Compare commits
22 Commits
1250247986
...
v0.10.0
| Author | SHA1 | Date | |
|---|---|---|---|
| 328c7a5693 | |||
| fe176a2abc | |||
| ab9218b124 | |||
| 8d6ab0eb56 | |||
| 76cd399c76 | |||
| bf1746a756 | |||
| 28bdc04fba | |||
| b67fae886e | |||
| 71a2eae87b | |||
| bd34ec57f8 | |||
| 97215ddb9b | |||
| dd7881acfb | |||
| ece31567b8 | |||
| 7ffc3dc603 | |||
| 4bdba6f2b7 | |||
| b184ca7cbd | |||
| 62a12dd661 | |||
| ac8d618111 | |||
| 5ddd3ee19c | |||
| 8be9b020d4 | |||
| 7f5a9c0357 | |||
| 7d591487e4 |
3
.gitignore
vendored
3
.gitignore
vendored
@@ -1,6 +1,5 @@
|
|||||||
# Compiled application binary and testing workspace
|
# Compiled application binary
|
||||||
/weatherreporter
|
/weatherreporter
|
||||||
/workspace
|
|
||||||
|
|
||||||
# ---> Go
|
# ---> Go
|
||||||
# If you prefer the allow list template instead of the deny list, see community template:
|
# If you prefer the allow list template instead of the deny list, see community template:
|
||||||
|
|||||||
14
README.md
14
README.md
@@ -1,25 +1,27 @@
|
|||||||
# weatherreporter
|
# weatherreporter
|
||||||
|
|
||||||
Weatherreporter is a Go CLI that turns normalized weather data into managed,
|
Weatherreporter is a Go CLI that turns normalized weather data into
|
||||||
human-facing Markdown reports.
|
human-facing Markdown reports.
|
||||||
|
|
||||||
It provides repeatable reports with inspectable local artifacts, so operators
|
It produces a Markdown report at an operator-owned destination and can upload
|
||||||
can review what was collected and generated for every run.
|
the completed output through Distributor.
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
weatherreporter generate today --out ./today.md
|
weatherreporter generate today
|
||||||
```
|
```
|
||||||
|
|
||||||
Configure a Weather API endpoint first; see the
|
Configure a Weather API endpoint first; see the
|
||||||
[configuration reference](docs/config.md).
|
[configuration reference](docs/config.md). The report is written to
|
||||||
|
`today.md` in the current directory; use `--out` to choose another destination.
|
||||||
|
See the [CLI reference](docs/cli.md) and [operations guide](docs/operations.md)
|
||||||
|
for command and operating details.
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
- [CLI reference](docs/cli.md)
|
- [CLI reference](docs/cli.md)
|
||||||
- [Configuration reference](docs/config.md)
|
- [Configuration reference](docs/config.md)
|
||||||
- [Operations guide](docs/operations.md)
|
- [Operations guide](docs/operations.md)
|
||||||
- [Troubleshooting](docs/troubleshooting.md)
|
|
||||||
- [Development guide](docs/development.md)
|
- [Development guide](docs/development.md)
|
||||||
- [Architecture policy](docs/policy/architecture.md)
|
- [Architecture policy](docs/policy/architecture.md)
|
||||||
|
|||||||
100
docs/adr/0001-stateless-execution.md
Normal file
100
docs/adr/0001-stateless-execution.md
Normal file
@@ -0,0 +1,100 @@
|
|||||||
|
# 0001: Make Weatherreporter Execution Stateless
|
||||||
|
|
||||||
|
Status: Accepted
|
||||||
|
|
||||||
|
Date: 2026-08-01
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Weather reports are ephemeral products. Forecasts and current conditions change
|
||||||
|
continuously, so the useful response to an old, failed, or superseded report is
|
||||||
|
normally a new generation rather than replaying or inspecting a prior run.
|
||||||
|
|
||||||
|
The existing run-addressed workspace retains module snapshots, prompt inputs,
|
||||||
|
execution receipts, generated text, rendered reports, metadata, and
|
||||||
|
notification receipts. That provenance store accumulates operational history
|
||||||
|
whose recovery and compatibility obligations are disproportionate to the value
|
||||||
|
of an ephemeral weather report. It also exists solely to support local Recent
|
||||||
|
Changes comparison for a rarely used report section.
|
||||||
|
|
||||||
|
The temporary roadmap that defined the feature scope and implementation plan
|
||||||
|
has been retired under the repository's documentation lifecycle. The
|
||||||
|
[architecture policy](../policy/architecture.md) defines the resulting system
|
||||||
|
invariants; this decision records their durable rationale.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Weatherreporter will operate as a stateless transformation pipeline:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Weather API input
|
||||||
|
-> deterministic facts and modules
|
||||||
|
-> Promptkit data package and generated text
|
||||||
|
-> repository-owned Markdown rendering
|
||||||
|
-> operator-owned report output
|
||||||
|
-> optional Distributor upload
|
||||||
|
```
|
||||||
|
|
||||||
|
Ordinary invocations will retain intermediate values only for the active
|
||||||
|
process and will publish one operator-owned Markdown output atomically. A
|
||||||
|
failed or canceled generation must not truncate or partially replace an
|
||||||
|
existing selected output. Single-report Distributor notification follows
|
||||||
|
successful publication; batch notification follows successful publication of
|
||||||
|
every planned report.
|
||||||
|
|
||||||
|
Weatherreporter will remove local Recent Changes comparison instead of
|
||||||
|
retaining application state to support it. It will remove run-addressed
|
||||||
|
workspace artifacts, historical inspection, and backward-compatible workspace
|
||||||
|
decoding. RunIDs may remain active correlation and Distributor idempotency
|
||||||
|
values, but will not identify retained application history.
|
||||||
|
|
||||||
|
Explicit `--llm-debug-dir` capture remains the sole diagnostic-file exception.
|
||||||
|
The operator selects and manages that secure location; ordinary execution does
|
||||||
|
not create an implicit debug location or a general logging store, and debug
|
||||||
|
capture must continue to exclude credentials.
|
||||||
|
|
||||||
|
Any future forecast comparison must use a structured product supplied by the
|
||||||
|
Weather API rather than local Weatherreporter history. The proposed
|
||||||
|
[Upstream Forecast Change Product](../roadmap/future.md#upstream-forecast-change-product)
|
||||||
|
defines the required upstream direction. A future integration must not add a
|
||||||
|
local snapshot fallback.
|
||||||
|
|
||||||
|
## Alternatives Considered
|
||||||
|
|
||||||
|
### Retain The Bounded Current-State Design
|
||||||
|
|
||||||
|
Retaining a managed workspace with current metadata, receipts, and snapshots
|
||||||
|
would preserve inspection and local comparison, but keeps an application-owned
|
||||||
|
history subsystem, artifact compatibility burden, and recovery surface that do
|
||||||
|
not match the report lifecycle.
|
||||||
|
|
||||||
|
### Time-Based Retention
|
||||||
|
|
||||||
|
Expiring workspace material after a fixed period reduces accumulation but still
|
||||||
|
requires retention policy, cleanup behavior, failure handling, and historical
|
||||||
|
format support. It does not remove the mismatch between retained provenance and
|
||||||
|
ephemeral report products.
|
||||||
|
|
||||||
|
### Bounded Run History
|
||||||
|
|
||||||
|
Keeping only a fixed number of prior runs limits storage volume but still makes
|
||||||
|
Weatherreporter responsible for run selection, comparison, inspection, and
|
||||||
|
state migration. It also creates arbitrary history gaps without establishing an
|
||||||
|
authoritative forecast baseline.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
The CLI, configuration, prompt-input, workspace, and inspection contracts will
|
||||||
|
change together. Legacy workspace material will not be migrated, decoded, or
|
||||||
|
automatically deleted; operators remain responsible for any desired cleanup.
|
||||||
|
|
||||||
|
Current action results will carry active identity, selected profile, safe
|
||||||
|
effective model information, output location, notification result, and safe
|
||||||
|
errors instead of historical artifact paths. Tests will protect atomic output,
|
||||||
|
batch and notification ordering, explicit secure debug capture, and the
|
||||||
|
absence of ordinary application-managed state.
|
||||||
|
|
||||||
|
This decision deliberately leaves the Weather API responsible for any future
|
||||||
|
forecast-history comparison. It avoids a cache, archive, retention engine,
|
||||||
|
manifest, resume mechanism, or replacement inspection surface in
|
||||||
|
Weatherreporter.
|
||||||
127
docs/cli.md
127
docs/cli.md
@@ -1,17 +1,17 @@
|
|||||||
# Weatherreporter CLI
|
# Weatherreporter CLI
|
||||||
|
|
||||||
`weatherreporter` generates weather reports, runs report batches, and inspects
|
`weatherreporter` generates Markdown weather reports and runs report batches.
|
||||||
artifacts already stored in its workspace.
|
It has no command for inspecting prior runs or application-owned state.
|
||||||
|
|
||||||
## Shortest Useful Command
|
## Shortest Useful Command
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
weatherreporter generate today --out ./today.md
|
weatherreporter generate today
|
||||||
```
|
```
|
||||||
|
|
||||||
The command uses the configured Weather API and writes an extra Markdown copy
|
The command uses the configured Weather API and writes an atomically replaced
|
||||||
at `./today.md`. See the [configuration reference](config.md) to supply the
|
`today.md` in the current directory. See the [configuration reference](config.md)
|
||||||
required Weather API endpoint.
|
to supply the required Weather API endpoint.
|
||||||
|
|
||||||
## Commands And Usage
|
## Commands And Usage
|
||||||
|
|
||||||
@@ -24,12 +24,6 @@ weatherreporter generate tomorrow [--config PATH] [--units VALUE] [--tz NAME] [-
|
|||||||
weatherreporter generate hourly [--config PATH] [--units VALUE] [--tz NAME] [--out PATH] [--llm-debug-dir PATH] [--quiet]
|
weatherreporter generate hourly [--config PATH] [--units VALUE] [--tz NAME] [--out PATH] [--llm-debug-dir PATH] [--quiet]
|
||||||
weatherreporter run morning [--config PATH] [--units VALUE] [--tz NAME] [--out-dir PATH] [--llm-debug-dir PATH] [--quiet]
|
weatherreporter run morning [--config PATH] [--units VALUE] [--tz NAME] [--out-dir PATH] [--llm-debug-dir PATH] [--quiet]
|
||||||
weatherreporter run evening [--config PATH] [--units VALUE] [--tz NAME] [--out-dir PATH] [--llm-debug-dir PATH] [--quiet]
|
weatherreporter run evening [--config PATH] [--units VALUE] [--tz NAME] [--out-dir PATH] [--llm-debug-dir PATH] [--quiet]
|
||||||
weatherreporter inspect reports [--config PATH] [--limit N]
|
|
||||||
weatherreporter inspect metadata [--config PATH] RUN_ID
|
|
||||||
weatherreporter inspect modules [--config PATH] RUN_ID
|
|
||||||
weatherreporter inspect data-package [--config PATH] RUN_ID
|
|
||||||
weatherreporter inspect prior [--config PATH] RUN_ID
|
|
||||||
weatherreporter inspect sources [--config PATH] RUN_ID
|
|
||||||
```
|
```
|
||||||
|
|
||||||
`weatherreporter --version` prints the version embedded in the executable.
|
`weatherreporter --version` prints the version embedded in the executable.
|
||||||
@@ -38,69 +32,81 @@ builds report `development`.
|
|||||||
|
|
||||||
| Command | Contract |
|
| Command | Contract |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `generate daily` | Requires `--date YYYY-MM-DD`; the date is interpreted in the effective report timezone. |
|
| `generate daily` | Requires `--date YYYY-MM-DD`; the date is interpreted in the effective report timezone. Its default filename is `daily-YYYY-MM-DD.md`. |
|
||||||
| `generate today` | Accepts an optional `--date YYYY-MM-DD`; without it, the current local date in the effective report timezone is used. |
|
| `generate today` | Accepts an optional `--date YYYY-MM-DD`; without it, the current local date in the effective report timezone is used. Its default filename is `today.md`. |
|
||||||
| `generate tomorrow` | Uses the next local civil day and accepts the common generate flags. |
|
| `generate tomorrow` | Uses the next local civil day and writes `tomorrow.md` by default. |
|
||||||
| `generate hourly` | Covers the next six hours in the effective report timezone. It does not accept `--date`, `--hours`, or `--duration`. |
|
| `generate hourly` | Covers the next six hours in the effective report timezone and writes `hourly.md` by default. It does not accept `--date`, `--hours`, or `--duration`. |
|
||||||
| `run morning` and `run evening` | Run their defined report batches. `--out-dir` writes extra Markdown copies; `--out` is not accepted. |
|
| `run morning` and `run evening` | Run their defined report batches and write each selected report beneath the current directory unless `--out-dir` selects another directory. `--out` is not accepted. |
|
||||||
|
|
||||||
`generate` accepts the four report command names shown above. `run` accepts
|
`generate` accepts the four report command names shown above. `run` accepts
|
||||||
only `morning` and `evening`. Batch membership, workspace artifacts, and
|
only `morning` and `evening`. Batch membership and notification ordering are
|
||||||
notification sequencing are described in the [operations guide](operations.md).
|
described in the [operations guide](operations.md).
|
||||||
|
|
||||||
## Output, Errors, And Quiet Mode
|
## Output, Errors, And Quiet Mode
|
||||||
|
|
||||||
|
For `generate`, the default output is the report's filename in the current
|
||||||
|
directory. `--out PATH` selects one output file instead. A relative path is
|
||||||
|
resolved from the current directory; an absolute path is used as given. For a
|
||||||
|
batch, the equivalent default is the current directory and `--out-dir PATH`
|
||||||
|
selects its output directory. Successful summaries always report the resulting
|
||||||
|
absolute `outputPath` values.
|
||||||
|
|
||||||
|
Outputs are written atomically. A generation, rendering, write, or cancellation
|
||||||
|
failure before publication leaves an existing destination unchanged. A
|
||||||
|
notification failure occurs after publication, so the newly written output
|
||||||
|
remains available.
|
||||||
|
|
||||||
Action commands (`generate` and `run`) write a JSON summary to stdout unless
|
Action commands (`generate` and `run`) write a JSON summary to stdout unless
|
||||||
`--quiet` is set. `run` also writes compact per-report and batch status lines
|
`--quiet` is set. `run` also writes compact per-report and batch status lines
|
||||||
to stderr. A pre-run error, such as an invalid flag, missing required argument,
|
to stderr. A pre-run error, such as an invalid flag, missing required argument,
|
||||||
or configuration-load failure, produces no partial JSON summary. When an action
|
or configuration-load failure, produces no partial JSON summary. When an
|
||||||
fails after it has produced a result, its summary has `"status": "failed"` and
|
action fails after it has produced a result, its summary has `"status": "failed"`
|
||||||
an `error` field.
|
and an `error` field.
|
||||||
|
|
||||||
`--quiet` is supported by action commands only. It suppresses action summaries
|
`--quiet` is supported by action commands only. It suppresses action summaries
|
||||||
and routine batch status output; it does not suppress command errors.
|
and routine batch status output; it does not suppress command errors.
|
||||||
|
|
||||||
Inspection commands always write their requested JSON value to stdout and do
|
|
||||||
not accept `--quiet`.
|
|
||||||
|
|
||||||
### Generate Summary
|
### Generate Summary
|
||||||
|
|
||||||
A generate summary always identifies the command, report, run, generation
|
A generate summary identifies the command, report, run, generation time, valid
|
||||||
time, valid period, and status:
|
period, prompt version, timezone, and status. Successful output has an absolute
|
||||||
|
`outputPath`:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"command": "generate",
|
"command": "generate",
|
||||||
"reportId": "today",
|
"reportId": "today",
|
||||||
"reportName": "Today Report",
|
|
||||||
"promptId": "weather.today_generated_text",
|
"promptId": "weather.today_generated_text",
|
||||||
|
"promptVersion": "2.0.0",
|
||||||
"runId": "20260529T120000.000000000Z_today",
|
"runId": "20260529T120000.000000000Z_today",
|
||||||
"status": "succeeded",
|
"status": "succeeded",
|
||||||
"generatedAt": "2026-05-29T12:00:00Z",
|
"timezone": "America/Chicago",
|
||||||
"validPeriod": {
|
"outputPath": "/srv/weather/today.md"
|
||||||
"start": "2026-05-29T00:00:00-05:00",
|
|
||||||
"end": "2026-05-30T00:00:00-05:00"
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
When available, the summary also includes `reportPath`, `metadataPath`,
|
When available, the summary also includes the effective `profileId`,
|
||||||
`dataPackagePath`, `preparationPath`, `executionPath`, `generatedTextRawPath`,
|
`backendId`, `modelName`, `sourceWarnings`, `validationStatus`, requested
|
||||||
`generatedTextPath`, `renderContextPath`, and `llmDebugPath`. `outputPath` is included only
|
`llmDebugPath`, and compact Distributor `notification` result. It does not
|
||||||
when `--out` wrote an extra copy. Distributor notification, when attempted,
|
include historical or transient artifact paths such as metadata, prompt input,
|
||||||
adds `notificationPath` and may add a compact `notification` object.
|
raw generated text, render context, or notification receipts.
|
||||||
|
|
||||||
### Run Summary And Stderr
|
### Run Summary And Stderr
|
||||||
|
|
||||||
A run summary contains `command`, `batch`, `status`, `startedAt`, `finishedAt`,
|
A run summary contains `command`, `batch`, `status`, `startedAt`, `finishedAt`,
|
||||||
`total`, `succeeded`, `failed`, and a `reports` array. It may also contain a
|
`total`, `succeeded`, `failed`, and a `reports` array. Each report item includes
|
||||||
top-level `notification` object and `error`. Batch status is `failed` if any
|
its identity, status, effective profile and model details when available,
|
||||||
report or the batch notification fails.
|
source warnings, validation status, and absolute `outputPath` after publication.
|
||||||
|
The top-level summary may also contain a batch `notification` object and
|
||||||
|
`error`. Batch status is `failed` if any report or the batch notification fails.
|
||||||
|
The `total`, `succeeded`, and `failed` counters describe report items only, so
|
||||||
|
a failed batch notification can leave `failed` at `0` while the top-level
|
||||||
|
notification and action status are `failed`.
|
||||||
|
|
||||||
Without `--quiet`, batch status lines use this form:
|
Without `--quiet`, batch status lines use this form:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
report=today status=succeeded output="reports/today.md"
|
report=today status=succeeded output="/srv/weather/reports/today.md"
|
||||||
batch=morning total=2 succeeded=2 failed=0
|
batch=morning total=2 succeeded=2 failed=0
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -112,12 +118,11 @@ batch=morning total=2 succeeded=2 failed=0
|
|||||||
| `--config PATH` | all commands | Load `PATH` instead of `/usr/local/etc/weatherreporter/config.yml`. |
|
| `--config PATH` | all commands | Load `PATH` instead of `/usr/local/etc/weatherreporter/config.yml`. |
|
||||||
| `--units VALUE` | `generate`, `run` | Override `weather_api.units` for this command. |
|
| `--units VALUE` | `generate`, `run` | Override `weather_api.units` for this command. |
|
||||||
| `--tz NAME` | `generate`, `run` | Override `weather_api.timezone` for this command. |
|
| `--tz NAME` | `generate`, `run` | Override `weather_api.timezone` for this command. |
|
||||||
| `--out PATH` | every `generate` command | Write an extra Markdown report copy. |
|
| `--out PATH` | every `generate` command | Write the report to this file instead of its current-directory default. |
|
||||||
| `--llm-debug-dir PATH` | every `generate` and `run` command | Write requested sensitive prompt diagnostics outside the managed workspace. The path must be absolute. |
|
| `--llm-debug-dir PATH` | every `generate` and `run` command | Write requested sensitive prompt diagnostics under this absolute path. |
|
||||||
| `--out-dir PATH` | `run morning`, `run evening` | Write extra Markdown report copies in `PATH`. |
|
| `--out-dir PATH` | `run morning`, `run evening` | Write batch reports beneath this directory instead of the current directory. |
|
||||||
| `--quiet` | `generate`, `run` | Suppress action summaries and routine batch status output. |
|
| `--quiet` | `generate`, `run` | Suppress action summaries and routine batch status output. |
|
||||||
| `--date YYYY-MM-DD` | `generate daily`, `generate today` | Required for Daily; optional for Today. |
|
| `--date YYYY-MM-DD` | `generate daily`, `generate today` | Required for Daily; optional for Today. |
|
||||||
| `--limit N` | `inspect reports` | Maximum runs to list. Defaults to `20`; `0` means no limit. |
|
|
||||||
|
|
||||||
Distributor notification is configured through `notify.distributor`; there are
|
Distributor notification is configured through `notify.distributor`; there are
|
||||||
no Distributor-specific CLI flags. See the [configuration reference](config.md).
|
no Distributor-specific CLI flags. See the [configuration reference](config.md).
|
||||||
@@ -125,33 +130,9 @@ no Distributor-specific CLI flags. See the [configuration reference](config.md).
|
|||||||
## Invocation Examples
|
## Invocation Examples
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
weatherreporter generate daily --date 2026-05-29 --out ./daily.md
|
weatherreporter generate daily --date 2026-05-29
|
||||||
weatherreporter generate today --date 2026-05-29 --out ./today.md
|
weatherreporter generate today --out ./reports/today.md
|
||||||
weatherreporter generate hourly --out ./hourly.md
|
weatherreporter generate hourly --out /srv/weather/hourly.md
|
||||||
weatherreporter generate today --llm-debug-dir /var/tmp/weatherreporter-debug
|
weatherreporter generate today --llm-debug-dir /var/tmp/weatherreporter-debug
|
||||||
weatherreporter run morning --out-dir ./reports --llm-debug-dir /var/tmp/weatherreporter-debug
|
weatherreporter run morning --out-dir ./reports --llm-debug-dir /var/tmp/weatherreporter-debug
|
||||||
```
|
```
|
||||||
|
|
||||||
## Inspection Commands
|
|
||||||
|
|
||||||
```sh
|
|
||||||
weatherreporter inspect reports --limit 10
|
|
||||||
weatherreporter inspect metadata 20260529T100000.000000000Z_today
|
|
||||||
weatherreporter inspect modules 20260529T100000.000000000Z_today
|
|
||||||
weatherreporter inspect data-package 20260529T100000.000000000Z_today
|
|
||||||
weatherreporter inspect prior 20260529T100000.000000000Z_today
|
|
||||||
weatherreporter inspect sources 20260529T100000.000000000Z_today
|
|
||||||
```
|
|
||||||
|
|
||||||
| Command | JSON returned |
|
|
||||||
| --- | --- |
|
|
||||||
| `inspect reports` | Recent generated runs, including artifact paths and source-warning counts. |
|
|
||||||
| `inspect metadata RUN_ID` | Persisted metadata for the run. |
|
|
||||||
| `inspect modules RUN_ID` | The run's persisted ordered module snapshot. |
|
|
||||||
| `inspect data-package RUN_ID` | The run's persisted prompt data package. |
|
|
||||||
| `inspect prior RUN_ID` | Prior comparable snapshot metadata, or `null` when none exists. |
|
|
||||||
| `inspect sources RUN_ID` | Source provenance and source warnings without full weather payloads. |
|
|
||||||
|
|
||||||
Inspection is read-only: it does not collect weather data or invoke Promptkit.
|
|
||||||
See the [operations guide](operations.md) for artifact lifecycle
|
|
||||||
and recovery.
|
|
||||||
|
|||||||
@@ -13,8 +13,8 @@ explicit `--config PATH` must exist. Values are applied in this order:
|
|||||||
2. the configuration file, when present; and
|
2. the configuration file, when present; and
|
||||||
3. the `--units` and `--tz` command-line overrides.
|
3. the `--units` and `--tz` command-line overrides.
|
||||||
|
|
||||||
Environment variables do not override configuration fields. Output flags write
|
Environment variables do not override configuration fields. Output flags select
|
||||||
extra report copies for a command and do not change configuration.
|
operator-owned destinations for one command and do not change configuration.
|
||||||
|
|
||||||
## Maintained Examples
|
## Maintained Examples
|
||||||
|
|
||||||
@@ -129,7 +129,7 @@ The default paths are:
|
|||||||
| `tomorrow` | `daily/{valid_start_date}/{run_id}.md`, `daily/{valid_start_date}/index.md`, `tomorrow/index.md` |
|
| `tomorrow` | `daily/{valid_start_date}/{run_id}.md`, `daily/{valid_start_date}/index.md`, `tomorrow/index.md` |
|
||||||
|
|
||||||
See the [operations guide](operations.md) for notification timing, uploaded
|
See the [operations guide](operations.md) for notification timing, uploaded
|
||||||
artifact selection, and failure handling.
|
output selection, and failure handling.
|
||||||
|
|
||||||
### `missing_source`
|
### `missing_source`
|
||||||
|
|
||||||
@@ -160,7 +160,7 @@ individual `generate` or `run` command when explicitly needed.
|
|||||||
`profile` selects an ID; `profile_file` and `profile_dir` supply definitions.
|
`profile` selects an ID; `profile_file` and `profile_dir` supply definitions.
|
||||||
They are separate decisions. An explicit `profile` applies to every selected
|
They are separate decisions. An explicit `profile` applies to every selected
|
||||||
report. Otherwise Hourly selects `weather-light`, while Daily, Today, and
|
report. Otherwise Hourly selects `weather-light`, while Daily, Today, and
|
||||||
Tomorrow select `weather-balanced` through their exact `1.1.0` prompt
|
Tomorrow select `weather-balanced` through their exact `2.0.0` prompt
|
||||||
definitions.
|
definitions.
|
||||||
|
|
||||||
Promptkit resolves a selected profile definition from a test or embedding
|
Promptkit resolves a selected profile definition from a test or embedding
|
||||||
@@ -179,21 +179,6 @@ model before use. An alternative profile may use `backend: local`; in that
|
|||||||
case `promptkit.local.endpoint` supplies the conventional local backend
|
case `promptkit.local.endpoint` supplies the conventional local backend
|
||||||
endpoint.
|
endpoint.
|
||||||
|
|
||||||
### `workspace`
|
|
||||||
|
|
||||||
| Field | Default |
|
|
||||||
| --- | --- |
|
|
||||||
| `root` | `workspace` |
|
|
||||||
| `snapshots_dir` | `snapshots` |
|
|
||||||
| `reports_dir` | `reports` |
|
|
||||||
| `data_packages_dir` | `data-packages` |
|
|
||||||
| `preflight_dir` | `preflight` |
|
|
||||||
| `notifications_dir` | `notifications` |
|
|
||||||
|
|
||||||
`workspace.root` is required. Each workspace subdirectory must be a relative
|
|
||||||
path that stays within the root. See the [operations guide](operations.md) for
|
|
||||||
the managed workspace layout and lifecycle.
|
|
||||||
|
|
||||||
### `dayparts`
|
### `dayparts`
|
||||||
|
|
||||||
`dayparts` is a non-empty list of named local-time windows used in forecast
|
`dayparts` is a non-empty list of named local-time windows used in forecast
|
||||||
@@ -202,18 +187,6 @@ derivation. Every item needs `name`, `start`, and `end`; start and end use
|
|||||||
(`06:00`–`10:00`), `midday` (`10:00`–`15:00`), `afternoon`
|
(`06:00`–`10:00`), `midday` (`10:00`–`15:00`), `afternoon`
|
||||||
(`15:00`–`17:00`), and `evening` (`17:00`–`24:00`).
|
(`15:00`–`17:00`), and `evening` (`17:00`–`24:00`).
|
||||||
|
|
||||||
### `recent_change`
|
|
||||||
|
|
||||||
| Field | Default |
|
|
||||||
| --- | --- |
|
|
||||||
| `temperature_degrees` | `5` |
|
|
||||||
| `precip_probability_points` | `20` |
|
|
||||||
| `wind_gust_miles_per_hour` | `10` |
|
|
||||||
| `precip_timing_shift_minutes` | `120` |
|
|
||||||
|
|
||||||
These thresholds control when Recent Changes are included in prompt input for a
|
|
||||||
prior comparable module snapshot.
|
|
||||||
|
|
||||||
### `reports`
|
### `reports`
|
||||||
|
|
||||||
`reports` optionally overrides a report's ordered deterministic modules and
|
`reports` optionally overrides a report's ordered deterministic modules and
|
||||||
|
|||||||
@@ -6,8 +6,8 @@ kind of change to its canonical documentation.
|
|||||||
|
|
||||||
Weatherreporter is a Go CLI that collects normalized weather data, derives
|
Weatherreporter is a Go CLI that collects normalized weather data, derives
|
||||||
deterministic report facts and module snapshots, executes Promptkit for
|
deterministic report facts and module snapshots, executes Promptkit for
|
||||||
single-report generated text, renders managed Markdown reports, and can upload completed
|
single-report generated text, renders Markdown reports, and can upload completed
|
||||||
reports through Distributor. Start with the [README](../README.md) for product
|
operator-owned outputs through Distributor. Start with the [README](../README.md) for product
|
||||||
context and the [architecture policy](policy/architecture.md) for system
|
context and the [architecture policy](policy/architecture.md) for system
|
||||||
boundaries and invariants.
|
boundaries and invariants.
|
||||||
|
|
||||||
@@ -21,16 +21,15 @@ boundaries and invariants.
|
|||||||
| Adding, changing, reviewing, or deleting tests | [Testing policy](policy/testing.md) and focused package tests | The policy defines risk-based sufficiency, durable test boundaries, doubles, and test-maintenance criteria. |
|
| Adding, changing, reviewing, or deleting tests | [Testing policy](policy/testing.md) and focused package tests | The policy defines risk-based sufficiency, durable test boundaries, doubles, and test-maintenance criteria. |
|
||||||
| CLI commands, flags, output, quiet mode, or command wiring | [CLI reference](cli.md) and [CLI internals](internal/cli.md) | The reference owns the user contract; the internal guide owns command composition and output flow. |
|
| CLI commands, flags, output, quiet mode, or command wiring | [CLI reference](cli.md) and [CLI internals](internal/cli.md) | The reference owns the user contract; the internal guide owns command composition and output flow. |
|
||||||
| Configuration fields, defaults, loading, overrides, validation, or secrets | [Configuration reference](config.md), [architecture policy](policy/architecture.md), and tests under `internal/config` | These separate the user-visible contract, architectural rules, and executable behavior. |
|
| Configuration fields, defaults, loading, overrides, validation, or secrets | [Configuration reference](config.md), [architecture policy](policy/architecture.md), and tests under `internal/config` | These separate the user-visible contract, architectural rules, and executable behavior. |
|
||||||
| Top-level generation, batch, collection, inspection, or notification workflow | [App orchestration internals](internal/app-orchestration.md) | It owns workflow ordering, persistence points, failure propagation, and orchestration invariants. |
|
| Top-level generation, batch, collection, output publication, or notification workflow | [App orchestration internals](internal/app-orchestration.md) | It owns workflow ordering, output publication, failure propagation, and orchestration invariants. |
|
||||||
| Weather API transport, source envelopes, source warnings, or collection | [Weather API integration](integrations/weatherapi.md), [weather-data internals](internal/weather-data.md), and [collection internals](internal/collect.md) | These separate the external contract, normalized source facts, and app-facing collection behavior. |
|
| Weather API transport, source envelopes, source warnings, or collection | [Weather API integration](integrations/weatherapi.md), [weather-data internals](internal/weather-data.md), and [collection internals](internal/collect.md) | These separate the external contract, normalized source facts, and app-facing collection behavior. |
|
||||||
| Forecast periods, weather derivation, collected facts, or derived facts | [Forecast derivation internals](internal/forecast-derivation.md) and [fact contracts](internal/facts.md) | They own deterministic derivation and the fact boundaries used by reports. |
|
| Forecast periods, weather derivation, collected facts, or derived facts | [Forecast derivation internals](internal/forecast-derivation.md) and [fact contracts](internal/facts.md) | They own deterministic derivation and the fact boundaries used by reports. |
|
||||||
| Report definitions, valid periods, report IDs, output naming, or batch composition | [Report registry internals](internal/report-registry.md) and [app orchestration internals](internal/app-orchestration.md) | Report definitions own selection and period rules; orchestration owns execution. |
|
| Report definitions, valid periods, report IDs, output naming, or batch composition | [Report registry internals](internal/report-registry.md) and [app orchestration internals](internal/app-orchestration.md) | Report definitions own selection and period rules; orchestration owns execution. |
|
||||||
| Module IDs, module composition, briefing values, or prompt-facing exports | [Module contract internals](internal/module.md), [module builder internals](internal/briefing.md), and [prompt-input internals](internal/prompt-input.md) | These own module contracts, value construction, and the curated prompt-package boundary. |
|
| Module IDs, module composition, briefing values, or prompt-facing exports | [Module contract internals](internal/module.md), [module builder internals](internal/briefing.md), and [prompt-input internals](internal/prompt-input.md) | These own module contracts, value construction, and the curated prompt-package boundary. |
|
||||||
| Recent Changes comparison | [Changes internals](internal/changes.md) and [operations guide](operations.md) | The internal guide owns structured comparison; operations owns user-visible artifact behavior. |
|
|
||||||
| Prompt execution, profiles, prompt inputs, or result handling | `internal/promptexec`, the Promptkit adapter, and [prompt-input internals](internal/prompt-input.md) | These separate the executor contract and input construction. |
|
| Prompt execution, profiles, prompt inputs, or result handling | `internal/promptexec`, the Promptkit adapter, and [prompt-input internals](internal/prompt-input.md) | These separate the executor contract and input construction. |
|
||||||
| Generated-text schemas, validation, render contexts, templates, or Markdown rendering | [Generated-text internals](internal/generatedtext.md), [report-template internals](internal/reporttemplate.md), and [report template guide](templates.md) | These own structured text, renderer implementation, and the maintainer-facing template surface. |
|
| Generated-text schemas, validation, render contexts, templates, or Markdown rendering | [Generated-text internals](internal/generatedtext.md), [report-template internals](internal/reporttemplate.md), and [report template guide](templates.md) | These own structured text, renderer implementation, and the maintainer-facing template surface. |
|
||||||
| Workspace paths, metadata, atomic persistence, lookup, inspection, or recovery | [State internals](internal/state.md), [operations guide](operations.md), and [troubleshooting guide](troubleshooting.md) | These separate implementation, operator workflows, and symptom-based recovery. |
|
| Output destinations, atomic publication, prompt diagnosis, or legacy cleanup | [Operations guide](operations.md) and [App orchestration internals](internal/app-orchestration.md) | Operations owns operator workflows; app internals owns the implementation boundary. |
|
||||||
| Distributor bundles, uploads, notification artifacts, or failures | [Distributor adapter internals](internal/distributor-adapter.md), [Distributor integration contracts](integrations/distributor/), and [operations guide](operations.md) | These separate adapter behavior, external contracts, and operational lifecycle. |
|
| Distributor bundles, uploads, notification results, or failures | [Distributor adapter internals](internal/distributor-adapter.md), [Distributor integration contracts](integrations/distributor/), and [operations guide](operations.md) | These separate adapter behavior, external contracts, and operational lifecycle. |
|
||||||
| Maintained example configuration | [Configuration reference](config.md) and files under `examples/` | The reference owns field meaning; examples own complete copyable files. |
|
| Maintained example configuration | [Configuration reference](config.md) and files under `examples/` | The reference owns field meaning; examples own complete copyable files. |
|
||||||
| Release preparation, tagging, publication, or verification | [Release procedure](release.md) | It owns version selection, release-note preparation, candidate validation, tag publication, CI behavior, and post-publication checks. |
|
| Release preparation, tagging, publication, or verification | [Release procedure](release.md) | It owns version selection, release-note preparation, candidate validation, tag publication, CI behavior, and post-publication checks. |
|
||||||
| Proposed, deferred, or unimplemented work | Documents under `docs/roadmap/` | Future behavior and implementation status belong only in roadmaps until implemented. |
|
| Proposed, deferred, or unimplemented work | Documents under `docs/roadmap/` | Future behavior and implementation status belong only in roadmaps until implemented. |
|
||||||
@@ -45,13 +44,13 @@ present before introducing a new package or abstraction.
|
|||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `cmd/weatherreporter` | Binary entry point. |
|
| `cmd/weatherreporter` | Binary entry point. |
|
||||||
| `internal/cli` | Command parsing, flags, help, output, and command wiring. |
|
| `internal/cli` | Command parsing, flags, help, output, and command wiring. |
|
||||||
| `internal/app` | Generation, batches, collection coordination, notification, and inspection orchestration. |
|
| `internal/app` | Stateless generation, batches, collection coordination, output publication, and notification. |
|
||||||
| `internal/config` | Configuration defaults, loading, precedence, secrets, and validation. |
|
| `internal/config` | Configuration defaults, loading, precedence, secrets, and validation. |
|
||||||
| `internal/adapters` | Weather API, Promptkit, and Distributor boundaries. |
|
| `internal/adapters` | Weather API, Promptkit, and Distributor boundaries. |
|
||||||
| `internal/weatherdata`, `internal/forecast`, `internal/facts` | Normalized source facts and deterministic derivation. |
|
| `internal/weatherdata`, `internal/forecast`, `internal/facts` | Normalized source facts and deterministic derivation. |
|
||||||
| `internal/report`, `internal/module`, `internal/briefing`, `internal/changes` | Report registry, module contracts and values, and structured comparison. |
|
| `internal/report`, `internal/module`, `internal/briefing` | Report registry plus module and briefing contracts. |
|
||||||
| `internal/promptinput`, `internal/generatedtext`, `internal/reporttemplate` | Prompt packages, generated-text validation, render contexts, and Markdown templates. |
|
| `internal/promptinput`, `internal/generatedtext`, `internal/reporttemplate` | Prompt packages, generated-text validation, render contexts, and Markdown templates. |
|
||||||
| `internal/state`, `internal/fileutil`, `internal/timeutil` | Durable artifacts, atomic file operations, clocks, dates, timezones, and periods. |
|
| `internal/fileutil`, `internal/timeutil` | Atomic output operations, clocks, dates, timezones, and periods. |
|
||||||
| `docs` | User, operator, integration, internal, policy, and roadmap documentation. |
|
| `docs` | User, operator, integration, internal, policy, and roadmap documentation. |
|
||||||
| `examples` | Maintained copyable configuration. |
|
| `examples` | Maintained copyable configuration. |
|
||||||
|
|
||||||
|
|||||||
@@ -58,8 +58,8 @@ application to record.
|
|||||||
|
|
||||||
Run and idempotency records are in-memory. Completed records expire according
|
Run and idempotency records are in-memory. Completed records expire according
|
||||||
to Distributor's `server.http.retention`, and a Distributor restart removes
|
to Distributor's `server.http.retention`, and a Distributor restart removes
|
||||||
retained status and idempotency state. Status polling decisions and persistence
|
retained status and idempotency state. Status polling decisions are internal
|
||||||
of notification artifacts are internal orchestration behavior; see the
|
orchestration behavior; see the
|
||||||
[Distributor adapter](../../internal/distributor-adapter.md) and
|
[Distributor adapter](../../internal/distributor-adapter.md) and
|
||||||
[application orchestration](../../internal/app-orchestration.md).
|
[application orchestration](../../internal/app-orchestration.md).
|
||||||
|
|
||||||
|
|||||||
@@ -8,15 +8,15 @@ the upload call returns.
|
|||||||
|
|
||||||
## File Mappings
|
## File Mappings
|
||||||
|
|
||||||
Every mapping pairs a managed Markdown report source with one bundle-relative
|
Every mapping pairs an operator-owned Markdown output with one bundle-relative
|
||||||
path. A single-report notification maps its one managed report to each rendered
|
path. A single-report notification maps its published output to each rendered
|
||||||
path configured for that report. A batch notification combines mappings for
|
path configured for that report. A batch notification combines mappings for
|
||||||
every included managed report and rejects duplicate bundle paths.
|
every included published output and rejects duplicate bundle paths.
|
||||||
|
|
||||||
The report source is never an `--out` copy or an arbitrary workspace scan. The
|
The report source is the output selected for that command; the application does
|
||||||
application selects it and renders notification paths; see the [operations guide](../../operations.md)
|
not scan local directories. It renders notification paths after publication;
|
||||||
for the managed-upload rule and the [Distributor adapter](../../internal/distributor-adapter.md)
|
see the [operations guide](../../operations.md) and the
|
||||||
for the adapter boundary.
|
[Distributor adapter](../../internal/distributor-adapter.md) for the boundary.
|
||||||
|
|
||||||
Bundle paths must be clean, relative, slash-separated paths. They cannot be
|
Bundle paths must be clean, relative, slash-separated paths. They cannot be
|
||||||
empty or absolute, contain backslashes, empty segments, `.` or `..`, or use
|
empty or absolute, contain backslashes, empty segments, `.` or `..`, or use
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ For each notification, Weatherreporter calls `UploadFiles` with:
|
|||||||
- the rendered pipeline ID;
|
- the rendered pipeline ID;
|
||||||
- the rendered bundle ID as the source manifest ID;
|
- the rendered bundle ID as the source manifest ID;
|
||||||
- the report or batch generation time as `Created`;
|
- the report or batch generation time as `Created`;
|
||||||
- the managed-report-to-bundle-path mappings described in the
|
- the published-output-to-bundle-path mappings described in the
|
||||||
[bundle mapping contract](pkg-bundle.md); and
|
[bundle mapping contract](pkg-bundle.md); and
|
||||||
- a rendered idempotency key.
|
- a rendered idempotency key.
|
||||||
|
|
||||||
@@ -38,8 +38,8 @@ adapter translates it to its own conflict error without exposing the token.
|
|||||||
The adapter then calls `Status` for the accepted run. A terminal `failed`
|
The adapter then calls `Status` for the accepted run. A terminal `failed`
|
||||||
status is a notification failure. A status lookup failure or a timeout before a
|
status is a notification failure. A status lookup failure or a timeout before a
|
||||||
terminal status remains attached to the otherwise accepted upload as diagnostic
|
terminal status remains attached to the otherwise accepted upload as diagnostic
|
||||||
status information. Polling cadence, final failure handling, redaction, and
|
status information. Polling cadence, final failure handling, and redaction are
|
||||||
notification artifact persistence are internal behavior documented in the
|
internal behavior documented in the
|
||||||
[Distributor adapter](../../internal/distributor-adapter.md) and
|
[Distributor adapter](../../internal/distributor-adapter.md) and
|
||||||
[application orchestration](../../internal/app-orchestration.md).
|
[application orchestration](../../internal/app-orchestration.md).
|
||||||
|
|
||||||
|
|||||||
@@ -1,15 +1,10 @@
|
|||||||
# Promptkit Integration
|
# Promptkit Integration
|
||||||
|
|
||||||
Weatherreporter uses Promptkit for all generated-text reports. The four logical
|
Weatherreporter uses Promptkit for all generated-text reports. The four logical prompts are `weather.daily_generated_text`, `weather.today_generated_text`, `weather.tomorrow_generated_text`, and `weather.hourly_generated_text`, each at version `2.0.0`. Their prompt assets, generated-text JSON Schemas, and Weatherreporter profile catalog are embedded by `internal/promptassets`.
|
||||||
prompts are `weather.daily_generated_text`, `weather.today_generated_text`,
|
|
||||||
`weather.tomorrow_generated_text`, and `weather.hourly_generated_text`, each at
|
|
||||||
version `1.1.0`. Their prompt assets, generated-text JSON Schemas, and
|
|
||||||
Weatherreporter profile catalog are embedded by `internal/promptassets`.
|
|
||||||
|
|
||||||
## Logical profile catalog
|
## Logical Profile Catalog
|
||||||
|
|
||||||
Prompt definitions select a stable Weatherreporter profile ID. The embedded
|
Prompt definitions select a stable Weatherreporter profile ID. The embedded definitions currently use Promptkit's `openrouter` backend:
|
||||||
definitions currently use Promptkit's `openrouter` backend:
|
|
||||||
|
|
||||||
| Profile ID | Model | Reasoning effort | Timeout | Service tier | Default reports |
|
| Profile ID | Model | Reasoning effort | Timeout | Service tier | Default reports |
|
||||||
| --- | --- | --- | --- | --- | --- |
|
| --- | --- | --- | --- | --- | --- |
|
||||||
@@ -17,44 +12,23 @@ definitions currently use Promptkit's `openrouter` backend:
|
|||||||
| `weather-balanced` | `~google/gemini-flash-latest` | `high` | 240 seconds | `flex` | Daily, Today, Tomorrow |
|
| `weather-balanced` | `~google/gemini-flash-latest` | `high` | 240 seconds | `flex` | Daily, Today, Tomorrow |
|
||||||
| `weather-deep` | `~anthropic/claude-sonnet-latest` | `high` | 240 seconds | `flex` | None |
|
| `weather-deep` | `~anthropic/claude-sonnet-latest` | `high` | 240 seconds | `flex` | None |
|
||||||
|
|
||||||
The `~` prefix is part of each OpenRouter rolling-alias model ID. The embedded
|
The `~` prefix is part of each OpenRouter rolling-alias model ID. The embedded profiles intentionally omit endpoints, credentials, temperature, `top_p`, and output-token limits.
|
||||||
profiles intentionally omit endpoints, credentials, temperature, `top_p`, and
|
|
||||||
output-token limits.
|
|
||||||
|
|
||||||
## Selection, lookup, and active execution
|
## Selection And Active Execution
|
||||||
|
|
||||||
Before collection, Weatherreporter inspects the exact prompt version and output
|
Before weather collection, Weatherreporter validates the exact prompt version, output contract, and selected profile. A nonblank `promptkit.profile` selects one profile ID for every report in the command; otherwise the prompt's declared default selects it. Promptkit resolves the selected definition in this order:
|
||||||
contract. A nonblank `promptkit.profile` selects one profile ID for every
|
|
||||||
report in the command; otherwise the prompt's declared default selects it.
|
|
||||||
Promptkit resolves the selected definition in this order:
|
|
||||||
|
|
||||||
1. explicit in-memory profiles used by an embedding consumer or test;
|
1. explicit in-memory profiles used by an embedding consumer or test;
|
||||||
2. the configured `profile_file` or `profile_dir`;
|
2. the configured `profile_file` or `profile_dir`;
|
||||||
3. Weatherreporter's embedded fallback profiles; and
|
3. Weatherreporter's embedded fallback profiles; and
|
||||||
4. Promptkit's built-in catalog.
|
4. Promptkit's built-in catalog.
|
||||||
|
|
||||||
A source falls through only when the selected ID is absent. Each source
|
A source falls through only when the selected ID is absent. Each source supplies a complete definition, so profile fields are not merged. A malformed matching operator definition is an error and does not fall back.
|
||||||
supplies a complete definition, so profile fields are not merged. A malformed
|
|
||||||
matching operator definition is an error and does not fall back.
|
|
||||||
|
|
||||||
Profiles that require a direct API key are unsupported; a profile that reports
|
Profiles that require a direct API key are unsupported; a profile that reports `APIKeyEnv` requires a nonblank value in that environment variable. Active results retain the selected logical profile ID and resolved backend and model. Ordinary errors, summaries, logs, and outputs exclude endpoints, credentials, rendered messages, schemas, request bodies, response bodies, and complete parameter maps.
|
||||||
`APIKeyEnv` requires a nonblank value in that environment variable. Inspection,
|
|
||||||
preparation, and execution retain the selected logical profile ID and resolved
|
|
||||||
backend and model through Weatherreporter's project-owned contract. Ordinary
|
|
||||||
errors, summaries, logs, and workspace state exclude endpoints, credentials,
|
|
||||||
rendered messages, schemas, request bodies, response bodies, and complete
|
|
||||||
parameter maps.
|
|
||||||
|
|
||||||
Execution receives the already-persisted YAML package, prepares it once, and returns structured
|
Promptkit receives the YAML data package as an inline input and returns structured JSON that Weatherreporter validates before rendering its own Markdown template. Safe active provenance remains in memory. Content-rich diagnostics are opt-in through `--llm-debug-dir`; see [operations](../operations.md) for retention and permissions.
|
||||||
JSON that Weatherreporter validates before rendering its own Markdown template. Preparation and
|
|
||||||
execution receipts are project-owned, safe provenance records. Content-rich diagnostics are
|
|
||||||
opt-in through `--llm-debug-dir`; see [operations](../operations.md) for retention and permissions.
|
|
||||||
|
|
||||||
The generated-text schemas require `summary`, `forecast_discussion`, and
|
The generated-text schemas require `summary`, `forecast_discussion`, and `precipitation_timing`, and reject additional properties. Prompts return an empty string for `precipitation_timing` when the deterministic package contains no precipitation windows.
|
||||||
`precipitation_timing`, and reject additional properties. Prompts return an empty string for
|
|
||||||
`precipitation_timing` when the deterministic package contains no precipitation windows.
|
|
||||||
|
|
||||||
Prompt/profile configuration and the maintained local override example are
|
Prompt/profile configuration and the maintained local override example are owned by the [configuration reference](../config.md). Adapter construction and mapping are documented in the [Promptkit adapter internals](../internal/promptkit-adapter.md).
|
||||||
owned by the [configuration reference](../config.md). Adapter construction and
|
|
||||||
mapping are documented in the [Promptkit adapter internals](../internal/promptkit-adapter.md).
|
|
||||||
Durable metadata compatibility is described in [state internals](../internal/state.md).
|
|
||||||
|
|||||||
@@ -1,48 +1,24 @@
|
|||||||
# Application Orchestration Internals
|
# Application Orchestration Internals
|
||||||
|
|
||||||
`internal/app` owns top-level generation, batch, collection, inspection, and
|
`internal/app` owns stateless report generation, batch execution, atomic output publication, and notification coordination after `internal/cli` has parsed arguments and loaded configuration. The user contract is owned by the [CLI reference](../cli.md) and [operations guide](../operations.md).
|
||||||
notification ordering after the CLI has parsed arguments and loaded configuration.
|
|
||||||
|
|
||||||
## Generation
|
## Single-Report Flow
|
||||||
|
|
||||||
`GenerateDetailed` resolves one of the four report definitions, initializes an
|
`GenerateDetailed` resolves the requested report and output destination, then initializes an optional explicit debug writer. It validates the exact Promptkit prompt and selected profile before collecting weather data. The resolved profile, backend, and model are carried in the active result.
|
||||||
optional debug root, and inspects the exact Promptkit prompt/profile before it
|
|
||||||
collects weather or writes managed state. A configured global profile selects
|
|
||||||
every report in the action; otherwise the exact prompt selects its default
|
|
||||||
logical profile. Inspection keeps only the selected profile ID and effective
|
|
||||||
backend/model needed by the project-owned execution contract. It then builds facts and modules,
|
|
||||||
saves the YAML data package, persists preparation metadata from the executor
|
|
||||||
callback, executes the prepared prompt, saves execution provenance and raw
|
|
||||||
output, validates generated text, renders Markdown, and optionally copies or
|
|
||||||
notifies from the managed report.
|
|
||||||
|
|
||||||
After a completed prompt run, each successfully written downstream artifact is
|
The workflow builds facts, a module snapshot, briefing metadata, and the YAML prompt package in memory. It executes Promptkit, validates the returned generated text, builds a render context, and renders Markdown. `fileutil` atomically writes the completed Markdown to the selected output path. Only after that write succeeds does single-report notification run.
|
||||||
atomically added to the execution record before the corresponding metadata
|
|
||||||
rewrite. Later failures therefore leave the original Promptkit outcome and its
|
|
||||||
last durable set of reached paths inspectable.
|
|
||||||
|
|
||||||
Failure results retain all safe paths reached so far. Validation rejection
|
Failures return an active partial result with safe identity, profile, warning, validation, debug, and output information when available. After rendering and immediately before publication, the workflow checks for cancellation or deadline expiry. Any failure before publication leaves an existing destination unchanged. A notification failure retains the newly published output.
|
||||||
persists raw output and execution provenance but does not render a report.
|
|
||||||
|
|
||||||
## Batches
|
## Batches
|
||||||
|
|
||||||
`RunBatchDetailed` constructs a single debug writer and uses the request's
|
`RunBatchDetailed` captures one output directory, creates at most one explicit debug writer, and uses one executor. Before collection it validates the prompt and profile candidates for the selected batch. It collects once, calculates the data-dependent plan, then validates and retains the final output path for every planned report before invoking the same generation core sequentially.
|
||||||
single executor. Before collection it inspects Today, Tomorrow, and Daily for
|
|
||||||
morning, or Tomorrow and Daily for evening, deduplicating inspection of a
|
|
||||||
shared selected profile. It then collects once, plans eligible Daily dates, and calls the
|
|
||||||
same prompt-generation core sequentially for each planned report. Per-report
|
|
||||||
notification is suppressed; a failed report does not stop later reports.
|
|
||||||
|
|
||||||
Batch notification is skipped when disabled or when any report failed.
|
Each item has an independent result. A failed item does not stop later items; successful items retain their published output paths. Per-report notification is suppressed during a batch. Batch notification runs only after every planned report has published successfully. It is skipped when any item failed. Batch result counters count report items only; a batch notification failure is represented by the top-level notification result and still produces a failed batch outcome.
|
||||||
Successful notification uses the completed managed report paths only. Batch
|
|
||||||
items retain preparation, execution, and optional debug paths when reached.
|
|
||||||
|
|
||||||
## Inspection And Boundaries
|
## Boundaries And Verification
|
||||||
|
|
||||||
Inspection loads persisted state only. It does not collect weather, invoke
|
The package does not parse flags, load YAML, implement transport, construct provider SDKs, or define report-period policy. Prompt, profile, weather, and Distributor implementations remain behind project-owned contracts.
|
||||||
Promptkit, or upload reports. The app coordinates project-owned contracts but
|
|
||||||
does not parse flags, load YAML, implement transport, construct provider SDKs,
|
|
||||||
or define report-period policy.
|
|
||||||
|
|
||||||
Focused checks:
|
Focused checks:
|
||||||
|
|
||||||
|
|||||||
@@ -66,4 +66,4 @@ go test ./internal/briefing
|
|||||||
```
|
```
|
||||||
|
|
||||||
Builders emit structured facts, never report prose. The app collects their
|
Builders emit structured facts, never report prose. The app collects their
|
||||||
outputs into a module snapshot, and state persists that snapshot.
|
outputs into an in-memory module snapshot for prompt input and rendering.
|
||||||
|
|||||||
@@ -1,56 +0,0 @@
|
|||||||
# Changes Internals
|
|
||||||
|
|
||||||
`internal/changes` deterministically compares a compatible prior module
|
|
||||||
snapshot with the current snapshot. It returns compact structured changes for
|
|
||||||
prompt input; it never reads state, finds a prior report, renders Markdown, or
|
|
||||||
compares generated text. Snapshot construction belongs to
|
|
||||||
[module internals](module.md), and prior-snapshot discovery belongs to
|
|
||||||
[state internals](state.md).
|
|
||||||
|
|
||||||
## Comparison inputs and output
|
|
||||||
|
|
||||||
Each comparator receives a prior snapshot, a current snapshot, and
|
|
||||||
`Thresholds`. A `Change` has a stable type and message plus previous and
|
|
||||||
current values where useful. Changes are sorted by type and then message, so
|
|
||||||
the same inputs always yield the same order.
|
|
||||||
|
|
||||||
Threshold values are supplied by application orchestration from the
|
|
||||||
[Recent Changes configuration](../config.md#recent_change); this package does
|
|
||||||
not load configuration or choose defaults. Numeric changes are emitted when
|
|
||||||
the absolute difference meets the configured threshold. Precipitation also
|
|
||||||
requires a change between its low, possible, likely, and high categories.
|
|
||||||
|
|
||||||
## Strategies
|
|
||||||
|
|
||||||
| Comparator | Required snapshot data | Compared values |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `CompareDaily` | `derived_daily_summary`, `derived_daypart_summaries` | Low and high temperature, daily precipitation probability and timing, peak gust, alerts, and aggregate indicators |
|
|
||||||
|
|
||||||
For daily comparison, `alert_digest` and `precip_timing` are optional: alerts
|
|
||||||
are compared when present, and timing is compared only when both snapshots
|
|
||||||
contain it.
|
|
||||||
|
|
||||||
The application selects a comparator only after state lookup establishes a
|
|
||||||
compatible prior snapshot. Daily, Today, and Tomorrow use the daily comparator.
|
|
||||||
Hourly reports do not produce a Recent Changes list.
|
|
||||||
|
|
||||||
## Missing data and failures
|
|
||||||
|
|
||||||
Required stanzas that are absent or cannot be decoded return an error with the
|
|
||||||
snapshot and stanza context. Optional stanzas may be absent. A snapshot with no
|
|
||||||
eligible predecessor is not a comparison failure: the caller supplies an empty
|
|
||||||
change list without invoking this package.
|
|
||||||
|
|
||||||
The package has no filesystem, transport, CLI, renderer, or persistence
|
|
||||||
behavior. It does not decide report compatibility or retain snapshots.
|
|
||||||
|
|
||||||
## Verification and invariants
|
|
||||||
|
|
||||||
Focused tests cover the daily strategy, threshold boundaries, indicator and
|
|
||||||
alert changes, and missing required stanzas:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/changes
|
|
||||||
```
|
|
||||||
|
|
||||||
Recent Changes always compare structured snapshot values, never report prose.
|
|
||||||
@@ -1,26 +1,14 @@
|
|||||||
# CLI Internals
|
# CLI Internals
|
||||||
|
|
||||||
`internal/cli` parses terminal arguments, loads configuration, constructs app
|
`internal/cli` parses terminal arguments, loads configuration, constructs app requests, and translates app results to bounded JSON summaries. The public contract belongs in the [CLI reference](../cli.md).
|
||||||
requests, and translates app results to bounded JSON summaries. The user
|
|
||||||
contract belongs in the [CLI reference](../cli.md).
|
|
||||||
|
|
||||||
The root `--version` flag reports the build version supplied by
|
The root `--version` flag reports the build version supplied by `internal/buildinfo`. Tagged release builds replace its development default at link time.
|
||||||
`internal/buildinfo`. Tagged release builds replace its development default at
|
|
||||||
link time.
|
|
||||||
|
|
||||||
For each `generate` or `run` action, `Runner` constructs one project-owned
|
For each `generate` or `run` action, `Runner` constructs one project-owned Promptkit executor after configuration loads. It captures an absolute working directory, resolves a relative output override against it, and passes the working directory, resolved override, and any `--llm-debug-dir` request to the app. With no override, the app derives the report filename in that working directory. `run` uses the same resolution rule for `--out-dir`.
|
||||||
Promptkit executor after configuration loads. It passes the executor and any
|
|
||||||
`--llm-debug-dir` request into the app. `run` accepts the debug flag as well
|
|
||||||
as `generate`; the app, not the CLI, secures and initializes the debug root.
|
|
||||||
|
|
||||||
Summaries include identity, status, safe artifact paths, and notification
|
The CLI dispatches only generation and batch actions. It has no persisted-run or inspection dispatch. Summaries include report identity, status, output path, effective profile/backend/model, source warnings, validation, requested debug path, and notification result when available. They intentionally exclude prompt input, raw generated text, render context, endpoints, credentials, and full Distributor payloads. A failed action with a partial result still emits its safe summary before its error is returned.
|
||||||
provenance. They intentionally exclude module values, YAML package bodies, raw
|
|
||||||
generated text, rendered prompts, schemas, endpoints, credentials, and full
|
|
||||||
Distributor payloads. A failed action with a partial result still emits its
|
|
||||||
safe summary before its error is returned.
|
|
||||||
|
|
||||||
CLI code owns no report policy, weather collection, persistence, provider
|
CLI code owns no report policy, weather collection, output publication, provider execution, or notification policy. Focused checks:
|
||||||
execution, or notification policy. Focused checks:
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go test ./internal/cli
|
go test ./internal/cli
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ status, and `RunStatus`, including pipeline ID, lifecycle timestamps, report,
|
|||||||
and remote error details.
|
and remote error details.
|
||||||
|
|
||||||
Status lookup or polling errors are preserved in `UploadResult.StatusError` so
|
Status lookup or polling errors are preserved in `UploadResult.StatusError` so
|
||||||
the caller can record an accepted-but-unconfirmed delivery. A terminal failed
|
the caller can report an accepted-but-unconfirmed delivery. A terminal failed
|
||||||
run returns that result and an error. Upload failures return no result. Upstream
|
run returns that result and an error. Upload failures return no result. Upstream
|
||||||
idempotency conflicts become the local `IdempotencyConflictError`, which adds
|
idempotency conflicts become the local `IdempotencyConflictError`, which adds
|
||||||
endpoint, pipeline, bundle, idempotency, and file-path context while redacting
|
endpoint, pipeline, bundle, idempotency, and file-path context while redacting
|
||||||
|
|||||||
@@ -60,5 +60,5 @@ go test ./internal/facts
|
|||||||
```
|
```
|
||||||
|
|
||||||
Facts are derived once for a resolved report from already collected data.
|
Facts are derived once for a resolved report from already collected data.
|
||||||
They remain reusable structured values: prompt wording, state persistence,
|
They remain reusable structured values for prompt input and template
|
||||||
prior-report comparison, and template presentation are owned elsewhere.
|
presentation, which are owned elsewhere.
|
||||||
|
|||||||
@@ -46,9 +46,7 @@ UTC when these APIs are called directly. Optional narrative, discussion, and
|
|||||||
alerts remain absent when their normalized products are absent.
|
alerts remain absent when their normalized products are absent.
|
||||||
|
|
||||||
Forecast thresholds used for brief indicators and precipitation timing are
|
Forecast thresholds used for brief indicators and precipitation timing are
|
||||||
implementation rules. User-configurable Recent Changes thresholds are applied
|
implementation rules.
|
||||||
by [changes internals](changes.md), whose defaults are documented in
|
|
||||||
[configuration](../config.md).
|
|
||||||
|
|
||||||
## Verification and invariants
|
## Verification and invariants
|
||||||
|
|
||||||
|
|||||||
@@ -34,8 +34,8 @@ template iteration rather than maps.
|
|||||||
Optional source stanzas become nil or fallback context fields. Missing required
|
Optional source stanzas become nil or fallback context fields. Missing required
|
||||||
stanzas, type-decoding failures, invalid metadata, or a generated-text type
|
stanzas, type-decoding failures, invalid metadata, or a generated-text type
|
||||||
that does not match the chosen handler fail before template execution. Prompt
|
that does not match the chosen handler fail before template execution. Prompt
|
||||||
packages, raw Promptkit output, state persistence, and template asset lookup
|
packages, raw Promptkit output handling, and template asset lookup remain
|
||||||
remain outside this package.
|
outside this package.
|
||||||
|
|
||||||
## Verification and invariants
|
## Verification and invariants
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# Module Contract Internals
|
# Module Contract Internals
|
||||||
|
|
||||||
`internal/module` defines the stable envelope between report composition,
|
`internal/module` defines the stable envelope between report composition,
|
||||||
module builders, snapshots, comparisons, templates, and prompt packages. It
|
module builders, in-memory snapshots, templates, and prompt packages. It
|
||||||
does not define a report, execute a builder, or choose prompt-export policy;
|
does not define a report, execute a builder, or choose prompt-export policy;
|
||||||
those responsibilities belong to [report registry](report-registry.md) and
|
those responsibilities belong to [report registry](report-registry.md) and
|
||||||
[briefing](briefing.md).
|
[briefing](briefing.md).
|
||||||
@@ -11,10 +11,10 @@ those responsibilities belong to [report registry](report-registry.md) and
|
|||||||
Each `Output` has a module ID, stanza name, rich `Value`, and runtime-only
|
Each `Output` has a module ID, stanza name, rich `Value`, and runtime-only
|
||||||
`PromptValue`. `DataPackageValue` returns the prompt value when present and
|
`PromptValue`. `DataPackageValue` returns the prompt value when present and
|
||||||
otherwise the rich value. This permits custom prompt exports without shrinking
|
otherwise the rich value. This permits custom prompt exports without shrinking
|
||||||
the template and inspection value.
|
the template value.
|
||||||
|
|
||||||
`NewSnapshot` builds the ordered `weatherreporter.modules.v1` snapshot and
|
`NewSnapshot` builds the ordered `weatherreporter.modules.v1` snapshot and
|
||||||
validates it. Snapshot JSON persists IDs, stanza names, and rich values only;
|
validates it. Its JSON representation contains IDs, stanza names, and rich values only;
|
||||||
`PromptValue` is deliberately excluded. `StanzaValue` decodes a named rich
|
`PromptValue` is deliberately excluded. `StanzaValue` decodes a named rich
|
||||||
stanza into a caller-supplied type, reporting a missing stanza separately from
|
stanza into a caller-supplied type, reporting a missing stanza separately from
|
||||||
a decoding error.
|
a decoding error.
|
||||||
@@ -47,7 +47,7 @@ are validated by the briefing registry.
|
|||||||
|
|
||||||
## Rich and prompt-facing values
|
## Rich and prompt-facing values
|
||||||
|
|
||||||
Rich values remain available to snapshots, comparisons, and render contexts.
|
Rich values remain available to module snapshots and render contexts.
|
||||||
Briefing attaches custom prompt exports only for current conditions, hourly
|
Briefing attaches custom prompt exports only for current conditions, hourly
|
||||||
forecast, and derived daypart summaries; all other current builders use
|
forecast, and derived daypart summaries; all other current builders use
|
||||||
pass-through values. The prompt package owns how exported stanzas are grouped
|
pass-through values. The prompt package owns how exported stanzas are grouped
|
||||||
|
|||||||
@@ -1,29 +1,16 @@
|
|||||||
# Prompt Input Internals
|
# Prompt Input Internals
|
||||||
|
|
||||||
`internal/promptinput` converts report metadata, an ordered module snapshot,
|
`internal/promptinput` converts report metadata, an ordered module snapshot, and source warnings into the YAML `data_package` supplied inline to Promptkit. It owns the package schema, grouping, serialization, loading, and validation; it does not choose an output destination, collect weather, execute a provider, or retain packages after a command ends.
|
||||||
Recent Changes, and source warnings into the YAML `data_package` consumed by
|
|
||||||
Promptkit. It owns this package's schema, grouping, serialization, loading,
|
|
||||||
and validation—not weather collection, module construction, path choice, or
|
|
||||||
provider execution.
|
|
||||||
|
|
||||||
## Package construction
|
## Package Construction
|
||||||
|
|
||||||
`Build` produces `weatherreporter.data_package.v3`. It copies the run ID;
|
`Build` produces `weatherreporter.data_package.v4`. It copies the run ID; report ID, variant, prompt ID, generation time, timezone, local current date, and valid period; ordered briefing stanzas; and source warnings. Prompt input contains no historical comparison section.
|
||||||
report ID, variant, prompt ID, generation time, timezone, local current date,
|
|
||||||
and valid period; ordered briefing stanzas; Recent Changes; and source
|
|
||||||
warnings. A nil Recent Changes slice becomes an empty `items` list.
|
|
||||||
|
|
||||||
Briefing starts as a flat snapshot order and stanza-value map. `Build` uses
|
Briefing is a flat ordered set of stanza values. `Build` uses each output's `DataPackageValue`, so curated prompt exports take precedence and rich values are used only as a fallback. Prompt exports are selected by the [briefing registry](briefing.md), while the rich-versus-prompt contract is in [module internals](module.md).
|
||||||
each output's `DataPackageValue`, so runtime prompt exports take precedence and
|
|
||||||
rich values are used only as a fallback. Prompt exports are selected by the
|
|
||||||
[briefing registry](briefing.md), while the rich-versus-prompt contract is in
|
|
||||||
[module internals](module.md).
|
|
||||||
|
|
||||||
## YAML ordering and grouping
|
## YAML Ordering And Validation
|
||||||
|
|
||||||
Serialization keeps `metadata` directly under `briefing`. Every other known
|
Serialization keeps `metadata` directly under `briefing`. Every other known stanza is placed in one category and emitted in category order while preserving its original module order:
|
||||||
stanza is placed in exactly one category, emitted in category order and in its
|
|
||||||
original snapshot order within that category:
|
|
||||||
|
|
||||||
| Category | Current stanzas |
|
| Category | Current stanzas |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
@@ -32,30 +19,10 @@ original snapshot order within that category:
|
|||||||
| `narrative_products` | narrative forecast, discussions, and weather story |
|
| `narrative_products` | narrative forecast, discussions, and weather story |
|
||||||
| `raw_data` | current conditions and hourly forecast |
|
| `raw_data` | current conditions and hourly forecast |
|
||||||
|
|
||||||
This YAML presentation does not alter the flat snapshot model. `LoadYAML`
|
`LoadYAML` accepts this layout and reconstructs the flat order and values. It rejects misplaced, duplicate, unknown, or uncategorized stanzas. `Validate` requires the v4 schema version, report identity and period fields, and at least one ordered briefing stanza. `MarshalYAML` and `LoadYAML` validate their result. `Save` remains a reusable atomic-file helper for callers that explicitly need one; normal application execution passes marshalled YAML directly to Promptkit.
|
||||||
accepts the same category layout and reconstructs flat `Order` and `Values`,
|
|
||||||
rejecting misplaced, duplicate, unknown, or uncategorized stanzas.
|
|
||||||
|
|
||||||
## Validation and persistence
|
Focused tests cover construction, curated exports, category ordering, YAML round trips, invalid layout, validation, and atomic saves:
|
||||||
|
|
||||||
`Validate` requires the current schema version, run and report identifiers,
|
|
||||||
prompt ID, generation timestamp, timezone, current local date, valid period,
|
|
||||||
and at least one ordered briefing stanza. It rejects duplicate stanza names,
|
|
||||||
missing values, and a missing category for every non-metadata stanza.
|
|
||||||
|
|
||||||
`MarshalYAML` and `LoadYAML` validate their result. `Save` writes the serialized
|
|
||||||
YAML atomically; managed workspace paths are owned by [state internals](state.md).
|
|
||||||
Generated-text artifacts and template render contexts are later workflow
|
|
||||||
artifacts, not members of this package.
|
|
||||||
|
|
||||||
## Verification and invariants
|
|
||||||
|
|
||||||
Focused tests cover construction, curated exports, category ordering, YAML
|
|
||||||
round trips, invalid layout, validation, and atomic saves:
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go test ./internal/promptinput
|
go test ./internal/promptinput
|
||||||
```
|
```
|
||||||
|
|
||||||
The package is narrower than a template render context and never infers changes
|
|
||||||
from report prose.
|
|
||||||
|
|||||||
@@ -1,26 +1,12 @@
|
|||||||
# Promptkit Adapter Internals
|
# Promptkit Adapter Internals
|
||||||
|
|
||||||
`internal/adapters/promptkit` maps Weatherreporter's project-owned executor contract to Promptkit.
|
`internal/adapters/promptkit` maps Weatherreporter's project-owned executor contract to Promptkit. The CLI maps `promptkit` configuration to a `PromptExecutorConfig` and constructs one executor per action. Promptkit dependency types do not escape the adapter.
|
||||||
The CLI maps `promptkit` configuration to a `PromptExecutorConfig` and constructs one executor
|
|
||||||
per action. Promptkit dependency types do not escape the adapter.
|
|
||||||
|
|
||||||
The adapter supplies Weatherreporter's embedded prompt, schema, and fallback
|
The adapter supplies Weatherreporter's embedded prompt, schema, and fallback profile filesystems to each engine. Promptkit resolves configured operator profile sources, the embedded fallback catalog, and its built-in catalog; the adapter does not parse profile YAML, merge sources, or probe endpoints.
|
||||||
profile filesystems to each engine. Promptkit remains responsible for resolving
|
|
||||||
the configured operator profile source, application fallback catalog, and its
|
|
||||||
built-in catalog; the adapter does not parse profile YAML, merge sources, or
|
|
||||||
probe endpoints.
|
|
||||||
|
|
||||||
The adapter exposes exact prompt and profile inspection plus prepared execution.
|
The adapter exposes exact prompt and profile validation plus prepared execution. It maps safe prompt identity, logical profile, effective backend/model, preparation, execution, validation, and optional debug values into `promptexec`. `Execute` passes the YAML package as an inline Promptkit input; it does not construct a filesystem URI or write a package file.
|
||||||
It maps Promptkit inspection values to project-owned prompt input,
|
|
||||||
output-contract, logical profile identity, effective backend/model,
|
|
||||||
preparation, execution, validation, and optional debug values. It classifies
|
|
||||||
adapter failures without copying provider secrets or unbounded response bodies
|
|
||||||
into application errors or normal state.
|
|
||||||
|
|
||||||
The app calls the executor's preparation callback before provider execution to persist safe preparation
|
The application uses the preparation callback to record active safe provenance in memory and optionally writes content-rich diagnostics only through an explicit debug writer. The adapter returns raw output for application validation and rendering. It does not retain application state, render Markdown, choose report definitions, or send Distributor notifications.
|
||||||
provenance. Completed executions are then persisted as safe execution provenance and raw generated text
|
|
||||||
is validated by `internal/generatedtext`. The adapter does not write workspace state, render Markdown,
|
|
||||||
choose report definitions, or send Distributor notifications.
|
|
||||||
|
|
||||||
Focused tests:
|
Focused tests:
|
||||||
|
|
||||||
@@ -28,5 +14,4 @@ Focused tests:
|
|||||||
go test ./internal/adapters/promptkit ./internal/cli ./internal/app
|
go test ./internal/adapters/promptkit ./internal/cli ./internal/app
|
||||||
```
|
```
|
||||||
|
|
||||||
The public logical prompt/profile/schema contract is owned by the
|
The public logical prompt/profile/schema contract is owned by the [Promptkit integration guide](../integrations/promptkit.md).
|
||||||
[Promptkit integration guide](../integrations/promptkit.md).
|
|
||||||
|
|||||||
@@ -1,72 +1,30 @@
|
|||||||
# Report Registry Internals
|
# Report Registry Internals
|
||||||
|
|
||||||
`internal/report` owns the registry of report identities and the data declared
|
`internal/report` owns report identities, valid-period resolution, exact prompt identity and version, output names, default module composition, and Distributor path declarations. Public command syntax belongs in the [CLI reference](../cli.md); configuration aliases and overrides belong in the [configuration reference](../config.md).
|
||||||
for each one: resolution, prompt identity and version, comparison policy,
|
|
||||||
artifact group, output-copy name, default module composition, and Distributor
|
|
||||||
path declarations. The public command syntax is owned by the
|
|
||||||
[CLI reference](../cli.md); configuration aliases and overrides are owned by
|
|
||||||
the [configuration reference](../config.md).
|
|
||||||
|
|
||||||
## Definitions and resolution
|
## Definitions And Resolution
|
||||||
|
|
||||||
Each `Definition` declares a stable ID and display name, prompt ID, generation
|
Each `Definition` declares a stable ID and display name, prompt ID and version, template and generated-text schema IDs, valid-period resolver, default output name, Distributor path templates, module list, and fixed batch eligibility. `Resolved` combines a definition with one valid period and run identity.
|
||||||
version, template and generated-text schema IDs, valid-period resolver,
|
|
||||||
comparison strategy, artifact group, batch-copy filename, Distributor path
|
|
||||||
templates, generation eligibility, compatible prior IDs, default modules, and
|
|
||||||
batch eligibility flags. `Resolved` combines that definition with the valid
|
|
||||||
period and run metadata for one invocation.
|
|
||||||
|
|
||||||
| Report ID | Prompt version | Default profile | Period policy | Comparison | Registry batch flag | Output copy |
|
| Report ID | Prompt version | Default profile | Period policy | Fixed batch flag | Default output |
|
||||||
| --- | --- | --- | --- | --- | --- | --- |
|
| --- | --- | --- | --- | --- | --- |
|
||||||
| `daily` | `1.1.0` | `weather-balanced` | Explicit local civil day | Same valid date | Dynamic Daily inclusion is app-owned | `daily.md` |
|
| `daily` | `2.0.0` | `weather-balanced` | Explicit local civil day | Dynamic Daily inclusion is app-owned | `daily-YYYY-MM-DD.md` |
|
||||||
| `today` | `1.1.0` | `weather-balanced` | Selected or current local civil day | Same valid date | Morning | `today.md` |
|
| `today` | `2.0.0` | `weather-balanced` | Selected or current local civil day | Morning | `today.md` |
|
||||||
| `tomorrow` | `1.1.0` | `weather-balanced` | Next local civil day | Same valid date | Evening | `tomorrow.md` |
|
| `tomorrow` | `2.0.0` | `weather-balanced` | Next local civil day | Evening | `tomorrow.md` |
|
||||||
| `hourly` | `1.1.0` | `weather-light` | Rolling six-hour interval | Rolling window | — | `hourly.md` |
|
| `hourly` | `2.0.0` | `weather-light` | Rolling six-hour interval | — | `hourly.md` |
|
||||||
|
|
||||||
Each report pairs its ID and prompt version with matching template and schema
|
Daily derives its filename from the resolved valid-period start in the effective timezone, so multiple Daily items have distinct destinations. Exact template fields and schema assets belong to [report templates](../templates.md) and [generated-text internals](generatedtext.md). Prompt assets own default profile selection; the registry stores no provider setting.
|
||||||
IDs. Exact template fields and schema assets belong to [report templates](../templates.md)
|
|
||||||
and [generated-text internals](generatedtext.md). Prompt assets own default
|
|
||||||
profile selection; the registry deliberately stores no provider setting. The
|
|
||||||
[Promptkit integration guide](../integrations/promptkit.md) owns profile
|
|
||||||
definitions and resolution.
|
|
||||||
|
|
||||||
All valid periods are half-open.
|
## Collaborators And Boundaries
|
||||||
|
|
||||||
## Registry collaborators
|
`DefaultRegistry`, `Lookup`, `Resolve`, and report-name helpers prevent callers from duplicating report identity rules. Registry overrides clone a recognized definition and replace its module list. `DistributorPathTemplates` are consumed by app orchestration; their rendered external bundle-path contract is documented in the [Distributor bundle guide](../integrations/distributor/pkg-bundle.md).
|
||||||
|
|
||||||
`DefaultRegistry` is the only source of the four report definitions.
|
`morning` and `evening` are registry-owned batch names. Fixed flags declare Today and Tomorrow eligibility; app orchestration determines data-dependent Daily membership and the actual batch plan.
|
||||||
`Lookup`, `Resolve`, and report-name helpers prevent callers from duplicating
|
|
||||||
report identity rules. Registry overrides clone a definition and replace its
|
|
||||||
module list only after the report ID is recognized.
|
|
||||||
|
|
||||||
The definition's `DistributorPathTemplates` are internal declarations consumed
|
The registry never collects weather data, parses CLI flags, writes output, executes Promptkit, or delivers a report.
|
||||||
by app orchestration. Their rendered external bundle paths and compatibility
|
|
||||||
contract are documented in the [Distributor bundle guide](../integrations/distributor/pkg-bundle.md), not repeated here.
|
|
||||||
|
|
||||||
`morning` and `evening` are registry-owned batch names. Registry flags declare
|
Focused tests cover definition completeness, command and alias lookup, period resolution, run IDs, output names, composition defaults, and override validation:
|
||||||
fixed report eligibility; app orchestration determines data-dependent Daily
|
|
||||||
membership and produces the actual batch plan.
|
|
||||||
|
|
||||||
## Module composition and failures
|
|
||||||
|
|
||||||
Each definition supplies an ordered `[]module.ConfigItem`; the complete
|
|
||||||
report-to-module mapping is maintained in [module internals](module.md).
|
|
||||||
`ArtifactGroup`, `BatchOutputName`, and comparison compatibility
|
|
||||||
are likewise consumed by state and orchestration rather than recomputed there.
|
|
||||||
|
|
||||||
Unknown report IDs or batch names return errors. The registry never collects
|
|
||||||
weather data, builds modules, parses CLI flags, writes state, executes
|
|
||||||
Promptkit, or delivers a report.
|
|
||||||
|
|
||||||
## Verification and invariants
|
|
||||||
|
|
||||||
Focused tests cover definition completeness, command and alias lookup, period
|
|
||||||
resolution, run IDs, path declarations, composition defaults, and override
|
|
||||||
validation:
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go test ./internal/report
|
go test ./internal/report
|
||||||
```
|
```
|
||||||
|
|
||||||
All report selection goes through the registry, and the registry is the source
|
|
||||||
of truth for report identity—not rendered report text or app-local constants.
|
|
||||||
|
|||||||
@@ -1,58 +0,0 @@
|
|||||||
# State Internals
|
|
||||||
|
|
||||||
`internal/state` owns safe workspace paths, atomic artifact writes, metadata,
|
|
||||||
prior-snapshot lookup, and read-only inspection. Operators should use the
|
|
||||||
[operations guide](../operations.md) for lifecycle and retention.
|
|
||||||
|
|
||||||
## Artifact Paths
|
|
||||||
|
|
||||||
For each run, paths are grouped by artifact group and valid start date:
|
|
||||||
|
|
||||||
| Artifact | Location |
|
|
||||||
| --- | --- |
|
|
||||||
| Module snapshot | `snapshots/<group>/<date>/modules.<run-id>.json` |
|
|
||||||
| Metadata | `snapshots/<group>/<date>/metadata.<run-id>.json` |
|
|
||||||
| Data package | `data-packages/<group>/<date>/data_package.<run-id>.yaml` |
|
|
||||||
| Prompt preparation | `preflight/<group>/<date>/prompt_preparation.<run-id>.json` |
|
|
||||||
| Prompt execution | `snapshots/<group>/<date>/prompt_execution.<run-id>.json` |
|
|
||||||
| Raw generated text | `snapshots/<group>/<date>/generated_text_raw.<run-id>.json` |
|
|
||||||
| Validated generated text | `snapshots/<group>/<date>/generated_text.<run-id>.json` |
|
|
||||||
| Render context | `snapshots/<group>/<date>/render_context.<run-id>.json` |
|
|
||||||
| Managed report | `reports/<group>/<date>/report.<run-id>.md` |
|
|
||||||
| Notification | `notifications/<group>/<date>/distributor.<run-id>.json` |
|
|
||||||
|
|
||||||
Batch notification records are `notifications/batches/<batch>/<local-date>/distributor.<batch-run-id>.json`.
|
|
||||||
|
|
||||||
## Metadata And Debug Storage
|
|
||||||
|
|
||||||
New metadata is `weatherreporter.metadata.v2` and gains preparation and
|
|
||||||
execution paths only after those artifacts are saved. Legacy V1 records remain
|
|
||||||
readable; their historic preflight and generated-text-result fields are mapped
|
|
||||||
to the corresponding preparation and execution views during inspection. New
|
|
||||||
runs never write V1 records.
|
|
||||||
|
|
||||||
Prompt preparation and execution records are validated on both save and load.
|
|
||||||
They require exact report/prompt identity, complete timing, internally
|
|
||||||
consistent provenance, and status-appropriate validation or bounded classified
|
|
||||||
errors. Completed preparation and execution provenance retain the selected
|
|
||||||
logical profile ID and resolved backend/model, but never profile endpoints or
|
|
||||||
credentials. Completed execution provenance keeps Promptkit's run identity
|
|
||||||
distinct from the Weatherreporter run identity.
|
|
||||||
|
|
||||||
For a completed prompt run, the execution record is atomically replaced after
|
|
||||||
each downstream artifact is saved. Its path set therefore records the raw and
|
|
||||||
normalized generated text, render context, managed report, requested output
|
|
||||||
copy, and notification artifact actually reached without changing the original
|
|
||||||
Promptkit outcome.
|
|
||||||
|
|
||||||
`PromptDebugWriter` is separate from workspace state. An empty root disables
|
|
||||||
it. An enabled absolute root is checked for safe directories and symlinks, then
|
|
||||||
stores `preparation.json` and `execution.json` beneath
|
|
||||||
`<root>/<report-id>/<valid-date>/<run-id>/`. Directories are `0700`; files are
|
|
||||||
atomic `0600`. Normal state discovery does not read this root.
|
|
||||||
|
|
||||||
Focused checks:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/state
|
|
||||||
```
|
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
# Weather Data Internals
|
# Weather Data Internals
|
||||||
|
|
||||||
`internal/weatherdata` owns the normalized, wire-independent weather bundle
|
`internal/weatherdata` owns the normalized, wire-independent weather bundle
|
||||||
that passes from collection through rendering and persistence. The Weather API
|
that passes from collection through rendering. The Weather API
|
||||||
adapter translates provider responses into these types; its request, response,
|
adapter translates provider responses into these types; its request, response,
|
||||||
and availability contract is documented in the
|
and availability contract is documented in the
|
||||||
[Weather API integration guide](../integrations/weatherapi.md).
|
[Weather API integration guide](../integrations/weatherapi.md).
|
||||||
@@ -54,8 +54,7 @@ local provenance and whole-run consumers see it. A policy that treats a missing
|
|||||||
source as an error returns no partial bundle.
|
source as an error returns no partial bundle.
|
||||||
|
|
||||||
Warnings describe data completeness, not rendering or delivery failures.
|
Warnings describe data completeness, not rendering or delivery failures.
|
||||||
Those failures are recorded by the application and state layers; see
|
Those failures are reported by [application orchestration](app-orchestration.md).
|
||||||
[application orchestration](app-orchestration.md) and [state internals](state.md).
|
|
||||||
|
|
||||||
## Boundaries and verification
|
## Boundaries and verification
|
||||||
|
|
||||||
|
|||||||
@@ -1,31 +1,66 @@
|
|||||||
# Weatherreporter Operations
|
# Weatherreporter Operations
|
||||||
|
|
||||||
This guide covers normal operation, managed workspace state, inspection,
|
This guide covers normal output handling, Distributor notification, secure
|
||||||
recovery, and operational caveats. See the [CLI reference](cli.md) for complete
|
prompt diagnostics, and cleanup of legacy application state. See the [CLI
|
||||||
command syntax and the [configuration reference](config.md) for fields,
|
reference](cli.md) for command syntax and the [configuration reference](config.md)
|
||||||
defaults, and notification templates. For symptom-based diagnosis, see
|
for fields, defaults, and notification templates.
|
||||||
[Troubleshooting](troubleshooting.md).
|
|
||||||
|
|
||||||
## Normal Operation
|
## Normal Operation
|
||||||
|
|
||||||
After configuring a Weather API endpoint, generate one report:
|
After configuring a Weather API endpoint, generate one report:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
weatherreporter generate today --out ./today.md
|
weatherreporter generate today
|
||||||
```
|
```
|
||||||
|
|
||||||
A generation collects weather data, resolves the report period, builds and
|
The command writes `today.md` in the current directory. Choose a different
|
||||||
persists the module snapshot and prompt data package, records Promptkit
|
operator-owned file with `--out`; a relative path is resolved from the current
|
||||||
preparation provenance before provider execution, then persists raw output and
|
directory and an absolute path is used directly. Weatherreporter renders in
|
||||||
execution provenance, validates the structured generated text, and renders the
|
memory and atomically replaces the selected destination only after generation
|
||||||
managed Markdown report from the validated text and deterministic values.
|
and rendering succeed. It does not create a default workspace, metadata,
|
||||||
The current receipts are transitional workspace state, not a cross-version
|
receipts, or intermediate output files.
|
||||||
profile-provenance contract.
|
|
||||||
|
|
||||||
The managed report and its final metadata are saved before single-report
|
Before a destination is published, provider, validation, rendering, write, and
|
||||||
Distributor notification is attempted. `--out` writes an extra operator copy;
|
cancellation failures leave an existing report unchanged. A notification
|
||||||
it never changes the managed report or upload source. A successful generate
|
failure happens after publication, so retain and use the completed Markdown
|
||||||
command prints its summary to stdout unless `--quiet` is used.
|
file while resolving the delivery error. The JSON result identifies the
|
||||||
|
absolute output path and active profile, backend, model, warnings, validation,
|
||||||
|
debug, and notification information; see the [CLI reference](cli.md) for its
|
||||||
|
exact fields.
|
||||||
|
|
||||||
|
## Batch Outputs And Distributor Notification
|
||||||
|
|
||||||
|
Run a scheduled batch with an explicit output directory when appropriate:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
weatherreporter run morning --out-dir ./reports
|
||||||
|
```
|
||||||
|
|
||||||
|
Without `--out-dir`, batch reports are written beneath the current directory.
|
||||||
|
Morning runs Today, Tomorrow, and every eligible dated Daily Report; evening
|
||||||
|
runs Tomorrow and the same eligible Daily Reports. Eligible Daily dates begin
|
||||||
|
after tomorrow and require complete hourly coverage for their local civil day.
|
||||||
|
A batch collects once, determines the complete report set, and validates every
|
||||||
|
final output destination before executing its first report prompt. A destination
|
||||||
|
collision, such as a directory named `tomorrow.md`, stops the batch before any
|
||||||
|
report output is created or replaced. After successful validation, each selected
|
||||||
|
report processes independently and successful outputs remain available if
|
||||||
|
another report fails.
|
||||||
|
|
||||||
|
When `notify.distributor.enabled` and batch notification are enabled,
|
||||||
|
Weatherreporter sends one Distributor upload only after every selected output
|
||||||
|
exists. If an item fails, the batch notification is skipped and successful
|
||||||
|
files remain at their selected destinations. A batch notification failure also
|
||||||
|
leaves all successfully published report files in place. Distributor source
|
||||||
|
files are those operator-owned Markdown outputs; rendered bundle paths and
|
||||||
|
delivery status appear in the result, not in a local notification receipt.
|
||||||
|
Report counters count report items only. A batch notification failure therefore
|
||||||
|
returns a failed batch status even when all report counters show success; the
|
||||||
|
top-level notification result contains the delivery diagnostic.
|
||||||
|
|
||||||
|
For a single report, Distributor notification follows the atomic output write.
|
||||||
|
See the [configuration reference](config.md) for pipeline, bundle,
|
||||||
|
idempotency-key, and per-report path templates.
|
||||||
|
|
||||||
## Local Prompt Profile Override
|
## Local Prompt Profile Override
|
||||||
|
|
||||||
@@ -37,19 +72,11 @@ set its `endpoint` and `model` for the local server, and configure the copy as
|
|||||||
completely replaces the embedded definition; it does not affect a report that
|
completely replaces the embedded definition; it does not affect a report that
|
||||||
selects another profile ID.
|
selects another profile ID.
|
||||||
|
|
||||||
For example, install the profile file at a known absolute path and set:
|
Prompt and profile validation occurs before weather collection. A malformed
|
||||||
|
profile file, missing required credential, or unsupported selected backend
|
||||||
```yaml
|
stops the command before collection. A reachable profile can still fail later
|
||||||
promptkit:
|
if its local model endpoint is unavailable; Weatherreporter does not switch to
|
||||||
profile_file: /etc/weatherreporter/weather-light-local-profile.yml
|
a remote profile.
|
||||||
```
|
|
||||||
|
|
||||||
Prompt inspection occurs before weather collection. A malformed profile file,
|
|
||||||
missing required credential, or unsupported selected backend stops the command
|
|
||||||
before collection. A reachable profile can still fail later if its local model
|
|
||||||
endpoint is unavailable; Weatherreporter does not switch to a remote profile.
|
|
||||||
See the [configuration reference](config.md) for field definitions and the
|
|
||||||
[troubleshooting guide](troubleshooting.md) for recovery.
|
|
||||||
|
|
||||||
## Optional Prompt Debug Capture
|
## Optional Prompt Debug Capture
|
||||||
|
|
||||||
@@ -59,153 +86,41 @@ Use `--llm-debug-dir` only when content-rich prompt diagnostics are required:
|
|||||||
weatherreporter generate today --llm-debug-dir /var/tmp/weatherreporter-debug
|
weatherreporter generate today --llm-debug-dir /var/tmp/weatherreporter-debug
|
||||||
```
|
```
|
||||||
|
|
||||||
The directory must be absolute and is initialized before prompt inspection or
|
The directory must be absolute. Requested captures are written with restrictive
|
||||||
weather collection. Capture files are stored outside the managed workspace,
|
permissions beneath the supplied directory, organized by report and run. They
|
||||||
with restrictive permissions, under the report ID, valid date, and RunID.
|
can contain rendered prompts and generated output, so limit access to trusted
|
||||||
They can contain rendered prompts and generated output, so the normal metadata,
|
operators and remove the captures when they are no longer needed. Normal output,
|
||||||
CLI summary, and routine logs contain only the optional directory path—not
|
summaries, and routine logs omit that sensitive content. Debug capture is never
|
||||||
their content. A capture-write failure stops that run before later work can
|
created for an ordinary command without `--llm-debug-dir`.
|
||||||
continue.
|
|
||||||
|
|
||||||
Run a scheduled batch with the same configured collection:
|
If capture creation or writing fails, the affected run fails rather than
|
||||||
|
silently continuing without the requested diagnostics.
|
||||||
|
|
||||||
|
## Diagnosing Failures
|
||||||
|
|
||||||
|
Start with the command error and JSON summary. For a report generation failure,
|
||||||
|
the selected destination was not replaced; for a notification failure, inspect
|
||||||
|
the completed destination and the notification result. For a batch failure,
|
||||||
|
use the per-report statuses and retain successful output files. Enable explicit
|
||||||
|
debug capture only when content-rich Promptkit diagnostics are necessary.
|
||||||
|
|
||||||
|
Weatherreporter does not retain runs for later inspection, resume failed work,
|
||||||
|
or provide automatic cleanup, archival, remote state, daemon operation, or
|
||||||
|
automatic storm monitoring.
|
||||||
|
|
||||||
|
## Manual Cleanup Of Legacy Workspaces
|
||||||
|
|
||||||
|
Older installations may have a directory named `workspace` containing reports,
|
||||||
|
snapshots, prompt inputs, or notification records from previous versions.
|
||||||
|
Current commands neither read nor update it. After confirming that no separate
|
||||||
|
retention requirement applies, remove that specific legacy directory manually;
|
||||||
|
do not use a broad cleanup command that could remove current operator outputs.
|
||||||
|
|
||||||
|
For example, from the directory that contains the old directory:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
weatherreporter run morning --out-dir ./reports --llm-debug-dir /var/tmp/weatherreporter-debug
|
rm -rf ./workspace
|
||||||
```
|
```
|
||||||
|
|
||||||
Each batch validates its configured prompt/profile candidates, then collects once before it plans reports. Morning runs Today, Tomorrow,
|
This removal cannot be recovered by Weatherreporter. Keep or archive any
|
||||||
and every eligible dated Daily Report; evening runs Tomorrow and the same
|
historical files that are still needed before deleting them.
|
||||||
eligible Daily Reports. Eligible Daily dates begin after tomorrow and require
|
|
||||||
complete hourly coverage for their entire local civil day. A batch continues
|
|
||||||
after an individual report fails and returns an aggregate failure when any
|
|
||||||
report or batch notification fails.
|
|
||||||
|
|
||||||
`--out-dir` writes extra copies such as `today.md`, `tomorrow.md`, and
|
|
||||||
`daily-YYYY-MM-DD.md`. These copies are never upload sources. Batch report
|
|
||||||
copies and notification behavior are summarized in the CLI result; use the
|
|
||||||
[CLI reference](cli.md) for its exact JSON and stderr contract.
|
|
||||||
|
|
||||||
## Managed Workspace
|
|
||||||
|
|
||||||
The default workspace root is `workspace`. Artifact paths use the report
|
|
||||||
definition's artifact group, the valid-period start date in the effective
|
|
||||||
timezone, and the RunID:
|
|
||||||
|
|
||||||
```text
|
|
||||||
workspace/
|
|
||||||
reports/<artifact_group>/<YYYY-MM-DD>/report.<run_id>.md
|
|
||||||
|
|
||||||
snapshots/<artifact_group>/<YYYY-MM-DD>/modules.<run_id>.json
|
|
||||||
snapshots/<artifact_group>/<YYYY-MM-DD>/metadata.<run_id>.json
|
|
||||||
snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text_raw.<run_id>.json
|
|
||||||
snapshots/<artifact_group>/<YYYY-MM-DD>/prompt_execution.<run_id>.json
|
|
||||||
snapshots/<artifact_group>/<YYYY-MM-DD>/generated_text.<run_id>.json
|
|
||||||
snapshots/<artifact_group>/<YYYY-MM-DD>/render_context.<run_id>.json
|
|
||||||
|
|
||||||
data-packages/<artifact_group>/<YYYY-MM-DD>/data_package.<run_id>.yaml
|
|
||||||
preflight/<artifact_group>/<YYYY-MM-DD>/prompt_preparation.<run_id>.json
|
|
||||||
|
|
||||||
notifications/<artifact_group>/<YYYY-MM-DD>/distributor.<run_id>.json
|
|
||||||
notifications/batches/<batch>/<YYYY-MM-DD>/distributor.<batch_run_id>.json
|
|
||||||
```
|
|
||||||
|
|
||||||
The generated-text and render-context artifacts are written for every completed
|
|
||||||
single-report generation.
|
|
||||||
A report's metadata links the module snapshot, data package, preparation and
|
|
||||||
execution receipts, managed report, generated-text artifacts, and any available single-report
|
|
||||||
notification artifact. These current-version receipts remain transitional; use
|
|
||||||
the active command's classified error and explicit secure debug capture for
|
|
||||||
prompt diagnosis rather than relying on them as a durable interface. Batch
|
|
||||||
notification artifacts are separate batch-level records under
|
|
||||||
`notifications/batches`.
|
|
||||||
|
|
||||||
RunIDs begin with the UTC generation timestamp and report ID. A Daily RunID
|
|
||||||
also contains its local valid date so multiple Daily reports in one batch have
|
|
||||||
different managed paths. Batch notification RunIDs contain the UTC batch start
|
|
||||||
timestamp and batch name.
|
|
||||||
|
|
||||||
## Distributor Notification
|
|
||||||
|
|
||||||
When `notify.distributor.enabled` is enabled, a successful `generate`
|
|
||||||
uploads only the managed Markdown report after final metadata has been saved.
|
|
||||||
The extra copy from `--out` is never uploaded. A notification attempt writes
|
|
||||||
a redacted debug artifact at
|
|
||||||
`notifications/<artifact_group>/<YYYY-MM-DD>/distributor.<run_id>.json`; its
|
|
||||||
path is then recorded in report metadata.
|
|
||||||
|
|
||||||
Batches suppress per-report notification. When both Distributor and its batch
|
|
||||||
notification are enabled, Weatherreporter submits one multi-report upload after
|
|
||||||
every planned report succeeds. If any report fails, it records a top-level
|
|
||||||
`skipped` notification with reason `one or more reports failed` and does not
|
|
||||||
call Distributor. If batch notification is disabled, a batch does not fall back
|
|
||||||
to individual uploads.
|
|
||||||
|
|
||||||
A batch notification attempt writes
|
|
||||||
`notifications/batches/<batch>/<YYYY-MM-DD>/distributor.<batch_run_id>.json`.
|
|
||||||
A notification failure makes the batch fail but does not change successful
|
|
||||||
individual report items into failed items. The debug artifacts contain rendered
|
|
||||||
identifiers, managed source and bundle paths, upload and status results, and
|
|
||||||
redacted errors; they do not contain tokens.
|
|
||||||
|
|
||||||
## Inspecting Stored Runs
|
|
||||||
|
|
||||||
Inspection is read-only: it neither collects weather data nor invokes
|
|
||||||
Promptkit or Distributor. Start by finding a RunID:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
weatherreporter inspect reports --limit 10
|
|
||||||
weatherreporter inspect metadata RUN_ID
|
|
||||||
```
|
|
||||||
|
|
||||||
| Command | Reads |
|
|
||||||
| --- | --- |
|
|
||||||
| `inspect reports` | Metadata files under the workspace snapshots tree. |
|
|
||||||
| `inspect metadata RUN_ID` | Metadata located by RunID. |
|
|
||||||
| `inspect modules RUN_ID` | The module snapshot path recorded in metadata. |
|
|
||||||
| `inspect data-package RUN_ID` | The data-package path recorded in metadata. |
|
|
||||||
| `inspect prior RUN_ID` | The run metadata, then compatible earlier metadata for its comparison policy. |
|
|
||||||
| `inspect sources RUN_ID` | Source provenance and warnings in the run metadata. |
|
|
||||||
|
|
||||||
A missing snapshots directory produces no listed reports. An unknown or empty
|
|
||||||
RunID is an error; use `inspect reports` to obtain a valid value.
|
|
||||||
|
|
||||||
New runs write `weatherreporter.metadata.v2`, including preparation and
|
|
||||||
execution references once those receipts exist. `inspect metadata` also reads
|
|
||||||
historic V1 records; their legacy preflight and generated-text-result fields
|
|
||||||
remain visible for compatibility, but Weatherreporter does not write them for
|
|
||||||
new runs.
|
|
||||||
|
|
||||||
## Recovery
|
|
||||||
|
|
||||||
Keep the workspace when a run fails: artifacts reached before the failure
|
|
||||||
remain available where they can be safely persisted.
|
|
||||||
|
|
||||||
- A preparation failure can leave its classified receipt and metadata.
|
|
||||||
- A report-generation failure can leave the managed report, module snapshot,
|
|
||||||
data package, and metadata.
|
|
||||||
- A completed prompt validation rejection leaves raw text, an execution receipt,
|
|
||||||
and metadata. Later generated-text failures can also leave validated text and
|
|
||||||
a render-context artifact, depending on where they stopped.
|
|
||||||
- A single-report notification failure preserves the report and final metadata,
|
|
||||||
including its notification artifact when it was written.
|
|
||||||
- A batch notification failure preserves each report's artifacts and adds the
|
|
||||||
top-level batch notification artifact.
|
|
||||||
|
|
||||||
Use the action summary and its classified error first. For prompt or provider
|
|
||||||
diagnosis, prefer an explicitly enabled secure debug capture; current-version
|
|
||||||
receipt paths may provide supplemental context when available. For a batch
|
|
||||||
failure, inspect the summary first, then inspect the affected report RunIDs or
|
|
||||||
the batch notification path. Do not remove the whole workspace as a first
|
|
||||||
response; retain it until the failure is understood.
|
|
||||||
|
|
||||||
## Operational Caveats
|
|
||||||
|
|
||||||
- Workspace files and generated reports can contain
|
|
||||||
sensitive operational context. Set appropriate filesystem permissions and do
|
|
||||||
not publish them unintentionally.
|
|
||||||
- Weatherreporter uses one configured Weather API endpoint and local workspace
|
|
||||||
state.
|
|
||||||
- Promptkit profile resolution does not discover local endpoints or fail over
|
|
||||||
between local and remote profiles.
|
|
||||||
- It does not provide automatic resume, cleanup, archival, remote state, daemon
|
|
||||||
operation, or automatic storm monitoring.
|
|
||||||
|
|||||||
@@ -10,29 +10,29 @@ package inventory; focused documents in `docs/internal/` own implementation deta
|
|||||||
|
|
||||||
Weatherreporter is a deterministic weather-report CLI. It collects normalized
|
Weatherreporter is a deterministic weather-report CLI. It collects normalized
|
||||||
weather data, derives facts and modules, builds a curated YAML data package,
|
weather data, derives facts and modules, builds a curated YAML data package,
|
||||||
compares prior snapshots, executes exact-version Promptkit prompts, validates
|
executes exact-version Promptkit prompts, validates structured generated prose,
|
||||||
structured generated prose, and renders repository-owned Markdown. Completed
|
and renders repository-owned Markdown in memory. Completed Markdown is
|
||||||
managed Markdown may be uploaded through Distributor.
|
atomically published to an operator-owned output destination and may then be
|
||||||
|
uploaded through Distributor.
|
||||||
|
|
||||||
The supported report products are Daily, Today, Tomorrow, and Hourly. A batch
|
The supported report products are Daily, Today, Tomorrow, and Hourly. A batch
|
||||||
collects once, validates its complete candidate prompt/profile set before
|
collects once, validates its complete candidate prompt/profile set before
|
||||||
collection, then executes planned reports sequentially with one executor. It
|
collection, then determines and validates every planned output destination
|
||||||
continues after independent report failures and sends a batch notification only
|
before executing reports sequentially with one executor. It continues after
|
||||||
after every planned report succeeds.
|
independent report failures and sends a batch notification only after every
|
||||||
|
planned report succeeds.
|
||||||
|
|
||||||
## Ownership And Boundaries
|
## Ownership And Boundaries
|
||||||
|
|
||||||
- `internal/cli` owns command parsing, help, summaries, and one executor
|
- `internal/cli` owns command parsing, help, summaries, and one executor
|
||||||
construction per action.
|
construction per action.
|
||||||
- `internal/config` owns defaults, loading, validation, and secret loading.
|
- `internal/config` owns defaults, loading, validation, and secret loading.
|
||||||
- `internal/app` owns workflow order, partial results, and notification
|
- `internal/app` owns in-memory workflow order, partial results, atomic output
|
||||||
coordination through project-owned contracts.
|
publication, and notification coordination through project-owned contracts.
|
||||||
- Deterministic domain packages own weather derivation, report periods, modules,
|
- Deterministic domain packages own weather derivation, report periods, modules,
|
||||||
generated-text validation, and template contexts.
|
generated-text validation, and template contexts.
|
||||||
- `internal/adapters/weatherapi`, `internal/adapters/promptkit`, and
|
- `internal/adapters/weatherapi`, `internal/adapters/promptkit`, and
|
||||||
`internal/adapters/distributor` own their external dependency mechanics.
|
`internal/adapters/distributor` own their external dependency mechanics.
|
||||||
- `internal/state` owns workspace paths, V2 metadata, atomic persistence, and
|
|
||||||
read-only inspection.
|
|
||||||
|
|
||||||
Dependency-specific Promptkit types remain inside its adapter. The application
|
Dependency-specific Promptkit types remain inside its adapter. The application
|
||||||
does not parse flags, construct provider clients, or render provider output
|
does not parse flags, construct provider clients, or render provider output
|
||||||
@@ -41,28 +41,32 @@ directly.
|
|||||||
## Prompt Execution Invariants
|
## Prompt Execution Invariants
|
||||||
|
|
||||||
- Prompts receive curated module packages, never unbounded raw weather payloads.
|
- Prompts receive curated module packages, never unbounded raw weather payloads.
|
||||||
- Every execution inspects the exact prompt version and output contract before
|
- Every execution validates the exact prompt version and output contract before
|
||||||
collection. The selected profile is configured explicitly or declared by the
|
collection. The selected profile is configured explicitly or declared by the
|
||||||
prompt; unsupported direct-key profiles and missing reported credentials fail
|
prompt; unsupported direct-key profiles and missing reported credentials fail
|
||||||
before collection.
|
before collection.
|
||||||
- Prepared execution persists safe preparation provenance before provider work.
|
- Prompt and profile validation completes before weather collection. Raw output
|
||||||
Completed execution persists safe execution provenance; raw output is
|
is validated before template rendering.
|
||||||
validated before template rendering.
|
|
||||||
- Generated text fills defined prose slots only. Deterministic facts remain
|
- Generated text fills defined prose slots only. Deterministic facts remain
|
||||||
authoritative and repository-owned templates produce all managed Markdown.
|
authoritative and repository-owned templates produce all Markdown output.
|
||||||
- Sensitive rendered prompts, schemas, input bodies, provider endpoints, and
|
- Sensitive rendered prompts, schemas, input bodies, provider endpoints, and
|
||||||
credentials never enter normal metadata, summaries, logs, or workspace
|
credentials never enter normal summaries or logs. They are written only to
|
||||||
artifacts. They are written only to an explicit secure debug root when
|
an explicit secure debug root when requested.
|
||||||
requested.
|
|
||||||
|
|
||||||
## State, Notification, And Testing Invariants
|
## Output, Notification, And Testing Invariants
|
||||||
|
|
||||||
- Managed writes are atomic where practical and stay beneath the configured
|
- Normal execution is stateless: it keeps weather data, prompt input, generated
|
||||||
workspace root. Reached artifacts remain inspectable after later failures.
|
text, and render context in memory and creates no application-owned durable
|
||||||
- New records use `weatherreporter.metadata.v2`; V1 records remain readable for
|
state.
|
||||||
inspection compatibility.
|
- Markdown writes are atomic at an operator-selected destination. A
|
||||||
- Distributor uploads use only the managed Markdown report, never output copies
|
pre-publication failure, including cancellation observed immediately before
|
||||||
or workspace scans. Notification follows report and final metadata success.
|
publication, does not replace an existing destination; a notification failure
|
||||||
|
does not remove a newly published output.
|
||||||
|
- Distributor uploads use only the published Markdown output, never a scan of
|
||||||
|
local files. Single notification follows publication; batch notification
|
||||||
|
follows publication of every selected report. Batch counters describe report
|
||||||
|
outcomes only; a failed batch notification is represented separately at the
|
||||||
|
batch level.
|
||||||
- Default tests are deterministic, offline, and use Promptkit/provider fakes
|
- Default tests are deterministic, offline, and use Promptkit/provider fakes
|
||||||
rather than live provider calls. See the [testing policy](testing.md).
|
rather than live provider calls. See the [testing policy](testing.md).
|
||||||
|
|
||||||
|
|||||||
@@ -85,9 +85,8 @@ mechanisms, not secret values.
|
|||||||
| Release procedure | `docs/release.md` | Version policy, release preparation, validation, tagging, automated publication, verification, failure handling, and release ordering. | General contributor workflow, product contracts, release-specific change summaries, and implementation history. |
|
| Release procedure | `docs/release.md` | Version policy, release preparation, validation, tagging, automated publication, verification, failure handling, and release ordering. | General contributor workflow, product contracts, release-specific change summaries, and implementation history. |
|
||||||
| Release notes | `docs/releases/` | One versioned, changelog-style summary for each release, including compatibility and operator action. The file at the tagged commit supplies the corresponding Gitea release body. | Current CLI, configuration, operations, integration, architecture, and internal contracts; release procedure; implementation plans. |
|
| Release notes | `docs/releases/` | One versioned, changelog-style summary for each release, including compatibility and operator action. The file at the tagged commit supplies the corresponding Gitea release body. | Current CLI, configuration, operations, integration, architecture, and internal contracts; release procedure; implementation plans. |
|
||||||
| CLI contract | `docs/cli.md` | Commands, arguments, flags, invocation semantics, stdout and stderr behavior, summaries, and exit behavior. | Configuration field definitions, complete operating procedures, runtime filesystem layout, and command implementation. |
|
| CLI contract | `docs/cli.md` | Commands, arguments, flags, invocation semantics, stdout and stderr behavior, summaries, and exit behavior. | Configuration field definitions, complete operating procedures, runtime filesystem layout, and command implementation. |
|
||||||
| Configuration contract | `docs/config.md` | Discovery and precedence, fields, defaults, secrets, validation rules, and user-selectable values. | Complete example files, CLI syntax, runtime state lifecycle, and loading implementation. |
|
| Configuration contract | `docs/config.md` | Discovery and precedence, fields, defaults, secrets, validation rules, and user-selectable values. | Complete example files, CLI syntax, output lifecycle, and loading implementation. |
|
||||||
| Operations | `docs/operations.md` | Normal workflows, physical workspace layout, artifacts and metadata, inspection, notification behavior, recovery, cleanup, permissions, and operational caveats. | Complete CLI syntax, configuration field definitions, logical external contracts, and implementation mechanics. |
|
| Operations | `docs/operations.md` | Normal output handling, atomic replacement, notification behavior, diagnosis, explicit debug capture, manual legacy-workspace cleanup, permissions, and operational caveats. | Complete CLI syntax, configuration field definitions, logical external contracts, and implementation mechanics. |
|
||||||
| Troubleshooting | `docs/troubleshooting.md` | Recurring symptoms, likely causes, diagnostic steps, safe fixes, and links to normal-operation references. | Complete command and configuration references, routine operating procedures, and implementation detail. |
|
|
||||||
| Report template surface | `docs/templates.md` | Implemented template files and partials, render-context fields, editing rules, and maintainer-facing template examples. | Weather derivation, module implementation, generated-text validation internals, and operator procedures. |
|
| Report template surface | `docs/templates.md` | Implemented template files and partials, render-context fields, editing rules, and maintainer-facing template examples. | Weather derivation, module implementation, generated-text validation internals, and operator procedures. |
|
||||||
| External and durable integration contracts | `docs/integrations/` | Weather API, Promptkit, Distributor, external formats and protocols, durable logical paths and schemas, compatibility behavior, and upstream or downstream responsibilities. | Physical runtime placement and lifecycle, internal transformations, CLI syntax, and configuration defaults. |
|
| External and durable integration contracts | `docs/integrations/` | Weather API, Promptkit, Distributor, external formats and protocols, durable logical paths and schemas, compatibility behavior, and upstream or downstream responsibilities. | Physical runtime placement and lifecycle, internal transformations, CLI syntax, and configuration defaults. |
|
||||||
| Internal subsystem behavior | `docs/internal/` | Implementation flow, internal collaborators and state transitions, package-local guarantees and failures, and relevant tests. | Global architecture invariants, user-facing contracts, external schemas, operator procedures, and future package plans. |
|
| Internal subsystem behavior | `docs/internal/` | Implementation flow, internal collaborators and state transitions, package-local guarantees and failures, and relevant tests. | Global architecture invariants, user-facing contracts, external schemas, operator procedures, and future package plans. |
|
||||||
@@ -110,13 +109,12 @@ structure and invariants. Focused internal documents own implementation
|
|||||||
behavior. These documents may link to one another but must not maintain
|
behavior. These documents may link to one another but must not maintain
|
||||||
parallel package or behavior references.
|
parallel package or behavior references.
|
||||||
|
|
||||||
### Commands, Configuration, Operations, And Troubleshooting
|
### Commands, Configuration, And Operations
|
||||||
|
|
||||||
CLI documentation answers how to invoke Weatherreporter and what its command
|
CLI documentation answers how to invoke Weatherreporter and what its command
|
||||||
interface does. Configuration documentation answers what settings mean.
|
interface does. Configuration documentation answers what settings mean.
|
||||||
Operations answers what happens to runtime state and how to operate or recover
|
Operations answers how to handle operator-owned outputs and runtime failures,
|
||||||
the application. Troubleshooting starts from a symptom and leads to diagnosis
|
including diagnosis, explicit debug capture, and safe legacy cleanup.
|
||||||
and a safe fix.
|
|
||||||
|
|
||||||
When a workflow crosses these topics, place the complete procedure with the
|
When a workflow crosses these topics, place the complete procedure with the
|
||||||
document that owns the task and link to the other contracts. Do not duplicate
|
document that owns the task and link to the other contracts. Do not duplicate
|
||||||
|
|||||||
@@ -96,8 +96,8 @@ Use each test type where it protects a distinct risk:
|
|||||||
- Integration tests use real deterministic collaborators when correctness
|
- Integration tests use real deterministic collaborators when correctness
|
||||||
depends on their interaction, while replacing live or nondeterministic
|
depends on their interaction, while replacing live or nondeterministic
|
||||||
external boundaries.
|
external boundaries.
|
||||||
- App and CLI tests protect representative assembled generation, batch,
|
- App and CLI tests protect representative assembled generation, batch, atomic
|
||||||
inspection, persistence, and notification workflows.
|
output, and notification workflows.
|
||||||
- Fixtures must be minimal, synthetic, versioned with the behavior they
|
- Fixtures must be minimal, synthetic, versioned with the behavior they
|
||||||
exercise, and free of credentials or private data.
|
exercise, and free of credentials or private data.
|
||||||
- Golden files are appropriate only when the complete output is intentionally
|
- Golden files are appropriate only when the complete output is intentionally
|
||||||
@@ -199,9 +199,9 @@ Each behavior should have a clear test owner:
|
|||||||
- Config tests own loading, precedence, defaults, secrets, and validation.
|
- Config tests own loading, precedence, defaults, secrets, and validation.
|
||||||
- Domain tests own weather transformations and invariants.
|
- Domain tests own weather transformations and invariants.
|
||||||
- Adapter tests own HTTP, Promptkit/provider, and upload boundaries.
|
- Adapter tests own HTTP, Promptkit/provider, and upload boundaries.
|
||||||
- Orchestrator tests own workflow ordering, persistence, partial success, and
|
- Orchestrator tests own workflow ordering, output publication, partial success,
|
||||||
failure propagation.
|
and failure propagation.
|
||||||
- State tests own path derivation, atomic artifacts, lookup, and round trips.
|
- Filesystem tests own atomic writes and destination-preservation behavior.
|
||||||
- Template and generated-text tests own schemas, render contexts, and rendered
|
- Template and generated-text tests own schemas, render contexts, and rendered
|
||||||
output contracts.
|
output contracts.
|
||||||
|
|
||||||
@@ -219,8 +219,8 @@ observation:
|
|||||||
3. Use stubs when a dependency only needs controlled responses.
|
3. Use stubs when a dependency only needs controlled responses.
|
||||||
4. Use mocks when the interaction itself is contractual.
|
4. Use mocks when the interaction itself is contractual.
|
||||||
|
|
||||||
Mocks are appropriate for requirements such as uploading exactly once, saving
|
Mocks are appropriate for requirements such as uploading exactly once,
|
||||||
metadata before notification, propagating cancellation to Promptkit, or
|
notifying only after output publication, propagating cancellation to Promptkit, or
|
||||||
avoiding an external call after an earlier workflow failure. Do not use mocks
|
avoiding an external call after an earlier workflow failure. Do not use mocks
|
||||||
merely to isolate every object or reproduce the implementation's call graph.
|
merely to isolate every object or reproduce the implementation's call graph.
|
||||||
|
|
||||||
|
|||||||
@@ -123,8 +123,8 @@ git diff --cached --check
|
|||||||
|
|
||||||
Follow every added or changed Markdown link and confirm that its local target
|
Follow every added or changed Markdown link and confirm that its local target
|
||||||
exists. Review the candidate for generated binaries, test output, credentials,
|
exists. Review the candidate for generated binaries, test output, credentials,
|
||||||
temporary files, workspace files, replacements, vendored dependencies, and
|
temporary files, replacements, vendored dependencies, and other files that do
|
||||||
other files that do not belong in source control.
|
not belong in source control.
|
||||||
|
|
||||||
## Publish The Candidate Commit
|
## Publish The Candidate Commit
|
||||||
|
|
||||||
|
|||||||
140
docs/releases/v0.10.0.md
Normal file
140
docs/releases/v0.10.0.md
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
# Weatherreporter v0.10.0
|
||||||
|
|
||||||
|
Weatherreporter `v0.10.0` makes report execution stateless, adds stable
|
||||||
|
weather-specific Promptkit profiles, and turns every successful generation
|
||||||
|
into one atomic operator-owned Markdown output.
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
- Ordinary generation no longer creates or depends on a managed workspace,
|
||||||
|
historical run artifacts, metadata, receipts, or prior snapshots.
|
||||||
|
- `generate` and `run` now publish directly to operator-selected paths, with
|
||||||
|
useful current-directory defaults when output flags are omitted.
|
||||||
|
- Local Recent Changes comparison and the historical `inspect` command family
|
||||||
|
have been removed.
|
||||||
|
- Promptkit `v0.5.0` and three embedded logical profiles provide a stable model
|
||||||
|
ladder with complete file- or directory-based overrides.
|
||||||
|
- Prompt input and generated-text contracts have been tightened, and output,
|
||||||
|
cancellation, batch preflight, notification, and partial-failure behavior
|
||||||
|
have focused offline coverage.
|
||||||
|
|
||||||
|
## Compatibility
|
||||||
|
|
||||||
|
This pre-`v1` minor release intentionally breaks CLI, configuration,
|
||||||
|
prompt-input, action-summary, and workspace contracts from `v0.9.0`.
|
||||||
|
|
||||||
|
- The `workspace:` and `recent_change:` configuration sections are no longer
|
||||||
|
supported. Strict configuration loading rejects them.
|
||||||
|
- The `inspect reports`, `inspect metadata`, `inspect modules`,
|
||||||
|
`inspect data-package`, `inspect prior`, and `inspect sources` commands have
|
||||||
|
been removed. Weatherreporter no longer reads V1 or V2 run metadata or other
|
||||||
|
historical workspace artifacts.
|
||||||
|
- Every successful `generate` writes exactly one Markdown file. Without
|
||||||
|
`--out`, Daily writes `daily-YYYY-MM-DD.md` and Today, Tomorrow, and Hourly
|
||||||
|
write `today.md`, `tomorrow.md`, and `hourly.md` in the invocation's current
|
||||||
|
directory. `--out` selects that file rather than creating an extra copy of a
|
||||||
|
separately managed report.
|
||||||
|
- `run` writes selected outputs beneath the current directory unless
|
||||||
|
`--out-dir` selects another directory. Successful items remain available
|
||||||
|
when another batch item fails.
|
||||||
|
- Action summaries no longer expose managed report, metadata, snapshot, data
|
||||||
|
package, prompt preparation, prompt execution, generated-text, render-context,
|
||||||
|
or notification-receipt paths. They retain the final `outputPath`, optional
|
||||||
|
`llmDebugPath`, safe effective profile/backend/model details, validation,
|
||||||
|
warnings, notification status, and safe errors.
|
||||||
|
- Batch report items no longer contain per-report notification fields. Batch
|
||||||
|
notification is represented once at the top level. The `total`, `succeeded`,
|
||||||
|
and `failed` counters describe reports only, so notification failure can
|
||||||
|
produce a failed action while `failed` remains `0`.
|
||||||
|
- The prompt data package advances from `weatherreporter.data_package.v3` to
|
||||||
|
`weatherreporter.data_package.v4` and removes `recent_changes`. All four
|
||||||
|
embedded prompts advance from `1.1.0` to `2.0.0`.
|
||||||
|
- Generated-text schemas now require string-valued `precipitation_timing`; the
|
||||||
|
model returns an empty string when there is no timing text. The unused
|
||||||
|
`confidence` field has been removed and is rejected as an unknown field.
|
||||||
|
|
||||||
|
Existing operator-owned Markdown files remain valid. Existing workspace trees
|
||||||
|
are ignored rather than migrated or deleted. Distributor continues to receive
|
||||||
|
the completed Markdown report, but its source is now the selected operator
|
||||||
|
output rather than a managed report copy.
|
||||||
|
|
||||||
|
## Upgrade
|
||||||
|
|
||||||
|
Before replacing `v0.9.0`:
|
||||||
|
|
||||||
|
1. Remove `workspace:` and `recent_change:` from configuration files.
|
||||||
|
2. Give scheduled commands a predictable working directory or explicit
|
||||||
|
`--out` or `--out-dir` destination. Confirm that these selected files may be
|
||||||
|
atomically replaced on later successful runs.
|
||||||
|
3. Remove historical `inspect` invocations and update action-summary consumers
|
||||||
|
to use `outputPath` and the remaining active-workflow fields.
|
||||||
|
4. Decide whether old workspace contents have any external retention value.
|
||||||
|
Weatherreporter no longer reads them; after review, they may be removed
|
||||||
|
manually using the narrowly scoped procedure in the operations guide.
|
||||||
|
5. Review Promptkit profile selection and credentials. Hourly defaults to
|
||||||
|
`weather-light`; Daily, Today, and Tomorrow default to `weather-balanced`.
|
||||||
|
A configured `promptkit.profile` still overrides every report in one action.
|
||||||
|
|
||||||
|
The embedded logical profiles are:
|
||||||
|
|
||||||
|
| Profile | OpenRouter model | Default use |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `weather-light` | `deepseek/deepseek-v4-flash` | Hourly |
|
||||||
|
| `weather-balanced` | `~google/gemini-flash-latest` | Daily, Today, Tomorrow |
|
||||||
|
| `weather-deep` | `~anthropic/claude-sonnet-latest` | Explicit selection |
|
||||||
|
|
||||||
|
Override a complete same-ID definition through `promptkit.profile_file` or
|
||||||
|
`promptkit.profile_dir` to use different models or a local OpenAI-compatible
|
||||||
|
endpoint. Definitions are replaced rather than field-merged, and a malformed
|
||||||
|
matching override fails instead of silently falling back.
|
||||||
|
|
||||||
|
See the [CLI reference](../cli.md), [configuration
|
||||||
|
reference](../config.md), [operations guide](../operations.md), and [Promptkit
|
||||||
|
integration](../integrations/promptkit.md) for the exact current contracts.
|
||||||
|
|
||||||
|
## Changes
|
||||||
|
|
||||||
|
### Stateless Execution And Operator-Owned Outputs
|
||||||
|
|
||||||
|
- Removed local forecast-change comparison, prior-snapshot selection, durable
|
||||||
|
module and prompt artifacts, managed reports, metadata compatibility, run
|
||||||
|
discovery, notification receipts, and the complete `internal/state`
|
||||||
|
subsystem.
|
||||||
|
- Added an Accepted architecture decision recording the stateless
|
||||||
|
transformation pipeline and operator-owned output boundary.
|
||||||
|
- Kept weather, facts, modules, prompt input, generated text, and render context
|
||||||
|
in memory during ordinary execution.
|
||||||
|
- Made output publication atomic and ensured cancellation or deadline expiry
|
||||||
|
observed before publication leaves an existing destination unchanged.
|
||||||
|
- Added complete batch-destination preflight before the first report prompt,
|
||||||
|
so a structural collision cannot leave an unreported partial batch.
|
||||||
|
- Preserved successful outputs after report or Distributor failure. Batch
|
||||||
|
notification runs only after every selected report succeeds.
|
||||||
|
|
||||||
|
### Promptkit Profiles And Prompt Contracts
|
||||||
|
|
||||||
|
- Upgraded Promptkit from `v0.4.0` to `v0.5.0`.
|
||||||
|
- Added embedded `weather-light`, `weather-balanced`, and `weather-deep`
|
||||||
|
profiles and mapped each exact prompt to its logical default.
|
||||||
|
- Added embedded-profile fallback after configured `profile_file` or
|
||||||
|
`profile_dir` lookup, allowing operators to replace a logical profile without
|
||||||
|
changing report definitions.
|
||||||
|
- Added a maintained local-endpoint example for replacing `weather-light`.
|
||||||
|
- Advanced the four prompt definitions to `2.0.0` and the curated data package
|
||||||
|
to v4 after removing Recent Changes.
|
||||||
|
- Required `precipitation_timing`, normalized whitespace-only timing to an
|
||||||
|
empty string, and removed the unused confidence value.
|
||||||
|
|
||||||
|
### CLI, Reliability, Documentation, And Testing
|
||||||
|
|
||||||
|
- Simplified action summaries to active workflow identity, output, model,
|
||||||
|
validation, warning, debug, notification, and safe error information.
|
||||||
|
- Made batch counters report-only while retaining failed action status and
|
||||||
|
non-zero exit behavior for batch notification failure.
|
||||||
|
- Kept prompt and profile inspection ahead of weather collection and validated
|
||||||
|
every batch candidate before collecting once.
|
||||||
|
- Replaced state-oriented workflow fixtures with focused generation, batch,
|
||||||
|
output, cancellation, profile-resolution, Distributor, and CLI coverage.
|
||||||
|
- Reconciled user, operator, integration, internal, policy, and ADR
|
||||||
|
documentation around the implemented stateless architecture and removed
|
||||||
|
completed temporary roadmaps.
|
||||||
@@ -1,283 +0,0 @@
|
|||||||
# Domain-Specific Prompt Profiles Roadmap
|
|
||||||
|
|
||||||
Status: Implemented.
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Weatherreporter should provide stable, domain-specific Promptkit profile IDs
|
|
||||||
that express the relative resource and analysis needs of its report products.
|
|
||||||
These logical profiles should give each report an appropriate default while
|
|
||||||
allowing operators to replace any definition through the existing configured
|
|
||||||
profile source.
|
|
||||||
|
|
||||||
This roadmap records the scope, policy, and implemented end state. The
|
|
||||||
companion [implementation plan](implementation.md) records the ordered work
|
|
||||||
and verification used to reach it.
|
|
||||||
|
|
||||||
## User Intent
|
|
||||||
|
|
||||||
The feature is intended to provide three related benefits:
|
|
||||||
|
|
||||||
- frequent reports can use a cost-effective model by default;
|
|
||||||
- reports needing broader synthesis can select a stronger default without
|
|
||||||
forcing the same cost on every invocation; and
|
|
||||||
- an installation can map a stable Weatherreporter profile ID to a model on a
|
|
||||||
local network endpoint without modifying embedded prompts or application
|
|
||||||
code.
|
|
||||||
|
|
||||||
`weather-light` describes the profile's intended resource tier, not a latency
|
|
||||||
guarantee. A locally hosted lightweight model may still generate slowly on the
|
|
||||||
available hardware.
|
|
||||||
|
|
||||||
## Pre-Implementation Baseline
|
|
||||||
|
|
||||||
Before implementation, Daily, Today, Tomorrow, and Hourly each declared
|
|
||||||
Promptkit's `gemini-flash-latest` profile as their prompt default. The optional
|
|
||||||
`promptkit.profile` setting overrode that default for every selected report in
|
|
||||||
an invocation.
|
|
||||||
|
|
||||||
Weatherreporter accepted either `promptkit.profile_file` or
|
|
||||||
`promptkit.profile_dir` and passed that source to Promptkit. A matching external
|
|
||||||
profile could override a Promptkit built-in profile, and the configured local
|
|
||||||
backend could support profiles that select `backend: local`. Endpoint-only
|
|
||||||
OpenAI-compatible profiles could also provide their own endpoint.
|
|
||||||
|
|
||||||
Weatherreporter did not own or embed execution profiles. Promptkit v0.5.0
|
|
||||||
provided the fallback-profile layer used to add them without changing the
|
|
||||||
existing operator-source precedence.
|
|
||||||
|
|
||||||
## Prerequisite
|
|
||||||
|
|
||||||
Promptkit v0.5.0 provides the application fallback profile capability defined
|
|
||||||
in the companion
|
|
||||||
[upstream feature request](promptkit-fallback-profiles-feature-request.md), and
|
|
||||||
Weatherreporter now depends on that tagged release. The dependency upgrade has
|
|
||||||
passed the repository test suite and an operator smoke test. Weatherreporter
|
|
||||||
must continue to use only Promptkit's public API rather than depending on its
|
|
||||||
internal packages or reproducing its profile repository behavior.
|
|
||||||
|
|
||||||
## Implemented End State
|
|
||||||
|
|
||||||
Weatherreporter embeds usable definitions for these exact logical profile IDs:
|
|
||||||
|
|
||||||
- `weather-light`
|
|
||||||
- `weather-balanced`
|
|
||||||
- `weather-deep`
|
|
||||||
|
|
||||||
The profiles are Weatherreporter-owned assets and remain behind the existing
|
|
||||||
Promptkit adapter boundary. Prompt definitions select the logical IDs, while
|
|
||||||
Promptkit resolves the effective backend, endpoint, model, and generation
|
|
||||||
settings.
|
|
||||||
|
|
||||||
An operator can place a profile with the same ID in `profile_file` or
|
|
||||||
`profile_dir`. The operator definition completely replaces the embedded
|
|
||||||
Weatherreporter definition for that ID. If the external source does not contain
|
|
||||||
the selected ID, lookup falls through to Weatherreporter's embedded profile and
|
|
||||||
then to Promptkit's built-in catalog.
|
|
||||||
|
|
||||||
The existing global `promptkit.profile` setting remains available as an
|
|
||||||
explicit all-report override. No new configuration field is required for the
|
|
||||||
initial feature.
|
|
||||||
|
|
||||||
## Profile Catalog And Report Assignment
|
|
||||||
|
|
||||||
| Profile | Meaning | Initial default reports |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `weather-light` | Lowest-cost supported tier for frequent, bounded synthesis. It makes no latency promise. | Hourly |
|
|
||||||
| `weather-balanced` | General-purpose tier for broader day-scale synthesis and forecast discussion. | Daily, Today, Tomorrow |
|
|
||||||
| `weather-deep` | Highest-capability tier for explicit operator use and future products whose measured quality benefit warrants the cost. | None initially |
|
|
||||||
|
|
||||||
The initial assignment recognizes that Weatherreporter's deterministic modules
|
|
||||||
already perform most weather selection and calculation. A higher-capability
|
|
||||||
model should not become a default merely because it is available. Moving an
|
|
||||||
existing report to `weather-deep` requires evidence that the stronger tier
|
|
||||||
materially improves supported reasoning or output quality.
|
|
||||||
|
|
||||||
The three profile IDs are capability policies, not permanent aliases for one
|
|
||||||
provider or model family. Their embedded definitions may change in a future
|
|
||||||
Weatherreporter release, with the change disclosed through normal release and
|
|
||||||
compatibility documentation.
|
|
||||||
|
|
||||||
## Selection And Definition Precedence
|
|
||||||
|
|
||||||
Profile ID selection and profile definition lookup are separate decisions.
|
|
||||||
|
|
||||||
Weatherreporter selects the profile ID in this order:
|
|
||||||
|
|
||||||
1. nonblank `promptkit.profile`; or
|
|
||||||
2. the exact prompt version's `default_profile`.
|
|
||||||
|
|
||||||
Promptkit then resolves the selected profile definition in this order:
|
|
||||||
|
|
||||||
1. explicit in-memory profiles, when used by an embedding consumer or test;
|
|
||||||
2. Weatherreporter's configured `profile_file` or `profile_dir` source;
|
|
||||||
3. Weatherreporter's embedded fallback profiles; and
|
|
||||||
4. Promptkit's embedded built-in profiles.
|
|
||||||
|
|
||||||
A higher-precedence source falls through only when the selected ID is absent.
|
|
||||||
A matching but malformed operator profile fails before weather collection and
|
|
||||||
must not silently use the embedded definition.
|
|
||||||
|
|
||||||
## Local Endpoint Experience
|
|
||||||
|
|
||||||
An operator should be able to override `weather-light` with an endpoint-only
|
|
||||||
profile whose model name is understood by the local OpenAI-compatible server.
|
|
||||||
This path does not require a separate Weatherreporter local-backend setting.
|
|
||||||
|
|
||||||
Alternatively, an override may select `backend: local`; in that case the
|
|
||||||
existing `promptkit.local.endpoint` and concurrency settings continue to own
|
|
||||||
the shared local backend definition.
|
|
||||||
|
|
||||||
The selected local profile is deterministic configuration, not a preference
|
|
||||||
hint. Weatherreporter will not probe for a local model and will not
|
|
||||||
automatically fall back to a remote or paid profile when the endpoint is
|
|
||||||
unavailable. The failure remains visible and attributable to the selected
|
|
||||||
profile.
|
|
||||||
|
|
||||||
## Embedded Profile Policy
|
|
||||||
|
|
||||||
Each embedded profile must be a complete, valid Promptkit profile and must be
|
|
||||||
usable in a default installation with the documented credential mechanism. The
|
|
||||||
initial embedded profiles are expected to use Promptkit's `openrouter` backend,
|
|
||||||
allowing them to inherit its endpoint and `OPENROUTER_API_KEY` environment
|
|
||||||
variable without embedding credentials.
|
|
||||||
|
|
||||||
Embedded definitions should include only settings that are intentional for the
|
|
||||||
selected model and supported by its backend. Avoid incidental generation
|
|
||||||
parameters that reduce portability or trigger provider-specific request
|
|
||||||
failures without a demonstrated quality benefit.
|
|
||||||
|
|
||||||
The initial profile definitions are:
|
|
||||||
|
|
||||||
| Profile ID | OpenRouter model | Reasoning effort | Timeout | Service tier |
|
|
||||||
| --- | --- | --- | --- | --- |
|
|
||||||
| `weather-light` | `deepseek/deepseek-v4-flash` | Provider default | 180 seconds | `flex` |
|
|
||||||
| `weather-balanced` | `~google/gemini-flash-latest` | `high` | 240 seconds | `flex` |
|
|
||||||
| `weather-deep` | `~anthropic/claude-sonnet-latest` | `high` | 240 seconds | `flex` |
|
|
||||||
|
|
||||||
These settings deliberately match the corresponding Promptkit v0.5.0
|
|
||||||
built-ins while exposing Weatherreporter-owned logical IDs. The `~` prefix is
|
|
||||||
part of each OpenRouter rolling-alias identifier. The profiles do not set
|
|
||||||
temperature, `top_p`, or output-token limits; omission preserves provider
|
|
||||||
defaults and avoids unsupported incidental parameters.
|
|
||||||
|
|
||||||
## Prompt And Active Execution Contract
|
|
||||||
|
|
||||||
Changing a prompt's `default_profile` is a material prompt-definition change.
|
|
||||||
The four prompt definitions should advance from `1.0.1` to `1.1.0` when the new
|
|
||||||
defaults are introduced. Prompt content and generated-text schemas need not
|
|
||||||
change solely for this feature.
|
|
||||||
|
|
||||||
Prompt inspection must continue to occur before weather collection. It should
|
|
||||||
report the selected logical profile ID and the resolved backend and model
|
|
||||||
without exposing endpoints or credentials.
|
|
||||||
|
|
||||||
The active execution contract should retain both the selected logical profile
|
|
||||||
identity and the resolved backend and model through inspection, preparation,
|
|
||||||
execution, errors, and command results where those values are already exposed.
|
|
||||||
This feature must not add a new durable-provenance or cross-version artifact
|
|
||||||
contract.
|
|
||||||
|
|
||||||
The accepted [ephemeral-state roadmap](ephemeral-state.md) makes historical
|
|
||||||
prompt provenance a non-goal. Existing workspace persistence may remain while
|
|
||||||
this feature lands, but it is transitional behavior and must not be expanded or
|
|
||||||
treated as part of the profile feature's desired end state. Prompt preparation
|
|
||||||
and execution artifacts written at `1.0.1` are not required to remain readable
|
|
||||||
after the prompt definitions advance to `1.1.0`.
|
|
||||||
|
|
||||||
## Evaluation Policy
|
|
||||||
|
|
||||||
Concrete model assignments should be evaluated with representative,
|
|
||||||
secret-free Daily, Today, Tomorrow, and Hourly data packages. Evaluation should
|
|
||||||
consider:
|
|
||||||
|
|
||||||
- strict-schema success rate;
|
|
||||||
- unsupported or invented weather claims;
|
|
||||||
- precipitation-timing accuracy and empty-string behavior;
|
|
||||||
- correct use of deterministic hazards, periods, and uncertainty;
|
|
||||||
- summary and forecast-discussion usefulness;
|
|
||||||
- generation latency;
|
|
||||||
- token use and provider cost; and
|
|
||||||
- behavior through a representative local OpenAI-compatible endpoint.
|
|
||||||
|
|
||||||
The purpose is to choose an appropriate default for each tier, not to add a
|
|
||||||
permanent benchmark framework or live-provider requirement to the ordinary
|
|
||||||
test suite. Repository tests remain offline and deterministic.
|
|
||||||
|
|
||||||
## Implemented Scope
|
|
||||||
|
|
||||||
The completed feature includes:
|
|
||||||
|
|
||||||
- Weatherreporter-owned embedded profile assets for all three logical IDs;
|
|
||||||
- Promptkit adapter wiring that supplies those assets as the application
|
|
||||||
fallback profile source;
|
|
||||||
- per-prompt default-profile assignments matching the catalog above;
|
|
||||||
- an exact prompt-version update for the changed definitions;
|
|
||||||
- preservation of the global profile override;
|
|
||||||
- same-ID override behavior through both supported external profile-source
|
|
||||||
forms;
|
|
||||||
- local-backend and endpoint-only override coverage;
|
|
||||||
- fail-fast inspection of missing, malformed, or unusable selected profiles;
|
|
||||||
- offline tests for selection, source precedence, effective model inspection,
|
|
||||||
batch reuse, and active execution behavior;
|
|
||||||
- maintained operator examples for overriding `weather-light` locally; and
|
|
||||||
- updates to the canonical configuration, Promptkit integration, report
|
|
||||||
registry, operations, troubleshooting, internal adapter, and release
|
|
||||||
documentation as applicable when implementation lands.
|
|
||||||
|
|
||||||
## Non-Goals
|
|
||||||
|
|
||||||
The feature does not include:
|
|
||||||
|
|
||||||
- automatic discovery, health checking, or benchmarking of local endpoints;
|
|
||||||
- implicit failover between local and remote profiles;
|
|
||||||
- retries with a more expensive tier after provider or validation failure;
|
|
||||||
- per-report profile configuration fields outside prompt defaults;
|
|
||||||
- profile inheritance, aliases, or field-level merging;
|
|
||||||
- runtime model selection based on weather severity, token count, or report
|
|
||||||
content;
|
|
||||||
- moving Weatherreporter profile policy into Promptkit's built-in catalog;
|
|
||||||
- exposing Promptkit types outside the adapter boundary; or
|
|
||||||
- making live provider calls part of the default repository test suite.
|
|
||||||
|
|
||||||
## Compatibility And Operational Policy
|
|
||||||
|
|
||||||
Existing configurations with a nonblank `promptkit.profile` retain their
|
|
||||||
all-report behavior. Existing `profile_file`, `profile_dir`, local-backend, and
|
|
||||||
credential configuration fields retain their meanings.
|
|
||||||
|
|
||||||
Configurations that rely on the omitted profile setting will intentionally
|
|
||||||
observe new per-report defaults. This is a user-visible model-selection and
|
|
||||||
cost change and must be called out in release notes. Operators who require the
|
|
||||||
old all-report model can preserve it by setting an explicit global profile.
|
|
||||||
|
|
||||||
The prompt-version transition does not provide backward compatibility for
|
|
||||||
historical prompt preparation or execution artifacts. This is consistent with
|
|
||||||
the accepted ephemeral-state direction; the profile feature does not otherwise
|
|
||||||
redesign or remove the current workspace layout.
|
|
||||||
|
|
||||||
An external same-ID override is an operator-owned compatibility commitment.
|
|
||||||
Weatherreporter may evolve its embedded definitions, but it must not rewrite or
|
|
||||||
silently merge an operator file.
|
|
||||||
|
|
||||||
## Completion Record
|
|
||||||
|
|
||||||
The following conditions are satisfied:
|
|
||||||
|
|
||||||
- a tagged Promptkit dependency supports the required fallback layer;
|
|
||||||
- every operational prompt selects its assigned logical profile at exact
|
|
||||||
version `1.1.0`;
|
|
||||||
- all three embedded profiles inspect successfully without an external profile
|
|
||||||
source;
|
|
||||||
- configured same-ID definitions override embedded definitions through both
|
|
||||||
`profile_file` and `profile_dir`;
|
|
||||||
- an invalid matching external definition fails without fallback;
|
|
||||||
- `weather-light` can resolve through an endpoint-only or configured-local
|
|
||||||
override without requiring code or prompt changes;
|
|
||||||
- global `promptkit.profile` still overrides every report in an invocation;
|
|
||||||
- active inspection and execution preserve the selected logical profile and
|
|
||||||
effective model through the project-owned execution contract;
|
|
||||||
- morning and evening batch preflight deduplicates inspection of shared
|
|
||||||
effective profile IDs as it does today;
|
|
||||||
- the default test suite remains offline and deterministic; and
|
|
||||||
- implemented behavior is documented by its canonical current-state owners.
|
|
||||||
@@ -1,382 +0,0 @@
|
|||||||
# Ephemeral Operational State Roadmap
|
|
||||||
|
|
||||||
Status: Accepted feature direction; implementation has not started.
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Weatherreporter should treat generated weather reports and their intermediate
|
|
||||||
artifacts as short-lived operational material rather than a durable audit
|
|
||||||
history. Forecasts and current conditions change continuously, and the normal
|
|
||||||
response to an old or failed report is to generate a new report, not to
|
|
||||||
reconstruct the provenance of the old one.
|
|
||||||
|
|
||||||
The application should retain only the bounded state needed to publish the
|
|
||||||
current report, calculate Recent Changes against the last successfully
|
|
||||||
published report for the same valid period, and complete the current
|
|
||||||
invocation safely. Detailed LLM diagnostics should remain an explicit,
|
|
||||||
operator-controlled exception outside ordinary workspace state.
|
|
||||||
|
|
||||||
This roadmap defines the intended state lifecycle, compatibility policy, and
|
|
||||||
architectural boundaries. A separate implementation plan will define the
|
|
||||||
ordered work after the roadmap is complete.
|
|
||||||
|
|
||||||
## User Intent
|
|
||||||
|
|
||||||
The state model should reflect these product expectations:
|
|
||||||
|
|
||||||
- weather reports are ephemeral products, not business records;
|
|
||||||
- old report provenance has no continuing operational value once conditions
|
|
||||||
and forecasts have changed;
|
|
||||||
- regenerating is preferable to recovering, replaying, or inspecting an old
|
|
||||||
generation;
|
|
||||||
- routine operation should not accumulate unbounded run-addressed artifacts;
|
|
||||||
- Recent Changes remains useful, but needs only one prior successful snapshot
|
|
||||||
for the same report and valid period; and
|
|
||||||
- sensitive prompt and response capture remains opt-in and explicitly managed
|
|
||||||
by the operator.
|
|
||||||
|
|
||||||
## Current State
|
|
||||||
|
|
||||||
Each generation currently writes a run-addressed collection containing a
|
|
||||||
module snapshot, data package, prompt preparation receipt, prompt execution
|
|
||||||
receipt, raw generated text, validated generated text, render context, managed
|
|
||||||
report, metadata, and optional notification receipt. Successful and failed
|
|
||||||
runs accumulate beneath the workspace.
|
|
||||||
|
|
||||||
Metadata links the collection and supports lookup by RunID. The CLI can list
|
|
||||||
historical runs and inspect their metadata, modules, data packages, prior
|
|
||||||
snapshots, and source provenance. New metadata uses the V2 format while the
|
|
||||||
reader retains V1 compatibility. Prompt artifacts are validated against
|
|
||||||
current report and prompt definitions when saved and loaded.
|
|
||||||
|
|
||||||
Most of this persistence exists for retrospective inspection and failure
|
|
||||||
recovery. Dedicated prompt preparation and execution load operations have no
|
|
||||||
ordinary production consumer. The important exception is module snapshot
|
|
||||||
state: generation actively loads the most recent compatible snapshot to build
|
|
||||||
the deterministic Recent Changes input for Daily, Today, and Tomorrow.
|
|
||||||
|
|
||||||
## Desired End State
|
|
||||||
|
|
||||||
Weatherreporter has three distinct state classes:
|
|
||||||
|
|
||||||
| State class | Lifecycle | Purpose |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| Invocation workspace | Temporary and unpublished | Hold intermediate values while one report or batch is running. |
|
|
||||||
| Current published state | Bounded and replaceable | Hold the current managed report and the minimal deterministic snapshot or manifest needed for normal operation. |
|
|
||||||
| Secure LLM debug capture | Explicitly enabled and operator-managed | Diagnose prompt rendering or provider output when the operator deliberately requests sensitive capture. |
|
|
||||||
|
|
||||||
Ordinary generation uses an invocation-scoped temporary directory on the same
|
|
||||||
filesystem as the managed workspace when atomic publication requires it.
|
|
||||||
Prompt preparation, prompt execution, raw generated text, validated generated
|
|
||||||
text, render contexts, data packages, and notification receipts may exist
|
|
||||||
there while needed, but they are not published as durable historical
|
|
||||||
artifacts.
|
|
||||||
|
|
||||||
A successful report atomically replaces the current published state for its
|
|
||||||
logical report key and valid period. A failed attempt leaves the last
|
|
||||||
successfully published report and comparison snapshot unchanged. Ordinary
|
|
||||||
temporary artifacts are removed after both success and handled failure;
|
|
||||||
cleanup failure is reported safely but must not replace the primary generation
|
|
||||||
error.
|
|
||||||
|
|
||||||
RunIDs remain useful as in-process correlation identifiers in action results,
|
|
||||||
logs, provider provenance, and optional debug paths. They no longer identify a
|
|
||||||
durable collection that Weatherreporter promises to locate or decode later.
|
|
||||||
|
|
||||||
## Published Report Policy
|
|
||||||
|
|
||||||
The managed Markdown report remains the authoritative upload source during an
|
|
||||||
invocation. The intended default is to retain only the current managed report
|
|
||||||
for each logical report key and valid period, replacing it atomically after a
|
|
||||||
new report has been fully rendered and validated.
|
|
||||||
|
|
||||||
An explicit `--out` or `--out-dir` copy remains operator-owned output outside
|
|
||||||
the managed-state lifecycle. Weatherreporter does not delete, rotate, or
|
|
||||||
rewrite those copies except when the same explicit destination is selected by
|
|
||||||
a later invocation.
|
|
||||||
|
|
||||||
Distributor continues to receive only a completed managed Markdown report.
|
|
||||||
Notification success or failure does not create a durable notification
|
|
||||||
history. A notification failure leaves the newly published report available
|
|
||||||
and returns a safe error through the current action result.
|
|
||||||
|
|
||||||
## Recent Changes State
|
|
||||||
|
|
||||||
Recent Changes must be preserved without preserving general report history.
|
|
||||||
For Daily, Today, and Tomorrow, Weatherreporter retains at most one compatible
|
|
||||||
module snapshot for each logical report key and valid period.
|
|
||||||
|
|
||||||
The retained snapshot represents the last successfully published report. A
|
|
||||||
new invocation reads it before constructing Recent Changes and replaces it
|
|
||||||
only when the new managed report has been successfully validated, rendered,
|
|
||||||
and published. A failed generation therefore does not become the baseline for
|
|
||||||
the next report and cannot hide changes that the user has not yet seen.
|
|
||||||
|
|
||||||
Hourly does not currently use the comparison strategy and should not retain a
|
|
||||||
comparison snapshot solely for symmetry. State whose valid period has ended
|
|
||||||
and can no longer participate in a supported comparison is eligible for safe
|
|
||||||
cleanup.
|
|
||||||
|
|
||||||
## Temporary Workspace And Failure Semantics
|
|
||||||
|
|
||||||
Temporary state must remain beneath a narrowly owned application directory and
|
|
||||||
use safe path construction, restrictive permissions where content is
|
|
||||||
sensitive, and atomic writes where practical. Publication must not expose a
|
|
||||||
partially rendered report or a snapshot that does not correspond to the
|
|
||||||
published report.
|
|
||||||
|
|
||||||
Normal results retain bounded error information and paths only for artifacts
|
|
||||||
that remain meaningful after the command: a previously or newly published
|
|
||||||
report, an explicit operator output, or an enabled secure debug capture.
|
|
||||||
Temporary intermediate paths are not emitted as if they were durable recovery
|
|
||||||
locations. A failed command is retried by starting a new generation.
|
|
||||||
|
|
||||||
Process interruption may leave an uncommitted temporary directory. Such a
|
|
||||||
directory is never considered published state, is never selected for Recent
|
|
||||||
Changes, and may be removed by a documented safe cleanup mechanism. Cleanup
|
|
||||||
must distinguish inactive temporary directories from concurrent active
|
|
||||||
invocations and must never recursively target the workspace root or an
|
|
||||||
unresolved configuration path.
|
|
||||||
|
|
||||||
## Inspection And Metadata Policy
|
|
||||||
|
|
||||||
Run-history discovery and inspection are not part of the desired product
|
|
||||||
contract. The historical `inspect reports`, `inspect metadata`, `inspect
|
|
||||||
modules`, `inspect data-package`, `inspect prior`, and `inspect sources`
|
|
||||||
surfaces are candidates for removal together rather than preservation through
|
|
||||||
a new storage representation.
|
|
||||||
|
|
||||||
Any manifest retained for atomic publication or Recent Changes is current
|
|
||||||
operational state, not an archival metadata record. It should contain only the
|
|
||||||
identity, valid period, safe paths, and deterministic snapshot information
|
|
||||||
needed to validate and use that current state. It does not need to preserve
|
|
||||||
prompt messages, generated prose intermediates, source provenance, provider
|
|
||||||
provenance, notification history, or a catalog of prior runs.
|
|
||||||
|
|
||||||
The application does not promise cross-version decoding of ordinary workspace
|
|
||||||
state. A new release may replace or ignore incompatible current-state files,
|
|
||||||
provided it fails safely, never mistakes stale state for a compatible Recent
|
|
||||||
Changes baseline, and documents any operator action required during upgrade.
|
|
||||||
|
|
||||||
## Prompt Execution And Debugging
|
|
||||||
|
|
||||||
Prompt inspection before weather collection and prepared execution remain
|
|
||||||
runtime safety requirements. They do not require durable preparation or
|
|
||||||
execution receipts.
|
|
||||||
|
|
||||||
The selected logical profile, effective backend and model, validation outcome,
|
|
||||||
and safe classified error remain available to the active workflow and its CLI
|
|
||||||
summary where useful. Weatherreporter does not retain them as long-term report
|
|
||||||
provenance after the invocation completes.
|
|
||||||
|
|
||||||
The existing explicit secure debug root remains outside ordinary state and may
|
|
||||||
retain rendered prompts, schemas, input bodies, generated bodies, and effective
|
|
||||||
parameters according to its documented contract. Weatherreporter does not
|
|
||||||
automatically clean that operator-selected location. Credentials must remain
|
|
||||||
excluded from debug capture.
|
|
||||||
|
|
||||||
## Compatibility And Upgrade Policy
|
|
||||||
|
|
||||||
This is an intentional breaking change to the workspace and inspection
|
|
||||||
contracts. Weatherreporter does not need to migrate historical V1 or V2
|
|
||||||
metadata, prompt receipts, intermediate generated-text artifacts, or managed
|
|
||||||
reports into the new representation.
|
|
||||||
|
|
||||||
Legacy workspace trees must not be silently interpreted as current published
|
|
||||||
state. They also must not be deleted automatically merely because a new
|
|
||||||
version starts: an operator may have placed or referenced files there despite
|
|
||||||
the absence of a continuing application compatibility promise. Release notes
|
|
||||||
and operations documentation must explain whether legacy data can be removed
|
|
||||||
manually and identify the exact safe target.
|
|
||||||
|
|
||||||
The change should land in a release whose notes clearly identify removed CLI
|
|
||||||
commands, obsolete paths and schemas, the new bounded state behavior, and any
|
|
||||||
upgrade action. Because Weatherreporter remains pre-1.0, the ordinary semantic
|
|
||||||
version policy may carry this breaking change without inventing a migration
|
|
||||||
framework.
|
|
||||||
|
|
||||||
## Required Architecture Decision Record
|
|
||||||
|
|
||||||
The implemented feature must include an Accepted ADR recording the durable
|
|
||||||
architectural decision to use ephemeral operational state. The ADR is not part
|
|
||||||
of this roadmap-writing pass and should not be created until implementation is
|
|
||||||
being prepared.
|
|
||||||
|
|
||||||
The ADR should record:
|
|
||||||
|
|
||||||
- the mismatch between run-addressed provenance storage and the ephemeral
|
|
||||||
weather-report lifecycle;
|
|
||||||
- the decision to retain bounded current report and comparison state rather
|
|
||||||
than historical runs;
|
|
||||||
- the distinction between temporary invocation state, published operational
|
|
||||||
state, explicit output copies, and secure debug capture;
|
|
||||||
- the removal of historical inspection and backward-compatibility guarantees;
|
|
||||||
- atomic publication and failed-run behavior;
|
|
||||||
- the alternatives considered, including retaining the current archive,
|
|
||||||
adding time-based retention, or keeping a bounded run history; and
|
|
||||||
- consequences for CLI compatibility, workspace layout, testing, operations,
|
|
||||||
and future schema changes.
|
|
||||||
|
|
||||||
Once accepted, the ADR owns the decision rationale. The architecture policy
|
|
||||||
owns the resulting current invariant, while focused state, CLI, operations,
|
|
||||||
and integration documents own the implemented contracts.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
The completed feature includes:
|
|
||||||
|
|
||||||
- an invocation-scoped temporary workspace for intermediate generation state;
|
|
||||||
- atomic publication of the current managed report and its minimal operational
|
|
||||||
state;
|
|
||||||
- a bounded comparison snapshot representing the last successfully published
|
|
||||||
report for each supported report key and valid period;
|
|
||||||
- safe cleanup behavior for normal completion, handled failure, and abandoned
|
|
||||||
temporary workspaces;
|
|
||||||
- removal of durable prompt preparation, prompt execution, generated-text,
|
|
||||||
render-context, data-package, notification, and run-metadata history;
|
|
||||||
- removal of run-history inspection commands and their application/state
|
|
||||||
contracts;
|
|
||||||
- removal of V1 metadata compatibility and current-version coupling for
|
|
||||||
historical prompt artifacts by removing the historical artifact contract;
|
|
||||||
- preservation of active-command partial status, safe errors, and paths to
|
|
||||||
genuinely retained published, operator-owned, or debug outputs;
|
|
||||||
- preservation of explicit output copies, Distributor upload behavior, and
|
|
||||||
opt-in secure LLM debug capture;
|
|
||||||
- risk-appropriate offline tests for atomic publication, comparison baselines,
|
|
||||||
failure isolation, cleanup safety, concurrent invocation safety, and absence
|
|
||||||
of unbounded state growth;
|
|
||||||
- an Accepted ADR documenting the architectural decision; and
|
|
||||||
- updates to canonical architecture, CLI, operations, configuration,
|
|
||||||
troubleshooting, integration, internal, testing, and release documentation
|
|
||||||
where their contracts change.
|
|
||||||
|
|
||||||
## Non-Goals
|
|
||||||
|
|
||||||
This feature does not include:
|
|
||||||
|
|
||||||
- a general-purpose cache, database, archival service, or retention engine;
|
|
||||||
- replaying or resuming interrupted generation;
|
|
||||||
- migrating legacy artifacts into the new representation;
|
|
||||||
- retaining a bounded number of historical runs for convenience;
|
|
||||||
- automatic upload or archival of state to remote storage;
|
|
||||||
- collecting additional provider telemetry or weather-source provenance;
|
|
||||||
- changing report content, prompt text, schemas, profile selection, weather
|
|
||||||
derivation, or batch membership;
|
|
||||||
- deleting operator-owned `--out`, `--out-dir`, or secure debug files;
|
|
||||||
- changing Distributor's report-content contract; or
|
|
||||||
- making live external services part of the default test suite.
|
|
||||||
|
|
||||||
## Safety And Testing Policy
|
|
||||||
|
|
||||||
The state refactoring must preserve Weatherreporter's existing path-safety and
|
|
||||||
atomicity expectations while reducing the amount of durable state. Tests
|
|
||||||
should emphasize observable lifecycle guarantees rather than private file
|
|
||||||
choreography.
|
|
||||||
|
|
||||||
Important risks requiring durable offline coverage include:
|
|
||||||
|
|
||||||
- a failed or canceled generation replacing a previously published report or
|
|
||||||
comparison baseline;
|
|
||||||
- a partially written report becoming visible as current;
|
|
||||||
- Recent Changes selecting an incompatible report, valid period, or failed
|
|
||||||
attempt;
|
|
||||||
- cleanup deleting published, operator-owned, debug, or concurrently active
|
|
||||||
files;
|
|
||||||
- batch partial success corrupting the state of another report;
|
|
||||||
- notification failure rolling back or obscuring a successfully published
|
|
||||||
report;
|
|
||||||
- stale or incompatible current state being treated as valid; and
|
|
||||||
- repeated successful and failed runs causing unbounded ordinary workspace
|
|
||||||
growth.
|
|
||||||
|
|
||||||
Tests remain deterministic, offline, credential-free, and based on real
|
|
||||||
temporary filesystems plus narrow external-boundary fakes. Race-enabled tests
|
|
||||||
are required where publication, cleanup, or concurrent invocation behavior
|
|
||||||
shares mutable filesystem state.
|
|
||||||
|
|
||||||
## Relationship To Domain-Specific Profiles
|
|
||||||
|
|
||||||
The domain-specific profile feature can be implemented before this refactor,
|
|
||||||
but it should not add new historical compatibility or durable-provenance
|
|
||||||
commitments. Profile inspection, selection, override precedence, and effective
|
|
||||||
model resolution remain active-workflow behavior and survive the state change.
|
|
||||||
|
|
||||||
The domain-profile roadmap and implementation plan should acknowledge that
|
|
||||||
prompt artifacts from version `1.0.1` need not remain readable after prompts
|
|
||||||
advance to `1.1.0`. Existing state persistence may remain temporarily while
|
|
||||||
the profile feature lands, but it should not be expanded or treated as the
|
|
||||||
target architecture.
|
|
||||||
|
|
||||||
## Completion Criteria
|
|
||||||
|
|
||||||
The roadmap's target state is achieved when:
|
|
||||||
|
|
||||||
- ordinary runs no longer create durable run-addressed artifact collections;
|
|
||||||
- a successful report atomically replaces only the corresponding current
|
|
||||||
published state;
|
|
||||||
- failed and canceled attempts leave the prior published report and Recent
|
|
||||||
Changes baseline unchanged;
|
|
||||||
- Daily, Today, and Tomorrow compare against at most one compatible snapshot
|
|
||||||
from the last successfully published report;
|
|
||||||
- expired comparison and published state can be removed safely without
|
|
||||||
touching operator-owned or active files;
|
|
||||||
- Hourly does not retain an unused comparison snapshot;
|
|
||||||
- historical inspection commands and V1/V2 archival compatibility code are
|
|
||||||
removed;
|
|
||||||
- temporary, published, explicit-output, and debug paths have distinct and
|
|
||||||
documented ownership and cleanup rules;
|
|
||||||
- Distributor and active-command summaries continue to receive the completed
|
|
||||||
report and safe status information they require;
|
|
||||||
- the default suite proves atomicity, bounded growth, cleanup safety, batch
|
|
||||||
isolation, and comparison correctness offline;
|
|
||||||
- an Accepted ADR records the architectural decision and alternatives; and
|
|
||||||
- canonical current-state documentation describes only the implemented
|
|
||||||
lifecycle.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
### Lifetime of the current managed report
|
|
||||||
|
|
||||||
Recommendation: retain one current managed report per logical report key and
|
|
||||||
valid period until it is replaced or its valid period expires. This preserves
|
|
||||||
the current default behavior for invocations without `--out` while bounding
|
|
||||||
growth.
|
|
||||||
|
|
||||||
Alternative: treat the managed report as temporary and retain output only when
|
|
||||||
the operator supplies `--out` or `--out-dir`. This minimizes state further but
|
|
||||||
makes a successful default invocation produce no durable report for the user
|
|
||||||
and complicates Distributor sequencing.
|
|
||||||
|
|
||||||
### Historical inspection replacement
|
|
||||||
|
|
||||||
Recommendation: remove the run-history inspection commands without adding a
|
|
||||||
replacement initially. Current command summaries, current managed files, and
|
|
||||||
opt-in debug capture cover the remaining supported workflows.
|
|
||||||
|
|
||||||
Alternative: add a narrow `inspect current REPORT` command backed only by the
|
|
||||||
current operational manifest. This provides discoverability without history,
|
|
||||||
but it creates a new public surface and may preserve metadata complexity that
|
|
||||||
the refactor is intended to remove.
|
|
||||||
|
|
||||||
### Abandoned temporary workspace cleanup
|
|
||||||
|
|
||||||
Recommendation: use an explicitly owned temporary subtree with per-invocation
|
|
||||||
ownership markers and a conservative age threshold. Normal cleanup removes the
|
|
||||||
current invocation synchronously; opportunistic cleanup removes only marked,
|
|
||||||
inactive directories old enough that they cannot reasonably belong to a live
|
|
||||||
invocation.
|
|
||||||
|
|
||||||
Alternative: perform only synchronous cleanup and document manual removal of
|
|
||||||
directories left by process termination. This minimizes destructive code and
|
|
||||||
concurrency risk, but crashed processes can still accumulate unbounded files.
|
|
||||||
|
|
||||||
### Legacy workspace cleanup
|
|
||||||
|
|
||||||
Recommendation: ignore legacy run-addressed trees and document a precise,
|
|
||||||
manual one-time cleanup procedure. Do not automatically delete them during
|
|
||||||
startup or upgrade.
|
|
||||||
|
|
||||||
Alternative: add an explicit cleanup command that previews and then removes
|
|
||||||
recognized legacy artifacts. This is more convenient for large installations
|
|
||||||
but introduces a destructive command and a legacy-format classifier that must
|
|
||||||
be maintained and tested.
|
|
||||||
@@ -3,6 +3,40 @@
|
|||||||
This roadmap contains future work only. Each section identifies its planning
|
This roadmap contains future work only. Each section identifies its planning
|
||||||
status; current behavior is documented outside `docs/roadmap/`.
|
status; current behavior is documented outside `docs/roadmap/`.
|
||||||
|
|
||||||
|
## Upstream Forecast Change Product
|
||||||
|
|
||||||
|
Status: Proposed upstream feature request; unimplemented.
|
||||||
|
|
||||||
|
Weatherreporter's local Recent Changes feature was removed by the accepted
|
||||||
|
[stateless execution decision](../adr/0001-stateless-execution.md). Forecast
|
||||||
|
version history and comparison are better owned by the Weather API, where the
|
||||||
|
underlying forecast issuances can be retained and compared consistently for
|
||||||
|
all consumers.
|
||||||
|
|
||||||
|
A future Weather API feature should expose a structured change product with:
|
||||||
|
|
||||||
|
- explicit current and baseline forecast issuance timestamps or identifiers;
|
||||||
|
- documented baseline selection, such as a requested comparison timestamp,
|
||||||
|
preceding issuance, or fixed rolling period;
|
||||||
|
- location, timezone, and half-open valid-period identity;
|
||||||
|
- typed changed values with previous and current values and units;
|
||||||
|
- stable change categories for temperature, precipitation probability and
|
||||||
|
timing, wind gusts, alerts, and aggregate hazards;
|
||||||
|
- an API-owned significance classification or enough structured information
|
||||||
|
for a stateless consumer to apply a documented presentation threshold; and
|
||||||
|
- deterministic ordering, missing-baseline behavior, and source metadata.
|
||||||
|
|
||||||
|
The API should compare forecast versions, not track a Weatherreporter client's
|
||||||
|
"previous run." It should not require consumer identity, mutable cursors, or
|
||||||
|
Weatherreporter-managed history. A missing baseline should be a normal empty
|
||||||
|
result rather than an error.
|
||||||
|
|
||||||
|
Once a stable upstream contract exists, a separate Weatherreporter roadmap may
|
||||||
|
reintroduce change commentary by collecting that product and mapping it into a
|
||||||
|
curated prompt-facing module. There must be no local snapshot fallback. The
|
||||||
|
ordinary Weatherreporter process must remain stateless, and the upstream
|
||||||
|
feature should have deterministic fixtures before adoption.
|
||||||
|
|
||||||
## Automatic Storm Monitoring
|
## Automatic Storm Monitoring
|
||||||
|
|
||||||
Status: Proposed and unimplemented.
|
Status: Proposed and unimplemented.
|
||||||
@@ -13,8 +47,10 @@ Possible direction:
|
|||||||
|
|
||||||
1. Detect candidate storm events from alerts, forecast discussion, weather
|
1. Detect candidate storm events from alerts, forecast discussion, weather
|
||||||
story context, hourly thresholds, and material forecast changes.
|
story context, hourly thresholds, and material forecast changes.
|
||||||
2. Evaluate candidates through Scriptorium or another narrow evaluator adapter.
|
2. Evaluate candidates through Promptkit or another narrow evaluator adapter.
|
||||||
3. Persist storm lifecycle state.
|
3. Keep any required storm lifecycle state in the upstream service or another
|
||||||
|
explicitly designed external owner rather than silently reintroducing a
|
||||||
|
Weatherreporter workspace.
|
||||||
4. Generate or update a storm report only when a meaningful event is present.
|
4. Generate or update a storm report only when a meaningful event is present.
|
||||||
5. Suppress ordinary low-impact thunder or rain chances.
|
5. Suppress ordinary low-impact thunder or rain chances.
|
||||||
|
|
||||||
@@ -54,8 +90,7 @@ Status: Proposed and unimplemented.
|
|||||||
Possible future modules:
|
Possible future modules:
|
||||||
|
|
||||||
- `hourly_table` for compact valid-period hourly facts
|
- `hourly_table` for compact valid-period hourly facts
|
||||||
- `forecast_delta` if a separate stanza is useful beyond current Recent
|
- `forecast_delta` after an upstream forecast-change product exists
|
||||||
Changes
|
|
||||||
- `weekend_planning` if weekend-specific planning guidance needs a dedicated
|
- `weekend_planning` if weekend-specific planning guidance needs a dedicated
|
||||||
deterministic stanza
|
deterministic stanza
|
||||||
- `storm_window_summary` if manual or automatic storm reports need a dedicated
|
- `storm_window_summary` if manual or automatic storm reports need a dedicated
|
||||||
@@ -79,7 +114,7 @@ contracts](../internal/facts.md), [module internals](../internal/module.md), and
|
|||||||
- keep broad reusable calculations in `DerivedFacts`
|
- keep broad reusable calculations in `DerivedFacts`
|
||||||
- keep prompt-facing field shape inside module builders
|
- keep prompt-facing field shape inside module builders
|
||||||
- use typed options for configurable module behavior
|
- use typed options for configurable module behavior
|
||||||
- keep module snapshots structured and deterministic for Recent Changes
|
- keep module output structured and deterministic
|
||||||
|
|
||||||
## Distributor Notification Enhancements
|
## Distributor Notification Enhancements
|
||||||
|
|
||||||
@@ -92,10 +127,8 @@ behavior is documented in the [Distributor adapter guide](../internal/distributo
|
|||||||
unimplemented:
|
unimplemented:
|
||||||
|
|
||||||
- `failure_policy: warn`
|
- `failure_policy: warn`
|
||||||
- uploading metadata, module snapshots, data packages, or preflight artifacts
|
|
||||||
- durable upload retry queues
|
- durable upload retry queues
|
||||||
- distributor-specific CLI flags
|
- distributor-specific CLI flags
|
||||||
- distributor workspace scanning
|
|
||||||
- destination routing, Markdown-to-HTML transformation, public URLs, or nginx
|
- destination routing, Markdown-to-HTML transformation, public URLs, or nginx
|
||||||
layout inside weatherreporter
|
layout inside weatherreporter
|
||||||
|
|
||||||
@@ -138,6 +171,7 @@ maintenance costs make the added abstraction worthwhile:
|
|||||||
- global test helper package
|
- global test helper package
|
||||||
- logging subsystem
|
- logging subsystem
|
||||||
|
|
||||||
Any future implementation should preserve the existing public CLI, artifact
|
Any future implementation should preserve the public CLI, report-output
|
||||||
paths, report identities, module boundaries, and adapter boundaries unless a
|
contract, report identities, module boundaries, and adapter boundaries in
|
||||||
separate roadmap explicitly changes them.
|
effect when that work begins unless a separate roadmap explicitly changes
|
||||||
|
them.
|
||||||
|
|||||||
@@ -1,599 +0,0 @@
|
|||||||
# Domain-Specific Prompt Profiles Implementation Plan
|
|
||||||
|
|
||||||
Status: Stages 1–7 completed; remediation Stage 8 ready.
|
|
||||||
|
|
||||||
## Purpose And Authority
|
|
||||||
|
|
||||||
This document records the implementation and post-implementation remediation
|
|
||||||
of the
|
|
||||||
[domain-specific prompt profiles roadmap](domain-profiles.md). The roadmap is
|
|
||||||
authoritative for scope, user intent, policy choices, and the intended end
|
|
||||||
state. This plan records implementation sequence, verification, audit findings,
|
|
||||||
and exit gates.
|
|
||||||
|
|
||||||
This plan follows the repository's
|
|
||||||
[architecture](../policy/architecture.md),
|
|
||||||
[documentation](../policy/documentation.md), and
|
|
||||||
[testing](../policy/testing.md) policies.
|
|
||||||
|
|
||||||
## Completed Prerequisite
|
|
||||||
|
|
||||||
Weatherreporter is already pinned to Promptkit v0.5.0. That release provides
|
|
||||||
the public `WithFallbackProfileFS` option and the required precedence across
|
|
||||||
inspection, preparation, and execution. The dependency upgrade passed
|
|
||||||
Weatherreporter's full offline test suite, race-enabled suite, CLI help check,
|
|
||||||
and an operator `generate hourly` smoke test. Do not repeat or replace the
|
|
||||||
dependency upgrade as part of these stages.
|
|
||||||
|
|
||||||
## Locked Product Decisions
|
|
||||||
|
|
||||||
Implement these exact Weatherreporter-owned profiles:
|
|
||||||
|
|
||||||
| Profile ID | Backend | Model | Reasoning effort | Timeout | Service tier |
|
|
||||||
| --- | --- | --- | --- | --- | --- |
|
|
||||||
| `weather-light` | `openrouter` | `deepseek/deepseek-v4-flash` | Omitted | 180 seconds | `flex` |
|
|
||||||
| `weather-balanced` | `openrouter` | `~google/gemini-flash-latest` | `high` | 240 seconds | `flex` |
|
|
||||||
| `weather-deep` | `openrouter` | `~anthropic/claude-sonnet-latest` | `high` | 240 seconds | `flex` |
|
|
||||||
|
|
||||||
Assign Hourly to `weather-light`; assign Daily, Today, and Tomorrow to
|
|
||||||
`weather-balanced`; assign no report to `weather-deep` initially. Advance all
|
|
||||||
four prompt definitions and matching report-registry entries from `1.0.1` to
|
|
||||||
`1.1.0` when their defaults change.
|
|
||||||
|
|
||||||
The leading `~` in the Gemini and Claude model IDs is required and denotes an
|
|
||||||
OpenRouter rolling alias. Do not substitute the unavailable non-tilde IDs or a
|
|
||||||
dated model version. Do not add temperature, `top_p`, maximum-token, endpoint,
|
|
||||||
or credential fields to the embedded definitions.
|
|
||||||
|
|
||||||
Prompt preparation and execution artifacts written at `1.0.1` are not required
|
|
||||||
to remain readable after the transition to `1.1.0`. Do not add a migration,
|
|
||||||
compatibility shim, or weaker historical-artifact validation for this feature.
|
|
||||||
|
|
||||||
Definition lookup must remain:
|
|
||||||
|
|
||||||
1. explicit Promptkit in-memory profiles used by tests or an embedding
|
|
||||||
consumer;
|
|
||||||
2. Weatherreporter's configured `profile_file` or `profile_dir` source;
|
|
||||||
3. Weatherreporter's embedded fallback profiles; and
|
|
||||||
4. Promptkit's built-in catalog.
|
|
||||||
|
|
||||||
Selection remains a separate concern: a nonblank global `promptkit.profile`
|
|
||||||
selects the profile for every report in the invocation; otherwise the exact
|
|
||||||
prompt definition's `default_profile` selects it. A malformed matching
|
|
||||||
higher-precedence profile is an error and never falls through.
|
|
||||||
|
|
||||||
## Continuing Invariants
|
|
||||||
|
|
||||||
- Keep all Promptkit types and mechanics inside
|
|
||||||
`internal/adapters/promptkit`, its focused tests, and asset contract tests.
|
|
||||||
- Keep prompt inspection before weather collection and provider work.
|
|
||||||
- Keep one Promptkit engine per command action and one shared engine across a
|
|
||||||
sequential batch.
|
|
||||||
- Preserve logical profile ID and effective backend/model information through
|
|
||||||
active inspection and execution where the project-owned contract already
|
|
||||||
exposes it. Do not add new durable-provenance fields or compatibility
|
|
||||||
guarantees.
|
|
||||||
- Leave existing workspace persistence behavior otherwise unchanged. The
|
|
||||||
accepted [ephemeral-state roadmap](ephemeral-state.md) owns its future
|
|
||||||
removal and must not be partially implemented here.
|
|
||||||
- Do not expose endpoints, credentials, rendered messages, schemas, request
|
|
||||||
bodies, response bodies, or complete parameter maps through ordinary errors,
|
|
||||||
logs, summaries, or state.
|
|
||||||
- Keep the default suite deterministic, offline, and credential-free.
|
|
||||||
- Do not add endpoint discovery, health probing, provider failover, retries at
|
|
||||||
a more expensive tier, profile merging, per-report configuration fields, or
|
|
||||||
severity-driven model selection.
|
|
||||||
- Update canonical current-state documentation only in the stage where the
|
|
||||||
corresponding behavior becomes implemented.
|
|
||||||
- Run `git diff --check` before completing every stage.
|
|
||||||
|
|
||||||
## Stage 1: Add The Embedded Weather Profile Catalog
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Create one repository-owned, embedded profile source containing exactly the
|
|
||||||
three locked logical profiles.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
1. Add strict YAML profile assets beneath `internal/promptassets` using the
|
|
||||||
exact IDs and definitions in this plan.
|
|
||||||
2. Extend `internal/promptassets` with a narrowly named accessor that returns
|
|
||||||
the embedded profile `fs.FS`. Follow the existing prompt and schema asset
|
|
||||||
pattern without exposing Promptkit types from the package.
|
|
||||||
3. Keep profile filenames and embed layout simple and deterministic. Do not
|
|
||||||
duplicate Promptkit's built-in directory taxonomy unless the application
|
|
||||||
assets require it.
|
|
||||||
4. Validate the assets through Promptkit's public engine/profile inspection
|
|
||||||
surface rather than adding a second YAML parser or a Weatherreporter-owned
|
|
||||||
profile representation.
|
|
||||||
|
|
||||||
### Tests
|
|
||||||
|
|
||||||
- Extend the asset contract tests to assert exactly the three logical IDs,
|
|
||||||
their exact effective model IDs, and the intentional parameters.
|
|
||||||
- Prove all three profiles inspect successfully offline when supplied as a
|
|
||||||
fallback source and no operator source is present.
|
|
||||||
- Assert that the catalog contains no endpoints, credentials, temperature,
|
|
||||||
`top_p`, or maximum-token settings.
|
|
||||||
- Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/promptassets
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
### Exit Gate
|
|
||||||
|
|
||||||
The embedded catalog is complete, strictly valid, safe, and independently
|
|
||||||
inspectable through Promptkit v0.5.0's public API.
|
|
||||||
|
|
||||||
## Stage 2: Wire Fallback Resolution And Protect Precedence
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Supply the embedded catalog through Promptkit's application fallback layer
|
|
||||||
without changing existing operator configuration or application boundaries.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
1. Add `promptkit.WithFallbackProfileFS(promptassets.ProfileFS(), ".")` to
|
|
||||||
normal adapter engine construction.
|
|
||||||
2. Preserve existing `profile_file`, `profile_dir`, configured local backend,
|
|
||||||
timeout, prompt filesystem, schema filesystem, and test-option behavior.
|
|
||||||
3. Ensure ordinary production construction and the adapter's test
|
|
||||||
construction path exercise the same fallback wiring. Test-only explicit
|
|
||||||
profiles may retain Promptkit's documented highest precedence.
|
|
||||||
4. Keep all fallback resolution in Promptkit. Do not add filesystem overlays,
|
|
||||||
existence checks, YAML parsing, or merge behavior to Weatherreporter.
|
|
||||||
|
|
||||||
### Tests
|
|
||||||
|
|
||||||
- At the adapter boundary, prove fallback-only inspection of all three
|
|
||||||
Weatherreporter profiles.
|
|
||||||
- Prove same-ID overrides through both configured `profile_file` and
|
|
||||||
`profile_dir`, including resolution of the override's effective backend and
|
|
||||||
model.
|
|
||||||
- Prove an absent operator match falls through, while a malformed matching
|
|
||||||
operator definition fails without using the embedded profile.
|
|
||||||
- Prove a selected Promptkit built-in that is absent from both higher layers
|
|
||||||
still resolves.
|
|
||||||
- Prove an explicit in-memory test profile retains highest precedence.
|
|
||||||
- Cover both local override forms required by the roadmap: an endpoint-only
|
|
||||||
OpenAI-compatible `weather-light` profile and a `backend: local` profile
|
|
||||||
using the configured local endpoint. No test may contact either endpoint.
|
|
||||||
- Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/adapters/promptkit
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
### Exit Gate
|
|
||||||
|
|
||||||
Inspection and prepared execution use Promptkit's exact four-layer precedence,
|
|
||||||
operator errors remain visible, and local overrides require no prompt or code
|
|
||||||
changes.
|
|
||||||
|
|
||||||
## Stage 3: Adopt Logical Defaults And Prompt Version 1.1.0
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Move operational prompts from provider-oriented defaults to the three-tier
|
|
||||||
Weatherreporter policy with an exact, synchronized version transition.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
1. Change Hourly's `default_profile` to `weather-light`.
|
|
||||||
2. Change Daily, Today, and Tomorrow to `weather-balanced`.
|
|
||||||
3. Advance the exact version in all four prompt YAML assets from `1.0.1` to
|
|
||||||
`1.1.0` without changing prompt text or generated-text schemas solely for
|
|
||||||
this feature.
|
|
||||||
4. Advance the four matching report-registry prompt versions to `1.1.0` in the
|
|
||||||
same change. Keep prompt IDs, report IDs, modules, periods, templates, and
|
|
||||||
output contracts unchanged.
|
|
||||||
5. Update fixtures and expectations that intentionally assert the current
|
|
||||||
prompt contract. Do not rewrite historical fixture versions or weaken tests
|
|
||||||
that protect actual compatibility.
|
|
||||||
|
|
||||||
### Tests
|
|
||||||
|
|
||||||
- Update asset and report-registry contract tests to require exact version
|
|
||||||
`1.1.0` and the report-to-profile assignments locked in this plan.
|
|
||||||
- Inspect every exact prompt version through the real embedded prompt, schema,
|
|
||||||
and fallback-profile filesystems.
|
|
||||||
- Prove Hourly resolves DeepSeek V4 Flash, the three day-scale reports resolve
|
|
||||||
Gemini Flash Latest, and `weather-deep` remains inspectable but unassigned.
|
|
||||||
- Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/promptassets ./internal/report ./internal/adapters/promptkit
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
### Exit Gate
|
|
||||||
|
|
||||||
Every operational prompt and registry definition agrees on exact version
|
|
||||||
`1.1.0`, selects its intended logical tier, and resolves its expected effective
|
|
||||||
model offline.
|
|
||||||
|
|
||||||
## Stage 4: Verify Application Selection And Batch Reuse
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Protect the assembled application behavior created by the new defaults and
|
|
||||||
confirm that logical identity is not lost during active effective-model
|
|
||||||
resolution.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
1. Preserve the current pre-collection inspection order and fail-fast behavior
|
|
||||||
for missing credentials, unknown profiles, malformed profiles, and unusable
|
|
||||||
backends.
|
|
||||||
2. Preserve the global `promptkit.profile` all-report override. Do not add a
|
|
||||||
second override mechanism or report-specific configuration fields.
|
|
||||||
3. Preserve batch preflight deduplication by selected effective profile ID:
|
|
||||||
Today and Tomorrow in the same batch should inspect their shared
|
|
||||||
`weather-balanced` selection once.
|
|
||||||
4. Preserve the selected logical profile ID and resolved backend/model through
|
|
||||||
active inspection, preparation, and execution using the existing
|
|
||||||
project-owned contract. Do not add state fields, expand persisted parameter
|
|
||||||
detail, or create a new historical compatibility guarantee.
|
|
||||||
|
|
||||||
### Tests
|
|
||||||
|
|
||||||
- Add or update representative app tests for default Hourly and day-scale
|
|
||||||
selection, a global-profile override, and a morning/evening batch sharing
|
|
||||||
`weather-balanced`.
|
|
||||||
- Assert inspection completes before weather collection and provider
|
|
||||||
generation, including malformed same-ID operator overrides.
|
|
||||||
- Assert active inspection and execution expose the logical profile ID and
|
|
||||||
effective model for both embedded and overridden profiles.
|
|
||||||
- Assert endpoints and credentials remain absent from errors, summaries,
|
|
||||||
normal logs, and ordinary state.
|
|
||||||
- Use project-owned executor fakes or Promptkit provider fakes; do not make live
|
|
||||||
provider calls.
|
|
||||||
- Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./internal/app ./internal/cli
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
### Exit Gate
|
|
||||||
|
|
||||||
Single-report and batch workflows select the intended tier, retain existing
|
|
||||||
override and preflight behavior, deduplicate shared batch inspection, and
|
|
||||||
preserve safe logical and effective model information during active execution
|
|
||||||
without adding a durable-provenance contract.
|
|
||||||
|
|
||||||
## Stage 5: Publish Canonical Operator And Maintainer Documentation
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Document the implemented feature once in each canonical owner and provide one
|
|
||||||
maintained, copyable local override example.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
1. Update `docs/config.md` to explain global profile selection versus
|
|
||||||
`profile_file`/`profile_dir` definition lookup and link to the maintained
|
|
||||||
example. Keep the field reference in this canonical document.
|
|
||||||
2. Update the Promptkit integration document with the logical profile catalog,
|
|
||||||
source precedence, exact prompt-version relationship, and safe active
|
|
||||||
inspection and execution contract. Avoid restating complete configuration
|
|
||||||
syntax or presenting transitional persistence as the target architecture.
|
|
||||||
3. Update the report-registry, Promptkit adapter, app-orchestration, and state
|
|
||||||
internal documents only where their implemented contracts changed.
|
|
||||||
4. Update `docs/operations.md` with the normal local-override workflow and
|
|
||||||
`docs/troubleshooting.md` with malformed override, unavailable local
|
|
||||||
endpoint, missing credential, and unexpected effective-model diagnostics.
|
|
||||||
5. Add or update one secret-free file under `examples/` showing a
|
|
||||||
`weather-light` override for a local OpenAI-compatible endpoint. Choose one
|
|
||||||
supported form as the complete example and mention the other form only in
|
|
||||||
its canonical reference.
|
|
||||||
6. Update the architecture policy only if implementation changed a normative
|
|
||||||
boundary or invariant. Do not add future behavior to current-state docs.
|
|
||||||
7. Keep the feature roadmap and this plan in their pre-implementation statuses
|
|
||||||
until the final repository gate passes. Do not create release notes before
|
|
||||||
a release version is chosen.
|
|
||||||
|
|
||||||
### Tests
|
|
||||||
|
|
||||||
- Verify every changed repository-relative link and every profile/model ID.
|
|
||||||
- Validate maintained YAML examples through the same strict configuration or
|
|
||||||
Promptkit profile path used by production where practical.
|
|
||||||
- Run the focused tests that own any executable examples, followed by:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
### Exit Gate
|
|
||||||
|
|
||||||
Users, operators, and maintainers can discover the tier defaults, precedence,
|
|
||||||
global override, local override, and failure behavior without duplicated or
|
|
||||||
future-state documentation.
|
|
||||||
|
|
||||||
## Stage 6: Complete Repository Verification And Roadmap Handoff
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Demonstrate that the complete feature is coherent, offline-testable, and ready
|
|
||||||
for review and a later release decision.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
1. Review the complete diff against the roadmap, this plan, and all three
|
|
||||||
policy documents. Remove stale identifiers, temporary helpers, redundant
|
|
||||||
tests, and documentation duplication.
|
|
||||||
2. Confirm `go.mod` and `go.sum` retain tagged Promptkit v0.5.0 without a local
|
|
||||||
replacement or dependency drift.
|
|
||||||
3. Confirm only the four supported report products exist and no retired report
|
|
||||||
surfaces were reintroduced.
|
|
||||||
4. Confirm the roadmap's completion criteria one by one. Change its status to
|
|
||||||
implemented and this plan's status to completed only after every criterion
|
|
||||||
and command below passes.
|
|
||||||
5. Do not require a live provider for completion. If credentials and network
|
|
||||||
access are deliberately supplied by an operator, record live smoke results
|
|
||||||
separately as release-candidate evidence rather than adding them to the
|
|
||||||
default suite.
|
|
||||||
|
|
||||||
### Verification
|
|
||||||
|
|
||||||
Run `gofmt -w` on every changed Go file, then run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./...
|
|
||||||
go test -race ./...
|
|
||||||
go run ./cmd/weatherreporter --help
|
|
||||||
git diff --check
|
|
||||||
git status --short
|
|
||||||
```
|
|
||||||
|
|
||||||
Also inspect all three logical profiles through the application's normal
|
|
||||||
preflight path using offline provider doubles, including one same-ID local
|
|
||||||
override and one explicit global override.
|
|
||||||
|
|
||||||
### Exit Gate
|
|
||||||
|
|
||||||
All roadmap completion criteria are satisfied, all verification commands pass,
|
|
||||||
the working tree contains only intentional changes, and the canonical
|
|
||||||
documentation describes the implemented state. The feature is ready for code
|
|
||||||
review and release preparation.
|
|
||||||
|
|
||||||
## Post-Implementation Review
|
|
||||||
|
|
||||||
Stages 1–6 implemented the intended production behavior and passed their
|
|
||||||
offline verification gates. A subsequent review found no high-severity runtime
|
|
||||||
defect, but identified three test-quality issues and one remaining validation
|
|
||||||
obligation:
|
|
||||||
|
|
||||||
- one app test asserted durable preparation and execution artifact provenance,
|
|
||||||
contrary to the active-execution boundary and accepted ephemeral-state
|
|
||||||
direction;
|
|
||||||
- an adapter-package test depended upward on app orchestration and duplicated
|
|
||||||
test ownership;
|
|
||||||
- embedded fallback profiles were inspected but not exercised through one
|
|
||||||
prepared execution with a provider fake; and
|
|
||||||
- the roadmap's representative model-evaluation policy had no recorded
|
|
||||||
evidence.
|
|
||||||
|
|
||||||
Stages 7 and 8 address those findings without changing the profile catalog,
|
|
||||||
selection precedence, report assignments, prompt content, generated-text
|
|
||||||
schemas, or default offline test contract.
|
|
||||||
|
|
||||||
## Stage 7: Correct Test Ownership And Fallback Execution Coverage
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Remove accidental durable-state and cross-layer test commitments while adding
|
|
||||||
one focused offline execution test for the embedded fallback path.
|
|
||||||
|
|
||||||
### Work
|
|
||||||
|
|
||||||
1. Rewrite `TestGenerateDetailedPreservesSelectedProfileThroughExecution` so
|
|
||||||
it protects active workflow behavior only:
|
|
||||||
|
|
||||||
- retain the Hourly default, day-scale default, and global-override cases;
|
|
||||||
- assert the profile ID sent in `promptexec.ExecuteRequest`;
|
|
||||||
- have the executor fake record the preparation and execution values it
|
|
||||||
emits, then assert their logical profile ID and effective backend/model;
|
|
||||||
- do not load preparation, execution, or metadata files to establish a
|
|
||||||
durable profile-provenance contract; and
|
|
||||||
- remove artifact-content scans whose fake inputs cannot contain an endpoint
|
|
||||||
or credential.
|
|
||||||
|
|
||||||
2. Preserve meaningful safety coverage at the boundary that can expose the
|
|
||||||
sensitive value:
|
|
||||||
|
|
||||||
- retain adapter mapping coverage proving an endpoint from a real Promptkit
|
|
||||||
profile does not enter `promptexec.ProfileInspection`;
|
|
||||||
- retain app error coverage proving dependency errors containing an endpoint
|
|
||||||
or credential are replaced by a bounded classified error; and
|
|
||||||
- do not add profile endpoints or credentials to project-owned execution
|
|
||||||
types merely to make a leakage test possible.
|
|
||||||
|
|
||||||
3. Remove `internal/app`, app configuration, and report-registry dependencies
|
|
||||||
from `internal/adapters/promptkit/adapter_test.go`. Move the assembled
|
|
||||||
application-preflight test to a new app-owned external integration test,
|
|
||||||
such as `internal/app/prompt_profile_integration_test.go` with package
|
|
||||||
`app_test`:
|
|
||||||
|
|
||||||
- construct the real Promptkit adapter through its public `New` function;
|
|
||||||
- call the public app prompt-inspection operation;
|
|
||||||
- supply a deterministic credential lookup rather than reading the process
|
|
||||||
environment; and
|
|
||||||
- cover Hourly, one representative day-scale default, the explicit
|
|
||||||
`weather-deep` global override, and a same-ID endpoint-only
|
|
||||||
`weather-light` override. The asset contract tests already own the exact
|
|
||||||
mapping for all four prompts, so the integration test need not repeat all
|
|
||||||
four.
|
|
||||||
|
|
||||||
4. Add one adapter-owned, offline fake-client execution test using the real
|
|
||||||
embedded Hourly prompt at `1.1.0` and selected profile `weather-light`.
|
|
||||||
Execute through the normal prepared adapter path and assert:
|
|
||||||
|
|
||||||
- the preparation callback runs before the fake provider;
|
|
||||||
- preparation and execution report logical profile `weather-light`, backend
|
|
||||||
`openrouter`, and model `deepseek/deepseek-v4-flash`;
|
|
||||||
- the fake provider request targets `deepseek/deepseek-v4-flash`; and
|
|
||||||
- schema validation completes without contacting a live service.
|
|
||||||
|
|
||||||
One execution case is sufficient because Promptkit owns uniform source
|
|
||||||
precedence and the adapter's inspection tests already cover fallback,
|
|
||||||
operator file, operator directory, built-in, and explicit in-memory layers.
|
|
||||||
|
|
||||||
5. Reconcile the profile-related current-state documentation:
|
|
||||||
|
|
||||||
- it may accurately describe fields present in current preparation and
|
|
||||||
execution receipts;
|
|
||||||
- it must not promise cross-version readability or characterize those
|
|
||||||
receipts as the profile feature's durable target architecture; and
|
|
||||||
- troubleshooting should prefer active command errors and explicit secure
|
|
||||||
debug capture, mentioning current-version receipts only as transitional
|
|
||||||
state if they remain useful before the ephemeral-state refactor.
|
|
||||||
|
|
||||||
6. Do not change production profile resolution, prompt definitions, state
|
|
||||||
schemas, artifact validators, or the ephemeral-state roadmap in this stage.
|
|
||||||
|
|
||||||
### Tests
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test -count=1 ./internal/promptassets ./internal/adapters/promptkit ./internal/app ./internal/cli
|
|
||||||
go test -count=1 -race ./internal/promptassets ./internal/adapters/promptkit ./internal/app ./internal/cli
|
|
||||||
go test -count=1 ./...
|
|
||||||
go vet ./...
|
|
||||||
go run ./cmd/weatherreporter --help
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
Review the changed tests against the testing policy and confirm that adapter
|
|
||||||
tests own adapter behavior, app tests own orchestration, and state tests remain
|
|
||||||
the sole owner of durable artifact format and validation details.
|
|
||||||
|
|
||||||
### Exit Gate
|
|
||||||
|
|
||||||
Active profile selection and effective-model propagation remain protected
|
|
||||||
without adding a durable-provenance commitment; the adapter test package no
|
|
||||||
longer imports the app layer; one embedded fallback profile completes prepared
|
|
||||||
execution through a provider fake; and every required check passes offline.
|
|
||||||
|
|
||||||
## Stage 8: Evaluate The Initial Model Ladder
|
|
||||||
|
|
||||||
### Goal
|
|
||||||
|
|
||||||
Produce explicit release-candidate evidence that the selected models are
|
|
||||||
acceptable for their intended report tiers and that a representative local
|
|
||||||
override provides the promised operator experience.
|
|
||||||
|
|
||||||
This is an opt-in evaluation stage, not an ordinary automated-test stage. It
|
|
||||||
requires operator-approved provider credentials, network access, and a local
|
|
||||||
OpenAI-compatible endpoint. Do not mark it complete when those prerequisites
|
|
||||||
are unavailable; report the missing prerequisite instead.
|
|
||||||
|
|
||||||
### Corpus
|
|
||||||
|
|
||||||
Use four representative, secret-free YAML data packages: one each for Daily,
|
|
||||||
Today, Tomorrow, and Hourly. The set must include at least one package with
|
|
||||||
precipitation windows and at least one with none. Remove precise private
|
|
||||||
location identifiers or other operationally sensitive values without changing
|
|
||||||
the meteorological relationships being evaluated.
|
|
||||||
|
|
||||||
Record a SHA-256 hash and a short, non-sensitive description for each package.
|
|
||||||
Do not commit full packages or generated prose unless the user separately
|
|
||||||
approves them as repository fixtures.
|
|
||||||
|
|
||||||
### Execution Matrix
|
|
||||||
|
|
||||||
Run these six evaluations from the exact package bytes:
|
|
||||||
|
|
||||||
| Case | Package | Profile |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| Hourly default | Hourly | `weather-light` |
|
|
||||||
| Daily default | Daily | `weather-balanced` |
|
|
||||||
| Today default | Today | `weather-balanced` |
|
|
||||||
| Tomorrow default | Tomorrow | `weather-balanced` |
|
|
||||||
| Deep comparison | The same Daily package used above | `weather-deep` |
|
|
||||||
| Local override | The same Hourly package used above | operator-defined `weather-light` endpoint profile |
|
|
||||||
|
|
||||||
After the successful local-override case, stop or deliberately address an
|
|
||||||
unavailable test endpoint and repeat it as a negative control. Confirm that the
|
|
||||||
request fails visibly and does not call or select an embedded remote profile.
|
|
||||||
This negative control is not an additional quality-evaluation case.
|
|
||||||
|
|
||||||
Use a temporary, untracked evaluation harness beneath the module when exact
|
|
||||||
package replay is needed. It should call the existing Promptkit adapter and
|
|
||||||
project-owned execution contract rather than duplicate prompt loading,
|
|
||||||
rendering, or schema validation. Remove the harness and all unapproved raw
|
|
||||||
outputs before completing the stage. Never print or record credentials.
|
|
||||||
|
|
||||||
### Evaluation Record
|
|
||||||
|
|
||||||
Add a concise `## Evaluation Record` section to
|
|
||||||
`docs/roadmap/domain-profiles.md`. For every case, record:
|
|
||||||
|
|
||||||
- evaluation date, logical profile, effective backend, and exact model
|
|
||||||
reported by execution;
|
|
||||||
- corpus hash, validation outcome, latency, prompt/completion/total token use,
|
|
||||||
and provider-reported or contemporaneously calculated cost;
|
|
||||||
- whether every generated claim is supported by the deterministic package;
|
|
||||||
- whether hazards, periods, uncertainty, and precipitation timing are used
|
|
||||||
correctly;
|
|
||||||
- whether `precipitation_timing` is exactly an empty string for the no-window
|
|
||||||
case;
|
|
||||||
- a short usefulness assessment for summary and forecast discussion; and
|
|
||||||
- any provider, alias, or local-endpoint caveat observed.
|
|
||||||
|
|
||||||
Do not include credentials, endpoints, complete effective parameter maps,
|
|
||||||
full data packages, rendered prompts, or full generated responses in the
|
|
||||||
record. The secure debug directory may be used temporarily for operator review
|
|
||||||
and remains operator-managed.
|
|
||||||
|
|
||||||
### Acceptance Rules
|
|
||||||
|
|
||||||
- Every case must complete strict JSON Schema validation without repair.
|
|
||||||
- Generated prose must contain no material unsupported weather claim or
|
|
||||||
contradiction of deterministic hazards, periods, or uncertainty.
|
|
||||||
- Precipitation timing must agree with the deterministic windows and use the
|
|
||||||
required empty-string representation when no window exists.
|
|
||||||
- The local override must select the operator model without modifying a prompt
|
|
||||||
or application code and must not fall back to a remote profile when the local
|
|
||||||
endpoint is unavailable.
|
|
||||||
- Latency, tokens, and cost must be recorded, but this initial evaluation does
|
|
||||||
not impose an invented numeric threshold. The operator decides whether the
|
|
||||||
observed tradeoff remains acceptable for the named tier.
|
|
||||||
- If a default case fails schema or factual acceptance, do not weaken the
|
|
||||||
schema or prompt to accommodate the model. Reopen the concrete model or
|
|
||||||
profile-setting decision in the feature roadmap and leave this stage
|
|
||||||
incomplete.
|
|
||||||
|
|
||||||
### Verification
|
|
||||||
|
|
||||||
After removing temporary evaluation material, run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test -count=1 ./...
|
|
||||||
git diff --check
|
|
||||||
git status --short
|
|
||||||
```
|
|
||||||
|
|
||||||
Confirm that the only intended repository change from this stage is the
|
|
||||||
concise evaluation record and any roadmap status correction required by its
|
|
||||||
result. Do not add live credentials, provider-dependent tests, a permanent
|
|
||||||
benchmark framework, or release notes before a release version is selected.
|
|
||||||
|
|
||||||
### Exit Gate
|
|
||||||
|
|
||||||
All six cases satisfy the acceptance rules, the roadmap contains concise and
|
|
||||||
safe evaluation evidence, no temporary corpus or response material remains in
|
|
||||||
the repository, and the default suite remains offline. Set this plan back to
|
|
||||||
`Status: Completed` only after both Stages 7 and 8 have passed.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None. The model identifiers, profile settings, report assignments, version
|
|
||||||
transition, precedence, compatibility behavior, test boundaries, and
|
|
||||||
documentation ownership are decision-complete.
|
|
||||||
@@ -1,169 +0,0 @@
|
|||||||
# Promptkit Feature Request: Application Fallback Profiles
|
|
||||||
|
|
||||||
Status: Implemented upstream in Promptkit v0.5.0.
|
|
||||||
|
|
||||||
Promptkit v0.5.0 resolved this request with the public
|
|
||||||
`WithFallbackProfileFS` engine option and the precedence and error semantics
|
|
||||||
specified below. This document is retained as the downstream rationale for
|
|
||||||
the capability.
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Promptkit should allow a consuming application to supply an embedded fallback
|
|
||||||
profile source that sits below operator-configured profiles and above
|
|
||||||
Promptkit's own built-in profile catalog.
|
|
||||||
|
|
||||||
This capability would let an application publish stable, domain-specific
|
|
||||||
profile IDs with useful defaults while preserving Promptkit's existing
|
|
||||||
operator-override behavior. The capability must remain application-neutral;
|
|
||||||
Promptkit should provide the source layer but should not own downstream profile
|
|
||||||
names, model assignments, or configuration policy.
|
|
||||||
|
|
||||||
## Downstream Use Case
|
|
||||||
|
|
||||||
Weatherreporter wants to embed profiles such as `weather-light`,
|
|
||||||
`weather-balanced`, and `weather-deep`. Report prompts would select those
|
|
||||||
logical profiles instead of naming provider- or model-specific Promptkit
|
|
||||||
profiles directly.
|
|
||||||
|
|
||||||
An installation could then place a profile with the same ID in its configured
|
|
||||||
profile directory. For example, a local `weather-light` definition could point
|
|
||||||
to an OpenAI-compatible endpoint on the deployment network. When no operator
|
|
||||||
definition exists, Weatherreporter's embedded definition would keep the
|
|
||||||
application usable without additional profile files.
|
|
||||||
|
|
||||||
This pattern is useful beyond Weatherreporter. Any Promptkit consumer may want
|
|
||||||
application-owned execution tiers or workload-specific defaults without
|
|
||||||
adding domain-specific profiles to Promptkit's general built-in catalog.
|
|
||||||
|
|
||||||
## Current Constraint
|
|
||||||
|
|
||||||
Promptkit currently resolves matching profile IDs in this order:
|
|
||||||
|
|
||||||
1. in-memory profiles supplied through `WithProfiles`;
|
|
||||||
2. one configured profile file, `fs.FS`, or directory source; and
|
|
||||||
3. Promptkit's embedded built-in profiles.
|
|
||||||
|
|
||||||
These layers do not express the desired application-default relationship:
|
|
||||||
|
|
||||||
- `WithProfiles` has higher precedence than the configured source, so it would
|
|
||||||
prevent an operator file from overriding an application profile with the
|
|
||||||
same ID.
|
|
||||||
- `WithProfileFS` can hold embedded application assets, but it occupies the
|
|
||||||
configured-source layer and therefore replaces rather than sits beneath a
|
|
||||||
configured profile directory or file.
|
|
||||||
- adding downstream profile IDs to Promptkit's built-in catalog would make the
|
|
||||||
library own application-specific policy.
|
|
||||||
|
|
||||||
A downstream application could build its own filesystem overlay, but that
|
|
||||||
would duplicate Promptkit's profile discovery, error, and precedence behavior
|
|
||||||
at the consumer boundary.
|
|
||||||
|
|
||||||
## Requested Capability
|
|
||||||
|
|
||||||
Add one optional application fallback profile source to engine construction.
|
|
||||||
When present, matching profile IDs should resolve in this order:
|
|
||||||
|
|
||||||
1. in-memory profiles supplied through `WithProfiles`;
|
|
||||||
2. the ordinary configured profile source selected through a profile option or
|
|
||||||
`Config.ProfileDir`;
|
|
||||||
3. the application fallback profile source; and
|
|
||||||
4. Promptkit's embedded built-in profiles.
|
|
||||||
|
|
||||||
When no application fallback is configured, existing source precedence and
|
|
||||||
behavior must remain unchanged.
|
|
||||||
|
|
||||||
The minimum useful public surface is an `fs.FS`-backed option because consumers
|
|
||||||
can embed YAML profile assets. A possible API shape is:
|
|
||||||
|
|
||||||
```go
|
|
||||||
promptkit.WithFallbackProfileFS(profileFS, ".")
|
|
||||||
```
|
|
||||||
|
|
||||||
The name is illustrative rather than prescriptive. A companion option for
|
|
||||||
validated `Profile` values could be added if Promptkit maintainers find it
|
|
||||||
generally useful, but it is not required for the Weatherreporter use case.
|
|
||||||
|
|
||||||
## Required Semantics
|
|
||||||
|
|
||||||
- A higher-precedence source falls through only when the requested profile ID
|
|
||||||
is absent.
|
|
||||||
- A malformed, unreadable, duplicate, ambiguous, or otherwise invalid matching
|
|
||||||
profile is an error and must not silently fall through.
|
|
||||||
- The fallback source uses the existing strict profile YAML format and profile
|
|
||||||
validation rules.
|
|
||||||
- Profile values are selected as a whole. This feature does not merge,
|
|
||||||
inherit, or partially overlay profile definitions.
|
|
||||||
- `InspectProfile`, `Prepare`, prepared execution, and ordinary execution use
|
|
||||||
the same profile-source precedence.
|
|
||||||
- An explicit request profile continues to take precedence over a prompt's
|
|
||||||
`default_profile`; this request concerns definition lookup after the profile
|
|
||||||
ID has been selected.
|
|
||||||
- Repeated fallback-source options should follow Promptkit's documented
|
|
||||||
same-category option convention, normally with the last value replacing the
|
|
||||||
earlier value.
|
|
||||||
- A canceled lookup, invalid fallback asset, or unknown resolved backend should
|
|
||||||
continue to cross the public facade through Promptkit's existing public error
|
|
||||||
identities.
|
|
||||||
- Exact profile inspection must remain side-effect free and must not contact a
|
|
||||||
model provider.
|
|
||||||
|
|
||||||
## Application And Library Boundaries
|
|
||||||
|
|
||||||
Promptkit should own:
|
|
||||||
|
|
||||||
- the additional repository layer;
|
|
||||||
- deterministic lookup and fallthrough behavior;
|
|
||||||
- validation of the supplied source through the existing profile contract;
|
|
||||||
- consistent use of the layer across inspection and execution; and
|
|
||||||
- public documentation and tests for the added precedence rule.
|
|
||||||
|
|
||||||
The consuming application should continue to own:
|
|
||||||
|
|
||||||
- whether it supplies fallback profiles;
|
|
||||||
- the profile IDs and their domain meaning;
|
|
||||||
- embedded profile contents and model choices;
|
|
||||||
- application configuration and override policy;
|
|
||||||
- report- or workload-to-profile assignment; and
|
|
||||||
- credential checks and operator-facing errors beyond Promptkit's public
|
|
||||||
contract.
|
|
||||||
|
|
||||||
## Non-Goals
|
|
||||||
|
|
||||||
This request does not ask Promptkit to add:
|
|
||||||
|
|
||||||
- Weatherreporter-specific profile IDs to its built-in catalog;
|
|
||||||
- profile inheritance, aliases, or field-level merging;
|
|
||||||
- automatic endpoint discovery or availability probing;
|
|
||||||
- provider failover or fallback from a failed selected profile;
|
|
||||||
- per-request model benchmarking or tier selection;
|
|
||||||
- application configuration discovery; or
|
|
||||||
- eager validation of every profile in every source.
|
|
||||||
|
|
||||||
## Compatibility
|
|
||||||
|
|
||||||
The feature can be additive. Engines that do not configure an application
|
|
||||||
fallback source should retain their current public behavior and precedence.
|
|
||||||
Existing uses of `WithProfiles`, `WithProfileFile`, `WithProfileFS`, and
|
|
||||||
`Config.ProfileDir` should not change meaning.
|
|
||||||
|
|
||||||
The application fallback is deliberately lower precedence than every existing
|
|
||||||
consumer-configured source. This preserves the established expectation that a
|
|
||||||
custom profile definition can override a packaged default with the same ID.
|
|
||||||
|
|
||||||
## Acceptance Criteria
|
|
||||||
|
|
||||||
The capability is sufficient for downstream adoption when Promptkit can
|
|
||||||
demonstrate that:
|
|
||||||
|
|
||||||
- a fallback-only profile can be inspected and used for preparation and
|
|
||||||
execution;
|
|
||||||
- a configured directory, file, or `fs.FS` profile with the same ID overrides
|
|
||||||
the fallback profile;
|
|
||||||
- an absent configured profile falls through to the application fallback;
|
|
||||||
- an invalid configured match fails instead of falling through;
|
|
||||||
- an absent application fallback profile continues to resolve from Promptkit's
|
|
||||||
built-in catalog;
|
|
||||||
- `WithProfiles` retains highest precedence;
|
|
||||||
- behavior is identical across inspection, preparation, and execution; and
|
|
||||||
- omitting the new option preserves existing tests and public contracts.
|
|
||||||
@@ -1,516 +0,0 @@
|
|||||||
# Promptkit Migration Roadmap
|
|
||||||
|
|
||||||
Status: Completed roadmap record.
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
This roadmap records the scope, decisions, and completed outcome of replacing
|
|
||||||
the external Scriptorium CLI integration with Promptkit. Canonical
|
|
||||||
documentation outside `docs/roadmap/` owns the implemented behavior.
|
|
||||||
|
|
||||||
## Pre-Migration Baseline
|
|
||||||
|
|
||||||
Status: Historical migration input.
|
|
||||||
|
|
||||||
Before the migration, Weatherreporter exposed seven report definitions, but
|
|
||||||
only four had complete prompt-backed report implementations:
|
|
||||||
|
|
||||||
- Daily Report: `weather.daily_generated_text`
|
|
||||||
- Today Report: `weather.today_generated_text`
|
|
||||||
- Tomorrow Report: `weather.tomorrow_generated_text`
|
|
||||||
- Hourly Report: `weather.hourly_generated_text`
|
|
||||||
|
|
||||||
The three-day, weekend, and storm commands and registry definitions had no
|
|
||||||
corresponding Scriptorium prompt or schema and never formed complete
|
|
||||||
operational report products. The `weather.daily_report` Scriptorium prompt was
|
|
||||||
legacy source material and was not selected by the registry.
|
|
||||||
|
|
||||||
The Scriptorium source corpus was retained temporarily under
|
|
||||||
`docs/roadmap/scriptorium/` as migration input. It contained the four
|
|
||||||
operational generated-text prompt definitions, their referenced content,
|
|
||||||
private response schemas, shared instructions, and the unused legacy Daily
|
|
||||||
Markdown prompt. The temporary corpus was removed after the runtime assets
|
|
||||||
were reconciled and embedded.
|
|
||||||
|
|
||||||
## Implemented End State
|
|
||||||
|
|
||||||
Status: Completed.
|
|
||||||
|
|
||||||
Weatherreporter pins
|
|
||||||
`gitea.maximumdirect.net/eric/promptkit` at `v0.4.0` and uses it as the
|
|
||||||
in-process engine for prompt inspection, prepared execution, provider calls,
|
|
||||||
and first-pass output validation.
|
|
||||||
|
|
||||||
The `scriptorium` executable, subprocess adapter, configuration, runtime
|
|
||||||
dependency, direct-Markdown execution path, and integration documentation have
|
|
||||||
been removed. The four operational reports continue to use structured
|
|
||||||
generated text followed by weatherreporter-owned validation and Markdown
|
|
||||||
templates.
|
|
||||||
|
|
||||||
The unfinished three-day, weekend, and storm reports are not implemented as
|
|
||||||
part of this migration. Their incomplete CLI, registry, documentation, and
|
|
||||||
generation declarations are removed from the implemented surface before the
|
|
||||||
migration is considered complete. Any future implementation of those products
|
|
||||||
requires separate roadmap scope, prompt and schema design, tests, and
|
|
||||||
documentation.
|
|
||||||
|
|
||||||
Weather selection, forecast derivation, valid periods, module construction,
|
|
||||||
Recent Changes, generated-text interpretation, Markdown templates, durable
|
|
||||||
state, inspection, output copies, and Distributor notification remain owned by
|
|
||||||
weatherreporter.
|
|
||||||
|
|
||||||
The four report prompts and private response schemas are versioned embedded
|
|
||||||
application assets. Operators configure Promptkit profiles without replacing
|
|
||||||
the report-owned corpus. One Promptkit engine is constructed per CLI
|
|
||||||
invocation and shared by every report in that invocation, including all
|
|
||||||
reports in a morning or evening batch.
|
|
||||||
|
|
||||||
Promptkit is isolated behind a weatherreporter-owned execution contract.
|
|
||||||
Promptkit request, result, validation, error, profile, backend, and provider
|
|
||||||
types do not leak into application orchestration, report definitions, domain
|
|
||||||
packages, CLI summaries, durable state contracts, or Distributor behavior.
|
|
||||||
|
|
||||||
## Goals
|
|
||||||
|
|
||||||
Status: Completed migration outcomes.
|
|
||||||
|
|
||||||
- Removed the Scriptorium runtime dependency and subprocess boundary.
|
|
||||||
- Migrated the four operational report prompts to Promptkit `v0.4.0`.
|
|
||||||
- Used prepared execution to persist preparation provenance before provider work
|
|
||||||
while executing the exact frozen snapshot.
|
|
||||||
- Validated report prompt and profile selections before weather collection when
|
|
||||||
the required information is available.
|
|
||||||
- Preserved deterministic module snapshots and structured Recent Changes.
|
|
||||||
- Preserved generated-text domain validation and repository-owned Markdown
|
|
||||||
rendering.
|
|
||||||
- Preserved context cancellation, actionable errors, secret redaction, and
|
|
||||||
inspectable failures.
|
|
||||||
- Improved durable prompt provenance with prompt, input, profile, model,
|
|
||||||
validation, usage, and timing metadata.
|
|
||||||
- Kept content-rich prompt and response diagnostics separate from routine
|
|
||||||
metadata and CLI output.
|
|
||||||
- Kept tests offline and deterministic through injected Promptkit model
|
|
||||||
clients and fixtures.
|
|
||||||
- Removed incomplete report declarations from the implemented product surface
|
|
||||||
rather than creating new report products during an integration migration.
|
|
||||||
|
|
||||||
## Non-Goals
|
|
||||||
|
|
||||||
Status: Completed migration constraints.
|
|
||||||
|
|
||||||
The completed migration did not:
|
|
||||||
|
|
||||||
- create prompts, schemas, templates, or completed products for three-day,
|
|
||||||
weekend, or storm reports;
|
|
||||||
- preserve the unused `weather.daily_report` legacy Markdown prompt as an
|
|
||||||
active runtime asset;
|
|
||||||
- preserve a direct-Markdown LLM generation mode;
|
|
||||||
- move meteorological selection, derivation, thresholds, or comparison logic
|
|
||||||
into prompts or Promptkit;
|
|
||||||
- send raw unbounded Weather API responses to the model;
|
|
||||||
- replace weatherreporter's generated-text domain validation or Markdown
|
|
||||||
template rendering;
|
|
||||||
- add a general workflow engine, provider plugin system, or arbitrary backend
|
|
||||||
registry;
|
|
||||||
- add automatic provider, validation, repair, or capacity retries;
|
|
||||||
- add concurrent report generation to the sequential batch workflow;
|
|
||||||
- expose Promptkit types as weatherreporter contracts;
|
|
||||||
- keep a production-selectable Scriptorium/Promptkit dual-run mode;
|
|
||||||
- require Promptkit eager source validation, structured generation errors, or
|
|
||||||
semantic execution-target fingerprints; or
|
|
||||||
- use an unpublished Promptkit commit, committed Go workspace, or committed
|
|
||||||
local module replacement.
|
|
||||||
|
|
||||||
## Locked Decisions
|
|
||||||
|
|
||||||
Status: Implemented migration decisions.
|
|
||||||
|
|
||||||
### Dependency And Upgrade Boundary
|
|
||||||
|
|
||||||
- The migration pins the tagged Promptkit `v0.4.0` release.
|
|
||||||
- Coordinated local development may temporarily use the sibling Promptkit
|
|
||||||
checkout, but committed module metadata must reference the tagged release.
|
|
||||||
- The adapter relies on the public root Promptkit package only.
|
|
||||||
- A future Promptkit upgrade requires explicit review of prepared-execution
|
|
||||||
lifecycle, prompt and profile inspection, prompt/profile/schema formats,
|
|
||||||
error identities, validation behavior, capacity behavior, and the outbound
|
|
||||||
provider contract.
|
|
||||||
- Promptkit's deferred eager source validation, structured generation errors,
|
|
||||||
and semantic execution-target fingerprints do not block this migration.
|
|
||||||
|
|
||||||
### Operational Report Scope
|
|
||||||
|
|
||||||
- The migration preserves these prompt IDs:
|
|
||||||
`weather.daily_generated_text`, `weather.today_generated_text`,
|
|
||||||
`weather.tomorrow_generated_text`, and `weather.hourly_generated_text`.
|
|
||||||
- Each operational report definition selects the exact embedded prompt version
|
|
||||||
`1.0.0`; execution does not rely on ambiguous single-version lookup.
|
|
||||||
- Morning and evening batch membership remains based on Today, Tomorrow, and
|
|
||||||
eligible future Daily reports.
|
|
||||||
- Three-day, weekend, and storm are removed from current CLI help, parsing,
|
|
||||||
report registry membership, tests that claim implemented generation, and
|
|
||||||
non-roadmap documentation.
|
|
||||||
- The future product concepts may remain under `docs/roadmap/`, but migration
|
|
||||||
verification does not invent outputs or compare nonexistent prompts.
|
|
||||||
|
|
||||||
### Application Boundary
|
|
||||||
|
|
||||||
- Promptkit remains an adapter boundary even though it runs in process.
|
|
||||||
- A weatherreporter-owned contract represents prompt identity, preparation,
|
|
||||||
execution, output, validation, usage, provenance, and neutral error
|
|
||||||
categories.
|
|
||||||
- The Promptkit adapter maps public Promptkit values into that contract.
|
|
||||||
- App orchestration and test fakes depend on the project-owned contract, not
|
|
||||||
Promptkit.
|
|
||||||
- Scriptorium-specific request, result, error, and generation-mode types are
|
|
||||||
removed rather than renamed and retained.
|
|
||||||
|
|
||||||
### Prompt And Schema Ownership
|
|
||||||
|
|
||||||
- Weatherreporter embeds the four operational prompt definitions, referenced
|
|
||||||
prompt content, shared prompt content, and private response schemas.
|
|
||||||
- Assets remain separate files rather than inline Go strings.
|
|
||||||
- The temporary corpus under `docs/roadmap/scriptorium/` is migration source
|
|
||||||
material, not the final runtime location.
|
|
||||||
- Weatherreporter's existing generated-text domain types, schemas, and
|
|
||||||
templates remain the canonical application contract. Imported Scriptorium
|
|
||||||
assets are reconciled with that contract rather than copied blindly or kept
|
|
||||||
as duplicate runtime schemas.
|
|
||||||
- The imported Daily schema's incorrect Today `$id` and title are corrected.
|
|
||||||
- `confidence` is handled consistently across each prompt, provider-facing
|
|
||||||
schema, generated-text domain type, and template. The existing optional
|
|
||||||
weatherreporter field remains supported unless a separate domain decision
|
|
||||||
removes it.
|
|
||||||
- Prompt input metadata identifies the serialized data package as YAML rather
|
|
||||||
than JSON.
|
|
||||||
- Imported `pipeline-weather/...` schema paths are replaced with paths valid
|
|
||||||
inside the embedded Promptkit schema source.
|
|
||||||
- Imported `repair_attempts: 2` values are removed or set to zero. The
|
|
||||||
migration does not rely on Promptkit's internal-only repair capability.
|
|
||||||
- The unused `weather.daily_report` prompt is not promoted into runtime assets.
|
|
||||||
- One centralized embedded prompt/schema source is sufficient; Weatherreporter
|
|
||||||
does not need Notarius's multi-module asset-flattening registry.
|
|
||||||
|
|
||||||
### Profiles, Backends, And Credentials
|
|
||||||
|
|
||||||
- Execution profiles remain operator-configurable rather than embedded report
|
|
||||||
policy.
|
|
||||||
- Each embedded operational prompt declares Promptkit's built-in
|
|
||||||
`gemini-flash-latest` profile as its default.
|
|
||||||
- `gemini-flash-latest` is intentionally a moving model alias. The execution
|
|
||||||
record captures the effective model identity, but operators who require a
|
|
||||||
pinned model must select an explicit external profile.
|
|
||||||
- Configuration supports at most one external profile source:
|
|
||||||
`promptkit.profile_file` or `promptkit.profile_dir`. The two fields are
|
|
||||||
mutually exclusive.
|
|
||||||
- A nonblank `promptkit.profile` is the explicit request profile for every
|
|
||||||
report in the invocation and takes precedence over each prompt's
|
|
||||||
`default_profile`. A blank value uses the prompt default.
|
|
||||||
- Promptkit's normal profile-source precedence remains intact: an external
|
|
||||||
matching profile takes precedence over an embedded built-in profile, and an
|
|
||||||
invalid matching external profile is an error rather than a reason to fall
|
|
||||||
back.
|
|
||||||
- Weatherreporter exposes Promptkit's conventional `local` backend through the
|
|
||||||
narrow `promptkit.local.endpoint` and
|
|
||||||
`promptkit.local.concurrency_limit` configuration fields. It does not expose
|
|
||||||
arbitrary backend registration.
|
|
||||||
- A configured local endpoint registers the engine-scoped `local` backend. An
|
|
||||||
operator-supplied external profile selects it with `backend: local` and owns
|
|
||||||
the model-specific settings; Weatherreporter does not invent a local model
|
|
||||||
profile.
|
|
||||||
- Local concurrency defaults to one. A value of zero means unlimited, matching
|
|
||||||
Promptkit, and a negative value is invalid. Queue capacity and general
|
|
||||||
backend parameters are not exposed.
|
|
||||||
- Credential values remain in environment variables or file-backed
|
|
||||||
environment secrets. Configuration contains only credential source names.
|
|
||||||
- Provider credentials never appear in logs, errors, CLI output, durable
|
|
||||||
metadata, preparation records, execution records, or debug summaries.
|
|
||||||
- Promptkit `InspectProfile` reports structural target and credential
|
|
||||||
requirements; Weatherreporter owns policy for checking configured
|
|
||||||
environment availability.
|
|
||||||
- Promptkit revalidates environment credentials at `RunPrepared`; a successful
|
|
||||||
preparation does not promise that execution-time credentials remain
|
|
||||||
available.
|
|
||||||
|
|
||||||
### Configuration Contract
|
|
||||||
|
|
||||||
The replacement configuration surface is:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
promptkit:
|
|
||||||
profile: ""
|
|
||||||
profile_file: ""
|
|
||||||
profile_dir: ""
|
|
||||||
timeout: 2m
|
|
||||||
|
|
||||||
local:
|
|
||||||
endpoint: ""
|
|
||||||
concurrency_limit: 1
|
|
||||||
```
|
|
||||||
|
|
||||||
- `timeout` remains the transport-wide provider-call safety cap.
|
|
||||||
- A blank local endpoint leaves the conventional local backend unregistered.
|
|
||||||
- Scriptorium's `binary`, `config_path`, and `extra_args` settings have no
|
|
||||||
Promptkit equivalents and are removed.
|
|
||||||
- Configuration validation rejects simultaneous `profile_file` and
|
|
||||||
`profile_dir` values, invalid local endpoints, negative concurrency, and
|
|
||||||
selected profiles that cannot resolve their backend.
|
|
||||||
|
|
||||||
### Engine Construction And Inspection
|
|
||||||
|
|
||||||
- One Promptkit engine is constructed per CLI invocation at the application
|
|
||||||
composition boundary.
|
|
||||||
- Single-report generation and every report in a batch use that same engine.
|
|
||||||
- Per-report orchestration does not construct a default engine.
|
|
||||||
- Promptkit backend capacity state and HTTP transport are shared consistently
|
|
||||||
for the invocation.
|
|
||||||
- Before collection, `InspectPrompt` checks every selected report's exact ID
|
|
||||||
and version, declared `data_package` input, default-profile metadata, prompt
|
|
||||||
hash availability, and declared output contract.
|
|
||||||
- `InspectPrompt` is a point-in-time structural check. It does not load a JSON
|
|
||||||
Schema, resolve a profile, or freeze later execution.
|
|
||||||
- Explicit profile overrides and relevant prompt defaults are checked with
|
|
||||||
`InspectProfile` before collection when application policy requires them.
|
|
||||||
- `InspectProfile` is also point-in-time and does not check credential values.
|
|
||||||
- Successful `PrepareExecution`, not inspection, is the per-run authority for
|
|
||||||
loaded schema, rendered content, frozen inputs, effective settings, and
|
|
||||||
durable execution provenance.
|
|
||||||
|
|
||||||
### Prompt Input
|
|
||||||
|
|
||||||
- Promptkit receives only the curated `data_package` produced by
|
|
||||||
`internal/promptinput`.
|
|
||||||
- Weatherreporter serializes the package once, atomically persists those exact
|
|
||||||
bytes, and supplies the same bytes with a Promptkit inline artifact.
|
|
||||||
- The managed data-package path may be supplied as non-secret provenance
|
|
||||||
through the inline artifact URI.
|
|
||||||
- Weatherreporter does not delegate unrestricted path loading to Promptkit's
|
|
||||||
default file artifact reader.
|
|
||||||
- Prompt inspection and adapter tests verify that `data_package` is required
|
|
||||||
and declared with the chosen YAML media type.
|
|
||||||
|
|
||||||
### Prepared Execution
|
|
||||||
|
|
||||||
- `Engine.PrepareExecution` replaces Scriptorium render preflight.
|
|
||||||
- Weatherreporter obtains `PreparedExecution.Details`, maps a safe subset into
|
|
||||||
its own preparation record, and persists that record before calling
|
|
||||||
`Engine.RunPrepared`.
|
|
||||||
- `RunPrepared` executes the frozen prompt, profile, schema, inputs, rendered
|
|
||||||
messages, target, and validation resources retained by the handle.
|
|
||||||
- Every acquired handle is followed immediately by `defer handle.Discard()`.
|
|
||||||
Discard is safe after execution and releases unused private execution state.
|
|
||||||
- Handles remain adapter-local, engine-bound, one-shot, in-process values.
|
|
||||||
They are never serialized, persisted, copied into app contracts, or treated
|
|
||||||
as restartable jobs.
|
|
||||||
- Preparation and execution use independent contexts. Execution receives the
|
|
||||||
active report workflow context.
|
|
||||||
- Capacity is not reserved during preparation. Capacity rejection can
|
|
||||||
therefore occur after a preparation record has been persisted.
|
|
||||||
- `RunPrepared` consumes the handle on success and every operational failure.
|
|
||||||
- Preparation details remain available from the adapter after execution or
|
|
||||||
discard, but rendered messages are not copied into routine durable state.
|
|
||||||
- Promptkit execution timing excludes preparation and consumer-held delay.
|
|
||||||
Weatherreporter records preparation timing and execution timing separately.
|
|
||||||
|
|
||||||
### Execution And Validation
|
|
||||||
|
|
||||||
- All four operational reports use Promptkit JSON Schema output validation.
|
|
||||||
- A completed Promptkit validation rejection returns a `RunResult`; the
|
|
||||||
adapter retains raw output and bounded validation details before failing the
|
|
||||||
report.
|
|
||||||
- An operational generation or validation error returns no partial
|
|
||||||
`RunResult`.
|
|
||||||
- Weatherreporter's `internal/generatedtext` validation remains the final
|
|
||||||
report-specific decode and domain boundary.
|
|
||||||
- Weatherreporter's `internal/reporttemplate` remains responsible for managed
|
|
||||||
Markdown rendering.
|
|
||||||
- Weatherreporter atomically persists Promptkit raw output and later artifacts
|
|
||||||
rather than asking Promptkit to choose managed filesystem paths.
|
|
||||||
- No Promptkit output-repair behavior is assumed or requested.
|
|
||||||
|
|
||||||
## Durable Artifacts And Observability
|
|
||||||
|
|
||||||
Status: Implemented design constraints.
|
|
||||||
|
|
||||||
Routine durable state retains useful non-secret provenance without persisting
|
|
||||||
full rendered prompts.
|
|
||||||
|
|
||||||
The preparation record contains:
|
|
||||||
|
|
||||||
- prompt ID and exact version;
|
|
||||||
- prompt definition hash;
|
|
||||||
- rendered prompt hash;
|
|
||||||
- input hashes;
|
|
||||||
- selected profile and backend identity;
|
|
||||||
- effective model identity;
|
|
||||||
- output contract summary;
|
|
||||||
- preparation start, end, and duration; and
|
|
||||||
- the path of the exact persisted data package.
|
|
||||||
|
|
||||||
The execution record and run metadata contain, when available:
|
|
||||||
|
|
||||||
- Promptkit run ID;
|
|
||||||
- prompt ID, version, and hashes;
|
|
||||||
- input hashes;
|
|
||||||
- selected profile, backend, and model identity;
|
|
||||||
- generated-content hash;
|
|
||||||
- token usage;
|
|
||||||
- execution start, end, and duration;
|
|
||||||
- validation status and bounded diagnostics; and
|
|
||||||
- paths of separately persisted raw output, normalized generated text, render
|
|
||||||
context, managed Markdown, and other artifacts reached by the workflow.
|
|
||||||
|
|
||||||
Provider endpoints, full effective model parameter maps, rendered messages,
|
|
||||||
schema bodies, data-package contents, and generated content do not belong in
|
|
||||||
routine metadata or CLI summaries.
|
|
||||||
|
|
||||||
Rendered messages and other content-rich preparation or response diagnostics
|
|
||||||
are available only when the operator supplies
|
|
||||||
`--llm-debug-dir <path>` to a single-report or batch command.
|
|
||||||
|
|
||||||
- There is no persistent YAML setting for debug capture.
|
|
||||||
- The debug root is validated or created before weather collection or provider
|
|
||||||
work. A requested destination that cannot be secured or written is an error.
|
|
||||||
- Artifacts are grouped beneath
|
|
||||||
`<path>/<report-id>/<valid-date>/<run-id>/`.
|
|
||||||
- Directories and files use owner-only permissions and atomic writes.
|
|
||||||
- Debug artifacts may contain rendered messages and content-rich preparation
|
|
||||||
or response diagnostics, but never credentials.
|
|
||||||
- The debug path appears in command output only when debug capture is enabled;
|
|
||||||
it is not added to routine durable metadata.
|
|
||||||
- Debug artifacts are not cache or comparison inputs. Their retention is owned
|
|
||||||
by the operator who selected the directory.
|
|
||||||
|
|
||||||
### Artifact Identities And Versions
|
|
||||||
|
|
||||||
Weatherreporter replaces Scriptorium-specific artifact identities rather than
|
|
||||||
reusing names whose meanings have changed:
|
|
||||||
|
|
||||||
- `PromptPreparationArtifact` uses schema version
|
|
||||||
`weatherreporter.prompt_preparation.v1`, is written as
|
|
||||||
`prompt_preparation.<runID>.json`, and is referenced by
|
|
||||||
`preparationPath`.
|
|
||||||
- `PromptExecutionArtifact` uses schema version
|
|
||||||
`weatherreporter.prompt_execution.v1`, is written as
|
|
||||||
`prompt_execution.<runID>.json`, and is referenced by `executionPath`.
|
|
||||||
- Run metadata advances to `weatherreporter.metadata.v2` and uses those new
|
|
||||||
path fields.
|
|
||||||
|
|
||||||
Preparation files remain beneath the existing configurable `preflight/`
|
|
||||||
directory, and execution files remain beneath the existing `snapshots/` tree.
|
|
||||||
The stable physical grouping limits deployment disruption without preserving
|
|
||||||
misleading Scriptorium-era filenames or field names. Raw generated output,
|
|
||||||
normalized generated text, render context, managed Markdown, and other
|
|
||||||
artifacts whose meanings have not changed retain their existing names and
|
|
||||||
locations.
|
|
||||||
|
|
||||||
Run inspection remains able to read `weatherreporter.metadata.v1` and its
|
|
||||||
legacy `preflightPath` and `generatedTextResultPath` references. New runs write
|
|
||||||
only the v2 metadata and new artifact names; Weatherreporter does not
|
|
||||||
dual-write deprecated aliases. CLI summary fields adopt `preparationPath` and
|
|
||||||
`executionPath` as an explicit, documented contract change.
|
|
||||||
|
|
||||||
## Failure Contract
|
|
||||||
|
|
||||||
Status: Implemented design constraints.
|
|
||||||
|
|
||||||
- A preparation failure produces a redacted weatherreporter-owned failure
|
|
||||||
receipt with report, RunID, prompt, stage, timing, and classified error
|
|
||||||
context. It does not fabricate Promptkit preparation details.
|
|
||||||
- An operational execution failure retains the successful preparation record
|
|
||||||
and adds a redacted execution failure receipt. No partial Promptkit result or
|
|
||||||
model output is invented.
|
|
||||||
- A Promptkit validation rejection retains the returned result, raw generated
|
|
||||||
output, validation details, and safe provenance before the report fails.
|
|
||||||
- A later generated-text decode, domain-validation, or template failure
|
|
||||||
retains every raw and validated artifact reached before that stage.
|
|
||||||
- Caller cancellation takes precedence when the active workflow context is
|
|
||||||
canceled.
|
|
||||||
- `promptkit.CapacityError` is recognized with `errors.As`; its backend ID is
|
|
||||||
copied into a weatherreporter-owned capacity error while
|
|
||||||
`ErrCapacityExceeded` remains the classification.
|
|
||||||
- Capacity rejection is an operational report failure, not invalid model
|
|
||||||
output, and does not trigger an automatic retry.
|
|
||||||
- Other Promptkit public error identities are translated into the narrow
|
|
||||||
weatherreporter error categories needed by CLI, metadata, and batch
|
|
||||||
behavior. Diagnostic prose is not parsed as a contract.
|
|
||||||
- Single-report commands return the classified failure with available
|
|
||||||
inspectable paths.
|
|
||||||
- Batch runs continue independent later reports under the existing batch
|
|
||||||
failure policy.
|
|
||||||
- Any future retry policy belongs to app orchestration, not the adapter.
|
|
||||||
|
|
||||||
## Compatibility Requirements
|
|
||||||
|
|
||||||
Status: Implemented design constraints.
|
|
||||||
|
|
||||||
- Daily, Today, Tomorrow, and Hourly report IDs, prompt IDs, valid periods,
|
|
||||||
artifact grouping, output names, and Distributor bundle behavior remain
|
|
||||||
stable.
|
|
||||||
- Morning and evening batch collection, planning, ordering, and continuation
|
|
||||||
behavior remains stable.
|
|
||||||
- Module snapshot and Recent Changes behavior remains deterministic.
|
|
||||||
- Promptkit receives only the existing curated prompt-input boundary.
|
|
||||||
- Managed Markdown remains the Distributor upload source.
|
|
||||||
- RunID lookup and inspection remain available for successful and failed runs.
|
|
||||||
- Existing managed paths remain stable where their meaning is unchanged.
|
|
||||||
Scriptorium-specific artifact names or schemas change when retaining them
|
|
||||||
would misrepresent the Promptkit contract.
|
|
||||||
- Existing v1 run metadata and referenced artifacts remain inspectable after
|
|
||||||
the migration. New runs use the v2 metadata and Promptkit-era artifact
|
|
||||||
identities without dual-writing deprecated aliases.
|
|
||||||
- Artifact or metadata schema changes are explicit, documented, and covered by
|
|
||||||
state and inspection tests.
|
|
||||||
- Prompt or generated content is not added to routine logs or CLI summaries.
|
|
||||||
- Tests do not require live providers or credentials.
|
|
||||||
- Removing incomplete three-day, weekend, and storm surfaces is documented as
|
|
||||||
correction of an unfinished product boundary, not as successful Promptkit
|
|
||||||
migration of those reports.
|
|
||||||
|
|
||||||
## Verification And Completion Criteria
|
|
||||||
|
|
||||||
Status: Completed and verified.
|
|
||||||
|
|
||||||
Completion was verified by the following outcomes:
|
|
||||||
|
|
||||||
- the four operational reports inspect, prepare, and execute through Promptkit
|
|
||||||
`v0.4.0` using embedded report-owned assets;
|
|
||||||
- every report uses exact prompt version `1.0.0`, requires the YAML
|
|
||||||
`data_package`, and declares the expected JSON Schema output contract;
|
|
||||||
- prepared execution persists a safe preparation record before provider work
|
|
||||||
and executes the same frozen snapshot;
|
|
||||||
- deterministic offline adapter and app tests cover success, preparation
|
|
||||||
failure, credential revalidation, capacity rejection, cancellation, timeout,
|
|
||||||
generation failure, Promptkit validation rejection, generated-text domain
|
|
||||||
failure, template failure, and handle discard;
|
|
||||||
- morning and evening batches construct one engine and preserve current
|
|
||||||
collection, planning, ordering, continuation, output, and notification
|
|
||||||
behavior;
|
|
||||||
- the temporary corpus has been reconciled into one runtime prompt/schema
|
|
||||||
source without duplicate provider-facing schemas;
|
|
||||||
- configuration examples load and contain no Scriptorium fields;
|
|
||||||
- CLI summaries and inspection commands expose the new project-owned artifact
|
|
||||||
contract without Promptkit types;
|
|
||||||
- Scriptorium code, configuration, tests, and runtime documentation have been
|
|
||||||
removed;
|
|
||||||
- incomplete three-day, weekend, and storm commands, registry entries, tests,
|
|
||||||
and current-behavior documentation have been removed or moved to roadmap
|
|
||||||
scope;
|
|
||||||
- non-roadmap documentation describes only the implemented Promptkit
|
|
||||||
integration;
|
|
||||||
- `go test ./...`, required focused or race-enabled checks, CLI help
|
|
||||||
validation, and `git diff --check` pass; and
|
|
||||||
- no committed `go.work`, local `replace`, live-provider test, or
|
|
||||||
secret-bearing fixture remains.
|
|
||||||
|
|
||||||
Fixture-based comparison with prior Scriptorium behavior is sufficient.
|
|
||||||
Production dual-run is not required because model calls are nondeterministic,
|
|
||||||
costly, and difficult to compare meaningfully.
|
|
||||||
|
|
||||||
## Decision Status
|
|
||||||
|
|
||||||
Status: Completed.
|
|
||||||
|
|
||||||
The roadmap has no remaining open product or architecture questions. Later
|
|
||||||
changes to this completed scope require new roadmap or decision-record scope
|
|
||||||
rather than implicit changes to this historical record.
|
|
||||||
@@ -1,75 +0,0 @@
|
|||||||
# Troubleshooting
|
|
||||||
|
|
||||||
Start with the command's classified error. When content-rich prompt diagnostics
|
|
||||||
are needed, enable a new run with `--llm-debug-dir` and handle the resulting
|
|
||||||
secure capture as sensitive. Current-version workspace receipts can provide
|
|
||||||
additional context when present, but are transitional state rather than a
|
|
||||||
long-term troubleshooting interface.
|
|
||||||
|
|
||||||
## Prompt inspection or credentials fail before collection
|
|
||||||
|
|
||||||
A prompt/version, contract, selected profile, unsupported direct-key profile,
|
|
||||||
or required environment credential can fail before weather collection. Correct
|
|
||||||
the configured `promptkit` profile or profile source, confirm the exact
|
|
||||||
Promptkit asset is available, and supply any reported environment credential.
|
|
||||||
Do not add provider keys to YAML. See [configuration](config.md).
|
|
||||||
|
|
||||||
## Local profile override is malformed or selects an unexpected model
|
|
||||||
|
|
||||||
`promptkit.profile_file` and `promptkit.profile_dir` supply complete profile
|
|
||||||
definitions. A same-ID definition replaces the embedded profile, and a malformed
|
|
||||||
matching definition fails before collection instead of falling back. Validate
|
|
||||||
the selected profile's YAML, ID, backend or endpoint, and model. If the model
|
|
||||||
is unexpected, first check the global `promptkit.profile` selection and then
|
|
||||||
look for a same-ID definition in the configured file or directory.
|
|
||||||
|
|
||||||
Current-version preparation and execution receipts may retain the selected
|
|
||||||
profile ID and effective backend/model, but not an endpoint or credential.
|
|
||||||
Use them only as supplemental context after the active command error or an
|
|
||||||
explicit secure debug capture. See the maintained
|
|
||||||
[local `weather-light` profile example](../examples/weather-light-local-profile.yml).
|
|
||||||
|
|
||||||
## Local model endpoint is unavailable
|
|
||||||
|
|
||||||
An endpoint-only `weather-light` override can pass preflight and still fail
|
|
||||||
during provider preparation or execution when the local server is unavailable
|
|
||||||
or does not accept the configured model. Start the local server, correct the
|
|
||||||
endpoint or model in the profile, and run the command again. Weatherreporter
|
|
||||||
does not probe endpoints or automatically use a remote profile instead.
|
|
||||||
|
|
||||||
## Preparation, capacity, or execution fails
|
|
||||||
|
|
||||||
A preparation failure occurs before provider work; an execution failure occurs
|
|
||||||
after preparation. A capacity error for one batch report does not retry that
|
|
||||||
report or prevent later independent reports. Correct the profile or backend
|
|
||||||
condition identified by the bounded command error, then create a new run.
|
|
||||||
Use explicit secure debug capture only when additional content-rich diagnostics
|
|
||||||
are necessary. See [operations](operations.md).
|
|
||||||
|
|
||||||
## Generated text fails validation
|
|
||||||
|
|
||||||
Raw generated output may be saved but Markdown is not rendered when the JSON
|
|
||||||
does not match the report schema. Correct the Promptkit prompt/profile behavior
|
|
||||||
or the matching schema and validator in source control; do not edit raw output
|
|
||||||
to treat it as validated. See [templates](templates.md).
|
|
||||||
|
|
||||||
## Debug capture fails
|
|
||||||
|
|
||||||
`--llm-debug-dir` must be an absolute secure directory outside workspace state.
|
|
||||||
A debug-write failure stops the affected report to avoid continuing without the
|
|
||||||
requested diagnostic. Repair the named path's ownership or permissions, then
|
|
||||||
rerun. Treat capture files as sensitive. See [operations](operations.md).
|
|
||||||
|
|
||||||
## Weather, state, output, or notification fails
|
|
||||||
|
|
||||||
Collection errors precede planning. Later filesystem, output-copy, template,
|
|
||||||
or Distributor errors retain the reached safe paths in the summary. Repair only
|
|
||||||
the reported endpoint or path, leave successful managed reports intact, and
|
|
||||||
rerun the affected report or batch. A batch notification is intentionally
|
|
||||||
skipped when any report item fails.
|
|
||||||
|
|
||||||
## Secrets cannot be loaded
|
|
||||||
|
|
||||||
Secret files must be regular non-symlink files directly beneath
|
|
||||||
`secrets.directory` with valid environment-variable basenames. Correct the
|
|
||||||
reported file or directory without placing secret values in YAML.
|
|
||||||
@@ -40,14 +40,6 @@ promptkit:
|
|||||||
local:
|
local:
|
||||||
concurrency_limit: 1
|
concurrency_limit: 1
|
||||||
|
|
||||||
workspace:
|
|
||||||
root: workspace
|
|
||||||
snapshots_dir: snapshots
|
|
||||||
reports_dir: reports
|
|
||||||
data_packages_dir: data-packages
|
|
||||||
preflight_dir: preflight
|
|
||||||
notifications_dir: notifications
|
|
||||||
|
|
||||||
dayparts:
|
dayparts:
|
||||||
- name: overnight
|
- name: overnight
|
||||||
start: "00:00"
|
start: "00:00"
|
||||||
@@ -65,12 +57,6 @@ dayparts:
|
|||||||
start: "17:00"
|
start: "17:00"
|
||||||
end: "24:00"
|
end: "24:00"
|
||||||
|
|
||||||
recent_change:
|
|
||||||
temperature_degrees: 5
|
|
||||||
precip_probability_points: 20
|
|
||||||
wind_gust_miles_per_hour: 10
|
|
||||||
precip_timing_shift_minutes: 120
|
|
||||||
|
|
||||||
reports:
|
reports:
|
||||||
daily:
|
daily:
|
||||||
distributor:
|
distributor:
|
||||||
|
|||||||
@@ -124,9 +124,7 @@ func (adapter *Adapter) Execute(ctx context.Context, request promptexec.ExecuteR
|
|||||||
PromptID: request.PromptID,
|
PromptID: request.PromptID,
|
||||||
PromptVersion: request.PromptVersion,
|
PromptVersion: request.PromptVersion,
|
||||||
ProfileID: request.ProfileID,
|
ProfileID: request.ProfileID,
|
||||||
Inputs: map[string]promptkit.ArtifactRef{
|
Inputs: map[string]promptkit.ArtifactRef{"data_package": promptkit.Inline(string(append([]byte(nil), request.DataPackage...)))},
|
||||||
"data_package": promptkit.InlineWithURI(request.DataPackagePath, string(append([]byte(nil), request.DataPackage...))),
|
|
||||||
},
|
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, classifyError(err)
|
return nil, classifyError(err)
|
||||||
@@ -134,7 +132,7 @@ func (adapter *Adapter) Execute(ctx context.Context, request promptexec.ExecuteR
|
|||||||
defer prepared.Discard()
|
defer prepared.Discard()
|
||||||
|
|
||||||
details := prepared.Details()
|
details := prepared.Details()
|
||||||
preparation, debug := preparationValues(details, request.DataPackagePath, request.CaptureDebug)
|
preparation, debug := preparationValues(details, request.CaptureDebug)
|
||||||
if preparedCallback != nil {
|
if preparedCallback != nil {
|
||||||
if err := preparedCallback(preparation, debug); err != nil {
|
if err := preparedCallback(preparation, debug); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -145,7 +143,7 @@ func (adapter *Adapter) Execute(ctx context.Context, request promptexec.ExecuteR
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, classifyError(err)
|
return nil, classifyError(err)
|
||||||
}
|
}
|
||||||
return executionValue(result, request.DataPackagePath, request.CaptureDebug), nil
|
return executionValue(result, request.CaptureDebug), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func outputContract(value promptkit.OutputContract) promptexec.OutputContract {
|
func outputContract(value promptkit.OutputContract) promptexec.OutputContract {
|
||||||
@@ -156,7 +154,7 @@ func outputContract(value promptkit.OutputContract) promptexec.OutputContract {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func preparationValues(value promptkit.PreparedRun, dataPackagePath string, captureDebug bool) (promptexec.Preparation, *promptexec.PreparationDebug) {
|
func preparationValues(value promptkit.PreparedRun, captureDebug bool) (promptexec.Preparation, *promptexec.PreparationDebug) {
|
||||||
preparation := promptexec.Preparation{
|
preparation := promptexec.Preparation{
|
||||||
PromptID: value.PromptID,
|
PromptID: value.PromptID,
|
||||||
PromptVersion: value.PromptVersion,
|
PromptVersion: value.PromptVersion,
|
||||||
@@ -170,7 +168,6 @@ func preparationValues(value promptkit.PreparedRun, dataPackagePath string, capt
|
|||||||
StartedAt: value.StartTime,
|
StartedAt: value.StartTime,
|
||||||
EndedAt: value.EndTime,
|
EndedAt: value.EndTime,
|
||||||
Duration: time.Duration(value.DurationMS) * time.Millisecond,
|
Duration: time.Duration(value.DurationMS) * time.Millisecond,
|
||||||
DataPackagePath: dataPackagePath,
|
|
||||||
}
|
}
|
||||||
if !captureDebug {
|
if !captureDebug {
|
||||||
return preparation, nil
|
return preparation, nil
|
||||||
@@ -186,7 +183,7 @@ func preparationValues(value promptkit.PreparedRun, dataPackagePath string, capt
|
|||||||
return preparation, debug
|
return preparation, debug
|
||||||
}
|
}
|
||||||
|
|
||||||
func executionValue(value *promptkit.RunResult, dataPackagePath string, captureDebug bool) *promptexec.Execution {
|
func executionValue(value *promptkit.RunResult, captureDebug bool) *promptexec.Execution {
|
||||||
if value == nil {
|
if value == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -218,7 +215,6 @@ func executionValue(value *promptkit.RunResult, dataPackagePath string, captureD
|
|||||||
EndedAt: value.EndTime,
|
EndedAt: value.EndTime,
|
||||||
Duration: value.Duration,
|
Duration: value.Duration,
|
||||||
Validation: validation,
|
Validation: validation,
|
||||||
DataPackagePath: dataPackagePath,
|
|
||||||
RawOutput: []byte(value.RawOutput),
|
RawOutput: []byte(value.RawOutput),
|
||||||
}
|
}
|
||||||
if captureDebug {
|
if captureDebug {
|
||||||
|
|||||||
@@ -68,11 +68,11 @@ func (client *fakeClient) request() promptkit.GenerateRequest {
|
|||||||
|
|
||||||
func TestInspectPromptAndProfile(t *testing.T) {
|
func TestInspectPromptAndProfile(t *testing.T) {
|
||||||
adapter := newTestAdapter(t, &fakeClient{})
|
adapter := newTestAdapter(t, &fakeClient{})
|
||||||
inspection, err := adapter.InspectPrompt(context.Background(), "weather.daily_generated_text", "1.1.0")
|
inspection, err := adapter.InspectPrompt(context.Background(), "weather.daily_generated_text", "2.0.0")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("InspectPrompt() error = %v", err)
|
t.Fatalf("InspectPrompt() error = %v", err)
|
||||||
}
|
}
|
||||||
if inspection.PromptID != "weather.daily_generated_text" || inspection.PromptVersion != "1.1.0" || inspection.DefaultProfileID != "weather-balanced" {
|
if inspection.PromptID != "weather.daily_generated_text" || inspection.PromptVersion != "2.0.0" || inspection.DefaultProfileID != "weather-balanced" {
|
||||||
t.Fatalf("inspection = %#v", inspection)
|
t.Fatalf("inspection = %#v", inspection)
|
||||||
}
|
}
|
||||||
if len(inspection.Inputs) != 1 || inspection.Inputs[0].Name != "data_package" || !inspection.Inputs[0].Required || inspection.Inputs[0].ContentType != "application/yaml" {
|
if len(inspection.Inputs) != 1 || inspection.Inputs[0].Name != "data_package" || !inspection.Inputs[0].Required || inspection.Inputs[0].ContentType != "application/yaml" {
|
||||||
@@ -210,7 +210,7 @@ func TestExecuteUsesPreparedInlineDataPackage(t *testing.T) {
|
|||||||
callbackCalls := 0
|
callbackCalls := 0
|
||||||
result, err := adapter.Execute(context.Background(), request, func(preparation promptexec.Preparation, debug *promptexec.PreparationDebug) error {
|
result, err := adapter.Execute(context.Background(), request, func(preparation promptexec.Preparation, debug *promptexec.PreparationDebug) error {
|
||||||
callbackCalls++
|
callbackCalls++
|
||||||
if preparation.PromptID != request.PromptID || preparation.PromptVersion != request.PromptVersion || preparation.DataPackagePath != request.DataPackagePath || preparation.ModelName != "test-model" {
|
if preparation.PromptID != request.PromptID || preparation.PromptVersion != request.PromptVersion || preparation.ModelName != "test-model" {
|
||||||
t.Fatalf("preparation = %#v", preparation)
|
t.Fatalf("preparation = %#v", preparation)
|
||||||
}
|
}
|
||||||
if debug != nil {
|
if debug != nil {
|
||||||
@@ -227,7 +227,7 @@ func TestExecuteUsesPreparedInlineDataPackage(t *testing.T) {
|
|||||||
if callbackCalls != 1 || client.callCount() != 1 {
|
if callbackCalls != 1 || client.callCount() != 1 {
|
||||||
t.Fatalf("callback/provider calls = %d/%d, want 1/1", callbackCalls, client.callCount())
|
t.Fatalf("callback/provider calls = %d/%d, want 1/1", callbackCalls, client.callCount())
|
||||||
}
|
}
|
||||||
if result == nil || result.Validation.Status != promptexec.ValidationPassed || string(result.RawOutput) != client.response.Content || result.DataPackagePath != request.DataPackagePath {
|
if result == nil || result.Validation.Status != promptexec.ValidationPassed || string(result.RawOutput) != client.response.Content {
|
||||||
t.Fatalf("result = %#v", result)
|
t.Fatalf("result = %#v", result)
|
||||||
}
|
}
|
||||||
if result.Debug != nil {
|
if result.Debug != nil {
|
||||||
@@ -251,10 +251,9 @@ func TestExecuteEmbeddedHourlyProfileThroughPreparedPath(t *testing.T) {
|
|||||||
}
|
}
|
||||||
request := promptexec.ExecuteRequest{
|
request := promptexec.ExecuteRequest{
|
||||||
PromptID: "weather.hourly_generated_text",
|
PromptID: "weather.hourly_generated_text",
|
||||||
PromptVersion: "1.1.0",
|
PromptVersion: "2.0.0",
|
||||||
ProfileID: "weather-light",
|
ProfileID: "weather-light",
|
||||||
DataPackage: []byte("report:\n id: hourly\nbriefing: {}\n"),
|
DataPackage: []byte("report:\n id: hourly\nbriefing: {}\n"),
|
||||||
DataPackagePath: "data-packages/hourly/data_package.yaml",
|
|
||||||
}
|
}
|
||||||
var preparation promptexec.Preparation
|
var preparation promptexec.Preparation
|
||||||
prepared := false
|
prepared := false
|
||||||
@@ -288,7 +287,7 @@ func TestExecuteUsesExactInlineDataPackageProvenance(t *testing.T) {
|
|||||||
if _, err := adapter.Execute(context.Background(), request, nil); err != nil {
|
if _, err := adapter.Execute(context.Background(), request, nil); err != nil {
|
||||||
t.Fatalf("Execute() error = %v", err)
|
t.Fatalf("Execute() error = %v", err)
|
||||||
}
|
}
|
||||||
if reader.ref.Type != promptkit.ArtifactRefInline || reader.ref.URI != request.DataPackagePath || reader.ref.Body != string(request.DataPackage) {
|
if reader.ref.Type != promptkit.ArtifactRefInline || reader.ref.URI != "" || reader.ref.Body != string(request.DataPackage) {
|
||||||
t.Fatalf("artifact ref = %#v, want exact inline data package provenance", reader.ref)
|
t.Fatalf("artifact ref = %#v, want exact inline data package provenance", reader.ref)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -528,10 +527,9 @@ func writeProfileFile(t *testing.T, profile string) string {
|
|||||||
func testExecuteRequest() promptexec.ExecuteRequest {
|
func testExecuteRequest() promptexec.ExecuteRequest {
|
||||||
return promptexec.ExecuteRequest{
|
return promptexec.ExecuteRequest{
|
||||||
PromptID: "weather.daily_generated_text",
|
PromptID: "weather.daily_generated_text",
|
||||||
PromptVersion: "1.1.0",
|
PromptVersion: "2.0.0",
|
||||||
ProfileID: "test-profile",
|
ProfileID: "test-profile",
|
||||||
DataPackage: []byte("report:\n id: daily\nbriefing: {}\n"),
|
DataPackage: []byte("report:\n id: daily\nbriefing: {}\n"),
|
||||||
DataPackagePath: "data-packages/daily/data_package.yaml",
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -4,22 +4,22 @@ package app
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
distributoradapter "gitea.maximumdirect.net/eric/weatherreporter/internal/adapters/distributor"
|
distributoradapter "gitea.maximumdirect.net/eric/weatherreporter/internal/adapters/distributor"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/changes"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/collect"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/collect"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/facts"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/facts"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/fileutil"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/forecast"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/forecast"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptdebug"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
||||||
)
|
)
|
||||||
@@ -43,6 +43,7 @@ const (
|
|||||||
type GenerateRequest struct {
|
type GenerateRequest struct {
|
||||||
Config config.Config
|
Config config.Config
|
||||||
Report ReportKind
|
Report ReportKind
|
||||||
|
WorkingDir string
|
||||||
OutputPath string
|
OutputPath string
|
||||||
LLMDebugDir string
|
LLMDebugDir string
|
||||||
Now time.Time
|
Now time.Time
|
||||||
@@ -50,26 +51,20 @@ type GenerateRequest struct {
|
|||||||
Collector Collector
|
Collector Collector
|
||||||
Notifier Notifier
|
Notifier Notifier
|
||||||
Executor promptexec.Executor
|
Executor promptexec.Executor
|
||||||
Store state.Store
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type BatchRequest struct {
|
type BatchRequest struct {
|
||||||
Config config.Config
|
Config config.Config
|
||||||
Batch BatchKind
|
Batch BatchKind
|
||||||
Now time.Time
|
Now time.Time
|
||||||
|
WorkingDir string
|
||||||
OutputDir string
|
OutputDir string
|
||||||
LLMDebugDir string
|
LLMDebugDir string
|
||||||
Collector Collector
|
Collector Collector
|
||||||
Executor promptexec.Executor
|
Executor promptexec.Executor
|
||||||
Store state.Store
|
|
||||||
Notifier Notifier
|
Notifier Notifier
|
||||||
}
|
}
|
||||||
|
|
||||||
type FetchBundleRequest struct {
|
|
||||||
Config config.Config
|
|
||||||
OutputPath string
|
|
||||||
}
|
|
||||||
|
|
||||||
type ModuleSnapshotRequest struct {
|
type ModuleSnapshotRequest struct {
|
||||||
Config config.Config
|
Config config.Config
|
||||||
Resolved report.Resolved
|
Resolved report.Resolved
|
||||||
@@ -81,23 +76,21 @@ type ReportFacts struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
type ReportResult struct {
|
type ReportResult struct {
|
||||||
ModuleSnapshot module.Snapshot
|
ReportID report.ID
|
||||||
ModuleSnapshotPath string
|
ReportName string
|
||||||
DataPackage promptinput.Package
|
PromptID string
|
||||||
DataPackagePath string
|
PromptVersion string
|
||||||
PreparationPath string
|
RunID string
|
||||||
ExecutionPath string
|
GeneratedAt time.Time
|
||||||
|
Timezone string
|
||||||
|
ValidPeriod timeutil.Period
|
||||||
|
ProfileID string
|
||||||
|
BackendID string
|
||||||
|
ModelName string
|
||||||
|
SourceWarnings []weatherdata.SourceWarning
|
||||||
|
ValidationStatus promptexec.ValidationStatus
|
||||||
LLMDebugPath string
|
LLMDebugPath string
|
||||||
ReportPath string
|
|
||||||
OutputPath string
|
OutputPath string
|
||||||
NotificationPath string
|
|
||||||
Metadata state.Metadata
|
|
||||||
MetadataPath string
|
|
||||||
PriorSnapshot *state.PriorSnapshot
|
|
||||||
RecentChanges []changes.Change
|
|
||||||
GeneratedTextRawPath string
|
|
||||||
GeneratedTextPath string
|
|
||||||
RenderContextPath string
|
|
||||||
Notification *NotificationResult
|
Notification *NotificationResult
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -119,7 +112,6 @@ type BatchNotificationResult struct {
|
|||||||
PipelineID string `json:"pipelineId,omitempty"`
|
PipelineID string `json:"pipelineId,omitempty"`
|
||||||
BundleID string `json:"bundleId,omitempty"`
|
BundleID string `json:"bundleId,omitempty"`
|
||||||
IdempotencyKey string `json:"idempotencyKey,omitempty"`
|
IdempotencyKey string `json:"idempotencyKey,omitempty"`
|
||||||
Path string `json:"path,omitempty"`
|
|
||||||
IncludedReports []BatchNotificationReport `json:"includedReports,omitempty"`
|
IncludedReports []BatchNotificationReport `json:"includedReports,omitempty"`
|
||||||
Error string `json:"error,omitempty"`
|
Error string `json:"error,omitempty"`
|
||||||
}
|
}
|
||||||
@@ -138,20 +130,16 @@ type BatchReportResult struct {
|
|||||||
RunID string `json:"runId"`
|
RunID string `json:"runId"`
|
||||||
Status string `json:"status"`
|
Status string `json:"status"`
|
||||||
Error string `json:"error,omitempty"`
|
Error string `json:"error,omitempty"`
|
||||||
NotificationStatus string `json:"notificationStatus,omitempty"`
|
|
||||||
NotificationRunID string `json:"notificationRunId,omitempty"`
|
|
||||||
NotificationPipelineID string `json:"notificationPipelineId,omitempty"`
|
|
||||||
NotificationError string `json:"notificationError,omitempty"`
|
|
||||||
NotificationPath string `json:"notificationPath,omitempty"`
|
|
||||||
GeneratedAt time.Time `json:"generatedAt"`
|
GeneratedAt time.Time `json:"generatedAt"`
|
||||||
ValidPeriod timeutil.Period `json:"validPeriod"`
|
ValidPeriod timeutil.Period `json:"validPeriod"`
|
||||||
DataPackagePath string `json:"dataPackagePath,omitempty"`
|
Timezone string `json:"timezone"`
|
||||||
PreparationPath string `json:"preparationPath,omitempty"`
|
ProfileID string `json:"profileId,omitempty"`
|
||||||
ExecutionPath string `json:"executionPath,omitempty"`
|
BackendID string `json:"backendId,omitempty"`
|
||||||
|
ModelName string `json:"modelName,omitempty"`
|
||||||
|
SourceWarnings []weatherdata.SourceWarning `json:"sourceWarnings,omitempty"`
|
||||||
|
ValidationStatus promptexec.ValidationStatus `json:"validationStatus,omitempty"`
|
||||||
LLMDebugPath string `json:"llmDebugPath,omitempty"`
|
LLMDebugPath string `json:"llmDebugPath,omitempty"`
|
||||||
ReportPath string `json:"reportPath,omitempty"`
|
|
||||||
OutputPath string `json:"outputPath,omitempty"`
|
OutputPath string `json:"outputPath,omitempty"`
|
||||||
MetadataPath string `json:"metadataPath,omitempty"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type BatchError struct {
|
type BatchError struct {
|
||||||
@@ -162,13 +150,14 @@ func (e BatchError) Error() string {
|
|||||||
if e.Result == nil {
|
if e.Result == nil {
|
||||||
return "batch failed"
|
return "batch failed"
|
||||||
}
|
}
|
||||||
if batchNotificationFailed(e.Result) && batchReportFailures(e.Result) == 0 {
|
failedReports := batchReportFailures(e.Result)
|
||||||
|
if batchNotificationFailed(e.Result) && failedReports == 0 {
|
||||||
if e.Result.Notification.Error != "" {
|
if e.Result.Notification.Error != "" {
|
||||||
return fmt.Sprintf("batch %s notification failed: %s", e.Result.Batch, e.Result.Notification.Error)
|
return fmt.Sprintf("batch %s notification failed: %s", e.Result.Batch, e.Result.Notification.Error)
|
||||||
}
|
}
|
||||||
return fmt.Sprintf("batch %s notification failed", e.Result.Batch)
|
return fmt.Sprintf("batch %s notification failed", e.Result.Batch)
|
||||||
}
|
}
|
||||||
return fmt.Sprintf("batch %s failed: %d of %d reports failed", e.Result.Batch, e.Result.Failed, e.Result.Total)
|
return fmt.Sprintf("batch %s failed: %d of %d reports failed", e.Result.Batch, failedReports, len(e.Result.Reports))
|
||||||
}
|
}
|
||||||
|
|
||||||
func batchNotificationFailed(result *BatchResult) bool {
|
func batchNotificationFailed(result *BatchResult) bool {
|
||||||
@@ -261,9 +250,15 @@ func GenerateDetailed(ctx context.Context, req GenerateRequest) (*ReportResult,
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
debugWriter, err := state.NewPromptDebugWriter(req.LLMDebugDir)
|
result := initialReportResult(req, resolved, PromptInspectionResult{})
|
||||||
|
outputPath, err := resolveReportOutputPath(req.WorkingDir, req.OutputPath, resolved)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, promptexec.NewError(promptexec.InvalidConfiguration, "initialize prompt debug", err)
|
return result, err
|
||||||
|
}
|
||||||
|
req.OutputPath = outputPath
|
||||||
|
debugWriter, err := promptdebug.NewPromptDebugWriter(req.LLMDebugDir)
|
||||||
|
if err != nil {
|
||||||
|
return result, promptexec.NewError(promptexec.InvalidConfiguration, "initialize prompt debug", err)
|
||||||
}
|
}
|
||||||
inspection, err := InspectPromptExecution(ctx, PromptInspectionRequest{
|
inspection, err := InspectPromptExecution(ctx, PromptInspectionRequest{
|
||||||
Resolved: resolved,
|
Resolved: resolved,
|
||||||
@@ -271,11 +266,12 @@ func GenerateDetailed(ctx context.Context, req GenerateRequest) (*ReportResult,
|
|||||||
Promptkit: req.Config.Promptkit,
|
Promptkit: req.Config.Promptkit,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return result, err
|
||||||
}
|
}
|
||||||
|
result.ProfileID, result.BackendID, result.ModelName = inspection.ProfileID, inspection.BackendID, inspection.ModelName
|
||||||
collection, err := collectWeather(ctx, req.Config, req.Collector)
|
collection, err := collectWeather(ctx, req.Config, req.Collector)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return result, err
|
||||||
}
|
}
|
||||||
return generatePromptReport(ctx, promptReportRequest{
|
return generatePromptReport(ctx, promptReportRequest{
|
||||||
GenerateRequest: req,
|
GenerateRequest: req,
|
||||||
@@ -283,6 +279,7 @@ func GenerateDetailed(ctx context.Context, req GenerateRequest) (*ReportResult,
|
|||||||
Collection: *collection,
|
Collection: *collection,
|
||||||
Inspection: inspection,
|
Inspection: inspection,
|
||||||
DebugWriter: debugWriter,
|
DebugWriter: debugWriter,
|
||||||
|
Result: result,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -291,7 +288,7 @@ func RunBatch(ctx context.Context, req BatchRequest) error {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if result.Failed > 0 {
|
if result.Failed > 0 || batchNotificationFailed(result) {
|
||||||
return BatchError{Result: result}
|
return BatchError{Result: result}
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
@@ -305,7 +302,12 @@ func RunBatchDetailed(ctx context.Context, req BatchRequest) (*BatchResult, erro
|
|||||||
if _, err := report.BatchForCommandName(string(req.Batch)); err != nil {
|
if _, err := report.BatchForCommandName(string(req.Batch)); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
debugWriter, err := state.NewPromptDebugWriter(req.LLMDebugDir)
|
outputDir, err := resolveOutputDir(req.WorkingDir, req.OutputDir)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
req.OutputDir = outputDir
|
||||||
|
debugWriter, err := promptdebug.NewPromptDebugWriter(req.LLMDebugDir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, promptexec.NewError(promptexec.InvalidConfiguration, "initialize prompt debug", err)
|
return nil, promptexec.NewError(promptexec.InvalidConfiguration, "initialize prompt debug", err)
|
||||||
}
|
}
|
||||||
@@ -329,28 +331,21 @@ func RunBatchDetailed(ctx context.Context, req BatchRequest) (*BatchResult, erro
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if req.Batch == BatchEvening || req.Batch == BatchMorning {
|
if err := prepareBatchOutputs(req.OutputDir, plannedReports); err != nil {
|
||||||
store := req.Store
|
|
||||||
if store == nil {
|
|
||||||
defaultStore, err := defaultStore(req.Config)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
store = defaultStore
|
if req.Batch == BatchEvening || req.Batch == BatchMorning {
|
||||||
}
|
|
||||||
startedAt := now
|
startedAt := now
|
||||||
result := &BatchResult{Batch: req.Batch, StartedAt: startedAt}
|
result := &BatchResult{Batch: req.Batch, StartedAt: startedAt}
|
||||||
for _, planned := range plannedReports {
|
for _, planned := range plannedReports {
|
||||||
resolved := planned.Resolved
|
resolved := planned.Resolved
|
||||||
item := batchReportResult(planned)
|
item := batchReportResult(planned)
|
||||||
outputPath := plannedBatchOutputPath(req.OutputDir, planned)
|
|
||||||
reportResult, err := generatePromptReport(ctx, promptReportRequest{
|
reportResult, err := generatePromptReport(ctx, promptReportRequest{
|
||||||
GenerateRequest: GenerateRequest{
|
GenerateRequest: GenerateRequest{
|
||||||
Config: req.Config,
|
Config: req.Config,
|
||||||
OutputPath: outputPath,
|
OutputPath: planned.OutputPath,
|
||||||
Notifier: req.Notifier,
|
Notifier: req.Notifier,
|
||||||
Executor: req.Executor,
|
Executor: req.Executor,
|
||||||
Store: store,
|
|
||||||
},
|
},
|
||||||
Resolved: resolved,
|
Resolved: resolved,
|
||||||
Collection: *collection,
|
Collection: *collection,
|
||||||
@@ -359,7 +354,7 @@ func RunBatchDetailed(ctx context.Context, req BatchRequest) (*BatchResult, erro
|
|||||||
noNotify: true,
|
noNotify: true,
|
||||||
})
|
})
|
||||||
if reportResult != nil {
|
if reportResult != nil {
|
||||||
copyBatchReportPaths(&item, reportResult)
|
copyBatchReportDetails(&item, reportResult)
|
||||||
}
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
item.Status = "failed"
|
item.Status = "failed"
|
||||||
@@ -372,33 +367,25 @@ func RunBatchDetailed(ctx context.Context, req BatchRequest) (*BatchResult, erro
|
|||||||
result.Reports = append(result.Reports, item)
|
result.Reports = append(result.Reports, item)
|
||||||
}
|
}
|
||||||
result.Total = len(result.Reports)
|
result.Total = len(result.Reports)
|
||||||
batchNotification, err := notifyBatch(ctx, req.Config, req.Batch, batchRunID(startedAt, req.Batch), startedAt, result, plannedReports, store, req.Notifier)
|
batchNotification := notifyBatch(ctx, req.Config, req.Batch, batchRunID(startedAt, req.Batch), startedAt, result, plannedReports, req.Notifier)
|
||||||
if batchNotification != nil {
|
if batchNotification != nil {
|
||||||
result.Notification = batchNotification
|
result.Notification = batchNotification
|
||||||
}
|
}
|
||||||
if err != nil {
|
|
||||||
result.Failed++
|
|
||||||
}
|
|
||||||
result.FinishedAt = time.Now()
|
result.FinishedAt = time.Now()
|
||||||
return result, nil
|
return result, nil
|
||||||
}
|
}
|
||||||
return nil, fmt.Errorf("run is not implemented")
|
return nil, fmt.Errorf("run is not implemented")
|
||||||
}
|
}
|
||||||
|
|
||||||
func copyBatchReportPaths(item *BatchReportResult, result *ReportResult) {
|
func copyBatchReportDetails(item *BatchReportResult, result *ReportResult) {
|
||||||
item.DataPackagePath = result.DataPackagePath
|
|
||||||
item.PreparationPath = result.PreparationPath
|
|
||||||
item.ExecutionPath = result.ExecutionPath
|
|
||||||
item.LLMDebugPath = result.LLMDebugPath
|
item.LLMDebugPath = result.LLMDebugPath
|
||||||
item.ReportPath = result.ReportPath
|
|
||||||
item.OutputPath = result.OutputPath
|
item.OutputPath = result.OutputPath
|
||||||
item.MetadataPath = result.MetadataPath
|
item.ProfileID = result.ProfileID
|
||||||
item.NotificationPath = result.NotificationPath
|
item.BackendID = result.BackendID
|
||||||
if result.Notification != nil {
|
item.ModelName = result.ModelName
|
||||||
item.NotificationStatus = result.Notification.Status
|
item.Timezone = result.Timezone
|
||||||
item.NotificationRunID = result.Notification.RunID
|
item.SourceWarnings = append([]weatherdata.SourceWarning(nil), result.SourceWarnings...)
|
||||||
item.NotificationPipelineID = result.Notification.PipelineID
|
item.ValidationStatus = result.ValidationStatus
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func batchInspectionCandidates(req BatchRequest, now time.Time) ([]report.Resolved, error) {
|
func batchInspectionCandidates(req BatchRequest, now time.Time) ([]report.Resolved, error) {
|
||||||
@@ -440,21 +427,106 @@ func batchReportResult(planned plannedBatchReport) BatchReportResult {
|
|||||||
RunID: metadata.RunID,
|
RunID: metadata.RunID,
|
||||||
GeneratedAt: metadata.GeneratedAt,
|
GeneratedAt: metadata.GeneratedAt,
|
||||||
ValidPeriod: metadata.ValidPeriod,
|
ValidPeriod: metadata.ValidPeriod,
|
||||||
|
Timezone: "",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func plannedBatchOutputPath(outputDir string, planned plannedBatchReport) string {
|
func plannedBatchOutputPath(outputDir string, planned plannedBatchReport) (string, error) {
|
||||||
if outputDir == "" {
|
outputName, err := planned.Resolved.OutputName()
|
||||||
return ""
|
if err != nil {
|
||||||
|
return "", err
|
||||||
}
|
}
|
||||||
outputCopyName := planned.OutputCopyName
|
return validateOutputPath(filepath.Join(outputDir, outputName))
|
||||||
if outputCopyName == "" {
|
|
||||||
outputCopyName = planned.Resolved.Definition.BatchOutputName
|
|
||||||
}
|
}
|
||||||
if outputCopyName == "" {
|
|
||||||
return ""
|
func prepareBatchOutputs(outputDir string, plannedReports []plannedBatchReport) error {
|
||||||
|
for index := range plannedReports {
|
||||||
|
outputPath, err := plannedBatchOutputPath(outputDir, plannedReports[index])
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
}
|
}
|
||||||
return filepath.Join(outputDir, outputCopyName)
|
plannedReports[index].OutputPath = outputPath
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveReportOutputPath(workingDir, override string, resolved report.Resolved) (string, error) {
|
||||||
|
outputName, err := resolved.OutputName()
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return resolveOutputPath(workingDir, override, outputName)
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveOutputDir(workingDir, override string) (string, error) {
|
||||||
|
workingDir, err := validateWorkingDir(workingDir)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if override == "" {
|
||||||
|
return workingDir, nil
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(override) == "" {
|
||||||
|
return "", fmt.Errorf("output directory is required")
|
||||||
|
}
|
||||||
|
directory := override
|
||||||
|
if !filepath.IsAbs(directory) {
|
||||||
|
directory = filepath.Join(workingDir, directory)
|
||||||
|
}
|
||||||
|
directory = filepath.Clean(directory)
|
||||||
|
if info, err := os.Stat(directory); err == nil && !info.IsDir() {
|
||||||
|
return "", fmt.Errorf("output directory %q is not a directory", directory)
|
||||||
|
} else if err != nil && !os.IsNotExist(err) {
|
||||||
|
return "", fmt.Errorf("inspect output directory %q: %w", directory, err)
|
||||||
|
}
|
||||||
|
return directory, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveOutputPath(workingDir, override, defaultName string) (string, error) {
|
||||||
|
workingDir, err := validateWorkingDir(workingDir)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
path := override
|
||||||
|
if path == "" {
|
||||||
|
path = defaultName
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(path) == "" {
|
||||||
|
return "", fmt.Errorf("final output path is required")
|
||||||
|
}
|
||||||
|
if !filepath.IsAbs(path) {
|
||||||
|
path = filepath.Join(workingDir, path)
|
||||||
|
}
|
||||||
|
return validateOutputPath(path)
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateWorkingDir(workingDir string) (string, error) {
|
||||||
|
if strings.TrimSpace(workingDir) == "" {
|
||||||
|
return "", fmt.Errorf("working directory is required")
|
||||||
|
}
|
||||||
|
if !filepath.IsAbs(workingDir) {
|
||||||
|
return "", fmt.Errorf("working directory %q must be absolute", workingDir)
|
||||||
|
}
|
||||||
|
return filepath.Clean(workingDir), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateOutputPath(path string) (string, error) {
|
||||||
|
if strings.TrimSpace(path) == "" {
|
||||||
|
return "", fmt.Errorf("final output path is required")
|
||||||
|
}
|
||||||
|
path = filepath.Clean(path)
|
||||||
|
if !filepath.IsAbs(path) {
|
||||||
|
return "", fmt.Errorf("final output path %q must be absolute", path)
|
||||||
|
}
|
||||||
|
if filepath.Dir(path) == path {
|
||||||
|
return "", fmt.Errorf("final output path %q must not be a filesystem root", path)
|
||||||
|
}
|
||||||
|
if info, err := os.Stat(path); err == nil && info.IsDir() {
|
||||||
|
return "", fmt.Errorf("final output path %q is a directory", path)
|
||||||
|
} else if err != nil && !os.IsNotExist(err) {
|
||||||
|
return "", fmt.Errorf("inspect final output path %q: %w", path, err)
|
||||||
|
}
|
||||||
|
return path, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func ResolveGenerate(req GenerateRequest, now time.Time) (report.Resolved, error) {
|
func ResolveGenerate(req GenerateRequest, now time.Time) (report.Resolved, error) {
|
||||||
@@ -489,14 +561,6 @@ func reportRegistry(cfg config.Config) (report.Registry, error) {
|
|||||||
return registry, nil
|
return registry, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func FetchBundle(ctx context.Context, req FetchBundleRequest) (*weatherdata.Bundle, error) {
|
|
||||||
result, err := collectWeather(ctx, req.Config, nil)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
return result.Bundle, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func collectWeather(ctx context.Context, cfg config.Config, collector Collector) (*collect.Result, error) {
|
func collectWeather(ctx context.Context, cfg config.Config, collector Collector) (*collect.Result, error) {
|
||||||
if collector == nil {
|
if collector == nil {
|
||||||
collector = defaultCollector{}
|
collector = defaultCollector{}
|
||||||
@@ -514,135 +578,23 @@ func collectWeather(ctx context.Context, cfg config.Config, collector Collector)
|
|||||||
return result, nil
|
return result, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func FetchAndSaveBundle(ctx context.Context, req FetchBundleRequest) (*weatherdata.Bundle, error) {
|
func notifyReport(ctx context.Context, cfg config.Config, resolved report.Resolved, outputPath, runID string, generatedAt time.Time, notifier Notifier) (*NotificationResult, error) {
|
||||||
if req.OutputPath == "" {
|
notifier, enabled := reportNotifier(cfg, notifier)
|
||||||
return nil, fmt.Errorf("output path is required")
|
if !enabled {
|
||||||
|
return nil, nil
|
||||||
}
|
}
|
||||||
bundle, err := FetchBundle(ctx, req)
|
notificationRequest, err := buildNotificationRequest(cfg, resolved, outputPath, runID, generatedAt)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if err := fileutil.WriteJSONAtomic(req.OutputPath, bundle); err != nil {
|
|
||||||
return nil, fmt.Errorf("save bundle: %w", err)
|
|
||||||
}
|
|
||||||
return bundle, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type finalizeRenderedReportRequest struct {
|
|
||||||
Config config.Config
|
|
||||||
Store state.Store
|
|
||||||
Resolved report.Resolved
|
|
||||||
Metadata state.Metadata
|
|
||||||
MetadataPath string
|
|
||||||
ExecutionArtifact *state.PromptExecutionArtifact
|
|
||||||
ManagedReportPath string
|
|
||||||
OutputPath string
|
|
||||||
Notifier Notifier
|
|
||||||
GenerationErr error
|
|
||||||
noNotify bool
|
|
||||||
}
|
|
||||||
|
|
||||||
type finalizeRenderedReportResult struct {
|
|
||||||
OutputPath string
|
|
||||||
NotificationPath string
|
|
||||||
Metadata state.Metadata
|
|
||||||
MetadataPath string
|
|
||||||
Notification *NotificationResult
|
|
||||||
}
|
|
||||||
|
|
||||||
func finalizeRenderedReport(ctx context.Context, req finalizeRenderedReportRequest) (finalizeRenderedReportResult, error) {
|
|
||||||
if req.Store == nil {
|
|
||||||
return finalizeRenderedReportResult{}, fmt.Errorf("state store is required")
|
|
||||||
}
|
|
||||||
if req.ManagedReportPath == "" {
|
|
||||||
return finalizeRenderedReportResult{}, fmt.Errorf("managed report path is required for report %q", req.Resolved.Definition.ID)
|
|
||||||
}
|
|
||||||
if req.ExecutionArtifact == nil {
|
|
||||||
return finalizeRenderedReportResult{}, fmt.Errorf("prompt execution artifact is required for report %q", req.Resolved.Definition.ID)
|
|
||||||
}
|
|
||||||
|
|
||||||
result := finalizeRenderedReportResult{Metadata: req.Metadata, MetadataPath: req.MetadataPath}
|
|
||||||
if req.OutputPath != "" && req.GenerationErr == nil {
|
|
||||||
if req.OutputPath != req.ManagedReportPath {
|
|
||||||
if err := fileutil.CopyFileAtomic(req.ManagedReportPath, req.OutputPath); err != nil {
|
|
||||||
return result, err
|
|
||||||
}
|
|
||||||
result.OutputPath = req.OutputPath
|
|
||||||
if err := persistReachedPromptPath(ctx, req.Store, req.Resolved, req.ExecutionArtifact, func(paths *state.PromptExecutionPaths) {
|
|
||||||
paths.OutputPath = req.OutputPath
|
|
||||||
}); err != nil {
|
|
||||||
return result, err
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
result.OutputPath = req.OutputPath
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
metadata := req.Metadata
|
|
||||||
metadata.RenderedReportPath = req.ManagedReportPath
|
|
||||||
metadataPath, err := req.Store.SaveMetadata(ctx, metadata)
|
|
||||||
if err != nil {
|
|
||||||
return result, err
|
|
||||||
}
|
|
||||||
result.Metadata = metadata
|
|
||||||
result.MetadataPath = metadataPath
|
|
||||||
if req.GenerationErr != nil {
|
|
||||||
return result, req.GenerationErr
|
|
||||||
}
|
|
||||||
if req.noNotify {
|
|
||||||
return result, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
notification, notificationPath, err := notifyReport(ctx, req.Config, req.Resolved, req.ManagedReportPath, metadata, req.Notifier, req.Store)
|
|
||||||
if notificationPath != "" {
|
|
||||||
result.NotificationPath = notificationPath
|
|
||||||
result.Notification = notification
|
|
||||||
metadata.NotificationPath = notificationPath
|
|
||||||
result.Metadata = metadata
|
|
||||||
if saveErr := persistReachedPromptPath(ctx, req.Store, req.Resolved, req.ExecutionArtifact, func(paths *state.PromptExecutionPaths) {
|
|
||||||
paths.NotificationPath = notificationPath
|
|
||||||
}); saveErr != nil {
|
|
||||||
return result, saveErr
|
|
||||||
}
|
|
||||||
metadataPath, saveErr := req.Store.SaveMetadata(ctx, metadata)
|
|
||||||
if saveErr != nil {
|
|
||||||
return result, saveErr
|
|
||||||
}
|
|
||||||
result.Metadata = metadata
|
|
||||||
result.MetadataPath = metadataPath
|
|
||||||
}
|
|
||||||
result.Notification = notification
|
|
||||||
if err != nil {
|
|
||||||
return result, err
|
|
||||||
}
|
|
||||||
return result, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func notifyReport(ctx context.Context, cfg config.Config, resolved report.Resolved, reportPath string, metadata state.Metadata, notifier Notifier, store state.Store) (*NotificationResult, string, error) {
|
|
||||||
notifier, enabled := reportNotifier(cfg, notifier)
|
|
||||||
if !enabled {
|
|
||||||
return nil, "", nil
|
|
||||||
}
|
|
||||||
notificationRequest, err := buildNotificationRequest(cfg, resolved, reportPath, metadata)
|
|
||||||
if err != nil {
|
|
||||||
notificationPath, saveErr := saveNotificationArtifact(ctx, store, resolved, cfg, metadata, NotificationRequest{}, nil, err)
|
|
||||||
if saveErr != nil {
|
|
||||||
return nil, "", saveErr
|
|
||||||
}
|
|
||||||
return nil, notificationPath, err
|
|
||||||
}
|
|
||||||
result, err := notifier.Notify(ctx, notificationRequest)
|
result, err := notifier.Notify(ctx, notificationRequest)
|
||||||
notificationPath, saveErr := saveNotificationArtifact(ctx, store, resolved, cfg, metadata, notificationRequest, result, err)
|
|
||||||
if saveErr != nil {
|
|
||||||
return nil, "", saveErr
|
|
||||||
}
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return result, notificationPath, &NotificationError{
|
return result, &NotificationError{
|
||||||
Request: notificationRequest,
|
Request: notificationRequest,
|
||||||
Err: fmt.Errorf("notify report %q run %q from managed report %q: %w", resolved.Definition.ID, metadata.RunID, reportPath, err),
|
Err: fmt.Errorf("notify report %q run %q from output %q: %w", resolved.Definition.ID, runID, outputPath, err),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return result, notificationPath, nil
|
return result, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func reportNotifier(cfg config.Config, notifier Notifier) (Notifier, bool) {
|
func reportNotifier(cfg config.Config, notifier Notifier) (Notifier, bool) {
|
||||||
@@ -657,8 +609,8 @@ func reportNotifier(cfg config.Config, notifier Notifier) (Notifier, bool) {
|
|||||||
}, true
|
}, true
|
||||||
}
|
}
|
||||||
|
|
||||||
func buildNotificationRequest(cfg config.Config, resolved report.Resolved, reportPath string, metadata state.Metadata) (NotificationRequest, error) {
|
func buildNotificationRequest(cfg config.Config, resolved report.Resolved, outputPath, runID string, generatedAt time.Time) (NotificationRequest, error) {
|
||||||
values, err := distributorTemplateValuesForReport(cfg, resolved, metadata.RunID, resolved.Definition.BatchOutputName)
|
values, err := distributorTemplateValuesForReport(cfg, resolved, runID, filepath.Base(outputPath))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return NotificationRequest{}, err
|
return NotificationRequest{}, err
|
||||||
}
|
}
|
||||||
@@ -675,32 +627,36 @@ func buildNotificationRequest(cfg config.Config, resolved report.Resolved, repor
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return NotificationRequest{}, err
|
return NotificationRequest{}, err
|
||||||
}
|
}
|
||||||
bundlePaths, err := renderDistributorReportBundlePaths(cfg, resolved, metadata.RunID, reportPath, values)
|
bundlePaths, err := renderDistributorReportBundlePaths(cfg, resolved, runID, outputPath, values)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return NotificationRequest{}, err
|
return NotificationRequest{}, err
|
||||||
}
|
}
|
||||||
return NotificationRequest{
|
return NotificationRequest{
|
||||||
ReportID: resolved.Definition.ID,
|
ReportID: resolved.Definition.ID,
|
||||||
RunID: metadata.RunID,
|
RunID: runID,
|
||||||
PipelineID: pipelineID,
|
PipelineID: pipelineID,
|
||||||
BundleID: bundleID,
|
BundleID: bundleID,
|
||||||
IdempotencyKey: idempotencyKey,
|
IdempotencyKey: idempotencyKey,
|
||||||
ReportPath: reportPath,
|
ReportPath: outputPath,
|
||||||
BundlePaths: bundlePaths,
|
BundlePaths: bundlePaths,
|
||||||
CreatedAt: metadata.GeneratedAt,
|
CreatedAt: generatedAt,
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func distributorTemplateValuesForReport(cfg config.Config, resolved report.Resolved, runID string, batchOutputName string) (config.DistributorTemplateValues, error) {
|
func distributorTemplateValuesForReport(cfg config.Config, resolved report.Resolved, runID string, outputName string) (config.DistributorTemplateValues, error) {
|
||||||
values := config.DistributorTemplateValues{
|
values := config.DistributorTemplateValues{
|
||||||
LocationID: cfg.Location.ID,
|
LocationID: cfg.Location.ID,
|
||||||
ReportID: string(resolved.Definition.ID),
|
ReportID: string(resolved.Definition.ID),
|
||||||
RunID: runID,
|
RunID: runID,
|
||||||
ArtifactGroup: resolved.Definition.ArtifactGroup,
|
ArtifactGroup: resolved.Definition.ArtifactGroup,
|
||||||
BatchOutputName: batchOutputName,
|
BatchOutputName: outputName,
|
||||||
}
|
}
|
||||||
if values.BatchOutputName == "" {
|
if values.BatchOutputName == "" {
|
||||||
values.BatchOutputName = resolved.Definition.BatchOutputName
|
var err error
|
||||||
|
values.BatchOutputName, err = resolved.OutputName()
|
||||||
|
if err != nil {
|
||||||
|
return config.DistributorTemplateValues{}, err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if err := addDistributorValidPeriodValues(&values, resolved.ValidPeriod, cfg.WeatherAPI.Timezone); err != nil {
|
if err := addDistributorValidPeriodValues(&values, resolved.ValidPeriod, cfg.WeatherAPI.Timezone); err != nil {
|
||||||
return config.DistributorTemplateValues{}, err
|
return config.DistributorTemplateValues{}, err
|
||||||
@@ -757,54 +713,6 @@ func addDistributorValidPeriodValues(values *config.DistributorTemplateValues, p
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func saveNotificationArtifact(ctx context.Context, store state.Store, resolved report.Resolved, cfg config.Config, metadata state.Metadata, req NotificationRequest, result *NotificationResult, notifyErr error) (string, error) {
|
|
||||||
if store == nil {
|
|
||||||
return "", fmt.Errorf("state store is required")
|
|
||||||
}
|
|
||||||
artifact := state.DistributorNotificationArtifact{
|
|
||||||
SchemaVersion: state.DistributorNotificationSchemaVersion,
|
|
||||||
RunID: metadata.RunID,
|
|
||||||
ReportID: resolved.Definition.ID,
|
|
||||||
AttemptedAt: time.Now(),
|
|
||||||
Endpoint: cfg.Notify.Distributor.Endpoint,
|
|
||||||
PipelineID: req.PipelineID,
|
|
||||||
BundleID: req.BundleID,
|
|
||||||
IdempotencyKey: req.IdempotencyKey,
|
|
||||||
SourcePath: req.ReportPath,
|
|
||||||
BundlePaths: append([]string(nil), req.BundlePaths...),
|
|
||||||
BundleCreated: req.CreatedAt,
|
|
||||||
Status: "attempted",
|
|
||||||
}
|
|
||||||
if result != nil {
|
|
||||||
artifact.Status = result.Status
|
|
||||||
artifact.Upload = &state.DistributorUploadResult{
|
|
||||||
RunID: result.RunID,
|
|
||||||
Status: result.UploadStatus,
|
|
||||||
}
|
|
||||||
if result.PipelineID != "" || !result.AcceptedAt.IsZero() || result.StartedAt != nil || result.FinishedAt != nil || len(result.Report) > 0 || result.Error != "" {
|
|
||||||
artifact.RunStatus = &state.DistributorRunStatus{
|
|
||||||
RunID: result.RunID,
|
|
||||||
PipelineID: result.PipelineID,
|
|
||||||
Status: result.Status,
|
|
||||||
AcceptedAt: result.AcceptedAt,
|
|
||||||
StartedAt: result.StartedAt,
|
|
||||||
FinishedAt: result.FinishedAt,
|
|
||||||
Report: append([]byte(nil), result.Report...),
|
|
||||||
Error: result.Error,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
artifact.StatusError = result.StatusError
|
|
||||||
}
|
|
||||||
if notifyErr != nil {
|
|
||||||
artifact.Status = "failed"
|
|
||||||
artifact.Error = notifyErr.Error()
|
|
||||||
}
|
|
||||||
if artifact.Status == "" {
|
|
||||||
artifact.Status = "unknown"
|
|
||||||
}
|
|
||||||
return store.SaveDistributorNotification(ctx, resolved, artifact)
|
|
||||||
}
|
|
||||||
|
|
||||||
type noopNotifier struct{}
|
type noopNotifier struct{}
|
||||||
|
|
||||||
func (noopNotifier) Notify(context.Context, NotificationRequest) (*NotificationResult, error) {
|
func (noopNotifier) Notify(context.Context, NotificationRequest) (*NotificationResult, error) {
|
||||||
@@ -951,7 +859,7 @@ func briefingBuildContext(cfg config.Config, resolved report.Resolved, collected
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func promptMetadata(metadata state.Metadata) promptinput.Metadata {
|
func promptMetadata(metadata briefing.Metadata) promptinput.Metadata {
|
||||||
return promptinput.Metadata{
|
return promptinput.Metadata{
|
||||||
RunID: metadata.RunID,
|
RunID: metadata.RunID,
|
||||||
ReportID: metadata.ReportID,
|
ReportID: metadata.ReportID,
|
||||||
@@ -994,32 +902,6 @@ func briefingLocation(cfg config.Config) *briefing.LocationContext {
|
|||||||
return &location
|
return &location
|
||||||
}
|
}
|
||||||
|
|
||||||
func defaultStore(cfg config.Config) (*state.FilesystemStore, error) {
|
|
||||||
return state.NewFilesystemStore(cfg.Workspace)
|
|
||||||
}
|
|
||||||
|
|
||||||
func recentChanges(ctx context.Context, store state.Store, priorSnapshot *state.PriorSnapshot, reportID report.ID, current module.Snapshot, cfg config.RecentChangeConfig) ([]changes.Change, error) {
|
|
||||||
if priorSnapshot == nil {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
previous, err := store.LoadModuleSnapshot(ctx, priorSnapshot.ModuleSnapshotPath)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
thresholds := changes.Thresholds{
|
|
||||||
TemperatureDegrees: cfg.TemperatureDegrees,
|
|
||||||
PrecipProbabilityPoints: cfg.PrecipProbabilityPoints,
|
|
||||||
WindGustMilesPerHour: cfg.WindGustMilesPerHour,
|
|
||||||
PrecipTimingShiftMinutes: cfg.PrecipTimingShiftMinutes,
|
|
||||||
}
|
|
||||||
switch reportID {
|
|
||||||
case report.Daily, report.Today, report.Tomorrow:
|
|
||||||
return changes.CompareDaily(previous, current, thresholds)
|
|
||||||
default:
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func generatedReportError(resolved report.Resolved, runID string, operation string, err error) error {
|
func generatedReportError(resolved report.Resolved, runID string, operation string, err error) error {
|
||||||
if err == nil {
|
if err == nil {
|
||||||
return nil
|
return nil
|
||||||
|
|||||||
@@ -1,85 +0,0 @@
|
|||||||
package app
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/collect"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestRunBatchDetailedInspectsEveryCandidateBeforeCollection(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
batch BatchKind
|
|
||||||
now string
|
|
||||||
wantPrompts int
|
|
||||||
}{
|
|
||||||
{name: "morning", batch: BatchMorning, now: "2026-05-29T08:00:00-05:00", wantPrompts: 3},
|
|
||||||
{name: "evening", batch: BatchEvening, now: "2026-05-29T18:00:00-05:00", wantPrompts: 2},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
cfg := config.Defaults()
|
|
||||||
cfg.Workspace.Root = t.TempDir()
|
|
||||||
now := mustParse(test.now)
|
|
||||||
req := BatchRequest{Config: cfg, Batch: test.batch, Now: now}
|
|
||||||
candidates, err := batchInspectionCandidates(req, now)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("batchInspectionCandidates() error = %v", err)
|
|
||||||
}
|
|
||||||
executor := &inspectionExecutor{profiles: map[string]promptexec.ProfileInspection{
|
|
||||||
"weather-balanced": {ProfileID: "weather-balanced", BackendID: "openrouter", ModelName: "~google/gemini-flash-latest"},
|
|
||||||
}, prompts: map[string]promptexec.PromptInspection{}}
|
|
||||||
for _, candidate := range candidates {
|
|
||||||
executor.prompts[candidate.Definition.PromptID] = logicalPromptInspection(candidate.Definition)
|
|
||||||
}
|
|
||||||
collector := collectorFunc(func(context.Context, collect.Request) (*collect.Result, error) {
|
|
||||||
return nil, errors.New("collection reached")
|
|
||||||
})
|
|
||||||
req.Executor = executor
|
|
||||||
req.Collector = collector
|
|
||||||
_, err = RunBatchDetailed(context.Background(), req)
|
|
||||||
if err == nil || err.Error() != "collection reached" {
|
|
||||||
t.Fatalf("RunBatchDetailed() error = %v, want collection error", err)
|
|
||||||
}
|
|
||||||
if len(executor.promptRequests) != test.wantPrompts || len(executor.profileRequests) != 1 || executor.profileRequests[0] != "weather-balanced" {
|
|
||||||
t.Fatalf("inspection calls = prompts %#v profiles %#v", executor.promptRequests, executor.profileRequests)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCopyBatchReportPathsLeavesUnreachedPathsEmpty(t *testing.T) {
|
|
||||||
item := BatchReportResult{}
|
|
||||||
copyBatchReportPaths(&item, &ReportResult{
|
|
||||||
DataPackagePath: "/runs/daily/data_package.yaml",
|
|
||||||
PreparationPath: "/runs/daily/preparation.json",
|
|
||||||
})
|
|
||||||
|
|
||||||
data, err := json.Marshal(item)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Marshal() error = %v", err)
|
|
||||||
}
|
|
||||||
text := string(data)
|
|
||||||
for _, omitted := range []string{"executionPath", "reportPath", "outputPath", "metadataPath", "notificationPath"} {
|
|
||||||
if strings.Contains(text, omitted) {
|
|
||||||
t.Fatalf("batch item includes unreached field %q:\n%s", omitted, text)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if !strings.Contains(text, "dataPackagePath") || !strings.Contains(text, "preparationPath") {
|
|
||||||
t.Fatalf("batch item omits reached paths:\n%s", text)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
type collectorFunc func(context.Context, collect.Request) (*collect.Result, error)
|
|
||||||
|
|
||||||
func (f collectorFunc) Run(ctx context.Context, req collect.Request) (*collect.Result, error) {
|
|
||||||
return f(ctx, req)
|
|
||||||
}
|
|
||||||
|
|
||||||
var _ Collector = collectorFunc(nil)
|
|
||||||
142
internal/app/batch_generation_test.go
Normal file
142
internal/app/batch_generation_test.go
Normal file
@@ -0,0 +1,142 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRunBatchDetailedKeepsSuccessfulOutputAndSkipsNotificationAfterPartialFailure(t *testing.T) {
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
bundle.Hourly.Periods = bundle.Hourly.Periods[:1]
|
||||||
|
notifier := &generationNotifier{}
|
||||||
|
executor := &generationExecutor{failedPrompt: generationDefinitionForPrompt("weather.tomorrow_generated_text").PromptID}
|
||||||
|
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
||||||
|
Config: generationDistributorConfig(), Batch: BatchMorning,
|
||||||
|
Now: generationTime("2026-05-29T08:30:00-05:00"), WorkingDir: t.TempDir(), OutputDir: t.TempDir(),
|
||||||
|
Collector: &generationCollector{bundle: &bundle}, Executor: executor, Notifier: notifier,
|
||||||
|
})
|
||||||
|
if err != nil || result == nil || result.Total != 2 || result.Succeeded != 1 || result.Failed != 1 || result.Notification == nil || result.Notification.Status != "skipped" || notifier.batchCalls != 0 {
|
||||||
|
t.Fatalf("RunBatchDetailed() result/error/notifier = %#v/%v/%#v", result, err, notifier)
|
||||||
|
}
|
||||||
|
if result.Reports[0].Status != "succeeded" || result.Reports[0].OutputPath == "" || result.Reports[1].Status != "failed" || result.Reports[1].OutputPath != "" {
|
||||||
|
t.Fatalf("report results = %#v", result.Reports)
|
||||||
|
}
|
||||||
|
if data, readErr := os.ReadFile(result.Reports[0].OutputPath); readErr != nil || len(data) == 0 {
|
||||||
|
t.Fatalf("successful output = %q, error = %v", data, readErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunBatchDetailedNotifiesOnlyAfterAllOutputsExist(t *testing.T) {
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
bundle.Hourly.Periods = bundle.Hourly.Periods[:1]
|
||||||
|
outputDir := t.TempDir()
|
||||||
|
notifier := &generationNotifier{}
|
||||||
|
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
||||||
|
Config: generationDistributorConfig(), Batch: BatchMorning,
|
||||||
|
Now: generationTime("2026-05-29T08:30:00-05:00"), WorkingDir: t.TempDir(), OutputDir: outputDir,
|
||||||
|
Collector: &generationCollector{bundle: &bundle}, Executor: &generationExecutor{}, Notifier: notifier,
|
||||||
|
})
|
||||||
|
if err != nil || result == nil || result.Total != 2 || result.Succeeded != 2 || result.Failed != 0 || notifier.batchCalls != 1 || result.Notification == nil || result.Notification.Status != "succeeded" {
|
||||||
|
t.Fatalf("RunBatchDetailed() result/error/notifier = %#v/%v/%#v", result, err, notifier)
|
||||||
|
}
|
||||||
|
if len(notifier.batchRequest.Files) < 2 || len(notifier.batchRequest.IncludedReports) != 2 {
|
||||||
|
t.Fatalf("batch notification = %#v", notifier.batchRequest)
|
||||||
|
}
|
||||||
|
if result.Reports[0].OutputPath == result.Reports[1].OutputPath {
|
||||||
|
t.Fatalf("batch reports share output path %q", result.Reports[0].OutputPath)
|
||||||
|
}
|
||||||
|
for _, file := range notifier.batchRequest.Files {
|
||||||
|
if filepath.Dir(file.SourcePath) != outputDir || file.BundlePath == "" {
|
||||||
|
t.Fatalf("notification file = %#v", file)
|
||||||
|
}
|
||||||
|
if _, statErr := os.Stat(file.SourcePath); statErr != nil {
|
||||||
|
t.Fatalf("notification source %q: %v", file.SourcePath, statErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunBatchDetailedPreflightsAllOutputPaths(t *testing.T) {
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
bundle.Hourly.Periods = bundle.Hourly.Periods[:1]
|
||||||
|
outputDir := t.TempDir()
|
||||||
|
if err := os.Mkdir(filepath.Join(outputDir, "tomorrow.md"), 0o700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
todayPath := filepath.Join(outputDir, "today.md")
|
||||||
|
const previousReport = "previous report"
|
||||||
|
if err := os.WriteFile(todayPath, []byte(previousReport), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
executor := &generationExecutor{}
|
||||||
|
promptInspectedBeforeCollection := false
|
||||||
|
collector := &generationCollector{
|
||||||
|
bundle: &bundle,
|
||||||
|
beforeRun: func() {
|
||||||
|
promptInspectedBeforeCollection = executor.promptInspections > 0
|
||||||
|
},
|
||||||
|
}
|
||||||
|
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
||||||
|
Config: generationDistributorConfig(), Batch: BatchMorning,
|
||||||
|
Now: generationTime("2026-05-29T08:30:00-05:00"), WorkingDir: t.TempDir(), OutputDir: outputDir,
|
||||||
|
Collector: collector, Executor: executor, Notifier: &generationNotifier{},
|
||||||
|
})
|
||||||
|
if err == nil || result != nil || !collector.called || !promptInspectedBeforeCollection || executor.called {
|
||||||
|
t.Fatalf("RunBatchDetailed() result/error/collection/inspection/execution = %#v/%v/%t/%t/%t", result, err, collector.called, promptInspectedBeforeCollection, executor.called)
|
||||||
|
}
|
||||||
|
if data, readErr := os.ReadFile(todayPath); readErr != nil || string(data) != previousReport {
|
||||||
|
t.Fatalf("earlier output = %q, error = %v", data, readErr)
|
||||||
|
}
|
||||||
|
if info, statErr := os.Stat(filepath.Join(outputDir, "tomorrow.md")); statErr != nil || !info.IsDir() {
|
||||||
|
t.Fatalf("blocked output info/error = %#v/%v", info, statErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunBatchDetailedRetainsReportCountsWhenNotificationFails(t *testing.T) {
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
bundle.Hourly.Periods = bundle.Hourly.Periods[:1]
|
||||||
|
outputDir := t.TempDir()
|
||||||
|
notifier := &generationNotifier{batchErr: errors.New("distributor unavailable")}
|
||||||
|
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
||||||
|
Config: generationDistributorConfig(), Batch: BatchMorning,
|
||||||
|
Now: generationTime("2026-05-29T08:30:00-05:00"), WorkingDir: t.TempDir(), OutputDir: outputDir,
|
||||||
|
Collector: &generationCollector{bundle: &bundle}, Executor: &generationExecutor{}, Notifier: notifier,
|
||||||
|
})
|
||||||
|
if err != nil || result == nil || result.Total != len(result.Reports) || result.Succeeded != len(result.Reports) || result.Failed != 0 || result.Notification == nil || result.Notification.Status != "failed" {
|
||||||
|
t.Fatalf("RunBatchDetailed() result/error = %#v/%v", result, err)
|
||||||
|
}
|
||||||
|
for _, item := range result.Reports {
|
||||||
|
if item.Status != "succeeded" || item.OutputPath == "" {
|
||||||
|
t.Fatalf("report result = %#v", item)
|
||||||
|
}
|
||||||
|
if _, statErr := os.Stat(item.OutputPath); statErr != nil {
|
||||||
|
t.Fatalf("published output %q: %v", item.OutputPath, statErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunBatchReturnsNotificationFailureWithoutReportFailureWording(t *testing.T) {
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
bundle.Hourly.Periods = bundle.Hourly.Periods[:1]
|
||||||
|
err := RunBatch(context.Background(), BatchRequest{
|
||||||
|
Config: generationDistributorConfig(), Batch: BatchMorning,
|
||||||
|
Now: generationTime("2026-05-29T08:30:00-05:00"), WorkingDir: t.TempDir(), OutputDir: t.TempDir(),
|
||||||
|
Collector: &generationCollector{bundle: &bundle}, Executor: &generationExecutor{}, Notifier: &generationNotifier{batchErr: errors.New("distributor unavailable")},
|
||||||
|
})
|
||||||
|
var batchErr BatchError
|
||||||
|
if !errors.As(err, &batchErr) || batchErr.Result == nil || batchErr.Result.Failed != 0 || batchErr.Result.Notification == nil || batchErr.Result.Notification.Status != "failed" || !strings.Contains(err.Error(), "notification failed") || strings.Contains(err.Error(), "reports failed") {
|
||||||
|
t.Fatalf("RunBatch() error/result = %v/%#v", err, batchErr.Result)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func generationDistributorConfig() config.Config {
|
||||||
|
cfg := generationConfig()
|
||||||
|
cfg.Notify.Distributor.Enabled = true
|
||||||
|
cfg.Notify.Distributor.PipelineIDTemplate = "weather"
|
||||||
|
return cfg
|
||||||
|
}
|
||||||
@@ -3,12 +3,12 @@ package app
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"path/filepath"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
distributoradapter "gitea.maximumdirect.net/eric/weatherreporter/internal/adapters/distributor"
|
distributoradapter "gitea.maximumdirect.net/eric/weatherreporter/internal/adapters/distributor"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -46,39 +46,31 @@ func batchRunID(startedAt time.Time, batch BatchKind) string {
|
|||||||
return startedAt.UTC().Format(runIDTimestampLayout) + "_" + string(batch)
|
return startedAt.UTC().Format(runIDTimestampLayout) + "_" + string(batch)
|
||||||
}
|
}
|
||||||
|
|
||||||
func notifyBatch(ctx context.Context, cfg config.Config, batch BatchKind, runID string, startedAt time.Time, result *BatchResult, planned []plannedBatchReport, store state.Store, notifier Notifier) (*BatchNotificationResult, error) {
|
func notifyBatch(ctx context.Context, cfg config.Config, batch BatchKind, runID string, startedAt time.Time, result *BatchResult, planned []plannedBatchReport, notifier Notifier) *BatchNotificationResult {
|
||||||
if !cfg.Notify.Distributor.Enabled {
|
if !cfg.Notify.Distributor.Enabled {
|
||||||
return nil, nil
|
return nil
|
||||||
}
|
}
|
||||||
if !cfg.Notify.Distributor.Batch.Enabled {
|
if !cfg.Notify.Distributor.Batch.Enabled {
|
||||||
return nil, nil
|
return nil
|
||||||
}
|
}
|
||||||
if result == nil {
|
if result == nil {
|
||||||
return nil, fmt.Errorf("batch result is required")
|
return failedBatchNotificationResult(batchNotificationRequest{}, fmt.Errorf("batch result is required"))
|
||||||
}
|
}
|
||||||
if result.Failed > 0 {
|
if result.Failed > 0 {
|
||||||
return &BatchNotificationResult{
|
return &BatchNotificationResult{
|
||||||
Status: "skipped",
|
Status: "skipped",
|
||||||
Reason: "one or more reports failed",
|
Reason: "one or more reports failed",
|
||||||
}, nil
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
req, err := buildBatchNotificationRequest(cfg, batch, runID, startedAt, result.Reports, planned)
|
req, err := buildBatchNotificationRequest(cfg, batch, runID, startedAt, result.Reports, planned)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
path, saveErr := saveBatchNotificationArtifact(ctx, store, cfg, batch, runID, startedAt, batchNotificationRequest{}, nil, err)
|
return failedBatchNotificationResult(batchNotificationRequest{}, err)
|
||||||
if saveErr != nil {
|
|
||||||
return nil, saveErr
|
|
||||||
}
|
|
||||||
return failedBatchNotificationResult(batchNotificationRequest{}, path, err), err
|
|
||||||
}
|
}
|
||||||
|
|
||||||
batchNotifier, err := resolveBatchNotifier(cfg, notifier)
|
batchNotifier, err := resolveBatchNotifier(cfg, notifier)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
path, saveErr := saveBatchNotificationArtifact(ctx, store, cfg, batch, runID, startedAt, req, nil, err)
|
return failedBatchNotificationResult(req, err)
|
||||||
if saveErr != nil {
|
|
||||||
return nil, saveErr
|
|
||||||
}
|
|
||||||
return failedBatchNotificationResult(req, path, err), err
|
|
||||||
}
|
}
|
||||||
|
|
||||||
notification, notifyErr := batchNotifier.NotifyBatch(ctx, req)
|
notification, notifyErr := batchNotifier.NotifyBatch(ctx, req)
|
||||||
@@ -86,18 +78,13 @@ func notifyBatch(ctx context.Context, cfg config.Config, batch BatchKind, runID
|
|||||||
if notifyErr != nil {
|
if notifyErr != nil {
|
||||||
wrappedErr = fmt.Errorf("notify batch %q run %q bundle %q: %w", batch, runID, req.BundleID, notifyErr)
|
wrappedErr = fmt.Errorf("notify batch %q run %q bundle %q: %w", batch, runID, req.BundleID, notifyErr)
|
||||||
}
|
}
|
||||||
path, saveErr := saveBatchNotificationArtifact(ctx, store, cfg, batch, runID, startedAt, req, notification, wrappedErr)
|
batchResult := batchNotificationResult(req, notification)
|
||||||
if saveErr != nil {
|
|
||||||
return nil, saveErr
|
|
||||||
}
|
|
||||||
|
|
||||||
batchResult := batchNotificationResult(req, notification, path)
|
|
||||||
if wrappedErr != nil {
|
if wrappedErr != nil {
|
||||||
batchResult.Status = "failed"
|
batchResult.Status = "failed"
|
||||||
batchResult.Error = wrappedErr.Error()
|
batchResult.Error = wrappedErr.Error()
|
||||||
return batchResult, wrappedErr
|
return batchResult
|
||||||
}
|
}
|
||||||
return batchResult, nil
|
return batchResult
|
||||||
}
|
}
|
||||||
|
|
||||||
func resolveBatchNotifier(cfg config.Config, notifier Notifier) (batchNotifier, error) {
|
func resolveBatchNotifier(cfg config.Config, notifier Notifier) (batchNotifier, error) {
|
||||||
@@ -153,15 +140,15 @@ func buildBatchNotificationRequest(cfg config.Config, batch BatchKind, runID str
|
|||||||
if item.ReportID != plannedReport.Resolved.Definition.ID {
|
if item.ReportID != plannedReport.Resolved.Definition.ID {
|
||||||
return batchNotificationRequest{}, fmt.Errorf("batch notification report %q run %q does not match planned report %q", item.ReportID, item.RunID, plannedReport.Resolved.Definition.ID)
|
return batchNotificationRequest{}, fmt.Errorf("batch notification report %q run %q does not match planned report %q", item.ReportID, item.RunID, plannedReport.Resolved.Definition.ID)
|
||||||
}
|
}
|
||||||
if item.ReportPath == "" {
|
if item.OutputPath == "" {
|
||||||
return batchNotificationRequest{}, fmt.Errorf("batch notification report %q run %q is missing managed report path", item.ReportID, item.RunID)
|
return batchNotificationRequest{}, fmt.Errorf("batch notification report %q run %q is missing output path", item.ReportID, item.RunID)
|
||||||
}
|
}
|
||||||
|
|
||||||
values, err := distributorTemplateValuesForReport(cfg, plannedReport.Resolved, item.RunID, plannedReport.OutputCopyName)
|
values, err := distributorTemplateValuesForReport(cfg, plannedReport.Resolved, item.RunID, filepath.Base(item.OutputPath))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return batchNotificationRequest{}, fmt.Errorf("batch notification report %q run %q source path %q: %w", item.ReportID, item.RunID, item.ReportPath, err)
|
return batchNotificationRequest{}, fmt.Errorf("batch notification report %q run %q source path %q: %w", item.ReportID, item.RunID, item.OutputPath, err)
|
||||||
}
|
}
|
||||||
bundlePaths, err := renderDistributorReportBundlePaths(cfg, plannedReport.Resolved, item.RunID, item.ReportPath, values)
|
bundlePaths, err := renderDistributorReportBundlePaths(cfg, plannedReport.Resolved, item.RunID, item.OutputPath, values)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return batchNotificationRequest{}, err
|
return batchNotificationRequest{}, err
|
||||||
}
|
}
|
||||||
@@ -169,18 +156,18 @@ func buildBatchNotificationRequest(cfg config.Config, batch BatchKind, runID str
|
|||||||
included := BatchNotificationReport{
|
included := BatchNotificationReport{
|
||||||
ReportID: item.ReportID,
|
ReportID: item.ReportID,
|
||||||
RunID: item.RunID,
|
RunID: item.RunID,
|
||||||
SourcePath: item.ReportPath,
|
SourcePath: item.OutputPath,
|
||||||
BundlePaths: append([]string(nil), bundlePaths...),
|
BundlePaths: append([]string(nil), bundlePaths...),
|
||||||
}
|
}
|
||||||
for _, bundlePath := range bundlePaths {
|
for _, bundlePath := range bundlePaths {
|
||||||
file := batchNotificationFile{
|
file := batchNotificationFile{
|
||||||
ReportID: item.ReportID,
|
ReportID: item.ReportID,
|
||||||
RunID: item.RunID,
|
RunID: item.RunID,
|
||||||
SourcePath: item.ReportPath,
|
SourcePath: item.OutputPath,
|
||||||
BundlePath: bundlePath,
|
BundlePath: bundlePath,
|
||||||
}
|
}
|
||||||
if previous, ok := seenBundlePaths[bundlePath]; ok {
|
if previous, ok := seenBundlePaths[bundlePath]; ok {
|
||||||
return batchNotificationRequest{}, fmt.Errorf("batch notification duplicate bundle path %q for report %q run %q source path %q; already used by report %q run %q source path %q", bundlePath, item.ReportID, item.RunID, item.ReportPath, previous.ReportID, previous.RunID, previous.SourcePath)
|
return batchNotificationRequest{}, fmt.Errorf("batch notification duplicate bundle path %q for report %q run %q source path %q; already used by report %q run %q source path %q", bundlePath, item.ReportID, item.RunID, item.OutputPath, previous.ReportID, previous.RunID, previous.SourcePath)
|
||||||
}
|
}
|
||||||
seenBundlePaths[bundlePath] = file
|
seenBundlePaths[bundlePath] = file
|
||||||
req.Files = append(req.Files, file)
|
req.Files = append(req.Files, file)
|
||||||
@@ -225,13 +212,12 @@ func batchDistributorUploadRequest(req batchNotificationRequest) distributoradap
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func batchNotificationResult(req batchNotificationRequest, result *NotificationResult, path string) *BatchNotificationResult {
|
func batchNotificationResult(req batchNotificationRequest, result *NotificationResult) *BatchNotificationResult {
|
||||||
notification := &BatchNotificationResult{
|
notification := &BatchNotificationResult{
|
||||||
Status: "unknown",
|
Status: "unknown",
|
||||||
PipelineID: req.PipelineID,
|
PipelineID: req.PipelineID,
|
||||||
BundleID: req.BundleID,
|
BundleID: req.BundleID,
|
||||||
IdempotencyKey: req.IdempotencyKey,
|
IdempotencyKey: req.IdempotencyKey,
|
||||||
Path: path,
|
|
||||||
IncludedReports: append([]BatchNotificationReport(nil), req.IncludedReports...),
|
IncludedReports: append([]BatchNotificationReport(nil), req.IncludedReports...),
|
||||||
}
|
}
|
||||||
if result != nil {
|
if result != nil {
|
||||||
@@ -256,8 +242,8 @@ func batchNotificationResult(req batchNotificationRequest, result *NotificationR
|
|||||||
return notification
|
return notification
|
||||||
}
|
}
|
||||||
|
|
||||||
func failedBatchNotificationResult(req batchNotificationRequest, path string, err error) *BatchNotificationResult {
|
func failedBatchNotificationResult(req batchNotificationRequest, err error) *BatchNotificationResult {
|
||||||
notification := batchNotificationResult(req, nil, path)
|
notification := batchNotificationResult(req, nil)
|
||||||
notification.Status = "failed"
|
notification.Status = "failed"
|
||||||
if err != nil {
|
if err != nil {
|
||||||
notification.Error = err.Error()
|
notification.Error = err.Error()
|
||||||
@@ -265,78 +251,6 @@ func failedBatchNotificationResult(req batchNotificationRequest, path string, er
|
|||||||
return notification
|
return notification
|
||||||
}
|
}
|
||||||
|
|
||||||
func saveBatchNotificationArtifact(ctx context.Context, store state.Store, cfg config.Config, batch BatchKind, runID string, startedAt time.Time, req batchNotificationRequest, result *NotificationResult, notifyErr error) (string, error) {
|
|
||||||
if store == nil {
|
|
||||||
return "", fmt.Errorf("state store is required")
|
|
||||||
}
|
|
||||||
location, err := timeutil.LoadLocation(cfg.WeatherAPI.Timezone)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("load batch notification timezone: %w", err)
|
|
||||||
}
|
|
||||||
artifact := state.BatchDistributorNotificationArtifact{
|
|
||||||
SchemaVersion: state.BatchDistributorNotificationSchemaVersion,
|
|
||||||
Batch: string(batch),
|
|
||||||
BatchRunID: runID,
|
|
||||||
AttemptedAt: time.Now(),
|
|
||||||
Endpoint: cfg.Notify.Distributor.Endpoint,
|
|
||||||
PipelineID: req.PipelineID,
|
|
||||||
BundleID: req.BundleID,
|
|
||||||
IdempotencyKey: req.IdempotencyKey,
|
|
||||||
BundleCreated: req.CreatedAt,
|
|
||||||
Reports: batchNotificationReportArtifacts(req.IncludedReports),
|
|
||||||
Status: "attempted",
|
|
||||||
}
|
|
||||||
if result != nil {
|
|
||||||
artifact.Status = result.Status
|
|
||||||
artifact.Upload = &state.DistributorUploadResult{
|
|
||||||
RunID: result.RunID,
|
|
||||||
Status: result.UploadStatus,
|
|
||||||
}
|
|
||||||
if result.PipelineID != "" || !result.AcceptedAt.IsZero() || result.StartedAt != nil || result.FinishedAt != nil || len(result.Report) > 0 || result.Error != "" {
|
|
||||||
artifact.RunStatus = &state.DistributorRunStatus{
|
|
||||||
RunID: result.RunID,
|
|
||||||
PipelineID: result.PipelineID,
|
|
||||||
Status: result.Status,
|
|
||||||
AcceptedAt: result.AcceptedAt,
|
|
||||||
StartedAt: result.StartedAt,
|
|
||||||
FinishedAt: result.FinishedAt,
|
|
||||||
Report: append([]byte(nil), result.Report...),
|
|
||||||
Error: result.Error,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
artifact.StatusError = result.StatusError
|
|
||||||
}
|
|
||||||
if notifyErr != nil {
|
|
||||||
artifact.Status = "failed"
|
|
||||||
artifact.Error = notifyErr.Error()
|
|
||||||
}
|
|
||||||
if artifact.Status == "" {
|
|
||||||
artifact.Status = "unknown"
|
|
||||||
}
|
|
||||||
return store.SaveBatchDistributorNotification(ctx, state.BatchDistributorNotificationRef{
|
|
||||||
Batch: string(batch),
|
|
||||||
BatchRunID: runID,
|
|
||||||
StartedAt: startedAt,
|
|
||||||
Location: location,
|
|
||||||
}, artifact)
|
|
||||||
}
|
|
||||||
|
|
||||||
func batchNotificationReportArtifacts(reports []BatchNotificationReport) []state.BatchDistributorNotificationReportArtifact {
|
|
||||||
if len(reports) == 0 {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
artifacts := make([]state.BatchDistributorNotificationReportArtifact, 0, len(reports))
|
|
||||||
for _, item := range reports {
|
|
||||||
artifacts = append(artifacts, state.BatchDistributorNotificationReportArtifact{
|
|
||||||
ReportID: item.ReportID,
|
|
||||||
RunID: item.RunID,
|
|
||||||
SourcePath: item.SourcePath,
|
|
||||||
BundlePaths: append([]string(nil), item.BundlePaths...),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
return artifacts
|
|
||||||
}
|
|
||||||
|
|
||||||
func renderBatchNotificationIdentity(cfg config.Config, batch BatchKind, runID string, startedAt time.Time) (batchNotificationIdentity, error) {
|
func renderBatchNotificationIdentity(cfg config.Config, batch BatchKind, runID string, startedAt time.Time) (batchNotificationIdentity, error) {
|
||||||
values, err := batchNotificationTemplateValues(cfg, batch, runID, startedAt)
|
values, err := batchNotificationTemplateValues(cfg, batch, runID, startedAt)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ import (
|
|||||||
|
|
||||||
type plannedBatchReport struct {
|
type plannedBatchReport struct {
|
||||||
Resolved report.Resolved
|
Resolved report.Resolved
|
||||||
OutputCopyName string
|
OutputPath string
|
||||||
}
|
}
|
||||||
|
|
||||||
func planBatchRun(req BatchRequest, now time.Time, collection collect.Result) ([]plannedBatchReport, error) {
|
func planBatchRun(req BatchRequest, now time.Time, collection collect.Result) ([]plannedBatchReport, error) {
|
||||||
@@ -36,16 +36,16 @@ func planBatchRun(req BatchRequest, now time.Time, collection collect.Result) ([
|
|||||||
var planned []plannedBatchReport
|
var planned []plannedBatchReport
|
||||||
switch batch {
|
switch batch {
|
||||||
case report.Morning:
|
case report.Morning:
|
||||||
planned, err = appendPlannedReport(planned, registry, report.Today, resolveReq, "")
|
planned, err = appendPlannedReport(planned, registry, report.Today, resolveReq)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
planned, err = appendPlannedReport(planned, registry, report.Tomorrow, resolveReq, "")
|
planned, err = appendPlannedReport(planned, registry, report.Tomorrow, resolveReq)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
case report.Evening:
|
case report.Evening:
|
||||||
planned, err = appendPlannedReport(planned, registry, report.Tomorrow, resolveReq, "")
|
planned, err = appendPlannedReport(planned, registry, report.Tomorrow, resolveReq)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -60,8 +60,7 @@ func planBatchRun(req BatchRequest, now time.Time, collection collect.Result) ([
|
|||||||
for _, date := range eligibleDailyDates(hourly, now, location) {
|
for _, date := range eligibleDailyDates(hourly, now, location) {
|
||||||
dailyReq := resolveReq
|
dailyReq := resolveReq
|
||||||
dailyReq.Date = date
|
dailyReq.Date = date
|
||||||
outputCopyName := "daily-" + date.In(location).Format(timeutil.DateLayout) + ".md"
|
planned, err = appendPlannedReport(planned, registry, report.Daily, dailyReq)
|
||||||
planned, err = appendPlannedReport(planned, registry, report.Daily, dailyReq, outputCopyName)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -69,15 +68,12 @@ func planBatchRun(req BatchRequest, now time.Time, collection collect.Result) ([
|
|||||||
return planned, nil
|
return planned, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func appendPlannedReport(planned []plannedBatchReport, registry report.Registry, id report.ID, req report.ResolveRequest, outputCopyName string) ([]plannedBatchReport, error) {
|
func appendPlannedReport(planned []plannedBatchReport, registry report.Registry, id report.ID, req report.ResolveRequest) ([]plannedBatchReport, error) {
|
||||||
resolved, err := registry.Resolve(id, req)
|
resolved, err := registry.Resolve(id, req)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
return append(planned, plannedBatchReport{
|
return append(planned, plannedBatchReport{Resolved: resolved}), nil
|
||||||
Resolved: resolved,
|
|
||||||
OutputCopyName: outputCopyName,
|
|
||||||
}), nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func eligibleDailyDates(hourly *weatherdata.ForecastRun, now time.Time, location *time.Location) []time.Time {
|
func eligibleDailyDates(hourly *weatherdata.ForecastRun, now time.Time, location *time.Location) []time.Time {
|
||||||
|
|||||||
@@ -55,7 +55,7 @@ func TestPlanBatchRunDynamicDailyDatesStartAfterTomorrow(t *testing.T) {
|
|||||||
assertPlanningPeriod(t, daily[1].Resolved.ValidPeriod, "2026-06-01T00:00:00-05:00", "2026-06-02T00:00:00-05:00")
|
assertPlanningPeriod(t, daily[1].Resolved.ValidPeriod, "2026-06-01T00:00:00-05:00", "2026-06-02T00:00:00-05:00")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPlanBatchRunDynamicDailyOutputCopyNames(t *testing.T) {
|
func TestPlanBatchRunUsesResolvedOutputNames(t *testing.T) {
|
||||||
location := mustLoadTestLocation(t, "America/Chicago")
|
location := mustLoadTestLocation(t, "America/Chicago")
|
||||||
hourly := hourlyRun(fullDayPeriods(t, "2026-05-31", location)...)
|
hourly := hourlyRun(fullDayPeriods(t, "2026-05-31", location)...)
|
||||||
|
|
||||||
@@ -68,11 +68,19 @@ func TestPlanBatchRunDynamicDailyOutputCopyNames(t *testing.T) {
|
|||||||
if len(daily) != 1 {
|
if len(daily) != 1 {
|
||||||
t.Fatalf("daily reports = %#v, want one Daily report", daily)
|
t.Fatalf("daily reports = %#v, want one Daily report", daily)
|
||||||
}
|
}
|
||||||
if daily[0].OutputCopyName != "daily-2026-05-31.md" {
|
outputName, err := daily[0].Resolved.OutputName()
|
||||||
t.Fatalf("OutputCopyName = %q, want date-qualified Daily name", daily[0].OutputCopyName)
|
if err != nil {
|
||||||
|
t.Fatalf("OutputName() error = %v", err)
|
||||||
}
|
}
|
||||||
if planned[0].OutputCopyName != "" {
|
if outputName != "daily-2026-05-31.md" {
|
||||||
t.Fatalf("Tomorrow OutputCopyName = %q, want definition batch output name to apply later", planned[0].OutputCopyName)
|
t.Fatalf("Daily output name = %q, want date-qualified name", outputName)
|
||||||
|
}
|
||||||
|
outputName, err = planned[0].Resolved.OutputName()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("OutputName() error = %v", err)
|
||||||
|
}
|
||||||
|
if outputName != "tomorrow.md" {
|
||||||
|
t.Fatalf("Tomorrow output name = %q, want tomorrow.md", outputName)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,500 +0,0 @@
|
|||||||
package app
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/collect"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
|
||||||
)
|
|
||||||
|
|
||||||
type assembledBatchExecutor struct {
|
|
||||||
definitions map[string]report.Definition
|
|
||||||
promptRequests []string
|
|
||||||
profileRequests []string
|
|
||||||
executeRequests []promptexec.ExecuteRequest
|
|
||||||
failures map[int]error
|
|
||||||
active int
|
|
||||||
maxActive int
|
|
||||||
}
|
|
||||||
|
|
||||||
func newAssembledBatchExecutor() *assembledBatchExecutor {
|
|
||||||
definitions := make(map[string]report.Definition)
|
|
||||||
for _, definition := range report.DefaultRegistry().All() {
|
|
||||||
definitions[definition.PromptID] = definition
|
|
||||||
}
|
|
||||||
return &assembledBatchExecutor{definitions: definitions, failures: make(map[int]error)}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *assembledBatchExecutor) InspectPrompt(_ context.Context, id, version string) (promptexec.PromptInspection, error) {
|
|
||||||
e.promptRequests = append(e.promptRequests, id+"@"+version)
|
|
||||||
definition, ok := e.definitions[id]
|
|
||||||
if !ok || definition.PromptVersion != version {
|
|
||||||
return promptexec.PromptInspection{}, errors.New("unexpected prompt inspection")
|
|
||||||
}
|
|
||||||
return logicalPromptInspection(definition), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *assembledBatchExecutor) InspectProfile(_ context.Context, id string) (promptexec.ProfileInspection, error) {
|
|
||||||
e.profileRequests = append(e.profileRequests, id)
|
|
||||||
return promptexec.ProfileInspection{ProfileID: id, BackendID: "fixture", ModelName: "fixture-model"}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *assembledBatchExecutor) Execute(_ context.Context, req promptexec.ExecuteRequest, callback promptexec.PreparationCallback) (*promptexec.Execution, error) {
|
|
||||||
call := len(e.executeRequests)
|
|
||||||
e.executeRequests = append(e.executeRequests, req)
|
|
||||||
e.active++
|
|
||||||
if e.active > e.maxActive {
|
|
||||||
e.maxActive = e.active
|
|
||||||
}
|
|
||||||
defer func() { e.active-- }()
|
|
||||||
stamp := time.Date(2026, 5, 29, 15, 0, 0, 0, time.UTC)
|
|
||||||
preparation := promptexec.Preparation{
|
|
||||||
PromptID: req.PromptID, PromptVersion: req.PromptVersion, PromptHash: "prompt-hash",
|
|
||||||
RenderedPromptHash: "rendered-hash", ProfileID: req.ProfileID, BackendID: "fixture",
|
|
||||||
ModelName: "fixture-model", DataPackagePath: req.DataPackagePath, StartedAt: stamp, EndedAt: stamp,
|
|
||||||
}
|
|
||||||
if err := callback(preparation, nil); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if err := e.failures[call]; err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
definition := e.definitions[req.PromptID]
|
|
||||||
return &promptexec.Execution{
|
|
||||||
RunID: "provider-run", PromptID: req.PromptID, PromptVersion: req.PromptVersion,
|
|
||||||
PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash", ProfileID: req.ProfileID,
|
|
||||||
BackendID: "fixture", ModelName: "fixture-model", GeneratedHash: "generated-hash",
|
|
||||||
StartedAt: stamp, EndedAt: stamp, DataPackagePath: req.DataPackagePath,
|
|
||||||
RawOutput: []byte(generatedTextForPrompt(req.PromptID)),
|
|
||||||
Validation: promptexec.NewValidation(promptexec.ValidationPassed, "json_schema", definition.GeneratedTextSchemaID+".generated_text.schema.json", nil),
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type assembledBatchNotifier struct {
|
|
||||||
reportRequests []NotificationRequest
|
|
||||||
batchRequests []batchNotificationRequest
|
|
||||||
batchResult *NotificationResult
|
|
||||||
batchErr error
|
|
||||||
}
|
|
||||||
|
|
||||||
func (n *assembledBatchNotifier) Notify(_ context.Context, req NotificationRequest) (*NotificationResult, error) {
|
|
||||||
n.reportRequests = append(n.reportRequests, req)
|
|
||||||
return nil, errors.New("per-report notification must be suppressed")
|
|
||||||
}
|
|
||||||
|
|
||||||
func (n *assembledBatchNotifier) NotifyBatch(_ context.Context, req batchNotificationRequest) (*NotificationResult, error) {
|
|
||||||
n.batchRequests = append(n.batchRequests, req)
|
|
||||||
if n.batchErr != nil {
|
|
||||||
return nil, n.batchErr
|
|
||||||
}
|
|
||||||
if n.batchResult != nil {
|
|
||||||
result := *n.batchResult
|
|
||||||
if result.PipelineID == "" {
|
|
||||||
result.PipelineID = req.PipelineID
|
|
||||||
}
|
|
||||||
if result.BundleID == "" {
|
|
||||||
result.BundleID = req.BundleID
|
|
||||||
}
|
|
||||||
if result.IdempotencyKey == "" {
|
|
||||||
result.IdempotencyKey = req.IdempotencyKey
|
|
||||||
}
|
|
||||||
return &result, nil
|
|
||||||
}
|
|
||||||
return &NotificationResult{
|
|
||||||
RunID: "batch-notification-run", PipelineID: req.PipelineID, BundleID: req.BundleID,
|
|
||||||
IdempotencyKey: req.IdempotencyKey, Status: "succeeded", UploadStatus: "accepted",
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunBatchDetailedExecutesRetainedReportsSequentially(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
batch BatchKind
|
|
||||||
now time.Time
|
|
||||||
wantIDs []report.ID
|
|
||||||
wantCopies []string
|
|
||||||
}{
|
|
||||||
{name: "morning", batch: BatchMorning, now: workflowTime("2026-05-29T08:00:00-05:00"), wantIDs: []report.ID{report.Today, report.Tomorrow, report.Daily}, wantCopies: []string{"today.md", "tomorrow.md", "daily-2026-05-31.md"}},
|
|
||||||
{name: "evening", batch: BatchEvening, now: workflowTime("2026-05-29T18:00:00-05:00"), wantIDs: []report.ID{report.Tomorrow, report.Daily}, wantCopies: []string{"tomorrow.md", "daily-2026-05-31.md"}},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
cfg := assembledBatchConfig(t, false)
|
|
||||||
bundle := assembledBatchBundle(t, "2026-05-31")
|
|
||||||
collector := &workflowCollector{result: &collect.Result{Bundle: &bundle}}
|
|
||||||
executor := newAssembledBatchExecutor()
|
|
||||||
outputDir := filepath.Join(t.TempDir(), "output")
|
|
||||||
debugRoot := filepath.Join(t.TempDir(), "debug")
|
|
||||||
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
|
||||||
Config: cfg, Batch: test.batch, Now: test.now, OutputDir: outputDir, LLMDebugDir: debugRoot,
|
|
||||||
Collector: collector, Executor: executor,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("RunBatchDetailed() error = %v", err)
|
|
||||||
}
|
|
||||||
if collector.calls != 1 || result.Total != len(test.wantIDs) || result.Succeeded != len(test.wantIDs) || result.Failed != 0 {
|
|
||||||
t.Fatalf("collection/summary = %d/%d/%d/%d", collector.calls, result.Total, result.Succeeded, result.Failed)
|
|
||||||
}
|
|
||||||
if len(executor.executeRequests) != len(test.wantIDs) || executor.maxActive != 1 {
|
|
||||||
t.Fatalf("executor calls/max active = %d/%d, want %d/1", len(executor.executeRequests), executor.maxActive, len(test.wantIDs))
|
|
||||||
}
|
|
||||||
if len(executor.profileRequests) != 1 || executor.profileRequests[0] != "weather-balanced" {
|
|
||||||
t.Fatalf("profile inspections = %#v, want one shared weather-balanced inspection", executor.profileRequests)
|
|
||||||
}
|
|
||||||
for index, item := range result.Reports {
|
|
||||||
if item.ReportID != test.wantIDs[index] || item.Status != "succeeded" {
|
|
||||||
t.Fatalf("report %d = %s/%s, want %s/succeeded", index, item.ReportID, item.Status, test.wantIDs[index])
|
|
||||||
}
|
|
||||||
if executor.executeRequests[index].PromptID != item.PromptID {
|
|
||||||
t.Fatalf("execution %d prompt = %q, want item prompt %q", index, executor.executeRequests[index].PromptID, item.PromptID)
|
|
||||||
}
|
|
||||||
if item.DataPackagePath == "" || item.PreparationPath == "" || item.ExecutionPath == "" || item.LLMDebugPath == "" || item.ReportPath == "" || item.OutputPath == "" || item.MetadataPath == "" {
|
|
||||||
t.Fatalf("successful report paths = %#v", item)
|
|
||||||
}
|
|
||||||
assertBatchItemMatchesMetadata(t, item)
|
|
||||||
if filepath.Base(item.OutputPath) != test.wantCopies[index] {
|
|
||||||
t.Fatalf("output copy = %q, want %q", item.OutputPath, test.wantCopies[index])
|
|
||||||
}
|
|
||||||
assertBatchPathsExist(t, item.DataPackagePath, item.PreparationPath, item.ExecutionPath, item.LLMDebugPath, item.ReportPath, item.OutputPath, item.MetadataPath)
|
|
||||||
managed, readErr := os.ReadFile(item.ReportPath)
|
|
||||||
if readErr != nil {
|
|
||||||
t.Fatalf("read managed report: %v", readErr)
|
|
||||||
}
|
|
||||||
copied, readErr := os.ReadFile(item.OutputPath)
|
|
||||||
if readErr != nil || !bytes.Equal(managed, copied) {
|
|
||||||
t.Fatalf("output copy mismatch/error = %v", readErr)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunBatchDetailedContinuesAfterCapacityRejection(t *testing.T) {
|
|
||||||
cfg := assembledBatchConfig(t, true)
|
|
||||||
bundle := assembledBatchBundle(t, "2026-05-31")
|
|
||||||
collector := &workflowCollector{result: &collect.Result{Bundle: &bundle}}
|
|
||||||
executor := newAssembledBatchExecutor()
|
|
||||||
executor.failures[1] = promptexec.NewError(promptexec.Capacity, "capacity rejected", nil)
|
|
||||||
notifier := &assembledBatchNotifier{}
|
|
||||||
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
|
||||||
Config: cfg, Batch: BatchMorning, Now: workflowTime("2026-05-29T08:00:00-05:00"),
|
|
||||||
OutputDir: filepath.Join(t.TempDir(), "output"), LLMDebugDir: filepath.Join(t.TempDir(), "debug"),
|
|
||||||
Collector: collector, Executor: executor, Notifier: notifier,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("RunBatchDetailed() error = %v", err)
|
|
||||||
}
|
|
||||||
if collector.calls != 1 || len(executor.executeRequests) != 3 || result.Total != 3 || result.Succeeded != 2 || result.Failed != 1 {
|
|
||||||
t.Fatalf("collection/execution/summary = %d/%d/%d/%d/%d", collector.calls, len(executor.executeRequests), result.Total, result.Succeeded, result.Failed)
|
|
||||||
}
|
|
||||||
if len(notifier.reportRequests) != 0 || len(notifier.batchRequests) != 0 || result.Notification == nil || result.Notification.Status != "skipped" {
|
|
||||||
t.Fatalf("notification state = reports %d batches %d result %#v", len(notifier.reportRequests), len(notifier.batchRequests), result.Notification)
|
|
||||||
}
|
|
||||||
for index, item := range result.Reports {
|
|
||||||
if index == 1 {
|
|
||||||
if item.ReportID != report.Tomorrow || item.Status != "failed" || !strings.Contains(item.Error, string(promptexec.Capacity)) {
|
|
||||||
t.Fatalf("failed item = %#v", item)
|
|
||||||
}
|
|
||||||
if item.DataPackagePath == "" || item.PreparationPath == "" || item.ExecutionPath == "" || item.LLMDebugPath == "" || item.MetadataPath == "" || item.ReportPath != "" || item.OutputPath != "" {
|
|
||||||
t.Fatalf("failed item reached paths = %#v", item)
|
|
||||||
}
|
|
||||||
assertBatchItemMatchesMetadata(t, item)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if item.Status != "succeeded" || item.ReportPath == "" || item.OutputPath == "" {
|
|
||||||
t.Fatalf("continued item %d = %#v", index, item)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunBatchDetailedNotificationLifecycle(t *testing.T) {
|
|
||||||
t.Run("disabled", func(t *testing.T) {
|
|
||||||
cfg := assembledBatchConfig(t, false)
|
|
||||||
bundle := assembledBatchBundle(t, "2026-05-31")
|
|
||||||
notifier := &assembledBatchNotifier{}
|
|
||||||
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
|
||||||
Config: cfg, Batch: BatchEvening, Now: workflowTime("2026-05-29T18:00:00-05:00"),
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &bundle}}, Executor: newAssembledBatchExecutor(), Notifier: notifier,
|
|
||||||
})
|
|
||||||
if err != nil || result.Notification != nil || result.Failed != 0 || len(notifier.reportRequests) != 0 || len(notifier.batchRequests) != 0 {
|
|
||||||
t.Fatalf("result/error/requests = %#v/%v/%d/%d", result, err, len(notifier.reportRequests), len(notifier.batchRequests))
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("batch disabled", func(t *testing.T) {
|
|
||||||
cfg := assembledBatchConfig(t, true)
|
|
||||||
cfg.Notify.Distributor.Batch.Enabled = false
|
|
||||||
bundle := assembledBatchBundle(t, "2026-05-31")
|
|
||||||
notifier := &assembledBatchNotifier{}
|
|
||||||
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
|
||||||
Config: cfg, Batch: BatchEvening, Now: workflowTime("2026-05-29T18:00:00-05:00"),
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &bundle}}, Executor: newAssembledBatchExecutor(), Notifier: notifier,
|
|
||||||
})
|
|
||||||
if err != nil || result.Notification != nil || len(notifier.reportRequests) != 0 || len(notifier.batchRequests) != 0 {
|
|
||||||
t.Fatalf("result/error/requests = %#v/%v/%d/%d", result, err, len(notifier.reportRequests), len(notifier.batchRequests))
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("all success", func(t *testing.T) {
|
|
||||||
cfg := assembledBatchConfig(t, true)
|
|
||||||
bundle := assembledBatchBundle(t, "2026-05-31")
|
|
||||||
notifier := &assembledBatchNotifier{batchResult: &NotificationResult{
|
|
||||||
RunID: "batch-notification-run", Status: "succeeded", UploadStatus: "accepted",
|
|
||||||
Report: []byte(`{"actions":[{"action":"replace_older"}]}`),
|
|
||||||
}}
|
|
||||||
outputDir := filepath.Join(t.TempDir(), "output")
|
|
||||||
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
|
||||||
Config: cfg, Batch: BatchMorning, Now: workflowTime("2026-05-29T08:00:00-05:00"), OutputDir: outputDir,
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &bundle}}, Executor: newAssembledBatchExecutor(), Notifier: notifier,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("RunBatchDetailed() error = %v", err)
|
|
||||||
}
|
|
||||||
if result.Failed != 0 || result.Notification == nil || result.Notification.Status != "succeeded" || result.Notification.Path == "" || len(notifier.reportRequests) != 0 || len(notifier.batchRequests) != 1 {
|
|
||||||
t.Fatalf("notification result/requests = %#v/%d/%d", result.Notification, len(notifier.reportRequests), len(notifier.batchRequests))
|
|
||||||
}
|
|
||||||
managedPaths := make(map[string]struct{}, len(result.Reports))
|
|
||||||
for _, item := range result.Reports {
|
|
||||||
managedPaths[item.ReportPath] = struct{}{}
|
|
||||||
if item.NotificationPath != "" {
|
|
||||||
t.Fatalf("report item contains per-report notification path: %#v", item)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
request := notifier.batchRequests[0]
|
|
||||||
if len(request.IncludedReports) != len(result.Reports) {
|
|
||||||
t.Fatalf("included reports = %d, want %d", len(request.IncludedReports), len(result.Reports))
|
|
||||||
}
|
|
||||||
for _, file := range request.Files {
|
|
||||||
if _, ok := managedPaths[file.SourcePath]; !ok || strings.HasPrefix(file.SourcePath, outputDir+string(filepath.Separator)) || file.BundlePath == "" {
|
|
||||||
t.Fatalf("notification file = %#v, want managed Markdown source", file)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
artifact := readBatchNotificationArtifact(t, result.Notification.Path)
|
|
||||||
if artifact.Status != "succeeded" || artifact.Upload == nil || artifact.Upload.RunID != "batch-notification-run" || artifact.RunStatus == nil || len(artifact.Reports) != len(result.Reports) {
|
|
||||||
t.Fatalf("notification artifact = %#v", artifact)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("upload failure", func(t *testing.T) {
|
|
||||||
cfg := assembledBatchConfig(t, true)
|
|
||||||
bundle := assembledBatchBundle(t, "2026-05-31")
|
|
||||||
notifier := &assembledBatchNotifier{batchErr: errors.New("batch upload rejected")}
|
|
||||||
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
|
||||||
Config: cfg, Batch: BatchEvening, Now: workflowTime("2026-05-29T18:00:00-05:00"),
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &bundle}}, Executor: newAssembledBatchExecutor(), Notifier: notifier,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("RunBatchDetailed() error = %v", err)
|
|
||||||
}
|
|
||||||
if result.Succeeded != 2 || result.Failed != 1 || result.Notification == nil || result.Notification.Status != "failed" || result.Notification.Path == "" {
|
|
||||||
t.Fatalf("result = %#v, want successful reports and failed notification", result)
|
|
||||||
}
|
|
||||||
for _, item := range result.Reports {
|
|
||||||
if item.Status != "succeeded" {
|
|
||||||
t.Fatalf("report item = %#v, want success despite notification failure", item)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
artifact := readBatchNotificationArtifact(t, result.Notification.Path)
|
|
||||||
if artifact.Status != "failed" || !strings.Contains(artifact.Error, "batch upload rejected") {
|
|
||||||
t.Fatalf("notification artifact = %#v", artifact)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("status report", func(t *testing.T) {
|
|
||||||
cfg := assembledBatchConfig(t, true)
|
|
||||||
bundle := assembledBatchBundle(t, "2026-05-31")
|
|
||||||
notifier := &assembledBatchNotifier{batchResult: &NotificationResult{
|
|
||||||
RunID: "batch-notification-run", Status: "accepted", UploadStatus: "accepted",
|
|
||||||
StatusError: "status lookup unavailable", Report: []byte(`{"actions":[{"action":"replace_older"}]}`),
|
|
||||||
}}
|
|
||||||
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
|
||||||
Config: cfg, Batch: BatchEvening, Now: workflowTime("2026-05-29T18:00:00-05:00"),
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &bundle}}, Executor: newAssembledBatchExecutor(), Notifier: notifier,
|
|
||||||
})
|
|
||||||
if err != nil || result.Failed != 0 || result.Notification == nil || result.Notification.Path == "" {
|
|
||||||
t.Fatalf("result/error = %#v/%v", result, err)
|
|
||||||
}
|
|
||||||
artifact := readBatchNotificationArtifact(t, result.Notification.Path)
|
|
||||||
if artifact.StatusError != "status lookup unavailable" || artifact.RunStatus == nil || !bytes.Contains(artifact.RunStatus.Report, []byte("replace_older")) {
|
|
||||||
t.Fatalf("notification artifact = %#v", artifact)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunBatchDetailedKeepsDynamicDailyArtifactsDistinct(t *testing.T) {
|
|
||||||
cfg := assembledBatchConfig(t, false)
|
|
||||||
bundle := assembledBatchBundle(t, "2026-05-31", "2026-06-01")
|
|
||||||
outputDir := filepath.Join(t.TempDir(), "output")
|
|
||||||
debugRoot := filepath.Join(t.TempDir(), "debug")
|
|
||||||
result, err := RunBatchDetailed(context.Background(), BatchRequest{
|
|
||||||
Config: cfg, Batch: BatchEvening, Now: workflowTime("2026-05-29T18:00:00-05:00"), OutputDir: outputDir, LLMDebugDir: debugRoot,
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &bundle}}, Executor: newAssembledBatchExecutor(),
|
|
||||||
})
|
|
||||||
if err != nil || result.Failed != 0 || len(result.Reports) != 3 {
|
|
||||||
t.Fatalf("result/error = %#v/%v", result, err)
|
|
||||||
}
|
|
||||||
seenRuns := make(map[string]struct{})
|
|
||||||
seenDebug := make(map[string]struct{})
|
|
||||||
dailyDates := make(map[string]BatchReportResult)
|
|
||||||
for _, item := range result.Reports {
|
|
||||||
if _, exists := seenRuns[item.RunID]; exists {
|
|
||||||
t.Fatalf("duplicate run ID %q", item.RunID)
|
|
||||||
}
|
|
||||||
seenRuns[item.RunID] = struct{}{}
|
|
||||||
if _, exists := seenDebug[item.LLMDebugPath]; exists {
|
|
||||||
t.Fatalf("duplicate debug path %q", item.LLMDebugPath)
|
|
||||||
}
|
|
||||||
seenDebug[item.LLMDebugPath] = struct{}{}
|
|
||||||
if item.ReportID == report.Daily {
|
|
||||||
date := item.ValidPeriod.Start.In(mustLoadTestLocation(t, "America/Chicago")).Format("2006-01-02")
|
|
||||||
dailyDates[date] = item
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for _, date := range []string{"2026-05-31", "2026-06-01"} {
|
|
||||||
item, ok := dailyDates[date]
|
|
||||||
if !ok {
|
|
||||||
t.Fatalf("daily items = %#v, want %s", dailyDates, date)
|
|
||||||
}
|
|
||||||
if !strings.HasSuffix(item.RunID, "_daily_"+date) || item.OutputPath != filepath.Join(outputDir, "daily-"+date+".md") {
|
|
||||||
t.Fatalf("daily identity/output = %q/%q", item.RunID, item.OutputPath)
|
|
||||||
}
|
|
||||||
wantDebugPrefix := filepath.Join(debugRoot, "daily", date, item.RunID)
|
|
||||||
if item.LLMDebugPath != wantDebugPrefix {
|
|
||||||
t.Fatalf("daily debug path = %q, want %q", item.LLMDebugPath, wantDebugPrefix)
|
|
||||||
}
|
|
||||||
assertBatchPathsExist(t, filepath.Join(item.LLMDebugPath, "preparation.json"), filepath.Join(item.LLMDebugPath, "execution.json"))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunBatchDetailedUsesPriorSnapshotsForPromptPackages(t *testing.T) {
|
|
||||||
cfg := assembledBatchConfig(t, false)
|
|
||||||
firstBundle := assembledBatchBundle(t, "2026-05-31")
|
|
||||||
setWorkflowTemperatures(&firstBundle, 45)
|
|
||||||
first, err := RunBatchDetailed(context.Background(), BatchRequest{
|
|
||||||
Config: cfg, Batch: BatchMorning, Now: workflowTime("2026-05-29T08:00:00-05:00"),
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &firstBundle}}, Executor: newAssembledBatchExecutor(),
|
|
||||||
})
|
|
||||||
if err != nil || first.Failed != 0 {
|
|
||||||
t.Fatalf("first result/error = %#v/%v", first, err)
|
|
||||||
}
|
|
||||||
secondBundle := assembledBatchBundle(t, "2026-05-31")
|
|
||||||
setWorkflowTemperatures(&secondBundle, 85)
|
|
||||||
second, err := RunBatchDetailed(context.Background(), BatchRequest{
|
|
||||||
Config: cfg, Batch: BatchMorning, Now: workflowTime("2026-05-29T08:30:00-05:00"),
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &secondBundle}}, Executor: newAssembledBatchExecutor(),
|
|
||||||
})
|
|
||||||
if err != nil || second.Failed != 0 || len(second.Reports) != len(first.Reports) {
|
|
||||||
t.Fatalf("second result/error = %#v/%v", second, err)
|
|
||||||
}
|
|
||||||
for _, item := range second.Reports {
|
|
||||||
pkg := loadBatchDataPackage(t, item.DataPackagePath)
|
|
||||||
if len(pkg.RecentChanges.Items) == 0 {
|
|
||||||
t.Fatalf("report %s data package has no changes from prior snapshot", item.ReportID)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func assembledBatchConfig(t *testing.T, notify bool) config.Config {
|
|
||||||
t.Helper()
|
|
||||||
cfg := workflowConfig(t)
|
|
||||||
cfg.Notify.Distributor.Enabled = notify
|
|
||||||
cfg.Notify.Distributor.Batch.Enabled = notify
|
|
||||||
return cfg
|
|
||||||
}
|
|
||||||
|
|
||||||
func assembledBatchBundle(t *testing.T, dates ...string) weatherdata.Bundle {
|
|
||||||
t.Helper()
|
|
||||||
bundle := workflowBundle(t)
|
|
||||||
location := mustLoadTestLocation(t, "America/Chicago")
|
|
||||||
for _, date := range dates {
|
|
||||||
periods := fullDayPeriods(t, date, location)
|
|
||||||
for index := range periods {
|
|
||||||
temperature := float64(60 + index)
|
|
||||||
periods[index].TemperatureF = &temperature
|
|
||||||
periods[index].TextDescription = "Partly cloudy"
|
|
||||||
}
|
|
||||||
bundle.Hourly.Periods = append(bundle.Hourly.Periods, periods...)
|
|
||||||
}
|
|
||||||
return bundle
|
|
||||||
}
|
|
||||||
|
|
||||||
func generatedTextForPrompt(promptID string) string {
|
|
||||||
switch promptID {
|
|
||||||
case "weather.today_generated_text":
|
|
||||||
return validTodayWorkflowJSON()
|
|
||||||
case "weather.tomorrow_generated_text":
|
|
||||||
return validTomorrowWorkflowJSON()
|
|
||||||
case "weather.daily_generated_text":
|
|
||||||
return validDailyWorkflowJSON()
|
|
||||||
case "weather.hourly_generated_text":
|
|
||||||
return validHourlyWorkflowJSON()
|
|
||||||
default:
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func assertBatchPathsExist(t *testing.T, paths ...string) {
|
|
||||||
t.Helper()
|
|
||||||
for _, path := range paths {
|
|
||||||
if _, err := os.Stat(path); err != nil {
|
|
||||||
t.Fatalf("expected path %q: %v", path, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func assertBatchItemMatchesMetadata(t *testing.T, item BatchReportResult) {
|
|
||||||
t.Helper()
|
|
||||||
data, err := os.ReadFile(item.MetadataPath)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read metadata %q: %v", item.MetadataPath, err)
|
|
||||||
}
|
|
||||||
var metadata state.Metadata
|
|
||||||
if err := json.Unmarshal(data, &metadata); err != nil {
|
|
||||||
t.Fatalf("decode metadata %q: %v", item.MetadataPath, err)
|
|
||||||
}
|
|
||||||
if item.ReportID != metadata.ReportID || item.RunID != metadata.RunID ||
|
|
||||||
item.DataPackagePath != metadata.DataPackagePath || item.PreparationPath != metadata.PreparationPath ||
|
|
||||||
item.ExecutionPath != metadata.ExecutionPath || item.ReportPath != metadata.RenderedReportPath ||
|
|
||||||
item.NotificationPath != metadata.NotificationPath {
|
|
||||||
t.Fatalf("batch item paths do not exactly match metadata: item=%#v metadata=%#v", item, metadata)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func readBatchNotificationArtifact(t *testing.T, path string) state.BatchDistributorNotificationArtifact {
|
|
||||||
t.Helper()
|
|
||||||
data, err := os.ReadFile(path)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read batch notification artifact: %v", err)
|
|
||||||
}
|
|
||||||
var artifact state.BatchDistributorNotificationArtifact
|
|
||||||
if err := json.Unmarshal(data, &artifact); err != nil {
|
|
||||||
t.Fatalf("decode batch notification artifact: %v", err)
|
|
||||||
}
|
|
||||||
return artifact
|
|
||||||
}
|
|
||||||
|
|
||||||
func loadBatchDataPackage(t *testing.T, path string) promptinput.Package {
|
|
||||||
t.Helper()
|
|
||||||
data, err := os.ReadFile(path)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read data package: %v", err)
|
|
||||||
}
|
|
||||||
pkg, err := promptinput.LoadYAML(data)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("LoadYAML() error = %v", err)
|
|
||||||
}
|
|
||||||
return pkg
|
|
||||||
}
|
|
||||||
288
internal/app/generation_test.go
Normal file
288
internal/app/generation_test.go
Normal file
@@ -0,0 +1,288 @@
|
|||||||
|
package app
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/collect"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
||||||
|
)
|
||||||
|
|
||||||
|
type generationCollector struct {
|
||||||
|
bundle *weatherdata.Bundle
|
||||||
|
err error
|
||||||
|
called bool
|
||||||
|
beforeRun func()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *generationCollector) Run(context.Context, collect.Request) (*collect.Result, error) {
|
||||||
|
if c.beforeRun != nil {
|
||||||
|
c.beforeRun()
|
||||||
|
}
|
||||||
|
c.called = true
|
||||||
|
return &collect.Result{Bundle: c.bundle}, c.err
|
||||||
|
}
|
||||||
|
|
||||||
|
type generationExecutor struct {
|
||||||
|
called bool
|
||||||
|
promptInspections int
|
||||||
|
inspectErr error
|
||||||
|
executeErr error
|
||||||
|
cancelBeforeReturn context.CancelFunc
|
||||||
|
validation promptexec.ValidationStatus
|
||||||
|
rawOutput []byte
|
||||||
|
failedPrompt string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *generationExecutor) InspectPrompt(_ context.Context, id, version string) (promptexec.PromptInspection, error) {
|
||||||
|
e.promptInspections++
|
||||||
|
if e.inspectErr != nil {
|
||||||
|
return promptexec.PromptInspection{}, e.inspectErr
|
||||||
|
}
|
||||||
|
definition := generationDefinitionForPrompt(id)
|
||||||
|
return promptexec.PromptInspection{PromptID: id, PromptVersion: version, PromptHash: "prompt-hash", DefaultProfileID: "fixture", Inputs: []promptexec.InputDefinition{{Name: "data_package", Required: true, ContentType: "application/yaml"}}, Output: promptexec.OutputContract{Format: "json", ValidationMode: "json_schema", SchemaPath: definition.GeneratedTextSchemaID + ".generated_text.schema.json"}}, nil
|
||||||
|
}
|
||||||
|
func (*generationExecutor) InspectProfile(_ context.Context, id string) (promptexec.ProfileInspection, error) {
|
||||||
|
return promptexec.ProfileInspection{ProfileID: id, BackendID: "fixture", ModelName: "fixture-model"}, nil
|
||||||
|
}
|
||||||
|
func (e *generationExecutor) Execute(_ context.Context, req promptexec.ExecuteRequest, callback promptexec.PreparationCallback) (*promptexec.Execution, error) {
|
||||||
|
stamp := time.Date(2026, 5, 29, 15, 0, 0, 0, time.UTC)
|
||||||
|
if err := callback(promptexec.Preparation{PromptID: req.PromptID, PromptVersion: req.PromptVersion, PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash", ProfileID: req.ProfileID, BackendID: "fixture", ModelName: "fixture-model", StartedAt: stamp, EndedAt: stamp}, nil); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
e.called = true
|
||||||
|
if e.executeErr != nil {
|
||||||
|
return nil, e.executeErr
|
||||||
|
}
|
||||||
|
status := e.validation
|
||||||
|
if status == "" {
|
||||||
|
status = promptexec.ValidationPassed
|
||||||
|
}
|
||||||
|
if e.failedPrompt == req.PromptID {
|
||||||
|
status = promptexec.ValidationFailed
|
||||||
|
}
|
||||||
|
rawOutput := e.rawOutput
|
||||||
|
if rawOutput == nil {
|
||||||
|
rawOutput = []byte(`{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast."],"precipitation_timing":"Rain is most likely during the afternoon."}`)
|
||||||
|
}
|
||||||
|
if e.cancelBeforeReturn != nil {
|
||||||
|
e.cancelBeforeReturn()
|
||||||
|
}
|
||||||
|
return &promptexec.Execution{RunID: "provider-run", PromptID: req.PromptID, PromptVersion: req.PromptVersion, PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash", ProfileID: req.ProfileID, BackendID: "fixture", ModelName: "fixture-model", StartedAt: stamp, EndedAt: stamp, RawOutput: rawOutput, Validation: promptexec.NewValidation(status, "json_schema", generationDefinitionForPrompt(req.PromptID).GeneratedTextSchemaID+".generated_text.schema.json", nil)}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func generationDefinitionForPrompt(promptID string) report.Definition {
|
||||||
|
for _, definition := range report.DefaultRegistry().All() {
|
||||||
|
if definition.PromptID == promptID {
|
||||||
|
return definition
|
||||||
|
}
|
||||||
|
}
|
||||||
|
panic("unknown fixture prompt " + promptID)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGenerateDetailedPublishesOnlySelectedOutput(t *testing.T) {
|
||||||
|
cfg := config.Defaults()
|
||||||
|
cfg.WeatherAPI.Timezone, cfg.Location.ID = "America/Chicago", "home"
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
executor := &generationExecutor{}
|
||||||
|
workingDir := t.TempDir()
|
||||||
|
result, err := GenerateDetailed(context.Background(), GenerateRequest{Config: cfg, Report: ReportDaily, Date: generationTime("2026-05-29T12:00:00-05:00"), Now: generationTime("2026-05-29T08:30:00-05:00"), WorkingDir: workingDir, Collector: &generationCollector{bundle: &bundle}, Executor: executor})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GenerateDetailed() error = %v", err)
|
||||||
|
}
|
||||||
|
if !executor.called || result.OutputPath != filepath.Join(workingDir, "daily-2026-05-29.md") || result.ValidationStatus != promptexec.ValidationPassed || result.ProfileID == "" || result.BackendID == "" || result.ModelName == "" {
|
||||||
|
t.Fatalf("result = %#v", result)
|
||||||
|
}
|
||||||
|
if result.LLMDebugPath != "" {
|
||||||
|
t.Fatalf("unexpected debug output = %q", result.LLMDebugPath)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(workingDir, "workspace")); !os.IsNotExist(err) {
|
||||||
|
t.Fatalf("unexpected default state directory: %v", err)
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(result.OutputPath)
|
||||||
|
if err != nil || len(data) == 0 {
|
||||||
|
t.Fatalf("output = %q, error = %v", data, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGenerateDetailedReturnsResolvedResultWhenCollectionFails(t *testing.T) {
|
||||||
|
cfg := config.Defaults()
|
||||||
|
cfg.WeatherAPI.Timezone, cfg.Location.ID = "America/Chicago", "home"
|
||||||
|
collectionErr := errors.New("weather source unavailable")
|
||||||
|
result, err := GenerateDetailed(context.Background(), GenerateRequest{
|
||||||
|
Config: cfg, Report: ReportDaily, Date: generationTime("2026-05-29T12:00:00-05:00"), Now: generationTime("2026-05-29T08:30:00-05:00"),
|
||||||
|
WorkingDir: t.TempDir(), Collector: &generationCollector{err: collectionErr}, Executor: &generationExecutor{},
|
||||||
|
})
|
||||||
|
if !errors.Is(err, collectionErr) {
|
||||||
|
t.Fatalf("GenerateDetailed() error = %v, want %v", err, collectionErr)
|
||||||
|
}
|
||||||
|
if result == nil || result.ReportID != report.Daily || result.RunID == "" || result.ProfileID != "fixture" || result.BackendID != "fixture" || result.ModelName != "fixture-model" || result.OutputPath != "" {
|
||||||
|
t.Fatalf("result = %#v", result)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGenerateDetailedInspectsPromptBeforeCollectingWeather(t *testing.T) {
|
||||||
|
cfg := generationConfig()
|
||||||
|
inspectionErr := errors.New("profile is invalid")
|
||||||
|
collector := &generationCollector{bundle: generationBundlePointer(t)}
|
||||||
|
result, err := GenerateDetailed(context.Background(), GenerateRequest{Config: cfg, Report: ReportDaily, Date: generationTime("2026-05-29T12:00:00-05:00"), Now: generationTime("2026-05-29T08:30:00-05:00"), WorkingDir: t.TempDir(), Collector: collector, Executor: &generationExecutor{inspectErr: inspectionErr}})
|
||||||
|
if !errors.Is(err, inspectionErr) || collector.called || result == nil {
|
||||||
|
t.Fatalf("GenerateDetailed() result/error/collector-called = %#v/%v/%t", result, err, collector.called)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGenerateDetailedPreservesDestinationBeforePublish(t *testing.T) {
|
||||||
|
for _, scenario := range []struct {
|
||||||
|
name string
|
||||||
|
executor generationExecutor
|
||||||
|
}{
|
||||||
|
{name: "generation", executor: generationExecutor{executeErr: errors.New("provider unavailable")}},
|
||||||
|
{name: "render", executor: generationExecutor{rawOutput: []byte(`{"summary":""}`)}},
|
||||||
|
} {
|
||||||
|
t.Run(scenario.name, func(t *testing.T) {
|
||||||
|
outputPath := filepath.Join(t.TempDir(), "daily.md")
|
||||||
|
if err := os.WriteFile(outputPath, []byte("previous report"), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
result, err := GenerateDetailed(context.Background(), GenerateRequest{Config: generationConfig(), Report: ReportDaily, Date: generationTime("2026-05-29T12:00:00-05:00"), Now: generationTime("2026-05-29T08:30:00-05:00"), WorkingDir: t.TempDir(), OutputPath: outputPath, Collector: &generationCollector{bundle: &bundle}, Executor: &scenario.executor})
|
||||||
|
data, readErr := os.ReadFile(outputPath)
|
||||||
|
if err == nil || result == nil || readErr != nil || string(data) != "previous report" {
|
||||||
|
t.Fatalf("GenerateDetailed() result/error/output = %#v/%v/%q (%v)", result, err, data, readErr)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGenerateDetailedPreservesDestinationWhenContextCancelsBeforePublication(t *testing.T) {
|
||||||
|
outputPath := filepath.Join(t.TempDir(), "daily.md")
|
||||||
|
const previousReport = "previous report"
|
||||||
|
if err := os.WriteFile(outputPath, []byte(previousReport), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
result, err := GenerateDetailed(ctx, GenerateRequest{
|
||||||
|
Config: generationConfig(), Report: ReportDaily,
|
||||||
|
Date: generationTime("2026-05-29T12:00:00-05:00"), Now: generationTime("2026-05-29T08:30:00-05:00"),
|
||||||
|
WorkingDir: t.TempDir(), OutputPath: outputPath, Collector: &generationCollector{bundle: &bundle}, Executor: &generationExecutor{cancelBeforeReturn: cancel},
|
||||||
|
})
|
||||||
|
data, readErr := os.ReadFile(outputPath)
|
||||||
|
if !errors.Is(err, context.Canceled) || promptexec.CategoryOf(err) != promptexec.Canceled || result == nil || result.OutputPath != "" || readErr != nil || string(data) != previousReport {
|
||||||
|
t.Fatalf("GenerateDetailed() result/error/output = %#v/%v/%q (%v)", result, err, data, readErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGenerateDetailedPreservesDestinationWhenContextDeadlineExpiresBeforePublication(t *testing.T) {
|
||||||
|
outputPath := filepath.Join(t.TempDir(), "daily.md")
|
||||||
|
const previousReport = "previous report"
|
||||||
|
if err := os.WriteFile(outputPath, []byte(previousReport), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithDeadline(context.Background(), time.Unix(0, 0))
|
||||||
|
defer cancel()
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
result, err := GenerateDetailed(ctx, GenerateRequest{
|
||||||
|
Config: generationConfig(), Report: ReportDaily,
|
||||||
|
Date: generationTime("2026-05-29T12:00:00-05:00"), Now: generationTime("2026-05-29T08:30:00-05:00"),
|
||||||
|
WorkingDir: t.TempDir(), OutputPath: outputPath, Collector: &generationCollector{bundle: &bundle}, Executor: &generationExecutor{},
|
||||||
|
})
|
||||||
|
data, readErr := os.ReadFile(outputPath)
|
||||||
|
if !errors.Is(err, context.DeadlineExceeded) || promptexec.CategoryOf(err) != promptexec.DeadlineExceeded || result == nil || result.OutputPath != "" || readErr != nil || string(data) != previousReport {
|
||||||
|
t.Fatalf("GenerateDetailed() result/error/output = %#v/%v/%q (%v)", result, err, data, readErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGenerateDetailedRetainsPublishedOutputWhenNotificationFails(t *testing.T) {
|
||||||
|
cfg := generationConfig()
|
||||||
|
cfg.Notify.Distributor.Enabled = true
|
||||||
|
cfg.Notify.Distributor.PipelineIDTemplate = "weather"
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
outputPath := filepath.Join(t.TempDir(), "daily.md")
|
||||||
|
notifier := &generationNotifier{err: errors.New("distributor unavailable")}
|
||||||
|
result, err := GenerateDetailed(context.Background(), GenerateRequest{Config: cfg, Report: ReportDaily, Date: generationTime("2026-05-29T12:00:00-05:00"), Now: generationTime("2026-05-29T08:30:00-05:00"), WorkingDir: t.TempDir(), OutputPath: outputPath, Collector: &generationCollector{bundle: &bundle}, Executor: &generationExecutor{}, Notifier: notifier})
|
||||||
|
if err == nil || result == nil || result.OutputPath != outputPath || notifier.request.ReportPath != outputPath || len(notifier.request.BundlePaths) == 0 {
|
||||||
|
t.Fatalf("GenerateDetailed() result/error/request = %#v/%v/%#v", result, err, notifier.request)
|
||||||
|
}
|
||||||
|
if data, readErr := os.ReadFile(outputPath); readErr != nil || len(data) == 0 {
|
||||||
|
t.Fatalf("published output = %q, error = %v", data, readErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGenerateDetailedDoesNotReplaceDirectoryOutput(t *testing.T) {
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
outputPath := filepath.Join(t.TempDir(), "daily.md")
|
||||||
|
if err := os.Mkdir(outputPath, 0o700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
result, err := GenerateDetailed(context.Background(), GenerateRequest{Config: generationConfig(), Report: ReportDaily, Date: generationTime("2026-05-29T12:00:00-05:00"), Now: generationTime("2026-05-29T08:30:00-05:00"), WorkingDir: t.TempDir(), OutputPath: outputPath, Collector: &generationCollector{bundle: &bundle}, Executor: &generationExecutor{}})
|
||||||
|
info, statErr := os.Stat(outputPath)
|
||||||
|
if err == nil || result == nil || statErr != nil || !info.IsDir() {
|
||||||
|
t.Fatalf("GenerateDetailed() result/error/output-info = %#v/%v/%#v (%v)", result, err, info, statErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func generationConfig() config.Config {
|
||||||
|
cfg := config.Defaults()
|
||||||
|
cfg.WeatherAPI.Timezone, cfg.Location.ID = "America/Chicago", "home"
|
||||||
|
return cfg
|
||||||
|
}
|
||||||
|
|
||||||
|
func generationBundlePointer(t *testing.T) *weatherdata.Bundle {
|
||||||
|
bundle := generationBundle(t)
|
||||||
|
return &bundle
|
||||||
|
}
|
||||||
|
|
||||||
|
type generationNotifier struct {
|
||||||
|
err error
|
||||||
|
batchErr error
|
||||||
|
request NotificationRequest
|
||||||
|
batchRequest batchNotificationRequest
|
||||||
|
batchCalls int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (n *generationNotifier) Notify(_ context.Context, request NotificationRequest) (*NotificationResult, error) {
|
||||||
|
n.request = request
|
||||||
|
return nil, n.err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (n *generationNotifier) NotifyBatch(_ context.Context, request batchNotificationRequest) (*NotificationResult, error) {
|
||||||
|
n.batchCalls++
|
||||||
|
n.batchRequest = request
|
||||||
|
for _, file := range request.Files {
|
||||||
|
if _, err := os.Stat(file.SourcePath); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &NotificationResult{Status: "succeeded", PipelineID: request.PipelineID, BundleID: request.BundleID}, n.batchErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func generationBundle(t *testing.T) weatherdata.Bundle {
|
||||||
|
t.Helper()
|
||||||
|
data, err := os.ReadFile(filepath.Join("..", "forecast", "testdata", "daily_bundle.json"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read bundle fixture: %v", err)
|
||||||
|
}
|
||||||
|
var bundle weatherdata.Bundle
|
||||||
|
if err := json.Unmarshal(data, &bundle); err != nil {
|
||||||
|
t.Fatalf("decode bundle fixture: %v", err)
|
||||||
|
}
|
||||||
|
return bundle
|
||||||
|
}
|
||||||
|
func generationTime(value string) time.Time {
|
||||||
|
parsed, _ := time.Parse(time.RFC3339, value)
|
||||||
|
return parsed
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ promptexec.Executor = (*generationExecutor)(nil)
|
||||||
|
var _ Collector = (*generationCollector)(nil)
|
||||||
|
var _ Notifier = (*generationNotifier)(nil)
|
||||||
|
var _ = report.Daily
|
||||||
@@ -1,126 +0,0 @@
|
|||||||
package app
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"fmt"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
|
||||||
)
|
|
||||||
|
|
||||||
type InspectReportsRequest struct {
|
|
||||||
Config config.Config
|
|
||||||
Limit int
|
|
||||||
}
|
|
||||||
|
|
||||||
type InspectRunRequest struct {
|
|
||||||
Config config.Config
|
|
||||||
RunID string
|
|
||||||
}
|
|
||||||
|
|
||||||
type SourceInspection struct {
|
|
||||||
RunID string `json:"runId"`
|
|
||||||
ReportID report.ID `json:"reportId"`
|
|
||||||
SourceLocation string `json:"sourceLocation,omitempty"`
|
|
||||||
Sources []briefing.SourceMetadata `json:"sources,omitempty"`
|
|
||||||
Warnings []weatherdata.SourceWarning `json:"warnings,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func InspectReports(ctx context.Context, req InspectReportsRequest) ([]state.ReportRecord, error) {
|
|
||||||
store, err := defaultStore(req.Config)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
return store.ListReports(ctx, req.Limit)
|
|
||||||
}
|
|
||||||
|
|
||||||
func InspectMetadata(ctx context.Context, req InspectRunRequest) (state.Metadata, error) {
|
|
||||||
inspection, err := inspectRun(ctx, req)
|
|
||||||
return inspection.metadata, err
|
|
||||||
}
|
|
||||||
|
|
||||||
func InspectModules(ctx context.Context, req InspectRunRequest) (module.Snapshot, error) {
|
|
||||||
inspection, err := inspectRun(ctx, req)
|
|
||||||
if err != nil {
|
|
||||||
return module.Snapshot{}, err
|
|
||||||
}
|
|
||||||
return inspection.store.LoadModuleSnapshot(ctx, inspection.metadata.ModuleSnapshotPath)
|
|
||||||
}
|
|
||||||
|
|
||||||
func InspectDataPackage(ctx context.Context, req InspectRunRequest) (promptinput.Package, error) {
|
|
||||||
inspection, err := inspectRun(ctx, req)
|
|
||||||
if err != nil {
|
|
||||||
return promptinput.Package{}, err
|
|
||||||
}
|
|
||||||
return inspection.store.LoadDataPackage(ctx, inspection.metadata.DataPackagePath)
|
|
||||||
}
|
|
||||||
|
|
||||||
func InspectPriorSnapshot(ctx context.Context, req InspectRunRequest) (*state.PriorSnapshot, error) {
|
|
||||||
inspection, err := inspectRun(ctx, req)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
resolved, err := resolvedFromMetadata(inspection.metadata)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
return inspection.store.FindPriorSnapshot(ctx, resolved)
|
|
||||||
}
|
|
||||||
|
|
||||||
func InspectSources(ctx context.Context, req InspectRunRequest) (SourceInspection, error) {
|
|
||||||
inspection, err := inspectRun(ctx, req)
|
|
||||||
if err != nil {
|
|
||||||
return SourceInspection{}, err
|
|
||||||
}
|
|
||||||
metadata := inspection.metadata
|
|
||||||
return SourceInspection{
|
|
||||||
RunID: metadata.RunID,
|
|
||||||
ReportID: metadata.ReportID,
|
|
||||||
SourceLocation: metadata.SourceLocation,
|
|
||||||
Sources: metadata.Sources,
|
|
||||||
Warnings: metadata.SourceWarnings,
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type runInspection struct {
|
|
||||||
store *state.FilesystemStore
|
|
||||||
metadata state.Metadata
|
|
||||||
}
|
|
||||||
|
|
||||||
func inspectRun(ctx context.Context, req InspectRunRequest) (runInspection, error) {
|
|
||||||
store, err := defaultStore(req.Config)
|
|
||||||
if err != nil {
|
|
||||||
return runInspection{}, err
|
|
||||||
}
|
|
||||||
metadata, _, err := store.LoadMetadataByRunID(ctx, req.RunID)
|
|
||||||
if err != nil {
|
|
||||||
return runInspection{}, err
|
|
||||||
}
|
|
||||||
return runInspection{store: store, metadata: metadata}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func resolvedFromMetadata(metadata state.Metadata) (report.Resolved, error) {
|
|
||||||
definition, err := report.DefaultRegistry().Lookup(metadata.ReportID)
|
|
||||||
if err != nil {
|
|
||||||
return report.Resolved{}, err
|
|
||||||
}
|
|
||||||
location, err := timeutil.LoadLocation(metadata.Timezone)
|
|
||||||
if err != nil {
|
|
||||||
return report.Resolved{}, err
|
|
||||||
}
|
|
||||||
if !metadata.ValidPeriod.IsValid() {
|
|
||||||
return report.Resolved{}, fmt.Errorf("metadata valid period for run id %q is invalid", metadata.RunID)
|
|
||||||
}
|
|
||||||
return report.Resolved{
|
|
||||||
Definition: definition,
|
|
||||||
GeneratedAt: metadata.GeneratedAt,
|
|
||||||
Timezone: location.String(),
|
|
||||||
ValidPeriod: metadata.ValidPeriod,
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
@@ -1,552 +0,0 @@
|
|||||||
package app
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/collect"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
|
||||||
failPromptExecution = "prompt execution"
|
|
||||||
failMetadata = "metadata"
|
|
||||||
failGeneratedText = "generated text"
|
|
||||||
failRenderContext = "render context"
|
|
||||||
failRenderedReportPath = "rendered report path"
|
|
||||||
failDistributorNotification = "distributor notification"
|
|
||||||
)
|
|
||||||
|
|
||||||
type failingPersistenceStore struct {
|
|
||||||
state.Store
|
|
||||||
failOperation string
|
|
||||||
failExecutionCall int
|
|
||||||
failMetadataCall int
|
|
||||||
executionCalls int
|
|
||||||
metadataCalls int
|
|
||||||
renderedReportPath string
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *failingPersistenceStore) SavePromptExecution(ctx context.Context, resolved report.Resolved, artifact state.PromptExecutionArtifact) (string, error) {
|
|
||||||
s.executionCalls++
|
|
||||||
if s.failOperation == failPromptExecution && (s.failExecutionCall == 0 || s.executionCalls == s.failExecutionCall) {
|
|
||||||
return "", errors.New("injected prompt execution persistence failure")
|
|
||||||
}
|
|
||||||
return s.Store.SavePromptExecution(ctx, resolved, artifact)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *failingPersistenceStore) SaveGeneratedText(ctx context.Context, resolved report.Resolved, data []byte) (string, error) {
|
|
||||||
if s.failOperation == failGeneratedText {
|
|
||||||
return "", errors.New("injected generated text persistence failure")
|
|
||||||
}
|
|
||||||
return s.Store.SaveGeneratedText(ctx, resolved, data)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *failingPersistenceStore) SaveRenderContext(ctx context.Context, resolved report.Resolved, value any) (string, error) {
|
|
||||||
if s.failOperation == failRenderContext {
|
|
||||||
return "", errors.New("injected render context persistence failure")
|
|
||||||
}
|
|
||||||
return s.Store.SaveRenderContext(ctx, resolved, value)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *failingPersistenceStore) PrepareRenderedReport(ctx context.Context, resolved report.Resolved) (string, error) {
|
|
||||||
if s.failOperation == failRenderedReportPath {
|
|
||||||
return s.renderedReportPath, nil
|
|
||||||
}
|
|
||||||
return s.Store.PrepareRenderedReport(ctx, resolved)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *failingPersistenceStore) SaveDistributorNotification(ctx context.Context, resolved report.Resolved, artifact state.DistributorNotificationArtifact) (string, error) {
|
|
||||||
if s.failOperation == failDistributorNotification {
|
|
||||||
return "", errors.New("injected notification persistence failure")
|
|
||||||
}
|
|
||||||
return s.Store.SaveDistributorNotification(ctx, resolved, artifact)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *failingPersistenceStore) SaveMetadata(ctx context.Context, metadata state.Metadata) (string, error) {
|
|
||||||
s.metadataCalls++
|
|
||||||
if s.failOperation == failMetadata && s.metadataCalls == s.failMetadataCall {
|
|
||||||
return "", errors.New("injected metadata persistence failure")
|
|
||||||
}
|
|
||||||
return s.Store.SaveMetadata(ctx, metadata)
|
|
||||||
}
|
|
||||||
|
|
||||||
type artifactPathExecutor struct {
|
|
||||||
beforePreparationErr error
|
|
||||||
afterPreparationErr error
|
|
||||||
validation promptexec.ValidationStatus
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e artifactPathExecutor) InspectPrompt(context.Context, string, string) (promptexec.PromptInspection, error) {
|
|
||||||
return promptexec.PromptInspection{}, errors.New("unexpected inspection")
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e artifactPathExecutor) InspectProfile(context.Context, string) (promptexec.ProfileInspection, error) {
|
|
||||||
return promptexec.ProfileInspection{}, errors.New("unexpected inspection")
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e artifactPathExecutor) Execute(_ context.Context, req promptexec.ExecuteRequest, callback promptexec.PreparationCallback) (*promptexec.Execution, error) {
|
|
||||||
if e.beforePreparationErr != nil {
|
|
||||||
return nil, e.beforePreparationErr
|
|
||||||
}
|
|
||||||
now := time.Date(2026, 5, 29, 15, 0, 0, 0, time.UTC)
|
|
||||||
if err := callback(promptexec.Preparation{
|
|
||||||
PromptID: req.PromptID, PromptVersion: req.PromptVersion, PromptHash: "prompt-hash",
|
|
||||||
RenderedPromptHash: "rendered-hash", ProfileID: req.ProfileID, BackendID: "test",
|
|
||||||
ModelName: "test-model", DataPackagePath: req.DataPackagePath, StartedAt: now, EndedAt: now,
|
|
||||||
}, nil); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if e.afterPreparationErr != nil {
|
|
||||||
return nil, e.afterPreparationErr
|
|
||||||
}
|
|
||||||
validation := e.validation
|
|
||||||
if validation == "" {
|
|
||||||
validation = promptexec.ValidationPassed
|
|
||||||
}
|
|
||||||
return &promptexec.Execution{
|
|
||||||
RunID: "provider-run", PromptID: req.PromptID, PromptVersion: req.PromptVersion,
|
|
||||||
PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash", ProfileID: req.ProfileID,
|
|
||||||
BackendID: "test", ModelName: "test-model", GeneratedHash: "generated-hash",
|
|
||||||
StartedAt: now, EndedAt: now, DataPackagePath: req.DataPackagePath,
|
|
||||||
RawOutput: []byte(`{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast."],"precipitation_timing":"Rain is most likely during the afternoon."}`),
|
|
||||||
Validation: promptexec.NewValidation(validation, "json_schema", "daily.generated_text.schema.json", nil),
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type successfulNotifier struct{}
|
|
||||||
|
|
||||||
func (successfulNotifier) Notify(context.Context, NotificationRequest) (*NotificationResult, error) {
|
|
||||||
return &NotificationResult{RunID: "notification-run", Status: "succeeded", UploadStatus: "accepted"}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type failingNotifier struct{}
|
|
||||||
|
|
||||||
func (failingNotifier) Notify(context.Context, NotificationRequest) (*NotificationResult, error) {
|
|
||||||
return nil, errors.New("injected notification failure")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGeneratePromptReportReturnsOnlyReachedArtifactPaths(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
failOperation string
|
|
||||||
failMetadataCall int
|
|
||||||
outputCopy bool
|
|
||||||
notify bool
|
|
||||||
want reachedPromptArtifacts
|
|
||||||
}{
|
|
||||||
{name: "preparation then metadata", failOperation: failMetadata, failMetadataCall: 1, want: reachedPromptArtifacts{preparation: true}},
|
|
||||||
{name: "raw output then execution", failOperation: failPromptExecution, want: reachedPromptArtifacts{preparation: true, metadata: true, raw: true}},
|
|
||||||
{name: "execution then metadata", failOperation: failMetadata, failMetadataCall: 2, want: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true}},
|
|
||||||
{name: "normalized output then metadata", failOperation: failMetadata, failMetadataCall: 3, want: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true}},
|
|
||||||
{name: "render context then metadata", failOperation: failMetadata, failMetadataCall: 4, want: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true}},
|
|
||||||
{name: "managed report then metadata", failOperation: failMetadata, failMetadataCall: 5, want: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true}},
|
|
||||||
{name: "output copy then metadata", failOperation: failMetadata, failMetadataCall: 5, outputCopy: true, want: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true, output: true}},
|
|
||||||
{name: "notification then metadata", failOperation: failMetadata, failMetadataCall: 6, notify: true, want: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true, notification: true}},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
req, paths := promptArtifactRequest(t, artifactPathExecutor{})
|
|
||||||
store := &failingPersistenceStore{Store: req.Store, failOperation: test.failOperation, failMetadataCall: test.failMetadataCall}
|
|
||||||
req.Store = store
|
|
||||||
if test.outputCopy {
|
|
||||||
req.OutputPath = filepath.Join(t.TempDir(), "daily.md")
|
|
||||||
paths.output = req.OutputPath
|
|
||||||
}
|
|
||||||
if test.notify {
|
|
||||||
req.Config.Notify.Distributor.Enabled = true
|
|
||||||
req.Config.Notify.Distributor.PipelineIDTemplate = "weatherreporter"
|
|
||||||
req.Notifier = successfulNotifier{}
|
|
||||||
req.noNotify = false
|
|
||||||
}
|
|
||||||
|
|
||||||
result, err := generatePromptReport(context.Background(), req)
|
|
||||||
if err == nil || result == nil {
|
|
||||||
t.Fatalf("generatePromptReport() result/error = %#v/%v, want partial result and failure", result, err)
|
|
||||||
}
|
|
||||||
assertReachedPromptArtifacts(t, result, paths, test.want)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGeneratePromptReportFailureReceiptsExposeReachedPaths(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
executor artifactPathExecutor
|
|
||||||
want reachedPromptArtifacts
|
|
||||||
wantExecutionStatus state.PromptExecutionStatus
|
|
||||||
wantExecutionPaths state.PromptExecutionPaths
|
|
||||||
wantRawExecution bool
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
name: "preparation failure",
|
|
||||||
executor: artifactPathExecutor{beforePreparationErr: promptexec.NewError(promptexec.Generation, "prepare failed", nil)},
|
|
||||||
want: reachedPromptArtifacts{preparation: true, metadata: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "operational execution failure",
|
|
||||||
executor: artifactPathExecutor{afterPreparationErr: promptexec.NewError(promptexec.Generation, "provider failed", nil)},
|
|
||||||
want: reachedPromptArtifacts{preparation: true, execution: true, metadata: true},
|
|
||||||
wantExecutionStatus: state.PromptExecutionFailed,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "completed validation rejection",
|
|
||||||
executor: artifactPathExecutor{validation: promptexec.ValidationFailed},
|
|
||||||
want: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true},
|
|
||||||
wantExecutionStatus: state.PromptExecutionValidationRejected,
|
|
||||||
wantRawExecution: true,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
req, paths := promptArtifactRequest(t, test.executor)
|
|
||||||
if test.wantRawExecution {
|
|
||||||
test.wantExecutionPaths.RawOutputPath = paths.GeneratedTextRaw
|
|
||||||
}
|
|
||||||
result, err := generatePromptReport(context.Background(), req)
|
|
||||||
if err == nil || result == nil {
|
|
||||||
t.Fatalf("generatePromptReport() result/error = %#v/%v, want partial result and failure", result, err)
|
|
||||||
}
|
|
||||||
assertReachedPromptArtifacts(t, result, paths, test.want)
|
|
||||||
if test.wantExecutionStatus != "" {
|
|
||||||
artifact, loadErr := req.Store.LoadPromptExecution(context.Background(), result.ExecutionPath)
|
|
||||||
if loadErr != nil {
|
|
||||||
t.Fatalf("LoadPromptExecution() error = %v", loadErr)
|
|
||||||
}
|
|
||||||
if artifact.Status != test.wantExecutionStatus || artifact.Paths != test.wantExecutionPaths {
|
|
||||||
t.Fatalf("execution outcome/paths = %q/%#v, want %q/%#v", artifact.Status, artifact.Paths, test.wantExecutionStatus, test.wantExecutionPaths)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCompletedExecutionArtifactTracksDownstreamLifecycle(t *testing.T) {
|
|
||||||
req, paths := promptArtifactRequest(t, artifactPathExecutor{})
|
|
||||||
req.OutputPath = filepath.Join(t.TempDir(), "daily.md")
|
|
||||||
paths.output = req.OutputPath
|
|
||||||
req.Config.Notify.Distributor.Enabled = true
|
|
||||||
req.Config.Notify.Distributor.PipelineIDTemplate = "weatherreporter"
|
|
||||||
req.Notifier = successfulNotifier{}
|
|
||||||
req.noNotify = false
|
|
||||||
|
|
||||||
result, err := generatePromptReport(context.Background(), req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("generatePromptReport() error = %v", err)
|
|
||||||
}
|
|
||||||
want := state.PromptExecutionPaths{
|
|
||||||
RawOutputPath: paths.GeneratedTextRaw, GeneratedTextPath: paths.GeneratedText,
|
|
||||||
RenderContextPath: paths.RenderContext, RenderedReportPath: paths.RenderedReport,
|
|
||||||
OutputPath: paths.output, NotificationPath: paths.Notification,
|
|
||||||
}
|
|
||||||
assertPersistedExecutionPaths(t, req.Store, result.ExecutionPath, want)
|
|
||||||
|
|
||||||
data, err := os.ReadFile(result.ExecutionPath)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read execution artifact: %v", err)
|
|
||||||
}
|
|
||||||
text := string(data)
|
|
||||||
for _, forbidden := range []string{
|
|
||||||
"Showers are possible during the selected day", `"rawOutput":`, `"debug":`,
|
|
||||||
`"renderedMessages":`, `"structuredSchema":`, `"endpoint":`, `"parametersJSON":`,
|
|
||||||
"credential", "secret-value",
|
|
||||||
} {
|
|
||||||
if strings.Contains(text, forbidden) {
|
|
||||||
t.Fatalf("execution artifact contains unsafe generated or provider detail %q:\n%s", forbidden, text)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCompletedExecutionArtifactRetainsLastPersistedCheckpoint(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
failOperation string
|
|
||||||
failExecutionCall int
|
|
||||||
failMetadataCall int
|
|
||||||
requestOutput bool
|
|
||||||
failOutputCopy bool
|
|
||||||
notify bool
|
|
||||||
notificationFailure bool
|
|
||||||
wantExecution reachedExecutionArtifacts
|
|
||||||
wantResult reachedPromptArtifacts
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
name: "normalized text write", failOperation: failGeneratedText,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "normalized text checkpoint", failOperation: failPromptExecution, failExecutionCall: 2,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "normalized text metadata", failOperation: failMetadata, failMetadataCall: 3,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "render context write", failOperation: failRenderContext,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "render context checkpoint", failOperation: failPromptExecution, failExecutionCall: 3,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "render context metadata", failOperation: failMetadata, failMetadataCall: 4,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true, renderContext: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "managed report write", failOperation: failRenderedReportPath,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true, renderContext: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "managed report checkpoint", failOperation: failPromptExecution, failExecutionCall: 4,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true, renderContext: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "output copy write", requestOutput: true, failOutputCopy: true,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true, renderContext: true, report: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "output copy checkpoint", failOperation: failPromptExecution, failExecutionCall: 5, requestOutput: true,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true, renderContext: true, report: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true, output: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "output copy metadata", failOperation: failMetadata, failMetadataCall: 5, requestOutput: true,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true, renderContext: true, report: true, output: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true, output: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "notification artifact write", failOperation: failDistributorNotification, requestOutput: true, notify: true,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true, renderContext: true, report: true, output: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true, output: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "notification checkpoint", failOperation: failPromptExecution, failExecutionCall: 6, requestOutput: true, notify: true,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true, renderContext: true, report: true, output: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true, output: true, notification: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "notification metadata", failOperation: failMetadata, failMetadataCall: 6, requestOutput: true, notify: true,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true, renderContext: true, report: true, output: true, notification: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true, output: true, notification: true},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "notification operation", requestOutput: true, notify: true, notificationFailure: true,
|
|
||||||
wantExecution: reachedExecutionArtifacts{raw: true, normalized: true, renderContext: true, report: true, output: true, notification: true},
|
|
||||||
wantResult: reachedPromptArtifacts{preparation: true, execution: true, metadata: true, raw: true, normalized: true, renderContext: true, report: true, output: true, notification: true},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
req, paths := promptArtifactRequest(t, artifactPathExecutor{})
|
|
||||||
store := &failingPersistenceStore{
|
|
||||||
Store: req.Store, failOperation: test.failOperation,
|
|
||||||
failExecutionCall: test.failExecutionCall, failMetadataCall: test.failMetadataCall,
|
|
||||||
}
|
|
||||||
if test.failOperation == failRenderedReportPath {
|
|
||||||
store.renderedReportPath = t.TempDir()
|
|
||||||
}
|
|
||||||
req.Store = store
|
|
||||||
if test.requestOutput {
|
|
||||||
req.OutputPath = filepath.Join(t.TempDir(), "daily.md")
|
|
||||||
paths.output = req.OutputPath
|
|
||||||
}
|
|
||||||
if test.failOutputCopy {
|
|
||||||
blocker := filepath.Join(t.TempDir(), "not-a-directory")
|
|
||||||
if err := os.WriteFile(blocker, []byte("block"), 0o600); err != nil {
|
|
||||||
t.Fatalf("write output blocker: %v", err)
|
|
||||||
}
|
|
||||||
req.OutputPath = filepath.Join(blocker, "daily.md")
|
|
||||||
paths.output = req.OutputPath
|
|
||||||
}
|
|
||||||
if test.notify {
|
|
||||||
req.Config.Notify.Distributor.Enabled = true
|
|
||||||
req.Config.Notify.Distributor.PipelineIDTemplate = "weatherreporter"
|
|
||||||
req.Notifier = successfulNotifier{}
|
|
||||||
req.noNotify = false
|
|
||||||
}
|
|
||||||
if test.notificationFailure {
|
|
||||||
req.Notifier = failingNotifier{}
|
|
||||||
}
|
|
||||||
|
|
||||||
result, err := generatePromptReport(context.Background(), req)
|
|
||||||
if err == nil || result == nil {
|
|
||||||
t.Fatalf("generatePromptReport() result/error = %#v/%v, want partial result and failure", result, err)
|
|
||||||
}
|
|
||||||
assertReachedPromptArtifacts(t, result, paths, test.wantResult)
|
|
||||||
assertPersistedExecutionPaths(t, store, result.ExecutionPath, executionPathsFor(paths, test.wantExecution))
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
type reachedExecutionArtifacts struct {
|
|
||||||
raw bool
|
|
||||||
normalized bool
|
|
||||||
renderContext bool
|
|
||||||
report bool
|
|
||||||
output bool
|
|
||||||
notification bool
|
|
||||||
}
|
|
||||||
|
|
||||||
func executionPathsFor(paths promptArtifactPaths, reached reachedExecutionArtifacts) state.PromptExecutionPaths {
|
|
||||||
result := state.PromptExecutionPaths{}
|
|
||||||
if reached.raw {
|
|
||||||
result.RawOutputPath = paths.GeneratedTextRaw
|
|
||||||
}
|
|
||||||
if reached.normalized {
|
|
||||||
result.GeneratedTextPath = paths.GeneratedText
|
|
||||||
}
|
|
||||||
if reached.renderContext {
|
|
||||||
result.RenderContextPath = paths.RenderContext
|
|
||||||
}
|
|
||||||
if reached.report {
|
|
||||||
result.RenderedReportPath = paths.RenderedReport
|
|
||||||
}
|
|
||||||
if reached.output {
|
|
||||||
result.OutputPath = paths.output
|
|
||||||
}
|
|
||||||
if reached.notification {
|
|
||||||
result.NotificationPath = paths.Notification
|
|
||||||
}
|
|
||||||
return result
|
|
||||||
}
|
|
||||||
|
|
||||||
func assertPersistedExecutionPaths(t *testing.T, store state.Store, path string, want state.PromptExecutionPaths) {
|
|
||||||
t.Helper()
|
|
||||||
artifact, err := store.LoadPromptExecution(context.Background(), path)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("LoadPromptExecution() error = %v", err)
|
|
||||||
}
|
|
||||||
if artifact.Status != state.PromptExecutionSucceeded || artifact.Validation == nil || artifact.Validation.Status != promptexec.ValidationPassed {
|
|
||||||
t.Fatalf("execution outcome changed after downstream write: %#v", artifact)
|
|
||||||
}
|
|
||||||
if artifact.Provenance == nil || artifact.Provenance.RunID != "provider-run" || artifact.Provenance.PromptHash != "prompt-hash" {
|
|
||||||
t.Fatalf("execution provenance changed after downstream write: %#v", artifact.Provenance)
|
|
||||||
}
|
|
||||||
if artifact.Paths != want {
|
|
||||||
t.Fatalf("execution paths = %#v, want %#v", artifact.Paths, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
type promptArtifactPaths struct {
|
|
||||||
state.ArtifactPaths
|
|
||||||
output string
|
|
||||||
}
|
|
||||||
|
|
||||||
type reachedPromptArtifacts struct {
|
|
||||||
preparation bool
|
|
||||||
execution bool
|
|
||||||
metadata bool
|
|
||||||
raw bool
|
|
||||||
normalized bool
|
|
||||||
renderContext bool
|
|
||||||
report bool
|
|
||||||
output bool
|
|
||||||
notification bool
|
|
||||||
}
|
|
||||||
|
|
||||||
func promptArtifactRequest(t *testing.T, executor promptexec.Executor) (promptReportRequest, promptArtifactPaths) {
|
|
||||||
t.Helper()
|
|
||||||
cfg := config.Defaults()
|
|
||||||
cfg.Workspace.Root = t.TempDir()
|
|
||||||
resolved, err := ResolveGenerate(GenerateRequest{
|
|
||||||
Config: cfg, Report: ReportDaily, Date: mustParse("2026-05-29T12:00:00-05:00"),
|
|
||||||
}, mustParse("2026-05-29T05:00:00-05:00"))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("ResolveGenerate() error = %v", err)
|
|
||||||
}
|
|
||||||
bundleData, err := os.ReadFile(filepath.Join("..", "forecast", "testdata", "daily_bundle.json"))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read daily fixture: %v", err)
|
|
||||||
}
|
|
||||||
var bundle weatherdata.Bundle
|
|
||||||
if err := json.Unmarshal(bundleData, &bundle); err != nil {
|
|
||||||
t.Fatalf("decode daily fixture: %v", err)
|
|
||||||
}
|
|
||||||
filesystemStore, err := state.NewFilesystemStore(cfg.Workspace)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
|
||||||
}
|
|
||||||
paths, err := filesystemStore.Paths(resolved)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Paths() error = %v", err)
|
|
||||||
}
|
|
||||||
debugWriter, err := state.NewPromptDebugWriter("")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("NewPromptDebugWriter() error = %v", err)
|
|
||||||
}
|
|
||||||
return promptReportRequest{
|
|
||||||
GenerateRequest: GenerateRequest{Config: cfg, Report: ReportDaily, Executor: executor, Store: filesystemStore},
|
|
||||||
Resolved: resolved, Collection: collect.Result{Bundle: &bundle},
|
|
||||||
Inspection: PromptInspectionResult{
|
|
||||||
PromptID: resolved.Definition.PromptID, PromptVersion: resolved.Definition.PromptVersion,
|
|
||||||
PromptHash: "prompt-hash", ProfileID: "test-profile", BackendID: "test", ModelName: "test-model",
|
|
||||||
},
|
|
||||||
DebugWriter: debugWriter, noNotify: true,
|
|
||||||
}, promptArtifactPaths{ArtifactPaths: paths}
|
|
||||||
}
|
|
||||||
|
|
||||||
func assertReachedPromptArtifacts(t *testing.T, result *ReportResult, paths promptArtifactPaths, want reachedPromptArtifacts) {
|
|
||||||
t.Helper()
|
|
||||||
if result.ModuleSnapshotPath != paths.ModuleSnapshot || result.DataPackagePath != paths.DataPackage {
|
|
||||||
t.Fatalf("base paths = module %q data %q, want %q and %q", result.ModuleSnapshotPath, result.DataPackagePath, paths.ModuleSnapshot, paths.DataPackage)
|
|
||||||
}
|
|
||||||
if result.Metadata.ModuleSnapshotPath != paths.ModuleSnapshot || result.Metadata.DataPackagePath != paths.DataPackage || result.Metadata.MetadataPath != paths.Metadata {
|
|
||||||
t.Fatalf("metadata base paths = %#v, want reached module/data paths and metadata destination", result.Metadata)
|
|
||||||
}
|
|
||||||
checks := []struct {
|
|
||||||
name string
|
|
||||||
got string
|
|
||||||
metadataGot string
|
|
||||||
inMetadata bool
|
|
||||||
path string
|
|
||||||
want bool
|
|
||||||
}{
|
|
||||||
{"preparation", result.PreparationPath, result.Metadata.PreparationPath, true, paths.Preparation, want.preparation},
|
|
||||||
{"execution", result.ExecutionPath, result.Metadata.ExecutionPath, true, paths.Execution, want.execution},
|
|
||||||
{"metadata", result.MetadataPath, "", false, paths.Metadata, want.metadata},
|
|
||||||
{"raw", result.GeneratedTextRawPath, result.Metadata.GeneratedTextRawPath, true, paths.GeneratedTextRaw, want.raw},
|
|
||||||
{"normalized", result.GeneratedTextPath, result.Metadata.GeneratedTextPath, true, paths.GeneratedText, want.normalized},
|
|
||||||
{"render context", result.RenderContextPath, result.Metadata.RenderContextPath, true, paths.RenderContext, want.renderContext},
|
|
||||||
{"report", result.ReportPath, result.Metadata.RenderedReportPath, true, paths.RenderedReport, want.report},
|
|
||||||
{"output", result.OutputPath, "", false, paths.output, want.output},
|
|
||||||
{"notification", result.NotificationPath, result.Metadata.NotificationPath, true, paths.Notification, want.notification},
|
|
||||||
}
|
|
||||||
for _, check := range checks {
|
|
||||||
if check.want && check.got != check.path {
|
|
||||||
t.Errorf("%s path = %q, want reached path %q", check.name, check.got, check.path)
|
|
||||||
}
|
|
||||||
if check.want && check.inMetadata && check.metadataGot != check.path {
|
|
||||||
t.Errorf("metadata %s path = %q, want reached path %q", check.name, check.metadataGot, check.path)
|
|
||||||
}
|
|
||||||
if !check.want && check.got != "" {
|
|
||||||
t.Errorf("%s path = %q, want empty because artifact was not reached", check.name, check.got)
|
|
||||||
}
|
|
||||||
if !check.want && check.inMetadata && check.metadataGot != "" {
|
|
||||||
t.Errorf("metadata %s path = %q, want empty because artifact was not reached", check.name, check.metadataGot)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -2,18 +2,19 @@ package app
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/collect"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/collect"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/fileutil"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/fileutil"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/generatedtext"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/generatedtext"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptdebug"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
||||||
)
|
)
|
||||||
|
|
||||||
type promptReportRequest struct {
|
type promptReportRequest struct {
|
||||||
@@ -21,10 +22,24 @@ type promptReportRequest struct {
|
|||||||
Resolved report.Resolved
|
Resolved report.Resolved
|
||||||
Collection collect.Result
|
Collection collect.Result
|
||||||
Inspection PromptInspectionResult
|
Inspection PromptInspectionResult
|
||||||
DebugWriter *state.PromptDebugWriter
|
DebugWriter *promptdebug.PromptDebugWriter
|
||||||
|
Result *ReportResult
|
||||||
noNotify bool
|
noNotify bool
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type promptReportWorkflow struct {
|
||||||
|
ctx context.Context
|
||||||
|
req promptReportRequest
|
||||||
|
result *ReportResult
|
||||||
|
briefingMetadata briefing.Metadata
|
||||||
|
reportFacts ReportFacts
|
||||||
|
moduleSnapshot module.Snapshot
|
||||||
|
dataPackage []byte
|
||||||
|
handler generatedtext.Handler
|
||||||
|
debugRef promptdebug.PromptDebugRef
|
||||||
|
callbackFailed bool
|
||||||
|
}
|
||||||
|
|
||||||
func generatePromptReport(ctx context.Context, req promptReportRequest) (*ReportResult, error) {
|
func generatePromptReport(ctx context.Context, req promptReportRequest) (*ReportResult, error) {
|
||||||
workflow, err := newPromptReportWorkflow(ctx, req)
|
workflow, err := newPromptReportWorkflow(ctx, req)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -33,380 +48,148 @@ func generatePromptReport(ctx context.Context, req promptReportRequest) (*Report
|
|||||||
if err := workflow.buildInputs(); err != nil {
|
if err := workflow.buildInputs(); err != nil {
|
||||||
return workflow.result, err
|
return workflow.result, err
|
||||||
}
|
}
|
||||||
|
execution, err := workflow.executePrompt()
|
||||||
execution, executeErr := workflow.executePrompt()
|
if err != nil {
|
||||||
if executeErr != nil {
|
if workflow.callbackFailed {
|
||||||
return workflow.result, workflow.handleExecutionFailure(executeErr)
|
return workflow.result, err
|
||||||
|
}
|
||||||
|
return workflow.result, workflow.reportError("execute prompt", classifiedPromptError("prompt execution failed", err))
|
||||||
}
|
}
|
||||||
if execution == nil {
|
if execution == nil {
|
||||||
err := promptexec.NewError(promptexec.Generation, "prompt executor returned no execution", nil)
|
return workflow.result, workflow.reportError("execute prompt", promptexec.NewError(promptexec.Generation, "prompt executor returned no execution", nil))
|
||||||
if saveErr := workflow.persistOperationalExecutionFailure(err); saveErr != nil {
|
|
||||||
return workflow.result, saveErr
|
|
||||||
}
|
}
|
||||||
return workflow.result, workflow.reportError("execute prompt", err)
|
workflow.result.ValidationStatus = execution.Validation.Status
|
||||||
}
|
if err := workflow.writeExecutionDebug(*execution); err != nil {
|
||||||
if err := workflow.persistExecutionDebug(*execution); err != nil {
|
|
||||||
return workflow.result, err
|
return workflow.result, err
|
||||||
}
|
}
|
||||||
|
|
||||||
if execution.Validation.Status != promptexec.ValidationPassed && execution.Validation.Status != promptexec.ValidationFailed {
|
if execution.Validation.Status != promptexec.ValidationPassed && execution.Validation.Status != promptexec.ValidationFailed {
|
||||||
err := promptexec.NewError(promptexec.OperationalValidation, "prompt execution did not complete validation", nil)
|
return workflow.result, workflow.reportError("validate prompt execution", promptexec.NewError(promptexec.OperationalValidation, "prompt execution did not complete validation", nil))
|
||||||
if saveErr := workflow.persistOperationalExecutionFailure(err); saveErr != nil {
|
|
||||||
return workflow.result, saveErr
|
|
||||||
}
|
|
||||||
return workflow.result, workflow.reportError("validate prompt execution", err)
|
|
||||||
}
|
|
||||||
if err := workflow.persistCompletedExecution(*execution); err != nil {
|
|
||||||
return workflow.result, err
|
|
||||||
}
|
}
|
||||||
if execution.Validation.Status == promptexec.ValidationFailed {
|
if execution.Validation.Status == promptexec.ValidationFailed {
|
||||||
err := promptexec.NewError(promptexec.ValidationRejected, "prompt output did not satisfy its schema", nil)
|
return workflow.result, workflow.reportError("validate prompt execution", promptexec.NewError(promptexec.ValidationRejected, "prompt output did not satisfy its schema", nil))
|
||||||
return workflow.result, workflow.reportError("validate prompt execution", err)
|
|
||||||
}
|
}
|
||||||
|
return workflow.renderAndPublish(execution.RawOutput)
|
||||||
rendered, err := workflow.persistGeneratedContent(execution.RawOutput)
|
|
||||||
if err != nil {
|
|
||||||
return workflow.result, err
|
|
||||||
}
|
|
||||||
return workflow.finalizeReport(rendered)
|
|
||||||
}
|
|
||||||
|
|
||||||
type promptReportWorkflow struct {
|
|
||||||
ctx context.Context
|
|
||||||
req promptReportRequest
|
|
||||||
store state.Store
|
|
||||||
result *ReportResult
|
|
||||||
metadata state.Metadata
|
|
||||||
briefingMetadata briefing.Metadata
|
|
||||||
reportFacts ReportFacts
|
|
||||||
moduleSnapshot module.Snapshot
|
|
||||||
dataPackageBytes []byte
|
|
||||||
handler generatedtext.Handler
|
|
||||||
executionArtifact state.PromptExecutionArtifact
|
|
||||||
debugRef state.PromptDebugRef
|
|
||||||
prepared bool
|
|
||||||
callbackFailed bool
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func newPromptReportWorkflow(ctx context.Context, req promptReportRequest) (*promptReportWorkflow, error) {
|
func newPromptReportWorkflow(ctx context.Context, req promptReportRequest) (*promptReportWorkflow, error) {
|
||||||
if req.Collection.Bundle == nil {
|
if req.Collection.Bundle == nil {
|
||||||
return nil, fmt.Errorf("collected weather bundle is required")
|
return nil, fmt.Errorf("collected weather bundle is required")
|
||||||
}
|
}
|
||||||
store := req.Store
|
result := req.Result
|
||||||
var err error
|
if result == nil {
|
||||||
if store == nil {
|
result = initialReportResult(req.GenerateRequest, req.Resolved, req.Inspection)
|
||||||
store, err = defaultStore(req.Config)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
}
|
||||||
|
return &promptReportWorkflow{
|
||||||
|
ctx: ctx, req: req,
|
||||||
|
result: result,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func initialReportResult(req GenerateRequest, resolved report.Resolved, inspection PromptInspectionResult) *ReportResult {
|
||||||
|
metadata := resolved.Metadata()
|
||||||
|
return &ReportResult{
|
||||||
|
ReportID: resolved.Definition.ID, ReportName: resolved.Definition.Name,
|
||||||
|
PromptID: resolved.Definition.PromptID, PromptVersion: resolved.Definition.PromptVersion,
|
||||||
|
RunID: metadata.RunID, GeneratedAt: metadata.GeneratedAt, Timezone: req.Config.WeatherAPI.Timezone,
|
||||||
|
ValidPeriod: metadata.ValidPeriod,
|
||||||
|
ProfileID: inspection.ProfileID, BackendID: inspection.BackendID, ModelName: inspection.ModelName,
|
||||||
}
|
}
|
||||||
return &promptReportWorkflow{ctx: ctx, req: req, store: store}, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (w *promptReportWorkflow) buildInputs() error {
|
func (w *promptReportWorkflow) buildInputs() error {
|
||||||
paths, err := w.store.Paths(w.req.Resolved)
|
var err error
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
w.result = &ReportResult{}
|
|
||||||
priorSnapshot, err := w.store.FindPriorSnapshot(w.ctx, w.req.Resolved)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
w.reportFacts, err = BuildReportFacts(ModuleSnapshotRequest{Config: w.req.Config, Resolved: w.req.Resolved}, w.req.Collection.Bundle)
|
w.reportFacts, err = BuildReportFacts(ModuleSnapshotRequest{Config: w.req.Config, Resolved: w.req.Resolved}, w.req.Collection.Bundle)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return generatedReportError(w.req.Resolved, w.req.Resolved.Metadata().RunID, "build report facts", err)
|
return w.reportError("build report facts", err)
|
||||||
}
|
}
|
||||||
w.moduleSnapshot, err = BuildModuleSnapshotFromFacts(ModuleSnapshotRequest{Config: w.req.Config, Resolved: w.req.Resolved}, w.reportFacts)
|
w.moduleSnapshot, err = BuildModuleSnapshotFromFacts(ModuleSnapshotRequest{Config: w.req.Config, Resolved: w.req.Resolved}, w.reportFacts)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return generatedReportError(w.req.Resolved, w.req.Resolved.Metadata().RunID, "build module snapshot", err)
|
return w.reportError("build module snapshot", err)
|
||||||
}
|
}
|
||||||
moduleSnapshotPath, err := w.store.SaveModuleSnapshot(w.ctx, w.req.Resolved, w.moduleSnapshot)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
w.result.ModuleSnapshot = w.moduleSnapshot
|
|
||||||
w.result.ModuleSnapshotPath = moduleSnapshotPath
|
|
||||||
w.result.PriorSnapshot = priorSnapshot
|
|
||||||
|
|
||||||
recent, err := recentChanges(w.ctx, w.store, priorSnapshot, w.req.Resolved.Definition.ID, w.moduleSnapshot, w.req.Config.RecentChange)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
w.result.RecentChanges = recent
|
|
||||||
w.briefingMetadata = briefing.BuildMetadata(briefingBuildContext(w.req.Config, w.req.Resolved, w.reportFacts.Collected))
|
w.briefingMetadata = briefing.BuildMetadata(briefingBuildContext(w.req.Config, w.req.Resolved, w.reportFacts.Collected))
|
||||||
w.metadata = state.BuildPromptMetadataFromBriefingMetadata(w.req.Resolved, w.briefingMetadata, state.ArtifactPaths{
|
w.result.SourceWarnings = append([]weatherdata.SourceWarning(nil), w.briefingMetadata.SourceWarnings...)
|
||||||
ModuleSnapshot: moduleSnapshotPath,
|
dataPackage, err := promptinput.Build(promptinput.BuildRequest{Metadata: promptMetadata(w.briefingMetadata), Modules: w.moduleSnapshot})
|
||||||
Metadata: paths.Metadata,
|
|
||||||
})
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
dataPackage, err := promptinput.Build(promptinput.BuildRequest{
|
|
||||||
Metadata: promptMetadata(w.metadata), Modules: w.moduleSnapshot, RecentChanges: recent,
|
|
||||||
})
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return w.reportError("build data package", err)
|
return w.reportError("build data package", err)
|
||||||
}
|
}
|
||||||
w.dataPackageBytes, err = promptinput.MarshalYAML(dataPackage)
|
w.dataPackage, err = promptinput.MarshalYAML(dataPackage)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return w.reportError("marshal data package", err)
|
||||||
}
|
}
|
||||||
dataPackagePath, err := w.store.SaveDataPackageBytes(w.ctx, w.req.Resolved, w.dataPackageBytes)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
w.metadata.DataPackagePath = dataPackagePath
|
|
||||||
w.result.DataPackage = dataPackage
|
|
||||||
w.result.DataPackagePath = dataPackagePath
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
w.handler, err = generatedtext.LookupDefinition(w.req.Resolved.Definition)
|
w.handler, err = generatedtext.LookupDefinition(w.req.Resolved.Definition)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return w.reportError("lookup generated text catalog", err)
|
return w.reportError("lookup generated text catalog", err)
|
||||||
}
|
}
|
||||||
w.debugRef = state.PromptDebugRef{
|
w.debugRef = promptdebug.PromptDebugRef{ReportID: w.result.ReportID, ValidDate: w.req.Resolved.ValidPeriod.Start.Format("2006-01-02"), RunID: w.result.RunID}
|
||||||
ReportID: w.req.Resolved.Definition.ID, ValidDate: w.req.Resolved.ValidPeriod.Start.Format("2006-01-02"), RunID: w.metadata.RunID,
|
|
||||||
}
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (w *promptReportWorkflow) executePrompt() (*promptexec.Execution, error) {
|
func (w *promptReportWorkflow) executePrompt() (*promptexec.Execution, error) {
|
||||||
return w.req.Executor.Execute(w.ctx, promptexec.ExecuteRequest{
|
captureDebug := w.req.DebugWriter != nil && w.req.DebugWriter.Enabled()
|
||||||
PromptID: w.req.Inspection.PromptID, PromptVersion: w.req.Inspection.PromptVersion,
|
return w.req.Executor.Execute(w.ctx, promptexec.ExecuteRequest{PromptID: w.req.Inspection.PromptID, PromptVersion: w.req.Inspection.PromptVersion, ProfileID: w.req.Inspection.ProfileID, DataPackage: w.dataPackage, CaptureDebug: captureDebug}, w.writePreparationDebug)
|
||||||
ProfileID: w.req.Inspection.ProfileID, DataPackage: w.dataPackageBytes,
|
|
||||||
DataPackagePath: w.result.DataPackagePath, CaptureDebug: w.req.DebugWriter.Enabled(),
|
|
||||||
}, w.persistPreparation)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (w *promptReportWorkflow) persistPreparation(preparation promptexec.Preparation, debug *promptexec.PreparationDebug) error {
|
func (w *promptReportWorkflow) writePreparationDebug(preparation promptexec.Preparation, debug *promptexec.PreparationDebug) error {
|
||||||
artifact := state.PromptPreparationArtifact{
|
w.result.ProfileID, w.result.BackendID, w.result.ModelName = preparation.ProfileID, preparation.BackendID, preparation.ModelName
|
||||||
SchemaVersion: state.PromptPreparationSchemaVersion, Status: state.PromptPreparationSucceeded,
|
if w.req.DebugWriter == nil {
|
||||||
ReportID: w.req.Resolved.Definition.ID, RunID: w.metadata.RunID,
|
return nil
|
||||||
PromptID: w.req.Inspection.PromptID, PromptVersion: w.req.Inspection.PromptVersion,
|
|
||||||
DataPackagePath: w.result.DataPackagePath, Preparation: &preparation,
|
|
||||||
StartedAt: preparation.StartedAt, EndedAt: preparation.EndedAt, Duration: preparation.Duration,
|
|
||||||
}
|
}
|
||||||
path, err := w.store.SavePromptPreparation(w.ctx, w.req.Resolved, artifact)
|
path, err := w.req.DebugWriter.WritePreparation(w.debugRef, preparation, debug)
|
||||||
if err != nil {
|
|
||||||
w.callbackFailed = true
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
w.prepared = true
|
|
||||||
w.result.PreparationPath = path
|
|
||||||
w.metadata.PreparationPath = path
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
debugPath, err := w.req.DebugWriter.WritePreparation(w.debugRef, preparation, debug)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
w.callbackFailed = true
|
w.callbackFailed = true
|
||||||
return promptDebugWriteError(err)
|
return promptDebugWriteError(err)
|
||||||
}
|
}
|
||||||
if debugPath != "" {
|
w.result.LLMDebugPath = path
|
||||||
w.result.LLMDebugPath = debugPath
|
|
||||||
}
|
|
||||||
if err := w.saveMetadata(); err != nil {
|
|
||||||
w.callbackFailed = true
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (w *promptReportWorkflow) handleExecutionFailure(executeErr error) error {
|
func (w *promptReportWorkflow) writeExecutionDebug(execution promptexec.Execution) error {
|
||||||
if w.callbackFailed {
|
if w.req.DebugWriter == nil {
|
||||||
return executeErr
|
return nil
|
||||||
}
|
}
|
||||||
executeErr = classifiedPromptError("prompt execution failed", executeErr)
|
path, err := w.req.DebugWriter.WriteExecution(w.debugRef, execution)
|
||||||
if !w.prepared {
|
|
||||||
if err := w.persistPreparationFailure(executeErr); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return w.reportError("prepare prompt", executeErr)
|
|
||||||
}
|
|
||||||
if promptexec.CategoryOf(executeErr) != "" {
|
|
||||||
if err := w.persistOperationalExecutionFailure(executeErr); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return w.reportError("execute prompt", executeErr)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (w *promptReportWorkflow) persistPreparationFailure(executeErr error) error {
|
|
||||||
startedAt, endedAt := time.Now(), time.Now()
|
|
||||||
artifact := state.PromptPreparationArtifact{
|
|
||||||
SchemaVersion: state.PromptPreparationSchemaVersion, Status: state.PromptPreparationFailed,
|
|
||||||
ReportID: w.req.Resolved.Definition.ID, RunID: w.metadata.RunID,
|
|
||||||
PromptID: w.req.Inspection.PromptID, PromptVersion: w.req.Inspection.PromptVersion,
|
|
||||||
DataPackagePath: w.result.DataPackagePath, StartedAt: startedAt, EndedAt: endedAt,
|
|
||||||
Error: state.NewPromptArtifactError(executeErr),
|
|
||||||
}
|
|
||||||
path, err := w.store.SavePromptPreparation(w.ctx, w.req.Resolved, artifact)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
w.result.PreparationPath = path
|
|
||||||
w.metadata.PreparationPath = path
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
return w.saveMetadata()
|
|
||||||
}
|
|
||||||
|
|
||||||
func (w *promptReportWorkflow) persistOperationalExecutionFailure(executeErr error) error {
|
|
||||||
artifact := failedPromptExecutionArtifact(w.req.Resolved, w.metadata, w.req.Inspection, executeErr)
|
|
||||||
path, err := w.store.SavePromptExecution(w.ctx, w.req.Resolved, artifact)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
w.result.ExecutionPath = path
|
|
||||||
w.metadata.ExecutionPath = path
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
return w.saveMetadata()
|
|
||||||
}
|
|
||||||
|
|
||||||
func (w *promptReportWorkflow) persistExecutionDebug(execution promptexec.Execution) error {
|
|
||||||
debugPath, err := w.req.DebugWriter.WriteExecution(w.debugRef, execution)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return w.reportError("write prompt debug", promptDebugWriteError(err))
|
return w.reportError("write prompt debug", promptDebugWriteError(err))
|
||||||
}
|
}
|
||||||
if debugPath != "" {
|
if path != "" {
|
||||||
w.result.LLMDebugPath = debugPath
|
w.result.LLMDebugPath = path
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (w *promptReportWorkflow) persistCompletedExecution(execution promptexec.Execution) error {
|
func (w *promptReportWorkflow) renderAndPublish(raw []byte) (*ReportResult, error) {
|
||||||
rawPath, err := w.store.SaveGeneratedTextRaw(w.ctx, w.req.Resolved, execution.RawOutput)
|
generatedText, _, err := w.handler.Validate(raw)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return w.result, w.reportError("validate generated text", err)
|
||||||
}
|
}
|
||||||
w.result.GeneratedTextRawPath = rawPath
|
|
||||||
w.metadata.GeneratedTextRawPath = rawPath
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
w.executionArtifact = state.PromptExecutionArtifact{
|
|
||||||
SchemaVersion: state.PromptExecutionSchemaVersion,
|
|
||||||
ReportID: w.req.Resolved.Definition.ID, RunID: w.metadata.RunID,
|
|
||||||
PromptID: w.req.Inspection.PromptID, PromptVersion: w.req.Inspection.PromptVersion,
|
|
||||||
Provenance: ptr(state.PromptExecutionProvenanceFrom(execution)), Validation: &execution.Validation,
|
|
||||||
Paths: state.PromptExecutionPaths{RawOutputPath: rawPath},
|
|
||||||
StartedAt: execution.StartedAt, EndedAt: execution.EndedAt, Duration: execution.Duration,
|
|
||||||
}
|
|
||||||
if execution.Validation.Status == promptexec.ValidationPassed {
|
|
||||||
w.executionArtifact.Status = state.PromptExecutionSucceeded
|
|
||||||
} else {
|
|
||||||
w.executionArtifact.Status = state.PromptExecutionValidationRejected
|
|
||||||
}
|
|
||||||
executionPath, err := w.store.SavePromptExecution(w.ctx, w.req.Resolved, w.executionArtifact)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
w.result.ExecutionPath = executionPath
|
|
||||||
w.metadata.ExecutionPath = executionPath
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
return w.saveMetadata()
|
|
||||||
}
|
|
||||||
|
|
||||||
func (w *promptReportWorkflow) persistGeneratedContent(raw []byte) ([]byte, error) {
|
|
||||||
generatedText, normalized, err := w.handler.Validate(raw)
|
|
||||||
if err != nil {
|
|
||||||
return nil, w.reportError("validate generated text", err)
|
|
||||||
}
|
|
||||||
generatedTextPath, err := w.store.SaveGeneratedText(w.ctx, w.req.Resolved, normalized)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
w.result.GeneratedTextPath = generatedTextPath
|
|
||||||
w.metadata.GeneratedTextPath = generatedTextPath
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
if err := w.persistReachedPathAndMetadata(func(paths *state.PromptExecutionPaths) { paths.GeneratedTextPath = generatedTextPath }); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
renderContext, err := w.handler.BuildRenderContext(w.briefingMetadata, w.moduleSnapshot, w.reportFacts.Collected, w.reportFacts.Derived, generatedText)
|
renderContext, err := w.handler.BuildRenderContext(w.briefingMetadata, w.moduleSnapshot, w.reportFacts.Collected, w.reportFacts.Derived, generatedText)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, w.reportError("build render context", err)
|
return w.result, w.reportError("build render context", err)
|
||||||
}
|
}
|
||||||
renderContextPath, err := w.store.SaveRenderContext(w.ctx, w.req.Resolved, renderContext)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
w.result.RenderContextPath = renderContextPath
|
|
||||||
w.metadata.RenderContextPath = renderContextPath
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
if err := w.persistReachedPathAndMetadata(func(paths *state.PromptExecutionPaths) { paths.RenderContextPath = renderContextPath }); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
rendered, err := w.handler.Render(renderContext)
|
rendered, err := w.handler.Render(renderContext)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, w.reportError("render template", err)
|
return w.result, w.reportError("render template", err)
|
||||||
}
|
}
|
||||||
return rendered, nil
|
if err := publicationContextError(w.ctx); err != nil {
|
||||||
|
return w.result, w.reportError("publish report", err)
|
||||||
}
|
}
|
||||||
|
if err := fileutil.WriteFileAtomic(w.req.OutputPath, rendered); err != nil {
|
||||||
func (w *promptReportWorkflow) finalizeReport(rendered []byte) (*ReportResult, error) {
|
return w.result, err
|
||||||
reportPath, err := w.store.PrepareRenderedReport(w.ctx, w.req.Resolved)
|
}
|
||||||
|
w.result.OutputPath = w.req.OutputPath
|
||||||
|
if w.req.noNotify {
|
||||||
|
return w.result, nil
|
||||||
|
}
|
||||||
|
notification, err := notifyReport(w.ctx, w.req.Config, w.req.Resolved, w.result.OutputPath, w.result.RunID, w.result.GeneratedAt, w.req.Notifier)
|
||||||
|
w.result.Notification = notification
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return w.result, err
|
return w.result, err
|
||||||
}
|
}
|
||||||
if err := fileutil.WriteFileAtomic(reportPath, rendered); err != nil {
|
return w.result, nil
|
||||||
return w.result, err
|
|
||||||
}
|
|
||||||
w.result.ReportPath = reportPath
|
|
||||||
w.metadata.RenderedReportPath = reportPath
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
if err := w.persistReachedPath(func(paths *state.PromptExecutionPaths) { paths.RenderedReportPath = reportPath }); err != nil {
|
|
||||||
return w.result, err
|
|
||||||
}
|
|
||||||
finalized, err := finalizeRenderedReport(w.ctx, finalizeRenderedReportRequest{
|
|
||||||
Config: w.req.Config, Store: w.store, Resolved: w.req.Resolved, Metadata: w.metadata, MetadataPath: w.result.MetadataPath,
|
|
||||||
ExecutionArtifact: &w.executionArtifact, ManagedReportPath: reportPath, OutputPath: w.req.OutputPath,
|
|
||||||
Notifier: w.req.Notifier, noNotify: w.req.noNotify,
|
|
||||||
})
|
|
||||||
w.result.OutputPath, w.result.NotificationPath = finalized.OutputPath, finalized.NotificationPath
|
|
||||||
w.result.Metadata, w.result.MetadataPath, w.result.Notification = finalized.Metadata, finalized.MetadataPath, finalized.Notification
|
|
||||||
return w.result, err
|
|
||||||
}
|
|
||||||
|
|
||||||
func (w *promptReportWorkflow) persistReachedPath(update func(*state.PromptExecutionPaths)) error {
|
|
||||||
return persistReachedPromptPath(w.ctx, w.store, w.req.Resolved, &w.executionArtifact, update)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (w *promptReportWorkflow) persistReachedPathAndMetadata(update func(*state.PromptExecutionPaths)) error {
|
|
||||||
if err := w.persistReachedPath(update); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return w.saveMetadata()
|
|
||||||
}
|
|
||||||
|
|
||||||
func (w *promptReportWorkflow) saveMetadata() error {
|
|
||||||
path, err := w.store.SaveMetadata(w.ctx, w.metadata)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
w.result.Metadata = w.metadata
|
|
||||||
w.result.MetadataPath = path
|
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (w *promptReportWorkflow) reportError(operation string, err error) error {
|
func (w *promptReportWorkflow) reportError(operation string, err error) error {
|
||||||
return generatedReportError(w.req.Resolved, w.metadata.RunID, operation, err)
|
return generatedReportError(w.req.Resolved, w.result.RunID, operation, err)
|
||||||
}
|
|
||||||
|
|
||||||
func persistReachedPromptPath(
|
|
||||||
ctx context.Context,
|
|
||||||
store state.Store,
|
|
||||||
resolved report.Resolved,
|
|
||||||
artifact *state.PromptExecutionArtifact,
|
|
||||||
update func(*state.PromptExecutionPaths),
|
|
||||||
) error {
|
|
||||||
update(&artifact.Paths)
|
|
||||||
_, err := store.SavePromptExecution(ctx, resolved, *artifact)
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
func failedPromptExecutionArtifact(resolved report.Resolved, metadata state.Metadata, inspection PromptInspectionResult, err error) state.PromptExecutionArtifact {
|
|
||||||
now := time.Now()
|
|
||||||
return state.PromptExecutionArtifact{
|
|
||||||
SchemaVersion: state.PromptExecutionSchemaVersion, Status: state.PromptExecutionFailed,
|
|
||||||
ReportID: resolved.Definition.ID, RunID: metadata.RunID, PromptID: inspection.PromptID,
|
|
||||||
PromptVersion: inspection.PromptVersion, StartedAt: now, EndedAt: now,
|
|
||||||
Error: state.NewPromptArtifactError(err),
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func classifiedPromptError(operation string, err error) error {
|
func classifiedPromptError(operation string, err error) error {
|
||||||
@@ -416,8 +199,16 @@ func classifiedPromptError(operation string, err error) error {
|
|||||||
return promptexec.NewError(promptexec.Generation, operation, err)
|
return promptexec.NewError(promptexec.Generation, operation, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func publicationContextError(ctx context.Context) error {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
if errors.Is(err, context.DeadlineExceeded) {
|
||||||
|
return promptexec.NewError(promptexec.DeadlineExceeded, "context expired before output publication", err)
|
||||||
|
}
|
||||||
|
return promptexec.NewError(promptexec.Canceled, "context canceled before output publication", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
func promptDebugWriteError(err error) error {
|
func promptDebugWriteError(err error) error {
|
||||||
return promptexec.NewError(promptexec.InvalidConfiguration, "write requested prompt debug artifact", err)
|
return promptexec.NewError(promptexec.InvalidConfiguration, "write requested prompt debug artifact", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
func ptr[T any](value T) *T { return &value }
|
|
||||||
|
|||||||
@@ -1,735 +0,0 @@
|
|||||||
package app
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/collect"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
|
||||||
)
|
|
||||||
|
|
||||||
type workflowCollector struct {
|
|
||||||
result *collect.Result
|
|
||||||
err error
|
|
||||||
calls int
|
|
||||||
}
|
|
||||||
|
|
||||||
func (c *workflowCollector) Run(context.Context, collect.Request) (*collect.Result, error) {
|
|
||||||
c.calls++
|
|
||||||
if c.err != nil {
|
|
||||||
return nil, c.err
|
|
||||||
}
|
|
||||||
return c.result, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type workflowExecutor struct {
|
|
||||||
definition report.Definition
|
|
||||||
raw []byte
|
|
||||||
prompt promptexec.PromptInspection
|
|
||||||
inspectionErr error
|
|
||||||
profile promptexec.ProfileInspection
|
|
||||||
profileErr error
|
|
||||||
beforePreparationErr error
|
|
||||||
afterCallbackErr error
|
|
||||||
afterPreparationErr error
|
|
||||||
validation promptexec.ValidationStatus
|
|
||||||
executeCalls int
|
|
||||||
providerCalls int
|
|
||||||
request promptexec.ExecuteRequest
|
|
||||||
beforeProvider func()
|
|
||||||
preparationDebug *promptexec.PreparationDebug
|
|
||||||
executionDebug *promptexec.ExecutionDebug
|
|
||||||
preparation *promptexec.Preparation
|
|
||||||
execution *promptexec.Execution
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *workflowExecutor) InspectPrompt(_ context.Context, id, version string) (promptexec.PromptInspection, error) {
|
|
||||||
if e.inspectionErr != nil {
|
|
||||||
return promptexec.PromptInspection{}, e.inspectionErr
|
|
||||||
}
|
|
||||||
if id != e.definition.PromptID || version != e.definition.PromptVersion {
|
|
||||||
return promptexec.PromptInspection{}, errors.New("unexpected prompt identity")
|
|
||||||
}
|
|
||||||
if e.prompt.PromptID != "" {
|
|
||||||
return e.prompt, nil
|
|
||||||
}
|
|
||||||
return validPromptInspection(e.definition), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *workflowExecutor) InspectProfile(_ context.Context, id string) (promptexec.ProfileInspection, error) {
|
|
||||||
if e.profileErr != nil {
|
|
||||||
return promptexec.ProfileInspection{}, e.profileErr
|
|
||||||
}
|
|
||||||
profile := e.profile
|
|
||||||
if profile.ProfileID == "" {
|
|
||||||
profile = promptexec.ProfileInspection{ProfileID: id, BackendID: "fixture", ModelName: "fixture-model"}
|
|
||||||
}
|
|
||||||
return profile, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *workflowExecutor) Execute(_ context.Context, req promptexec.ExecuteRequest, callback promptexec.PreparationCallback) (*promptexec.Execution, error) {
|
|
||||||
e.executeCalls++
|
|
||||||
e.request = req
|
|
||||||
if e.beforePreparationErr != nil {
|
|
||||||
return nil, e.beforePreparationErr
|
|
||||||
}
|
|
||||||
stamp := time.Date(2026, 5, 29, 15, 0, 0, 0, time.UTC)
|
|
||||||
profile := e.profile
|
|
||||||
if profile.ProfileID == "" {
|
|
||||||
profile = promptexec.ProfileInspection{ProfileID: req.ProfileID, BackendID: "fixture", ModelName: "fixture-model"}
|
|
||||||
}
|
|
||||||
preparation := promptexec.Preparation{
|
|
||||||
PromptID: req.PromptID, PromptVersion: req.PromptVersion, PromptHash: "prompt-hash",
|
|
||||||
RenderedPromptHash: "rendered-hash", ProfileID: req.ProfileID, BackendID: profile.BackendID,
|
|
||||||
ModelName: profile.ModelName, DataPackagePath: req.DataPackagePath, StartedAt: stamp, EndedAt: stamp,
|
|
||||||
}
|
|
||||||
e.preparation = &preparation
|
|
||||||
if err := callback(preparation, e.preparationDebug); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if e.afterCallbackErr != nil {
|
|
||||||
return nil, e.afterCallbackErr
|
|
||||||
}
|
|
||||||
if e.beforeProvider != nil {
|
|
||||||
e.beforeProvider()
|
|
||||||
}
|
|
||||||
e.providerCalls++
|
|
||||||
if e.afterPreparationErr != nil {
|
|
||||||
return nil, e.afterPreparationErr
|
|
||||||
}
|
|
||||||
validation := e.validation
|
|
||||||
if validation == "" {
|
|
||||||
validation = promptexec.ValidationPassed
|
|
||||||
}
|
|
||||||
execution := &promptexec.Execution{
|
|
||||||
RunID: "provider-run", PromptID: req.PromptID, PromptVersion: req.PromptVersion,
|
|
||||||
PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash", ProfileID: req.ProfileID,
|
|
||||||
BackendID: profile.BackendID, ModelName: profile.ModelName, GeneratedHash: "generated-hash",
|
|
||||||
StartedAt: stamp, EndedAt: stamp, DataPackagePath: req.DataPackagePath, RawOutput: e.raw,
|
|
||||||
Debug: e.executionDebug,
|
|
||||||
Validation: promptexec.NewValidation(validation, "json_schema", e.definition.GeneratedTextSchemaID+".generated_text.schema.json", nil),
|
|
||||||
}
|
|
||||||
e.execution = execution
|
|
||||||
return execution, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type workflowNotifier struct {
|
|
||||||
requests []NotificationRequest
|
|
||||||
err error
|
|
||||||
}
|
|
||||||
|
|
||||||
func (n *workflowNotifier) Notify(_ context.Context, req NotificationRequest) (*NotificationResult, error) {
|
|
||||||
n.requests = append(n.requests, req)
|
|
||||||
if n.err != nil {
|
|
||||||
return nil, n.err
|
|
||||||
}
|
|
||||||
return &NotificationResult{
|
|
||||||
RunID: "notification-run", PipelineID: req.PipelineID, BundleID: req.BundleID,
|
|
||||||
IdempotencyKey: req.IdempotencyKey, Status: "succeeded", UploadStatus: "accepted",
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGenerateDetailedCompletesRetainedReportWorkflows(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
kind ReportKind
|
|
||||||
id report.ID
|
|
||||||
date time.Time
|
|
||||||
raw string
|
|
||||||
wantOutput string
|
|
||||||
}{
|
|
||||||
{name: "daily", kind: ReportDaily, id: report.Daily, date: workflowTime("2026-05-29T12:00:00-05:00"), raw: validDailyWorkflowJSON(), wantOutput: "Showers are possible during the selected day."},
|
|
||||||
{name: "today", kind: ReportToday, id: report.Today, raw: validTodayWorkflowJSON(), wantOutput: "Today starts with showers before improving."},
|
|
||||||
{name: "tomorrow", kind: ReportTomorrow, id: report.Tomorrow, raw: validTomorrowWorkflowJSON(), wantOutput: "Tomorrow starts with showers before improving."},
|
|
||||||
{name: "hourly", kind: ReportHourly, id: report.Hourly, raw: validHourlyWorkflowJSON(), wantOutput: "Storm chances increase through late morning."},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
cfg := workflowConfig(t)
|
|
||||||
definition := report.DefaultRegistry().MustLookup(test.id)
|
|
||||||
bundle := workflowBundle(t)
|
|
||||||
collector := &workflowCollector{result: &collect.Result{Bundle: &bundle}}
|
|
||||||
executor := &workflowExecutor{definition: definition, raw: []byte(test.raw)}
|
|
||||||
notifier := &workflowNotifier{}
|
|
||||||
outputPath := filepath.Join(t.TempDir(), test.name+".md")
|
|
||||||
|
|
||||||
result, err := GenerateDetailed(context.Background(), GenerateRequest{
|
|
||||||
Config: cfg, Report: test.kind, Date: test.date, Now: workflowTime("2026-05-29T08:30:00-05:00"),
|
|
||||||
OutputPath: outputPath, Collector: collector, Executor: executor, Notifier: notifier,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("GenerateDetailed() error = %v", err)
|
|
||||||
}
|
|
||||||
if result.Metadata.ReportID != test.id || result.Metadata.PromptID != definition.PromptID {
|
|
||||||
t.Fatalf("metadata identity = %q/%q, want %q/%q", result.Metadata.ReportID, result.Metadata.PromptID, test.id, definition.PromptID)
|
|
||||||
}
|
|
||||||
if executor.request.PromptVersion != definition.PromptVersion {
|
|
||||||
t.Fatalf("prompt version = %q, want %q", executor.request.PromptVersion, definition.PromptVersion)
|
|
||||||
}
|
|
||||||
filesystem, storeErr := state.NewFilesystemStore(cfg.Workspace)
|
|
||||||
if storeErr != nil {
|
|
||||||
t.Fatalf("NewFilesystemStore() error = %v", storeErr)
|
|
||||||
}
|
|
||||||
preparation, loadErr := filesystem.LoadPromptPreparation(context.Background(), result.PreparationPath)
|
|
||||||
if loadErr != nil || preparation.PromptVersion != definition.PromptVersion {
|
|
||||||
t.Fatalf("persisted preparation prompt version = %q, error %v, want %q", preparation.PromptVersion, loadErr, definition.PromptVersion)
|
|
||||||
}
|
|
||||||
if collector.calls != 1 || executor.executeCalls != 1 || executor.providerCalls != 1 {
|
|
||||||
t.Fatalf("calls = collect %d execute %d provider %d, want one each", collector.calls, executor.executeCalls, executor.providerCalls)
|
|
||||||
}
|
|
||||||
persisted, readErr := os.ReadFile(result.DataPackagePath)
|
|
||||||
if readErr != nil {
|
|
||||||
t.Fatalf("read data package: %v", readErr)
|
|
||||||
}
|
|
||||||
if !bytes.Equal(executor.request.DataPackage, persisted) {
|
|
||||||
t.Fatal("executor data package differs from exact persisted YAML bytes")
|
|
||||||
}
|
|
||||||
managed, readErr := os.ReadFile(result.ReportPath)
|
|
||||||
if readErr != nil || !strings.Contains(string(managed), test.wantOutput) {
|
|
||||||
t.Fatalf("managed report = %q, error %v, want generated template output %q", managed, readErr, test.wantOutput)
|
|
||||||
}
|
|
||||||
copied, readErr := os.ReadFile(outputPath)
|
|
||||||
if readErr != nil || !bytes.Equal(copied, managed) || result.OutputPath != outputPath {
|
|
||||||
t.Fatalf("output copy mismatch/error/path = %v/%q", readErr, result.OutputPath)
|
|
||||||
}
|
|
||||||
if len(notifier.requests) != 1 || notifier.requests[0].ReportPath != result.ReportPath || notifier.requests[0].ReportPath == outputPath {
|
|
||||||
t.Fatalf("notification requests = %#v, want managed report source", notifier.requests)
|
|
||||||
}
|
|
||||||
wantPipeline := "reports." + string(test.id) + "." + definition.ArtifactGroup
|
|
||||||
if notifier.requests[0].PipelineID != wantPipeline {
|
|
||||||
t.Fatalf("pipeline = %q, want %q", notifier.requests[0].PipelineID, wantPipeline)
|
|
||||||
}
|
|
||||||
validDate := result.Metadata.ValidPeriod.Start.Format("2006-01-02")
|
|
||||||
wantBundlePaths := workflowBundlePaths(test.id, validDate, result.Metadata.RunID)
|
|
||||||
if strings.Join(notifier.requests[0].BundlePaths, "\n") != strings.Join(wantBundlePaths, "\n") {
|
|
||||||
t.Fatalf("bundle paths = %#v, want %#v", notifier.requests[0].BundlePaths, wantBundlePaths)
|
|
||||||
}
|
|
||||||
managedName := filepath.Base(result.ReportPath)
|
|
||||||
if !strings.HasPrefix(managedName, "report.") || !strings.Contains(managedName, "_"+test.name) || !strings.HasSuffix(managedName, ".md") || filepath.Base(result.OutputPath) != test.name+".md" {
|
|
||||||
t.Fatalf("output names = managed %q copy %q", result.ReportPath, result.OutputPath)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGenerateDetailedPreservesSelectedProfileThroughExecution(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
kind ReportKind
|
|
||||||
id report.ID
|
|
||||||
raw string
|
|
||||||
override string
|
|
||||||
profile promptexec.ProfileInspection
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
name: "hourly default", kind: ReportHourly, id: report.Hourly, raw: validHourlyWorkflowJSON(),
|
|
||||||
profile: promptexec.ProfileInspection{ProfileID: "weather-light", BackendID: "openrouter", ModelName: "deepseek/deepseek-v4-flash"},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "daily default", kind: ReportDaily, id: report.Daily, raw: validDailyWorkflowJSON(),
|
|
||||||
profile: promptexec.ProfileInspection{ProfileID: "weather-balanced", BackendID: "openrouter", ModelName: "~google/gemini-flash-latest"},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "global override", kind: ReportDaily, id: report.Daily, raw: validDailyWorkflowJSON(), override: "operator-profile",
|
|
||||||
profile: promptexec.ProfileInspection{ProfileID: "operator-profile", BackendID: "local", ModelName: "local-weather-model"},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
cfg := workflowConfig(t)
|
|
||||||
cfg.Promptkit.Profile = test.override
|
|
||||||
definition := report.DefaultRegistry().MustLookup(test.id)
|
|
||||||
executor := &workflowExecutor{
|
|
||||||
definition: definition, prompt: logicalPromptInspection(definition), profile: test.profile, raw: []byte(test.raw),
|
|
||||||
}
|
|
||||||
bundle := workflowBundle(t)
|
|
||||||
_, err := GenerateDetailed(context.Background(), GenerateRequest{
|
|
||||||
Config: cfg, Report: test.kind, Date: workflowTime("2026-05-29T12:00:00-05:00"), Now: workflowTime("2026-05-29T08:30:00-05:00"),
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &bundle}}, Executor: executor, Notifier: &workflowNotifier{},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("GenerateDetailed() error = %v", err)
|
|
||||||
}
|
|
||||||
if executor.request.ProfileID != test.profile.ProfileID {
|
|
||||||
t.Fatalf("execution profile = %q, want %q", executor.request.ProfileID, test.profile.ProfileID)
|
|
||||||
}
|
|
||||||
if executor.preparation == nil || executor.preparation.ProfileID != test.profile.ProfileID || executor.preparation.BackendID != test.profile.BackendID || executor.preparation.ModelName != test.profile.ModelName {
|
|
||||||
t.Fatalf("prepared profile = %#v, want %q/%q/%q", executor.preparation, test.profile.ProfileID, test.profile.BackendID, test.profile.ModelName)
|
|
||||||
}
|
|
||||||
if executor.execution == nil || executor.execution.ProfileID != test.profile.ProfileID || executor.execution.BackendID != test.profile.BackendID || executor.execution.ModelName != test.profile.ModelName {
|
|
||||||
t.Fatalf("executed profile = %#v, want %q/%q/%q", executor.execution, test.profile.ProfileID, test.profile.BackendID, test.profile.ModelName)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
type preparationFailingStore struct {
|
|
||||||
state.Store
|
|
||||||
}
|
|
||||||
|
|
||||||
type renderContextFailingStore struct {
|
|
||||||
state.Store
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s renderContextFailingStore) SaveModuleSnapshot(ctx context.Context, resolved report.Resolved, snapshot module.Snapshot) (string, error) {
|
|
||||||
path, err := s.Store.SaveModuleSnapshot(ctx, resolved, snapshot)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
for index := range snapshot.Outputs {
|
|
||||||
snapshot.Outputs[index].Value = "invalid module value"
|
|
||||||
}
|
|
||||||
return path, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s preparationFailingStore) SavePromptPreparation(context.Context, report.Resolved, state.PromptPreparationArtifact) (string, error) {
|
|
||||||
return "", errors.New("injected preparation persistence failure")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGenerateDetailedStopsAtConsequentialPromptFailures(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
configure func(*workflowExecutor)
|
|
||||||
wantCategory promptexec.ErrorCategory
|
|
||||||
wantPreparation bool
|
|
||||||
wantExecution bool
|
|
||||||
wantRaw bool
|
|
||||||
wantProviderCall int
|
|
||||||
}{
|
|
||||||
{name: "preparation", configure: func(e *workflowExecutor) {
|
|
||||||
e.beforePreparationErr = promptexec.NewError(promptexec.Generation, "preparation failed", nil)
|
|
||||||
}, wantCategory: promptexec.Generation, wantPreparation: true},
|
|
||||||
{name: "credential disappears", configure: func(e *workflowExecutor) {
|
|
||||||
e.afterCallbackErr = promptexec.NewError(promptexec.MissingCredential, "credential unavailable", nil)
|
|
||||||
}, wantCategory: promptexec.MissingCredential, wantPreparation: true, wantExecution: true},
|
|
||||||
{name: "capacity is not retried", configure: func(e *workflowExecutor) {
|
|
||||||
e.afterPreparationErr = promptexec.NewError(promptexec.Capacity, "capacity rejected", nil)
|
|
||||||
}, wantCategory: promptexec.Capacity, wantPreparation: true, wantExecution: true, wantProviderCall: 1},
|
|
||||||
{name: "canceled", configure: func(e *workflowExecutor) {
|
|
||||||
e.afterPreparationErr = promptexec.NewError(promptexec.Canceled, "request canceled", context.Canceled)
|
|
||||||
}, wantCategory: promptexec.Canceled, wantPreparation: true, wantExecution: true, wantProviderCall: 1},
|
|
||||||
{name: "deadline", configure: func(e *workflowExecutor) {
|
|
||||||
e.afterPreparationErr = promptexec.NewError(promptexec.DeadlineExceeded, "deadline exceeded", context.DeadlineExceeded)
|
|
||||||
}, wantCategory: promptexec.DeadlineExceeded, wantPreparation: true, wantExecution: true, wantProviderCall: 1},
|
|
||||||
{name: "generation", configure: func(e *workflowExecutor) {
|
|
||||||
e.afterPreparationErr = promptexec.NewError(promptexec.Generation, "generation failed", nil)
|
|
||||||
}, wantCategory: promptexec.Generation, wantPreparation: true, wantExecution: true, wantProviderCall: 1},
|
|
||||||
{name: "operational validation error", configure: func(e *workflowExecutor) {
|
|
||||||
e.afterPreparationErr = promptexec.NewError(promptexec.OperationalValidation, "validator failed", nil)
|
|
||||||
}, wantCategory: promptexec.OperationalValidation, wantPreparation: true, wantExecution: true, wantProviderCall: 1},
|
|
||||||
{name: "operational validation incomplete", configure: func(e *workflowExecutor) { e.validation = promptexec.ValidationSkipped }, wantCategory: promptexec.OperationalValidation, wantPreparation: true, wantExecution: true, wantProviderCall: 1},
|
|
||||||
{name: "schema rejection", configure: func(e *workflowExecutor) { e.validation = promptexec.ValidationFailed }, wantCategory: promptexec.ValidationRejected, wantPreparation: true, wantExecution: true, wantRaw: true, wantProviderCall: 1},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
cfg := workflowConfig(t)
|
|
||||||
cfg.Notify.Distributor.Enabled = false
|
|
||||||
definition := report.DefaultRegistry().MustLookup(report.Daily)
|
|
||||||
executor := &workflowExecutor{definition: definition, raw: []byte(validDailyWorkflowJSON())}
|
|
||||||
test.configure(executor)
|
|
||||||
bundle := workflowBundle(t)
|
|
||||||
collector := &workflowCollector{result: &collect.Result{Bundle: &bundle}}
|
|
||||||
result, err := GenerateDetailed(context.Background(), GenerateRequest{
|
|
||||||
Config: cfg, Report: ReportDaily, Date: workflowTime("2026-05-29T12:00:00-05:00"), Now: workflowTime("2026-05-29T08:30:00-05:00"),
|
|
||||||
Collector: collector, Executor: executor,
|
|
||||||
})
|
|
||||||
if err == nil || result == nil || promptexec.CategoryOf(err) != test.wantCategory {
|
|
||||||
t.Fatalf("result/error/category = %#v/%v/%q, want partial result and %q", result, err, promptexec.CategoryOf(err), test.wantCategory)
|
|
||||||
}
|
|
||||||
if (result.PreparationPath != "") != test.wantPreparation || (result.ExecutionPath != "") != test.wantExecution || (result.GeneratedTextRawPath != "") != test.wantRaw {
|
|
||||||
t.Fatalf("paths = preparation %q execution %q raw %q", result.PreparationPath, result.ExecutionPath, result.GeneratedTextRawPath)
|
|
||||||
}
|
|
||||||
if executor.executeCalls != 1 || executor.providerCalls != test.wantProviderCall {
|
|
||||||
t.Fatalf("calls = execute %d provider %d, want 1/%d", executor.executeCalls, executor.providerCalls, test.wantProviderCall)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGenerateDetailedRejectsInspectionAndCredentialsBeforeCollection(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
configure func(*workflowExecutor)
|
|
||||||
wantCategory promptexec.ErrorCategory
|
|
||||||
}{
|
|
||||||
{name: "inspection", configure: func(e *workflowExecutor) { e.inspectionErr = errors.New("inspection unavailable") }, wantCategory: promptexec.InvalidConfiguration},
|
|
||||||
{name: "unknown profile", configure: func(e *workflowExecutor) { e.profileErr = errors.New("unknown selected profile") }, wantCategory: promptexec.InvalidConfiguration},
|
|
||||||
{name: "malformed profile", configure: func(e *workflowExecutor) {
|
|
||||||
e.profileErr = errors.New("malformed profile at https://operator.example/v1 api_key=secret")
|
|
||||||
}, wantCategory: promptexec.InvalidConfiguration},
|
|
||||||
{name: "unusable backend", configure: func(e *workflowExecutor) { e.profileErr = errors.New("unsupported backend") }, wantCategory: promptexec.InvalidConfiguration},
|
|
||||||
{name: "credential", configure: func(e *workflowExecutor) {
|
|
||||||
e.profile = promptexec.ProfileInspection{ProfileID: "default-profile", CredentialRequired: true}
|
|
||||||
}, wantCategory: promptexec.MissingCredential},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
cfg := workflowConfig(t)
|
|
||||||
definition := report.DefaultRegistry().MustLookup(report.Daily)
|
|
||||||
executor := &workflowExecutor{definition: definition}
|
|
||||||
test.configure(executor)
|
|
||||||
collector := &workflowCollector{err: errors.New("collector must not run")}
|
|
||||||
result, err := GenerateDetailed(context.Background(), GenerateRequest{
|
|
||||||
Config: cfg, Report: ReportDaily, Date: workflowTime("2026-05-29T12:00:00-05:00"), Now: workflowTime("2026-05-29T08:30:00-05:00"),
|
|
||||||
Collector: collector, Executor: executor,
|
|
||||||
})
|
|
||||||
if err == nil || result != nil || promptexec.CategoryOf(err) != test.wantCategory || collector.calls != 0 || executor.executeCalls != 0 {
|
|
||||||
t.Fatalf("result/error/category/collect/execute = %#v/%v/%q/%d/%d", result, err, promptexec.CategoryOf(err), collector.calls, executor.executeCalls)
|
|
||||||
}
|
|
||||||
if strings.Contains(err.Error(), "operator.example") || strings.Contains(err.Error(), "secret") {
|
|
||||||
t.Fatalf("error leaks profile details: %v", err)
|
|
||||||
}
|
|
||||||
entries, readErr := os.ReadDir(cfg.Workspace.Root)
|
|
||||||
if readErr != nil || len(entries) != 0 {
|
|
||||||
t.Fatalf("workspace entries/error = %#v/%v, want no writes before collection", entries, readErr)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGenerateDetailedStopsProviderWhenPreparationCannotPersist(t *testing.T) {
|
|
||||||
cfg := workflowConfig(t)
|
|
||||||
cfg.Notify.Distributor.Enabled = false
|
|
||||||
filesystem, err := state.NewFilesystemStore(cfg.Workspace)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
|
||||||
}
|
|
||||||
definition := report.DefaultRegistry().MustLookup(report.Daily)
|
|
||||||
executor := &workflowExecutor{definition: definition, raw: []byte(validDailyWorkflowJSON())}
|
|
||||||
bundle := workflowBundle(t)
|
|
||||||
result, err := GenerateDetailed(context.Background(), GenerateRequest{
|
|
||||||
Config: cfg, Report: ReportDaily, Date: workflowTime("2026-05-29T12:00:00-05:00"), Now: workflowTime("2026-05-29T08:30:00-05:00"),
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &bundle}}, Executor: executor, Store: preparationFailingStore{Store: filesystem},
|
|
||||||
})
|
|
||||||
if err == nil || result == nil || result.PreparationPath != "" || executor.providerCalls != 0 {
|
|
||||||
t.Fatalf("result/error/preparation/provider = %#v/%v/%q/%d", result, err, result.PreparationPath, executor.providerCalls)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGenerateDetailedPersistsPreparationBeforeProviderExecution(t *testing.T) {
|
|
||||||
cfg := workflowConfig(t)
|
|
||||||
cfg.Notify.Distributor.Enabled = false
|
|
||||||
now := workflowTime("2026-05-29T08:30:00-05:00")
|
|
||||||
request := GenerateRequest{Config: cfg, Report: ReportDaily, Date: workflowTime("2026-05-29T12:00:00-05:00"), Now: now}
|
|
||||||
resolved, err := ResolveGenerate(request, now)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("ResolveGenerate() error = %v", err)
|
|
||||||
}
|
|
||||||
filesystem, err := state.NewFilesystemStore(cfg.Workspace)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
|
||||||
}
|
|
||||||
paths, err := filesystem.Paths(resolved)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Paths() error = %v", err)
|
|
||||||
}
|
|
||||||
checked := false
|
|
||||||
executor := &workflowExecutor{definition: resolved.Definition, raw: []byte(validDailyWorkflowJSON())}
|
|
||||||
executor.beforeProvider = func() {
|
|
||||||
checked = true
|
|
||||||
if _, statErr := os.Stat(paths.Preparation); statErr != nil {
|
|
||||||
t.Fatalf("preparation was not durable before provider execution: %v", statErr)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
bundle := workflowBundle(t)
|
|
||||||
request.Collector = &workflowCollector{result: &collect.Result{Bundle: &bundle}}
|
|
||||||
request.Executor = executor
|
|
||||||
request.Store = filesystem
|
|
||||||
result, err := GenerateDetailed(context.Background(), request)
|
|
||||||
if err != nil || result == nil || !checked || result.OutputPath != "" {
|
|
||||||
t.Fatalf("result/error/checked/output = %#v/%v/%t/%q", result, err, checked, result.OutputPath)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGenerateDetailedRetainsInspectableArtifactsAfterApplicationFailures(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
raw string
|
|
||||||
configure func(*GenerateRequest, *workflowNotifier)
|
|
||||||
wantRaw bool
|
|
||||||
wantNormalized bool
|
|
||||||
wantContext bool
|
|
||||||
wantReport bool
|
|
||||||
wantOutput bool
|
|
||||||
wantNotify bool
|
|
||||||
}{
|
|
||||||
{name: "generated text decode", raw: `{`, wantRaw: true},
|
|
||||||
{name: "generated text domain", raw: `{}`, wantRaw: true},
|
|
||||||
{name: "render context build", raw: validDailyWorkflowJSON(), configure: func(req *GenerateRequest, _ *workflowNotifier) {
|
|
||||||
req.Store = renderContextFailingStore{Store: req.Store}
|
|
||||||
}, wantRaw: true, wantNormalized: true},
|
|
||||||
{name: "render context persistence", raw: validDailyWorkflowJSON(), configure: func(req *GenerateRequest, _ *workflowNotifier) {
|
|
||||||
req.Store = &failingPersistenceStore{Store: req.Store, failOperation: failRenderContext}
|
|
||||||
}, wantRaw: true, wantNormalized: true},
|
|
||||||
{name: "template write", raw: validDailyWorkflowJSON(), configure: func(req *GenerateRequest, _ *workflowNotifier) {
|
|
||||||
blocker := filepath.Join(t.TempDir(), "report-blocker")
|
|
||||||
if err := os.Mkdir(blocker, 0o700); err != nil {
|
|
||||||
t.Fatalf("create report blocker: %v", err)
|
|
||||||
}
|
|
||||||
req.Store = &failingPersistenceStore{Store: req.Store, failOperation: failRenderedReportPath, renderedReportPath: blocker}
|
|
||||||
}, wantRaw: true, wantNormalized: true, wantContext: true},
|
|
||||||
{name: "output copy", raw: validDailyWorkflowJSON(), configure: func(req *GenerateRequest, _ *workflowNotifier) {
|
|
||||||
req.OutputPath = t.TempDir()
|
|
||||||
}, wantRaw: true, wantNormalized: true, wantContext: true, wantReport: true},
|
|
||||||
{name: "notification", raw: validDailyWorkflowJSON(), configure: func(_ *GenerateRequest, notifier *workflowNotifier) {
|
|
||||||
notifier.err = errors.New("notification rejected")
|
|
||||||
}, wantRaw: true, wantNormalized: true, wantContext: true, wantReport: true, wantOutput: true, wantNotify: true},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
cfg := workflowConfig(t)
|
|
||||||
definition := report.DefaultRegistry().MustLookup(report.Daily)
|
|
||||||
executor := &workflowExecutor{definition: definition, raw: []byte(test.raw)}
|
|
||||||
notifier := &workflowNotifier{}
|
|
||||||
bundle := workflowBundle(t)
|
|
||||||
filesystem, err := state.NewFilesystemStore(cfg.Workspace)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
|
||||||
}
|
|
||||||
req := GenerateRequest{
|
|
||||||
Config: cfg, Report: ReportDaily, Date: workflowTime("2026-05-29T12:00:00-05:00"), Now: workflowTime("2026-05-29T08:30:00-05:00"),
|
|
||||||
OutputPath: filepath.Join(t.TempDir(), "daily.md"), Collector: &workflowCollector{result: &collect.Result{Bundle: &bundle}},
|
|
||||||
Executor: executor, Notifier: notifier, Store: filesystem,
|
|
||||||
}
|
|
||||||
if test.configure != nil {
|
|
||||||
test.configure(&req, notifier)
|
|
||||||
}
|
|
||||||
result, err := GenerateDetailed(context.Background(), req)
|
|
||||||
if err == nil || result == nil {
|
|
||||||
t.Fatalf("result/error = %#v/%v, want partial result and error", result, err)
|
|
||||||
}
|
|
||||||
if (result.GeneratedTextRawPath != "") != test.wantRaw || (result.GeneratedTextPath != "") != test.wantNormalized ||
|
|
||||||
(result.RenderContextPath != "") != test.wantContext || (result.ReportPath != "") != test.wantReport ||
|
|
||||||
(result.OutputPath != "") != test.wantOutput || (result.NotificationPath != "") != test.wantNotify {
|
|
||||||
t.Fatalf("reached paths = raw %q normalized %q context %q report %q output %q notification %q", result.GeneratedTextRawPath, result.GeneratedTextPath, result.RenderContextPath, result.ReportPath, result.OutputPath, result.NotificationPath)
|
|
||||||
}
|
|
||||||
if test.wantRaw {
|
|
||||||
persisted, readErr := os.ReadFile(result.GeneratedTextRawPath)
|
|
||||||
if readErr != nil || !bytes.Equal(persisted, []byte(test.raw)) {
|
|
||||||
t.Fatalf("retained raw output = %q, error %v", persisted, readErr)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if test.name == "notification" && (len(notifier.requests) != 1 || notifier.requests[0].ReportPath != result.ReportPath) {
|
|
||||||
t.Fatalf("notification requests = %#v", notifier.requests)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGenerateDetailedDebugFailuresRespectProviderBoundary(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
createCollision func(string, report.Resolved) error
|
|
||||||
wantProviderCalls int
|
|
||||||
wantPreparationFile bool
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
name: "preparation debug",
|
|
||||||
createCollision: func(root string, resolved report.Resolved) error {
|
|
||||||
path := workflowDebugRunPath(root, resolved)
|
|
||||||
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return os.WriteFile(path, []byte("not a directory"), 0o600)
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "execution debug", wantProviderCalls: 1, wantPreparationFile: true,
|
|
||||||
createCollision: func(root string, resolved report.Resolved) error {
|
|
||||||
return os.MkdirAll(filepath.Join(workflowDebugRunPath(root, resolved), "execution.json"), 0o700)
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
cfg := workflowConfig(t)
|
|
||||||
cfg.Notify.Distributor.Enabled = false
|
|
||||||
now := workflowTime("2026-05-29T08:30:00-05:00")
|
|
||||||
request := GenerateRequest{Config: cfg, Report: ReportDaily, Date: workflowTime("2026-05-29T12:00:00-05:00"), Now: now}
|
|
||||||
resolved, err := ResolveGenerate(request, now)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("ResolveGenerate() error = %v", err)
|
|
||||||
}
|
|
||||||
debugRoot := filepath.Join(t.TempDir(), "prompt-debug")
|
|
||||||
if err := test.createCollision(debugRoot, resolved); err != nil {
|
|
||||||
t.Fatalf("create debug collision: %v", err)
|
|
||||||
}
|
|
||||||
bundle := workflowBundle(t)
|
|
||||||
executor := &workflowExecutor{definition: resolved.Definition, raw: []byte(validDailyWorkflowJSON())}
|
|
||||||
request.Collector = &workflowCollector{result: &collect.Result{Bundle: &bundle}}
|
|
||||||
request.Executor = executor
|
|
||||||
request.LLMDebugDir = debugRoot
|
|
||||||
result, err := GenerateDetailed(context.Background(), request)
|
|
||||||
if err == nil || result == nil || promptexec.CategoryOf(err) != promptexec.InvalidConfiguration {
|
|
||||||
t.Fatalf("result/error/category = %#v/%v/%q", result, err, promptexec.CategoryOf(err))
|
|
||||||
}
|
|
||||||
if executor.providerCalls != test.wantProviderCalls || result.PreparationPath == "" || result.ExecutionPath != "" || result.GeneratedTextRawPath != "" {
|
|
||||||
t.Fatalf("provider/preparation/metadata/execution/raw = %d/%q/%q/%q/%q", executor.providerCalls, result.PreparationPath, result.MetadataPath, result.ExecutionPath, result.GeneratedTextRawPath)
|
|
||||||
}
|
|
||||||
if test.wantPreparationFile && result.MetadataPath == "" {
|
|
||||||
t.Fatal("execution debug failure lost previously persisted metadata")
|
|
||||||
}
|
|
||||||
preparationDebug := filepath.Join(workflowDebugRunPath(debugRoot, resolved), "preparation.json")
|
|
||||||
_, statErr := os.Stat(preparationDebug)
|
|
||||||
if (statErr == nil) != test.wantPreparationFile {
|
|
||||||
t.Fatalf("preparation debug stat error = %v, want file %t", statErr, test.wantPreparationFile)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func workflowDebugRunPath(root string, resolved report.Resolved) string {
|
|
||||||
return filepath.Join(root, string(resolved.Definition.ID), resolved.ValidPeriod.Start.Format("2006-01-02"), resolved.Metadata().RunID)
|
|
||||||
}
|
|
||||||
|
|
||||||
func workflowBundlePaths(id report.ID, validDate, runID string) []string {
|
|
||||||
switch id {
|
|
||||||
case report.Daily:
|
|
||||||
return []string{"daily/" + validDate + "/" + runID + ".md", "daily/" + validDate + "/index.md"}
|
|
||||||
case report.Today:
|
|
||||||
return []string{"daily/" + validDate + "/" + runID + ".md", "daily/" + validDate + "/index.md", "today/index.md"}
|
|
||||||
case report.Tomorrow:
|
|
||||||
return []string{"daily/" + validDate + "/" + runID + ".md", "daily/" + validDate + "/index.md", "tomorrow/index.md"}
|
|
||||||
case report.Hourly:
|
|
||||||
return []string{"hourly/index.md"}
|
|
||||||
default:
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGenerateDetailedSelectsPriorSnapshotsForRetainedReports(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
kind ReportKind
|
|
||||||
id report.ID
|
|
||||||
date time.Time
|
|
||||||
raw string
|
|
||||||
wantPrior bool
|
|
||||||
wantRecentChanges bool
|
|
||||||
}{
|
|
||||||
{name: "daily", kind: ReportDaily, id: report.Daily, date: workflowTime("2026-05-29T12:00:00-05:00"), raw: validDailyWorkflowJSON(), wantPrior: true, wantRecentChanges: true},
|
|
||||||
{name: "today", kind: ReportToday, id: report.Today, raw: validTodayWorkflowJSON(), wantPrior: true, wantRecentChanges: true},
|
|
||||||
{name: "tomorrow", kind: ReportTomorrow, id: report.Tomorrow, raw: validTomorrowWorkflowJSON(), wantPrior: true, wantRecentChanges: true},
|
|
||||||
{name: "hourly", kind: ReportHourly, id: report.Hourly, raw: validHourlyWorkflowJSON()},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
cfg := workflowConfig(t)
|
|
||||||
cfg.Notify.Distributor.Enabled = false
|
|
||||||
definition := report.DefaultRegistry().MustLookup(test.id)
|
|
||||||
firstBundle := workflowBundle(t)
|
|
||||||
setWorkflowTemperatures(&firstBundle, 45)
|
|
||||||
first, err := GenerateDetailed(context.Background(), GenerateRequest{
|
|
||||||
Config: cfg, Report: test.kind, Date: test.date, Now: workflowTime("2026-05-29T08:00:00-05:00"),
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &firstBundle}},
|
|
||||||
Executor: &workflowExecutor{definition: definition, raw: []byte(test.raw)},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("first GenerateDetailed() error = %v", err)
|
|
||||||
}
|
|
||||||
secondBundle := workflowBundle(t)
|
|
||||||
setWorkflowTemperatures(&secondBundle, 85)
|
|
||||||
second, err := GenerateDetailed(context.Background(), GenerateRequest{
|
|
||||||
Config: cfg, Report: test.kind, Date: test.date, Now: workflowTime("2026-05-29T08:30:00-05:00"),
|
|
||||||
Collector: &workflowCollector{result: &collect.Result{Bundle: &secondBundle}},
|
|
||||||
Executor: &workflowExecutor{definition: definition, raw: []byte(test.raw)},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("second GenerateDetailed() error = %v", err)
|
|
||||||
}
|
|
||||||
if test.wantPrior && (second.PriorSnapshot == nil || second.PriorSnapshot.Metadata.RunID != first.Metadata.RunID || second.PriorSnapshot.Metadata.ReportID != test.id) {
|
|
||||||
t.Fatalf("prior snapshot = %#v, want first %s run %q", second.PriorSnapshot, test.id, first.Metadata.RunID)
|
|
||||||
}
|
|
||||||
if !test.wantPrior && second.PriorSnapshot != nil {
|
|
||||||
t.Fatalf("prior snapshot = %#v, want none for non-overlapping rolling window", second.PriorSnapshot)
|
|
||||||
}
|
|
||||||
if (len(second.RecentChanges) > 0) != test.wantRecentChanges {
|
|
||||||
t.Fatalf("recent changes = %#v, want present %t", second.RecentChanges, test.wantRecentChanges)
|
|
||||||
}
|
|
||||||
if (len(second.DataPackage.RecentChanges.Items) > 0) != test.wantRecentChanges {
|
|
||||||
t.Fatalf("data package recent changes = %#v, want present %t", second.DataPackage.RecentChanges.Items, test.wantRecentChanges)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func setWorkflowTemperatures(bundle *weatherdata.Bundle, temperature float64) {
|
|
||||||
for index := range bundle.Hourly.Periods {
|
|
||||||
value := temperature
|
|
||||||
bundle.Hourly.Periods[index].TemperatureF = &value
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func workflowConfig(t *testing.T) config.Config {
|
|
||||||
t.Helper()
|
|
||||||
cfg := config.Defaults()
|
|
||||||
cfg.Workspace.Root = t.TempDir()
|
|
||||||
cfg.WeatherAPI.Timezone = "America/Chicago"
|
|
||||||
cfg.Location.ID = "home"
|
|
||||||
cfg.Location.Name = "Testville"
|
|
||||||
cfg.Location.Region = "MO"
|
|
||||||
cfg.Notify.Distributor.Enabled = true
|
|
||||||
cfg.Notify.Distributor.PipelineIDTemplate = "reports.{report_id}.{artifact_group}"
|
|
||||||
return cfg
|
|
||||||
}
|
|
||||||
|
|
||||||
func workflowBundle(t *testing.T) weatherdata.Bundle {
|
|
||||||
t.Helper()
|
|
||||||
data, err := os.ReadFile(filepath.Join("..", "forecast", "testdata", "daily_bundle.json"))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read bundle fixture: %v", err)
|
|
||||||
}
|
|
||||||
var bundle weatherdata.Bundle
|
|
||||||
if err := json.Unmarshal(data, &bundle); err != nil {
|
|
||||||
t.Fatalf("decode bundle fixture: %v", err)
|
|
||||||
}
|
|
||||||
future := bundle.Hourly.Periods[0]
|
|
||||||
future.StartTime = workflowTime("2026-05-30T06:00:00-05:00")
|
|
||||||
future.EndTime = workflowTime("2026-05-30T07:00:00-05:00")
|
|
||||||
bundle.Hourly.Periods = append(bundle.Hourly.Periods, future)
|
|
||||||
futureNarrative := bundle.Narrative.Periods[0]
|
|
||||||
futureNarrative.StartTime = workflowTime("2026-05-30T06:00:00-05:00")
|
|
||||||
futureNarrative.EndTime = workflowTime("2026-05-30T18:00:00-05:00")
|
|
||||||
futureNarrative.Name = "Tomorrow"
|
|
||||||
bundle.Narrative.Periods = append(bundle.Narrative.Periods, futureNarrative)
|
|
||||||
return bundle
|
|
||||||
}
|
|
||||||
|
|
||||||
func workflowTime(value string) time.Time {
|
|
||||||
parsed, err := time.Parse(time.RFC3339, value)
|
|
||||||
if err != nil {
|
|
||||||
panic(err)
|
|
||||||
}
|
|
||||||
return parsed
|
|
||||||
}
|
|
||||||
|
|
||||||
func validHourlyWorkflowJSON() string {
|
|
||||||
return `{"summary":"Storm chances increase through late morning.","forecast_discussion":"A front will keep the region unsettled.","precipitation_timing":"A cold front is moving into the region."}`
|
|
||||||
}
|
|
||||||
|
|
||||||
func validTomorrowWorkflowJSON() string {
|
|
||||||
return `{"summary":"Tomorrow starts with showers before improving.","forecast_discussion":["Morning showers should taper as drier air arrives.","Afternoon conditions trend quieter."],"precipitation_timing":"The best rain chance is during the morning."}`
|
|
||||||
}
|
|
||||||
|
|
||||||
func validTodayWorkflowJSON() string {
|
|
||||||
return `{"summary":"Today starts with showers before improving.","forecast_discussion":["Morning showers should taper as drier air arrives.","Afternoon conditions trend quieter."],"precipitation_timing":"The best rain chance is during the morning."}`
|
|
||||||
}
|
|
||||||
|
|
||||||
func validDailyWorkflowJSON() string {
|
|
||||||
return `{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast.","Temperatures stay seasonable by afternoon."],"precipitation_timing":"Rain is most likely during the afternoon."}`
|
|
||||||
}
|
|
||||||
@@ -1,313 +0,0 @@
|
|||||||
// Package changes compares structured module snapshots.
|
|
||||||
package changes
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"math"
|
|
||||||
"sort"
|
|
||||||
"strconv"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
|
||||||
)
|
|
||||||
|
|
||||||
type Thresholds struct {
|
|
||||||
TemperatureDegrees float64
|
|
||||||
PrecipProbabilityPoints int
|
|
||||||
WindGustMilesPerHour int
|
|
||||||
PrecipTimingShiftMinutes int
|
|
||||||
}
|
|
||||||
|
|
||||||
type Change struct {
|
|
||||||
Type string `json:"type"`
|
|
||||||
Message string `json:"message"`
|
|
||||||
Previous string `json:"previous,omitempty"`
|
|
||||||
Current string `json:"current,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func CompareDaily(previous module.Snapshot, current module.Snapshot, thresholds Thresholds) ([]Change, error) {
|
|
||||||
previousSummary, err := requiredStanza[dailySummaryStanza](previous, "derived_daily_summary")
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("previous daily summary: %w", err)
|
|
||||||
}
|
|
||||||
currentSummary, err := requiredStanza[dailySummaryStanza](current, "derived_daily_summary")
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("current daily summary: %w", err)
|
|
||||||
}
|
|
||||||
previousDayparts, err := requiredStanza[map[string]daypartSummaryStanza](previous, "derived_daypart_summaries")
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("previous daypart summaries: %w", err)
|
|
||||||
}
|
|
||||||
currentDayparts, err := requiredStanza[map[string]daypartSummaryStanza](current, "derived_daypart_summaries")
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("current daypart summaries: %w", err)
|
|
||||||
}
|
|
||||||
previousAlerts, _, err := module.StanzaValue[alertDigestStanza](previous, "alert_digest")
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
currentAlerts, _, err := module.StanzaValue[alertDigestStanza](current, "alert_digest")
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
previousTiming, previousHasTiming, err := module.StanzaValue[precipTimingStanza](previous, "precip_timing")
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
currentTiming, currentHasTiming, err := module.StanzaValue[precipTimingStanza](current, "precip_timing")
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
var changes []Change
|
|
||||||
changes = append(changes, compareTemperatureValues("Low", previousSummary.LowTempF, currentSummary.LowTempF, thresholds.TemperatureDegrees)...)
|
|
||||||
changes = append(changes, compareTemperatureValues("High", previousSummary.HighTempF, currentSummary.HighTempF, thresholds.TemperatureDegrees)...)
|
|
||||||
changes = append(changes, comparePrecipitationValues(previousSummary.DailyPrecipitationProbability, currentSummary.DailyPrecipitationProbability, thresholds.PrecipProbabilityPoints, "")...)
|
|
||||||
if previousHasTiming && currentHasTiming {
|
|
||||||
changes = append(changes, comparePrecipTiming(previousTiming.MaxPopTime, currentTiming.MaxPopTime, thresholds.PrecipTimingShiftMinutes, "")...)
|
|
||||||
}
|
|
||||||
changes = append(changes, compareWindValues(previousSummary.MaxWindGustMph, currentSummary.MaxWindGustMph, thresholds.WindGustMilesPerHour, "")...)
|
|
||||||
changes = append(changes, compareAlerts(previousAlerts.Relevant, currentAlerts.Relevant)...)
|
|
||||||
changes = append(changes, compareIndicators(aggregateIndicators(previousDayparts), aggregateIndicators(currentDayparts), "")...)
|
|
||||||
sortChanges(changes)
|
|
||||||
return changes, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type dailySummaryStanza struct {
|
|
||||||
Date string `json:"date,omitempty"`
|
|
||||||
HighTempF *int `json:"high_temp_f,omitempty"`
|
|
||||||
LowTempF *int `json:"low_temp_f,omitempty"`
|
|
||||||
DailyPrecipitationProbability *int `json:"daily_precipitation_probability,omitempty"`
|
|
||||||
MaxWindGustMph *int `json:"max_wind_gust_mph,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type daypartSummaryStanza struct {
|
|
||||||
Date string `json:"date,omitempty"`
|
|
||||||
PeriodBegins string `json:"period_begins,omitempty"`
|
|
||||||
PeriodEnds string `json:"period_ends,omitempty"`
|
|
||||||
TempRangeF string `json:"temp_range_f,omitempty"`
|
|
||||||
MaxPopPercent *int `json:"max_pop_percent,omitempty"`
|
|
||||||
MaxPopTime string `json:"max_pop_time,omitempty"`
|
|
||||||
MaxWindGustMph *int `json:"max_wind_gust_mph,omitempty"`
|
|
||||||
Snow bool `json:"snow,omitempty"`
|
|
||||||
Ice bool `json:"ice,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type precipTimingStanza struct {
|
|
||||||
MaxPopPercent *int `json:"max_pop_percent,omitempty"`
|
|
||||||
MaxPopTime string `json:"max_pop_time,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type alertDigestStanza struct {
|
|
||||||
Relevant []alertSummaryStanza `json:"relevant,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type alertSummaryStanza struct {
|
|
||||||
Event string `json:"event,omitempty"`
|
|
||||||
Headline string `json:"headline,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type indicators struct {
|
|
||||||
Snow bool
|
|
||||||
Ice bool
|
|
||||||
}
|
|
||||||
|
|
||||||
func requiredStanza[T any](snapshot module.Snapshot, name string) (T, error) {
|
|
||||||
value, ok, err := module.StanzaValue[T](snapshot, name)
|
|
||||||
if err != nil {
|
|
||||||
return value, err
|
|
||||||
}
|
|
||||||
if !ok {
|
|
||||||
return value, fmt.Errorf("stanza %q is required", name)
|
|
||||||
}
|
|
||||||
return value, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func compareTemperatureValues(label string, previous *int, current *int, threshold float64) []Change {
|
|
||||||
if previous == nil || current == nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
if !differenceAtLeast(float64(*previous), float64(*current), threshold) {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
return []Change{{
|
|
||||||
Type: "temperature_shift",
|
|
||||||
Message: fmt.Sprintf("%s temperature changed from %d to %d.", label, *previous, *current),
|
|
||||||
Previous: fmt.Sprintf("%d", *previous),
|
|
||||||
Current: fmt.Sprintf("%d", *current),
|
|
||||||
}}
|
|
||||||
}
|
|
||||||
|
|
||||||
func comparePrecipitationValues(previous *int, current *int, threshold int, prefix string) []Change {
|
|
||||||
if previous == nil || current == nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
previousCategory := precipitationCategory(float64(*previous))
|
|
||||||
currentCategory := precipitationCategory(float64(*current))
|
|
||||||
if previousCategory == currentCategory && !differenceAtLeast(float64(*previous), float64(*current), float64(threshold)) {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
changeType := prefix + "precip_probability_change"
|
|
||||||
return []Change{{
|
|
||||||
Type: changeType,
|
|
||||||
Message: fmt.Sprintf("Peak precipitation chance changed from %d%% (%s) to %d%% (%s).", *previous, previousCategory, *current, currentCategory),
|
|
||||||
Previous: fmt.Sprintf("%d%% %s", *previous, previousCategory),
|
|
||||||
Current: fmt.Sprintf("%d%% %s", *current, currentCategory),
|
|
||||||
}}
|
|
||||||
}
|
|
||||||
|
|
||||||
func comparePrecipTiming(previous string, current string, thresholdMinutes int, prefix string) []Change {
|
|
||||||
if thresholdMinutes <= 0 || previous == "" || current == "" || previous == current {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
previousTime, previousOK := parseClock(previous)
|
|
||||||
currentTime, currentOK := parseClock(current)
|
|
||||||
if !previousOK || !currentOK {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
if int(math.Abs(currentTime.Sub(previousTime).Minutes())) < thresholdMinutes {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
return []Change{{
|
|
||||||
Type: prefix + "precip_timing_shift",
|
|
||||||
Message: fmt.Sprintf("Peak precipitation timing shifted from %s to %s.", previous, current),
|
|
||||||
Previous: previous,
|
|
||||||
Current: current,
|
|
||||||
}}
|
|
||||||
}
|
|
||||||
|
|
||||||
func compareWindValues(previous *int, current *int, threshold int, prefix string) []Change {
|
|
||||||
if previous == nil || current == nil || !differenceAtLeast(float64(*previous), float64(*current), float64(threshold)) {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
return []Change{{
|
|
||||||
Type: prefix + "wind_gust_change",
|
|
||||||
Message: fmt.Sprintf("Peak wind gust changed from %d mph to %d mph.", *previous, *current),
|
|
||||||
Previous: fmt.Sprintf("%d mph", *previous),
|
|
||||||
Current: fmt.Sprintf("%d mph", *current),
|
|
||||||
}}
|
|
||||||
}
|
|
||||||
|
|
||||||
func compareAlerts(previous []alertSummaryStanza, current []alertSummaryStanza) []Change {
|
|
||||||
previousSet := alertSet(previous)
|
|
||||||
currentSet := alertSet(current)
|
|
||||||
var changes []Change
|
|
||||||
for event := range currentSet {
|
|
||||||
if _, ok := previousSet[event]; !ok {
|
|
||||||
changes = append(changes, Change{Type: "alert_added", Message: fmt.Sprintf("Alert added: %s.", event), Current: event})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for event := range previousSet {
|
|
||||||
if _, ok := currentSet[event]; !ok {
|
|
||||||
changes = append(changes, Change{Type: "alert_removed", Message: fmt.Sprintf("Alert removed: %s.", event), Previous: event})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
sortChanges(changes)
|
|
||||||
return changes
|
|
||||||
}
|
|
||||||
|
|
||||||
func compareIndicators(previous indicators, current indicators, prefix string) []Change {
|
|
||||||
var changes []Change
|
|
||||||
for _, item := range []struct {
|
|
||||||
name string
|
|
||||||
previous bool
|
|
||||||
current bool
|
|
||||||
}{
|
|
||||||
{name: "snow", previous: previous.Snow, current: current.Snow},
|
|
||||||
{name: "ice", previous: previous.Ice, current: current.Ice},
|
|
||||||
} {
|
|
||||||
if item.previous == item.current {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
changeType := prefix + item.name + "_risk_change"
|
|
||||||
if item.current {
|
|
||||||
changes = append(changes, Change{Type: changeType, Message: fmt.Sprintf("%s risk is now present.", item.name), Current: "present"})
|
|
||||||
} else {
|
|
||||||
changes = append(changes, Change{Type: changeType, Message: fmt.Sprintf("%s risk is no longer present.", item.name), Previous: "present"})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return changes
|
|
||||||
}
|
|
||||||
|
|
||||||
func aggregateIndicators(dayparts map[string]daypartSummaryStanza) indicators {
|
|
||||||
out := indicators{}
|
|
||||||
for _, daypart := range dayparts {
|
|
||||||
out.Snow = out.Snow || daypart.Snow
|
|
||||||
out.Ice = out.Ice || daypart.Ice
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
func alertSet(alerts []alertSummaryStanza) map[string]struct{} {
|
|
||||||
out := map[string]struct{}{}
|
|
||||||
for _, alert := range alerts {
|
|
||||||
event := alert.Event
|
|
||||||
if event == "" {
|
|
||||||
event = alert.Headline
|
|
||||||
}
|
|
||||||
if event != "" {
|
|
||||||
out[event] = struct{}{}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
func precipitationCategory(value float64) string {
|
|
||||||
switch {
|
|
||||||
case value >= 70:
|
|
||||||
return "high"
|
|
||||||
case value >= 50:
|
|
||||||
return "likely"
|
|
||||||
case value >= 20:
|
|
||||||
return "possible"
|
|
||||||
default:
|
|
||||||
return "low"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func differenceAtLeast(previous float64, current float64, threshold float64) bool {
|
|
||||||
if threshold <= 0 {
|
|
||||||
return previous != current
|
|
||||||
}
|
|
||||||
return math.Abs(current-previous) >= threshold
|
|
||||||
}
|
|
||||||
|
|
||||||
func sortChanges(items []Change) {
|
|
||||||
sort.SliceStable(items, func(i, j int) bool {
|
|
||||||
if items[i].Type == items[j].Type {
|
|
||||||
return items[i].Message < items[j].Message
|
|
||||||
}
|
|
||||||
return items[i].Type < items[j].Type
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func parseClock(value string) (time.Time, bool) {
|
|
||||||
value = strings.TrimSpace(value)
|
|
||||||
for _, layout := range []string{"3 PM", "3:04 PM", "15:04"} {
|
|
||||||
if parsed, err := time.Parse(layout, value); err == nil {
|
|
||||||
return parsed, true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return time.Time{}, false
|
|
||||||
}
|
|
||||||
|
|
||||||
func parseTempRange(value string) (*int, *int) {
|
|
||||||
value = strings.TrimSpace(value)
|
|
||||||
if value == "" {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
parts := strings.Split(value, "-")
|
|
||||||
if len(parts) == 1 {
|
|
||||||
if parsed, err := strconv.Atoi(strings.TrimSpace(parts[0])); err == nil {
|
|
||||||
return &parsed, &parsed
|
|
||||||
}
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
minValue, minErr := strconv.Atoi(strings.TrimSpace(parts[0]))
|
|
||||||
maxValue, maxErr := strconv.Atoi(strings.TrimSpace(parts[len(parts)-1]))
|
|
||||||
if minErr != nil || maxErr != nil {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
return &minValue, &maxValue
|
|
||||||
}
|
|
||||||
@@ -1,138 +0,0 @@
|
|||||||
package changes
|
|
||||||
|
|
||||||
import (
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestCompareDailyNoMeaningfulChanges(t *testing.T) {
|
|
||||||
previous := dailySnapshot(t, 60, 70, 30, "8 AM", nil, false)
|
|
||||||
current := dailySnapshot(t, 61, 71, 35, "8:30 AM", nil, false)
|
|
||||||
|
|
||||||
changes, err := CompareDaily(previous, current, testThresholds())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("CompareDaily() error = %v", err)
|
|
||||||
}
|
|
||||||
if len(changes) != 0 {
|
|
||||||
t.Fatalf("changes = %#v, want none", changes)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCompareDailyTemperatureThreshold(t *testing.T) {
|
|
||||||
previous := dailySnapshot(t, 50, 70, 10, "8 AM", nil, false)
|
|
||||||
current := dailySnapshot(t, 58, 79, 10, "8 AM", nil, false)
|
|
||||||
|
|
||||||
changes, err := CompareDaily(previous, current, testThresholds())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("CompareDaily() error = %v", err)
|
|
||||||
}
|
|
||||||
if countType(changes, "temperature_shift") != 2 {
|
|
||||||
t.Fatalf("changes = %#v, want low and high temperature changes", changes)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCompareDailyPrecipTimingShift(t *testing.T) {
|
|
||||||
previous := dailySnapshot(t, 60, 70, 60, "8 AM", nil, false)
|
|
||||||
current := dailySnapshot(t, 60, 70, 60, "11 AM", nil, false)
|
|
||||||
|
|
||||||
changes, err := CompareDaily(previous, current, testThresholds())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("CompareDaily() error = %v", err)
|
|
||||||
}
|
|
||||||
if countType(changes, "precip_timing_shift") != 1 {
|
|
||||||
t.Fatalf("changes = %#v, want timing shift", changes)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCompareDailyAlertAddedAndRemoved(t *testing.T) {
|
|
||||||
previous := dailySnapshot(t, 60, 70, 10, "8 AM", []string{"Wind Advisory"}, false)
|
|
||||||
current := dailySnapshot(t, 60, 70, 10, "8 AM", []string{"Flood Watch"}, false)
|
|
||||||
|
|
||||||
changes, err := CompareDaily(previous, current, testThresholds())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("CompareDaily() error = %v", err)
|
|
||||||
}
|
|
||||||
if countType(changes, "alert_added") != 1 || countType(changes, "alert_removed") != 1 {
|
|
||||||
t.Fatalf("changes = %#v, want one alert added and one removed", changes)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCompareDailyIndicatorChange(t *testing.T) {
|
|
||||||
previous := dailySnapshot(t, 60, 70, 10, "8 AM", nil, false)
|
|
||||||
current := dailySnapshot(t, 60, 70, 10, "8 AM", nil, true)
|
|
||||||
|
|
||||||
changes, err := CompareDaily(previous, current, testThresholds())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("CompareDaily() error = %v", err)
|
|
||||||
}
|
|
||||||
if countType(changes, "snow_risk_change") != 1 {
|
|
||||||
t.Fatalf("changes = %#v, want snow risk change", changes)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCompareDailyRequiresComparisonStanzas(t *testing.T) {
|
|
||||||
_, err := CompareDaily(snapshot(t), dailySnapshot(t, 60, 70, 10, "8 AM", nil, false), testThresholds())
|
|
||||||
if err == nil {
|
|
||||||
t.Fatal("CompareDaily() error = nil, want missing stanza error")
|
|
||||||
}
|
|
||||||
if !strings.Contains(err.Error(), "derived_daily_summary") {
|
|
||||||
t.Fatalf("error = %q, want derived_daily_summary context", err.Error())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func dailySnapshot(t *testing.T, low int, high int, precip int, precipTime string, alerts []string, snow bool) module.Snapshot {
|
|
||||||
t.Helper()
|
|
||||||
relevant := make([]alertSummaryStanza, 0, len(alerts))
|
|
||||||
for _, alert := range alerts {
|
|
||||||
relevant = append(relevant, alertSummaryStanza{Event: alert})
|
|
||||||
}
|
|
||||||
return snapshot(t,
|
|
||||||
module.Output{ID: module.DerivedDailySummary, StanzaName: "derived_daily_summary", Value: dailySummaryStanza{
|
|
||||||
Date: "2026-05-29",
|
|
||||||
HighTempF: &high,
|
|
||||||
LowTempF: &low,
|
|
||||||
DailyPrecipitationProbability: &precip,
|
|
||||||
}},
|
|
||||||
module.Output{ID: module.DerivedDaypartSummaries, StanzaName: "derived_daypart_summaries", Value: map[string]daypartSummaryStanza{
|
|
||||||
"morning": {
|
|
||||||
Date: "2026-05-29",
|
|
||||||
PeriodBegins: "2026-05-29 at 6:00 AM",
|
|
||||||
PeriodEnds: "2026-05-29 at 10:00 AM",
|
|
||||||
TempRangeF: "60-70",
|
|
||||||
Snow: snow,
|
|
||||||
},
|
|
||||||
}},
|
|
||||||
module.Output{ID: module.PrecipTiming, StanzaName: "precip_timing", Value: precipTimingStanza{MaxPopPercent: &precip, MaxPopTime: precipTime}},
|
|
||||||
module.Output{ID: module.AlertDigest, StanzaName: "alert_digest", Value: alertDigestStanza{Relevant: relevant}},
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func snapshot(t *testing.T, outputs ...module.Output) module.Snapshot {
|
|
||||||
t.Helper()
|
|
||||||
snapshot, err := module.NewSnapshot(outputs)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("NewSnapshot() error = %v", err)
|
|
||||||
}
|
|
||||||
return snapshot
|
|
||||||
}
|
|
||||||
|
|
||||||
func testThresholds() Thresholds {
|
|
||||||
return Thresholds{
|
|
||||||
TemperatureDegrees: 5,
|
|
||||||
PrecipProbabilityPoints: 20,
|
|
||||||
WindGustMilesPerHour: 10,
|
|
||||||
PrecipTimingShiftMinutes: 120,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func countType(changes []Change, changeType string) int {
|
|
||||||
var count int
|
|
||||||
for _, change := range changes {
|
|
||||||
if change.Type == changeType {
|
|
||||||
count++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return count
|
|
||||||
}
|
|
||||||
@@ -33,21 +33,11 @@ func writeBatchStatus(stderr io.Writer, result *app.BatchResult) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
for _, item := range result.Reports {
|
for _, item := range result.Reports {
|
||||||
notificationFields := ""
|
|
||||||
if item.NotificationStatus != "" {
|
|
||||||
notificationFields += fmt.Sprintf(" notificationStatus=%q", item.NotificationStatus)
|
|
||||||
}
|
|
||||||
if item.NotificationRunID != "" {
|
|
||||||
notificationFields += fmt.Sprintf(" notificationRunId=%q", item.NotificationRunID)
|
|
||||||
}
|
|
||||||
if item.NotificationError != "" {
|
|
||||||
notificationFields += fmt.Sprintf(" notificationError=%q", item.NotificationError)
|
|
||||||
}
|
|
||||||
if item.Status == "failed" {
|
if item.Status == "failed" {
|
||||||
_, _ = fmt.Fprintf(stderr, "report=%s status=failed error=%q%s\n", item.ReportID, item.Error, notificationFields)
|
_, _ = fmt.Fprintf(stderr, "report=%s status=failed error=%q\n", item.ReportID, item.Error)
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
_, _ = fmt.Fprintf(stderr, "report=%s status=succeeded output=%q%s\n", item.ReportID, item.OutputPath, notificationFields)
|
_, _ = fmt.Fprintf(stderr, "report=%s status=succeeded output=%q\n", item.ReportID, item.OutputPath)
|
||||||
}
|
}
|
||||||
if result.Notification != nil {
|
if result.Notification != nil {
|
||||||
_, _ = fmt.Fprintf(stderr, "batchNotification status=%q", result.Notification.Status)
|
_, _ = fmt.Fprintf(stderr, "batchNotification status=%q", result.Notification.Status)
|
||||||
@@ -63,9 +53,6 @@ func writeBatchStatus(stderr io.Writer, result *app.BatchResult) {
|
|||||||
if result.Notification.BundleID != "" {
|
if result.Notification.BundleID != "" {
|
||||||
_, _ = fmt.Fprintf(stderr, " bundleId=%q", result.Notification.BundleID)
|
_, _ = fmt.Fprintf(stderr, " bundleId=%q", result.Notification.BundleID)
|
||||||
}
|
}
|
||||||
if result.Notification.Path != "" {
|
|
||||||
_, _ = fmt.Fprintf(stderr, " path=%q", result.Notification.Path)
|
|
||||||
}
|
|
||||||
if result.Notification.Error != "" {
|
if result.Notification.Error != "" {
|
||||||
_, _ = fmt.Fprintf(stderr, " error=%q", result.Notification.Error)
|
_, _ = fmt.Fprintf(stderr, " error=%q", result.Notification.Error)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/app"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/app"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -25,17 +26,15 @@ type generateSummary struct {
|
|||||||
Status string `json:"status"`
|
Status string `json:"status"`
|
||||||
GeneratedAt time.Time `json:"generatedAt"`
|
GeneratedAt time.Time `json:"generatedAt"`
|
||||||
ValidPeriod timeutil.Period `json:"validPeriod"`
|
ValidPeriod timeutil.Period `json:"validPeriod"`
|
||||||
ReportPath string `json:"reportPath,omitempty"`
|
|
||||||
OutputPath string `json:"outputPath,omitempty"`
|
OutputPath string `json:"outputPath,omitempty"`
|
||||||
MetadataPath string `json:"metadataPath,omitempty"`
|
|
||||||
DataPackagePath string `json:"dataPackagePath,omitempty"`
|
|
||||||
PreparationPath string `json:"preparationPath,omitempty"`
|
|
||||||
ExecutionPath string `json:"executionPath,omitempty"`
|
|
||||||
LLMDebugPath string `json:"llmDebugPath,omitempty"`
|
LLMDebugPath string `json:"llmDebugPath,omitempty"`
|
||||||
GeneratedTextRawPath string `json:"generatedTextRawPath,omitempty"`
|
PromptVersion string `json:"promptVersion"`
|
||||||
GeneratedTextPath string `json:"generatedTextPath,omitempty"`
|
Timezone string `json:"timezone"`
|
||||||
RenderContextPath string `json:"renderContextPath,omitempty"`
|
ProfileID string `json:"profileId,omitempty"`
|
||||||
NotificationPath string `json:"notificationPath,omitempty"`
|
BackendID string `json:"backendId,omitempty"`
|
||||||
|
ModelName string `json:"modelName,omitempty"`
|
||||||
|
SourceWarnings []weatherdata.SourceWarning `json:"sourceWarnings,omitempty"`
|
||||||
|
ValidationStatus string `json:"validationStatus,omitempty"`
|
||||||
Notification *generateNotificationSummary `json:"notification,omitempty"`
|
Notification *generateNotificationSummary `json:"notification,omitempty"`
|
||||||
Error string `json:"error,omitempty"`
|
Error string `json:"error,omitempty"`
|
||||||
}
|
}
|
||||||
@@ -74,25 +73,20 @@ func newGenerateSummary(result *app.ReportResult, err error) generateSummary {
|
|||||||
return summary
|
return summary
|
||||||
}
|
}
|
||||||
|
|
||||||
metadata := result.Metadata
|
summary.ReportID = result.ReportID
|
||||||
summary.ReportID = metadata.ReportID
|
summary.ReportName = result.ReportName
|
||||||
summary.ReportName = reportName(metadata.ReportID)
|
summary.PromptID = result.PromptID
|
||||||
summary.PromptID = metadata.PromptID
|
summary.PromptVersion = result.PromptVersion
|
||||||
summary.RunID = metadata.RunID
|
summary.RunID = result.RunID
|
||||||
summary.Status = summaryStatusSucceeded
|
summary.Status = summaryStatusSucceeded
|
||||||
summary.GeneratedAt = metadata.GeneratedAt
|
summary.GeneratedAt = result.GeneratedAt
|
||||||
summary.ValidPeriod = metadata.ValidPeriod
|
summary.ValidPeriod = result.ValidPeriod
|
||||||
summary.ReportPath = result.ReportPath
|
summary.Timezone = result.Timezone
|
||||||
|
summary.ProfileID, summary.BackendID, summary.ModelName = result.ProfileID, result.BackendID, result.ModelName
|
||||||
|
summary.SourceWarnings = append([]weatherdata.SourceWarning(nil), result.SourceWarnings...)
|
||||||
|
summary.ValidationStatus = string(result.ValidationStatus)
|
||||||
summary.OutputPath = result.OutputPath
|
summary.OutputPath = result.OutputPath
|
||||||
summary.MetadataPath = result.MetadataPath
|
|
||||||
summary.DataPackagePath = result.DataPackagePath
|
|
||||||
summary.PreparationPath = result.PreparationPath
|
|
||||||
summary.ExecutionPath = result.ExecutionPath
|
|
||||||
summary.LLMDebugPath = result.LLMDebugPath
|
summary.LLMDebugPath = result.LLMDebugPath
|
||||||
summary.GeneratedTextRawPath = result.GeneratedTextRawPath
|
|
||||||
summary.GeneratedTextPath = result.GeneratedTextPath
|
|
||||||
summary.RenderContextPath = result.RenderContextPath
|
|
||||||
summary.NotificationPath = result.NotificationPath
|
|
||||||
summary.Notification = newGenerateNotificationSummary(result.Notification)
|
summary.Notification = newGenerateNotificationSummary(result.Notification)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
summary.Status = summaryStatusFailed
|
summary.Status = summaryStatusFailed
|
||||||
|
|||||||
@@ -2,259 +2,30 @@ package cli
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/app"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/app"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestNewGenerateSummaryForGeneratedTextReport(t *testing.T) {
|
func TestGenerateSummaryUsesActiveResultFields(t *testing.T) {
|
||||||
generatedAt := time.Date(2026, 5, 29, 13, 30, 0, 0, time.UTC)
|
generatedAt := time.Date(2026, 5, 29, 13, 30, 0, 0, time.UTC)
|
||||||
acceptedAt := generatedAt.Add(time.Minute)
|
summary := newGenerateSummary(&app.ReportResult{ReportID: report.Daily, ReportName: "Daily Report", PromptID: "weather.daily_generated_text", PromptVersion: "2.0.0", RunID: "run-123", GeneratedAt: generatedAt, Timezone: "America/Chicago", ValidPeriod: timeutil.Period{Start: generatedAt, End: generatedAt.Add(24 * time.Hour)}, ProfileID: "weather-balanced", BackendID: "openrouter", ModelName: "model", SourceWarnings: []weatherdata.SourceWarning{{Source: "alerts", Message: "source unavailable"}}, ValidationStatus: promptexec.ValidationPassed, OutputPath: "/reports/daily.md"}, nil)
|
||||||
startedAt := acceptedAt.Add(time.Minute)
|
if summary.OutputPath == "" || summary.ProfileID == "" || summary.ValidationStatus != string(promptexec.ValidationPassed) || len(summary.SourceWarnings) != 1 {
|
||||||
finishedAt := startedAt.Add(time.Minute)
|
t.Fatalf("summary = %#v", summary)
|
||||||
result := &app.ReportResult{
|
|
||||||
DataPackagePath: "/runs/hourly/data_package.yaml",
|
|
||||||
PreparationPath: "/runs/hourly/preparation.json",
|
|
||||||
ExecutionPath: "/runs/hourly/execution.json",
|
|
||||||
LLMDebugPath: "/operator-debug/hourly/2026-05-29/run-123",
|
|
||||||
ReportPath: "/runs/hourly/report.md",
|
|
||||||
OutputPath: "/copies/hourly.md",
|
|
||||||
MetadataPath: "/runs/hourly/metadata.json",
|
|
||||||
GeneratedTextRawPath: "/runs/hourly/generated_text_raw.json",
|
|
||||||
GeneratedTextPath: "/runs/hourly/generated_text.json",
|
|
||||||
RenderContextPath: "/runs/hourly/render_context.json",
|
|
||||||
NotificationPath: "/runs/hourly/notification.json",
|
|
||||||
Metadata: state.Metadata{
|
|
||||||
ReportID: report.Hourly,
|
|
||||||
PromptID: "weather.hourly_generated_text",
|
|
||||||
RunID: "20260529T133000Z_hourly",
|
|
||||||
GeneratedAt: generatedAt,
|
|
||||||
ValidPeriod: testSummaryPeriod(generatedAt),
|
|
||||||
},
|
|
||||||
Notification: &app.NotificationResult{
|
|
||||||
Status: "succeeded",
|
|
||||||
UploadStatus: "accepted",
|
|
||||||
RunID: "distributor-run",
|
|
||||||
PipelineID: "weatherreporter.hourly",
|
|
||||||
BundleID: "weatherreporter.home.hourly",
|
|
||||||
IdempotencyKey: "weatherreporter.home.hourly.20260529T133000Z_hourly",
|
|
||||||
AcceptedAt: acceptedAt,
|
|
||||||
StartedAt: &startedAt,
|
|
||||||
FinishedAt: &finishedAt,
|
|
||||||
Report: []byte(`{"actions":[{"action":"replace_older"}]}`),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
summary := newGenerateSummary(result, nil)
|
|
||||||
|
|
||||||
if summary.Command != "generate" || summary.Status != "succeeded" {
|
|
||||||
t.Fatalf("summary command/status = %q/%q, want generate/succeeded", summary.Command, summary.Status)
|
|
||||||
}
|
|
||||||
if summary.ReportID != report.Hourly || summary.ReportName != "Hourly Report" || summary.PromptID != "weather.hourly_generated_text" || summary.RunID != "20260529T133000Z_hourly" {
|
|
||||||
t.Fatalf("summary identity = %#v, want hourly report identity", summary)
|
|
||||||
}
|
|
||||||
if summary.PreparationPath == "" || summary.ExecutionPath == "" || summary.LLMDebugPath == "" || summary.GeneratedTextRawPath == "" || summary.GeneratedTextPath == "" || summary.RenderContextPath == "" {
|
|
||||||
t.Fatalf("generated-text paths = %#v, want generated-text artifact paths", summary)
|
|
||||||
}
|
|
||||||
if summary.Notification == nil || summary.Notification.RunID != "distributor-run" || summary.Notification.AcceptedAt == nil || !summary.Notification.AcceptedAt.Equal(acceptedAt) {
|
|
||||||
t.Fatalf("notification = %#v, want summarized distributor result", summary.Notification)
|
|
||||||
}
|
}
|
||||||
data, err := json.Marshal(summary)
|
data, err := json.Marshal(summary)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Marshal() error = %v", err)
|
t.Fatalf("Marshal() error = %v", err)
|
||||||
}
|
}
|
||||||
if strings.Contains(string(data), "replace_older") || strings.Contains(string(data), "actions") {
|
for _, forbidden := range []string{"reportPath", "metadataPath", "dataPackagePath", "preparationPath", "executionPath", "generatedTextRawPath", "generatedTextPath", "renderContextPath"} {
|
||||||
t.Fatalf("summary JSON includes raw distributor report payload:\n%s", string(data))
|
if strings.Contains(string(data), forbidden) {
|
||||||
}
|
t.Fatalf("summary includes %q: %s", forbidden, data)
|
||||||
if strings.Contains(string(data), "preflightPath") || strings.Contains(string(data), "generatedTextResultPath") || !strings.Contains(string(data), "preparationPath") || !strings.Contains(string(data), "executionPath") {
|
|
||||||
t.Fatalf("summary JSON does not use prompt artifact path names:\n%s", string(data))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestNewGenerateSummaryOmitsNotificationWhenNotAttempted(t *testing.T) {
|
|
||||||
generatedAt := time.Date(2026, 5, 29, 13, 30, 0, 0, time.UTC)
|
|
||||||
result := &app.ReportResult{
|
|
||||||
DataPackagePath: "/runs/daily/data_package.yaml",
|
|
||||||
PreparationPath: "/runs/daily/preparation.json",
|
|
||||||
ReportPath: "/runs/daily/report.md",
|
|
||||||
OutputPath: "/copies/daily.md",
|
|
||||||
MetadataPath: "/runs/daily/metadata.json",
|
|
||||||
Metadata: state.Metadata{
|
|
||||||
ReportID: report.Daily,
|
|
||||||
PromptID: "weather.daily_generated_text",
|
|
||||||
RunID: "20260529T133000Z_daily",
|
|
||||||
GeneratedAt: generatedAt,
|
|
||||||
ValidPeriod: testSummaryPeriod(generatedAt),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
summary := newGenerateSummary(result, nil)
|
|
||||||
|
|
||||||
if summary.ReportID != report.Daily || summary.ReportName != "Daily Report" || summary.Status != "succeeded" {
|
|
||||||
t.Fatalf("summary = %#v, want successful daily summary", summary)
|
|
||||||
}
|
|
||||||
if summary.Notification != nil || summary.NotificationPath != "" {
|
|
||||||
t.Fatalf("notification summary/path = %#v/%q, want omitted", summary.Notification, summary.NotificationPath)
|
|
||||||
}
|
|
||||||
data, err := json.Marshal(summary)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Marshal() error = %v", err)
|
|
||||||
}
|
|
||||||
for _, omitted := range []string{"notification"} {
|
|
||||||
if strings.Contains(string(data), omitted) {
|
|
||||||
t.Fatalf("summary JSON contains %q, want omitted:\n%s", omitted, string(data))
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestNewGenerateSummaryOmitsUnreachedArtifactPaths(t *testing.T) {
|
|
||||||
result := &app.ReportResult{
|
|
||||||
DataPackagePath: "/runs/daily/data_package.yaml",
|
|
||||||
PreparationPath: "/runs/daily/preparation.json",
|
|
||||||
Metadata: state.Metadata{
|
|
||||||
ReportID: report.Daily,
|
|
||||||
RunID: "20260529T133000Z_daily",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
data, err := json.Marshal(newGenerateSummary(result, errors.New("metadata write failed")))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Marshal() error = %v", err)
|
|
||||||
}
|
|
||||||
text := string(data)
|
|
||||||
for _, omitted := range []string{"executionPath", "reportPath", "outputPath", "metadataPath", "generatedTextRawPath", "generatedTextPath", "renderContextPath", "notificationPath"} {
|
|
||||||
if strings.Contains(text, omitted) {
|
|
||||||
t.Fatalf("partial summary includes unreached field %q:\n%s", omitted, text)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if !strings.Contains(text, "dataPackagePath") || !strings.Contains(text, "preparationPath") {
|
|
||||||
t.Fatalf("partial summary omits reached paths:\n%s", text)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestNewGenerateSummaryForNotificationFailure(t *testing.T) {
|
|
||||||
generatedAt := time.Date(2026, 5, 29, 13, 30, 0, 0, time.UTC)
|
|
||||||
result := &app.ReportResult{
|
|
||||||
DataPackagePath: "/runs/hourly/data_package.yaml",
|
|
||||||
PreparationPath: "/runs/hourly/preparation.json",
|
|
||||||
ReportPath: "/runs/hourly/report.md",
|
|
||||||
OutputPath: "/copies/hourly.md",
|
|
||||||
MetadataPath: "/runs/hourly/metadata.json",
|
|
||||||
NotificationPath: "/runs/hourly/notification.json",
|
|
||||||
Metadata: state.Metadata{
|
|
||||||
ReportID: report.Hourly,
|
|
||||||
PromptID: "weather.hourly_generated_text",
|
|
||||||
RunID: "20260529T133000Z_hourly",
|
|
||||||
GeneratedAt: generatedAt,
|
|
||||||
ValidPeriod: testSummaryPeriod(generatedAt),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
err := errors.New(`notify report "hourly" run "20260529T133000Z_hourly": upload rejected`)
|
|
||||||
|
|
||||||
summary := newGenerateSummary(result, err)
|
|
||||||
|
|
||||||
if summary.Status != "failed" || summary.Error != err.Error() {
|
|
||||||
t.Fatalf("status/error = %q/%q, want failed notification error", summary.Status, summary.Error)
|
|
||||||
}
|
|
||||||
if summary.NotificationPath != "/runs/hourly/notification.json" || summary.ReportPath == "" || summary.MetadataPath == "" {
|
|
||||||
t.Fatalf("artifact paths = report %q metadata %q notification %q, want inspectable paths", summary.ReportPath, summary.MetadataPath, summary.NotificationPath)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestNewBatchSummaryStatusDerivation(t *testing.T) {
|
|
||||||
startedAt := time.Date(2026, 5, 29, 12, 0, 0, 0, time.UTC)
|
|
||||||
finishedAt := startedAt.Add(2 * time.Minute)
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
result *app.BatchResult
|
|
||||||
wantStatus string
|
|
||||||
wantError string
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
name: "success",
|
|
||||||
result: &app.BatchResult{
|
|
||||||
Batch: app.BatchMorning,
|
|
||||||
StartedAt: startedAt,
|
|
||||||
FinishedAt: finishedAt,
|
|
||||||
Total: 1,
|
|
||||||
Succeeded: 1,
|
|
||||||
Reports: []app.BatchReportResult{{ReportID: report.Today, Status: "succeeded"}},
|
|
||||||
},
|
|
||||||
wantStatus: "succeeded",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "report failure",
|
|
||||||
result: &app.BatchResult{
|
|
||||||
Batch: app.BatchMorning,
|
|
||||||
Total: 2,
|
|
||||||
Succeeded: 1,
|
|
||||||
Failed: 1,
|
|
||||||
Reports: []app.BatchReportResult{
|
|
||||||
{ReportID: report.Today, Status: "succeeded"},
|
|
||||||
{ReportID: report.Tomorrow, Status: "failed", Error: "render failed"},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
wantStatus: "failed",
|
|
||||||
wantError: "batch morning failed: 1 of 2 reports failed",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "skipped notification",
|
|
||||||
result: &app.BatchResult{
|
|
||||||
Batch: app.BatchEvening,
|
|
||||||
Total: 2,
|
|
||||||
Succeeded: 1,
|
|
||||||
Failed: 1,
|
|
||||||
Reports: []app.BatchReportResult{{ReportID: report.Tomorrow, Status: "failed"}},
|
|
||||||
Notification: &app.BatchNotificationResult{
|
|
||||||
Status: "skipped",
|
|
||||||
Reason: "one or more reports failed",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
wantStatus: "failed",
|
|
||||||
wantError: "batch evening failed: 1 of 2 reports failed",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "failed notification",
|
|
||||||
result: &app.BatchResult{
|
|
||||||
Batch: app.BatchEvening,
|
|
||||||
Total: 1,
|
|
||||||
Succeeded: 1,
|
|
||||||
Reports: []app.BatchReportResult{{ReportID: report.Tomorrow, Status: "succeeded"}},
|
|
||||||
Notification: &app.BatchNotificationResult{
|
|
||||||
Status: "failed",
|
|
||||||
Error: "notify batch evening: upload rejected",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
wantStatus: "failed",
|
|
||||||
wantError: "batch evening notification failed: notify batch evening: upload rejected",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range tests {
|
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
|
||||||
summary := newBatchSummary(tt.result)
|
|
||||||
if summary.Command != "run" || summary.Status != tt.wantStatus {
|
|
||||||
t.Fatalf("command/status = %q/%q, want run/%s", summary.Command, summary.Status, tt.wantStatus)
|
|
||||||
}
|
|
||||||
if summary.Error != tt.wantError {
|
|
||||||
t.Fatalf("error = %q, want %q", summary.Error, tt.wantError)
|
|
||||||
}
|
|
||||||
if len(summary.Reports) != len(tt.result.Reports) {
|
|
||||||
t.Fatalf("reports = %#v, want copied report list", summary.Reports)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func testSummaryPeriod(start time.Time) timeutil.Period {
|
|
||||||
return timeutil.Period{
|
|
||||||
Start: start,
|
|
||||||
End: start.Add(6 * time.Hour),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -5,6 +5,8 @@ import (
|
|||||||
"flag"
|
"flag"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/app"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/app"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/buildinfo"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/buildinfo"
|
||||||
@@ -24,12 +26,6 @@ Usage:
|
|||||||
weatherreporter generate hourly [--config PATH] [--units VALUE] [--tz NAME] [--out PATH] [--llm-debug-dir PATH] [--quiet]
|
weatherreporter generate hourly [--config PATH] [--units VALUE] [--tz NAME] [--out PATH] [--llm-debug-dir PATH] [--quiet]
|
||||||
weatherreporter run morning [--config PATH] [--units VALUE] [--tz NAME] [--out-dir PATH] [--llm-debug-dir PATH] [--quiet]
|
weatherreporter run morning [--config PATH] [--units VALUE] [--tz NAME] [--out-dir PATH] [--llm-debug-dir PATH] [--quiet]
|
||||||
weatherreporter run evening [--config PATH] [--units VALUE] [--tz NAME] [--out-dir PATH] [--llm-debug-dir PATH] [--quiet]
|
weatherreporter run evening [--config PATH] [--units VALUE] [--tz NAME] [--out-dir PATH] [--llm-debug-dir PATH] [--quiet]
|
||||||
weatherreporter inspect reports [--config PATH] [--limit N]
|
|
||||||
weatherreporter inspect metadata [--config PATH] RUN_ID
|
|
||||||
weatherreporter inspect modules [--config PATH] RUN_ID
|
|
||||||
weatherreporter inspect data-package [--config PATH] RUN_ID
|
|
||||||
weatherreporter inspect prior [--config PATH] RUN_ID
|
|
||||||
weatherreporter inspect sources [--config PATH] RUN_ID
|
|
||||||
|
|
||||||
Options:
|
Options:
|
||||||
-h, --help Show this help message.
|
-h, --help Show this help message.
|
||||||
@@ -37,9 +33,9 @@ Options:
|
|||||||
--config PATH Load configuration from PATH instead of /usr/local/etc/weatherreporter/config.yml.
|
--config PATH Load configuration from PATH instead of /usr/local/etc/weatherreporter/config.yml.
|
||||||
--units VALUE Override weather API units.
|
--units VALUE Override weather API units.
|
||||||
--tz NAME Override weather API timezone.
|
--tz NAME Override weather API timezone.
|
||||||
--out PATH Write an extra Markdown report copy where supported by the generate command.
|
--out PATH Write the generated Markdown report to PATH.
|
||||||
--llm-debug-dir PATH Write sensitive prompt debug artifacts outside the managed workspace.
|
--llm-debug-dir PATH Write sensitive prompt debug artifacts under PATH.
|
||||||
--out-dir PATH Write extra Markdown report copies for run commands.
|
--out-dir PATH Write generated Markdown reports beneath PATH for run commands.
|
||||||
--quiet Suppress successful generate and run output.
|
--quiet Suppress successful generate and run output.
|
||||||
`
|
`
|
||||||
|
|
||||||
@@ -47,6 +43,8 @@ type Runner struct {
|
|||||||
Clock timeutil.Clock
|
Clock timeutil.Clock
|
||||||
ExecutorFactory ExecutorFactory
|
ExecutorFactory ExecutorFactory
|
||||||
Version string
|
Version string
|
||||||
|
WorkingDir string
|
||||||
|
runBatchDetailed func(context.Context, app.BatchRequest) (*app.BatchResult, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
func Run(ctx context.Context, args []string, stdout io.Writer, stderr io.Writer) error {
|
func Run(ctx context.Context, args []string, stdout io.Writer, stderr io.Writer) error {
|
||||||
@@ -92,7 +90,11 @@ func (r Runner) Run(ctx context.Context, args []string, stdout io.Writer, stderr
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
result, err := app.RunBatchDetailed(ctx, req)
|
runBatchDetailed := r.runBatchDetailed
|
||||||
|
if runBatchDetailed == nil {
|
||||||
|
runBatchDetailed = app.RunBatchDetailed
|
||||||
|
}
|
||||||
|
result, err := runBatchDetailed(ctx, req)
|
||||||
if result != nil {
|
if result != nil {
|
||||||
summary := newBatchSummary(result)
|
summary := newBatchSummary(result)
|
||||||
if encodeErr := writeActionResult(stdout, stderr, summary, outputOptions{Quiet: opts.Quiet}, func(w io.Writer) {
|
if encodeErr := writeActionResult(stdout, stderr, summary, outputOptions{Quiet: opts.Quiet}, func(w io.Writer) {
|
||||||
@@ -105,8 +107,6 @@ func (r Runner) Run(ctx context.Context, args []string, stdout io.Writer, stderr
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
return err
|
return err
|
||||||
case "inspect":
|
|
||||||
return r.runInspect(ctx, args[1:], stdout)
|
|
||||||
default:
|
default:
|
||||||
return fmt.Errorf("unknown command %q", args[0])
|
return fmt.Errorf("unknown command %q", args[0])
|
||||||
}
|
}
|
||||||
@@ -127,81 +127,6 @@ type generateOptions struct {
|
|||||||
Date string
|
Date string
|
||||||
}
|
}
|
||||||
|
|
||||||
type inspectOptions struct {
|
|
||||||
ConfigPath string
|
|
||||||
Limit int
|
|
||||||
RunID string
|
|
||||||
}
|
|
||||||
|
|
||||||
type inspectRunCommand struct {
|
|
||||||
Name string
|
|
||||||
Inspect func(context.Context, app.InspectRunRequest) (any, error)
|
|
||||||
}
|
|
||||||
|
|
||||||
var inspectRunCommands = []inspectRunCommand{
|
|
||||||
{Name: "metadata", Inspect: func(ctx context.Context, req app.InspectRunRequest) (any, error) {
|
|
||||||
return app.InspectMetadata(ctx, req)
|
|
||||||
}},
|
|
||||||
{Name: "modules", Inspect: func(ctx context.Context, req app.InspectRunRequest) (any, error) {
|
|
||||||
return app.InspectModules(ctx, req)
|
|
||||||
}},
|
|
||||||
{Name: "data-package", Inspect: func(ctx context.Context, req app.InspectRunRequest) (any, error) {
|
|
||||||
return app.InspectDataPackage(ctx, req)
|
|
||||||
}},
|
|
||||||
{Name: "prior", Inspect: func(ctx context.Context, req app.InspectRunRequest) (any, error) {
|
|
||||||
return app.InspectPriorSnapshot(ctx, req)
|
|
||||||
}},
|
|
||||||
{Name: "sources", Inspect: func(ctx context.Context, req app.InspectRunRequest) (any, error) {
|
|
||||||
return app.InspectSources(ctx, req)
|
|
||||||
}},
|
|
||||||
}
|
|
||||||
|
|
||||||
func (r Runner) runInspect(ctx context.Context, args []string, stdout io.Writer) error {
|
|
||||||
if len(args) == 0 {
|
|
||||||
return fmt.Errorf("inspect requires a command")
|
|
||||||
}
|
|
||||||
command := args[0]
|
|
||||||
switch command {
|
|
||||||
case "reports":
|
|
||||||
opts, err := parseInspectReportsFlags(args[1:])
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
cfg, err := config.Load(config.LoadOptions{Path: opts.ConfigPath})
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
records, err := app.InspectReports(ctx, app.InspectReportsRequest{Config: cfg, Limit: opts.Limit})
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return writeJSON(stdout, records)
|
|
||||||
default:
|
|
||||||
for _, candidate := range inspectRunCommands {
|
|
||||||
if candidate.Name == command {
|
|
||||||
return runInspectRunCommand(ctx, stdout, candidate, args[1:])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return fmt.Errorf("unknown inspect command %q", command)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func runInspectRunCommand(ctx context.Context, stdout io.Writer, command inspectRunCommand, args []string) error {
|
|
||||||
opts, err := parseInspectRunFlags(command.Name, args)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
cfg, err := config.Load(config.LoadOptions{Path: opts.ConfigPath})
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
value, err := command.Inspect(ctx, app.InspectRunRequest{Config: cfg, RunID: opts.RunID})
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return writeJSON(stdout, value)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (r Runner) resolveGenerate(args []string) (app.GenerateRequest, error) {
|
func (r Runner) resolveGenerate(args []string) (app.GenerateRequest, error) {
|
||||||
req, _, err := r.resolveGenerateAction(args)
|
req, _, err := r.resolveGenerateAction(args)
|
||||||
return req, err
|
return req, err
|
||||||
@@ -240,10 +165,20 @@ func (r Runner) resolveGenerateAction(args []string) (app.GenerateRequest, commo
|
|||||||
return app.GenerateRequest{}, commonOptions{}, err
|
return app.GenerateRequest{}, commonOptions{}, err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
workingDir, err := r.workingDir()
|
||||||
|
if err != nil {
|
||||||
|
return app.GenerateRequest{}, commonOptions{}, err
|
||||||
|
}
|
||||||
|
outputPath, err := resolveOutputOverride(workingDir, opts.Output)
|
||||||
|
if err != nil {
|
||||||
|
return app.GenerateRequest{}, commonOptions{}, err
|
||||||
|
}
|
||||||
|
|
||||||
req := app.GenerateRequest{
|
req := app.GenerateRequest{
|
||||||
Config: cfg,
|
Config: cfg,
|
||||||
Report: reportKind,
|
Report: reportKind,
|
||||||
OutputPath: opts.Output,
|
WorkingDir: workingDir,
|
||||||
|
OutputPath: outputPath,
|
||||||
LLMDebugDir: opts.LLMDebugDir,
|
LLMDebugDir: opts.LLMDebugDir,
|
||||||
Now: r.Clock.Now(),
|
Now: r.Clock.Now(),
|
||||||
Executor: executor,
|
Executor: executor,
|
||||||
@@ -304,7 +239,15 @@ func (r Runner) resolveRunAction(args []string) (app.BatchRequest, commonOptions
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return app.BatchRequest{}, commonOptions{}, err
|
return app.BatchRequest{}, commonOptions{}, err
|
||||||
}
|
}
|
||||||
return app.BatchRequest{Config: cfg, Batch: batch, Now: r.Clock.Now(), OutputDir: opts.OutputDir, LLMDebugDir: opts.LLMDebugDir, Executor: executor}, opts, nil
|
workingDir, err := r.workingDir()
|
||||||
|
if err != nil {
|
||||||
|
return app.BatchRequest{}, commonOptions{}, err
|
||||||
|
}
|
||||||
|
outputDir, err := resolveOutputOverride(workingDir, opts.OutputDir)
|
||||||
|
if err != nil {
|
||||||
|
return app.BatchRequest{}, commonOptions{}, err
|
||||||
|
}
|
||||||
|
return app.BatchRequest{Config: cfg, Batch: batch, Now: r.Clock.Now(), WorkingDir: workingDir, OutputDir: outputDir, LLMDebugDir: opts.LLMDebugDir, Executor: executor}, opts, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func resolveRun(args []string) (app.BatchRequest, error) {
|
func resolveRun(args []string) (app.BatchRequest, error) {
|
||||||
@@ -334,7 +277,7 @@ func parseRunFlags(args []string) (commonOptions, error) {
|
|||||||
fs.SetOutput(io.Discard)
|
fs.SetOutput(io.Discard)
|
||||||
opts := commonOptions{}
|
opts := commonOptions{}
|
||||||
addCommonFlags(fs, &opts, false)
|
addCommonFlags(fs, &opts, false)
|
||||||
fs.StringVar(&opts.OutputDir, "out-dir", "", "extra Markdown report copy directory")
|
fs.StringVar(&opts.OutputDir, "out-dir", "", "generated Markdown report directory")
|
||||||
fs.BoolVar(&opts.Quiet, "quiet", false, "suppress successful action output")
|
fs.BoolVar(&opts.Quiet, "quiet", false, "suppress successful action output")
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := fs.Parse(args); err != nil {
|
||||||
return commonOptions{}, err
|
return commonOptions{}, err
|
||||||
@@ -345,45 +288,37 @@ func parseRunFlags(args []string) (commonOptions, error) {
|
|||||||
return opts, nil
|
return opts, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func parseInspectReportsFlags(args []string) (inspectOptions, error) {
|
|
||||||
fs := flag.NewFlagSet("inspect reports", flag.ContinueOnError)
|
|
||||||
fs.SetOutput(io.Discard)
|
|
||||||
opts := inspectOptions{Limit: 20}
|
|
||||||
fs.StringVar(&opts.ConfigPath, "config", "", "configuration file path")
|
|
||||||
fs.IntVar(&opts.Limit, "limit", 20, "maximum reports to list")
|
|
||||||
if err := fs.Parse(args); err != nil {
|
|
||||||
return inspectOptions{}, err
|
|
||||||
}
|
|
||||||
if fs.NArg() > 0 {
|
|
||||||
return inspectOptions{}, fmt.Errorf("unexpected argument %q", fs.Arg(0))
|
|
||||||
}
|
|
||||||
if opts.Limit < 0 {
|
|
||||||
return inspectOptions{}, fmt.Errorf("limit must be zero or greater")
|
|
||||||
}
|
|
||||||
return opts, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func parseInspectRunFlags(command string, args []string) (inspectOptions, error) {
|
|
||||||
fs := flag.NewFlagSet("inspect "+command, flag.ContinueOnError)
|
|
||||||
fs.SetOutput(io.Discard)
|
|
||||||
opts := inspectOptions{}
|
|
||||||
fs.StringVar(&opts.ConfigPath, "config", "", "configuration file path")
|
|
||||||
if err := fs.Parse(args); err != nil {
|
|
||||||
return inspectOptions{}, err
|
|
||||||
}
|
|
||||||
if fs.NArg() != 1 {
|
|
||||||
return inspectOptions{}, fmt.Errorf("inspect %s requires a run id", command)
|
|
||||||
}
|
|
||||||
opts.RunID = fs.Arg(0)
|
|
||||||
return opts, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func addCommonFlags(fs *flag.FlagSet, opts *commonOptions, includeOutput bool) {
|
func addCommonFlags(fs *flag.FlagSet, opts *commonOptions, includeOutput bool) {
|
||||||
fs.StringVar(&opts.ConfigPath, "config", "", "configuration file path")
|
fs.StringVar(&opts.ConfigPath, "config", "", "configuration file path")
|
||||||
fs.StringVar(&opts.Units, "units", "", "weather API units")
|
fs.StringVar(&opts.Units, "units", "", "weather API units")
|
||||||
fs.StringVar(&opts.Timezone, "tz", "", "weather API timezone")
|
fs.StringVar(&opts.Timezone, "tz", "", "weather API timezone")
|
||||||
fs.StringVar(&opts.LLMDebugDir, "llm-debug-dir", "", "write sensitive prompt debug artifacts under PATH")
|
fs.StringVar(&opts.LLMDebugDir, "llm-debug-dir", "", "write sensitive prompt debug artifacts under PATH")
|
||||||
if includeOutput {
|
if includeOutput {
|
||||||
fs.StringVar(&opts.Output, "out", "", "extra Markdown report copy path")
|
fs.StringVar(&opts.Output, "out", "", "generated Markdown report path")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (r Runner) workingDir() (string, error) {
|
||||||
|
workingDir := r.WorkingDir
|
||||||
|
if workingDir == "" {
|
||||||
|
var err error
|
||||||
|
workingDir, err = os.Getwd()
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("get working directory: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !filepath.IsAbs(workingDir) {
|
||||||
|
return "", fmt.Errorf("working directory %q must be absolute", workingDir)
|
||||||
|
}
|
||||||
|
return filepath.Clean(workingDir), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveOutputOverride(workingDir, value string) (string, error) {
|
||||||
|
if value == "" {
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
|
if !filepath.IsAbs(value) {
|
||||||
|
value = filepath.Join(workingDir, value)
|
||||||
|
}
|
||||||
|
return filepath.Clean(value), nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,619 +0,0 @@
|
|||||||
package cli
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"errors"
|
|
||||||
"net/http"
|
|
||||||
"net/http/httptest"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/app"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/state"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
|
||||||
testRenderedPrompt = "PRIVATE RENDERED PROMPT"
|
|
||||||
testSchemaBody = `{"private":"schema"}`
|
|
||||||
testDataBody = "PRIVATE DATA PACKAGE"
|
|
||||||
testGeneratedBody = "PRIVATE GENERATED BODY"
|
|
||||||
testEndpoint = "https://user:credential@example.invalid/v1?token=credential"
|
|
||||||
testParameters = `{"temperature":0.2,"private":"parameter"}`
|
|
||||||
testCredential = "cli-secret-credential"
|
|
||||||
)
|
|
||||||
|
|
||||||
type commandOutput struct {
|
|
||||||
stdout string
|
|
||||||
stderr string
|
|
||||||
}
|
|
||||||
|
|
||||||
type cliExecutor struct {
|
|
||||||
fail bool
|
|
||||||
failPrompt string
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e cliExecutor) InspectPrompt(_ context.Context, id, version string) (promptexec.PromptInspection, error) {
|
|
||||||
name := strings.TrimSuffix(strings.TrimPrefix(id, "weather."), "_generated_text")
|
|
||||||
return promptexec.PromptInspection{
|
|
||||||
PromptID: id, PromptVersion: version, PromptHash: "prompt-hash", DefaultProfileID: "offline-profile",
|
|
||||||
Inputs: []promptexec.InputDefinition{{Name: "data_package", Required: true, ContentType: "application/yaml"}},
|
|
||||||
Output: promptexec.OutputContract{Format: "json", ValidationMode: "json_schema", SchemaPath: name + ".generated_text.schema.json"},
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e cliExecutor) InspectProfile(_ context.Context, id string) (promptexec.ProfileInspection, error) {
|
|
||||||
return promptexec.ProfileInspection{ProfileID: id, BackendID: "offline", ModelName: "offline-model"}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e cliExecutor) Execute(_ context.Context, req promptexec.ExecuteRequest, callback promptexec.PreparationCallback) (*promptexec.Execution, error) {
|
|
||||||
now := time.Date(2026, 5, 29, 12, 1, 0, 0, time.UTC)
|
|
||||||
preparation := promptexec.Preparation{
|
|
||||||
PromptID: req.PromptID, PromptVersion: req.PromptVersion, PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash",
|
|
||||||
ProfileID: req.ProfileID, BackendID: "offline", ModelName: "offline-model", DataPackagePath: req.DataPackagePath,
|
|
||||||
StartedAt: now, EndedAt: now,
|
|
||||||
}
|
|
||||||
if err := callback(preparation, nil); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if e.fail || req.PromptID == e.failPrompt {
|
|
||||||
return nil, errors.New(strings.Join([]string{
|
|
||||||
"provider failed", testEndpoint, testCredential, testRenderedPrompt, testSchemaBody, testDataBody, testGeneratedBody, testParameters,
|
|
||||||
}, " "))
|
|
||||||
}
|
|
||||||
raw := []byte(`{"summary":"Showers are possible.","forecast_discussion":["Rain chances continue."],"precipitation_timing":"Rain is most likely this afternoon."}`)
|
|
||||||
if req.PromptID == "weather.hourly_generated_text" {
|
|
||||||
raw = []byte(`{"summary":"Storm chances increase.","forecast_discussion":"A front keeps the area unsettled.","precipitation_timing":"Rain is most likely late this morning."}`)
|
|
||||||
}
|
|
||||||
return &promptexec.Execution{
|
|
||||||
RunID: "offline-run", PromptID: req.PromptID, PromptVersion: req.PromptVersion, PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash",
|
|
||||||
ProfileID: req.ProfileID, BackendID: "offline", ModelName: "offline-model", GeneratedHash: "generated-hash",
|
|
||||||
StartedAt: now, EndedAt: now, DataPackagePath: req.DataPackagePath, RawOutput: raw,
|
|
||||||
Validation: promptexec.NewValidation(promptexec.ValidationPassed, "json_schema", "generated_text.schema.json", nil),
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunnerHelpListsOnlySupportedCommands(t *testing.T) {
|
|
||||||
output, err := runCLICommand(Runner{}, "--help")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Run(--help) error = %v", err)
|
|
||||||
}
|
|
||||||
for _, command := range []string{
|
|
||||||
"--version",
|
|
||||||
"generate daily", "generate today", "generate tomorrow", "generate hourly", "run morning", "run evening",
|
|
||||||
"inspect reports", "inspect metadata", "inspect modules", "inspect data-package", "inspect prior", "inspect sources",
|
|
||||||
} {
|
|
||||||
if !strings.Contains(output.stdout, command) {
|
|
||||||
t.Fatalf("help missing %q:\n%s", command, output.stdout)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for _, retired := range []string{"near-term", "three-day", "weekend", "storm"} {
|
|
||||||
if strings.Contains(output.stdout, retired) {
|
|
||||||
t.Fatalf("help contains retired command %q:\n%s", retired, output.stdout)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunnerVersion(t *testing.T) {
|
|
||||||
for _, test := range []struct {
|
|
||||||
name string
|
|
||||||
runner Runner
|
|
||||||
version string
|
|
||||||
}{
|
|
||||||
{name: "development default", runner: Runner{}, version: "development"},
|
|
||||||
{name: "injected release", runner: Runner{Version: "v0.9.0-test"}, version: "v0.9.0-test"},
|
|
||||||
} {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
output, err := runCLICommand(test.runner, "--version")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Run(--version) error = %v", err)
|
|
||||||
}
|
|
||||||
if output.stdout != "weatherreporter "+test.version+"\n" || output.stderr != "" {
|
|
||||||
t.Fatalf("Run(--version) output = stdout %q stderr %q", output.stdout, output.stderr)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
if _, err := runCLICommand(Runner{Version: "v0.9.0-test"}, "--version", "extra"); err == nil {
|
|
||||||
t.Fatal("Run(--version extra) error = nil")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestResolveSupportedCommandsAndFlags(t *testing.T) {
|
|
||||||
configPath := writeCLIConfig(t, t.TempDir(), "")
|
|
||||||
runner, constructions := countingRunner(cliExecutor{})
|
|
||||||
|
|
||||||
for _, tt := range []struct {
|
|
||||||
name string
|
|
||||||
args []string
|
|
||||||
want app.ReportKind
|
|
||||||
}{
|
|
||||||
{name: "daily", args: []string{"daily", "--date", "2026-05-29"}, want: app.ReportDaily},
|
|
||||||
{name: "today", args: []string{"today"}, want: app.ReportToday},
|
|
||||||
{name: "tomorrow", args: []string{"tomorrow"}, want: app.ReportTomorrow},
|
|
||||||
{name: "hourly", args: []string{"hourly"}, want: app.ReportHourly},
|
|
||||||
} {
|
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
|
||||||
args := append(tt.args, "--config", configPath)
|
|
||||||
req, err := runner.resolveGenerate(args)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("resolveGenerate() error = %v", err)
|
|
||||||
}
|
|
||||||
if req.Report != tt.want || req.Executor == nil {
|
|
||||||
t.Fatalf("request = %#v, want report %q with executor", req, tt.want)
|
|
||||||
}
|
|
||||||
if tt.want == app.ReportDaily || tt.want == app.ReportToday {
|
|
||||||
if got := req.Date.Format(timeutil.DateLayout); got != "2026-05-29" {
|
|
||||||
t.Fatalf("resolved date = %q, want 2026-05-29", got)
|
|
||||||
}
|
|
||||||
} else if !req.Date.IsZero() {
|
|
||||||
t.Fatalf("resolved date = %s, want unset", req.Date)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range []struct {
|
|
||||||
name string
|
|
||||||
want app.BatchKind
|
|
||||||
}{
|
|
||||||
{name: "morning", want: app.BatchMorning},
|
|
||||||
{name: "evening", want: app.BatchEvening},
|
|
||||||
} {
|
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
|
||||||
req, err := runner.resolveRun([]string{tt.name, "--config", configPath})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("resolveRun() error = %v", err)
|
|
||||||
}
|
|
||||||
if req.Batch != tt.want || req.Executor == nil {
|
|
||||||
t.Fatalf("request = %#v, want batch %q with executor", req, tt.want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
if *constructions != 6 {
|
|
||||||
t.Fatalf("executor constructions = %d, want one per resolved action", *constructions)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestResolveGenerateAndRunApplySharedActionFlags(t *testing.T) {
|
|
||||||
configPath := writeCLIConfig(t, t.TempDir(), "")
|
|
||||||
runner, _ := countingRunner(cliExecutor{})
|
|
||||||
|
|
||||||
generate, generateOpts, err := runner.resolveGenerateAction([]string{
|
|
||||||
"daily", "--config", configPath, "--date", "2026-05-30", "--units", "metric", "--tz", "UTC",
|
|
||||||
"--out", "daily.md", "--llm-debug-dir", "/safe/debug", "--quiet",
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("resolveGenerateAction() error = %v", err)
|
|
||||||
}
|
|
||||||
if generate.Config.WeatherAPI.Units != "metric" || generate.Config.WeatherAPI.Timezone != "UTC" || generate.OutputPath != "daily.md" || generate.LLMDebugDir != "/safe/debug" || !generateOpts.Quiet {
|
|
||||||
t.Fatalf("generate request/options = %#v/%#v", generate, generateOpts)
|
|
||||||
}
|
|
||||||
if got := generate.Date.Format(timeutil.DateLayout); got != "2026-05-30" {
|
|
||||||
t.Fatalf("generate date = %q, want 2026-05-30", got)
|
|
||||||
}
|
|
||||||
|
|
||||||
batch, batchOpts, err := runner.resolveRunAction([]string{
|
|
||||||
"evening", "--config", configPath, "--units", "metric", "--tz", "UTC", "--out-dir", "reports",
|
|
||||||
"--llm-debug-dir", "/safe/debug", "--quiet",
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("resolveRunAction() error = %v", err)
|
|
||||||
}
|
|
||||||
if batch.Config.WeatherAPI.Units != "metric" || batch.Config.WeatherAPI.Timezone != "UTC" || batch.OutputDir != "reports" || batch.LLMDebugDir != "/safe/debug" || !batchOpts.Quiet {
|
|
||||||
t.Fatalf("batch request/options = %#v/%#v", batch, batchOpts)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGenerateFlagContracts(t *testing.T) {
|
|
||||||
for _, kind := range []app.ReportKind{app.ReportDaily, app.ReportToday, app.ReportTomorrow, app.ReportHourly} {
|
|
||||||
t.Run(string(kind), func(t *testing.T) {
|
|
||||||
opts, err := parseGenerateFlags(kind, []string{"--llm-debug-dir", "/safe/debug", "--quiet", "--out", "report.md"})
|
|
||||||
if err != nil || opts.LLMDebugDir != "/safe/debug" || !opts.Quiet || opts.Output != "report.md" {
|
|
||||||
t.Fatalf("parseGenerateFlags() = %#v, %v", opts, err)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
runner, _ := countingRunner(cliExecutor{})
|
|
||||||
for _, tt := range []struct {
|
|
||||||
name string
|
|
||||||
args []string
|
|
||||||
want string
|
|
||||||
}{
|
|
||||||
{name: "daily requires date", args: []string{"daily"}, want: "requires --date"},
|
|
||||||
{name: "malformed daily date", args: []string{"daily", "--date", "bad-date"}, want: "YYYY-MM-DD"},
|
|
||||||
{name: "malformed today date", args: []string{"today", "--date", "bad-date"}, want: "YYYY-MM-DD"},
|
|
||||||
{name: "tomorrow rejects date", args: []string{"tomorrow", "--date", "2026-05-29"}},
|
|
||||||
{name: "hourly rejects date", args: []string{"hourly", "--date", "2026-05-29"}},
|
|
||||||
{name: "hourly rejects hours", args: []string{"hourly", "--hours", "6"}},
|
|
||||||
{name: "hourly rejects duration", args: []string{"hourly", "--duration", "6h"}},
|
|
||||||
{name: "batch rejects output", args: []string{"run", "--out", "report.md"}},
|
|
||||||
} {
|
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
|
||||||
var err error
|
|
||||||
if tt.args[0] == "run" {
|
|
||||||
_, err = runner.resolveRun(append([]string{"morning"}, tt.args[1:]...))
|
|
||||||
} else {
|
|
||||||
_, err = runner.resolveGenerate(tt.args)
|
|
||||||
}
|
|
||||||
if err == nil || (tt.want != "" && !strings.Contains(err.Error(), tt.want)) {
|
|
||||||
t.Fatalf("error = %v, want rejection containing %q", err, tt.want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestResolversRejectRetiredAndUnknownNames(t *testing.T) {
|
|
||||||
runner, _ := countingRunner(cliExecutor{})
|
|
||||||
for _, name := range []string{"near-term", "three-day", "weekend", "storm"} {
|
|
||||||
if _, err := runner.resolveGenerate([]string{name}); err == nil || !strings.Contains(err.Error(), "unknown generate report") {
|
|
||||||
t.Fatalf("resolveGenerate(%q) error = %v", name, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for _, name := range []string{"daily", "weekend", "storm"} {
|
|
||||||
if _, err := runner.resolveRun([]string{name}); err == nil || !strings.Contains(err.Error(), "unknown run batch") {
|
|
||||||
t.Fatalf("resolveRun(%q) error = %v", name, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunnerSuccessfulSingleAndBatchActions(t *testing.T) {
|
|
||||||
for _, tt := range []struct {
|
|
||||||
name string
|
|
||||||
args func(string, string) []string
|
|
||||||
}{
|
|
||||||
{name: "single", args: func(configPath, outputPath string) []string {
|
|
||||||
return []string{"generate", "today", "--config", configPath, "--out", outputPath}
|
|
||||||
}},
|
|
||||||
{name: "batch", args: func(configPath, outputPath string) []string {
|
|
||||||
return []string{"run", "evening", "--config", configPath, "--out-dir", outputPath}
|
|
||||||
}},
|
|
||||||
} {
|
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
|
||||||
fixture := newCLIFixture(t)
|
|
||||||
runner, constructions := countingRunner(cliExecutor{})
|
|
||||||
output, err := runCLICommand(runner, tt.args(fixture.configPath, fixture.path("copies"))...)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Run() error = %v", err)
|
|
||||||
}
|
|
||||||
if *constructions != 1 {
|
|
||||||
t.Fatalf("executor constructions = %d, want 1", *constructions)
|
|
||||||
}
|
|
||||||
assertRoutineOutputSafe(t, output)
|
|
||||||
if tt.name == "single" {
|
|
||||||
summary := decodeGenerateSummary(t, output.stdout)
|
|
||||||
if summary.Status != summaryStatusSucceeded || summary.ReportID != report.Today || summary.ReportPath == "" || summary.OutputPath == "" || summary.MetadataPath == "" || summary.DataPackagePath == "" || summary.PreparationPath == "" || summary.ExecutionPath == "" {
|
|
||||||
t.Fatalf("single summary = %#v", summary)
|
|
||||||
}
|
|
||||||
if strings.Contains(output.stdout, `"notification"`) || strings.Contains(output.stdout, `"llmDebugPath"`) {
|
|
||||||
t.Fatalf("single summary contains absent optional fields:\n%s", output.stdout)
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
summary := decodeBatchSummary(t, output.stdout)
|
|
||||||
if summary.Status != summaryStatusSucceeded || summary.Batch != app.BatchEvening || summary.Total != 1 || len(summary.Reports) != 1 || summary.Reports[0].OutputPath == "" {
|
|
||||||
t.Fatalf("batch summary = %#v", summary)
|
|
||||||
}
|
|
||||||
if !strings.Contains(output.stderr, "report=tomorrow status=succeeded") || !strings.Contains(output.stderr, "batch=evening total=1 succeeded=1 failed=0") {
|
|
||||||
t.Fatalf("batch status = %q", output.stderr)
|
|
||||||
}
|
|
||||||
if strings.Contains(output.stdout, `"notification"`) || strings.Contains(output.stdout, `"llmDebugPath"`) {
|
|
||||||
t.Fatalf("batch summary contains absent optional fields:\n%s", output.stdout)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunnerPreRunFailureAndQuietMode(t *testing.T) {
|
|
||||||
runner, constructions := countingRunner(cliExecutor{})
|
|
||||||
output, err := runCLICommand(runner, "generate", "daily")
|
|
||||||
if err == nil || output.stdout != "" || output.stderr != "" {
|
|
||||||
t.Fatalf("pre-run output/error = %#v/%v, want error without summary", output, err)
|
|
||||||
}
|
|
||||||
if *constructions != 1 {
|
|
||||||
t.Fatalf("executor constructions = %d, want one action-scoped construction", *constructions)
|
|
||||||
}
|
|
||||||
|
|
||||||
fixture := newCLIFixture(t)
|
|
||||||
runner, _ = countingRunner(cliExecutor{})
|
|
||||||
output, err = runCLICommand(runner, "generate", "today", "--config", fixture.configPath, "--quiet")
|
|
||||||
if err != nil || output.stdout != "" || output.stderr != "" {
|
|
||||||
t.Fatalf("quiet output/error = %#v/%v", output, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunnerFailedActionReportsSafePartialSummary(t *testing.T) {
|
|
||||||
fixture := newCLIFixture(t)
|
|
||||||
runner, constructions := countingRunner(cliExecutor{fail: true})
|
|
||||||
output, err := runCLICommand(runner, "generate", "today", "--config", fixture.configPath)
|
|
||||||
if err == nil {
|
|
||||||
t.Fatal("Run() error = nil, want execution failure")
|
|
||||||
}
|
|
||||||
if *constructions != 1 {
|
|
||||||
t.Fatalf("executor constructions = %d, want 1", *constructions)
|
|
||||||
}
|
|
||||||
summary := decodeGenerateSummary(t, output.stdout)
|
|
||||||
if summary.Status != summaryStatusFailed || summary.RunID == "" || summary.MetadataPath == "" || summary.DataPackagePath == "" || summary.PreparationPath == "" || summary.ExecutionPath == "" || summary.ReportPath != "" {
|
|
||||||
t.Fatalf("failed summary paths = %#v", summary)
|
|
||||||
}
|
|
||||||
if !strings.Contains(summary.Error, "prompt execution failed") {
|
|
||||||
t.Fatalf("failed summary error = %q", summary.Error)
|
|
||||||
}
|
|
||||||
assertRoutineOutputSafe(t, output)
|
|
||||||
|
|
||||||
for _, command := range []string{"metadata", "modules", "data-package", "sources"} {
|
|
||||||
inspected, inspectErr := runCLICommand(runner, "inspect", command, "--config", fixture.configPath, summary.RunID)
|
|
||||||
if inspectErr != nil {
|
|
||||||
t.Fatalf("inspect %s error = %v", command, inspectErr)
|
|
||||||
}
|
|
||||||
if !strings.Contains(inspected.stdout, summary.RunID) {
|
|
||||||
t.Fatalf("inspect %s missing failed run id:\n%s", command, inspected.stdout)
|
|
||||||
}
|
|
||||||
assertRoutineOutputSafe(t, inspected)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunnerMixedBatchReportsSafePartialFailure(t *testing.T) {
|
|
||||||
fixture := newCLIFixture(t)
|
|
||||||
runner, constructions := countingRunner(cliExecutor{failPrompt: "weather.tomorrow_generated_text"})
|
|
||||||
output, err := runCLICommand(runner, "run", "morning", "--config", fixture.configPath)
|
|
||||||
if err == nil {
|
|
||||||
t.Fatal("Run() error = nil, want aggregate batch failure")
|
|
||||||
}
|
|
||||||
if *constructions != 1 {
|
|
||||||
t.Fatalf("executor constructions = %d, want 1", *constructions)
|
|
||||||
}
|
|
||||||
summary := decodeBatchSummary(t, output.stdout)
|
|
||||||
if summary.Status != summaryStatusFailed || summary.Total != 2 || summary.Succeeded != 1 || summary.Failed != 1 || len(summary.Reports) != 2 {
|
|
||||||
t.Fatalf("failed batch summary = %#v", summary)
|
|
||||||
}
|
|
||||||
var succeeded, failed *app.BatchReportResult
|
|
||||||
for index := range summary.Reports {
|
|
||||||
item := &summary.Reports[index]
|
|
||||||
if item.Status == summaryStatusSucceeded {
|
|
||||||
succeeded = item
|
|
||||||
} else if item.Status == summaryStatusFailed {
|
|
||||||
failed = item
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if succeeded == nil || succeeded.ReportPath == "" || succeeded.MetadataPath == "" || succeeded.ExecutionPath == "" {
|
|
||||||
t.Fatalf("successful batch item paths = %#v", succeeded)
|
|
||||||
}
|
|
||||||
if failed == nil || failed.ReportPath != "" || failed.MetadataPath == "" || failed.DataPackagePath == "" || failed.PreparationPath == "" || failed.ExecutionPath == "" {
|
|
||||||
t.Fatalf("failed batch item paths = %#v", failed)
|
|
||||||
}
|
|
||||||
if !strings.Contains(output.stderr, "status=succeeded") || !strings.Contains(output.stderr, "status=failed") || !strings.Contains(output.stderr, "batch=morning total=2 succeeded=1 failed=1") {
|
|
||||||
t.Fatalf("partial batch status = %q", output.stderr)
|
|
||||||
}
|
|
||||||
assertRoutineOutputSafe(t, output)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunnerInspectsReportsAndCurrentArtifacts(t *testing.T) {
|
|
||||||
fixture := newCLIFixture(t)
|
|
||||||
runner, _ := countingRunner(cliExecutor{})
|
|
||||||
first := runSuccessfulGenerate(t, runner, fixture.configPath, time.Date(2026, 5, 29, 11, 0, 0, 0, time.UTC))
|
|
||||||
second := runSuccessfulGenerate(t, runner, fixture.configPath, time.Date(2026, 5, 29, 12, 0, 0, 0, time.UTC))
|
|
||||||
|
|
||||||
listed, err := runCLICommand(runner, "inspect", "reports", "--config", fixture.configPath, "--limit", "2")
|
|
||||||
if err != nil || !strings.Contains(listed.stdout, first.RunID) || !strings.Contains(listed.stdout, second.RunID) {
|
|
||||||
t.Fatalf("inspect reports output/error = %s/%v", listed.stdout, err)
|
|
||||||
}
|
|
||||||
for _, command := range []string{"metadata", "modules", "data-package", "sources"} {
|
|
||||||
output, inspectErr := runCLICommand(runner, "inspect", command, "--config", fixture.configPath, second.RunID)
|
|
||||||
if inspectErr != nil || !strings.Contains(output.stdout, second.RunID) {
|
|
||||||
t.Fatalf("inspect %s output/error = %s/%v", command, output.stdout, inspectErr)
|
|
||||||
}
|
|
||||||
assertRoutineOutputSafe(t, output)
|
|
||||||
}
|
|
||||||
prior, err := runCLICommand(runner, "inspect", "prior", "--config", fixture.configPath, second.RunID)
|
|
||||||
if err != nil || !strings.Contains(prior.stdout, first.RunID) {
|
|
||||||
t.Fatalf("inspect prior output/error = %s/%v", prior.stdout, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRunnerInspectsHistoricalMetadataAndArtifacts(t *testing.T) {
|
|
||||||
fixture := newCLIFixture(t)
|
|
||||||
runIDs := []string{
|
|
||||||
writeHistoricalInspectionFixture(t, fixture.workspaceRoot, report.ID("three_day"), time.Date(2026, 5, 20, 12, 0, 0, 0, time.UTC)),
|
|
||||||
writeHistoricalInspectionFixture(t, fixture.workspaceRoot, report.ID("weekend"), time.Date(2026, 5, 21, 12, 0, 0, 0, time.UTC)),
|
|
||||||
writeHistoricalInspectionFixture(t, fixture.workspaceRoot, report.ID("storm"), time.Date(2026, 5, 22, 12, 0, 0, 0, time.UTC)),
|
|
||||||
}
|
|
||||||
runID := runIDs[0]
|
|
||||||
runner, _ := countingRunner(cliExecutor{})
|
|
||||||
|
|
||||||
listed, err := runCLICommand(runner, "inspect", "reports", "--config", fixture.configPath)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("inspect historical reports output/error = %s/%v", listed.stdout, err)
|
|
||||||
}
|
|
||||||
for _, want := range []string{runIDs[0], runIDs[1], runIDs[2], `"reportId": "three_day"`, `"reportId": "weekend"`, `"reportId": "storm"`} {
|
|
||||||
if !strings.Contains(listed.stdout, want) {
|
|
||||||
t.Fatalf("inspect historical reports missing %q:\n%s", want, listed.stdout)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for _, command := range []string{"metadata", "modules", "data-package", "sources"} {
|
|
||||||
output, inspectErr := runCLICommand(runner, "inspect", command, "--config", fixture.configPath, runID)
|
|
||||||
if inspectErr != nil || !strings.Contains(output.stdout, runID) {
|
|
||||||
t.Fatalf("inspect historical %s output/error = %s/%v", command, output.stdout, inspectErr)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
metadata, err := runCLICommand(runner, "inspect", "metadata", "--config", fixture.configPath, runID)
|
|
||||||
if err != nil || !strings.Contains(metadata.stdout, `"schemaVersion": "weatherreporter.metadata.v1"`) || !strings.Contains(metadata.stdout, `"preflightPath"`) || strings.Contains(metadata.stdout, `"preparationPath"`) {
|
|
||||||
t.Fatalf("historical metadata aliases/output = %s/%v", metadata.stdout, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func countingRunner(executor promptexec.Executor) (Runner, *int) {
|
|
||||||
count := new(int)
|
|
||||||
return Runner{
|
|
||||||
Clock: timeutil.FixedClock{Time: time.Date(2026, 5, 29, 12, 0, 0, 0, time.UTC)},
|
|
||||||
ExecutorFactory: func(PromptExecutorConfig) (promptexec.Executor, error) {
|
|
||||||
*count++
|
|
||||||
return executor, nil
|
|
||||||
},
|
|
||||||
}, count
|
|
||||||
}
|
|
||||||
|
|
||||||
func runCLICommand(runner Runner, args ...string) (commandOutput, error) {
|
|
||||||
var stdout bytes.Buffer
|
|
||||||
var stderr bytes.Buffer
|
|
||||||
err := runner.Run(context.Background(), args, &stdout, &stderr)
|
|
||||||
return commandOutput{stdout: stdout.String(), stderr: stderr.String()}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
func runSuccessfulGenerate(t *testing.T, base Runner, configPath string, now time.Time) generateSummary {
|
|
||||||
t.Helper()
|
|
||||||
base.Clock = timeutil.FixedClock{Time: now}
|
|
||||||
output, err := runCLICommand(base, "generate", "today", "--config", configPath)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("generate current report: %v", err)
|
|
||||||
}
|
|
||||||
return decodeGenerateSummary(t, output.stdout)
|
|
||||||
}
|
|
||||||
|
|
||||||
func decodeGenerateSummary(t *testing.T, text string) generateSummary {
|
|
||||||
t.Helper()
|
|
||||||
var summary generateSummary
|
|
||||||
if err := json.Unmarshal([]byte(text), &summary); err != nil {
|
|
||||||
t.Fatalf("decode generate summary: %v\n%s", err, text)
|
|
||||||
}
|
|
||||||
return summary
|
|
||||||
}
|
|
||||||
|
|
||||||
func decodeBatchSummary(t *testing.T, text string) batchSummary {
|
|
||||||
t.Helper()
|
|
||||||
var summary batchSummary
|
|
||||||
if err := json.Unmarshal([]byte(text), &summary); err != nil {
|
|
||||||
t.Fatalf("decode batch summary: %v\n%s", err, text)
|
|
||||||
}
|
|
||||||
return summary
|
|
||||||
}
|
|
||||||
|
|
||||||
func assertRoutineOutputSafe(t *testing.T, output commandOutput) {
|
|
||||||
t.Helper()
|
|
||||||
combined := output.stdout + output.stderr
|
|
||||||
for _, forbidden := range []string{testRenderedPrompt, testSchemaBody, testDataBody, testGeneratedBody, testEndpoint, testParameters, testCredential, "credential@example.invalid"} {
|
|
||||||
if strings.Contains(combined, forbidden) {
|
|
||||||
t.Fatalf("routine output contains sensitive value %q:\n%s", forbidden, combined)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
type cliFixture struct {
|
|
||||||
tempDir string
|
|
||||||
workspaceRoot string
|
|
||||||
configPath string
|
|
||||||
}
|
|
||||||
|
|
||||||
func newCLIFixture(t *testing.T) cliFixture {
|
|
||||||
t.Helper()
|
|
||||||
tempDir := t.TempDir()
|
|
||||||
workspaceRoot := filepath.Join(tempDir, "workspace")
|
|
||||||
server := weatherServer(t)
|
|
||||||
return cliFixture{tempDir: tempDir, workspaceRoot: workspaceRoot, configPath: writeCLIConfig(t, workspaceRoot, server.URL+"/")}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (f cliFixture) path(name string) string { return filepath.Join(f.tempDir, name) }
|
|
||||||
|
|
||||||
func writeCLIConfig(t *testing.T, workspaceRoot, baseURL string) string {
|
|
||||||
t.Helper()
|
|
||||||
configPath := filepath.Join(t.TempDir(), "config.yml")
|
|
||||||
body := "weather_api:\n timezone: America/Chicago\n"
|
|
||||||
if baseURL != "" {
|
|
||||||
body += " base_url: " + baseURL + "\n"
|
|
||||||
}
|
|
||||||
body += "workspace:\n root: " + workspaceRoot + "\n"
|
|
||||||
if err := os.WriteFile(configPath, []byte(body), 0o600); err != nil {
|
|
||||||
t.Fatalf("write config: %v", err)
|
|
||||||
}
|
|
||||||
return configPath
|
|
||||||
}
|
|
||||||
|
|
||||||
func weatherServer(t *testing.T) *httptest.Server {
|
|
||||||
t.Helper()
|
|
||||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
switch r.URL.Path {
|
|
||||||
case "/observations":
|
|
||||||
_, _ = w.Write([]byte(`{"data":{"timestamp":"2026-05-29T14:00:00Z","conditionCode":3}}`))
|
|
||||||
case "/conditions/current":
|
|
||||||
_, _ = w.Write([]byte(`{"data":{"conditionText":"Clear"}}`))
|
|
||||||
case "/forecast/hourly":
|
|
||||||
_, _ = w.Write([]byte(`{"data":{"locationId":"test-grid","locationName":"Testville","issuedAt":"2026-05-29T10:30:00-05:00","product":"hourly","periods":[{"startTime":"2026-05-29T06:00:00-05:00","endTime":"2026-05-29T07:00:00-05:00","textDescription":"Showers","temperatureF":66,"probabilityOfPrecipitationPercent":80},{"startTime":"2026-05-30T06:00:00-05:00","endTime":"2026-05-30T07:00:00-05:00","textDescription":"Showers","temperatureF":67,"probabilityOfPrecipitationPercent":70}]}}`))
|
|
||||||
case "/forecast/narrative":
|
|
||||||
_, _ = w.Write([]byte(`{"data":{"issuedAt":"2026-05-29T10:30:00-05:00","product":"narrative","periods":[{"startTime":"2026-05-29T06:00:00-05:00","endTime":"2026-05-29T18:00:00-05:00","textDescription":"Morning showers."},{"startTime":"2026-05-30T06:00:00-05:00","endTime":"2026-05-30T18:00:00-05:00","textDescription":"Tomorrow starts showery."}]}}`))
|
|
||||||
case "/alerts/active":
|
|
||||||
_, _ = w.Write([]byte(`{"data":{"alerts":[]}}`))
|
|
||||||
case "/discussion":
|
|
||||||
_, _ = w.Write([]byte(`{"data":{"product":"discussion","issuedAt":"2026-05-29T09:25:00-05:00","keyMessages":["Showers remain possible."]}}`))
|
|
||||||
case "/weatherstories/latest":
|
|
||||||
_, _ = w.Write([]byte(`{"data":null}`))
|
|
||||||
case "/outlooks/convective":
|
|
||||||
_, _ = w.Write([]byte(`{"data":{"asOf":"2026-05-29T16:00:00Z","outlooks":[],"discussions":[]}}`))
|
|
||||||
default:
|
|
||||||
http.NotFound(w, r)
|
|
||||||
}
|
|
||||||
}))
|
|
||||||
t.Cleanup(server.Close)
|
|
||||||
return server
|
|
||||||
}
|
|
||||||
|
|
||||||
func writeHistoricalInspectionFixture(t *testing.T, workspaceRoot string, reportID report.ID, generatedAt time.Time) string {
|
|
||||||
t.Helper()
|
|
||||||
runID := "historical-" + string(reportID)
|
|
||||||
date := generatedAt.Format(timeutil.DateLayout)
|
|
||||||
dir := filepath.Join(workspaceRoot, "snapshots", string(reportID), date)
|
|
||||||
modulePath := filepath.Join(dir, "modules."+runID+".json")
|
|
||||||
dataPath := filepath.Join(workspaceRoot, "data-packages", string(reportID), date, "data_package."+runID+".yaml")
|
|
||||||
metadataPath := filepath.Join(dir, "metadata."+runID+".json")
|
|
||||||
if err := os.MkdirAll(filepath.Dir(dataPath), 0o755); err != nil {
|
|
||||||
t.Fatalf("create historical fixture directory: %v", err)
|
|
||||||
}
|
|
||||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
|
||||||
t.Fatalf("create historical metadata directory: %v", err)
|
|
||||||
}
|
|
||||||
snapshot, err := module.NewSnapshot([]module.Output{{ID: module.Metadata, StanzaName: "metadata", Value: map[string]any{"run_id": runID}}})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("build historical module snapshot: %v", err)
|
|
||||||
}
|
|
||||||
moduleData, err := json.Marshal(snapshot)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("marshal historical module snapshot: %v", err)
|
|
||||||
}
|
|
||||||
if err := os.WriteFile(modulePath, moduleData, 0o600); err != nil {
|
|
||||||
t.Fatalf("write historical module snapshot: %v", err)
|
|
||||||
}
|
|
||||||
period := timeutil.Period{Start: generatedAt, End: generatedAt.Add(24 * time.Hour)}
|
|
||||||
pkg, err := promptinput.Build(promptinput.BuildRequest{Metadata: promptinput.Metadata{
|
|
||||||
RunID: runID, ReportID: reportID, PromptID: "weather." + string(reportID), GeneratedAt: generatedAt, Timezone: "UTC", ValidPeriod: period,
|
|
||||||
}, Modules: snapshot})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("build historical data package: %v", err)
|
|
||||||
}
|
|
||||||
data, err := promptinput.MarshalYAML(pkg)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("marshal historical data package: %v", err)
|
|
||||||
}
|
|
||||||
if err := os.WriteFile(dataPath, data, 0o600); err != nil {
|
|
||||||
t.Fatalf("write historical data package: %v", err)
|
|
||||||
}
|
|
||||||
metadata := state.Metadata{
|
|
||||||
SchemaVersion: state.MetadataSchemaVersionV1, RunID: runID, ReportID: reportID, PromptID: "weather." + string(reportID),
|
|
||||||
GeneratedAt: generatedAt, Timezone: "UTC", ValidPeriod: period, SourceLocation: "historical archive",
|
|
||||||
ModuleSnapshotPath: modulePath, DataPackagePath: dataPath, PreflightPath: "/archive/preflight.json", GeneratedTextResultPath: "/archive/result.json",
|
|
||||||
}
|
|
||||||
metadataData, err := json.Marshal(metadata)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("marshal historical metadata: %v", err)
|
|
||||||
}
|
|
||||||
if err := os.WriteFile(metadataPath, metadataData, 0o600); err != nil {
|
|
||||||
t.Fatalf("write historical metadata: %v", err)
|
|
||||||
}
|
|
||||||
return runID
|
|
||||||
}
|
|
||||||
@@ -1,11 +1,17 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/app"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
||||||
)
|
)
|
||||||
@@ -21,7 +27,7 @@ func TestResolveRunActionConstructsOneExecutor(t *testing.T) {
|
|||||||
for _, command := range []string{"morning", "evening"} {
|
for _, command := range []string{"morning", "evening"} {
|
||||||
t.Run(command, func(t *testing.T) {
|
t.Run(command, func(t *testing.T) {
|
||||||
configPath := filepath.Join(t.TempDir(), "config.yml")
|
configPath := filepath.Join(t.TempDir(), "config.yml")
|
||||||
if err := os.WriteFile(configPath, []byte("workspace:\n root: "+filepath.Join(t.TempDir(), "workspace")+"\n"), 0o600); err != nil {
|
if err := os.WriteFile(configPath, []byte("weather_api:\n base_url: https://weather.api.example.com/\n"), 0o600); err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
calls := 0
|
calls := 0
|
||||||
@@ -40,3 +46,98 @@ func TestResolveRunActionConstructsOneExecutor(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestResolveGenerateActionUsesInjectedWorkingDirectoryForOutputOverrides(t *testing.T) {
|
||||||
|
workingDir := t.TempDir()
|
||||||
|
configPath := filepath.Join(t.TempDir(), "config.yml")
|
||||||
|
if err := os.WriteFile(configPath, []byte("weather_api:\n base_url: https://weather.api.example.com/\n"), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
absoluteOutput := filepath.Join(t.TempDir(), "daily.md")
|
||||||
|
for _, scenario := range []struct {
|
||||||
|
name string
|
||||||
|
out string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{name: "default", want: ""},
|
||||||
|
{name: "relative", out: "reports/daily.md", want: filepath.Join(workingDir, "reports", "daily.md")},
|
||||||
|
{name: "absolute", out: absoluteOutput, want: absoluteOutput},
|
||||||
|
} {
|
||||||
|
t.Run(scenario.name, func(t *testing.T) {
|
||||||
|
runner := Runner{
|
||||||
|
Clock: timeutil.FixedClock{Time: time.Date(2026, 5, 29, 8, 0, 0, 0, time.UTC)},
|
||||||
|
WorkingDir: workingDir,
|
||||||
|
ExecutorFactory: func(PromptExecutorConfig) (promptexec.Executor, error) {
|
||||||
|
return &factoryExecutor{}, nil
|
||||||
|
},
|
||||||
|
}
|
||||||
|
args := []string{"daily", "--date", "2026-05-29", "--config", configPath}
|
||||||
|
if scenario.out != "" {
|
||||||
|
args = append(args, "--out", scenario.out)
|
||||||
|
}
|
||||||
|
req, _, err := runner.resolveGenerateAction(args)
|
||||||
|
if err != nil || req.WorkingDir != workingDir || req.OutputPath != scenario.want {
|
||||||
|
t.Fatalf("resolveGenerateAction() request/error = %#v/%v", req, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunActionReturnsFailureForBatchNotificationFailure(t *testing.T) {
|
||||||
|
configPath := filepath.Join(t.TempDir(), "config.yml")
|
||||||
|
if err := os.WriteFile(configPath, []byte("weather_api:\n base_url: https://weather.api.example.com/\n"), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
result := &app.BatchResult{
|
||||||
|
Batch: app.BatchMorning, Total: 2, Succeeded: 2,
|
||||||
|
Reports: []app.BatchReportResult{
|
||||||
|
{ReportID: "today", Status: "succeeded", OutputPath: "/reports/today.md"},
|
||||||
|
{ReportID: "tomorrow", Status: "succeeded", OutputPath: "/reports/tomorrow.md"},
|
||||||
|
},
|
||||||
|
Notification: &app.BatchNotificationResult{Status: "failed", Error: "distributor unavailable"},
|
||||||
|
}
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
runner := Runner{
|
||||||
|
Clock: timeutil.FixedClock{Time: time.Date(2026, 5, 29, 8, 0, 0, 0, time.UTC)},
|
||||||
|
ExecutorFactory: func(PromptExecutorConfig) (promptexec.Executor, error) {
|
||||||
|
return &factoryExecutor{}, nil
|
||||||
|
},
|
||||||
|
runBatchDetailed: func(context.Context, app.BatchRequest) (*app.BatchResult, error) {
|
||||||
|
return result, nil
|
||||||
|
},
|
||||||
|
}
|
||||||
|
err := runner.Run(context.Background(), []string{"run", "morning", "--config", configPath}, &stdout, &stderr)
|
||||||
|
var batchErr app.BatchError
|
||||||
|
if !errors.As(err, &batchErr) || !strings.Contains(err.Error(), "notification failed") {
|
||||||
|
t.Fatalf("Run() error = %v", err)
|
||||||
|
}
|
||||||
|
var summary batchSummary
|
||||||
|
if err := json.Unmarshal(stdout.Bytes(), &summary); err != nil {
|
||||||
|
t.Fatalf("decode summary: %v", err)
|
||||||
|
}
|
||||||
|
if summary.Status != summaryStatusFailed || summary.Total != 2 || summary.Succeeded != 2 || summary.Failed != 0 || summary.Notification == nil || summary.Notification.Status != "failed" {
|
||||||
|
t.Fatalf("summary = %#v", summary)
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), `batchNotification status="failed"`) {
|
||||||
|
t.Fatalf("stdout/stderr = %q/%q", stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
for _, field := range []string{"notificationStatus", "notificationRunId", "notificationPipelineId", "notificationError"} {
|
||||||
|
if strings.Contains(stdout.String(), field) || strings.Contains(stderr.String(), field) {
|
||||||
|
t.Fatalf("stdout/stderr includes removed field %q: %q/%q", field, stdout.String(), stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInspectCommandIsUnknownAndAbsentFromHelp(t *testing.T) {
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
err := (Runner{}).Run(context.Background(), []string{"inspect", "reports"}, &stdout, &stderr)
|
||||||
|
if err == nil || err.Error() != `unknown command "inspect"` {
|
||||||
|
t.Fatalf("Run(inspect) error = %v", err)
|
||||||
|
}
|
||||||
|
if err := (Runner{}).Run(context.Background(), []string{"--help"}, &stdout, &stderr); err != nil {
|
||||||
|
t.Fatalf("Run(--help) error = %v", err)
|
||||||
|
}
|
||||||
|
if strings.Contains(stdout.String(), "inspect") {
|
||||||
|
t.Fatalf("help contains removed inspect command:\n%s", stdout.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -28,9 +28,7 @@ type Config struct {
|
|||||||
Notify NotifyConfig `yaml:"notify"`
|
Notify NotifyConfig `yaml:"notify"`
|
||||||
MissingSource MissingSourceConfig `yaml:"missing_source"`
|
MissingSource MissingSourceConfig `yaml:"missing_source"`
|
||||||
Promptkit PromptkitConfig `yaml:"promptkit"`
|
Promptkit PromptkitConfig `yaml:"promptkit"`
|
||||||
Workspace WorkspaceConfig `yaml:"workspace"`
|
|
||||||
Dayparts []DaypartConfig `yaml:"dayparts"`
|
Dayparts []DaypartConfig `yaml:"dayparts"`
|
||||||
RecentChange RecentChangeConfig `yaml:"recent_change"`
|
|
||||||
Reports map[string]ReportConfig `yaml:"reports"`
|
Reports map[string]ReportConfig `yaml:"reports"`
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -94,28 +92,12 @@ type PromptkitLocalConfig struct {
|
|||||||
ConcurrencyLimit int `yaml:"concurrency_limit"`
|
ConcurrencyLimit int `yaml:"concurrency_limit"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type WorkspaceConfig struct {
|
|
||||||
Root string `yaml:"root"`
|
|
||||||
SnapshotsDir string `yaml:"snapshots_dir"`
|
|
||||||
ReportsDir string `yaml:"reports_dir"`
|
|
||||||
DataPackagesDir string `yaml:"data_packages_dir"`
|
|
||||||
PreflightDir string `yaml:"preflight_dir"`
|
|
||||||
NotificationsDir string `yaml:"notifications_dir"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type DaypartConfig struct {
|
type DaypartConfig struct {
|
||||||
Name string `yaml:"name"`
|
Name string `yaml:"name"`
|
||||||
Start string `yaml:"start"`
|
Start string `yaml:"start"`
|
||||||
End string `yaml:"end"`
|
End string `yaml:"end"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type RecentChangeConfig struct {
|
|
||||||
TemperatureDegrees float64 `yaml:"temperature_degrees"`
|
|
||||||
PrecipProbabilityPoints int `yaml:"precip_probability_points"`
|
|
||||||
WindGustMilesPerHour int `yaml:"wind_gust_miles_per_hour"`
|
|
||||||
PrecipTimingShiftMinutes int `yaml:"precip_timing_shift_minutes"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type ReportConfig struct {
|
type ReportConfig struct {
|
||||||
DeterministicModules []ModuleConfigItem `yaml:"deterministic_modules"`
|
DeterministicModules []ModuleConfigItem `yaml:"deterministic_modules"`
|
||||||
Distributor ReportDistributorConfig `yaml:"distributor"`
|
Distributor ReportDistributorConfig `yaml:"distributor"`
|
||||||
|
|||||||
@@ -147,12 +147,6 @@ func TestLoadMinimalExampleConfig(t *testing.T) {
|
|||||||
if cfg.Promptkit.Timeout != 2*time.Minute || cfg.Promptkit.Local.ConcurrencyLimit != 1 {
|
if cfg.Promptkit.Timeout != 2*time.Minute || cfg.Promptkit.Local.ConcurrencyLimit != 1 {
|
||||||
t.Fatalf("Promptkit defaults = %#v", cfg.Promptkit)
|
t.Fatalf("Promptkit defaults = %#v", cfg.Promptkit)
|
||||||
}
|
}
|
||||||
if cfg.Workspace.Root != "workspace" {
|
|
||||||
t.Fatalf("Workspace.Root = %q, want default workspace", cfg.Workspace.Root)
|
|
||||||
}
|
|
||||||
if cfg.Workspace.NotificationsDir != "notifications" {
|
|
||||||
t.Fatalf("Workspace.NotificationsDir = %q, want notifications", cfg.Workspace.NotificationsDir)
|
|
||||||
}
|
|
||||||
if cfg.Location.Name != "Brentwood" {
|
if cfg.Location.Name != "Brentwood" {
|
||||||
t.Fatalf("Location.Name = %q, want default Brentwood", cfg.Location.Name)
|
t.Fatalf("Location.Name = %q, want default Brentwood", cfg.Location.Name)
|
||||||
}
|
}
|
||||||
@@ -165,6 +159,20 @@ func TestLoadRejectsRetiredExecutionConfiguration(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestLoadRejectsRemovedRecentChangeConfiguration(t *testing.T) {
|
||||||
|
_, err := LoadFile(writeConfig(t, "recent_change:\n temperature_degrees: 5\n"))
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "recent_change") {
|
||||||
|
t.Fatalf("LoadFile() error = %v, want removed configuration rejection", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadRejectsRemovedWorkspaceConfiguration(t *testing.T) {
|
||||||
|
_, err := LoadFile(writeConfig(t, "workspace:\n root: workspace\n"))
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "workspace") {
|
||||||
|
t.Fatalf("LoadFile() error = %v, want removed configuration rejection", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestLoadReportModuleOverrides(t *testing.T) {
|
func TestLoadReportModuleOverrides(t *testing.T) {
|
||||||
path := writeConfig(t, `
|
path := writeConfig(t, `
|
||||||
reports:
|
reports:
|
||||||
|
|||||||
@@ -49,14 +49,6 @@ func Defaults() Config {
|
|||||||
ConcurrencyLimit: 1,
|
ConcurrencyLimit: 1,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
Workspace: WorkspaceConfig{
|
|
||||||
Root: "workspace",
|
|
||||||
SnapshotsDir: "snapshots",
|
|
||||||
ReportsDir: "reports",
|
|
||||||
DataPackagesDir: "data-packages",
|
|
||||||
PreflightDir: "preflight",
|
|
||||||
NotificationsDir: "notifications",
|
|
||||||
},
|
|
||||||
Dayparts: []DaypartConfig{
|
Dayparts: []DaypartConfig{
|
||||||
{Name: "overnight", Start: "00:00", End: "06:00"},
|
{Name: "overnight", Start: "00:00", End: "06:00"},
|
||||||
{Name: "morning", Start: "06:00", End: "10:00"},
|
{Name: "morning", Start: "06:00", End: "10:00"},
|
||||||
@@ -64,12 +56,6 @@ func Defaults() Config {
|
|||||||
{Name: "afternoon", Start: "15:00", End: "17:00"},
|
{Name: "afternoon", Start: "15:00", End: "17:00"},
|
||||||
{Name: "evening", Start: "17:00", End: "24:00"},
|
{Name: "evening", Start: "17:00", End: "24:00"},
|
||||||
},
|
},
|
||||||
RecentChange: RecentChangeConfig{
|
|
||||||
TemperatureDegrees: 5,
|
|
||||||
PrecipProbabilityPoints: 20,
|
|
||||||
WindGustMilesPerHour: 10,
|
|
||||||
PrecipTimingShiftMinutes: 120,
|
|
||||||
},
|
|
||||||
Reports: map[string]ReportConfig{},
|
Reports: map[string]ReportConfig{},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
package config
|
package config
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bytes"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
@@ -62,7 +63,9 @@ func mergeFile(cfg *Config, path string) error {
|
|||||||
if err := rejectRetiredExecutionConfig(data); err != nil {
|
if err := rejectRetiredExecutionConfig(data); err != nil {
|
||||||
return fmt.Errorf("parse config %q: %w", path, err)
|
return fmt.Errorf("parse config %q: %w", path, err)
|
||||||
}
|
}
|
||||||
if err := yaml.Unmarshal(data, cfg); err != nil {
|
decoder := yaml.NewDecoder(bytes.NewReader(data))
|
||||||
|
decoder.KnownFields(true)
|
||||||
|
if err := decoder.Decode(cfg); err != nil {
|
||||||
return fmt.Errorf("parse config %q: %w", path, err)
|
return fmt.Errorf("parse config %q: %w", path, err)
|
||||||
}
|
}
|
||||||
if cfg.MissingSource.Sources == nil {
|
if cfg.MissingSource.Sources == nil {
|
||||||
|
|||||||
@@ -62,9 +62,6 @@ func Validate(cfg Config) error {
|
|||||||
if err := validatePromptkit(cfg.Promptkit); err != nil {
|
if err := validatePromptkit(cfg.Promptkit); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if cfg.Workspace.Root == "" {
|
|
||||||
return fmt.Errorf("workspace.root is required")
|
|
||||||
}
|
|
||||||
if len(cfg.Dayparts) == 0 {
|
if len(cfg.Dayparts) == 0 {
|
||||||
return fmt.Errorf("dayparts must contain at least one entry")
|
return fmt.Errorf("dayparts must contain at least one entry")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
// Package fileutil provides narrow filesystem helpers for durable artifacts.
|
// Package fileutil provides narrow filesystem helpers for operator-owned outputs.
|
||||||
package fileutil
|
package fileutil
|
||||||
|
|
||||||
import (
|
import (
|
||||||
@@ -38,11 +38,3 @@ func WriteJSONAtomic(path string, value any) error {
|
|||||||
}
|
}
|
||||||
return WriteFileAtomic(path, data)
|
return WriteFileAtomic(path, data)
|
||||||
}
|
}
|
||||||
|
|
||||||
func CopyFileAtomic(source string, target string) error {
|
|
||||||
data, err := os.ReadFile(source)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("read %q: %w", source, err)
|
|
||||||
}
|
|
||||||
return WriteFileAtomic(target, data)
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -80,24 +80,3 @@ func TestWriteJSONAtomic(t *testing.T) {
|
|||||||
t.Fatalf("json = %q, want indented object", data)
|
t.Fatalf("json = %q, want indented object", data)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCopyFileAtomic(t *testing.T) {
|
|
||||||
dir := t.TempDir()
|
|
||||||
source := filepath.Join(dir, "source.txt")
|
|
||||||
target := filepath.Join(dir, "nested", "target.txt")
|
|
||||||
if err := os.WriteFile(source, []byte("copied"), 0o600); err != nil {
|
|
||||||
t.Fatalf("WriteFile() error = %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := CopyFileAtomic(source, target); err != nil {
|
|
||||||
t.Fatalf("CopyFileAtomic() error = %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
data, err := os.ReadFile(target)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("ReadFile() error = %v", err)
|
|
||||||
}
|
|
||||||
if string(data) != "copied" {
|
|
||||||
t.Fatalf("data = %q, want copied", data)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ You are WeatherReporter, a concise personal weather briefing writer.
|
|||||||
|
|
||||||
You generate local weather forecast analysis from structured data packages prepared by the weatherreporter application.
|
You generate local weather forecast analysis from structured data packages prepared by the weatherreporter application.
|
||||||
|
|
||||||
Use only the provided data package as your source of truth. Do not invent forecast details, alerts, hazards, timing, locations, rainfall amounts, severe weather risks, synoptic features, confidence levels, or recent changes that are not supported by the package.
|
Use only the provided data package as your source of truth. Do not invent forecast details, alerts, hazards, timing, locations, rainfall amounts, severe weather risks, synoptic features, or confidence levels that are not supported by the package.
|
||||||
|
|
||||||
The reader is intelligent and weather-literate, but not a professional meteorologist. If asked to provide narrative analysis or commentary, write in plain, precise, meteorologically informed language. Avoid hype, filler, generic safety advice, and TV-weather style. Provide polished prose that avoids highly technical meteorological jargon or shorthand.
|
The reader is intelligent and weather-literate, but not a professional meteorologist. If asked to provide narrative analysis or commentary, write in plain, precise, meteorologically informed language. Avoid hype, filler, generic safety advice, and TV-weather style. Provide polished prose that avoids highly technical meteorological jargon or shorthand.
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
id: weather.daily_generated_text
|
id: weather.daily_generated_text
|
||||||
version: "1.1.0"
|
version: "2.0.0"
|
||||||
default_profile: weather-balanced
|
default_profile: weather-balanced
|
||||||
description: Daily weather report analysis prompt.
|
description: Daily weather report analysis prompt.
|
||||||
inputs:
|
inputs:
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
id: weather.hourly_generated_text
|
id: weather.hourly_generated_text
|
||||||
version: "1.1.0"
|
version: "2.0.0"
|
||||||
default_profile: weather-light
|
default_profile: weather-light
|
||||||
description: Hourly weather report analysis prompt.
|
description: Hourly weather report analysis prompt.
|
||||||
inputs:
|
inputs:
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
id: weather.today_generated_text
|
id: weather.today_generated_text
|
||||||
version: "1.1.0"
|
version: "2.0.0"
|
||||||
default_profile: weather-balanced
|
default_profile: weather-balanced
|
||||||
description: Today's weather report analysis prompt.
|
description: Today's weather report analysis prompt.
|
||||||
inputs:
|
inputs:
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
id: weather.tomorrow_generated_text
|
id: weather.tomorrow_generated_text
|
||||||
version: "1.1.0"
|
version: "2.0.0"
|
||||||
default_profile: weather-balanced
|
default_profile: weather-balanced
|
||||||
description: Tomorrow's weather report analysis prompt.
|
description: Tomorrow's weather report analysis prompt.
|
||||||
inputs:
|
inputs:
|
||||||
|
|||||||
@@ -68,8 +68,8 @@ func TestPromptAssetsDeclareTheFourGeneratedTextPrompts(t *testing.T) {
|
|||||||
if err := yaml.Unmarshal(data, &definition); err != nil {
|
if err := yaml.Unmarshal(data, &definition); err != nil {
|
||||||
t.Fatalf("decode prompt definition: %v", err)
|
t.Fatalf("decode prompt definition: %v", err)
|
||||||
}
|
}
|
||||||
if definition.ID != tc.id || definition.Version != "1.1.0" || definition.DefaultProfile != tc.profile {
|
if definition.ID != tc.id || definition.Version != "2.0.0" || definition.DefaultProfile != tc.profile {
|
||||||
t.Fatalf("definition = %#v, want %s version 1.1.0 and profile %s", definition, tc.id, tc.profile)
|
t.Fatalf("definition = %#v, want %s version 2.0.0 and profile %s", definition, tc.id, tc.profile)
|
||||||
}
|
}
|
||||||
if len(definition.Inputs) != 1 || definition.Inputs[0].Name != "data_package" || !definition.Inputs[0].Required || definition.Inputs[0].ContentType != "application/yaml" {
|
if len(definition.Inputs) != 1 || definition.Inputs[0].Name != "data_package" || !definition.Inputs[0].Required || definition.Inputs[0].ContentType != "application/yaml" {
|
||||||
t.Fatalf("inputs = %#v, want one required YAML data_package", definition.Inputs)
|
t.Fatalf("inputs = %#v, want one required YAML data_package", definition.Inputs)
|
||||||
@@ -140,11 +140,11 @@ func TestPromptkitInspectsEmbeddedPromptsOffline(t *testing.T) {
|
|||||||
{"weather.hourly_generated_text", "weather-light", "deepseek/deepseek-v4-flash"},
|
{"weather.hourly_generated_text", "weather-light", "deepseek/deepseek-v4-flash"},
|
||||||
} {
|
} {
|
||||||
t.Run(want.id, func(t *testing.T) {
|
t.Run(want.id, func(t *testing.T) {
|
||||||
inspection, err := engine.InspectPrompt(context.Background(), want.id, "1.1.0")
|
inspection, err := engine.InspectPrompt(context.Background(), want.id, "2.0.0")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("InspectPrompt() error = %v", err)
|
t.Fatalf("InspectPrompt() error = %v", err)
|
||||||
}
|
}
|
||||||
if inspection.PromptID != want.id || inspection.PromptVersion != "1.1.0" || inspection.DefaultProfileID != want.profile {
|
if inspection.PromptID != want.id || inspection.PromptVersion != "2.0.0" || inspection.DefaultProfileID != want.profile {
|
||||||
t.Fatalf("inspection = %#v", inspection)
|
t.Fatalf("inspection = %#v", inspection)
|
||||||
}
|
}
|
||||||
profile, err := engine.InspectProfile(context.Background(), inspection.DefaultProfileID)
|
profile, err := engine.InspectProfile(context.Background(), inspection.DefaultProfileID)
|
||||||
|
|||||||
@@ -1,4 +1,6 @@
|
|||||||
package state
|
// Package promptdebug writes explicitly requested prompt diagnostics outside
|
||||||
|
// ordinary application state.
|
||||||
|
package promptdebug
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
@@ -14,14 +16,14 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
promptPreparationDebugSchemaVersion = "weatherreporter.prompt_preparation_debug.v1"
|
promptPreparationDebugSchemaVersion = "weatherreporter.prompt_preparation_debug.v2"
|
||||||
promptExecutionDebugSchemaVersion = "weatherreporter.prompt_execution_debug.v1"
|
promptExecutionDebugSchemaVersion = "weatherreporter.prompt_execution_debug.v2"
|
||||||
debugDirectoryMode = 0o700
|
debugDirectoryMode = 0o700
|
||||||
debugFileMode = 0o600
|
debugFileMode = 0o600
|
||||||
)
|
)
|
||||||
|
|
||||||
// PromptDebugWriter stores explicitly requested content-rich diagnostics outside
|
// PromptDebugWriter stores explicitly requested content-rich diagnostics. A
|
||||||
// the managed report workspace. A writer created without a root is disabled.
|
// writer created without a root is disabled.
|
||||||
type PromptDebugWriter struct {
|
type PromptDebugWriter struct {
|
||||||
root string
|
root string
|
||||||
}
|
}
|
||||||
@@ -62,7 +64,6 @@ type PromptDebugPreparation struct {
|
|||||||
StartedAt time.Time `json:"startedAt"`
|
StartedAt time.Time `json:"startedAt"`
|
||||||
EndedAt time.Time `json:"endedAt"`
|
EndedAt time.Time `json:"endedAt"`
|
||||||
Duration time.Duration `json:"duration"`
|
Duration time.Duration `json:"duration"`
|
||||||
DataPackagePath string `json:"dataPackagePath"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// PromptPreparationDebugArtifact is the on-disk preparation debug record.
|
// PromptPreparationDebugArtifact is the on-disk preparation debug record.
|
||||||
@@ -112,7 +113,6 @@ type PromptDebugExecution struct {
|
|||||||
StartedAt time.Time `json:"startedAt"`
|
StartedAt time.Time `json:"startedAt"`
|
||||||
EndedAt time.Time `json:"endedAt"`
|
EndedAt time.Time `json:"endedAt"`
|
||||||
Duration time.Duration `json:"duration"`
|
Duration time.Duration `json:"duration"`
|
||||||
DataPackagePath string `json:"dataPackagePath"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// PromptExecutionDebugArtifact is the on-disk execution debug record.
|
// PromptExecutionDebugArtifact is the on-disk execution debug record.
|
||||||
@@ -338,7 +338,7 @@ func promptDebugPreparation(value promptexec.Preparation) PromptDebugPreparation
|
|||||||
RenderedPromptHash: value.RenderedPromptHash, InputHashes: copyPromptDebugMap(value.InputHashes),
|
RenderedPromptHash: value.RenderedPromptHash, InputHashes: copyPromptDebugMap(value.InputHashes),
|
||||||
ProfileID: value.ProfileID, BackendID: value.BackendID, ModelName: value.ModelName,
|
ProfileID: value.ProfileID, BackendID: value.BackendID, ModelName: value.ModelName,
|
||||||
Output: PromptDebugOutput{Format: value.Output.Format, ValidationMode: value.Output.ValidationMode, SchemaPath: value.Output.SchemaPath},
|
Output: PromptDebugOutput{Format: value.Output.Format, ValidationMode: value.Output.ValidationMode, SchemaPath: value.Output.SchemaPath},
|
||||||
StartedAt: value.StartedAt, EndedAt: value.EndedAt, Duration: value.Duration, DataPackagePath: value.DataPackagePath,
|
StartedAt: value.StartedAt, EndedAt: value.EndedAt, Duration: value.Duration,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -349,7 +349,7 @@ func promptDebugExecution(value promptexec.Execution) PromptDebugExecution {
|
|||||||
InputHashes: copyPromptDebugMap(value.InputHashes), ProfileID: value.ProfileID,
|
InputHashes: copyPromptDebugMap(value.InputHashes), ProfileID: value.ProfileID,
|
||||||
BackendID: value.BackendID, ModelName: value.ModelName, GeneratedHash: value.GeneratedHash,
|
BackendID: value.BackendID, ModelName: value.ModelName, GeneratedHash: value.GeneratedHash,
|
||||||
Usage: PromptDebugUsage{PromptTokens: value.Usage.PromptTokens, CompletionTokens: value.Usage.CompletionTokens, TotalTokens: value.Usage.TotalTokens, CachedTokens: value.Usage.CachedTokens, CacheWriteTokens: value.Usage.CacheWriteTokens},
|
Usage: PromptDebugUsage{PromptTokens: value.Usage.PromptTokens, CompletionTokens: value.Usage.CompletionTokens, TotalTokens: value.Usage.TotalTokens, CachedTokens: value.Usage.CachedTokens, CacheWriteTokens: value.Usage.CacheWriteTokens},
|
||||||
StartedAt: value.StartedAt, EndedAt: value.EndedAt, Duration: value.Duration, DataPackagePath: value.DataPackagePath,
|
StartedAt: value.StartedAt, EndedAt: value.EndedAt, Duration: value.Duration,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package state
|
package promptdebug
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"os"
|
"os"
|
||||||
@@ -40,19 +40,19 @@ func TestPromptDebugWriterWritesIsolatedArtifacts(t *testing.T) {
|
|||||||
t.Fatalf("WriteExecution() directory = %q, want %q", executionDir, preparationDir)
|
t.Fatalf("WriteExecution() directory = %q, want %q", executionDir, preparationDir)
|
||||||
}
|
}
|
||||||
preparationData := readPromptDebugFile(t, filepath.Join(preparationDir, "preparation.json"))
|
preparationData := readPromptDebugFile(t, filepath.Join(preparationDir, "preparation.json"))
|
||||||
for _, want := range []string{"Use the supplied weather facts.", `"type": "object"`, "https://llm.example.test/v1/chat?api_key=%5Bredacted%5D", `"temperature": 0.2`, `"api_key": "[redacted]"`} {
|
for _, want := range []string{"weatherreporter.prompt_preparation_debug.v2", "Use the supplied weather facts.", `"type": "object"`, "https://llm.example.test/v1/chat?api_key=%5Bredacted%5D", `"temperature": 0.2`, `"api_key": "[redacted]"`} {
|
||||||
if !strings.Contains(string(preparationData), want) {
|
if !strings.Contains(string(preparationData), want) {
|
||||||
t.Fatalf("preparation debug artifact missing %q:\n%s", want, preparationData)
|
t.Fatalf("preparation debug artifact missing %q:\n%s", want, preparationData)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
executionData := readPromptDebugFile(t, filepath.Join(executionDir, "execution.json"))
|
executionData := readPromptDebugFile(t, filepath.Join(executionDir, "execution.json"))
|
||||||
for _, want := range []string{"Generated forecast prose.", "validation details", `"status": "passed"`} {
|
for _, want := range []string{"weatherreporter.prompt_execution_debug.v2", "Generated forecast prose.", "validation details", `"status": "passed"`} {
|
||||||
if !strings.Contains(string(executionData), want) {
|
if !strings.Contains(string(executionData), want) {
|
||||||
t.Fatalf("execution debug artifact missing %q:\n%s", want, executionData)
|
t.Fatalf("execution debug artifact missing %q:\n%s", want, executionData)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
for _, data := range [][]byte{preparationData, executionData} {
|
for _, data := range [][]byte{preparationData, executionData} {
|
||||||
if strings.Contains(string(data), "credential") || strings.Contains(string(data), "resolved-secret-value") {
|
if strings.Contains(string(data), "credential") || strings.Contains(string(data), "resolved-secret-value") || strings.Contains(string(data), "dataPackagePath") {
|
||||||
t.Fatalf("debug artifact contains credentials:\n%s", data)
|
t.Fatalf("debug artifact contains credentials:\n%s", data)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -169,7 +169,7 @@ func promptDebugPreparationFixture() promptexec.Preparation {
|
|||||||
PromptID: "weather.daily", PromptVersion: "v1", PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash",
|
PromptID: "weather.daily", PromptVersion: "v1", PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash",
|
||||||
InputHashes: map[string]string{"data_package": "input-hash"}, ProfileID: "local", BackendID: "local", ModelName: "weather-model",
|
InputHashes: map[string]string{"data_package": "input-hash"}, ProfileID: "local", BackendID: "local", ModelName: "weather-model",
|
||||||
Output: promptexec.OutputContract{Format: "json_schema", ValidationMode: "strict", SchemaPath: "schemas/daily.json"},
|
Output: promptexec.OutputContract{Format: "json_schema", ValidationMode: "strict", SchemaPath: "schemas/daily.json"},
|
||||||
StartedAt: startedAt, EndedAt: startedAt.Add(time.Second), Duration: time.Second, DataPackagePath: "/packages/daily.yaml",
|
StartedAt: startedAt, EndedAt: startedAt.Add(time.Second), Duration: time.Second,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -181,7 +181,7 @@ func promptDebugExecutionFixture() promptexec.Execution {
|
|||||||
GeneratedHash: "generated-hash", Usage: promptexec.TokenUsage{PromptTokens: 10, CompletionTokens: 5, TotalTokens: 15},
|
GeneratedHash: "generated-hash", Usage: promptexec.TokenUsage{PromptTokens: 10, CompletionTokens: 5, TotalTokens: 15},
|
||||||
StartedAt: startedAt, EndedAt: startedAt.Add(time.Second), Duration: time.Second,
|
StartedAt: startedAt, EndedAt: startedAt.Add(time.Second), Duration: time.Second,
|
||||||
Validation: promptexec.NewValidation(promptexec.ValidationPassed, "strict", "schemas/daily.json", []string{"validation details"}),
|
Validation: promptexec.NewValidation(promptexec.ValidationPassed, "strict", "schemas/daily.json", []string{"validation details"}),
|
||||||
DataPackagePath: "/packages/daily.yaml", RawOutput: []byte("Generated forecast prose."),
|
RawOutput: []byte("Generated forecast prose."),
|
||||||
Debug: &promptexec.ExecutionDebug{ValidationDiagnostics: []string{"validation details"}},
|
Debug: &promptexec.ExecutionDebug{ValidationDiagnostics: []string{"validation details"}},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -61,14 +61,12 @@ type ProfileInspection struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ExecuteRequest selects one exact prompt execution. DataPackage is the exact
|
// ExecuteRequest selects one exact prompt execution. DataPackage is the exact
|
||||||
// YAML input; implementations must copy it before retaining it. DataPackagePath
|
// YAML input; implementations must copy it before retaining it.
|
||||||
// is provenance for the inline input, not a provider-readable file reference.
|
|
||||||
type ExecuteRequest struct {
|
type ExecuteRequest struct {
|
||||||
PromptID string
|
PromptID string
|
||||||
PromptVersion string
|
PromptVersion string
|
||||||
ProfileID string
|
ProfileID string
|
||||||
DataPackage []byte
|
DataPackage []byte
|
||||||
DataPackagePath string
|
|
||||||
CaptureDebug bool
|
CaptureDebug bool
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -90,7 +88,6 @@ type Preparation struct {
|
|||||||
StartedAt time.Time
|
StartedAt time.Time
|
||||||
EndedAt time.Time
|
EndedAt time.Time
|
||||||
Duration time.Duration
|
Duration time.Duration
|
||||||
DataPackagePath string
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// PreparationDebug contains content-rich preparation details for an explicitly
|
// PreparationDebug contains content-rich preparation details for an explicitly
|
||||||
@@ -127,7 +124,6 @@ type Execution struct {
|
|||||||
EndedAt time.Time
|
EndedAt time.Time
|
||||||
Duration time.Duration
|
Duration time.Duration
|
||||||
Validation Validation
|
Validation Validation
|
||||||
DataPackagePath string
|
|
||||||
RawOutput []byte
|
RawOutput []byte
|
||||||
Debug *ExecutionDebug
|
Debug *ExecutionDebug
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -42,7 +42,7 @@ func (executor *lifecycleExecutor) Execute(_ context.Context, request ExecuteReq
|
|||||||
if executor.operationalFailure != nil {
|
if executor.operationalFailure != nil {
|
||||||
return nil, executor.operationalFailure
|
return nil, executor.operationalFailure
|
||||||
}
|
}
|
||||||
preparation := Preparation{PromptID: request.PromptID, PromptVersion: request.PromptVersion, DataPackagePath: request.DataPackagePath}
|
preparation := Preparation{PromptID: request.PromptID, PromptVersion: request.PromptVersion}
|
||||||
var debug *PreparationDebug
|
var debug *PreparationDebug
|
||||||
if request.CaptureDebug {
|
if request.CaptureDebug {
|
||||||
debug = &PreparationDebug{RenderedMessages: []RenderedMessage{{Role: "user", Content: "sensitive rendered message"}}}
|
debug = &PreparationDebug{RenderedMessages: []RenderedMessage{{Role: "user", Content: "sensitive rendered message"}}}
|
||||||
@@ -57,7 +57,7 @@ func (executor *lifecycleExecutor) Execute(_ context.Context, request ExecuteReq
|
|||||||
if executor.validationRejected {
|
if executor.validationRejected {
|
||||||
status = ValidationFailed
|
status = ValidationFailed
|
||||||
}
|
}
|
||||||
result := Execution{PromptID: request.PromptID, PromptVersion: request.PromptVersion, DataPackagePath: request.DataPackagePath, Validation: Validation{Status: status}, RawOutput: []byte("generated content")}
|
result := Execution{PromptID: request.PromptID, PromptVersion: request.PromptVersion, Validation: Validation{Status: status}, RawOutput: []byte("generated content")}
|
||||||
if request.CaptureDebug {
|
if request.CaptureDebug {
|
||||||
result.Debug = &ExecutionDebug{RawOutput: []byte("generated content")}
|
result.Debug = &ExecutionDebug{RawOutput: []byte("generated content")}
|
||||||
}
|
}
|
||||||
@@ -65,7 +65,7 @@ func (executor *lifecycleExecutor) Execute(_ context.Context, request ExecuteReq
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestExecutorLifecycleFixtures(t *testing.T) {
|
func TestExecutorLifecycleFixtures(t *testing.T) {
|
||||||
request := ExecuteRequest{PromptID: "weather.daily_generated_text", PromptVersion: "1.0.0", DataPackagePath: "data_package.yaml"}
|
request := ExecuteRequest{PromptID: "weather.daily_generated_text", PromptVersion: "1.0.0"}
|
||||||
t.Run("callback failure prevents provider execution", func(t *testing.T) {
|
t.Run("callback failure prevents provider execution", func(t *testing.T) {
|
||||||
executor := &lifecycleExecutor{}
|
executor := &lifecycleExecutor{}
|
||||||
callbackError := errors.New("persistence failed")
|
callbackError := errors.New("persistence failed")
|
||||||
@@ -239,7 +239,6 @@ func TestSafeContractValuesExcludeSensitiveFields(t *testing.T) {
|
|||||||
BackendID: "local",
|
BackendID: "local",
|
||||||
ModelName: "model-name",
|
ModelName: "model-name",
|
||||||
Output: OutputContract{Format: "json", ValidationMode: "json_schema", SchemaPath: "daily.generated_text.schema.json"},
|
Output: OutputContract{Format: "json", ValidationMode: "json_schema", SchemaPath: "daily.generated_text.schema.json"},
|
||||||
DataPackagePath: "data-packages/daily/data_package.yaml",
|
|
||||||
}
|
}
|
||||||
execution := Execution{
|
execution := Execution{
|
||||||
RunID: "run-id",
|
RunID: "run-id",
|
||||||
@@ -254,7 +253,7 @@ func TestSafeContractValuesExcludeSensitiveFields(t *testing.T) {
|
|||||||
GeneratedHash: "generated-hash",
|
GeneratedHash: "generated-hash",
|
||||||
RawOutput: []byte("generated content"),
|
RawOutput: []byte("generated content"),
|
||||||
}
|
}
|
||||||
text := preparation.PromptID + preparation.PromptVersion + preparation.PromptHash + preparation.RenderedPromptHash + preparation.ProfileID + preparation.BackendID + preparation.ModelName + preparation.Output.SchemaPath + preparation.DataPackagePath + execution.RunID + execution.GeneratedHash
|
text := preparation.PromptID + preparation.PromptVersion + preparation.PromptHash + preparation.RenderedPromptHash + preparation.ProfileID + preparation.BackendID + preparation.ModelName + preparation.Output.SchemaPath + execution.RunID + execution.GeneratedHash
|
||||||
for _, unwanted := range []string{"https://provider.example", "API_KEY_ENV", "rendered message", "schema body", "input body", "provider response body", "full parameters"} {
|
for _, unwanted := range []string{"https://provider.example", "API_KEY_ENV", "rendered message", "schema body", "input body", "provider response body", "full parameters"} {
|
||||||
if strings.Contains(text, unwanted) {
|
if strings.Contains(text, unwanted) {
|
||||||
t.Fatalf("safe values contain %q: %s", unwanted, text)
|
t.Fatalf("safe values contain %q: %s", unwanted, text)
|
||||||
|
|||||||
@@ -7,7 +7,6 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/changes"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/fileutil"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/fileutil"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
||||||
@@ -16,7 +15,7 @@ import (
|
|||||||
"gopkg.in/yaml.v3"
|
"gopkg.in/yaml.v3"
|
||||||
)
|
)
|
||||||
|
|
||||||
const SchemaVersion = "weatherreporter.data_package.v3"
|
const SchemaVersion = "weatherreporter.data_package.v4"
|
||||||
|
|
||||||
const (
|
const (
|
||||||
metadataStanza = "metadata"
|
metadataStanza = "metadata"
|
||||||
@@ -54,7 +53,6 @@ var briefingStanzaCategories = map[string]string{
|
|||||||
type BuildRequest struct {
|
type BuildRequest struct {
|
||||||
Metadata Metadata
|
Metadata Metadata
|
||||||
Modules module.Snapshot
|
Modules module.Snapshot
|
||||||
RecentChanges []changes.Change
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type Metadata struct {
|
type Metadata struct {
|
||||||
@@ -73,7 +71,6 @@ type Package struct {
|
|||||||
RunID string `json:"runId" yaml:"run_id"`
|
RunID string `json:"runId" yaml:"run_id"`
|
||||||
Report Report `json:"report" yaml:"report"`
|
Report Report `json:"report" yaml:"report"`
|
||||||
Briefing BriefingStanzas `json:"briefing" yaml:"briefing"`
|
Briefing BriefingStanzas `json:"briefing" yaml:"briefing"`
|
||||||
RecentChanges RecentChanges `json:"recentChanges" yaml:"recent_changes"`
|
|
||||||
SourceWarnings []weatherdata.SourceWarning `json:"sourceWarnings,omitempty" yaml:"source_warnings,omitempty"`
|
SourceWarnings []weatherdata.SourceWarning `json:"sourceWarnings,omitempty" yaml:"source_warnings,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -92,20 +89,11 @@ type BriefingStanzas struct {
|
|||||||
Values map[string]any `json:"-" yaml:"-"`
|
Values map[string]any `json:"-" yaml:"-"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type RecentChanges struct {
|
|
||||||
Items []changes.Change `json:"items" yaml:"items"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func Build(req BuildRequest) (Package, error) {
|
func Build(req BuildRequest) (Package, error) {
|
||||||
localDate, err := currentLocalDate(req.Metadata.GeneratedAt, req.Metadata.Timezone)
|
localDate, err := currentLocalDate(req.Metadata.GeneratedAt, req.Metadata.Timezone)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return Package{}, err
|
return Package{}, err
|
||||||
}
|
}
|
||||||
items := make([]changes.Change, len(req.RecentChanges))
|
|
||||||
copy(items, req.RecentChanges)
|
|
||||||
if items == nil {
|
|
||||||
items = []changes.Change{}
|
|
||||||
}
|
|
||||||
pkg := Package{
|
pkg := Package{
|
||||||
SchemaVersion: SchemaVersion,
|
SchemaVersion: SchemaVersion,
|
||||||
RunID: req.Metadata.RunID,
|
RunID: req.Metadata.RunID,
|
||||||
@@ -119,7 +107,6 @@ func Build(req BuildRequest) (Package, error) {
|
|||||||
ValidPeriod: req.Metadata.ValidPeriod,
|
ValidPeriod: req.Metadata.ValidPeriod,
|
||||||
},
|
},
|
||||||
Briefing: stanzasFromSnapshot(req.Modules),
|
Briefing: stanzasFromSnapshot(req.Modules),
|
||||||
RecentChanges: RecentChanges{Items: items},
|
|
||||||
SourceWarnings: append([]weatherdata.SourceWarning(nil), req.Metadata.SourceWarnings...),
|
SourceWarnings: append([]weatherdata.SourceWarning(nil), req.Metadata.SourceWarnings...),
|
||||||
}
|
}
|
||||||
if err := Validate(pkg); err != nil {
|
if err := Validate(pkg); err != nil {
|
||||||
|
|||||||
@@ -34,9 +34,6 @@ func TestBuildDailyDataPackage(t *testing.T) {
|
|||||||
if got := pkg.Briefing.Values["current_conditions"].(map[string]string)["condition_text"]; got != "Partly cloudy" {
|
if got := pkg.Briefing.Values["current_conditions"].(map[string]string)["condition_text"]; got != "Partly cloudy" {
|
||||||
t.Fatalf("current_conditions.condition_text = %q, want Partly cloudy", got)
|
t.Fatalf("current_conditions.condition_text = %q, want Partly cloudy", got)
|
||||||
}
|
}
|
||||||
if pkg.RecentChanges.Items == nil || len(pkg.RecentChanges.Items) != 0 {
|
|
||||||
t.Fatalf("RecentChanges.Items = %#v, want empty slice", pkg.RecentChanges.Items)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBuildCurrentLocalDateUsesReportTimezone(t *testing.T) {
|
func TestBuildCurrentLocalDateUsesReportTimezone(t *testing.T) {
|
||||||
@@ -181,7 +178,7 @@ func TestMarshalYAMLIsDeterministicAndGroupsNamedStanzas(t *testing.T) {
|
|||||||
if string(first) != string(second) {
|
if string(first) != string(second) {
|
||||||
t.Fatalf("YAML output changed between marshals:\n%s\n---\n%s", string(first), string(second))
|
t.Fatalf("YAML output changed between marshals:\n%s\n---\n%s", string(first), string(second))
|
||||||
}
|
}
|
||||||
if !strings.Contains(string(first), "schema_version: weatherreporter.data_package.v3") ||
|
if !strings.Contains(string(first), "schema_version: weatherreporter.data_package.v4") ||
|
||||||
!strings.Contains(string(first), "briefing:\n") ||
|
!strings.Contains(string(first), "briefing:\n") ||
|
||||||
!strings.Contains(string(first), " applicable_risk_products:\n") ||
|
!strings.Contains(string(first), " applicable_risk_products:\n") ||
|
||||||
!strings.Contains(string(first), " derived_summaries:\n") ||
|
!strings.Contains(string(first), " derived_summaries:\n") ||
|
||||||
@@ -191,6 +188,9 @@ func TestMarshalYAMLIsDeterministicAndGroupsNamedStanzas(t *testing.T) {
|
|||||||
!strings.Contains(string(first), " condition_text: Partly cloudy") {
|
!strings.Contains(string(first), " condition_text: Partly cloudy") {
|
||||||
t.Fatalf("YAML output missing expected grouped stanzas:\n%s", string(first))
|
t.Fatalf("YAML output missing expected grouped stanzas:\n%s", string(first))
|
||||||
}
|
}
|
||||||
|
if strings.Contains(string(first), "recent_changes") {
|
||||||
|
t.Fatalf("YAML output contains removed recent_changes stanza:\n%s", string(first))
|
||||||
|
}
|
||||||
for _, pair := range []struct {
|
for _, pair := range []struct {
|
||||||
before string
|
before string
|
||||||
after string
|
after string
|
||||||
@@ -289,7 +289,7 @@ func TestMarshalYAMLRejectsUncategorizedStanza(t *testing.T) {
|
|||||||
|
|
||||||
func TestLoadYAMLRejectsMisplacedStanza(t *testing.T) {
|
func TestLoadYAMLRejectsMisplacedStanza(t *testing.T) {
|
||||||
data := []byte(`
|
data := []byte(`
|
||||||
schema_version: weatherreporter.data_package.v3
|
schema_version: weatherreporter.data_package.v4
|
||||||
run_id: 20260529T100000Z_daily
|
run_id: 20260529T100000Z_daily
|
||||||
report:
|
report:
|
||||||
id: daily
|
id: daily
|
||||||
@@ -306,8 +306,6 @@ briefing:
|
|||||||
raw_data:
|
raw_data:
|
||||||
alert_digest:
|
alert_digest:
|
||||||
checked: true
|
checked: true
|
||||||
recent_changes:
|
|
||||||
items: []
|
|
||||||
`)
|
`)
|
||||||
|
|
||||||
_, err := LoadYAML(data)
|
_, err := LoadYAML(data)
|
||||||
@@ -325,10 +323,10 @@ func TestLoadYAMLRejectsOldSchemaVersion(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("MarshalYAML() error = %v", err)
|
t.Fatalf("MarshalYAML() error = %v", err)
|
||||||
}
|
}
|
||||||
data = []byte(strings.Replace(string(data), "weatherreporter.data_package.v3", "weatherreporter.data_package.v2", 1))
|
data = []byte(strings.Replace(string(data), "weatherreporter.data_package.v4", "weatherreporter.data_package.v3", 1))
|
||||||
|
|
||||||
_, err = LoadYAML(data)
|
_, err = LoadYAML(data)
|
||||||
if err == nil || !strings.Contains(err.Error(), "schemaVersion must be weatherreporter.data_package.v3") {
|
if err == nil || !strings.Contains(err.Error(), "schemaVersion must be weatherreporter.data_package.v4") {
|
||||||
t.Fatalf("LoadYAML() error = %v, want current schema version error", err)
|
t.Fatalf("LoadYAML() error = %v, want current schema version error", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,17 +12,15 @@ func dailyDefinition() Definition {
|
|||||||
ID: Daily,
|
ID: Daily,
|
||||||
Name: "Daily Report",
|
Name: "Daily Report",
|
||||||
PromptID: "weather.daily_generated_text",
|
PromptID: "weather.daily_generated_text",
|
||||||
PromptVersion: "1.1.0",
|
PromptVersion: "2.0.0",
|
||||||
TemplateID: "daily",
|
TemplateID: "daily",
|
||||||
GeneratedTextSchemaID: "daily",
|
GeneratedTextSchemaID: "daily",
|
||||||
ComparisonStrategy: CompareSameValidDate,
|
|
||||||
ArtifactGroup: "daily",
|
ArtifactGroup: "daily",
|
||||||
BatchOutputName: "daily.md",
|
OutputName: "daily.md",
|
||||||
DistributorPathTemplates: []string{
|
DistributorPathTemplates: []string{
|
||||||
"daily/{valid_start_date}/{run_id}.md",
|
"daily/{valid_start_date}/{run_id}.md",
|
||||||
"daily/{valid_start_date}/index.md",
|
"daily/{valid_start_date}/index.md",
|
||||||
},
|
},
|
||||||
CompatiblePriorIDs: []ID{Daily},
|
|
||||||
Modules: dailyModules(),
|
Modules: dailyModules(),
|
||||||
resolve: resolveDaily,
|
resolve: resolveDaily,
|
||||||
runIDDisambiguator: validStartDateRunIDDisambiguator,
|
runIDDisambiguator: validStartDateRunIDDisambiguator,
|
||||||
|
|||||||
@@ -19,13 +19,6 @@ const (
|
|||||||
Hourly ID = "hourly"
|
Hourly ID = "hourly"
|
||||||
)
|
)
|
||||||
|
|
||||||
type ComparisonStrategy string
|
|
||||||
|
|
||||||
const (
|
|
||||||
CompareSameValidDate ComparisonStrategy = "same_valid_date"
|
|
||||||
CompareRollingWindow ComparisonStrategy = "rolling_window"
|
|
||||||
)
|
|
||||||
|
|
||||||
type Batch string
|
type Batch string
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -40,11 +33,9 @@ type Definition struct {
|
|||||||
PromptVersion string
|
PromptVersion string
|
||||||
TemplateID string
|
TemplateID string
|
||||||
GeneratedTextSchemaID string
|
GeneratedTextSchemaID string
|
||||||
ComparisonStrategy ComparisonStrategy
|
|
||||||
ArtifactGroup string
|
ArtifactGroup string
|
||||||
BatchOutputName string
|
OutputName string
|
||||||
DistributorPathTemplates []string
|
DistributorPathTemplates []string
|
||||||
CompatiblePriorIDs []ID
|
|
||||||
Modules []module.ConfigItem
|
Modules []module.ConfigItem
|
||||||
Morning bool
|
Morning bool
|
||||||
Evening bool
|
Evening bool
|
||||||
@@ -52,6 +43,23 @@ type Definition struct {
|
|||||||
runIDDisambiguator func(Resolved) string
|
runIDDisambiguator func(Resolved) string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (r Resolved) OutputName() (string, error) {
|
||||||
|
if r.Definition.ID == Daily {
|
||||||
|
if r.ValidPeriod.Start.IsZero() {
|
||||||
|
return "", fmt.Errorf("daily report has no valid-period start for output naming")
|
||||||
|
}
|
||||||
|
location, err := timeutil.LoadLocation(r.Timezone)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("load report timezone for output naming: %w", err)
|
||||||
|
}
|
||||||
|
return "daily-" + r.ValidPeriod.Start.In(location).Format(timeutil.DateLayout) + ".md", nil
|
||||||
|
}
|
||||||
|
if r.Definition.OutputName == "" {
|
||||||
|
return "", fmt.Errorf("report %q has no output name", r.Definition.ID)
|
||||||
|
}
|
||||||
|
return r.Definition.OutputName, nil
|
||||||
|
}
|
||||||
|
|
||||||
func (d Definition) ResolvePeriod(req ResolveRequest) (timeutil.Period, error) {
|
func (d Definition) ResolvePeriod(req ResolveRequest) (timeutil.Period, error) {
|
||||||
if d.resolve == nil {
|
if d.resolve == nil {
|
||||||
return timeutil.Period{}, fmt.Errorf("report %q has no valid-period resolver", d.ID)
|
return timeutil.Period{}, fmt.Errorf("report %q has no valid-period resolver", d.ID)
|
||||||
@@ -59,15 +67,6 @@ func (d Definition) ResolvePeriod(req ResolveRequest) (timeutil.Period, error) {
|
|||||||
return d.resolve(req)
|
return d.resolve(req)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (d Definition) CompatibleWithPrior(id ID) bool {
|
|
||||||
for _, compatibleID := range d.CompatiblePriorIDs {
|
|
||||||
if id == compatibleID {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
func (d Definition) ModuleIDs() []module.ID {
|
func (d Definition) ModuleIDs() []module.ID {
|
||||||
ids := make([]module.ID, 0, len(d.Modules))
|
ids := make([]module.ID, 0, len(d.Modules))
|
||||||
for _, item := range d.Modules {
|
for _, item := range d.Modules {
|
||||||
|
|||||||
@@ -14,16 +14,14 @@ func hourlyDefinition() Definition {
|
|||||||
ID: Hourly,
|
ID: Hourly,
|
||||||
Name: "Hourly Report",
|
Name: "Hourly Report",
|
||||||
PromptID: "weather.hourly_generated_text",
|
PromptID: "weather.hourly_generated_text",
|
||||||
PromptVersion: "1.1.0",
|
PromptVersion: "2.0.0",
|
||||||
TemplateID: "hourly",
|
TemplateID: "hourly",
|
||||||
GeneratedTextSchemaID: "hourly",
|
GeneratedTextSchemaID: "hourly",
|
||||||
ComparisonStrategy: CompareRollingWindow,
|
|
||||||
ArtifactGroup: "hourly",
|
ArtifactGroup: "hourly",
|
||||||
BatchOutputName: "hourly.md",
|
OutputName: "hourly.md",
|
||||||
DistributorPathTemplates: []string{
|
DistributorPathTemplates: []string{
|
||||||
"hourly/index.md",
|
"hourly/index.md",
|
||||||
},
|
},
|
||||||
CompatiblePriorIDs: []ID{Hourly},
|
|
||||||
Modules: hourlyModules(),
|
Modules: hourlyModules(),
|
||||||
resolve: resolveHourly,
|
resolve: resolveHourly,
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -51,8 +51,8 @@ func TestRegistryContainsOnlyPromptBackedReports(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
for _, definition := range definitions {
|
for _, definition := range definitions {
|
||||||
if definition.PromptVersion != "1.1.0" {
|
if definition.PromptVersion != "2.0.0" {
|
||||||
t.Fatalf("%s PromptVersion = %q, want 1.1.0", definition.ID, definition.PromptVersion)
|
t.Fatalf("%s PromptVersion = %q, want 2.0.0", definition.ID, definition.PromptVersion)
|
||||||
}
|
}
|
||||||
if definition.PromptID == "" {
|
if definition.PromptID == "" {
|
||||||
t.Fatalf("%s PromptID is empty", definition.ID)
|
t.Fatalf("%s PromptID is empty", definition.ID)
|
||||||
@@ -87,23 +87,22 @@ func TestCommandAndConfigurationNamesRejectRetiredReports(t *testing.T) {
|
|||||||
func TestRegistryDefinitionsPreserveRetainedContracts(t *testing.T) {
|
func TestRegistryDefinitionsPreserveRetainedContracts(t *testing.T) {
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
id ID
|
id ID
|
||||||
comparison ComparisonStrategy
|
|
||||||
morning bool
|
morning bool
|
||||||
evening bool
|
evening bool
|
||||||
outputName string
|
outputName string
|
||||||
paths []string
|
paths []string
|
||||||
}{
|
}{
|
||||||
{Daily, CompareSameValidDate, false, false, "daily.md", []string{"daily/{valid_start_date}/{run_id}.md", "daily/{valid_start_date}/index.md"}},
|
{Daily, false, false, "daily.md", []string{"daily/{valid_start_date}/{run_id}.md", "daily/{valid_start_date}/index.md"}},
|
||||||
{Today, CompareSameValidDate, true, false, "today.md", []string{"daily/{valid_start_date}/{run_id}.md", "daily/{valid_start_date}/index.md", "today/index.md"}},
|
{Today, true, false, "today.md", []string{"daily/{valid_start_date}/{run_id}.md", "daily/{valid_start_date}/index.md", "today/index.md"}},
|
||||||
{Tomorrow, CompareSameValidDate, false, true, "tomorrow.md", []string{"daily/{valid_start_date}/{run_id}.md", "daily/{valid_start_date}/index.md", "tomorrow/index.md"}},
|
{Tomorrow, false, true, "tomorrow.md", []string{"daily/{valid_start_date}/{run_id}.md", "daily/{valid_start_date}/index.md", "tomorrow/index.md"}},
|
||||||
{Hourly, CompareRollingWindow, false, false, "hourly.md", []string{"hourly/index.md"}},
|
{Hourly, false, false, "hourly.md", []string{"hourly/index.md"}},
|
||||||
}
|
}
|
||||||
|
|
||||||
registry := DefaultRegistry()
|
registry := DefaultRegistry()
|
||||||
for _, tt := range tests {
|
for _, tt := range tests {
|
||||||
t.Run(string(tt.id), func(t *testing.T) {
|
t.Run(string(tt.id), func(t *testing.T) {
|
||||||
definition := registry.MustLookup(tt.id)
|
definition := registry.MustLookup(tt.id)
|
||||||
if definition.ComparisonStrategy != tt.comparison || definition.Morning != tt.morning || definition.Evening != tt.evening || definition.BatchOutputName != tt.outputName {
|
if definition.Morning != tt.morning || definition.Evening != tt.evening || definition.OutputName != tt.outputName {
|
||||||
t.Fatalf("definition = %#v, want retained report contract", definition)
|
t.Fatalf("definition = %#v, want retained report contract", definition)
|
||||||
}
|
}
|
||||||
if strings.Join(definition.DistributorPathTemplates, "|") != strings.Join(tt.paths, "|") {
|
if strings.Join(definition.DistributorPathTemplates, "|") != strings.Join(tt.paths, "|") {
|
||||||
|
|||||||
@@ -10,18 +10,16 @@ func todayDefinition() Definition {
|
|||||||
ID: Today,
|
ID: Today,
|
||||||
Name: "Today Report",
|
Name: "Today Report",
|
||||||
PromptID: "weather.today_generated_text",
|
PromptID: "weather.today_generated_text",
|
||||||
PromptVersion: "1.1.0",
|
PromptVersion: "2.0.0",
|
||||||
TemplateID: "today",
|
TemplateID: "today",
|
||||||
GeneratedTextSchemaID: "today",
|
GeneratedTextSchemaID: "today",
|
||||||
ComparisonStrategy: CompareSameValidDate,
|
|
||||||
ArtifactGroup: "today",
|
ArtifactGroup: "today",
|
||||||
BatchOutputName: "today.md",
|
OutputName: "today.md",
|
||||||
DistributorPathTemplates: []string{
|
DistributorPathTemplates: []string{
|
||||||
"daily/{valid_start_date}/{run_id}.md",
|
"daily/{valid_start_date}/{run_id}.md",
|
||||||
"daily/{valid_start_date}/index.md",
|
"daily/{valid_start_date}/index.md",
|
||||||
"today/index.md",
|
"today/index.md",
|
||||||
},
|
},
|
||||||
CompatiblePriorIDs: []ID{Today},
|
|
||||||
Modules: todayModules(),
|
Modules: todayModules(),
|
||||||
Morning: true,
|
Morning: true,
|
||||||
resolve: resolveToday,
|
resolve: resolveToday,
|
||||||
|
|||||||
@@ -10,18 +10,16 @@ func tomorrowDefinition() Definition {
|
|||||||
ID: Tomorrow,
|
ID: Tomorrow,
|
||||||
Name: "Tomorrow Report",
|
Name: "Tomorrow Report",
|
||||||
PromptID: "weather.tomorrow_generated_text",
|
PromptID: "weather.tomorrow_generated_text",
|
||||||
PromptVersion: "1.1.0",
|
PromptVersion: "2.0.0",
|
||||||
TemplateID: "tomorrow",
|
TemplateID: "tomorrow",
|
||||||
GeneratedTextSchemaID: "tomorrow",
|
GeneratedTextSchemaID: "tomorrow",
|
||||||
ComparisonStrategy: CompareSameValidDate,
|
|
||||||
ArtifactGroup: "tomorrow",
|
ArtifactGroup: "tomorrow",
|
||||||
BatchOutputName: "tomorrow.md",
|
OutputName: "tomorrow.md",
|
||||||
DistributorPathTemplates: []string{
|
DistributorPathTemplates: []string{
|
||||||
"daily/{valid_start_date}/{run_id}.md",
|
"daily/{valid_start_date}/{run_id}.md",
|
||||||
"daily/{valid_start_date}/index.md",
|
"daily/{valid_start_date}/index.md",
|
||||||
"tomorrow/index.md",
|
"tomorrow/index.md",
|
||||||
},
|
},
|
||||||
CompatiblePriorIDs: []ID{Tomorrow},
|
|
||||||
Modules: tomorrowModules(),
|
Modules: tomorrowModules(),
|
||||||
Evening: true,
|
Evening: true,
|
||||||
resolve: resolveTomorrow,
|
resolve: resolveTomorrow,
|
||||||
|
|||||||
@@ -1,582 +0,0 @@
|
|||||||
package state
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"sort"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/fileutil"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
)
|
|
||||||
|
|
||||||
type FilesystemStore struct {
|
|
||||||
root string
|
|
||||||
snapshotsDir string
|
|
||||||
reportsDir string
|
|
||||||
dataPackagesDir string
|
|
||||||
preflightDir string
|
|
||||||
notificationsDir string
|
|
||||||
}
|
|
||||||
|
|
||||||
type ArtifactPaths struct {
|
|
||||||
ModuleSnapshot string `json:"moduleSnapshot"`
|
|
||||||
Metadata string `json:"metadata"`
|
|
||||||
DataPackage string `json:"dataPackage"`
|
|
||||||
Preparation string `json:"preparation,omitempty"`
|
|
||||||
Execution string `json:"execution,omitempty"`
|
|
||||||
Notification string `json:"notification,omitempty"`
|
|
||||||
RenderedReport string `json:"renderedReport,omitempty"`
|
|
||||||
GeneratedTextRaw string `json:"generatedTextRaw,omitempty"`
|
|
||||||
GeneratedText string `json:"generatedText,omitempty"`
|
|
||||||
RenderContext string `json:"renderContext,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type ReportRecord struct {
|
|
||||||
RunID string `json:"runId"`
|
|
||||||
ReportID report.ID `json:"reportId"`
|
|
||||||
Variant string `json:"variant,omitempty"`
|
|
||||||
PromptID string `json:"promptId"`
|
|
||||||
GeneratedAt string `json:"generatedAt"`
|
|
||||||
ValidStart string `json:"validStart"`
|
|
||||||
ValidEnd string `json:"validEnd"`
|
|
||||||
MetadataPath string `json:"metadataPath"`
|
|
||||||
ReportPath string `json:"reportPath,omitempty"`
|
|
||||||
Warnings int `json:"warnings"`
|
|
||||||
metadata Metadata
|
|
||||||
}
|
|
||||||
|
|
||||||
func NewFilesystemStore(cfg config.WorkspaceConfig) (*FilesystemStore, error) {
|
|
||||||
if cfg.Root == "" {
|
|
||||||
return nil, fmt.Errorf("workspace root is required")
|
|
||||||
}
|
|
||||||
for name, value := range map[string]string{
|
|
||||||
"snapshots_dir": cfg.SnapshotsDir,
|
|
||||||
"reports_dir": cfg.ReportsDir,
|
|
||||||
"data_packages_dir": cfg.DataPackagesDir,
|
|
||||||
"preflight_dir": cfg.PreflightDir,
|
|
||||||
"notifications_dir": cfg.NotificationsDir,
|
|
||||||
} {
|
|
||||||
if err := validateRelativeDir(name, value); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return &FilesystemStore{
|
|
||||||
root: filepath.Clean(cfg.Root),
|
|
||||||
snapshotsDir: filepath.Clean(cfg.SnapshotsDir),
|
|
||||||
reportsDir: filepath.Clean(cfg.ReportsDir),
|
|
||||||
dataPackagesDir: filepath.Clean(cfg.DataPackagesDir),
|
|
||||||
preflightDir: filepath.Clean(cfg.PreflightDir),
|
|
||||||
notificationsDir: filepath.Clean(cfg.NotificationsDir),
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) Paths(resolved report.Resolved) (ArtifactPaths, error) {
|
|
||||||
if s == nil {
|
|
||||||
return ArtifactPaths{}, fmt.Errorf("state store is required")
|
|
||||||
}
|
|
||||||
metadata := resolved.Metadata()
|
|
||||||
if err := validatePathSegment("run id", metadata.RunID); err != nil {
|
|
||||||
return ArtifactPaths{}, err
|
|
||||||
}
|
|
||||||
group := resolved.Definition.ArtifactGroup
|
|
||||||
if group == "" {
|
|
||||||
return ArtifactPaths{}, fmt.Errorf("report %q has no artifact group", resolved.Definition.ID)
|
|
||||||
}
|
|
||||||
validDate := resolved.ValidPeriod.Start.Format("2006-01-02")
|
|
||||||
return ArtifactPaths{
|
|
||||||
ModuleSnapshot: s.join(s.snapshotsDir, group, validDate, "modules."+metadata.RunID+".json"),
|
|
||||||
Metadata: s.join(s.snapshotsDir, group, validDate, "metadata."+metadata.RunID+".json"),
|
|
||||||
DataPackage: s.join(s.dataPackagesDir, group, validDate, "data_package."+metadata.RunID+".yaml"),
|
|
||||||
Preparation: s.join(s.preflightDir, group, validDate, "prompt_preparation."+metadata.RunID+".json"),
|
|
||||||
Execution: s.join(s.snapshotsDir, group, validDate, "prompt_execution."+metadata.RunID+".json"),
|
|
||||||
Notification: s.join(s.notificationsDir, group, validDate, "distributor."+metadata.RunID+".json"),
|
|
||||||
RenderedReport: s.join(s.reportsDir, group, validDate, "report."+metadata.RunID+".md"),
|
|
||||||
GeneratedTextRaw: s.join(s.snapshotsDir, group, validDate, "generated_text_raw."+metadata.RunID+".json"),
|
|
||||||
GeneratedText: s.join(s.snapshotsDir, group, validDate, "generated_text."+metadata.RunID+".json"),
|
|
||||||
RenderContext: s.join(s.snapshotsDir, group, validDate, "render_context."+metadata.RunID+".json"),
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SaveModuleSnapshot(_ context.Context, resolved report.Resolved, snapshot module.Snapshot) (string, error) {
|
|
||||||
if err := snapshot.Validate(); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return s.saveResolvedJSON(resolved, func(paths ArtifactPaths) string {
|
|
||||||
return paths.ModuleSnapshot
|
|
||||||
}, snapshot)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SaveDataPackage(ctx context.Context, resolved report.Resolved, pkg promptinput.Package) (string, error) {
|
|
||||||
data, err := promptinput.MarshalYAML(pkg)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return s.SaveDataPackageBytes(ctx, resolved, data)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SaveDataPackageBytes(_ context.Context, resolved report.Resolved, data []byte) (string, error) {
|
|
||||||
paths, err := s.Paths(resolved)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
if err := fileutil.WriteFileAtomic(paths.DataPackage, data); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return paths.DataPackage, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SavePromptPreparation(_ context.Context, resolved report.Resolved, artifact PromptPreparationArtifact) (string, error) {
|
|
||||||
if artifact.SchemaVersion == "" {
|
|
||||||
artifact.SchemaVersion = PromptPreparationSchemaVersion
|
|
||||||
}
|
|
||||||
if err := artifact.Validate(); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return s.saveResolvedJSON(resolved, func(paths ArtifactPaths) string {
|
|
||||||
return paths.Preparation
|
|
||||||
}, artifact)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SavePromptExecution(_ context.Context, resolved report.Resolved, artifact PromptExecutionArtifact) (string, error) {
|
|
||||||
if artifact.SchemaVersion == "" {
|
|
||||||
artifact.SchemaVersion = PromptExecutionSchemaVersion
|
|
||||||
}
|
|
||||||
if err := artifact.Validate(); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return s.saveResolvedJSON(resolved, func(paths ArtifactPaths) string {
|
|
||||||
return paths.Execution
|
|
||||||
}, artifact)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SaveDistributorNotification(_ context.Context, resolved report.Resolved, artifact DistributorNotificationArtifact) (string, error) {
|
|
||||||
if artifact.SchemaVersion == "" {
|
|
||||||
artifact.SchemaVersion = DistributorNotificationSchemaVersion
|
|
||||||
}
|
|
||||||
return s.saveResolvedJSON(resolved, func(paths ArtifactPaths) string {
|
|
||||||
return paths.Notification
|
|
||||||
}, artifact)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) BatchDistributorNotificationPath(ref BatchDistributorNotificationRef) (string, error) {
|
|
||||||
if s == nil {
|
|
||||||
return "", fmt.Errorf("state store is required")
|
|
||||||
}
|
|
||||||
if err := validateBatchNotificationRef(ref); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
localDate := ref.StartedAt.In(ref.Location).Format("2006-01-02")
|
|
||||||
return s.join(s.notificationsDir, "batches", ref.Batch, localDate, "distributor."+ref.BatchRunID+".json"), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SaveBatchDistributorNotification(_ context.Context, ref BatchDistributorNotificationRef, artifact BatchDistributorNotificationArtifact) (string, error) {
|
|
||||||
path, err := s.BatchDistributorNotificationPath(ref)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
if artifact.SchemaVersion == "" {
|
|
||||||
artifact.SchemaVersion = BatchDistributorNotificationSchemaVersion
|
|
||||||
}
|
|
||||||
artifact.Batch = ref.Batch
|
|
||||||
artifact.BatchRunID = ref.BatchRunID
|
|
||||||
if err := fileutil.WriteJSONAtomic(path, artifact); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return path, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SaveGeneratedTextRaw(_ context.Context, resolved report.Resolved, data []byte) (string, error) {
|
|
||||||
return s.saveResolvedBytes(resolved, func(paths ArtifactPaths) string {
|
|
||||||
return paths.GeneratedTextRaw
|
|
||||||
}, data)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SaveGeneratedText(_ context.Context, resolved report.Resolved, data []byte) (string, error) {
|
|
||||||
return s.saveResolvedBytes(resolved, func(paths ArtifactPaths) string {
|
|
||||||
return paths.GeneratedText
|
|
||||||
}, data)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SaveRenderContext(_ context.Context, resolved report.Resolved, value any) (string, error) {
|
|
||||||
return s.saveResolvedJSON(resolved, func(paths ArtifactPaths) string {
|
|
||||||
return paths.RenderContext
|
|
||||||
}, value)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) saveResolvedJSON(resolved report.Resolved, selectPath func(ArtifactPaths) string, value any) (string, error) {
|
|
||||||
paths, err := s.Paths(resolved)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
path := selectPath(paths)
|
|
||||||
if err := fileutil.WriteJSONAtomic(path, value); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return path, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) saveResolvedBytes(resolved report.Resolved, selectPath func(ArtifactPaths) string, data []byte) (string, error) {
|
|
||||||
paths, err := s.Paths(resolved)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
path := selectPath(paths)
|
|
||||||
if err := fileutil.WriteFileAtomic(path, data); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return path, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) PrepareRenderedReport(_ context.Context, resolved report.Resolved) (string, error) {
|
|
||||||
paths, err := s.Paths(resolved)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
if err := os.MkdirAll(filepath.Dir(paths.RenderedReport), 0o755); err != nil {
|
|
||||||
return "", fmt.Errorf("create rendered report directory %q: %w", filepath.Dir(paths.RenderedReport), err)
|
|
||||||
}
|
|
||||||
return paths.RenderedReport, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) SaveMetadata(_ context.Context, metadata Metadata) (string, error) {
|
|
||||||
if metadata.SchemaVersion != MetadataSchemaVersion {
|
|
||||||
return "", fmt.Errorf("new metadata must use schema version %q", MetadataSchemaVersion)
|
|
||||||
}
|
|
||||||
if err := metadata.Validate(); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
if err := s.validateManagedPath("metadata path", metadata.MetadataPath); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
if err := fileutil.WriteJSONAtomic(metadata.MetadataPath, metadata); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return metadata.MetadataPath, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) FindPriorSnapshot(_ context.Context, resolved report.Resolved) (*PriorSnapshot, error) {
|
|
||||||
if resolved.Definition.ComparisonStrategy != report.CompareSameValidDate {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
group := resolved.Definition.ArtifactGroup
|
|
||||||
if group == "" {
|
|
||||||
return nil, fmt.Errorf("report %q has no artifact group", resolved.Definition.ID)
|
|
||||||
}
|
|
||||||
dirs, err := s.metadataDirectories(resolved, group)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
var candidates []Metadata
|
|
||||||
for _, dir := range dirs {
|
|
||||||
entries, err := os.ReadDir(dir)
|
|
||||||
if err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
return nil, fmt.Errorf("read snapshot metadata directory %q: %w", dir, err)
|
|
||||||
}
|
|
||||||
for _, entry := range entries {
|
|
||||||
if entry.IsDir() || !isMetadataFilename(entry.Name()) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
path := filepath.Join(dir, entry.Name())
|
|
||||||
var metadata Metadata
|
|
||||||
if err := readJSON(path, &metadata); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if metadata.RunID == resolved.Metadata().RunID {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if !resolved.Definition.CompatibleWithPrior(metadata.ReportID) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if !comparablePeriod(metadata, resolved) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if !metadata.GeneratedAt.Before(resolved.GeneratedAt) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
candidates = append(candidates, metadata)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if len(candidates) == 0 {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
sort.Slice(candidates, func(i, j int) bool {
|
|
||||||
return candidates[i].GeneratedAt.After(candidates[j].GeneratedAt)
|
|
||||||
})
|
|
||||||
return &PriorSnapshot{
|
|
||||||
Metadata: candidates[0],
|
|
||||||
ModuleSnapshotPath: candidates[0].ModuleSnapshotPath,
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) ListReports(_ context.Context, limit int) ([]ReportRecord, error) {
|
|
||||||
if s == nil {
|
|
||||||
return nil, fmt.Errorf("state store is required")
|
|
||||||
}
|
|
||||||
root := s.join(s.snapshotsDir)
|
|
||||||
if _, err := os.Stat(root); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
return nil, fmt.Errorf("inspect %q: %w", root, err)
|
|
||||||
}
|
|
||||||
var records []ReportRecord
|
|
||||||
err := filepath.WalkDir(root, func(path string, entry os.DirEntry, err error) error {
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("inspect %q: %w", path, err)
|
|
||||||
}
|
|
||||||
if entry.IsDir() || !isMetadataFilename(entry.Name()) {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
record, err := s.reportRecord(path)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
records = append(records, record)
|
|
||||||
return nil
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
sort.Slice(records, func(i, j int) bool {
|
|
||||||
return records[i].metadata.GeneratedAt.After(records[j].metadata.GeneratedAt)
|
|
||||||
})
|
|
||||||
if limit > 0 && len(records) > limit {
|
|
||||||
records = records[:limit]
|
|
||||||
}
|
|
||||||
return records, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) LoadMetadataByRunID(ctx context.Context, runID string) (Metadata, string, error) {
|
|
||||||
if strings.TrimSpace(runID) == "" {
|
|
||||||
return Metadata{}, "", fmt.Errorf("run id is required")
|
|
||||||
}
|
|
||||||
records, err := s.ListReports(ctx, 0)
|
|
||||||
if err != nil {
|
|
||||||
return Metadata{}, "", err
|
|
||||||
}
|
|
||||||
for _, record := range records {
|
|
||||||
if record.RunID == runID {
|
|
||||||
return record.metadata, record.MetadataPath, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return Metadata{}, "", fmt.Errorf("metadata for run id %q was not found", runID)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) LoadDataPackage(_ context.Context, path string) (promptinput.Package, error) {
|
|
||||||
if path == "" {
|
|
||||||
return promptinput.Package{}, fmt.Errorf("data package path is required")
|
|
||||||
}
|
|
||||||
data, err := os.ReadFile(path)
|
|
||||||
if err != nil {
|
|
||||||
return promptinput.Package{}, fmt.Errorf("read %q: %w", path, err)
|
|
||||||
}
|
|
||||||
pkg, err := promptinput.LoadYAML(data)
|
|
||||||
if err != nil {
|
|
||||||
return promptinput.Package{}, err
|
|
||||||
}
|
|
||||||
return pkg, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) LoadModuleSnapshot(_ context.Context, path string) (module.Snapshot, error) {
|
|
||||||
if path == "" {
|
|
||||||
return module.Snapshot{}, fmt.Errorf("module snapshot path is required")
|
|
||||||
}
|
|
||||||
var snapshot module.Snapshot
|
|
||||||
if err := readJSON(path, &snapshot); err != nil {
|
|
||||||
return module.Snapshot{}, err
|
|
||||||
}
|
|
||||||
if err := snapshot.Validate(); err != nil {
|
|
||||||
return module.Snapshot{}, err
|
|
||||||
}
|
|
||||||
return snapshot, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) LoadGeneratedText(_ context.Context, path string) ([]byte, error) {
|
|
||||||
if path == "" {
|
|
||||||
return nil, fmt.Errorf("generated text path is required")
|
|
||||||
}
|
|
||||||
data, err := os.ReadFile(path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("read %q: %w", path, err)
|
|
||||||
}
|
|
||||||
return data, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) LoadPromptPreparation(_ context.Context, path string) (PromptPreparationArtifact, error) {
|
|
||||||
if path == "" {
|
|
||||||
return PromptPreparationArtifact{}, fmt.Errorf("prompt preparation path is required")
|
|
||||||
}
|
|
||||||
var artifact PromptPreparationArtifact
|
|
||||||
if err := readJSON(path, &artifact); err != nil {
|
|
||||||
return PromptPreparationArtifact{}, err
|
|
||||||
}
|
|
||||||
if err := artifact.Validate(); err != nil {
|
|
||||||
return PromptPreparationArtifact{}, err
|
|
||||||
}
|
|
||||||
return artifact, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) LoadPromptExecution(_ context.Context, path string) (PromptExecutionArtifact, error) {
|
|
||||||
if path == "" {
|
|
||||||
return PromptExecutionArtifact{}, fmt.Errorf("prompt execution path is required")
|
|
||||||
}
|
|
||||||
var artifact PromptExecutionArtifact
|
|
||||||
if err := readJSON(path, &artifact); err != nil {
|
|
||||||
return PromptExecutionArtifact{}, err
|
|
||||||
}
|
|
||||||
if err := artifact.Validate(); err != nil {
|
|
||||||
return PromptExecutionArtifact{}, err
|
|
||||||
}
|
|
||||||
return artifact, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) LoadRenderContext(_ context.Context, path string, target any) error {
|
|
||||||
if path == "" {
|
|
||||||
return fmt.Errorf("render context path is required")
|
|
||||||
}
|
|
||||||
if target == nil {
|
|
||||||
return fmt.Errorf("render context target is required")
|
|
||||||
}
|
|
||||||
return readJSON(path, target)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) reportRecord(path string) (ReportRecord, error) {
|
|
||||||
var metadata Metadata
|
|
||||||
if err := readJSON(path, &metadata); err != nil {
|
|
||||||
return ReportRecord{}, err
|
|
||||||
}
|
|
||||||
metadata.MetadataPath = path
|
|
||||||
return ReportRecord{
|
|
||||||
RunID: metadata.RunID,
|
|
||||||
ReportID: metadata.ReportID,
|
|
||||||
Variant: metadata.Variant,
|
|
||||||
PromptID: metadata.PromptID,
|
|
||||||
GeneratedAt: metadata.GeneratedAt.Format(time.RFC3339Nano),
|
|
||||||
ValidStart: metadata.ValidPeriod.Start.Format(time.RFC3339Nano),
|
|
||||||
ValidEnd: metadata.ValidPeriod.End.Format(time.RFC3339Nano),
|
|
||||||
MetadataPath: path,
|
|
||||||
ReportPath: metadata.RenderedReportPath,
|
|
||||||
Warnings: len(metadata.SourceWarnings),
|
|
||||||
metadata: metadata,
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) metadataDirectories(resolved report.Resolved, group string) ([]string, error) {
|
|
||||||
paths, err := s.Paths(resolved)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
return []string{filepath.Dir(paths.Metadata)}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) join(parts ...string) string {
|
|
||||||
all := append([]string{s.root}, parts...)
|
|
||||||
return filepath.Join(all...)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (s *FilesystemStore) validateManagedPath(name, path string) error {
|
|
||||||
if s == nil {
|
|
||||||
return fmt.Errorf("state store is required")
|
|
||||||
}
|
|
||||||
root, err := filepath.Abs(s.root)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("resolve workspace root: %w", err)
|
|
||||||
}
|
|
||||||
target, err := filepath.Abs(path)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("resolve %s: %w", name, err)
|
|
||||||
}
|
|
||||||
relative, err := filepath.Rel(root, target)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("resolve %s relative to workspace root: %w", name, err)
|
|
||||||
}
|
|
||||||
if relative == "." || relative == ".." || strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
|
|
||||||
return fmt.Errorf("%s must stay within workspace root", name)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func validateRelativeDir(name string, value string) error {
|
|
||||||
if value == "" {
|
|
||||||
return fmt.Errorf("%s is required", name)
|
|
||||||
}
|
|
||||||
if filepath.IsAbs(value) {
|
|
||||||
return fmt.Errorf("%s must be relative to workspace root", name)
|
|
||||||
}
|
|
||||||
cleaned := filepath.Clean(value)
|
|
||||||
if cleaned == "." || cleaned == ".." || strings.HasPrefix(cleaned, ".."+string(filepath.Separator)) {
|
|
||||||
return fmt.Errorf("%s must stay within workspace root", name)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func validateBatchNotificationRef(ref BatchDistributorNotificationRef) error {
|
|
||||||
if err := validatePathSegment("batch kind", ref.Batch); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := validatePathSegment("batch run id", ref.BatchRunID); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if ref.StartedAt.IsZero() {
|
|
||||||
return fmt.Errorf("batch started time is required")
|
|
||||||
}
|
|
||||||
if ref.Location == nil {
|
|
||||||
return fmt.Errorf("batch location is required")
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func validatePathSegment(name string, value string) error {
|
|
||||||
if strings.TrimSpace(value) == "" {
|
|
||||||
return fmt.Errorf("%s is required", name)
|
|
||||||
}
|
|
||||||
if strings.ContainsAny(value, `/\`) {
|
|
||||||
return fmt.Errorf("%s must not contain path separators", name)
|
|
||||||
}
|
|
||||||
if value == "." || value == ".." {
|
|
||||||
return fmt.Errorf("%s must be a safe path segment", name)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func isMetadataFilename(name string) bool {
|
|
||||||
if !strings.HasPrefix(name, "metadata.") || !strings.HasSuffix(name, ".json") {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
runID := strings.TrimSuffix(strings.TrimPrefix(name, "metadata."), ".json")
|
|
||||||
return strings.TrimSpace(runID) != "" && !strings.ContainsAny(runID, `/\`) && runID != "." && runID != ".."
|
|
||||||
}
|
|
||||||
|
|
||||||
func readJSON(path string, target any) error {
|
|
||||||
data, err := os.ReadFile(path)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("read %q: %w", path, err)
|
|
||||||
}
|
|
||||||
if err := json.Unmarshal(data, target); err != nil {
|
|
||||||
return fmt.Errorf("decode %q: %w", path, err)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func sameValidDate(metadata Metadata, resolved report.Resolved) bool {
|
|
||||||
return metadata.ValidPeriod.Start.Format("2006-01-02") == resolved.ValidPeriod.Start.Format("2006-01-02")
|
|
||||||
}
|
|
||||||
|
|
||||||
func comparablePeriod(metadata Metadata, resolved report.Resolved) bool {
|
|
||||||
return resolved.Definition.ComparisonStrategy == report.CompareSameValidDate && sameValidDate(metadata, resolved)
|
|
||||||
}
|
|
||||||
@@ -1,396 +0,0 @@
|
|||||||
package state
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/config"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestFilesystemPathsUseExactPromptArtifactLayout(t *testing.T) {
|
|
||||||
store := newFilesystemTestStore(t)
|
|
||||||
tests := []struct {
|
|
||||||
id report.ID
|
|
||||||
now string
|
|
||||||
date string
|
|
||||||
group string
|
|
||||||
validDate string
|
|
||||||
runID string
|
|
||||||
}{
|
|
||||||
{report.Daily, "2026-05-29T05:00:00-05:00", "2026-05-29T12:00:00-05:00", "daily", "2026-05-29", "20260529T100000.000000000Z_daily_2026-05-29"},
|
|
||||||
{report.Today, "2026-05-29T05:00:00-05:00", "", "today", "2026-05-29", "20260529T100000.000000000Z_today"},
|
|
||||||
{report.Tomorrow, "2026-05-29T18:00:00-05:00", "", "tomorrow", "2026-05-30", "20260529T230000.000000000Z_tomorrow"},
|
|
||||||
{report.Hourly, "2026-05-29T05:00:00-05:00", "", "hourly", "2026-05-29", "20260529T100000.000000000Z_hourly"},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(string(test.id), func(t *testing.T) {
|
|
||||||
resolved := resolveStateReport(t, test.id, test.now, test.date)
|
|
||||||
paths, err := store.Paths(resolved)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Paths() error = %v", err)
|
|
||||||
}
|
|
||||||
want := ArtifactPaths{
|
|
||||||
ModuleSnapshot: filepath.Join(store.root, "snapshots", test.group, test.validDate, "modules."+test.runID+".json"),
|
|
||||||
Metadata: filepath.Join(store.root, "snapshots", test.group, test.validDate, "metadata."+test.runID+".json"),
|
|
||||||
DataPackage: filepath.Join(store.root, "data-packages", test.group, test.validDate, "data_package."+test.runID+".yaml"),
|
|
||||||
Preparation: filepath.Join(store.root, "preflight", test.group, test.validDate, "prompt_preparation."+test.runID+".json"),
|
|
||||||
Execution: filepath.Join(store.root, "snapshots", test.group, test.validDate, "prompt_execution."+test.runID+".json"),
|
|
||||||
Notification: filepath.Join(store.root, "notifications", test.group, test.validDate, "distributor."+test.runID+".json"),
|
|
||||||
RenderedReport: filepath.Join(store.root, "reports", test.group, test.validDate, "report."+test.runID+".md"),
|
|
||||||
GeneratedTextRaw: filepath.Join(store.root, "snapshots", test.group, test.validDate, "generated_text_raw."+test.runID+".json"),
|
|
||||||
GeneratedText: filepath.Join(store.root, "snapshots", test.group, test.validDate, "generated_text."+test.runID+".json"),
|
|
||||||
RenderContext: filepath.Join(store.root, "snapshots", test.group, test.validDate, "render_context."+test.runID+".json"),
|
|
||||||
}
|
|
||||||
if paths != want {
|
|
||||||
t.Fatalf("Paths() = %#v, want %#v", paths, want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestPromptArtifactsAndMetadataRoundTrip(t *testing.T) {
|
|
||||||
store := newFilesystemTestStore(t)
|
|
||||||
resolved := resolveStateReport(t, report.Daily, "2026-05-29T05:00:00-05:00", "2026-05-29T12:00:00-05:00")
|
|
||||||
paths, err := store.Paths(resolved)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Paths() error = %v", err)
|
|
||||||
}
|
|
||||||
preparation := preparationArtifactFor(resolved, paths)
|
|
||||||
preparationPath, err := store.SavePromptPreparation(context.Background(), resolved, preparation)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("SavePromptPreparation() error = %v", err)
|
|
||||||
}
|
|
||||||
execution := executionArtifactFor(resolved, paths)
|
|
||||||
executionPath, err := store.SavePromptExecution(context.Background(), resolved, execution)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("SavePromptExecution() error = %v", err)
|
|
||||||
}
|
|
||||||
loadedPreparation, err := store.LoadPromptPreparation(context.Background(), preparationPath)
|
|
||||||
if err != nil || loadedPreparation.Preparation == nil || loadedPreparation.Preparation.PromptHash != "prompt-hash" {
|
|
||||||
t.Fatalf("LoadPromptPreparation() = %#v, %v", loadedPreparation, err)
|
|
||||||
}
|
|
||||||
loadedExecution, err := store.LoadPromptExecution(context.Background(), executionPath)
|
|
||||||
if err != nil || loadedExecution.Provenance == nil || loadedExecution.Provenance.RunID != "provider-run" || loadedExecution.Validation == nil {
|
|
||||||
t.Fatalf("LoadPromptExecution() = %#v, %v", loadedExecution, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
metadata := promptMetadataFor(resolved, paths)
|
|
||||||
metadata.PreparationPath = preparationPath
|
|
||||||
metadata.ExecutionPath = executionPath
|
|
||||||
metadata.GeneratedTextRawPath = paths.GeneratedTextRaw
|
|
||||||
metadataPath, err := store.SaveMetadata(context.Background(), metadata)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("SaveMetadata() error = %v", err)
|
|
||||||
}
|
|
||||||
data, err := os.ReadFile(metadataPath)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read metadata: %v", err)
|
|
||||||
}
|
|
||||||
text := string(data)
|
|
||||||
if strings.Contains(text, "preflightPath") || strings.Contains(text, "generatedTextResultPath") || strings.Contains(text, "metadataPath") {
|
|
||||||
t.Fatalf("v2 metadata contains legacy or runtime aliases: %s", text)
|
|
||||||
}
|
|
||||||
loadedMetadata, loadedPath, err := store.LoadMetadataByRunID(context.Background(), metadata.RunID)
|
|
||||||
if err != nil || loadedPath != metadataPath || loadedMetadata.PreparationPath != preparationPath || loadedMetadata.ExecutionPath != executionPath {
|
|
||||||
t.Fatalf("LoadMetadataByRunID() = %#v, %q, %v", loadedMetadata, loadedPath, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestMetadataLegacyCompatibilityAndV2OnlyWrites(t *testing.T) {
|
|
||||||
legacy := Metadata{
|
|
||||||
SchemaVersion: MetadataSchemaVersionV1, RunID: "legacy-run", ReportID: report.ID("three_day"),
|
|
||||||
PromptID: "weather.three_day", ModuleSnapshotPath: "/archive/modules.json", DataPackagePath: "/archive/data.yaml",
|
|
||||||
PreflightPath: "/archive/render.json", GeneratedTextResultPath: "/archive/result.json",
|
|
||||||
}
|
|
||||||
data, err := json.Marshal(legacy)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Marshal() error = %v", err)
|
|
||||||
}
|
|
||||||
text := string(data)
|
|
||||||
if !strings.Contains(text, "preflightPath") || !strings.Contains(text, "generatedTextResultPath") || strings.Contains(text, "preparationPath") || strings.Contains(text, "executionPath") {
|
|
||||||
t.Fatalf("v1 metadata wire fields = %s", text)
|
|
||||||
}
|
|
||||||
var decoded Metadata
|
|
||||||
if err := json.Unmarshal(data, &decoded); err != nil {
|
|
||||||
t.Fatalf("Unmarshal() error = %v", err)
|
|
||||||
}
|
|
||||||
if decoded.PreparationPath != legacy.PreflightPath || decoded.ExecutionPath != legacy.GeneratedTextResultPath {
|
|
||||||
t.Fatalf("normalized compatibility aliases = %#v", decoded)
|
|
||||||
}
|
|
||||||
remarshaled, err := json.Marshal(decoded)
|
|
||||||
if err != nil || !strings.Contains(string(remarshaled), "preflightPath") || strings.Contains(string(remarshaled), "preparationPath") {
|
|
||||||
t.Fatalf("remarshaled v1 metadata = %s, %v", remarshaled, err)
|
|
||||||
}
|
|
||||||
if err := json.Unmarshal([]byte(`{"schemaVersion":"weatherreporter.metadata.v99"}`), &decoded); err == nil {
|
|
||||||
t.Fatal("Unmarshal() error = nil, want unknown schema rejection")
|
|
||||||
}
|
|
||||||
|
|
||||||
store := newFilesystemTestStore(t)
|
|
||||||
legacy.MetadataPath = filepath.Join(store.root, "snapshots", "legacy", "metadata.legacy-run.json")
|
|
||||||
if _, err := store.SaveMetadata(context.Background(), legacy); err == nil {
|
|
||||||
t.Fatal("SaveMetadata(v1) error = nil, want v2-only write rejection")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestReportDiscoveryAndArtifactInspection(t *testing.T) {
|
|
||||||
store := newFilesystemTestStore(t)
|
|
||||||
older := resolveStateReport(t, report.Daily, "2026-05-29T05:00:00-05:00", "2026-05-29T12:00:00-05:00")
|
|
||||||
newer := resolveStateReport(t, report.Today, "2026-05-29T08:00:00-05:00", "")
|
|
||||||
olderPaths := saveStateMetadata(t, store, older)
|
|
||||||
newerPaths := saveStateMetadata(t, store, newer)
|
|
||||||
|
|
||||||
snapshot, err := module.NewSnapshot([]module.Output{{ID: module.Metadata, StanzaName: "metadata", Value: map[string]any{"run_id": older.Metadata().RunID}}})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("NewSnapshot() error = %v", err)
|
|
||||||
}
|
|
||||||
modulePath, err := store.SaveModuleSnapshot(context.Background(), older, snapshot)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("SaveModuleSnapshot() error = %v", err)
|
|
||||||
}
|
|
||||||
pkg, err := promptinput.Build(promptinput.BuildRequest{
|
|
||||||
Metadata: promptinput.Metadata{
|
|
||||||
RunID: older.Metadata().RunID, ReportID: older.Definition.ID, PromptID: older.Definition.PromptID,
|
|
||||||
GeneratedAt: older.GeneratedAt, Timezone: older.Timezone, ValidPeriod: older.ValidPeriod,
|
|
||||||
},
|
|
||||||
Modules: snapshot,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Build() error = %v", err)
|
|
||||||
}
|
|
||||||
dataPath, err := store.SaveDataPackage(context.Background(), older, pkg)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("SaveDataPackage() error = %v", err)
|
|
||||||
}
|
|
||||||
loadedSnapshot, err := store.LoadModuleSnapshot(context.Background(), modulePath)
|
|
||||||
if err != nil || len(loadedSnapshot.Outputs) != 1 {
|
|
||||||
t.Fatalf("LoadModuleSnapshot() = %#v, %v", loadedSnapshot, err)
|
|
||||||
}
|
|
||||||
loadedPackage, err := store.LoadDataPackage(context.Background(), dataPath)
|
|
||||||
if err != nil || loadedPackage.RunID != older.Metadata().RunID {
|
|
||||||
t.Fatalf("LoadDataPackage() = %#v, %v", loadedPackage, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
records, err := store.ListReports(context.Background(), 0)
|
|
||||||
if err != nil || len(records) != 2 {
|
|
||||||
t.Fatalf("ListReports() = %#v, %v", records, err)
|
|
||||||
}
|
|
||||||
if records[0].RunID != newer.Metadata().RunID || records[0].MetadataPath != newerPaths.Metadata || records[1].MetadataPath != olderPaths.Metadata {
|
|
||||||
t.Fatalf("ordered report records = %#v", records)
|
|
||||||
}
|
|
||||||
loadedMetadata, loadedPath, err := store.LoadMetadataByRunID(context.Background(), older.Metadata().RunID)
|
|
||||||
if err != nil || loadedPath != olderPaths.Metadata || len(loadedMetadata.Sources) != 1 || loadedMetadata.Sources[0].Name != "weather-api" {
|
|
||||||
t.Fatalf("LoadMetadataByRunID() = %#v, %q, %v", loadedMetadata, loadedPath, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestListReportsRetainsHistoricalV1ReportIDs(t *testing.T) {
|
|
||||||
store := newFilesystemTestStore(t)
|
|
||||||
for i, id := range []report.ID{"three_day", "weekend", "storm"} {
|
|
||||||
generatedAt := time.Date(2026, 5, 20+i, 12, 0, 0, 0, time.UTC)
|
|
||||||
metadata := Metadata{
|
|
||||||
SchemaVersion: MetadataSchemaVersionV1, RunID: "historical-" + string(id), ReportID: id,
|
|
||||||
PromptID: "weather." + string(id), GeneratedAt: generatedAt, Timezone: "UTC",
|
|
||||||
ValidPeriod: timeutil.Period{Start: generatedAt, End: generatedAt.Add(24 * time.Hour)},
|
|
||||||
ModuleSnapshotPath: "/archive/modules.json", DataPackagePath: "/archive/data.yaml",
|
|
||||||
PreflightPath: "/archive/render.json", GeneratedTextResultPath: "/archive/result.json",
|
|
||||||
}
|
|
||||||
path := filepath.Join(store.root, "snapshots", string(id), "2026-05-20", "metadata."+metadata.RunID+".json")
|
|
||||||
writeJSONFixture(t, path, metadata)
|
|
||||||
}
|
|
||||||
records, err := store.ListReports(context.Background(), 0)
|
|
||||||
if err != nil || len(records) != 3 {
|
|
||||||
t.Fatalf("ListReports() = %#v, %v", records, err)
|
|
||||||
}
|
|
||||||
for _, record := range records {
|
|
||||||
if record.ReportID != "three_day" && record.ReportID != "weekend" && record.ReportID != "storm" {
|
|
||||||
t.Fatalf("unexpected historical report record: %#v", record)
|
|
||||||
}
|
|
||||||
metadata, path, err := store.LoadMetadataByRunID(context.Background(), record.RunID)
|
|
||||||
if err != nil || path != record.MetadataPath || metadata.PreparationPath != "/archive/render.json" || metadata.ExecutionPath != "/archive/result.json" {
|
|
||||||
t.Fatalf("LoadMetadataByRunID(%q) = %#v, %q, %v", record.RunID, metadata, path, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestFilesystemWritesAreAtomicAndRejectUnsafePaths(t *testing.T) {
|
|
||||||
store := newFilesystemTestStore(t)
|
|
||||||
resolved := resolveStateReport(t, report.Hourly, "2026-05-29T05:00:00-05:00", "")
|
|
||||||
path, err := store.SaveGeneratedTextRaw(context.Background(), resolved, []byte(`{"value":"first"}`))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("first SaveGeneratedTextRaw() error = %v", err)
|
|
||||||
}
|
|
||||||
if _, err := store.SaveGeneratedTextRaw(context.Background(), resolved, []byte(`{"value":"second"}`)); err != nil {
|
|
||||||
t.Fatalf("second SaveGeneratedTextRaw() error = %v", err)
|
|
||||||
}
|
|
||||||
data, err := os.ReadFile(path)
|
|
||||||
if err != nil || string(data) != `{"value":"second"}` {
|
|
||||||
t.Fatalf("atomic replacement = %q, %v", data, err)
|
|
||||||
}
|
|
||||||
entries, err := os.ReadDir(filepath.Dir(path))
|
|
||||||
if err != nil || len(entries) != 1 || entries[0].Name() != filepath.Base(path) {
|
|
||||||
t.Fatalf("artifact directory after atomic write = %#v, %v", entries, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
metadata := promptMetadataFor(resolved, mustStatePaths(t, store, resolved))
|
|
||||||
metadata.PreparationPath = "/saved/preparation.json"
|
|
||||||
metadata.MetadataPath = filepath.Join(t.TempDir(), "outside.json")
|
|
||||||
if _, err := store.SaveMetadata(context.Background(), metadata); err == nil {
|
|
||||||
t.Fatal("SaveMetadata(outside workspace) error = nil")
|
|
||||||
}
|
|
||||||
|
|
||||||
cfg := config.Defaults().Workspace
|
|
||||||
cfg.Root = t.TempDir()
|
|
||||||
cfg.SnapshotsDir = "../snapshots"
|
|
||||||
if _, err := NewFilesystemStore(cfg); err == nil {
|
|
||||||
t.Fatal("NewFilesystemStore(unsafe directory) error = nil")
|
|
||||||
}
|
|
||||||
unsafeResolved := resolved
|
|
||||||
unsafeResolved.Definition.ID = report.ID("hourly/bad")
|
|
||||||
if _, err := store.Paths(unsafeResolved); err == nil {
|
|
||||||
t.Fatal("Paths(unsafe run id) error = nil")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestFindPriorSnapshotForSupportedReports(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
id report.ID
|
|
||||||
firstNow string
|
|
||||||
secondNow string
|
|
||||||
date string
|
|
||||||
wantPrior bool
|
|
||||||
}{
|
|
||||||
{report.Daily, "2026-05-29T05:00:00-05:00", "2026-05-29T08:00:00-05:00", "2026-05-29T12:00:00-05:00", true},
|
|
||||||
{report.Today, "2026-05-29T05:00:00-05:00", "2026-05-29T08:00:00-05:00", "", true},
|
|
||||||
{report.Tomorrow, "2026-05-29T17:00:00-05:00", "2026-05-29T18:00:00-05:00", "", true},
|
|
||||||
{report.Hourly, "2026-05-29T05:00:00-05:00", "2026-05-29T06:00:00-05:00", "", false},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(string(test.id), func(t *testing.T) {
|
|
||||||
store := newFilesystemTestStore(t)
|
|
||||||
first := resolveStateReport(t, test.id, test.firstNow, test.date)
|
|
||||||
second := resolveStateReport(t, test.id, test.secondNow, test.date)
|
|
||||||
paths := saveStateMetadata(t, store, first)
|
|
||||||
prior, err := store.FindPriorSnapshot(context.Background(), second)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("FindPriorSnapshot() error = %v", err)
|
|
||||||
}
|
|
||||||
if !test.wantPrior && prior != nil {
|
|
||||||
t.Fatalf("FindPriorSnapshot() = %#v, want nil", prior)
|
|
||||||
}
|
|
||||||
if test.wantPrior && (prior == nil || prior.Metadata.RunID != first.Metadata().RunID || prior.ModuleSnapshotPath != paths.ModuleSnapshot) {
|
|
||||||
t.Fatalf("FindPriorSnapshot() = %#v, want run %q", prior, first.Metadata().RunID)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func newFilesystemTestStore(t *testing.T) *FilesystemStore {
|
|
||||||
t.Helper()
|
|
||||||
cfg := config.Defaults().Workspace
|
|
||||||
cfg.Root = t.TempDir()
|
|
||||||
store, err := NewFilesystemStore(cfg)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
|
||||||
}
|
|
||||||
return store
|
|
||||||
}
|
|
||||||
|
|
||||||
func resolveStateReport(t *testing.T, id report.ID, nowValue, dateValue string) report.Resolved {
|
|
||||||
t.Helper()
|
|
||||||
location, err := time.LoadLocation("America/Chicago")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("LoadLocation() error = %v", err)
|
|
||||||
}
|
|
||||||
now, err := time.Parse(time.RFC3339, nowValue)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("parse now: %v", err)
|
|
||||||
}
|
|
||||||
req := report.ResolveRequest{Now: now, Location: location}
|
|
||||||
if dateValue != "" {
|
|
||||||
req.Date, err = time.Parse(time.RFC3339, dateValue)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("parse date: %v", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
resolved, err := report.DefaultRegistry().Resolve(id, req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Resolve() error = %v", err)
|
|
||||||
}
|
|
||||||
return resolved
|
|
||||||
}
|
|
||||||
|
|
||||||
func mustStatePaths(t *testing.T, store *FilesystemStore, resolved report.Resolved) ArtifactPaths {
|
|
||||||
t.Helper()
|
|
||||||
paths, err := store.Paths(resolved)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Paths() error = %v", err)
|
|
||||||
}
|
|
||||||
return paths
|
|
||||||
}
|
|
||||||
|
|
||||||
func promptMetadataFor(resolved report.Resolved, paths ArtifactPaths) Metadata {
|
|
||||||
metadata := BuildPromptMetadataFromBriefingMetadata(resolved, briefing.Metadata{
|
|
||||||
Sources: []briefing.SourceMetadata{{Name: "weather-api", FetchedAt: resolved.GeneratedAt}},
|
|
||||||
SourceWarnings: []weatherdata.SourceWarning{},
|
|
||||||
}, ArtifactPaths{ModuleSnapshot: paths.ModuleSnapshot, Metadata: paths.Metadata, DataPackage: paths.DataPackage})
|
|
||||||
return metadata
|
|
||||||
}
|
|
||||||
|
|
||||||
func saveStateMetadata(t *testing.T, store *FilesystemStore, resolved report.Resolved) ArtifactPaths {
|
|
||||||
t.Helper()
|
|
||||||
paths := mustStatePaths(t, store, resolved)
|
|
||||||
metadata := promptMetadataFor(resolved, paths)
|
|
||||||
metadata.PreparationPath = paths.Preparation
|
|
||||||
if _, err := store.SaveMetadata(context.Background(), metadata); err != nil {
|
|
||||||
t.Fatalf("SaveMetadata() error = %v", err)
|
|
||||||
}
|
|
||||||
return paths
|
|
||||||
}
|
|
||||||
|
|
||||||
func preparationArtifactFor(resolved report.Resolved, paths ArtifactPaths) PromptPreparationArtifact {
|
|
||||||
artifact := validPreparationArtifact()
|
|
||||||
metadata := resolved.Metadata()
|
|
||||||
artifact.ReportID, artifact.RunID = metadata.ReportID, metadata.RunID
|
|
||||||
artifact.PromptID, artifact.PromptVersion = resolved.Definition.PromptID, resolved.Definition.PromptVersion
|
|
||||||
artifact.DataPackagePath = paths.DataPackage
|
|
||||||
artifact.Preparation.PromptID, artifact.Preparation.PromptVersion = artifact.PromptID, artifact.PromptVersion
|
|
||||||
artifact.Preparation.PromptHash = "prompt-hash"
|
|
||||||
artifact.Preparation.DataPackagePath = paths.DataPackage
|
|
||||||
return artifact
|
|
||||||
}
|
|
||||||
|
|
||||||
func executionArtifactFor(resolved report.Resolved, paths ArtifactPaths) PromptExecutionArtifact {
|
|
||||||
artifact := validExecutionArtifact()
|
|
||||||
metadata := resolved.Metadata()
|
|
||||||
artifact.ReportID, artifact.RunID = metadata.ReportID, metadata.RunID
|
|
||||||
artifact.PromptID, artifact.PromptVersion = resolved.Definition.PromptID, resolved.Definition.PromptVersion
|
|
||||||
artifact.Provenance.PromptID, artifact.Provenance.PromptVersion = artifact.PromptID, artifact.PromptVersion
|
|
||||||
artifact.Provenance.DataPackagePath = paths.DataPackage
|
|
||||||
artifact.Paths.RawOutputPath = paths.GeneratedTextRaw
|
|
||||||
return artifact
|
|
||||||
}
|
|
||||||
|
|
||||||
func writeJSONFixture(t *testing.T, path string, value any) {
|
|
||||||
t.Helper()
|
|
||||||
data, err := json.Marshal(value)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("marshal fixture: %v", err)
|
|
||||||
}
|
|
||||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
|
||||||
t.Fatalf("create fixture directory: %v", err)
|
|
||||||
}
|
|
||||||
if err := os.WriteFile(path, data, 0o600); err != nil {
|
|
||||||
t.Fatalf("write fixture: %v", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,196 +0,0 @@
|
|||||||
package state
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/timeutil"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/weatherdata"
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
|
||||||
MetadataSchemaVersionV1 = "weatherreporter.metadata.v1"
|
|
||||||
MetadataSchemaVersion = "weatherreporter.metadata.v2"
|
|
||||||
)
|
|
||||||
|
|
||||||
// Metadata is the durable record used for report discovery and inspection.
|
|
||||||
// V1 fields remain internal compatibility values and are emitted only for V1
|
|
||||||
// records; V2 records have no legacy aliases in their JSON representation.
|
|
||||||
type Metadata struct {
|
|
||||||
SchemaVersion string `json:"schemaVersion"`
|
|
||||||
RunID string `json:"runId"`
|
|
||||||
MetadataPath string `json:"-"`
|
|
||||||
ReportID report.ID `json:"reportId"`
|
|
||||||
Variant string `json:"variant,omitempty"`
|
|
||||||
PromptID string `json:"promptId"`
|
|
||||||
GeneratedAt time.Time `json:"generatedAt"`
|
|
||||||
Timezone string `json:"timezone"`
|
|
||||||
ValidPeriod timeutil.Period `json:"validPeriod"`
|
|
||||||
Location *briefing.LocationContext `json:"location,omitempty"`
|
|
||||||
SourceLocationID string `json:"sourceLocationId,omitempty"`
|
|
||||||
SourceLocation string `json:"sourceLocation,omitempty"`
|
|
||||||
Sources []briefing.SourceMetadata `json:"sources,omitempty"`
|
|
||||||
SourceWarnings []weatherdata.SourceWarning `json:"sourceWarnings,omitempty"`
|
|
||||||
ModuleSnapshotPath string `json:"moduleSnapshotPath"`
|
|
||||||
DataPackagePath string `json:"dataPackagePath"`
|
|
||||||
PreparationPath string `json:"preparationPath,omitempty"`
|
|
||||||
ExecutionPath string `json:"executionPath,omitempty"`
|
|
||||||
NotificationPath string `json:"notificationPath,omitempty"`
|
|
||||||
RenderedReportPath string `json:"renderedReportPath,omitempty"`
|
|
||||||
GeneratedTextSchemaID string `json:"generatedTextSchemaId,omitempty"`
|
|
||||||
GeneratedTextRawPath string `json:"generatedTextRawPath,omitempty"`
|
|
||||||
GeneratedTextPath string `json:"generatedTextPath,omitempty"`
|
|
||||||
RenderContextPath string `json:"renderContextPath,omitempty"`
|
|
||||||
|
|
||||||
// These paths are retained only to read legacy V1 records. They are never
|
|
||||||
// emitted in V2 metadata.
|
|
||||||
PreflightPath string `json:"-"`
|
|
||||||
GeneratedTextResultPath string `json:"-"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type metadataJSON struct {
|
|
||||||
SchemaVersion string `json:"schemaVersion"`
|
|
||||||
RunID string `json:"runId"`
|
|
||||||
ReportID report.ID `json:"reportId"`
|
|
||||||
Variant string `json:"variant,omitempty"`
|
|
||||||
PromptID string `json:"promptId"`
|
|
||||||
GeneratedAt time.Time `json:"generatedAt"`
|
|
||||||
Timezone string `json:"timezone"`
|
|
||||||
ValidPeriod timeutil.Period `json:"validPeriod"`
|
|
||||||
Location *briefing.LocationContext `json:"location,omitempty"`
|
|
||||||
SourceLocationID string `json:"sourceLocationId,omitempty"`
|
|
||||||
SourceLocation string `json:"sourceLocation,omitempty"`
|
|
||||||
Sources []briefing.SourceMetadata `json:"sources,omitempty"`
|
|
||||||
SourceWarnings []weatherdata.SourceWarning `json:"sourceWarnings,omitempty"`
|
|
||||||
ModuleSnapshotPath string `json:"moduleSnapshotPath"`
|
|
||||||
DataPackagePath string `json:"dataPackagePath"`
|
|
||||||
PreparationPath string `json:"preparationPath,omitempty"`
|
|
||||||
ExecutionPath string `json:"executionPath,omitempty"`
|
|
||||||
PreflightPath string `json:"preflightPath,omitempty"`
|
|
||||||
NotificationPath string `json:"notificationPath,omitempty"`
|
|
||||||
RenderedReportPath string `json:"renderedReportPath,omitempty"`
|
|
||||||
GeneratedTextSchemaID string `json:"generatedTextSchemaId,omitempty"`
|
|
||||||
GeneratedTextRawPath string `json:"generatedTextRawPath,omitempty"`
|
|
||||||
GeneratedTextResultPath string `json:"generatedTextResultPath,omitempty"`
|
|
||||||
GeneratedTextPath string `json:"generatedTextPath,omitempty"`
|
|
||||||
RenderContextPath string `json:"renderContextPath,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func (m Metadata) MarshalJSON() ([]byte, error) {
|
|
||||||
w := metadataJSON{
|
|
||||||
SchemaVersion: m.SchemaVersion, RunID: m.RunID, ReportID: m.ReportID,
|
|
||||||
Variant: m.Variant, PromptID: m.PromptID, GeneratedAt: m.GeneratedAt,
|
|
||||||
Timezone: m.Timezone, ValidPeriod: m.ValidPeriod, Location: m.Location,
|
|
||||||
SourceLocationID: m.SourceLocationID, SourceLocation: m.SourceLocation,
|
|
||||||
Sources: m.Sources, SourceWarnings: m.SourceWarnings,
|
|
||||||
ModuleSnapshotPath: m.ModuleSnapshotPath, DataPackagePath: m.DataPackagePath,
|
|
||||||
NotificationPath: m.NotificationPath, RenderedReportPath: m.RenderedReportPath,
|
|
||||||
GeneratedTextSchemaID: m.GeneratedTextSchemaID, GeneratedTextRawPath: m.GeneratedTextRawPath,
|
|
||||||
GeneratedTextPath: m.GeneratedTextPath, RenderContextPath: m.RenderContextPath,
|
|
||||||
}
|
|
||||||
switch m.SchemaVersion {
|
|
||||||
case MetadataSchemaVersionV1:
|
|
||||||
w.PreflightPath = m.PreflightPath
|
|
||||||
w.GeneratedTextResultPath = m.GeneratedTextResultPath
|
|
||||||
case MetadataSchemaVersion:
|
|
||||||
w.PreparationPath = m.PreparationPath
|
|
||||||
w.ExecutionPath = m.ExecutionPath
|
|
||||||
default:
|
|
||||||
return nil, fmt.Errorf("unsupported metadata schema version %q", m.SchemaVersion)
|
|
||||||
}
|
|
||||||
return json.Marshal(w)
|
|
||||||
}
|
|
||||||
|
|
||||||
func (m *Metadata) UnmarshalJSON(data []byte) error {
|
|
||||||
var header struct {
|
|
||||||
SchemaVersion string `json:"schemaVersion"`
|
|
||||||
}
|
|
||||||
if err := json.Unmarshal(data, &header); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if header.SchemaVersion != MetadataSchemaVersionV1 && header.SchemaVersion != MetadataSchemaVersion {
|
|
||||||
return fmt.Errorf("unsupported metadata schema version %q", header.SchemaVersion)
|
|
||||||
}
|
|
||||||
var w metadataJSON
|
|
||||||
if err := json.Unmarshal(data, &w); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
*m = Metadata{
|
|
||||||
SchemaVersion: w.SchemaVersion, RunID: w.RunID, ReportID: w.ReportID,
|
|
||||||
Variant: w.Variant, PromptID: w.PromptID, GeneratedAt: w.GeneratedAt,
|
|
||||||
Timezone: w.Timezone, ValidPeriod: w.ValidPeriod, Location: w.Location,
|
|
||||||
SourceLocationID: w.SourceLocationID, SourceLocation: w.SourceLocation,
|
|
||||||
Sources: w.Sources, SourceWarnings: w.SourceWarnings,
|
|
||||||
ModuleSnapshotPath: w.ModuleSnapshotPath, DataPackagePath: w.DataPackagePath,
|
|
||||||
NotificationPath: w.NotificationPath, RenderedReportPath: w.RenderedReportPath,
|
|
||||||
GeneratedTextSchemaID: w.GeneratedTextSchemaID, GeneratedTextRawPath: w.GeneratedTextRawPath,
|
|
||||||
GeneratedTextPath: w.GeneratedTextPath, RenderContextPath: w.RenderContextPath,
|
|
||||||
}
|
|
||||||
if w.SchemaVersion == MetadataSchemaVersionV1 {
|
|
||||||
m.PreflightPath = w.PreflightPath
|
|
||||||
m.GeneratedTextResultPath = w.GeneratedTextResultPath
|
|
||||||
m.PreparationPath = w.PreflightPath
|
|
||||||
m.ExecutionPath = w.GeneratedTextResultPath
|
|
||||||
} else {
|
|
||||||
m.PreparationPath = w.PreparationPath
|
|
||||||
m.ExecutionPath = w.ExecutionPath
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (m Metadata) Validate() error {
|
|
||||||
if strings.TrimSpace(m.RunID) == "" {
|
|
||||||
return fmt.Errorf("metadata run id is required")
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(m.ModuleSnapshotPath) == "" {
|
|
||||||
return fmt.Errorf("metadata module snapshot path is required")
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(m.DataPackagePath) == "" {
|
|
||||||
return fmt.Errorf("metadata data package path is required")
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(m.MetadataPath) == "" {
|
|
||||||
return fmt.Errorf("metadata path is required")
|
|
||||||
}
|
|
||||||
switch m.SchemaVersion {
|
|
||||||
case MetadataSchemaVersionV1:
|
|
||||||
if strings.TrimSpace(m.PreflightPath) == "" {
|
|
||||||
return fmt.Errorf("metadata preflight path is required")
|
|
||||||
}
|
|
||||||
case MetadataSchemaVersion:
|
|
||||||
if strings.TrimSpace(m.PreparationPath) == "" {
|
|
||||||
return fmt.Errorf("metadata preparation path is required")
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
return fmt.Errorf("unsupported metadata schema version %q", m.SchemaVersion)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// BuildPromptMetadataFromBriefingMetadata creates the V2 record used by the
|
|
||||||
// prompt execution workflow. Callers populate preparation and execution paths
|
|
||||||
// only after their corresponding artifacts have been saved.
|
|
||||||
func BuildPromptMetadataFromBriefingMetadata(resolved report.Resolved, briefingMetadata briefing.Metadata, paths ArtifactPaths) Metadata {
|
|
||||||
metadata := resolved.Metadata()
|
|
||||||
return Metadata{
|
|
||||||
SchemaVersion: MetadataSchemaVersion, RunID: metadata.RunID, MetadataPath: paths.Metadata,
|
|
||||||
ReportID: metadata.ReportID, Variant: briefingMetadata.Variant, PromptID: metadata.PromptID,
|
|
||||||
GeneratedAt: metadata.GeneratedAt, Timezone: metadata.Timezone, ValidPeriod: metadata.ValidPeriod,
|
|
||||||
Location: copyLocation(briefingMetadata.Location), SourceLocationID: briefingMetadata.SourceLocationID,
|
|
||||||
SourceLocation: briefingMetadata.SourceLocation, Sources: briefingMetadata.Sources,
|
|
||||||
SourceWarnings: briefingMetadata.SourceWarnings, ModuleSnapshotPath: paths.ModuleSnapshot,
|
|
||||||
DataPackagePath: paths.DataPackage,
|
|
||||||
GeneratedTextSchemaID: resolved.Definition.GeneratedTextSchemaID,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func copyLocation(location *briefing.LocationContext) *briefing.LocationContext {
|
|
||||||
if location == nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
copied := *location
|
|
||||||
return &copied
|
|
||||||
}
|
|
||||||
@@ -1,38 +0,0 @@
|
|||||||
package state
|
|
||||||
|
|
||||||
import (
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/briefing"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestBuildPromptMetadataIncludesOnlyExistingArtifacts(t *testing.T) {
|
|
||||||
resolved, err := report.DefaultRegistry().Resolve(report.Daily, report.ResolveRequest{
|
|
||||||
Now: time.Date(2026, 5, 29, 12, 0, 0, 0, time.UTC),
|
|
||||||
Date: time.Date(2026, 5, 29, 0, 0, 0, 0, time.UTC), Location: time.UTC,
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Resolve() error = %v", err)
|
|
||||||
}
|
|
||||||
metadata := BuildPromptMetadataFromBriefingMetadata(resolved, briefing.Metadata{}, ArtifactPaths{
|
|
||||||
ModuleSnapshot: "/saved/modules.json",
|
|
||||||
Metadata: "/destination/metadata.json",
|
|
||||||
DataPackage: "/saved/data.yaml",
|
|
||||||
Preparation: "/future/preparation.json",
|
|
||||||
Execution: "/future/execution.json",
|
|
||||||
Notification: "/future/notification.json",
|
|
||||||
RenderedReport: "/future/report.md",
|
|
||||||
GeneratedTextRaw: "/future/raw.json",
|
|
||||||
GeneratedText: "/future/generated.json",
|
|
||||||
RenderContext: "/future/context.json",
|
|
||||||
})
|
|
||||||
|
|
||||||
if metadata.ModuleSnapshotPath != "/saved/modules.json" || metadata.DataPackagePath != "/saved/data.yaml" || metadata.MetadataPath != "/destination/metadata.json" {
|
|
||||||
t.Fatalf("existing paths = %#v, want module, data package, and metadata destination", metadata)
|
|
||||||
}
|
|
||||||
if metadata.PreparationPath != "" || metadata.ExecutionPath != "" || metadata.NotificationPath != "" || metadata.RenderedReportPath != "" || metadata.GeneratedTextRawPath != "" || metadata.GeneratedTextPath != "" || metadata.RenderContextPath != "" {
|
|
||||||
t.Fatalf("metadata includes paths for unreached artifacts: %#v", metadata)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,281 +0,0 @@
|
|||||||
package state
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
"unicode/utf8"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
|
||||||
PromptPreparationSchemaVersion = "weatherreporter.prompt_preparation.v1"
|
|
||||||
PromptExecutionSchemaVersion = "weatherreporter.prompt_execution.v1"
|
|
||||||
promptArtifactErrorLimit = 2048
|
|
||||||
)
|
|
||||||
|
|
||||||
type PromptPreparationStatus string
|
|
||||||
|
|
||||||
const (
|
|
||||||
PromptPreparationSucceeded PromptPreparationStatus = "succeeded"
|
|
||||||
PromptPreparationFailed PromptPreparationStatus = "failed"
|
|
||||||
)
|
|
||||||
|
|
||||||
// PromptArtifactError is the bounded, classified failure detail retained with a
|
|
||||||
// prompt execution artifact. It deliberately excludes provider error bodies.
|
|
||||||
type PromptArtifactError struct {
|
|
||||||
Category promptexec.ErrorCategory `json:"category"`
|
|
||||||
Message string `json:"message"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// NewPromptArtifactError converts an execution error into bounded, durable
|
|
||||||
// diagnostic information without retaining its underlying cause.
|
|
||||||
func NewPromptArtifactError(err error) *PromptArtifactError {
|
|
||||||
if err == nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
message := strings.ToValidUTF8(err.Error(), "<22>")
|
|
||||||
if len(message) > promptArtifactErrorLimit {
|
|
||||||
message = message[:promptArtifactErrorLimit]
|
|
||||||
for !utf8.ValidString(message) {
|
|
||||||
message = message[:len(message)-1]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return &PromptArtifactError{Category: promptexec.CategoryOf(err), Message: message}
|
|
||||||
}
|
|
||||||
|
|
||||||
// PromptPreparationArtifact records the safe provenance available before a
|
|
||||||
// provider is invoked. Preparation debug payloads are never stored here.
|
|
||||||
type PromptPreparationArtifact struct {
|
|
||||||
SchemaVersion string `json:"schemaVersion"`
|
|
||||||
Status PromptPreparationStatus `json:"status"`
|
|
||||||
ReportID report.ID `json:"reportId"`
|
|
||||||
RunID string `json:"runId"`
|
|
||||||
PromptID string `json:"promptId"`
|
|
||||||
PromptVersion string `json:"promptVersion,omitempty"`
|
|
||||||
DataPackagePath string `json:"dataPackagePath"`
|
|
||||||
Preparation *promptexec.Preparation `json:"preparation,omitempty"`
|
|
||||||
StartedAt time.Time `json:"startedAt"`
|
|
||||||
EndedAt time.Time `json:"endedAt"`
|
|
||||||
Duration time.Duration `json:"duration"`
|
|
||||||
Error *PromptArtifactError `json:"error,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func (a PromptPreparationArtifact) Validate() error {
|
|
||||||
if a.SchemaVersion != PromptPreparationSchemaVersion {
|
|
||||||
return fmt.Errorf("unsupported prompt preparation schema version %q", a.SchemaVersion)
|
|
||||||
}
|
|
||||||
if err := validatePromptArtifactIdentity("prompt preparation", a.ReportID, a.RunID, a.PromptID, a.PromptVersion); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(a.DataPackagePath) == "" {
|
|
||||||
return fmt.Errorf("prompt preparation data package path is required")
|
|
||||||
}
|
|
||||||
if err := validatePromptArtifactTiming("prompt preparation", a.StartedAt, a.EndedAt, a.Duration); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
switch a.Status {
|
|
||||||
case PromptPreparationSucceeded:
|
|
||||||
if a.Preparation == nil || a.Error != nil {
|
|
||||||
return fmt.Errorf("successful prompt preparation requires preparation without an error")
|
|
||||||
}
|
|
||||||
if a.Preparation.PromptID != a.PromptID || a.Preparation.PromptVersion != a.PromptVersion || a.Preparation.DataPackagePath != a.DataPackagePath {
|
|
||||||
return fmt.Errorf("successful prompt preparation provenance must match the artifact")
|
|
||||||
}
|
|
||||||
case PromptPreparationFailed:
|
|
||||||
if !validPromptArtifactError(a.Error) {
|
|
||||||
return fmt.Errorf("failed prompt preparation requires a classified error")
|
|
||||||
}
|
|
||||||
if a.Preparation != nil {
|
|
||||||
return fmt.Errorf("failed prompt preparation must not include preparation provenance")
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
return fmt.Errorf("unsupported prompt preparation status %q", a.Status)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
type PromptExecutionStatus string
|
|
||||||
|
|
||||||
const (
|
|
||||||
PromptExecutionSucceeded PromptExecutionStatus = "succeeded"
|
|
||||||
PromptExecutionValidationRejected PromptExecutionStatus = "validation_rejected"
|
|
||||||
PromptExecutionFailed PromptExecutionStatus = "failed"
|
|
||||||
)
|
|
||||||
|
|
||||||
// PromptExecutionProvenance is the safe subset of promptexec.Execution. The
|
|
||||||
// generated content and debug payload are intentionally excluded.
|
|
||||||
type PromptExecutionProvenance struct {
|
|
||||||
RunID string `json:"runId"`
|
|
||||||
PromptID string `json:"promptId"`
|
|
||||||
PromptVersion string `json:"promptVersion"`
|
|
||||||
PromptHash string `json:"promptHash"`
|
|
||||||
RenderedPromptHash string `json:"renderedPromptHash"`
|
|
||||||
InputHashes map[string]string `json:"inputHashes,omitempty"`
|
|
||||||
ProfileID string `json:"profileId"`
|
|
||||||
BackendID string `json:"backendId"`
|
|
||||||
ModelName string `json:"modelName"`
|
|
||||||
GeneratedHash string `json:"generatedHash,omitempty"`
|
|
||||||
Usage promptexec.TokenUsage `json:"usage"`
|
|
||||||
StartedAt time.Time `json:"startedAt"`
|
|
||||||
EndedAt time.Time `json:"endedAt"`
|
|
||||||
Duration time.Duration `json:"duration"`
|
|
||||||
DataPackagePath string `json:"dataPackagePath"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// PromptExecutionPaths records only destinations reached by a completed run.
|
|
||||||
// It contains paths, never generated content or debug information.
|
|
||||||
type PromptExecutionPaths struct {
|
|
||||||
RawOutputPath string `json:"rawOutputPath,omitempty"`
|
|
||||||
GeneratedTextPath string `json:"generatedTextPath,omitempty"`
|
|
||||||
RenderContextPath string `json:"renderContextPath,omitempty"`
|
|
||||||
RenderedReportPath string `json:"renderedReportPath,omitempty"`
|
|
||||||
OutputPath string `json:"outputPath,omitempty"`
|
|
||||||
NotificationPath string `json:"notificationPath,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// PromptExecutionArtifact records safe execution provenance and its validation
|
|
||||||
// outcome. It never embeds generated output or content-rich debug data.
|
|
||||||
type PromptExecutionArtifact struct {
|
|
||||||
SchemaVersion string `json:"schemaVersion"`
|
|
||||||
Status PromptExecutionStatus `json:"status"`
|
|
||||||
ReportID report.ID `json:"reportId"`
|
|
||||||
RunID string `json:"runId"`
|
|
||||||
PromptID string `json:"promptId"`
|
|
||||||
PromptVersion string `json:"promptVersion,omitempty"`
|
|
||||||
Provenance *PromptExecutionProvenance `json:"provenance,omitempty"`
|
|
||||||
Validation *promptexec.Validation `json:"validation,omitempty"`
|
|
||||||
Paths PromptExecutionPaths `json:"paths,omitempty"`
|
|
||||||
StartedAt time.Time `json:"startedAt"`
|
|
||||||
EndedAt time.Time `json:"endedAt"`
|
|
||||||
Duration time.Duration `json:"duration"`
|
|
||||||
Error *PromptArtifactError `json:"error,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func PromptExecutionProvenanceFrom(value promptexec.Execution) PromptExecutionProvenance {
|
|
||||||
inputHashes := make(map[string]string, len(value.InputHashes))
|
|
||||||
for key, item := range value.InputHashes {
|
|
||||||
inputHashes[key] = item
|
|
||||||
}
|
|
||||||
return PromptExecutionProvenance{
|
|
||||||
RunID: value.RunID, PromptID: value.PromptID, PromptVersion: value.PromptVersion,
|
|
||||||
PromptHash: value.PromptHash, RenderedPromptHash: value.RenderedPromptHash,
|
|
||||||
InputHashes: inputHashes, ProfileID: value.ProfileID, BackendID: value.BackendID,
|
|
||||||
ModelName: value.ModelName, GeneratedHash: value.GeneratedHash, Usage: value.Usage,
|
|
||||||
StartedAt: value.StartedAt, EndedAt: value.EndedAt, Duration: value.Duration,
|
|
||||||
DataPackagePath: value.DataPackagePath,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (a PromptExecutionArtifact) Validate() error {
|
|
||||||
if a.SchemaVersion != PromptExecutionSchemaVersion {
|
|
||||||
return fmt.Errorf("unsupported prompt execution schema version %q", a.SchemaVersion)
|
|
||||||
}
|
|
||||||
if err := validatePromptArtifactIdentity("prompt execution", a.ReportID, a.RunID, a.PromptID, a.PromptVersion); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := validatePromptArtifactTiming("prompt execution", a.StartedAt, a.EndedAt, a.Duration); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
switch a.Status {
|
|
||||||
case PromptExecutionSucceeded:
|
|
||||||
if a.Provenance == nil || a.Validation == nil || a.Validation.Status != promptexec.ValidationPassed || a.Error != nil {
|
|
||||||
return fmt.Errorf("successful prompt execution requires passed validation without an error")
|
|
||||||
}
|
|
||||||
if err := validatePromptExecutionProvenance(a); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
case PromptExecutionValidationRejected:
|
|
||||||
if a.Provenance == nil || a.Validation == nil || a.Validation.Status != promptexec.ValidationFailed || a.Error != nil {
|
|
||||||
return fmt.Errorf("validation-rejected prompt execution requires failed validation without an error")
|
|
||||||
}
|
|
||||||
if err := validatePromptExecutionProvenance(a); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
case PromptExecutionFailed:
|
|
||||||
if !validPromptArtifactError(a.Error) {
|
|
||||||
return fmt.Errorf("failed prompt execution requires a classified error")
|
|
||||||
}
|
|
||||||
if a.Provenance != nil || a.Validation != nil {
|
|
||||||
return fmt.Errorf("failed prompt execution must not include completed provenance or validation")
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
return fmt.Errorf("unsupported prompt execution status %q", a.Status)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func validPromptArtifactError(value *PromptArtifactError) bool {
|
|
||||||
return value != nil && validPromptErrorCategory(value.Category) && strings.TrimSpace(value.Message) != "" && len(value.Message) <= promptArtifactErrorLimit && utf8.ValidString(value.Message)
|
|
||||||
}
|
|
||||||
|
|
||||||
func validatePromptArtifactIdentity(kind string, reportID report.ID, runID, promptID, promptVersion string) error {
|
|
||||||
if reportID == "" || strings.TrimSpace(runID) == "" || strings.TrimSpace(promptID) == "" {
|
|
||||||
return fmt.Errorf("%s identity is required", kind)
|
|
||||||
}
|
|
||||||
definition, err := report.DefaultRegistry().Lookup(reportID)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("%s report id is unsupported: %w", kind, err)
|
|
||||||
}
|
|
||||||
if promptID != definition.PromptID {
|
|
||||||
return fmt.Errorf("%s prompt id must match report %q", kind, reportID)
|
|
||||||
}
|
|
||||||
if promptVersion != definition.PromptVersion {
|
|
||||||
return fmt.Errorf("%s prompt version must be %q", kind, definition.PromptVersion)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func validatePromptArtifactTiming(kind string, startedAt, endedAt time.Time, duration time.Duration) error {
|
|
||||||
if startedAt.IsZero() || endedAt.IsZero() {
|
|
||||||
return fmt.Errorf("%s start and end times are required", kind)
|
|
||||||
}
|
|
||||||
if duration < 0 {
|
|
||||||
return fmt.Errorf("%s duration must not be negative", kind)
|
|
||||||
}
|
|
||||||
if endedAt.Before(startedAt) {
|
|
||||||
return fmt.Errorf("%s end time must not be earlier than its start time", kind)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func validatePromptExecutionProvenance(artifact PromptExecutionArtifact) error {
|
|
||||||
value := artifact.Provenance
|
|
||||||
if value == nil {
|
|
||||||
return fmt.Errorf("completed prompt execution provenance is required")
|
|
||||||
}
|
|
||||||
if value.PromptID != artifact.PromptID || value.PromptVersion != artifact.PromptVersion {
|
|
||||||
return fmt.Errorf("completed prompt execution provenance must match the artifact")
|
|
||||||
}
|
|
||||||
for _, required := range []struct {
|
|
||||||
name string
|
|
||||||
value string
|
|
||||||
}{
|
|
||||||
{"run id", value.RunID}, {"prompt hash", value.PromptHash}, {"rendered prompt hash", value.RenderedPromptHash},
|
|
||||||
{"profile id", value.ProfileID}, {"backend id", value.BackendID}, {"model name", value.ModelName},
|
|
||||||
{"data package path", value.DataPackagePath},
|
|
||||||
} {
|
|
||||||
if strings.TrimSpace(required.value) == "" {
|
|
||||||
return fmt.Errorf("completed prompt execution provenance %s is required", required.name)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if err := validatePromptArtifactTiming("completed prompt execution provenance", value.StartedAt, value.EndedAt, value.Duration); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func validPromptErrorCategory(category promptexec.ErrorCategory) bool {
|
|
||||||
switch category {
|
|
||||||
case promptexec.InvalidConfiguration, promptexec.InvalidRequest, promptexec.PromptNotFound,
|
|
||||||
promptexec.PromptLoad, promptexec.ProfileNotFound, promptexec.ProfileLoad,
|
|
||||||
promptexec.MissingCredential, promptexec.ArtifactLoad, promptexec.PromptRender,
|
|
||||||
promptexec.Capacity, promptexec.Generation, promptexec.OperationalValidation,
|
|
||||||
promptexec.ValidationRejected, promptexec.Canceled, promptexec.DeadlineExceeded:
|
|
||||||
return true
|
|
||||||
default:
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,207 +0,0 @@
|
|||||||
package state
|
|
||||||
|
|
||||||
import (
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptexec"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestPromptPreparationArtifactValidation(t *testing.T) {
|
|
||||||
valid := validPreparationArtifact()
|
|
||||||
if err := valid.Validate(); err != nil {
|
|
||||||
t.Fatalf("valid successful preparation: %v", err)
|
|
||||||
}
|
|
||||||
failed := valid
|
|
||||||
failed.Status = PromptPreparationFailed
|
|
||||||
failed.Preparation = nil
|
|
||||||
failed.Error = &PromptArtifactError{Category: promptexec.Generation, Message: "provider unavailable"}
|
|
||||||
if err := failed.Validate(); err != nil {
|
|
||||||
t.Fatalf("valid failed preparation: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
mutate func(*PromptPreparationArtifact)
|
|
||||||
}{
|
|
||||||
{"report id", func(a *PromptPreparationArtifact) { a.ReportID = "" }},
|
|
||||||
{"run id", func(a *PromptPreparationArtifact) { a.RunID = "" }},
|
|
||||||
{"prompt id", func(a *PromptPreparationArtifact) { a.PromptID = "" }},
|
|
||||||
{"prompt id for another report", func(a *PromptPreparationArtifact) { a.PromptID = "weather.hourly_generated_text" }},
|
|
||||||
{"prompt version", func(a *PromptPreparationArtifact) { a.PromptVersion = "latest" }},
|
|
||||||
{"data package", func(a *PromptPreparationArtifact) { a.DataPackagePath = "" }},
|
|
||||||
{"start time", func(a *PromptPreparationArtifact) { a.StartedAt = time.Time{} }},
|
|
||||||
{"end time", func(a *PromptPreparationArtifact) { a.EndedAt = time.Time{} }},
|
|
||||||
{"negative duration", func(a *PromptPreparationArtifact) { a.Duration = -time.Second }},
|
|
||||||
{"reversed times", func(a *PromptPreparationArtifact) { a.EndedAt = a.StartedAt.Add(-time.Second) }},
|
|
||||||
{"missing provenance", func(a *PromptPreparationArtifact) { a.Preparation = nil }},
|
|
||||||
{"success error", func(a *PromptPreparationArtifact) {
|
|
||||||
a.Error = &PromptArtifactError{Category: promptexec.Generation, Message: "failed"}
|
|
||||||
}},
|
|
||||||
{"provenance prompt id", func(a *PromptPreparationArtifact) { a.Preparation.PromptID = "other" }},
|
|
||||||
{"provenance prompt version", func(a *PromptPreparationArtifact) { a.Preparation.PromptVersion = "other" }},
|
|
||||||
{"provenance data package", func(a *PromptPreparationArtifact) { a.Preparation.DataPackagePath = "/other/data.yaml" }},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
artifact := validPreparationArtifact()
|
|
||||||
test.mutate(&artifact)
|
|
||||||
if err := artifact.Validate(); err == nil {
|
|
||||||
t.Fatalf("Validate() error = nil for %#v", artifact)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestFailedPromptPreparationRejectsContradictoryDetails(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
mutate func(*PromptPreparationArtifact)
|
|
||||||
}{
|
|
||||||
{"missing error", func(a *PromptPreparationArtifact) { a.Error = nil }},
|
|
||||||
{"unknown category", func(a *PromptPreparationArtifact) { a.Error.Category = promptexec.ErrorCategory("other") }},
|
|
||||||
{"empty message", func(a *PromptPreparationArtifact) { a.Error.Message = " " }},
|
|
||||||
{"oversized message", func(a *PromptPreparationArtifact) { a.Error.Message = strings.Repeat("x", promptArtifactErrorLimit+1) }},
|
|
||||||
{"invented provenance", func(a *PromptPreparationArtifact) { a.Preparation = validPreparationArtifact().Preparation }},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
artifact := validPreparationArtifact()
|
|
||||||
artifact.Status = PromptPreparationFailed
|
|
||||||
artifact.Preparation = nil
|
|
||||||
artifact.Error = &PromptArtifactError{Category: promptexec.Generation, Message: "provider unavailable"}
|
|
||||||
test.mutate(&artifact)
|
|
||||||
if err := artifact.Validate(); err == nil {
|
|
||||||
t.Fatalf("Validate() error = nil for %#v", artifact)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestPromptExecutionArtifactValidation(t *testing.T) {
|
|
||||||
valid := validExecutionArtifact()
|
|
||||||
valid.Provenance.RunID = "provider-run-different-from-weatherreporter"
|
|
||||||
valid.Provenance.GeneratedHash = ""
|
|
||||||
valid.Provenance.Usage = promptexec.TokenUsage{}
|
|
||||||
if err := valid.Validate(); err != nil {
|
|
||||||
t.Fatalf("valid completed execution with provider run identity and omitted counters: %v", err)
|
|
||||||
}
|
|
||||||
rejected := validExecutionArtifact()
|
|
||||||
rejected.Status = PromptExecutionValidationRejected
|
|
||||||
validation := promptexec.NewValidation(promptexec.ValidationFailed, "json_schema", "daily.generated_text.schema.json", []string{"schema mismatch"})
|
|
||||||
rejected.Validation = &validation
|
|
||||||
if err := rejected.Validate(); err != nil {
|
|
||||||
t.Fatalf("valid validation rejection: %v", err)
|
|
||||||
}
|
|
||||||
failed := validFailedExecutionArtifact()
|
|
||||||
if err := failed.Validate(); err != nil {
|
|
||||||
t.Fatalf("valid operational failure: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
mutate func(*PromptExecutionArtifact)
|
|
||||||
}{
|
|
||||||
{"report id", func(a *PromptExecutionArtifact) { a.ReportID = "" }},
|
|
||||||
{"run id", func(a *PromptExecutionArtifact) { a.RunID = "" }},
|
|
||||||
{"prompt id", func(a *PromptExecutionArtifact) { a.PromptID = "" }},
|
|
||||||
{"prompt version", func(a *PromptExecutionArtifact) { a.PromptVersion = "latest" }},
|
|
||||||
{"start time", func(a *PromptExecutionArtifact) { a.StartedAt = time.Time{} }},
|
|
||||||
{"end time", func(a *PromptExecutionArtifact) { a.EndedAt = time.Time{} }},
|
|
||||||
{"negative duration", func(a *PromptExecutionArtifact) { a.Duration = -time.Second }},
|
|
||||||
{"reversed times", func(a *PromptExecutionArtifact) { a.EndedAt = a.StartedAt.Add(-time.Second) }},
|
|
||||||
{"missing provenance", func(a *PromptExecutionArtifact) { a.Provenance = nil }},
|
|
||||||
{"missing validation", func(a *PromptExecutionArtifact) { a.Validation = nil }},
|
|
||||||
{"wrong validation", func(a *PromptExecutionArtifact) {
|
|
||||||
value := promptexec.NewValidation(promptexec.ValidationFailed, "json_schema", "schema.json", nil)
|
|
||||||
a.Validation = &value
|
|
||||||
}},
|
|
||||||
{"operational error", func(a *PromptExecutionArtifact) {
|
|
||||||
a.Error = &PromptArtifactError{Category: promptexec.Generation, Message: "failed"}
|
|
||||||
}},
|
|
||||||
{"provenance prompt id", func(a *PromptExecutionArtifact) { a.Provenance.PromptID = "other" }},
|
|
||||||
{"provenance prompt version", func(a *PromptExecutionArtifact) { a.Provenance.PromptVersion = "other" }},
|
|
||||||
{"provenance run id", func(a *PromptExecutionArtifact) { a.Provenance.RunID = "" }},
|
|
||||||
{"prompt hash", func(a *PromptExecutionArtifact) { a.Provenance.PromptHash = "" }},
|
|
||||||
{"rendered hash", func(a *PromptExecutionArtifact) { a.Provenance.RenderedPromptHash = "" }},
|
|
||||||
{"profile id", func(a *PromptExecutionArtifact) { a.Provenance.ProfileID = "" }},
|
|
||||||
{"backend id", func(a *PromptExecutionArtifact) { a.Provenance.BackendID = "" }},
|
|
||||||
{"model name", func(a *PromptExecutionArtifact) { a.Provenance.ModelName = "" }},
|
|
||||||
{"data package", func(a *PromptExecutionArtifact) { a.Provenance.DataPackagePath = "" }},
|
|
||||||
{"provenance start time", func(a *PromptExecutionArtifact) { a.Provenance.StartedAt = time.Time{} }},
|
|
||||||
{"provenance end time", func(a *PromptExecutionArtifact) { a.Provenance.EndedAt = time.Time{} }},
|
|
||||||
{"provenance negative duration", func(a *PromptExecutionArtifact) { a.Provenance.Duration = -time.Second }},
|
|
||||||
{"provenance reversed times", func(a *PromptExecutionArtifact) { a.Provenance.EndedAt = a.Provenance.StartedAt.Add(-time.Second) }},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
artifact := validExecutionArtifact()
|
|
||||||
test.mutate(&artifact)
|
|
||||||
if err := artifact.Validate(); err == nil {
|
|
||||||
t.Fatalf("Validate() error = nil for %#v", artifact)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestFailedPromptExecutionRejectsContradictoryDetails(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
mutate func(*PromptExecutionArtifact)
|
|
||||||
}{
|
|
||||||
{"missing error", func(a *PromptExecutionArtifact) { a.Error = nil }},
|
|
||||||
{"unknown category", func(a *PromptExecutionArtifact) { a.Error.Category = promptexec.ErrorCategory("other") }},
|
|
||||||
{"provenance", func(a *PromptExecutionArtifact) { a.Provenance = validExecutionArtifact().Provenance }},
|
|
||||||
{"completed validation", func(a *PromptExecutionArtifact) { a.Validation = validExecutionArtifact().Validation }},
|
|
||||||
}
|
|
||||||
for _, test := range tests {
|
|
||||||
t.Run(test.name, func(t *testing.T) {
|
|
||||||
artifact := validFailedExecutionArtifact()
|
|
||||||
test.mutate(&artifact)
|
|
||||||
if err := artifact.Validate(); err == nil {
|
|
||||||
t.Fatalf("Validate() error = nil for %#v", artifact)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func validPreparationArtifact() PromptPreparationArtifact {
|
|
||||||
started := time.Date(2026, 5, 29, 15, 0, 0, 0, time.UTC)
|
|
||||||
return PromptPreparationArtifact{
|
|
||||||
SchemaVersion: PromptPreparationSchemaVersion, Status: PromptPreparationSucceeded,
|
|
||||||
ReportID: report.Daily, RunID: "weatherreporter-run", PromptID: "weather.daily_generated_text",
|
|
||||||
PromptVersion: "1.1.0", DataPackagePath: "/workspace/data.yaml",
|
|
||||||
Preparation: &promptexec.Preparation{
|
|
||||||
PromptID: "weather.daily_generated_text", PromptVersion: "1.1.0",
|
|
||||||
DataPackagePath: "/workspace/data.yaml",
|
|
||||||
},
|
|
||||||
StartedAt: started, EndedAt: started.Add(time.Second), Duration: time.Second,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func validExecutionArtifact() PromptExecutionArtifact {
|
|
||||||
started := time.Date(2026, 5, 29, 15, 0, 0, 0, time.UTC)
|
|
||||||
validation := promptexec.NewValidation(promptexec.ValidationPassed, "json_schema", "daily.generated_text.schema.json", nil)
|
|
||||||
return PromptExecutionArtifact{
|
|
||||||
SchemaVersion: PromptExecutionSchemaVersion, Status: PromptExecutionSucceeded,
|
|
||||||
ReportID: report.Daily, RunID: "weatherreporter-run", PromptID: "weather.daily_generated_text", PromptVersion: "1.1.0",
|
|
||||||
Provenance: &PromptExecutionProvenance{
|
|
||||||
RunID: "provider-run", PromptID: "weather.daily_generated_text", PromptVersion: "1.1.0",
|
|
||||||
PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash", ProfileID: "profile",
|
|
||||||
BackendID: "backend", ModelName: "model", DataPackagePath: "/workspace/data.yaml",
|
|
||||||
StartedAt: started, EndedAt: started.Add(time.Second), Duration: time.Second,
|
|
||||||
},
|
|
||||||
Validation: &validation, StartedAt: started, EndedAt: started.Add(time.Second), Duration: time.Second,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func validFailedExecutionArtifact() PromptExecutionArtifact {
|
|
||||||
started := time.Date(2026, 5, 29, 15, 0, 0, 0, time.UTC)
|
|
||||||
return PromptExecutionArtifact{
|
|
||||||
SchemaVersion: PromptExecutionSchemaVersion, Status: PromptExecutionFailed,
|
|
||||||
ReportID: report.Daily, RunID: "weatherreporter-run", PromptID: "weather.daily_generated_text", PromptVersion: "1.1.0",
|
|
||||||
StartedAt: started, EndedAt: started, Error: &PromptArtifactError{Category: promptexec.Generation, Message: "provider unavailable"},
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,109 +0,0 @@
|
|||||||
// Package state persists report artifacts and metadata.
|
|
||||||
package state
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/module"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/promptinput"
|
|
||||||
"gitea.maximumdirect.net/eric/weatherreporter/internal/report"
|
|
||||||
)
|
|
||||||
|
|
||||||
type Store interface {
|
|
||||||
Paths(report.Resolved) (ArtifactPaths, error)
|
|
||||||
SaveModuleSnapshot(context.Context, report.Resolved, module.Snapshot) (string, error)
|
|
||||||
SaveDataPackage(context.Context, report.Resolved, promptinput.Package) (string, error)
|
|
||||||
SaveDataPackageBytes(context.Context, report.Resolved, []byte) (string, error)
|
|
||||||
SavePromptPreparation(context.Context, report.Resolved, PromptPreparationArtifact) (string, error)
|
|
||||||
SavePromptExecution(context.Context, report.Resolved, PromptExecutionArtifact) (string, error)
|
|
||||||
SaveDistributorNotification(context.Context, report.Resolved, DistributorNotificationArtifact) (string, error)
|
|
||||||
SaveBatchDistributorNotification(context.Context, BatchDistributorNotificationRef, BatchDistributorNotificationArtifact) (string, error)
|
|
||||||
SaveGeneratedTextRaw(context.Context, report.Resolved, []byte) (string, error)
|
|
||||||
SaveGeneratedText(context.Context, report.Resolved, []byte) (string, error)
|
|
||||||
SaveRenderContext(context.Context, report.Resolved, any) (string, error)
|
|
||||||
PrepareRenderedReport(context.Context, report.Resolved) (string, error)
|
|
||||||
SaveMetadata(context.Context, Metadata) (string, error)
|
|
||||||
FindPriorSnapshot(context.Context, report.Resolved) (*PriorSnapshot, error)
|
|
||||||
LoadModuleSnapshot(context.Context, string) (module.Snapshot, error)
|
|
||||||
LoadGeneratedText(context.Context, string) ([]byte, error)
|
|
||||||
LoadPromptPreparation(context.Context, string) (PromptPreparationArtifact, error)
|
|
||||||
LoadPromptExecution(context.Context, string) (PromptExecutionArtifact, error)
|
|
||||||
LoadRenderContext(context.Context, string, any) error
|
|
||||||
}
|
|
||||||
|
|
||||||
type PriorSnapshot struct {
|
|
||||||
Metadata Metadata
|
|
||||||
ModuleSnapshotPath string
|
|
||||||
}
|
|
||||||
|
|
||||||
const DistributorNotificationSchemaVersion = "weatherreporter.distributor_notification.v1"
|
|
||||||
const BatchDistributorNotificationSchemaVersion = "weatherreporter.batch_distributor_notification.v1"
|
|
||||||
|
|
||||||
type BatchDistributorNotificationRef struct {
|
|
||||||
Batch string
|
|
||||||
BatchRunID string
|
|
||||||
StartedAt time.Time
|
|
||||||
Location *time.Location
|
|
||||||
}
|
|
||||||
|
|
||||||
type DistributorNotificationArtifact struct {
|
|
||||||
SchemaVersion string `json:"schemaVersion"`
|
|
||||||
RunID string `json:"runId"`
|
|
||||||
ReportID report.ID `json:"reportId"`
|
|
||||||
AttemptedAt time.Time `json:"attemptedAt"`
|
|
||||||
Endpoint string `json:"endpoint"`
|
|
||||||
PipelineID string `json:"pipelineId,omitempty"`
|
|
||||||
BundleID string `json:"bundleId,omitempty"`
|
|
||||||
IdempotencyKey string `json:"idempotencyKey,omitempty"`
|
|
||||||
SourcePath string `json:"sourcePath,omitempty"`
|
|
||||||
BundlePaths []string `json:"bundlePaths,omitempty"`
|
|
||||||
BundleCreated time.Time `json:"bundleCreated,omitempty"`
|
|
||||||
Status string `json:"status"`
|
|
||||||
Upload *DistributorUploadResult `json:"upload,omitempty"`
|
|
||||||
RunStatus *DistributorRunStatus `json:"runStatus,omitempty"`
|
|
||||||
StatusError string `json:"statusError,omitempty"`
|
|
||||||
Error string `json:"error,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type DistributorUploadResult struct {
|
|
||||||
RunID string `json:"runId,omitempty"`
|
|
||||||
Status string `json:"status,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type DistributorRunStatus struct {
|
|
||||||
RunID string `json:"runId,omitempty"`
|
|
||||||
PipelineID string `json:"pipelineId,omitempty"`
|
|
||||||
Status string `json:"status,omitempty"`
|
|
||||||
AcceptedAt time.Time `json:"acceptedAt,omitempty"`
|
|
||||||
StartedAt *time.Time `json:"startedAt,omitempty"`
|
|
||||||
FinishedAt *time.Time `json:"finishedAt,omitempty"`
|
|
||||||
Report json.RawMessage `json:"report,omitempty"`
|
|
||||||
Error string `json:"error,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type BatchDistributorNotificationArtifact struct {
|
|
||||||
SchemaVersion string `json:"schemaVersion"`
|
|
||||||
Batch string `json:"batch"`
|
|
||||||
BatchRunID string `json:"batchRunId"`
|
|
||||||
AttemptedAt time.Time `json:"attemptedAt"`
|
|
||||||
Endpoint string `json:"endpoint"`
|
|
||||||
PipelineID string `json:"pipelineId,omitempty"`
|
|
||||||
BundleID string `json:"bundleId,omitempty"`
|
|
||||||
IdempotencyKey string `json:"idempotencyKey,omitempty"`
|
|
||||||
BundleCreated time.Time `json:"bundleCreated,omitempty"`
|
|
||||||
Reports []BatchDistributorNotificationReportArtifact `json:"includedReports,omitempty"`
|
|
||||||
Status string `json:"status"`
|
|
||||||
Upload *DistributorUploadResult `json:"upload,omitempty"`
|
|
||||||
RunStatus *DistributorRunStatus `json:"runStatus,omitempty"`
|
|
||||||
StatusError string `json:"statusError,omitempty"`
|
|
||||||
Error string `json:"error,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type BatchDistributorNotificationReportArtifact struct {
|
|
||||||
ReportID report.ID `json:"reportId"`
|
|
||||||
RunID string `json:"runId"`
|
|
||||||
SourcePath string `json:"sourcePath"`
|
|
||||||
BundlePaths []string `json:"bundlePaths"`
|
|
||||||
}
|
|
||||||
Reference in New Issue
Block a user