eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Document profile comparison workflow

Browse Source
This commit is contained in:
Eric Rakestraw
2026-08-02 06:10:15 +00:00
parent ccf6b66880
commit e0229d9c90
14 changed files with 322 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

48
docs/operations.md
View File

@@ -72,6 +72,38 @@ For a single report, Distributor notification follows the atomic output write.
See the [configuration reference](config.md) for pipeline, bundle,
idempotency-key, and per-report path templates.
## Comparison Bundles
Use `compare` when an operator needs to evaluate explicit Promptkit profiles
against the same report input. The command writes one flat, operator-owned
bundle directory and never sends a Distributor notification. Command syntax,
profile validation, JSON output, and exit behavior belong to the
[CLI reference](cli.md); the durable file contract belongs to the
[comparison bundle contract](integrations/comparison-bundle.md).
The output destination follows the normal `output.directory` fallback. An
explicit `--out-dir` takes precedence and names the exact bundle directory,
not a parent to be combined with another name. The standard names are derived
from the report output name, such as `comparison-today` and
`comparison-daily-2026-05-29`; see the [configuration reference](config.md)
for output-directory resolution.
A comparison bundle contains the shared data package, a manifest, and one
Markdown file for every successful profile. Treat all of these files as
potentially sensitive: the data package and generated reports can contain
location or forecast context. Weatherreporter creates no application-owned
history, retention store, or cleanup job. Retain, archive, or remove only the
specific bundle directories your operating policy permits.
The destination is preflighted before prompt inspection and collection, then
rechecked immediately before an atomic publish. A missing or empty directory
is usable. A nonempty directory can be replaced only when `--replace` is given
and it is recognized as a current Weatherreporter comparison bundle; ordinary
directories, symlinks, and unsafe destinations are rejected. Cancellation and
all failures before publication preserve an existing bundle. Profile failures
are different: the command publishes a complete partial bundle, with failed
profiles represented in the manifest and no Markdown file for those profiles.
## Local Prompt Profile Override
Hourly normally selects the embedded `weather-light` profile. To use a local
@@ -99,9 +131,11 @@ weatherreporter generate today --llm-debug-dir /var/tmp/weatherreporter-debug
The directory must be absolute. Requested captures are written with restrictive
permissions beneath the supplied directory, organized by report and run. They
can contain rendered prompts and generated output, so limit access to trusted
operators and remove the captures when they are no longer needed. Normal output,
summaries, and routine logs omit that sensitive content. Debug capture is never
created for an ordinary command without `--llm-debug-dir`.
operators and remove the captures when they are no longer needed. Comparison
captures additionally identify each selected profile so concurrent executions
remain distinct. Normal output, summaries, and routine logs omit that sensitive
content. Debug capture is never created for an ordinary command without
`--llm-debug-dir`.
If capture creation or writing fails, the affected run fails rather than
silently continuing without the requested diagnostics.
@@ -111,8 +145,12 @@ silently continuing without the requested diagnostics.
Start with the command error and JSON summary. For a report generation failure,
the selected destination was not replaced; for a notification failure, inspect
the completed destination and the notification result. For a batch failure,
use the per-report statuses and retain successful output files. Enable explicit
debug capture only when content-rich Promptkit diagnostics are necessary.
use the per-report statuses and retain successful output files. For a comparison
failure, inspect the published manifest when its path is present: individual
profile failures retain their safe result and successful Markdown files, while
cancellation and pre-publication errors leave the prior destination unchanged.
Enable explicit debug capture only when content-rich Promptkit diagnostics are
necessary.
Weatherreporter does not retain runs for later inspection, resume failed work,
or provide automatic cleanup, archival, remote state, daemon operation, or