eric/weatherreporter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Document profile comparison workflow

Browse Source
This commit is contained in:
Eric Rakestraw
2026-08-02 06:10:15 +00:00
parent ccf6b66880
commit e0229d9c90
14 changed files with 322 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

92
docs/integrations/comparison-bundle.md Normal file
View File

@@ -0,0 +1,92 @@
# Comparison Bundle Contract
A comparison bundle is the durable, flat artifact produced when one report is
executed with multiple explicit Promptkit profiles. This document is the
canonical contract for consumers of those bundles. Command invocation and JSON
action summaries belong to the [CLI reference](../cli.md); destination handling
and retention belong to the [operations guide](../operations.md).
## Version And Layout
The current and only supported manifest schema version is
`weatherreporter.comparison.v1`. A bundle directory contains exactly these
regular, non-symlinked files:
```text
comparison.json
data-package.yml
NN-profile-slug.md
```
`comparison.json` is the manifest and `data-package.yml` is the exact YAML
input supplied to every selected profile. There is one Markdown file for each
successful result and none for failed results. `NN` is the one-based selected
profile position, zero padded to at least two digits (and widened only when
needed for 100 or more profiles). The profile slug preserves ASCII letters,
digits, `-`, and `_`; each run of other characters becomes one `-`; edge `-`
and `_` characters are removed; the value is capped at 64 bytes; and an empty
slug becomes `profile`. Logical profile IDs remain authoritative in the
manifest.
All manifest paths are basenames relative to the bundle root. They never use
path separators, `.` or `..`. The CLI reports absolute paths only after a
bundle has been published.
## Manifest Schema
The manifest is UTF-8 JSON, encoded as two-space-indented JSON with one
trailing newline. Its fields appear in this order:
```text
schemaVersion, comparisonId, startedAt, finishedAt, reportId, validPeriod,
timezone, promptId, promptVersion, promptHash, dataPackage, total, succeeded,
failed, results
```
`validPeriod` contains `start` and `end`; it is a nonempty half-open period.
`dataPackage` contains `path` (always `data-package.yml`) and `sha256` (the
lowercase, 64-character SHA-256 digest of that file's exact bytes). `results`
is in the explicit profile-selection order. Its result-object fields appear in
this order:
```text
position, profileId, backendId, modelName, status, validationStatus,
reportPath, error
```
`startedAt` and `finishedAt` are nonzero UTC timestamps, and the latter is not
earlier than the former. `validPeriod` retains its resolved time offset.
`reportId`, `timezone`, prompt identity, model name, and comparison ID are
nonblank. `promptHash` and `dataPackage.sha256` are lowercase SHA-256 digests.
## Result Invariants
`total` is at least two and equals the number of results. Positions are
contiguous from one, profile IDs are distinct and nonblank, and
`succeeded + failed == total`.
A successful result has `status: "succeeded"`, `validationStatus: "passed"`,
a unique Markdown `reportPath`, and no `error`. A failed result has
`status: "failed"`, no `reportPath`, and an `error` object with nonblank
`category` and `message`. Its validation status is absent, `failed`, or
`skipped`. Error messages are valid UTF-8 and no longer than 1,024 bytes.
`backendId` and `validationStatus` are omitted when unavailable.
Every successful Markdown file is declared by exactly one successful result.
The directory contains no extra entries. Consumers can therefore verify the
data-package digest and the full manifest-to-file mapping without scanning a
larger workspace.
## Compatibility And Sensitivity
Weatherreporter recognizes a replaceable bundle only when it exactly satisfies
the current version, schema, file set, file types, relative-path rules, and
data-package digest. It rejects unknown manifest fields, multiple JSON values,
extra entries, symlinks, and future or otherwise unsupported versions. Treat a
bundle that fails recognition as an ordinary directory, not as a compatible
bundle.
The manifest contains safe operational provenance, but `data-package.yml` and
the generated Markdown can contain sensitive weather or location context. Do
not assume these artifacts are safe for public distribution. Handle retention,
access, and deletion according to the [operations guide](../operations.md).

16
docs/integrations/promptkit.md
View File

@@ -29,6 +29,22 @@ Profiles that require a direct API key are unsupported; a profile that reports `
Promptkit receives the YAML data package as an inline input and returns structured JSON that Weatherreporter validates before rendering its own Markdown template. Safe active provenance remains in memory. Content-rich diagnostics are opt-in through `--llm-debug-dir`; see [operations](../operations.md) for retention and permissions.
## Comparison Execution
For `compare`, Weatherreporter validates one exact prompt and every explicitly
selected profile before weather collection. It prepares one deterministic YAML
data package, retains immutable copies of the report inputs, and executes every
profile against the same exact data-package bytes. Each profile remains an
independent Promptkit execution: one provider or validation failure does not
stop its peers, while caller cancellation applies to every in-flight execution.
Weatherreporter starts selected profile executions concurrently and does not
add an application-level concurrency limit. Promptkit owns backend capacity and
any profile or backend concurrency policy. The durable comparison output and
its compatibility rules are defined by the
[comparison bundle contract](comparison-bundle.md); the user-facing command
contract is in the [CLI reference](../cli.md).
The generated-text schemas require `summary`, `forecast_discussion`, and `precipitation_timing`, and reject additional properties. Prompts return an empty string for `precipitation_timing` when the deterministic package contains no precipitation windows.
Prompt/profile configuration and the maintained local override example are owned by the [configuration reference](../config.md). Adapter construction and mapping are documented in the [Promptkit adapter internals](../internal/promptkit-adapter.md).