Compare commits
3 Commits
v0.9.0
...
51c35f7c22
| Author | SHA1 | Date | |
|---|---|---|---|
| 51c35f7c22 | |||
| f014a078ee | |||
| 8c19ad763b |
@@ -3,7 +3,7 @@
|
|||||||
Weatherreporter uses Promptkit for all generated-text reports. The four logical prompts are
|
Weatherreporter uses Promptkit for all generated-text reports. The four logical prompts are
|
||||||
`weather.daily_generated_text`, `weather.today_generated_text`,
|
`weather.daily_generated_text`, `weather.today_generated_text`,
|
||||||
`weather.tomorrow_generated_text`, and `weather.hourly_generated_text`, each at version
|
`weather.tomorrow_generated_text`, and `weather.hourly_generated_text`, each at version
|
||||||
`1.0.0`. Their prompt assets and generated-text JSON Schemas are embedded by
|
`1.0.1`. Their prompt assets and generated-text JSON Schemas are embedded by
|
||||||
`internal/promptassets`.
|
`internal/promptassets`.
|
||||||
|
|
||||||
Before collection, Weatherreporter inspects the exact prompt version, requires one required
|
Before collection, Weatherreporter inspects the exact prompt version, requires one required
|
||||||
@@ -17,6 +17,10 @@ JSON that Weatherreporter validates before rendering its own Markdown template.
|
|||||||
execution receipts are project-owned, safe provenance records. Content-rich diagnostics are
|
execution receipts are project-owned, safe provenance records. Content-rich diagnostics are
|
||||||
opt-in through `--llm-debug-dir`; see [operations](../operations.md) for retention and permissions.
|
opt-in through `--llm-debug-dir`; see [operations](../operations.md) for retention and permissions.
|
||||||
|
|
||||||
|
The generated-text schemas require `summary`, `forecast_discussion`, and
|
||||||
|
`precipitation_timing`, and reject additional properties. Prompts return an empty string for
|
||||||
|
`precipitation_timing` when the deterministic package contains no precipitation windows.
|
||||||
|
|
||||||
Prompt/profile configuration is owned by the [configuration reference](../config.md). Adapter
|
Prompt/profile configuration is owned by the [configuration reference](../config.md). Adapter
|
||||||
construction and mapping are documented in the [Promptkit adapter internals](../internal/promptkit-adapter.md).
|
construction and mapping are documented in the [Promptkit adapter internals](../internal/promptkit-adapter.md).
|
||||||
Durable metadata compatibility is described in [state internals](../internal/state.md).
|
Durable metadata compatibility is described in [state internals](../internal/state.md).
|
||||||
|
|||||||
@@ -17,8 +17,9 @@ value and canonical normalized JSON, loads its canonical schema through
|
|||||||
|
|
||||||
Daily, Today, and Tomorrow use a day-style value with required trimmed summary
|
Daily, Today, and Tomorrow use a day-style value with required trimmed summary
|
||||||
and one or more nonblank discussion paragraphs. Hourly requires trimmed summary
|
and one or more nonblank discussion paragraphs. Hourly requires trimmed summary
|
||||||
and a single trimmed discussion string. Each form permits optional trimmed
|
and a single trimmed discussion string. Every form also requires the
|
||||||
precipitation-timing and confidence prose. Typed decoding rejects unknown JSON
|
`precipitation_timing` field; an empty string means there is no supported timing
|
||||||
|
prose to render. Typed decoding rejects missing required fields and unknown JSON
|
||||||
fields; no general-purpose JSON Schema engine is used at runtime.
|
fields; no general-purpose JSON Schema engine is used at runtime.
|
||||||
|
|
||||||
## Render contexts
|
## Render contexts
|
||||||
|
|||||||
@@ -18,10 +18,10 @@ period and run metadata for one invocation.
|
|||||||
|
|
||||||
| Report ID | Prompt version | Period policy | Comparison | Registry batch flag | Output copy |
|
| Report ID | Prompt version | Period policy | Comparison | Registry batch flag | Output copy |
|
||||||
| --- | --- | --- | --- | --- | --- |
|
| --- | --- | --- | --- | --- | --- |
|
||||||
| `daily` | `1.0.0` | Explicit local civil day | Same valid date | Dynamic Daily inclusion is app-owned | `daily.md` |
|
| `daily` | `1.0.1` | Explicit local civil day | Same valid date | Dynamic Daily inclusion is app-owned | `daily.md` |
|
||||||
| `today` | `1.0.0` | Selected or current local civil day | Same valid date | Morning | `today.md` |
|
| `today` | `1.0.1` | Selected or current local civil day | Same valid date | Morning | `today.md` |
|
||||||
| `tomorrow` | `1.0.0` | Next local civil day | Same valid date | Evening | `tomorrow.md` |
|
| `tomorrow` | `1.0.1` | Next local civil day | Same valid date | Evening | `tomorrow.md` |
|
||||||
| `hourly` | `1.0.0` | Rolling six-hour interval | Rolling window | — | `hourly.md` |
|
| `hourly` | `1.0.1` | Rolling six-hour interval | Rolling window | — | `hourly.md` |
|
||||||
|
|
||||||
Each report pairs its ID and prompt version with matching template and schema
|
Each report pairs its ID and prompt version with matching template and schema
|
||||||
IDs. Exact template fields and schema assets belong to [report templates](../templates.md)
|
IDs. Exact template fields and schema assets belong to [report templates](../templates.md)
|
||||||
|
|||||||
294
docs/roadmap/domain-profiles.md
Normal file
294
docs/roadmap/domain-profiles.md
Normal file
@@ -0,0 +1,294 @@
|
|||||||
|
# Domain-Specific Prompt Profiles Roadmap
|
||||||
|
|
||||||
|
Status: Accepted feature direction; implementation has not started.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Weatherreporter should provide stable, domain-specific Promptkit profile IDs
|
||||||
|
that express the relative resource and analysis needs of its report products.
|
||||||
|
These logical profiles should give each report an appropriate default while
|
||||||
|
allowing operators to replace any definition through the existing configured
|
||||||
|
profile source.
|
||||||
|
|
||||||
|
This roadmap defines the scope, policy, and intended end state. A later staged
|
||||||
|
implementation plan will translate it into implementation work. This document
|
||||||
|
does not prescribe implementation stages.
|
||||||
|
|
||||||
|
## User Intent
|
||||||
|
|
||||||
|
The feature is intended to provide three related benefits:
|
||||||
|
|
||||||
|
- frequent reports can use a cost-effective model by default;
|
||||||
|
- reports needing broader synthesis can select a stronger default without
|
||||||
|
forcing the same cost on every invocation; and
|
||||||
|
- an installation can map a stable Weatherreporter profile ID to a model on a
|
||||||
|
local network endpoint without modifying embedded prompts or application
|
||||||
|
code.
|
||||||
|
|
||||||
|
`weather-light` describes the profile's intended resource tier, not a latency
|
||||||
|
guarantee. A locally hosted lightweight model may still generate slowly on the
|
||||||
|
available hardware.
|
||||||
|
|
||||||
|
## Current State
|
||||||
|
|
||||||
|
Daily, Today, Tomorrow, and Hourly each declare Promptkit's
|
||||||
|
`gemini-flash-latest` profile as their prompt default. The optional
|
||||||
|
`promptkit.profile` setting overrides that default for every selected report in
|
||||||
|
an invocation.
|
||||||
|
|
||||||
|
Weatherreporter accepts either `promptkit.profile_file` or
|
||||||
|
`promptkit.profile_dir` and passes that source to Promptkit. A matching external
|
||||||
|
profile can override a Promptkit built-in profile, and the configured local
|
||||||
|
backend can support profiles that select `backend: local`. Endpoint-only
|
||||||
|
OpenAI-compatible profiles can also provide their own endpoint.
|
||||||
|
|
||||||
|
Weatherreporter does not currently own or embed execution profiles. Promptkit's
|
||||||
|
current public source precedence also has no layer where application-owned
|
||||||
|
fallback profiles can remain overridable by the ordinary configured source.
|
||||||
|
|
||||||
|
## Prerequisite
|
||||||
|
|
||||||
|
Promptkit must first provide the application fallback profile capability
|
||||||
|
defined in the companion
|
||||||
|
[upstream feature request](promptkit-fallback-profiles-feature-request.md).
|
||||||
|
|
||||||
|
Weatherreporter should adopt a tagged Promptkit release containing that public
|
||||||
|
capability. It should not depend on Promptkit internal packages, reproduce
|
||||||
|
Promptkit's profile repository behavior, or commit a local module replacement
|
||||||
|
as the production dependency.
|
||||||
|
|
||||||
|
## Desired End State
|
||||||
|
|
||||||
|
Weatherreporter embeds usable definitions for these exact logical profile IDs:
|
||||||
|
|
||||||
|
- `weather-light`
|
||||||
|
- `weather-balanced`
|
||||||
|
- `weather-deep`
|
||||||
|
|
||||||
|
The profiles are Weatherreporter-owned assets and remain behind the existing
|
||||||
|
Promptkit adapter boundary. Prompt definitions select the logical IDs, while
|
||||||
|
Promptkit resolves the effective backend, endpoint, model, and generation
|
||||||
|
settings.
|
||||||
|
|
||||||
|
An operator can place a profile with the same ID in `profile_file` or
|
||||||
|
`profile_dir`. The operator definition completely replaces the embedded
|
||||||
|
Weatherreporter definition for that ID. If the external source does not contain
|
||||||
|
the selected ID, lookup falls through to Weatherreporter's embedded profile and
|
||||||
|
then to Promptkit's built-in catalog.
|
||||||
|
|
||||||
|
The existing global `promptkit.profile` setting remains available as an
|
||||||
|
explicit all-report override. No new configuration field is required for the
|
||||||
|
initial feature.
|
||||||
|
|
||||||
|
## Profile Catalog And Report Assignment
|
||||||
|
|
||||||
|
| Profile | Meaning | Initial default reports |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `weather-light` | Lowest-cost supported tier for frequent, bounded synthesis. It makes no latency promise. | Hourly |
|
||||||
|
| `weather-balanced` | General-purpose tier for broader day-scale synthesis and forecast discussion. | Daily, Today, Tomorrow |
|
||||||
|
| `weather-deep` | Highest-capability tier for explicit operator use and future products whose measured quality benefit warrants the cost. | None initially |
|
||||||
|
|
||||||
|
The initial assignment recognizes that Weatherreporter's deterministic modules
|
||||||
|
already perform most weather selection and calculation. A higher-capability
|
||||||
|
model should not become a default merely because it is available. Moving an
|
||||||
|
existing report to `weather-deep` requires evidence that the stronger tier
|
||||||
|
materially improves supported reasoning or output quality.
|
||||||
|
|
||||||
|
The three profile IDs are capability policies, not permanent aliases for one
|
||||||
|
provider or model family. Their embedded definitions may change in a future
|
||||||
|
Weatherreporter release, with the change disclosed through normal release and
|
||||||
|
compatibility documentation.
|
||||||
|
|
||||||
|
## Selection And Definition Precedence
|
||||||
|
|
||||||
|
Profile ID selection and profile definition lookup are separate decisions.
|
||||||
|
|
||||||
|
Weatherreporter selects the profile ID in this order:
|
||||||
|
|
||||||
|
1. nonblank `promptkit.profile`; or
|
||||||
|
2. the exact prompt version's `default_profile`.
|
||||||
|
|
||||||
|
Promptkit then resolves the selected profile definition in this order:
|
||||||
|
|
||||||
|
1. explicit in-memory profiles, when used by an embedding consumer or test;
|
||||||
|
2. Weatherreporter's configured `profile_file` or `profile_dir` source;
|
||||||
|
3. Weatherreporter's embedded fallback profiles; and
|
||||||
|
4. Promptkit's embedded built-in profiles.
|
||||||
|
|
||||||
|
A higher-precedence source falls through only when the selected ID is absent.
|
||||||
|
A matching but malformed operator profile fails before weather collection and
|
||||||
|
must not silently use the embedded definition.
|
||||||
|
|
||||||
|
## Local Endpoint Experience
|
||||||
|
|
||||||
|
An operator should be able to override `weather-light` with an endpoint-only
|
||||||
|
profile whose model name is understood by the local OpenAI-compatible server.
|
||||||
|
This path does not require a separate Weatherreporter local-backend setting.
|
||||||
|
|
||||||
|
Alternatively, an override may select `backend: local`; in that case the
|
||||||
|
existing `promptkit.local.endpoint` and concurrency settings continue to own
|
||||||
|
the shared local backend definition.
|
||||||
|
|
||||||
|
The selected local profile is deterministic configuration, not a preference
|
||||||
|
hint. Weatherreporter will not probe for a local model and will not
|
||||||
|
automatically fall back to a remote or paid profile when the endpoint is
|
||||||
|
unavailable. The failure remains visible and attributable to the selected
|
||||||
|
profile.
|
||||||
|
|
||||||
|
## Embedded Profile Policy
|
||||||
|
|
||||||
|
Each embedded profile must be a complete, valid Promptkit profile and must be
|
||||||
|
usable in a default installation with the documented credential mechanism. The
|
||||||
|
initial embedded profiles are expected to use Promptkit's `openrouter` backend,
|
||||||
|
allowing them to inherit its endpoint and `OPENROUTER_API_KEY` environment
|
||||||
|
variable without embedding credentials.
|
||||||
|
|
||||||
|
Embedded definitions should include only settings that are intentional for the
|
||||||
|
selected model and supported by its backend. Avoid incidental generation
|
||||||
|
parameters that reduce portability or trigger provider-specific request
|
||||||
|
failures without a demonstrated quality benefit.
|
||||||
|
|
||||||
|
The initial model candidates are:
|
||||||
|
|
||||||
|
- `weather-light`: the maintained OpenRouter identifier for the Gemma 4 31B
|
||||||
|
instruction model;
|
||||||
|
- `weather-balanced`: Promptkit's maintained Gemini Flash family target; and
|
||||||
|
- `weather-deep`: Promptkit's maintained Gemini Pro family target.
|
||||||
|
|
||||||
|
The exact identifiers and generation settings must be finalized against the
|
||||||
|
available tagged Promptkit catalog and a representative Weatherreporter
|
||||||
|
evaluation corpus before the staged implementation plan is approved. The
|
||||||
|
logical profile IDs and report assignments do not depend on those concrete
|
||||||
|
choices.
|
||||||
|
|
||||||
|
## Prompt And Provenance Contract
|
||||||
|
|
||||||
|
Changing a prompt's `default_profile` is a material prompt-definition change.
|
||||||
|
The four prompt definitions should advance from `1.0.1` to `1.1.0` when the new
|
||||||
|
defaults are introduced. Prompt content and generated-text schemas need not
|
||||||
|
change solely for this feature.
|
||||||
|
|
||||||
|
Prompt inspection must continue to occur before weather collection. It should
|
||||||
|
report the selected logical profile ID and the resolved backend and model
|
||||||
|
without exposing endpoints or credentials.
|
||||||
|
|
||||||
|
Preparation, execution, and failure artifacts should retain both the logical
|
||||||
|
profile identity and effective model provenance already available through the
|
||||||
|
project-owned execution contract. This feature should not replace a logical
|
||||||
|
profile ID with only the concrete model name in durable records.
|
||||||
|
|
||||||
|
## Evaluation Policy
|
||||||
|
|
||||||
|
Concrete model assignments should be evaluated with representative,
|
||||||
|
secret-free Daily, Today, Tomorrow, and Hourly data packages. Evaluation should
|
||||||
|
consider:
|
||||||
|
|
||||||
|
- strict-schema success rate;
|
||||||
|
- unsupported or invented weather claims;
|
||||||
|
- precipitation-timing accuracy and empty-string behavior;
|
||||||
|
- correct use of deterministic hazards, periods, and uncertainty;
|
||||||
|
- summary and forecast-discussion usefulness;
|
||||||
|
- generation latency;
|
||||||
|
- token use and provider cost; and
|
||||||
|
- behavior through a representative local OpenAI-compatible endpoint.
|
||||||
|
|
||||||
|
The purpose is to choose an appropriate default for each tier, not to add a
|
||||||
|
permanent benchmark framework or live-provider requirement to the ordinary
|
||||||
|
test suite. Repository tests remain offline and deterministic.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
The completed feature includes:
|
||||||
|
|
||||||
|
- Weatherreporter-owned embedded profile assets for all three logical IDs;
|
||||||
|
- Promptkit adapter wiring that supplies those assets as the application
|
||||||
|
fallback profile source;
|
||||||
|
- per-prompt default-profile assignments matching the catalog above;
|
||||||
|
- an exact prompt-version update for the changed definitions;
|
||||||
|
- preservation of the global profile override;
|
||||||
|
- same-ID override behavior through both supported external profile-source
|
||||||
|
forms;
|
||||||
|
- local-backend and endpoint-only override coverage;
|
||||||
|
- fail-fast inspection of missing, malformed, or unusable selected profiles;
|
||||||
|
- offline tests for selection, source precedence, effective model inspection,
|
||||||
|
batch reuse, and durable provenance;
|
||||||
|
- maintained operator examples for overriding `weather-light` locally; and
|
||||||
|
- updates to the canonical configuration, Promptkit integration, report
|
||||||
|
registry, operations, troubleshooting, internal adapter, and release
|
||||||
|
documentation as applicable when implementation lands.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
The feature does not include:
|
||||||
|
|
||||||
|
- automatic discovery, health checking, or benchmarking of local endpoints;
|
||||||
|
- implicit failover between local and remote profiles;
|
||||||
|
- retries with a more expensive tier after provider or validation failure;
|
||||||
|
- per-report profile configuration fields outside prompt defaults;
|
||||||
|
- profile inheritance, aliases, or field-level merging;
|
||||||
|
- runtime model selection based on weather severity, token count, or report
|
||||||
|
content;
|
||||||
|
- moving Weatherreporter profile policy into Promptkit's built-in catalog;
|
||||||
|
- exposing Promptkit types outside the adapter boundary; or
|
||||||
|
- making live provider calls part of the default repository test suite.
|
||||||
|
|
||||||
|
## Compatibility And Operational Policy
|
||||||
|
|
||||||
|
Existing configurations with a nonblank `promptkit.profile` retain their
|
||||||
|
all-report behavior. Existing `profile_file`, `profile_dir`, local-backend, and
|
||||||
|
credential configuration fields retain their meanings.
|
||||||
|
|
||||||
|
Configurations that rely on the omitted profile setting will intentionally
|
||||||
|
observe new per-report defaults. This is a user-visible model-selection and
|
||||||
|
cost change and must be called out in release notes. Operators who require the
|
||||||
|
old all-report model can preserve it by setting an explicit global profile.
|
||||||
|
|
||||||
|
An external same-ID override is an operator-owned compatibility commitment.
|
||||||
|
Weatherreporter may evolve its embedded definitions, but it must not rewrite or
|
||||||
|
silently merge an operator file.
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
|
||||||
|
The roadmap is complete when:
|
||||||
|
|
||||||
|
- a tagged Promptkit dependency supports the required fallback layer;
|
||||||
|
- every operational prompt selects its assigned logical profile at exact
|
||||||
|
version `1.1.0`;
|
||||||
|
- all three embedded profiles inspect successfully without an external profile
|
||||||
|
source;
|
||||||
|
- configured same-ID definitions override embedded definitions through both
|
||||||
|
`profile_file` and `profile_dir`;
|
||||||
|
- an invalid matching external definition fails without fallback;
|
||||||
|
- `weather-light` can resolve through an endpoint-only or configured-local
|
||||||
|
override without requiring code or prompt changes;
|
||||||
|
- global `promptkit.profile` still overrides every report in an invocation;
|
||||||
|
- prompt preparation and execution artifacts record the selected logical
|
||||||
|
profile and effective model;
|
||||||
|
- morning and evening batch preflight deduplicates inspection of shared
|
||||||
|
effective profile IDs as it does today;
|
||||||
|
- the default test suite remains offline and deterministic; and
|
||||||
|
- implemented behavior is documented by its canonical current-state owners.
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
### Exact initial model identifiers and settings
|
||||||
|
|
||||||
|
Recommendation: evaluate the current Gemma 4 31B instruction target for
|
||||||
|
`weather-light`, the maintained Gemini Flash target for `weather-balanced`, and
|
||||||
|
the maintained Gemini Pro target for `weather-deep`. Pin the exact profile
|
||||||
|
contents in this roadmap before deriving the staged implementation plan.
|
||||||
|
|
||||||
|
Alternative: duplicate three current Promptkit built-in definitions without a
|
||||||
|
Weatherreporter-specific evaluation. This is faster, but it provides less
|
||||||
|
evidence that their cost, schema reliability, and weather reasoning match the
|
||||||
|
tier policy.
|
||||||
|
|
||||||
|
### Future default use of `weather-deep`
|
||||||
|
|
||||||
|
Recommendation: ship the profile as an explicit override with no default
|
||||||
|
report assignment. Assign it later only when a report or evaluation shows a
|
||||||
|
material benefit over `weather-balanced`.
|
||||||
|
|
||||||
|
Alternative: assign Daily or Tomorrow to `weather-deep` immediately. This may
|
||||||
|
improve difficult synthesis, but it increases routine cost before the benefit
|
||||||
|
has been measured.
|
||||||
164
docs/roadmap/promptkit-fallback-profiles-feature-request.md
Normal file
164
docs/roadmap/promptkit-fallback-profiles-feature-request.md
Normal file
@@ -0,0 +1,164 @@
|
|||||||
|
# Promptkit Feature Request: Application Fallback Profiles
|
||||||
|
|
||||||
|
Status: Proposed upstream capability.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Promptkit should allow a consuming application to supply an embedded fallback
|
||||||
|
profile source that sits below operator-configured profiles and above
|
||||||
|
Promptkit's own built-in profile catalog.
|
||||||
|
|
||||||
|
This capability would let an application publish stable, domain-specific
|
||||||
|
profile IDs with useful defaults while preserving Promptkit's existing
|
||||||
|
operator-override behavior. The capability must remain application-neutral;
|
||||||
|
Promptkit should provide the source layer but should not own downstream profile
|
||||||
|
names, model assignments, or configuration policy.
|
||||||
|
|
||||||
|
## Downstream Use Case
|
||||||
|
|
||||||
|
Weatherreporter wants to embed profiles such as `weather-light`,
|
||||||
|
`weather-balanced`, and `weather-deep`. Report prompts would select those
|
||||||
|
logical profiles instead of naming provider- or model-specific Promptkit
|
||||||
|
profiles directly.
|
||||||
|
|
||||||
|
An installation could then place a profile with the same ID in its configured
|
||||||
|
profile directory. For example, a local `weather-light` definition could point
|
||||||
|
to an OpenAI-compatible endpoint on the deployment network. When no operator
|
||||||
|
definition exists, Weatherreporter's embedded definition would keep the
|
||||||
|
application usable without additional profile files.
|
||||||
|
|
||||||
|
This pattern is useful beyond Weatherreporter. Any Promptkit consumer may want
|
||||||
|
application-owned execution tiers or workload-specific defaults without
|
||||||
|
adding domain-specific profiles to Promptkit's general built-in catalog.
|
||||||
|
|
||||||
|
## Current Constraint
|
||||||
|
|
||||||
|
Promptkit currently resolves matching profile IDs in this order:
|
||||||
|
|
||||||
|
1. in-memory profiles supplied through `WithProfiles`;
|
||||||
|
2. one configured profile file, `fs.FS`, or directory source; and
|
||||||
|
3. Promptkit's embedded built-in profiles.
|
||||||
|
|
||||||
|
These layers do not express the desired application-default relationship:
|
||||||
|
|
||||||
|
- `WithProfiles` has higher precedence than the configured source, so it would
|
||||||
|
prevent an operator file from overriding an application profile with the
|
||||||
|
same ID.
|
||||||
|
- `WithProfileFS` can hold embedded application assets, but it occupies the
|
||||||
|
configured-source layer and therefore replaces rather than sits beneath a
|
||||||
|
configured profile directory or file.
|
||||||
|
- adding downstream profile IDs to Promptkit's built-in catalog would make the
|
||||||
|
library own application-specific policy.
|
||||||
|
|
||||||
|
A downstream application could build its own filesystem overlay, but that
|
||||||
|
would duplicate Promptkit's profile discovery, error, and precedence behavior
|
||||||
|
at the consumer boundary.
|
||||||
|
|
||||||
|
## Requested Capability
|
||||||
|
|
||||||
|
Add one optional application fallback profile source to engine construction.
|
||||||
|
When present, matching profile IDs should resolve in this order:
|
||||||
|
|
||||||
|
1. in-memory profiles supplied through `WithProfiles`;
|
||||||
|
2. the ordinary configured profile source selected through a profile option or
|
||||||
|
`Config.ProfileDir`;
|
||||||
|
3. the application fallback profile source; and
|
||||||
|
4. Promptkit's embedded built-in profiles.
|
||||||
|
|
||||||
|
When no application fallback is configured, existing source precedence and
|
||||||
|
behavior must remain unchanged.
|
||||||
|
|
||||||
|
The minimum useful public surface is an `fs.FS`-backed option because consumers
|
||||||
|
can embed YAML profile assets. A possible API shape is:
|
||||||
|
|
||||||
|
```go
|
||||||
|
promptkit.WithFallbackProfileFS(profileFS, ".")
|
||||||
|
```
|
||||||
|
|
||||||
|
The name is illustrative rather than prescriptive. A companion option for
|
||||||
|
validated `Profile` values could be added if Promptkit maintainers find it
|
||||||
|
generally useful, but it is not required for the Weatherreporter use case.
|
||||||
|
|
||||||
|
## Required Semantics
|
||||||
|
|
||||||
|
- A higher-precedence source falls through only when the requested profile ID
|
||||||
|
is absent.
|
||||||
|
- A malformed, unreadable, duplicate, ambiguous, or otherwise invalid matching
|
||||||
|
profile is an error and must not silently fall through.
|
||||||
|
- The fallback source uses the existing strict profile YAML format and profile
|
||||||
|
validation rules.
|
||||||
|
- Profile values are selected as a whole. This feature does not merge,
|
||||||
|
inherit, or partially overlay profile definitions.
|
||||||
|
- `InspectProfile`, `Prepare`, prepared execution, and ordinary execution use
|
||||||
|
the same profile-source precedence.
|
||||||
|
- An explicit request profile continues to take precedence over a prompt's
|
||||||
|
`default_profile`; this request concerns definition lookup after the profile
|
||||||
|
ID has been selected.
|
||||||
|
- Repeated fallback-source options should follow Promptkit's documented
|
||||||
|
same-category option convention, normally with the last value replacing the
|
||||||
|
earlier value.
|
||||||
|
- A canceled lookup, invalid fallback asset, or unknown resolved backend should
|
||||||
|
continue to cross the public facade through Promptkit's existing public error
|
||||||
|
identities.
|
||||||
|
- Exact profile inspection must remain side-effect free and must not contact a
|
||||||
|
model provider.
|
||||||
|
|
||||||
|
## Application And Library Boundaries
|
||||||
|
|
||||||
|
Promptkit should own:
|
||||||
|
|
||||||
|
- the additional repository layer;
|
||||||
|
- deterministic lookup and fallthrough behavior;
|
||||||
|
- validation of the supplied source through the existing profile contract;
|
||||||
|
- consistent use of the layer across inspection and execution; and
|
||||||
|
- public documentation and tests for the added precedence rule.
|
||||||
|
|
||||||
|
The consuming application should continue to own:
|
||||||
|
|
||||||
|
- whether it supplies fallback profiles;
|
||||||
|
- the profile IDs and their domain meaning;
|
||||||
|
- embedded profile contents and model choices;
|
||||||
|
- application configuration and override policy;
|
||||||
|
- report- or workload-to-profile assignment; and
|
||||||
|
- credential checks and operator-facing errors beyond Promptkit's public
|
||||||
|
contract.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
This request does not ask Promptkit to add:
|
||||||
|
|
||||||
|
- Weatherreporter-specific profile IDs to its built-in catalog;
|
||||||
|
- profile inheritance, aliases, or field-level merging;
|
||||||
|
- automatic endpoint discovery or availability probing;
|
||||||
|
- provider failover or fallback from a failed selected profile;
|
||||||
|
- per-request model benchmarking or tier selection;
|
||||||
|
- application configuration discovery; or
|
||||||
|
- eager validation of every profile in every source.
|
||||||
|
|
||||||
|
## Compatibility
|
||||||
|
|
||||||
|
The feature can be additive. Engines that do not configure an application
|
||||||
|
fallback source should retain their current public behavior and precedence.
|
||||||
|
Existing uses of `WithProfiles`, `WithProfileFile`, `WithProfileFS`, and
|
||||||
|
`Config.ProfileDir` should not change meaning.
|
||||||
|
|
||||||
|
The application fallback is deliberately lower precedence than every existing
|
||||||
|
consumer-configured source. This preserves the established expectation that a
|
||||||
|
custom profile definition can override a packaged default with the same ID.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
The capability is sufficient for downstream adoption when Promptkit can
|
||||||
|
demonstrate that:
|
||||||
|
|
||||||
|
- a fallback-only profile can be inspected and used for preparation and
|
||||||
|
execution;
|
||||||
|
- a configured directory, file, or `fs.FS` profile with the same ID overrides
|
||||||
|
the fallback profile;
|
||||||
|
- an absent configured profile falls through to the application fallback;
|
||||||
|
- an invalid configured match fails instead of falling through;
|
||||||
|
- an absent application fallback profile continues to resolve from Promptkit's
|
||||||
|
built-in catalog;
|
||||||
|
- `WithProfiles` retains highest precedence;
|
||||||
|
- behavior is identical across inspection, preparation, and execution; and
|
||||||
|
- omitting the new option preserves existing tests and public contracts.
|
||||||
@@ -128,8 +128,7 @@ It is not a source for deterministic weather facts.
|
|||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| `.GeneratedText.Summary` | `string` | `string` | Required. |
|
| `.GeneratedText.Summary` | `string` | `string` | Required. |
|
||||||
| `.GeneratedText.ForecastDiscussion` | `string` | `[]string` | Required; range over the day-style paragraph slice. |
|
| `.GeneratedText.ForecastDiscussion` | `string` | `[]string` | Required; range over the day-style paragraph slice. |
|
||||||
| `.GeneratedText.PrecipitationTiming` | `string` | `string` | Optional prose used by the precipitation partial when deterministic windows exist. |
|
| `.GeneratedText.PrecipitationTiming` | `string` | `string` | Required field; an empty string represents no supported prose. The precipitation partial uses nonempty prose only when deterministic windows exist. |
|
||||||
| `.GeneratedText.Confidence` | `string` | `string` | Optional validated prose; the current templates do not render it. |
|
|
||||||
|
|
||||||
The JSON schema rejects unknown properties and defines the required fields, but
|
The JSON schema rejects unknown properties and defines the required fields, but
|
||||||
the schema body and validation behavior are documented in [Generated Text
|
the schema body and validation behavior are documented in [Generated Text
|
||||||
|
|||||||
2
go.mod
2
go.mod
@@ -6,7 +6,7 @@ require gopkg.in/yaml.v3 v3.0.1
|
|||||||
|
|
||||||
require (
|
require (
|
||||||
gitea.maximumdirect.net/eric/distributor v0.5.0
|
gitea.maximumdirect.net/eric/distributor v0.5.0
|
||||||
gitea.maximumdirect.net/eric/promptkit v0.4.0
|
gitea.maximumdirect.net/eric/promptkit v0.5.0
|
||||||
)
|
)
|
||||||
|
|
||||||
require (
|
require (
|
||||||
|
|||||||
4
go.sum
4
go.sum
@@ -1,7 +1,7 @@
|
|||||||
gitea.maximumdirect.net/eric/distributor v0.5.0 h1:+al7Bw+kMv6V35a3Sm5rUtCTQhwOn5b9x3RsclPMKJk=
|
gitea.maximumdirect.net/eric/distributor v0.5.0 h1:+al7Bw+kMv6V35a3Sm5rUtCTQhwOn5b9x3RsclPMKJk=
|
||||||
gitea.maximumdirect.net/eric/distributor v0.5.0/go.mod h1:G03FCFZPHpsUKC6SeMgTdbfNRpPQBdyTtDUj04e1Tu8=
|
gitea.maximumdirect.net/eric/distributor v0.5.0/go.mod h1:G03FCFZPHpsUKC6SeMgTdbfNRpPQBdyTtDUj04e1Tu8=
|
||||||
gitea.maximumdirect.net/eric/promptkit v0.4.0 h1:WHRQEt3BVBAR7hQePBaGtNXpzrs59mlr/42nQzwgOz4=
|
gitea.maximumdirect.net/eric/promptkit v0.5.0 h1:jnpazLyyNhWrB2xzwwtUkNUfktkTdkENTwuSPnKiYrc=
|
||||||
gitea.maximumdirect.net/eric/promptkit v0.4.0/go.mod h1:R95NM6fbMDGDC0/UomgnSBP6ui2ns+8SZb8bESNvrDQ=
|
gitea.maximumdirect.net/eric/promptkit v0.5.0/go.mod h1:R95NM6fbMDGDC0/UomgnSBP6ui2ns+8SZb8bESNvrDQ=
|
||||||
github.com/aws/aws-sdk-go-v2 v1.41.9 h1:/rYeyO2+HrMztAmxAq9++XJtFMqSIpSsNA0yDGALYq4=
|
github.com/aws/aws-sdk-go-v2 v1.41.9 h1:/rYeyO2+HrMztAmxAq9++XJtFMqSIpSsNA0yDGALYq4=
|
||||||
github.com/aws/aws-sdk-go-v2 v1.41.9/go.mod h1:+HsoOEX80qAVUitj1A2DhCNTjmb3edVyuDypb6LNEeo=
|
github.com/aws/aws-sdk-go-v2 v1.41.9/go.mod h1:+HsoOEX80qAVUitj1A2DhCNTjmb3edVyuDypb6LNEeo=
|
||||||
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11 h1:h5+3VT69KUBK24grGuuA5saDJTj2IIjLb9au668Fo5I=
|
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11 h1:h5+3VT69KUBK24grGuuA5saDJTj2IIjLb9au668Fo5I=
|
||||||
|
|||||||
@@ -68,11 +68,11 @@ func (client *fakeClient) request() promptkit.GenerateRequest {
|
|||||||
|
|
||||||
func TestInspectPromptAndProfile(t *testing.T) {
|
func TestInspectPromptAndProfile(t *testing.T) {
|
||||||
adapter := newTestAdapter(t, &fakeClient{})
|
adapter := newTestAdapter(t, &fakeClient{})
|
||||||
inspection, err := adapter.InspectPrompt(context.Background(), "weather.daily_generated_text", "1.0.0")
|
inspection, err := adapter.InspectPrompt(context.Background(), "weather.daily_generated_text", "1.0.1")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("InspectPrompt() error = %v", err)
|
t.Fatalf("InspectPrompt() error = %v", err)
|
||||||
}
|
}
|
||||||
if inspection.PromptID != "weather.daily_generated_text" || inspection.PromptVersion != "1.0.0" || inspection.DefaultProfileID != "gemini-flash-latest" {
|
if inspection.PromptID != "weather.daily_generated_text" || inspection.PromptVersion != "1.0.1" || inspection.DefaultProfileID != "gemini-flash-latest" {
|
||||||
t.Fatalf("inspection = %#v", inspection)
|
t.Fatalf("inspection = %#v", inspection)
|
||||||
}
|
}
|
||||||
if len(inspection.Inputs) != 1 || inspection.Inputs[0].Name != "data_package" || !inspection.Inputs[0].Required || inspection.Inputs[0].ContentType != "application/yaml" {
|
if len(inspection.Inputs) != 1 || inspection.Inputs[0].Name != "data_package" || !inspection.Inputs[0].Required || inspection.Inputs[0].ContentType != "application/yaml" {
|
||||||
@@ -369,7 +369,7 @@ func testProfileDirectory(t *testing.T, profile string) string {
|
|||||||
func testExecuteRequest() promptexec.ExecuteRequest {
|
func testExecuteRequest() promptexec.ExecuteRequest {
|
||||||
return promptexec.ExecuteRequest{
|
return promptexec.ExecuteRequest{
|
||||||
PromptID: "weather.daily_generated_text",
|
PromptID: "weather.daily_generated_text",
|
||||||
PromptVersion: "1.0.0",
|
PromptVersion: "1.0.1",
|
||||||
ProfileID: "test-profile",
|
ProfileID: "test-profile",
|
||||||
DataPackage: []byte("report:\n id: daily\nbriefing: {}\n"),
|
DataPackage: []byte("report:\n id: daily\nbriefing: {}\n"),
|
||||||
DataPackagePath: "data-packages/daily/data_package.yaml",
|
DataPackagePath: "data-packages/daily/data_package.yaml",
|
||||||
@@ -378,7 +378,7 @@ func testExecuteRequest() promptexec.ExecuteRequest {
|
|||||||
|
|
||||||
func validResponse() *promptkit.GenerateResponse {
|
func validResponse() *promptkit.GenerateResponse {
|
||||||
return &promptkit.GenerateResponse{
|
return &promptkit.GenerateResponse{
|
||||||
Content: `{"summary":"A quiet day is expected.","forecast_discussion":["High pressure keeps conditions settled."],"confidence":"High."}`,
|
Content: `{"summary":"A quiet day is expected.","forecast_discussion":["High pressure keeps conditions settled."],"precipitation_timing":""}`,
|
||||||
Usage: promptkit.TokenUsage{PromptTokens: 12, CompletionTokens: 8, TotalTokens: 20},
|
Usage: promptkit.TokenUsage{PromptTokens: 12, CompletionTokens: 8, TotalTokens: 20},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -119,7 +119,7 @@ func (e artifactPathExecutor) Execute(_ context.Context, req promptexec.ExecuteR
|
|||||||
PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash", ProfileID: req.ProfileID,
|
PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash", ProfileID: req.ProfileID,
|
||||||
BackendID: "test", ModelName: "test-model", GeneratedHash: "generated-hash",
|
BackendID: "test", ModelName: "test-model", GeneratedHash: "generated-hash",
|
||||||
StartedAt: now, EndedAt: now, DataPackagePath: req.DataPackagePath,
|
StartedAt: now, EndedAt: now, DataPackagePath: req.DataPackagePath,
|
||||||
RawOutput: []byte(`{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast."],"precipitation_timing":"Rain is most likely during the afternoon.","confidence":"Medium"}`),
|
RawOutput: []byte(`{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast."],"precipitation_timing":"Rain is most likely during the afternoon."}`),
|
||||||
Validation: promptexec.NewValidation(validation, "json_schema", "daily.generated_text.schema.json", nil),
|
Validation: promptexec.NewValidation(validation, "json_schema", "daily.generated_text.schema.json", nil),
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -643,7 +643,7 @@ func workflowTime(value string) time.Time {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func validHourlyWorkflowJSON() string {
|
func validHourlyWorkflowJSON() string {
|
||||||
return `{"summary":"Storm chances increase through late morning.","forecast_discussion":"A front will keep the region unsettled.","precipitation_timing":"A cold front is moving into the region.","confidence":"Medium"}`
|
return `{"summary":"Storm chances increase through late morning.","forecast_discussion":"A front will keep the region unsettled.","precipitation_timing":"A cold front is moving into the region."}`
|
||||||
}
|
}
|
||||||
|
|
||||||
func validTomorrowWorkflowJSON() string {
|
func validTomorrowWorkflowJSON() string {
|
||||||
@@ -655,5 +655,5 @@ func validTodayWorkflowJSON() string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func validDailyWorkflowJSON() string {
|
func validDailyWorkflowJSON() string {
|
||||||
return `{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast.","Temperatures stay seasonable by afternoon."],"precipitation_timing":"Rain is most likely during the afternoon.","confidence":"Medium"}`
|
return `{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast.","Temperatures stay seasonable by afternoon."],"precipitation_timing":"Rain is most likely during the afternoon."}`
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -147,7 +147,8 @@ func TestCatalogValidationDispatchSupportsKnownSchemas(t *testing.T) {
|
|||||||
}
|
}
|
||||||
hourly, normalized, err := hourlyHandler.Validate([]byte(`{
|
hourly, normalized, err := hourlyHandler.Validate([]byte(`{
|
||||||
"summary": " Storm chances increase. ",
|
"summary": " Storm chances increase. ",
|
||||||
"forecast_discussion": " A front will keep the region unsettled. "
|
"forecast_discussion": " A front will keep the region unsettled. ",
|
||||||
|
"precipitation_timing": ""
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Validate(hourly) error = %v", err)
|
t.Fatalf("Validate(hourly) error = %v", err)
|
||||||
@@ -165,7 +166,8 @@ func TestCatalogValidationDispatchSupportsKnownSchemas(t *testing.T) {
|
|||||||
}
|
}
|
||||||
tomorrow, normalized, err := tomorrowHandler.Validate([]byte(`{
|
tomorrow, normalized, err := tomorrowHandler.Validate([]byte(`{
|
||||||
"summary": " Storms become more likely tomorrow. ",
|
"summary": " Storms become more likely tomorrow. ",
|
||||||
"forecast_discussion": [" A front will keep showers in the forecast. ", ""]
|
"forecast_discussion": [" A front will keep showers in the forecast. ", ""],
|
||||||
|
"precipitation_timing": ""
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Validate(tomorrow) error = %v", err)
|
t.Fatalf("Validate(tomorrow) error = %v", err)
|
||||||
@@ -187,7 +189,8 @@ func TestCatalogValidationDispatchSupportsKnownSchemas(t *testing.T) {
|
|||||||
}
|
}
|
||||||
today, normalized, err := todayHandler.Validate([]byte(`{
|
today, normalized, err := todayHandler.Validate([]byte(`{
|
||||||
"summary": " Showers are likely today. ",
|
"summary": " Showers are likely today. ",
|
||||||
"forecast_discussion": [" A front will keep rain chances elevated. ", ""]
|
"forecast_discussion": [" A front will keep rain chances elevated. ", ""],
|
||||||
|
"precipitation_timing": ""
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Validate(today) error = %v", err)
|
t.Fatalf("Validate(today) error = %v", err)
|
||||||
@@ -209,7 +212,8 @@ func TestCatalogValidationDispatchSupportsKnownSchemas(t *testing.T) {
|
|||||||
}
|
}
|
||||||
daily, normalized, err := dailyHandler.Validate([]byte(`{
|
daily, normalized, err := dailyHandler.Validate([]byte(`{
|
||||||
"summary": " Showers are possible during the selected day. ",
|
"summary": " Showers are possible during the selected day. ",
|
||||||
"forecast_discussion": [" A front will keep rain chances in the forecast. ", ""]
|
"forecast_discussion": [" A front will keep rain chances in the forecast. ", ""],
|
||||||
|
"precipitation_timing": ""
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Validate(daily) error = %v", err)
|
t.Fatalf("Validate(daily) error = %v", err)
|
||||||
|
|||||||
@@ -3,8 +3,7 @@ package generatedtext
|
|||||||
type Daily struct {
|
type Daily struct {
|
||||||
Summary string `json:"summary"`
|
Summary string `json:"summary"`
|
||||||
ForecastDiscussion []string `json:"forecast_discussion"`
|
ForecastDiscussion []string `json:"forecast_discussion"`
|
||||||
PrecipitationTiming string `json:"precipitation_timing,omitempty"`
|
PrecipitationTiming string `json:"precipitation_timing"`
|
||||||
Confidence string `json:"confidence,omitempty"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func ValidateDaily(data []byte) (Daily, []byte, error) {
|
func ValidateDaily(data []byte) (Daily, []byte, error) {
|
||||||
@@ -16,7 +15,6 @@ func (d *Daily) dayStyleFields() dayStyleFields {
|
|||||||
Summary: d.Summary,
|
Summary: d.Summary,
|
||||||
ForecastDiscussion: d.ForecastDiscussion,
|
ForecastDiscussion: d.ForecastDiscussion,
|
||||||
PrecipitationTiming: d.PrecipitationTiming,
|
PrecipitationTiming: d.PrecipitationTiming,
|
||||||
Confidence: d.Confidence,
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -24,5 +22,4 @@ func (d *Daily) setDayStyleFields(fields dayStyleFields) {
|
|||||||
d.Summary = fields.Summary
|
d.Summary = fields.Summary
|
||||||
d.ForecastDiscussion = fields.ForecastDiscussion
|
d.ForecastDiscussion = fields.ForecastDiscussion
|
||||||
d.PrecipitationTiming = fields.PrecipitationTiming
|
d.PrecipitationTiming = fields.PrecipitationTiming
|
||||||
d.Confidence = fields.Confidence
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -13,8 +13,7 @@ func TestValidateDailyNormalizesJSON(t *testing.T) {
|
|||||||
"",
|
"",
|
||||||
" Temperatures stay seasonable by afternoon. "
|
" Temperatures stay seasonable by afternoon. "
|
||||||
],
|
],
|
||||||
"precipitation_timing": " Rain is most likely during the afternoon. ",
|
"precipitation_timing": " Rain is most likely during the afternoon. "
|
||||||
"confidence": " Medium "
|
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ValidateDaily() error = %v", err)
|
t.Fatalf("ValidateDaily() error = %v", err)
|
||||||
@@ -28,23 +27,22 @@ func TestValidateDailyNormalizesJSON(t *testing.T) {
|
|||||||
if value.PrecipitationTiming != "Rain is most likely during the afternoon." {
|
if value.PrecipitationTiming != "Rain is most likely during the afternoon." {
|
||||||
t.Fatalf("PrecipitationTiming = %q, want trimmed precipitation timing", value.PrecipitationTiming)
|
t.Fatalf("PrecipitationTiming = %q, want trimmed precipitation timing", value.PrecipitationTiming)
|
||||||
}
|
}
|
||||||
want := `{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast.","Temperatures stay seasonable by afternoon."],"precipitation_timing":"Rain is most likely during the afternoon.","confidence":"Medium"}`
|
want := `{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast.","Temperatures stay seasonable by afternoon."],"precipitation_timing":"Rain is most likely during the afternoon."}`
|
||||||
if string(normalized) != want {
|
if string(normalized) != want {
|
||||||
t.Fatalf("normalized = %s, want %s", normalized, want)
|
t.Fatalf("normalized = %s, want %s", normalized, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestValidateDailyOmitsEmptyOptionalFields(t *testing.T) {
|
func TestValidateDailyPreservesRequiredEmptyPrecipitationTiming(t *testing.T) {
|
||||||
_, normalized, err := ValidateDaily([]byte(`{
|
_, normalized, err := ValidateDaily([]byte(`{
|
||||||
"summary": "Showers are possible during the selected day.",
|
"summary": "Showers are possible during the selected day.",
|
||||||
"forecast_discussion": ["A front will keep rain chances in the forecast."],
|
"forecast_discussion": ["A front will keep rain chances in the forecast."],
|
||||||
"precipitation_timing": " ",
|
"precipitation_timing": " "
|
||||||
"confidence": " "
|
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ValidateDaily() error = %v", err)
|
t.Fatalf("ValidateDaily() error = %v", err)
|
||||||
}
|
}
|
||||||
want := `{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast."]}`
|
want := `{"summary":"Showers are possible during the selected day.","forecast_discussion":["A front will keep rain chances in the forecast."],"precipitation_timing":""}`
|
||||||
if string(normalized) != want {
|
if string(normalized) != want {
|
||||||
t.Fatalf("normalized = %s, want %s", normalized, want)
|
t.Fatalf("normalized = %s, want %s", normalized, want)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -9,7 +9,6 @@ type dayStyleFields struct {
|
|||||||
Summary string
|
Summary string
|
||||||
ForecastDiscussion []string
|
ForecastDiscussion []string
|
||||||
PrecipitationTiming string
|
PrecipitationTiming string
|
||||||
Confidence string
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type dayStyleGeneratedText interface {
|
type dayStyleGeneratedText interface {
|
||||||
@@ -31,7 +30,6 @@ func validateDayStyleGeneratedText[T any, PT interface {
|
|||||||
fields := pointer.dayStyleFields()
|
fields := pointer.dayStyleFields()
|
||||||
fields.Summary = strings.TrimSpace(fields.Summary)
|
fields.Summary = strings.TrimSpace(fields.Summary)
|
||||||
fields.PrecipitationTiming = strings.TrimSpace(fields.PrecipitationTiming)
|
fields.PrecipitationTiming = strings.TrimSpace(fields.PrecipitationTiming)
|
||||||
fields.Confidence = strings.TrimSpace(fields.Confidence)
|
|
||||||
fields.ForecastDiscussion = trimNonEmpty(fields.ForecastDiscussion)
|
fields.ForecastDiscussion = trimNonEmpty(fields.ForecastDiscussion)
|
||||||
if fields.Summary == "" {
|
if fields.Summary == "" {
|
||||||
var zero T
|
var zero T
|
||||||
@@ -41,6 +39,10 @@ func validateDayStyleGeneratedText[T any, PT interface {
|
|||||||
var zero T
|
var zero T
|
||||||
return zero, nil, fmt.Errorf("%s generated text forecast discussion is required", name)
|
return zero, nil, fmt.Errorf("%s generated text forecast discussion is required", name)
|
||||||
}
|
}
|
||||||
|
if err := requireGeneratedTextStringField(data, name, "precipitation_timing"); err != nil {
|
||||||
|
var zero T
|
||||||
|
return zero, nil, err
|
||||||
|
}
|
||||||
pointer.setDayStyleFields(fields)
|
pointer.setDayStyleFields(fields)
|
||||||
|
|
||||||
normalized, err := normalizeGeneratedText(value, name)
|
normalized, err := normalizeGeneratedText(value, name)
|
||||||
|
|||||||
@@ -60,8 +60,7 @@ func TestValidateDayStyleGeneratedTextSharedBehavior(t *testing.T) {
|
|||||||
"",
|
"",
|
||||||
" Second paragraph. "
|
" Second paragraph. "
|
||||||
],
|
],
|
||||||
"precipitation_timing": " Afternoon. ",
|
"precipitation_timing": " Afternoon. "
|
||||||
"confidence": " Medium "
|
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("validate() error = %v", err)
|
t.Fatalf("validate() error = %v", err)
|
||||||
@@ -76,31 +75,35 @@ func TestValidateDayStyleGeneratedTextSharedBehavior(t *testing.T) {
|
|||||||
if fields.PrecipitationTiming != "Afternoon." {
|
if fields.PrecipitationTiming != "Afternoon." {
|
||||||
t.Fatalf("PrecipitationTiming = %q, want trimmed precipitation timing", fields.PrecipitationTiming)
|
t.Fatalf("PrecipitationTiming = %q, want trimmed precipitation timing", fields.PrecipitationTiming)
|
||||||
}
|
}
|
||||||
if fields.Confidence != "Medium" {
|
want := `{"summary":"Shared summary.","forecast_discussion":["First paragraph.","Second paragraph."],"precipitation_timing":"Afternoon."}`
|
||||||
t.Fatalf("Confidence = %q, want trimmed confidence", fields.Confidence)
|
|
||||||
}
|
|
||||||
want := `{"summary":"Shared summary.","forecast_discussion":["First paragraph.","Second paragraph."],"precipitation_timing":"Afternoon.","confidence":"Medium"}`
|
|
||||||
if string(normalized) != want {
|
if string(normalized) != want {
|
||||||
t.Fatalf("normalized = %s, want %s", normalized, want)
|
t.Fatalf("normalized = %s, want %s", normalized, want)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
t.Run("omits empty optional fields", func(t *testing.T) {
|
t.Run("preserves required empty precipitation timing", func(t *testing.T) {
|
||||||
_, normalized, err := report.validate([]byte(`{
|
_, normalized, err := report.validate([]byte(`{
|
||||||
"summary": "Shared summary.",
|
"summary": "Shared summary.",
|
||||||
"forecast_discussion": ["First paragraph."],
|
"forecast_discussion": ["First paragraph."],
|
||||||
"precipitation_timing": " ",
|
"precipitation_timing": " "
|
||||||
"confidence": " "
|
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("validate() error = %v", err)
|
t.Fatalf("validate() error = %v", err)
|
||||||
}
|
}
|
||||||
want := `{"summary":"Shared summary.","forecast_discussion":["First paragraph."]}`
|
want := `{"summary":"Shared summary.","forecast_discussion":["First paragraph."],"precipitation_timing":""}`
|
||||||
if string(normalized) != want {
|
if string(normalized) != want {
|
||||||
t.Fatalf("normalized = %s, want %s", normalized, want)
|
t.Fatalf("normalized = %s, want %s", normalized, want)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
t.Run("requires precipitation timing field", func(t *testing.T) {
|
||||||
|
_, _, err := report.validate([]byte(`{"summary":"Shared summary.","forecast_discussion":["First paragraph."]}`))
|
||||||
|
want := fmt.Sprintf("%s generated text precipitation timing is required", report.name)
|
||||||
|
if err == nil || err.Error() != want {
|
||||||
|
t.Fatalf("validate() error = %v, want %q", err, want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
t.Run("rejects unknown fields", func(t *testing.T) {
|
t.Run("rejects unknown fields", func(t *testing.T) {
|
||||||
_, _, err := report.validate([]byte(`{"summary":"Shared summary.","forecast_discussion":["First paragraph."],"extra":"value"}`))
|
_, _, err := report.validate([]byte(`{"summary":"Shared summary.","forecast_discussion":["First paragraph."],"extra":"value"}`))
|
||||||
if err == nil {
|
if err == nil {
|
||||||
@@ -110,6 +113,13 @@ func TestValidateDayStyleGeneratedTextSharedBehavior(t *testing.T) {
|
|||||||
t.Fatalf("validate() error = %v, want unknown field error", err)
|
t.Fatalf("validate() error = %v, want unknown field error", err)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
t.Run("rejects retired confidence field", func(t *testing.T) {
|
||||||
|
_, _, err := report.validate([]byte(`{"summary":"Shared summary.","forecast_discussion":["First paragraph."],"precipitation_timing":"","confidence":"Medium"}`))
|
||||||
|
if err == nil || !strings.Contains(err.Error(), `unknown field "confidence"`) {
|
||||||
|
t.Fatalf("validate() error = %v, want retired confidence field rejection", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -9,8 +9,7 @@ import (
|
|||||||
type Hourly struct {
|
type Hourly struct {
|
||||||
Summary string `json:"summary"`
|
Summary string `json:"summary"`
|
||||||
ForecastDiscussion string `json:"forecast_discussion"`
|
ForecastDiscussion string `json:"forecast_discussion"`
|
||||||
PrecipitationTiming string `json:"precipitation_timing,omitempty"`
|
PrecipitationTiming string `json:"precipitation_timing"`
|
||||||
Confidence string `json:"confidence,omitempty"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func ValidateHourly(data []byte) (Hourly, []byte, error) {
|
func ValidateHourly(data []byte) (Hourly, []byte, error) {
|
||||||
@@ -22,13 +21,15 @@ func ValidateHourly(data []byte) (Hourly, []byte, error) {
|
|||||||
value.Summary = strings.TrimSpace(value.Summary)
|
value.Summary = strings.TrimSpace(value.Summary)
|
||||||
value.ForecastDiscussion = strings.TrimSpace(value.ForecastDiscussion)
|
value.ForecastDiscussion = strings.TrimSpace(value.ForecastDiscussion)
|
||||||
value.PrecipitationTiming = strings.TrimSpace(value.PrecipitationTiming)
|
value.PrecipitationTiming = strings.TrimSpace(value.PrecipitationTiming)
|
||||||
value.Confidence = strings.TrimSpace(value.Confidence)
|
|
||||||
if value.Summary == "" {
|
if value.Summary == "" {
|
||||||
return Hourly{}, nil, fmt.Errorf("hourly generated text summary is required")
|
return Hourly{}, nil, fmt.Errorf("hourly generated text summary is required")
|
||||||
}
|
}
|
||||||
if value.ForecastDiscussion == "" {
|
if value.ForecastDiscussion == "" {
|
||||||
return Hourly{}, nil, fmt.Errorf("hourly generated text forecast discussion is required")
|
return Hourly{}, nil, fmt.Errorf("hourly generated text forecast discussion is required")
|
||||||
}
|
}
|
||||||
|
if err := requireGeneratedTextStringField(data, "hourly", "precipitation_timing"); err != nil {
|
||||||
|
return Hourly{}, nil, err
|
||||||
|
}
|
||||||
|
|
||||||
normalized, err := normalizeGeneratedText(value, "hourly")
|
normalized, err := normalizeGeneratedText(value, "hourly")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -9,8 +9,7 @@ func TestValidateHourlyNormalizesJSON(t *testing.T) {
|
|||||||
value, normalized, err := ValidateHourly([]byte(`{
|
value, normalized, err := ValidateHourly([]byte(`{
|
||||||
"summary": " Storm chances increase. ",
|
"summary": " Storm chances increase. ",
|
||||||
"forecast_discussion": " A front will keep the region unsettled. ",
|
"forecast_discussion": " A front will keep the region unsettled. ",
|
||||||
"precipitation_timing": " Showers are most likely early this afternoon. ",
|
"precipitation_timing": " Showers are most likely early this afternoon. "
|
||||||
"confidence": " Medium "
|
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ValidateHourly() error = %v", err)
|
t.Fatalf("ValidateHourly() error = %v", err)
|
||||||
@@ -24,23 +23,22 @@ func TestValidateHourlyNormalizesJSON(t *testing.T) {
|
|||||||
if value.PrecipitationTiming != "Showers are most likely early this afternoon." {
|
if value.PrecipitationTiming != "Showers are most likely early this afternoon." {
|
||||||
t.Fatalf("PrecipitationTiming = %q, want trimmed precipitation timing", value.PrecipitationTiming)
|
t.Fatalf("PrecipitationTiming = %q, want trimmed precipitation timing", value.PrecipitationTiming)
|
||||||
}
|
}
|
||||||
want := `{"summary":"Storm chances increase.","forecast_discussion":"A front will keep the region unsettled.","precipitation_timing":"Showers are most likely early this afternoon.","confidence":"Medium"}`
|
want := `{"summary":"Storm chances increase.","forecast_discussion":"A front will keep the region unsettled.","precipitation_timing":"Showers are most likely early this afternoon."}`
|
||||||
if string(normalized) != want {
|
if string(normalized) != want {
|
||||||
t.Fatalf("normalized = %s, want %s", normalized, want)
|
t.Fatalf("normalized = %s, want %s", normalized, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestValidateHourlyOmitsEmptyConfidence(t *testing.T) {
|
func TestValidateHourlyPreservesRequiredEmptyPrecipitationTiming(t *testing.T) {
|
||||||
_, normalized, err := ValidateHourly([]byte(`{
|
_, normalized, err := ValidateHourly([]byte(`{
|
||||||
"summary": "Storm chances increase.",
|
"summary": "Storm chances increase.",
|
||||||
"forecast_discussion": "A front will keep the region unsettled.",
|
"forecast_discussion": "A front will keep the region unsettled.",
|
||||||
"precipitation_timing": " ",
|
"precipitation_timing": " "
|
||||||
"confidence": " "
|
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ValidateHourly() error = %v", err)
|
t.Fatalf("ValidateHourly() error = %v", err)
|
||||||
}
|
}
|
||||||
want := `{"summary":"Storm chances increase.","forecast_discussion":"A front will keep the region unsettled."}`
|
want := `{"summary":"Storm chances increase.","forecast_discussion":"A front will keep the region unsettled.","precipitation_timing":""}`
|
||||||
if string(normalized) != want {
|
if string(normalized) != want {
|
||||||
t.Fatalf("normalized = %s, want %s", normalized, want)
|
t.Fatalf("normalized = %s, want %s", normalized, want)
|
||||||
}
|
}
|
||||||
@@ -72,6 +70,21 @@ func TestValidateHourlyRejectsInvalidInput(t *testing.T) {
|
|||||||
in: `{"summary":"Storm chances increase.","forecast_discussion":" "}`,
|
in: `{"summary":"Storm chances increase.","forecast_discussion":" "}`,
|
||||||
want: "forecast discussion is required",
|
want: "forecast discussion is required",
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
name: "missing precipitation timing",
|
||||||
|
in: `{"summary":"Storm chances increase.","forecast_discussion":"A front will keep the region unsettled."}`,
|
||||||
|
want: "precipitation timing is required",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "null precipitation timing",
|
||||||
|
in: `{"summary":"Storm chances increase.","forecast_discussion":"A front will keep the region unsettled.","precipitation_timing":null}`,
|
||||||
|
want: "precipitation timing must be a string",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "retired confidence field rejected",
|
||||||
|
in: `{"summary":"Storm chances increase.","forecast_discussion":"A front will keep the region unsettled.","precipitation_timing":"","confidence":"Medium"}`,
|
||||||
|
want: `unknown field "confidence"`,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
name: "old timing field rejected",
|
name: "old timing field rejected",
|
||||||
in: `{"summary":"Storm chances increase.","forecast_discussion":"A front will keep the region unsettled.","timing":"Late morning."}`,
|
in: `{"summary":"Storm chances increase.","forecast_discussion":"A front will keep the region unsettled.","timing":"Late morning."}`,
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import (
|
|||||||
"encoding/json"
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"strings"
|
||||||
)
|
)
|
||||||
|
|
||||||
func decodeGeneratedText[T any](data []byte, name string) (T, error) {
|
func decodeGeneratedText[T any](data []byte, name string) (T, error) {
|
||||||
@@ -24,6 +25,21 @@ func decodeGeneratedText[T any](data []byte, name string) (T, error) {
|
|||||||
return value, fmt.Errorf("decode %s generated text: multiple JSON values", name)
|
return value, fmt.Errorf("decode %s generated text: multiple JSON values", name)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func requireGeneratedTextStringField(data []byte, name, field string) error {
|
||||||
|
var fields map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(data, &fields); err != nil {
|
||||||
|
return fmt.Errorf("decode %s generated text: %w", name, err)
|
||||||
|
}
|
||||||
|
raw, ok := fields[field]
|
||||||
|
if !ok {
|
||||||
|
return fmt.Errorf("%s generated text %s is required", name, strings.ReplaceAll(field, "_", " "))
|
||||||
|
}
|
||||||
|
if bytes.Equal(bytes.TrimSpace(raw), []byte("null")) {
|
||||||
|
return fmt.Errorf("%s generated text %s must be a string", name, strings.ReplaceAll(field, "_", " "))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
func normalizeGeneratedText[T any](value T, name string) ([]byte, error) {
|
func normalizeGeneratedText[T any](value T, name string) ([]byte, error) {
|
||||||
normalized, err := json.Marshal(value)
|
normalized, err := json.Marshal(value)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -21,7 +21,6 @@ func TestBuildHourlyRenderContext(t *testing.T) {
|
|||||||
Summary: "Storm chances increase through late morning.",
|
Summary: "Storm chances increase through late morning.",
|
||||||
ForecastDiscussion: "A front will keep the region unsettled.",
|
ForecastDiscussion: "A front will keep the region unsettled.",
|
||||||
PrecipitationTiming: "A cold front is moving into the region.",
|
PrecipitationTiming: "A cold front is moving into the region.",
|
||||||
Confidence: "Medium confidence in timing.",
|
|
||||||
}
|
}
|
||||||
collected := testCollected()
|
collected := testCollected()
|
||||||
derived := testDerived()
|
derived := testDerived()
|
||||||
|
|||||||
@@ -3,8 +3,7 @@ package generatedtext
|
|||||||
type Today struct {
|
type Today struct {
|
||||||
Summary string `json:"summary"`
|
Summary string `json:"summary"`
|
||||||
ForecastDiscussion []string `json:"forecast_discussion"`
|
ForecastDiscussion []string `json:"forecast_discussion"`
|
||||||
PrecipitationTiming string `json:"precipitation_timing,omitempty"`
|
PrecipitationTiming string `json:"precipitation_timing"`
|
||||||
Confidence string `json:"confidence,omitempty"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func ValidateToday(data []byte) (Today, []byte, error) {
|
func ValidateToday(data []byte) (Today, []byte, error) {
|
||||||
@@ -16,7 +15,6 @@ func (t *Today) dayStyleFields() dayStyleFields {
|
|||||||
Summary: t.Summary,
|
Summary: t.Summary,
|
||||||
ForecastDiscussion: t.ForecastDiscussion,
|
ForecastDiscussion: t.ForecastDiscussion,
|
||||||
PrecipitationTiming: t.PrecipitationTiming,
|
PrecipitationTiming: t.PrecipitationTiming,
|
||||||
Confidence: t.Confidence,
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -24,5 +22,4 @@ func (t *Today) setDayStyleFields(fields dayStyleFields) {
|
|||||||
t.Summary = fields.Summary
|
t.Summary = fields.Summary
|
||||||
t.ForecastDiscussion = fields.ForecastDiscussion
|
t.ForecastDiscussion = fields.ForecastDiscussion
|
||||||
t.PrecipitationTiming = fields.PrecipitationTiming
|
t.PrecipitationTiming = fields.PrecipitationTiming
|
||||||
t.Confidence = fields.Confidence
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -13,8 +13,7 @@ func TestValidateTodayNormalizesJSON(t *testing.T) {
|
|||||||
"",
|
"",
|
||||||
" Temperatures stay mild through the afternoon. "
|
" Temperatures stay mild through the afternoon. "
|
||||||
],
|
],
|
||||||
"precipitation_timing": " Rain is most likely during the afternoon. ",
|
"precipitation_timing": " Rain is most likely during the afternoon. "
|
||||||
"confidence": " Medium "
|
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ValidateToday() error = %v", err)
|
t.Fatalf("ValidateToday() error = %v", err)
|
||||||
@@ -28,23 +27,22 @@ func TestValidateTodayNormalizesJSON(t *testing.T) {
|
|||||||
if value.PrecipitationTiming != "Rain is most likely during the afternoon." {
|
if value.PrecipitationTiming != "Rain is most likely during the afternoon." {
|
||||||
t.Fatalf("PrecipitationTiming = %q, want trimmed precipitation timing", value.PrecipitationTiming)
|
t.Fatalf("PrecipitationTiming = %q, want trimmed precipitation timing", value.PrecipitationTiming)
|
||||||
}
|
}
|
||||||
want := `{"summary":"Showers are likely today.","forecast_discussion":["A front will keep rain chances elevated.","Temperatures stay mild through the afternoon."],"precipitation_timing":"Rain is most likely during the afternoon.","confidence":"Medium"}`
|
want := `{"summary":"Showers are likely today.","forecast_discussion":["A front will keep rain chances elevated.","Temperatures stay mild through the afternoon."],"precipitation_timing":"Rain is most likely during the afternoon."}`
|
||||||
if string(normalized) != want {
|
if string(normalized) != want {
|
||||||
t.Fatalf("normalized = %s, want %s", normalized, want)
|
t.Fatalf("normalized = %s, want %s", normalized, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestValidateTodayOmitsEmptyOptionalFields(t *testing.T) {
|
func TestValidateTodayPreservesRequiredEmptyPrecipitationTiming(t *testing.T) {
|
||||||
_, normalized, err := ValidateToday([]byte(`{
|
_, normalized, err := ValidateToday([]byte(`{
|
||||||
"summary": "Showers are likely today.",
|
"summary": "Showers are likely today.",
|
||||||
"forecast_discussion": ["A front will keep rain chances elevated."],
|
"forecast_discussion": ["A front will keep rain chances elevated."],
|
||||||
"precipitation_timing": " ",
|
"precipitation_timing": " "
|
||||||
"confidence": " "
|
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ValidateToday() error = %v", err)
|
t.Fatalf("ValidateToday() error = %v", err)
|
||||||
}
|
}
|
||||||
want := `{"summary":"Showers are likely today.","forecast_discussion":["A front will keep rain chances elevated."]}`
|
want := `{"summary":"Showers are likely today.","forecast_discussion":["A front will keep rain chances elevated."],"precipitation_timing":""}`
|
||||||
if string(normalized) != want {
|
if string(normalized) != want {
|
||||||
t.Fatalf("normalized = %s, want %s", normalized, want)
|
t.Fatalf("normalized = %s, want %s", normalized, want)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,8 +3,7 @@ package generatedtext
|
|||||||
type Tomorrow struct {
|
type Tomorrow struct {
|
||||||
Summary string `json:"summary"`
|
Summary string `json:"summary"`
|
||||||
ForecastDiscussion []string `json:"forecast_discussion"`
|
ForecastDiscussion []string `json:"forecast_discussion"`
|
||||||
PrecipitationTiming string `json:"precipitation_timing,omitempty"`
|
PrecipitationTiming string `json:"precipitation_timing"`
|
||||||
Confidence string `json:"confidence,omitempty"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func ValidateTomorrow(data []byte) (Tomorrow, []byte, error) {
|
func ValidateTomorrow(data []byte) (Tomorrow, []byte, error) {
|
||||||
@@ -16,7 +15,6 @@ func (t *Tomorrow) dayStyleFields() dayStyleFields {
|
|||||||
Summary: t.Summary,
|
Summary: t.Summary,
|
||||||
ForecastDiscussion: t.ForecastDiscussion,
|
ForecastDiscussion: t.ForecastDiscussion,
|
||||||
PrecipitationTiming: t.PrecipitationTiming,
|
PrecipitationTiming: t.PrecipitationTiming,
|
||||||
Confidence: t.Confidence,
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -24,5 +22,4 @@ func (t *Tomorrow) setDayStyleFields(fields dayStyleFields) {
|
|||||||
t.Summary = fields.Summary
|
t.Summary = fields.Summary
|
||||||
t.ForecastDiscussion = fields.ForecastDiscussion
|
t.ForecastDiscussion = fields.ForecastDiscussion
|
||||||
t.PrecipitationTiming = fields.PrecipitationTiming
|
t.PrecipitationTiming = fields.PrecipitationTiming
|
||||||
t.Confidence = fields.Confidence
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -13,8 +13,7 @@ func TestValidateTomorrowNormalizesJSON(t *testing.T) {
|
|||||||
"",
|
"",
|
||||||
" Temperatures stay seasonable by afternoon. "
|
" Temperatures stay seasonable by afternoon. "
|
||||||
],
|
],
|
||||||
"precipitation_timing": " Rain is most likely before sunrise. ",
|
"precipitation_timing": " Rain is most likely before sunrise. "
|
||||||
"confidence": " Medium "
|
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ValidateTomorrow() error = %v", err)
|
t.Fatalf("ValidateTomorrow() error = %v", err)
|
||||||
@@ -28,23 +27,22 @@ func TestValidateTomorrowNormalizesJSON(t *testing.T) {
|
|||||||
if value.PrecipitationTiming != "Rain is most likely before sunrise." {
|
if value.PrecipitationTiming != "Rain is most likely before sunrise." {
|
||||||
t.Fatalf("PrecipitationTiming = %q, want trimmed precipitation timing", value.PrecipitationTiming)
|
t.Fatalf("PrecipitationTiming = %q, want trimmed precipitation timing", value.PrecipitationTiming)
|
||||||
}
|
}
|
||||||
want := `{"summary":"Storms become more likely tomorrow.","forecast_discussion":["A front will keep showers in the forecast.","Temperatures stay seasonable by afternoon."],"precipitation_timing":"Rain is most likely before sunrise.","confidence":"Medium"}`
|
want := `{"summary":"Storms become more likely tomorrow.","forecast_discussion":["A front will keep showers in the forecast.","Temperatures stay seasonable by afternoon."],"precipitation_timing":"Rain is most likely before sunrise."}`
|
||||||
if string(normalized) != want {
|
if string(normalized) != want {
|
||||||
t.Fatalf("normalized = %s, want %s", normalized, want)
|
t.Fatalf("normalized = %s, want %s", normalized, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestValidateTomorrowOmitsEmptyOptionalFields(t *testing.T) {
|
func TestValidateTomorrowPreservesRequiredEmptyPrecipitationTiming(t *testing.T) {
|
||||||
_, normalized, err := ValidateTomorrow([]byte(`{
|
_, normalized, err := ValidateTomorrow([]byte(`{
|
||||||
"summary": "Storms become more likely tomorrow.",
|
"summary": "Storms become more likely tomorrow.",
|
||||||
"forecast_discussion": ["A front will keep showers in the forecast."],
|
"forecast_discussion": ["A front will keep showers in the forecast."],
|
||||||
"precipitation_timing": " ",
|
"precipitation_timing": " "
|
||||||
"confidence": " "
|
|
||||||
}`))
|
}`))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("ValidateTomorrow() error = %v", err)
|
t.Fatalf("ValidateTomorrow() error = %v", err)
|
||||||
}
|
}
|
||||||
want := `{"summary":"Storms become more likely tomorrow.","forecast_discussion":["A front will keep showers in the forecast."]}`
|
want := `{"summary":"Storms become more likely tomorrow.","forecast_discussion":["A front will keep showers in the forecast."],"precipitation_timing":""}`
|
||||||
if string(normalized) != want {
|
if string(normalized) != want {
|
||||||
t.Fatalf("normalized = %s, want %s", normalized, want)
|
t.Fatalf("normalized = %s, want %s", normalized, want)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -8,8 +8,7 @@ Return these fields:
|
|||||||
|
|
||||||
- `summary`: required. One or two sentences summarizing the main weather story for the valid period.
|
- `summary`: required. One or two sentences summarizing the main weather story for the valid period.
|
||||||
- `forecast_discussion`: required. Three paragraphs explaining the broader setup, trend, or forecast reasoning most relevant to the valid period.
|
- `forecast_discussion`: required. Three paragraphs explaining the broader setup, trend, or forecast reasoning most relevant to the valid period.
|
||||||
- `precipitation_timing`: optional. Include only when the deterministic `precip_timing` module contains precipitation windows.
|
- `precipitation_timing`: required. When the deterministic `precip_timing` module contains precipitation windows, provide the supported timing prose described below. Otherwise, return an empty string (`""`).
|
||||||
- `confidence`: optional. Include only if uncertainty, timing spread, or conflicting signals materially affect how the reader should interpret the forecast.
|
|
||||||
|
|
||||||
Return JSON only.
|
Return JSON only.
|
||||||
|
|
||||||
@@ -27,7 +26,7 @@ In most cases, include three paragraphs: a two-to-four sentence relevant local o
|
|||||||
|
|
||||||
# Precipitation timing
|
# Precipitation timing
|
||||||
|
|
||||||
Include this only if precipitation is forecast. Use one to four sentences to give practical context about a supported frontal, convective, or stratiform setup; expected type, intensity, and duration; and uncertainty in onset or duration.
|
When precipitation windows are present, use one to four sentences to give practical context about a supported frontal, convective, or stratiform setup; expected type, intensity, and duration; and uncertainty in onset or duration. When no precipitation windows are present, return an empty string (`""`) for `precipitation_timing`.
|
||||||
|
|
||||||
# Narrative source selection
|
# Narrative source selection
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
id: weather.daily_generated_text
|
id: weather.daily_generated_text
|
||||||
version: "1.0.0"
|
version: "1.0.1"
|
||||||
default_profile: gemini-flash-latest
|
default_profile: gemini-flash-latest
|
||||||
description: Daily weather report analysis prompt.
|
description: Daily weather report analysis prompt.
|
||||||
inputs:
|
inputs:
|
||||||
|
|||||||
@@ -8,8 +8,7 @@ Return these fields:
|
|||||||
|
|
||||||
- `summary`: required. One or two sentences summarizing the main weather story for the valid period.
|
- `summary`: required. One or two sentences summarizing the main weather story for the valid period.
|
||||||
- `forecast_discussion`: required. Two or three sentences explaining the broader setup, trend, or forecast reasoning most relevant to the valid period.
|
- `forecast_discussion`: required. Two or three sentences explaining the broader setup, trend, or forecast reasoning most relevant to the valid period.
|
||||||
- `precipitation_timing`: optional. Include only when the deterministic `precip_timing` module contains precipitation windows.
|
- `precipitation_timing`: required. When the deterministic `precip_timing` module contains precipitation windows, provide the supported timing prose described below. Otherwise, return an empty string (`""`).
|
||||||
- `confidence`: optional. Include only if uncertainty, timing spread, or conflicting signals materially affect how the reader should interpret the forecast.
|
|
||||||
|
|
||||||
Return JSON only.
|
Return JSON only.
|
||||||
|
|
||||||
@@ -25,4 +24,4 @@ Use narrative products to explain the “why” behind the local forecast when u
|
|||||||
|
|
||||||
# Precipitation timing
|
# Precipitation timing
|
||||||
|
|
||||||
Include this only if precipitation is forecast. Use one to four sentences to give practical context about a supported frontal, convective, or stratiform setup; expected type, intensity, and duration; and uncertainty in onset or duration.
|
When precipitation windows are present, use one to four sentences to give practical context about a supported frontal, convective, or stratiform setup; expected type, intensity, and duration; and uncertainty in onset or duration. When no precipitation windows are present, return an empty string (`""`) for `precipitation_timing`.
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
id: weather.hourly_generated_text
|
id: weather.hourly_generated_text
|
||||||
version: "1.0.0"
|
version: "1.0.1"
|
||||||
default_profile: gemini-flash-latest
|
default_profile: gemini-flash-latest
|
||||||
description: Hourly weather report analysis prompt.
|
description: Hourly weather report analysis prompt.
|
||||||
inputs:
|
inputs:
|
||||||
|
|||||||
@@ -8,8 +8,7 @@ Return these fields:
|
|||||||
|
|
||||||
- `summary`: required. One or two sentences summarizing the main weather story for the valid period.
|
- `summary`: required. One or two sentences summarizing the main weather story for the valid period.
|
||||||
- `forecast_discussion`: required. Three paragraphs explaining the broader setup, trend, or forecast reasoning most relevant to the valid period.
|
- `forecast_discussion`: required. Three paragraphs explaining the broader setup, trend, or forecast reasoning most relevant to the valid period.
|
||||||
- `precipitation_timing`: optional. Include only when the deterministic `precip_timing` module contains precipitation windows.
|
- `precipitation_timing`: required. When the deterministic `precip_timing` module contains precipitation windows, provide the supported timing prose described below. Otherwise, return an empty string (`""`).
|
||||||
- `confidence`: optional. Include only if uncertainty, timing spread, or conflicting signals materially affect how the reader should interpret the forecast.
|
|
||||||
|
|
||||||
Return JSON only.
|
Return JSON only.
|
||||||
|
|
||||||
@@ -27,4 +26,4 @@ In most cases, include three paragraphs: a two-to-four sentence relevant local o
|
|||||||
|
|
||||||
# Precipitation timing
|
# Precipitation timing
|
||||||
|
|
||||||
Include this only if precipitation is forecast. Use one to four sentences to give practical context about a supported frontal, convective, or stratiform setup; expected type, intensity, and duration; and uncertainty in onset or duration.
|
When precipitation windows are present, use one to four sentences to give practical context about a supported frontal, convective, or stratiform setup; expected type, intensity, and duration; and uncertainty in onset or duration. When no precipitation windows are present, return an empty string (`""`) for `precipitation_timing`.
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
id: weather.today_generated_text
|
id: weather.today_generated_text
|
||||||
version: "1.0.0"
|
version: "1.0.1"
|
||||||
default_profile: gemini-flash-latest
|
default_profile: gemini-flash-latest
|
||||||
description: Today's weather report analysis prompt.
|
description: Today's weather report analysis prompt.
|
||||||
inputs:
|
inputs:
|
||||||
|
|||||||
@@ -8,8 +8,7 @@ Return these fields:
|
|||||||
|
|
||||||
- `summary`: required. One or two sentences summarizing the main weather story for the valid period.
|
- `summary`: required. One or two sentences summarizing the main weather story for the valid period.
|
||||||
- `forecast_discussion`: required. Three paragraphs explaining the broader setup, trend, or forecast reasoning most relevant to the valid period.
|
- `forecast_discussion`: required. Three paragraphs explaining the broader setup, trend, or forecast reasoning most relevant to the valid period.
|
||||||
- `precipitation_timing`: optional. Include only when the deterministic `precip_timing` module contains precipitation windows.
|
- `precipitation_timing`: required. When the deterministic `precip_timing` module contains precipitation windows, provide the supported timing prose described below. Otherwise, return an empty string (`""`).
|
||||||
- `confidence`: optional. Include only if uncertainty, timing spread, or conflicting signals materially affect how the reader should interpret the forecast.
|
|
||||||
|
|
||||||
Return JSON only.
|
Return JSON only.
|
||||||
|
|
||||||
@@ -27,4 +26,4 @@ In most cases, include three paragraphs: a two-to-four sentence relevant local o
|
|||||||
|
|
||||||
# Precipitation timing
|
# Precipitation timing
|
||||||
|
|
||||||
Use one or two sentences to give practical context about a supported frontal, convective, or stratiform setup; expected type, intensity, and duration; and uncertainty in onset or duration.
|
When precipitation windows are present, use one or two sentences to give practical context about a supported frontal, convective, or stratiform setup; expected type, intensity, and duration; and uncertainty in onset or duration. When no precipitation windows are present, return an empty string (`""`) for `precipitation_timing`.
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
id: weather.tomorrow_generated_text
|
id: weather.tomorrow_generated_text
|
||||||
version: "1.0.0"
|
version: "1.0.1"
|
||||||
default_profile: gemini-flash-latest
|
default_profile: gemini-flash-latest
|
||||||
description: Tomorrow's weather report analysis prompt.
|
description: Tomorrow's weather report analysis prompt.
|
||||||
inputs:
|
inputs:
|
||||||
|
|||||||
@@ -4,11 +4,10 @@
|
|||||||
"title": "Daily GeneratedText",
|
"title": "Daily GeneratedText",
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"additionalProperties": false,
|
"additionalProperties": false,
|
||||||
"required": ["summary", "forecast_discussion"],
|
"required": ["summary", "forecast_discussion", "precipitation_timing"],
|
||||||
"properties": {
|
"properties": {
|
||||||
"summary": {"type": "string"},
|
"summary": {"type": "string"},
|
||||||
"forecast_discussion": {"type": "array", "items": {"type": "string"}, "minItems": 1},
|
"forecast_discussion": {"type": "array", "items": {"type": "string"}, "minItems": 1},
|
||||||
"precipitation_timing": {"type": "string"},
|
"precipitation_timing": {"type": "string"}
|
||||||
"confidence": {"type": "string"}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,11 +4,10 @@
|
|||||||
"title": "Hourly GeneratedText",
|
"title": "Hourly GeneratedText",
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"additionalProperties": false,
|
"additionalProperties": false,
|
||||||
"required": ["summary", "forecast_discussion"],
|
"required": ["summary", "forecast_discussion", "precipitation_timing"],
|
||||||
"properties": {
|
"properties": {
|
||||||
"summary": {"type": "string"},
|
"summary": {"type": "string"},
|
||||||
"forecast_discussion": {"type": "string"},
|
"forecast_discussion": {"type": "string"},
|
||||||
"precipitation_timing": {"type": "string"},
|
"precipitation_timing": {"type": "string"}
|
||||||
"confidence": {"type": "string"}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,11 +4,10 @@
|
|||||||
"title": "Today GeneratedText",
|
"title": "Today GeneratedText",
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"additionalProperties": false,
|
"additionalProperties": false,
|
||||||
"required": ["summary", "forecast_discussion"],
|
"required": ["summary", "forecast_discussion", "precipitation_timing"],
|
||||||
"properties": {
|
"properties": {
|
||||||
"summary": {"type": "string"},
|
"summary": {"type": "string"},
|
||||||
"forecast_discussion": {"type": "array", "items": {"type": "string"}, "minItems": 1},
|
"forecast_discussion": {"type": "array", "items": {"type": "string"}, "minItems": 1},
|
||||||
"precipitation_timing": {"type": "string"},
|
"precipitation_timing": {"type": "string"}
|
||||||
"confidence": {"type": "string"}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,11 +4,10 @@
|
|||||||
"title": "Tomorrow GeneratedText",
|
"title": "Tomorrow GeneratedText",
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"additionalProperties": false,
|
"additionalProperties": false,
|
||||||
"required": ["summary", "forecast_discussion"],
|
"required": ["summary", "forecast_discussion", "precipitation_timing"],
|
||||||
"properties": {
|
"properties": {
|
||||||
"summary": {"type": "string"},
|
"summary": {"type": "string"},
|
||||||
"forecast_discussion": {"type": "array", "items": {"type": "string"}, "minItems": 1},
|
"forecast_discussion": {"type": "array", "items": {"type": "string"}, "minItems": 1},
|
||||||
"precipitation_timing": {"type": "string"},
|
"precipitation_timing": {"type": "string"}
|
||||||
"confidence": {"type": "string"}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -67,8 +67,8 @@ func TestPromptAssetsDeclareTheFourGeneratedTextPrompts(t *testing.T) {
|
|||||||
if err := yaml.Unmarshal(data, &definition); err != nil {
|
if err := yaml.Unmarshal(data, &definition); err != nil {
|
||||||
t.Fatalf("decode prompt definition: %v", err)
|
t.Fatalf("decode prompt definition: %v", err)
|
||||||
}
|
}
|
||||||
if definition.ID != tc.id || definition.Version != "1.0.0" || definition.DefaultProfile != "gemini-flash-latest" {
|
if definition.ID != tc.id || definition.Version != "1.0.1" || definition.DefaultProfile != "gemini-flash-latest" {
|
||||||
t.Fatalf("definition = %#v, want %s version 1.0.0 and gemini-flash-latest", definition, tc.id)
|
t.Fatalf("definition = %#v, want %s version 1.0.1 and gemini-flash-latest", definition, tc.id)
|
||||||
}
|
}
|
||||||
if len(definition.Inputs) != 1 || definition.Inputs[0].Name != "data_package" || !definition.Inputs[0].Required || definition.Inputs[0].ContentType != "application/yaml" {
|
if len(definition.Inputs) != 1 || definition.Inputs[0].Name != "data_package" || !definition.Inputs[0].Required || definition.Inputs[0].ContentType != "application/yaml" {
|
||||||
t.Fatalf("inputs = %#v, want one required YAML data_package", definition.Inputs)
|
t.Fatalf("inputs = %#v, want one required YAML data_package", definition.Inputs)
|
||||||
@@ -101,11 +101,11 @@ func TestSchemasAreCanonicalAndIndependent(t *testing.T) {
|
|||||||
if err := json.Unmarshal(data, &schema); err != nil {
|
if err := json.Unmarshal(data, &schema); err != nil {
|
||||||
t.Fatalf("decode schema: %v", err)
|
t.Fatalf("decode schema: %v", err)
|
||||||
}
|
}
|
||||||
if schema.Type != "object" || schema.AdditionalProperties || strings.Join(schema.Required, ",") != "summary,forecast_discussion" {
|
if schema.Type != "object" || schema.AdditionalProperties || strings.Join(schema.Required, ",") != "summary,forecast_discussion,precipitation_timing" {
|
||||||
t.Fatalf("schema = %#v, want strict generated-text object", schema)
|
t.Fatalf("schema = %#v, want strict generated-text object", schema)
|
||||||
}
|
}
|
||||||
if _, ok := schema.Properties["confidence"]; !ok {
|
if _, ok := schema.Properties["confidence"]; ok {
|
||||||
t.Fatalf("schema properties = %#v, want confidence", schema.Properties)
|
t.Fatalf("schema properties = %#v, do not want retired confidence field", schema.Properties)
|
||||||
}
|
}
|
||||||
if id == "daily" && (schema.ID != "weatherreporter.daily.generated_text.schema.json" || schema.Title != "Daily GeneratedText") {
|
if id == "daily" && (schema.ID != "weatherreporter.daily.generated_text.schema.json" || schema.Title != "Daily GeneratedText") {
|
||||||
t.Fatalf("daily schema identity = %q/%q, want corrected Daily identity", schema.ID, schema.Title)
|
t.Fatalf("daily schema identity = %q/%q, want corrected Daily identity", schema.ID, schema.Title)
|
||||||
@@ -129,11 +129,11 @@ func TestPromptkitInspectsEmbeddedPromptsOffline(t *testing.T) {
|
|||||||
}
|
}
|
||||||
for _, id := range []string{"weather.daily_generated_text", "weather.today_generated_text", "weather.tomorrow_generated_text", "weather.hourly_generated_text"} {
|
for _, id := range []string{"weather.daily_generated_text", "weather.today_generated_text", "weather.tomorrow_generated_text", "weather.hourly_generated_text"} {
|
||||||
t.Run(id, func(t *testing.T) {
|
t.Run(id, func(t *testing.T) {
|
||||||
inspection, err := engine.InspectPrompt(context.Background(), id, "1.0.0")
|
inspection, err := engine.InspectPrompt(context.Background(), id, "1.0.1")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("InspectPrompt() error = %v", err)
|
t.Fatalf("InspectPrompt() error = %v", err)
|
||||||
}
|
}
|
||||||
if inspection.PromptID != id || inspection.PromptVersion != "1.0.0" || inspection.DefaultProfileID != "gemini-flash-latest" {
|
if inspection.PromptID != id || inspection.PromptVersion != "1.0.1" || inspection.DefaultProfileID != "gemini-flash-latest" {
|
||||||
t.Fatalf("inspection = %#v", inspection)
|
t.Fatalf("inspection = %#v", inspection)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ func dailyDefinition() Definition {
|
|||||||
ID: Daily,
|
ID: Daily,
|
||||||
Name: "Daily Report",
|
Name: "Daily Report",
|
||||||
PromptID: "weather.daily_generated_text",
|
PromptID: "weather.daily_generated_text",
|
||||||
PromptVersion: "1.0.0",
|
PromptVersion: "1.0.1",
|
||||||
TemplateID: "daily",
|
TemplateID: "daily",
|
||||||
GeneratedTextSchemaID: "daily",
|
GeneratedTextSchemaID: "daily",
|
||||||
ComparisonStrategy: CompareSameValidDate,
|
ComparisonStrategy: CompareSameValidDate,
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ func hourlyDefinition() Definition {
|
|||||||
ID: Hourly,
|
ID: Hourly,
|
||||||
Name: "Hourly Report",
|
Name: "Hourly Report",
|
||||||
PromptID: "weather.hourly_generated_text",
|
PromptID: "weather.hourly_generated_text",
|
||||||
PromptVersion: "1.0.0",
|
PromptVersion: "1.0.1",
|
||||||
TemplateID: "hourly",
|
TemplateID: "hourly",
|
||||||
GeneratedTextSchemaID: "hourly",
|
GeneratedTextSchemaID: "hourly",
|
||||||
ComparisonStrategy: CompareRollingWindow,
|
ComparisonStrategy: CompareRollingWindow,
|
||||||
|
|||||||
@@ -51,8 +51,8 @@ func TestRegistryContainsOnlyPromptBackedReports(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
for _, definition := range definitions {
|
for _, definition := range definitions {
|
||||||
if definition.PromptVersion != "1.0.0" {
|
if definition.PromptVersion != "1.0.1" {
|
||||||
t.Fatalf("%s PromptVersion = %q, want 1.0.0", definition.ID, definition.PromptVersion)
|
t.Fatalf("%s PromptVersion = %q, want 1.0.1", definition.ID, definition.PromptVersion)
|
||||||
}
|
}
|
||||||
if definition.TemplateID == "" || definition.GeneratedTextSchemaID == "" {
|
if definition.TemplateID == "" || definition.GeneratedTextSchemaID == "" {
|
||||||
t.Fatalf("%s template/schema = %q/%q, want both set", definition.ID, definition.TemplateID, definition.GeneratedTextSchemaID)
|
t.Fatalf("%s template/schema = %q/%q, want both set", definition.ID, definition.TemplateID, definition.GeneratedTextSchemaID)
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ func todayDefinition() Definition {
|
|||||||
ID: Today,
|
ID: Today,
|
||||||
Name: "Today Report",
|
Name: "Today Report",
|
||||||
PromptID: "weather.today_generated_text",
|
PromptID: "weather.today_generated_text",
|
||||||
PromptVersion: "1.0.0",
|
PromptVersion: "1.0.1",
|
||||||
TemplateID: "today",
|
TemplateID: "today",
|
||||||
GeneratedTextSchemaID: "today",
|
GeneratedTextSchemaID: "today",
|
||||||
ComparisonStrategy: CompareSameValidDate,
|
ComparisonStrategy: CompareSameValidDate,
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ func tomorrowDefinition() Definition {
|
|||||||
ID: Tomorrow,
|
ID: Tomorrow,
|
||||||
Name: "Tomorrow Report",
|
Name: "Tomorrow Report",
|
||||||
PromptID: "weather.tomorrow_generated_text",
|
PromptID: "weather.tomorrow_generated_text",
|
||||||
PromptVersion: "1.0.0",
|
PromptVersion: "1.0.1",
|
||||||
TemplateID: "tomorrow",
|
TemplateID: "tomorrow",
|
||||||
GeneratedTextSchemaID: "tomorrow",
|
GeneratedTextSchemaID: "tomorrow",
|
||||||
ComparisonStrategy: CompareSameValidDate,
|
ComparisonStrategy: CompareSameValidDate,
|
||||||
|
|||||||
@@ -64,7 +64,7 @@ func TestSchemaLookup(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Schema() error = %v", err)
|
t.Fatalf("Schema() error = %v", err)
|
||||||
}
|
}
|
||||||
assertStringSchema(t, data, "summary,forecast_discussion", []string{"summary", "forecast_discussion", "precipitation_timing", "confidence"})
|
assertStringSchema(t, data, "summary,forecast_discussion,precipitation_timing", []string{"summary", "forecast_discussion", "precipitation_timing"})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestTomorrowSchemaLookup(t *testing.T) {
|
func TestTomorrowSchemaLookup(t *testing.T) {
|
||||||
@@ -72,7 +72,7 @@ func TestTomorrowSchemaLookup(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Schema() error = %v", err)
|
t.Fatalf("Schema() error = %v", err)
|
||||||
}
|
}
|
||||||
schema := assertSchema(t, data, "summary,forecast_discussion")
|
schema := assertSchema(t, data, "summary,forecast_discussion,precipitation_timing")
|
||||||
property, ok := schema.Properties["forecast_discussion"].(map[string]any)
|
property, ok := schema.Properties["forecast_discussion"].(map[string]any)
|
||||||
if !ok {
|
if !ok {
|
||||||
t.Fatal("schema property forecast_discussion missing or invalid")
|
t.Fatal("schema property forecast_discussion missing or invalid")
|
||||||
@@ -87,7 +87,7 @@ func TestTomorrowSchemaLookup(t *testing.T) {
|
|||||||
if !ok || items["type"] != "string" {
|
if !ok || items["type"] != "string" {
|
||||||
t.Fatalf("forecast_discussion items = %#v, want string items", property["items"])
|
t.Fatalf("forecast_discussion items = %#v, want string items", property["items"])
|
||||||
}
|
}
|
||||||
for _, field := range []string{"summary", "precipitation_timing", "confidence"} {
|
for _, field := range []string{"summary", "precipitation_timing"} {
|
||||||
property, ok := schema.Properties[field].(map[string]any)
|
property, ok := schema.Properties[field].(map[string]any)
|
||||||
if !ok {
|
if !ok {
|
||||||
t.Fatalf("schema property %q missing or invalid", field)
|
t.Fatalf("schema property %q missing or invalid", field)
|
||||||
@@ -103,7 +103,7 @@ func TestDailySchemaLookup(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Schema() error = %v", err)
|
t.Fatalf("Schema() error = %v", err)
|
||||||
}
|
}
|
||||||
schema := assertSchema(t, data, "summary,forecast_discussion")
|
schema := assertSchema(t, data, "summary,forecast_discussion,precipitation_timing")
|
||||||
property, ok := schema.Properties["forecast_discussion"].(map[string]any)
|
property, ok := schema.Properties["forecast_discussion"].(map[string]any)
|
||||||
if !ok {
|
if !ok {
|
||||||
t.Fatal("schema property forecast_discussion missing or invalid")
|
t.Fatal("schema property forecast_discussion missing or invalid")
|
||||||
@@ -118,7 +118,7 @@ func TestDailySchemaLookup(t *testing.T) {
|
|||||||
if !ok || items["type"] != "string" {
|
if !ok || items["type"] != "string" {
|
||||||
t.Fatalf("forecast_discussion items = %#v, want string items", property["items"])
|
t.Fatalf("forecast_discussion items = %#v, want string items", property["items"])
|
||||||
}
|
}
|
||||||
for _, field := range []string{"summary", "precipitation_timing", "confidence"} {
|
for _, field := range []string{"summary", "precipitation_timing"} {
|
||||||
property, ok := schema.Properties[field].(map[string]any)
|
property, ok := schema.Properties[field].(map[string]any)
|
||||||
if !ok {
|
if !ok {
|
||||||
t.Fatalf("schema property %q missing or invalid", field)
|
t.Fatalf("schema property %q missing or invalid", field)
|
||||||
@@ -134,7 +134,7 @@ func TestTodaySchemaLookup(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Schema() error = %v", err)
|
t.Fatalf("Schema() error = %v", err)
|
||||||
}
|
}
|
||||||
schema := assertSchema(t, data, "summary,forecast_discussion")
|
schema := assertSchema(t, data, "summary,forecast_discussion,precipitation_timing")
|
||||||
property, ok := schema.Properties["forecast_discussion"].(map[string]any)
|
property, ok := schema.Properties["forecast_discussion"].(map[string]any)
|
||||||
if !ok {
|
if !ok {
|
||||||
t.Fatal("schema property forecast_discussion missing or invalid")
|
t.Fatal("schema property forecast_discussion missing or invalid")
|
||||||
@@ -149,7 +149,7 @@ func TestTodaySchemaLookup(t *testing.T) {
|
|||||||
if !ok || items["type"] != "string" {
|
if !ok || items["type"] != "string" {
|
||||||
t.Fatalf("forecast_discussion items = %#v, want string items", property["items"])
|
t.Fatalf("forecast_discussion items = %#v, want string items", property["items"])
|
||||||
}
|
}
|
||||||
for _, field := range []string{"summary", "precipitation_timing", "confidence"} {
|
for _, field := range []string{"summary", "precipitation_timing"} {
|
||||||
property, ok := schema.Properties[field].(map[string]any)
|
property, ok := schema.Properties[field].(map[string]any)
|
||||||
if !ok {
|
if !ok {
|
||||||
t.Fatalf("schema property %q missing or invalid", field)
|
t.Fatalf("schema property %q missing or invalid", field)
|
||||||
@@ -214,7 +214,6 @@ func TestRenderHourly(t *testing.T) {
|
|||||||
Summary: "Storm chances increase through late morning.",
|
Summary: "Storm chances increase through late morning.",
|
||||||
ForecastDiscussion: "A front will keep the region unsettled.",
|
ForecastDiscussion: "A front will keep the region unsettled.",
|
||||||
PrecipitationTiming: "A cold front is moving into the region.",
|
PrecipitationTiming: "A cold front is moving into the region.",
|
||||||
Confidence: "Medium confidence in timing.",
|
|
||||||
},
|
},
|
||||||
Modules: testModules{
|
Modules: testModules{
|
||||||
CurrentConditions: &testCurrentConditions{
|
CurrentConditions: &testCurrentConditions{
|
||||||
@@ -282,7 +281,7 @@ func TestRenderHourly(t *testing.T) {
|
|||||||
t.Fatalf("rendered template missing %q:\n%s", want, text)
|
t.Fatalf("rendered template missing %q:\n%s", want, text)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if strings.Contains(text, "19%") || strings.Contains(text, "wind S") || strings.Contains(text, "## Confidence") {
|
if strings.Contains(text, "19%") || strings.Contains(text, "wind S") {
|
||||||
t.Fatalf("rendered template included omitted details:\n%s", text)
|
t.Fatalf("rendered template included omitted details:\n%s", text)
|
||||||
}
|
}
|
||||||
for _, unwanted := range []string{"Avoid low-water crossings.", "Slight risk for severe thunderstorms"} {
|
for _, unwanted := range []string{"Avoid low-water crossings.", "Slight risk for severe thunderstorms"} {
|
||||||
@@ -955,7 +954,6 @@ type testGeneratedText struct {
|
|||||||
Summary string
|
Summary string
|
||||||
ForecastDiscussion string
|
ForecastDiscussion string
|
||||||
PrecipitationTiming string
|
PrecipitationTiming string
|
||||||
Confidence string
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type testTomorrowReportContext struct {
|
type testTomorrowReportContext struct {
|
||||||
@@ -980,14 +978,12 @@ type testTomorrowGeneratedText struct {
|
|||||||
Summary string
|
Summary string
|
||||||
ForecastDiscussion []string
|
ForecastDiscussion []string
|
||||||
PrecipitationTiming string
|
PrecipitationTiming string
|
||||||
Confidence string
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type testDailyGeneratedText struct {
|
type testDailyGeneratedText struct {
|
||||||
Summary string
|
Summary string
|
||||||
ForecastDiscussion []string
|
ForecastDiscussion []string
|
||||||
PrecipitationTiming string
|
PrecipitationTiming string
|
||||||
Confidence string
|
|
||||||
}
|
}
|
||||||
|
|
||||||
type testModules struct {
|
type testModules struct {
|
||||||
|
|||||||
@@ -172,9 +172,9 @@ func validPreparationArtifact() PromptPreparationArtifact {
|
|||||||
return PromptPreparationArtifact{
|
return PromptPreparationArtifact{
|
||||||
SchemaVersion: PromptPreparationSchemaVersion, Status: PromptPreparationSucceeded,
|
SchemaVersion: PromptPreparationSchemaVersion, Status: PromptPreparationSucceeded,
|
||||||
ReportID: report.Daily, RunID: "weatherreporter-run", PromptID: "weather.daily_generated_text",
|
ReportID: report.Daily, RunID: "weatherreporter-run", PromptID: "weather.daily_generated_text",
|
||||||
PromptVersion: "1.0.0", DataPackagePath: "/workspace/data.yaml",
|
PromptVersion: "1.0.1", DataPackagePath: "/workspace/data.yaml",
|
||||||
Preparation: &promptexec.Preparation{
|
Preparation: &promptexec.Preparation{
|
||||||
PromptID: "weather.daily_generated_text", PromptVersion: "1.0.0",
|
PromptID: "weather.daily_generated_text", PromptVersion: "1.0.1",
|
||||||
DataPackagePath: "/workspace/data.yaml",
|
DataPackagePath: "/workspace/data.yaml",
|
||||||
},
|
},
|
||||||
StartedAt: started, EndedAt: started.Add(time.Second), Duration: time.Second,
|
StartedAt: started, EndedAt: started.Add(time.Second), Duration: time.Second,
|
||||||
@@ -186,9 +186,9 @@ func validExecutionArtifact() PromptExecutionArtifact {
|
|||||||
validation := promptexec.NewValidation(promptexec.ValidationPassed, "json_schema", "daily.generated_text.schema.json", nil)
|
validation := promptexec.NewValidation(promptexec.ValidationPassed, "json_schema", "daily.generated_text.schema.json", nil)
|
||||||
return PromptExecutionArtifact{
|
return PromptExecutionArtifact{
|
||||||
SchemaVersion: PromptExecutionSchemaVersion, Status: PromptExecutionSucceeded,
|
SchemaVersion: PromptExecutionSchemaVersion, Status: PromptExecutionSucceeded,
|
||||||
ReportID: report.Daily, RunID: "weatherreporter-run", PromptID: "weather.daily_generated_text", PromptVersion: "1.0.0",
|
ReportID: report.Daily, RunID: "weatherreporter-run", PromptID: "weather.daily_generated_text", PromptVersion: "1.0.1",
|
||||||
Provenance: &PromptExecutionProvenance{
|
Provenance: &PromptExecutionProvenance{
|
||||||
RunID: "provider-run", PromptID: "weather.daily_generated_text", PromptVersion: "1.0.0",
|
RunID: "provider-run", PromptID: "weather.daily_generated_text", PromptVersion: "1.0.1",
|
||||||
PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash", ProfileID: "profile",
|
PromptHash: "prompt-hash", RenderedPromptHash: "rendered-hash", ProfileID: "profile",
|
||||||
BackendID: "backend", ModelName: "model", DataPackagePath: "/workspace/data.yaml",
|
BackendID: "backend", ModelName: "model", DataPackagePath: "/workspace/data.yaml",
|
||||||
StartedAt: started, EndedAt: started.Add(time.Second), Duration: time.Second,
|
StartedAt: started, EndedAt: started.Add(time.Second), Duration: time.Second,
|
||||||
@@ -201,7 +201,7 @@ func validFailedExecutionArtifact() PromptExecutionArtifact {
|
|||||||
started := time.Date(2026, 5, 29, 15, 0, 0, 0, time.UTC)
|
started := time.Date(2026, 5, 29, 15, 0, 0, 0, time.UTC)
|
||||||
return PromptExecutionArtifact{
|
return PromptExecutionArtifact{
|
||||||
SchemaVersion: PromptExecutionSchemaVersion, Status: PromptExecutionFailed,
|
SchemaVersion: PromptExecutionSchemaVersion, Status: PromptExecutionFailed,
|
||||||
ReportID: report.Daily, RunID: "weatherreporter-run", PromptID: "weather.daily_generated_text", PromptVersion: "1.0.0",
|
ReportID: report.Daily, RunID: "weatherreporter-run", PromptID: "weather.daily_generated_text", PromptVersion: "1.0.1",
|
||||||
StartedAt: started, EndedAt: started, Error: &PromptArtifactError{Category: promptexec.Generation, Message: "provider unavailable"},
|
StartedAt: started, EndedAt: started, Error: &PromptArtifactError{Category: promptexec.Generation, Message: "provider unavailable"},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user