weatherreporter/docs/roadmap/domain-profiles.md

Domain-Specific Prompt Profiles Roadmap

Status: Accepted feature direction; implementation has not started.

Purpose

Weatherreporter should provide stable, domain-specific Promptkit profile IDs that express the relative resource and analysis needs of its report products. These logical profiles should give each report an appropriate default while allowing operators to replace any definition through the existing configured profile source.

This roadmap defines the scope, policy, and intended end state. A later staged implementation plan will translate it into implementation work. This document does not prescribe implementation stages.

User Intent

The feature is intended to provide three related benefits:

• frequent reports can use a cost-effective model by default;
• reports needing broader synthesis can select a stronger default without forcing the same cost on every invocation; and
• an installation can map a stable Weatherreporter profile ID to a model on a local network endpoint without modifying embedded prompts or application code.

weather-light describes the profile's intended resource tier, not a latency guarantee. A locally hosted lightweight model may still generate slowly on the available hardware.

Current State

Daily, Today, Tomorrow, and Hourly each declare Promptkit's gemini-flash-latest profile as their prompt default. The optional promptkit.profile setting overrides that default for every selected report in an invocation.

Weatherreporter accepts either promptkit.profile_file or promptkit.profile_dir and passes that source to Promptkit. A matching external profile can override a Promptkit built-in profile, and the configured local backend can support profiles that select backend: local. Endpoint-only OpenAI-compatible profiles can also provide their own endpoint.

Weatherreporter does not currently own or embed execution profiles. Promptkit's current public source precedence also has no layer where application-owned fallback profiles can remain overridable by the ordinary configured source.

Prerequisite

Promptkit must first provide the application fallback profile capability defined in the companion upstream feature request.

Weatherreporter should adopt a tagged Promptkit release containing that public capability. It should not depend on Promptkit internal packages, reproduce Promptkit's profile repository behavior, or commit a local module replacement as the production dependency.

Desired End State

Weatherreporter embeds usable definitions for these exact logical profile IDs:

• weather-light
• weather-balanced
• weather-deep

The profiles are Weatherreporter-owned assets and remain behind the existing Promptkit adapter boundary. Prompt definitions select the logical IDs, while Promptkit resolves the effective backend, endpoint, model, and generation settings.

An operator can place a profile with the same ID in profile_file or profile_dir. The operator definition completely replaces the embedded Weatherreporter definition for that ID. If the external source does not contain the selected ID, lookup falls through to Weatherreporter's embedded profile and then to Promptkit's built-in catalog.

The existing global promptkit.profile setting remains available as an explicit all-report override. No new configuration field is required for the initial feature.

Profile Catalog And Report Assignment

Profile	Meaning	Initial default reports
weather-light	Lowest-cost supported tier for frequent, bounded synthesis. It makes no latency promise.	Hourly
weather-balanced	General-purpose tier for broader day-scale synthesis and forecast discussion.	Daily, Today, Tomorrow
weather-deep	Highest-capability tier for explicit operator use and future products whose measured quality benefit warrants the cost.	None initially

The initial assignment recognizes that Weatherreporter's deterministic modules already perform most weather selection and calculation. A higher-capability model should not become a default merely because it is available. Moving an existing report to weather-deep requires evidence that the stronger tier materially improves supported reasoning or output quality.

The three profile IDs are capability policies, not permanent aliases for one provider or model family. Their embedded definitions may change in a future Weatherreporter release, with the change disclosed through normal release and compatibility documentation.

Selection And Definition Precedence

Profile ID selection and profile definition lookup are separate decisions.

Weatherreporter selects the profile ID in this order:

1. nonblank promptkit.profile; or
2. the exact prompt version's default_profile.

Promptkit then resolves the selected profile definition in this order:

1. explicit in-memory profiles, when used by an embedding consumer or test;
2. Weatherreporter's configured profile_file or profile_dir source;
3. Weatherreporter's embedded fallback profiles; and
4. Promptkit's embedded built-in profiles.

A higher-precedence source falls through only when the selected ID is absent. A matching but malformed operator profile fails before weather collection and must not silently use the embedded definition.

Local Endpoint Experience

An operator should be able to override weather-light with an endpoint-only profile whose model name is understood by the local OpenAI-compatible server. This path does not require a separate Weatherreporter local-backend setting.

Alternatively, an override may select backend: local; in that case the existing promptkit.local.endpoint and concurrency settings continue to own the shared local backend definition.

The selected local profile is deterministic configuration, not a preference hint. Weatherreporter will not probe for a local model and will not automatically fall back to a remote or paid profile when the endpoint is unavailable. The failure remains visible and attributable to the selected profile.

Embedded Profile Policy

Each embedded profile must be a complete, valid Promptkit profile and must be usable in a default installation with the documented credential mechanism. The initial embedded profiles are expected to use Promptkit's openrouter backend, allowing them to inherit its endpoint and OPENROUTER_API_KEY environment variable without embedding credentials.

Embedded definitions should include only settings that are intentional for the selected model and supported by its backend. Avoid incidental generation parameters that reduce portability or trigger provider-specific request failures without a demonstrated quality benefit.

The initial model candidates are:

• weather-light: the maintained OpenRouter identifier for the Gemma 4 31B instruction model;
• weather-balanced: Promptkit's maintained Gemini Flash family target; and
• weather-deep: Promptkit's maintained Gemini Pro family target.

The exact identifiers and generation settings must be finalized against the available tagged Promptkit catalog and a representative Weatherreporter evaluation corpus before the staged implementation plan is approved. The logical profile IDs and report assignments do not depend on those concrete choices.

Prompt And Provenance Contract

Changing a prompt's default_profile is a material prompt-definition change. The four prompt definitions should advance from 1.0.1 to 1.1.0 when the new defaults are introduced. Prompt content and generated-text schemas need not change solely for this feature.

Prompt inspection must continue to occur before weather collection. It should report the selected logical profile ID and the resolved backend and model without exposing endpoints or credentials.

Preparation, execution, and failure artifacts should retain both the logical profile identity and effective model provenance already available through the project-owned execution contract. This feature should not replace a logical profile ID with only the concrete model name in durable records.

Evaluation Policy

Concrete model assignments should be evaluated with representative, secret-free Daily, Today, Tomorrow, and Hourly data packages. Evaluation should consider:

• strict-schema success rate;
• unsupported or invented weather claims;
• precipitation-timing accuracy and empty-string behavior;
• correct use of deterministic hazards, periods, and uncertainty;
• summary and forecast-discussion usefulness;
• generation latency;
• token use and provider cost; and
• behavior through a representative local OpenAI-compatible endpoint.

The purpose is to choose an appropriate default for each tier, not to add a permanent benchmark framework or live-provider requirement to the ordinary test suite. Repository tests remain offline and deterministic.

Scope

The completed feature includes:

• Weatherreporter-owned embedded profile assets for all three logical IDs;
• Promptkit adapter wiring that supplies those assets as the application fallback profile source;
• per-prompt default-profile assignments matching the catalog above;
• an exact prompt-version update for the changed definitions;
• preservation of the global profile override;
• same-ID override behavior through both supported external profile-source forms;
• local-backend and endpoint-only override coverage;
• fail-fast inspection of missing, malformed, or unusable selected profiles;
• offline tests for selection, source precedence, effective model inspection, batch reuse, and durable provenance;
• maintained operator examples for overriding weather-light locally; and
• updates to the canonical configuration, Promptkit integration, report registry, operations, troubleshooting, internal adapter, and release documentation as applicable when implementation lands.

Non-Goals

The feature does not include:

• automatic discovery, health checking, or benchmarking of local endpoints;
• implicit failover between local and remote profiles;
• retries with a more expensive tier after provider or validation failure;
• per-report profile configuration fields outside prompt defaults;
• profile inheritance, aliases, or field-level merging;
• runtime model selection based on weather severity, token count, or report content;
• moving Weatherreporter profile policy into Promptkit's built-in catalog;
• exposing Promptkit types outside the adapter boundary; or
• making live provider calls part of the default repository test suite.

Compatibility And Operational Policy

Existing configurations with a nonblank promptkit.profile retain their all-report behavior. Existing profile_file, profile_dir, local-backend, and credential configuration fields retain their meanings.

Configurations that rely on the omitted profile setting will intentionally observe new per-report defaults. This is a user-visible model-selection and cost change and must be called out in release notes. Operators who require the old all-report model can preserve it by setting an explicit global profile.

An external same-ID override is an operator-owned compatibility commitment. Weatherreporter may evolve its embedded definitions, but it must not rewrite or silently merge an operator file.

Completion Criteria

The roadmap is complete when:

• a tagged Promptkit dependency supports the required fallback layer;
• every operational prompt selects its assigned logical profile at exact version 1.1.0;
• all three embedded profiles inspect successfully without an external profile source;
• configured same-ID definitions override embedded definitions through both profile_file and profile_dir;
• an invalid matching external definition fails without fallback;
• weather-light can resolve through an endpoint-only or configured-local override without requiring code or prompt changes;
• global promptkit.profile still overrides every report in an invocation;
• prompt preparation and execution artifacts record the selected logical profile and effective model;
• morning and evening batch preflight deduplicates inspection of shared effective profile IDs as it does today;
• the default test suite remains offline and deterministic; and
• implemented behavior is documented by its canonical current-state owners.

Open Questions

Exact initial model identifiers and settings

Recommendation: evaluate the current Gemma 4 31B instruction target for weather-light, the maintained Gemini Flash target for weather-balanced, and the maintained Gemini Pro target for weather-deep. Pin the exact profile contents in this roadmap before deriving the staged implementation plan.

Alternative: duplicate three current Promptkit built-in definitions without a Weatherreporter-specific evaluation. This is faster, but it provides less evidence that their cost, schema reliability, and weather reasoning match the tier policy.

Future default use of weather-deep

Recommendation: ship the profile as an explicit override with no default report assignment. Assign it later only when a report or evaluation shows a material benefit over weather-balanced.

Alternative: assign Daily or Tomorrow to weather-deep immediately. This may improve difficult synthesis, but it increases routine cost before the benefit has been measured.