weatherreporter/docs/roadmap/domain-profiles.md

Domain-Specific Prompt Profiles Roadmap

Status: Accepted; implementation plan ready.

Purpose

Weatherreporter should provide stable, domain-specific Promptkit profile IDs that express the relative resource and analysis needs of its report products. These logical profiles should give each report an appropriate default while allowing operators to replace any definition through the existing configured profile source.

This roadmap defines the scope, policy, and intended end state. The companion implementation plan owns the ordered work needed to reach that state.

User Intent

The feature is intended to provide three related benefits:

• frequent reports can use a cost-effective model by default;
• reports needing broader synthesis can select a stronger default without forcing the same cost on every invocation; and
• an installation can map a stable Weatherreporter profile ID to a model on a local network endpoint without modifying embedded prompts or application code.

weather-light describes the profile's intended resource tier, not a latency guarantee. A locally hosted lightweight model may still generate slowly on the available hardware.

Current State

Daily, Today, Tomorrow, and Hourly each declare Promptkit's gemini-flash-latest profile as their prompt default. The optional promptkit.profile setting overrides that default for every selected report in an invocation.

Weatherreporter accepts either promptkit.profile_file or promptkit.profile_dir and passes that source to Promptkit. A matching external profile can override a Promptkit built-in profile, and the configured local backend can support profiles that select backend: local. Endpoint-only OpenAI-compatible profiles can also provide their own endpoint.

Weatherreporter does not currently own or embed execution profiles. Promptkit v0.5.0 now provides the fallback-profile layer needed to add them without changing the existing operator-source precedence.

Prerequisite

Promptkit v0.5.0 provides the application fallback profile capability defined in the companion upstream feature request, and Weatherreporter now depends on that tagged release. The dependency upgrade has passed the repository test suite and an operator smoke test. Weatherreporter must continue to use only Promptkit's public API rather than depending on its internal packages or reproducing its profile repository behavior.

Desired End State

Weatherreporter embeds usable definitions for these exact logical profile IDs:

• weather-light
• weather-balanced
• weather-deep

The profiles are Weatherreporter-owned assets and remain behind the existing Promptkit adapter boundary. Prompt definitions select the logical IDs, while Promptkit resolves the effective backend, endpoint, model, and generation settings.

An operator can place a profile with the same ID in profile_file or profile_dir. The operator definition completely replaces the embedded Weatherreporter definition for that ID. If the external source does not contain the selected ID, lookup falls through to Weatherreporter's embedded profile and then to Promptkit's built-in catalog.

The existing global promptkit.profile setting remains available as an explicit all-report override. No new configuration field is required for the initial feature.

Profile Catalog And Report Assignment

Profile	Meaning	Initial default reports
weather-light	Lowest-cost supported tier for frequent, bounded synthesis. It makes no latency promise.	Hourly
weather-balanced	General-purpose tier for broader day-scale synthesis and forecast discussion.	Daily, Today, Tomorrow
weather-deep	Highest-capability tier for explicit operator use and future products whose measured quality benefit warrants the cost.	None initially

The initial assignment recognizes that Weatherreporter's deterministic modules already perform most weather selection and calculation. A higher-capability model should not become a default merely because it is available. Moving an existing report to weather-deep requires evidence that the stronger tier materially improves supported reasoning or output quality.

The three profile IDs are capability policies, not permanent aliases for one provider or model family. Their embedded definitions may change in a future Weatherreporter release, with the change disclosed through normal release and compatibility documentation.

Selection And Definition Precedence

Profile ID selection and profile definition lookup are separate decisions.

Weatherreporter selects the profile ID in this order:

1. nonblank promptkit.profile; or
2. the exact prompt version's default_profile.

Promptkit then resolves the selected profile definition in this order:

1. explicit in-memory profiles, when used by an embedding consumer or test;
2. Weatherreporter's configured profile_file or profile_dir source;
3. Weatherreporter's embedded fallback profiles; and
4. Promptkit's embedded built-in profiles.

A higher-precedence source falls through only when the selected ID is absent. A matching but malformed operator profile fails before weather collection and must not silently use the embedded definition.

Local Endpoint Experience

An operator should be able to override weather-light with an endpoint-only profile whose model name is understood by the local OpenAI-compatible server. This path does not require a separate Weatherreporter local-backend setting.

Alternatively, an override may select backend: local; in that case the existing promptkit.local.endpoint and concurrency settings continue to own the shared local backend definition.

The selected local profile is deterministic configuration, not a preference hint. Weatherreporter will not probe for a local model and will not automatically fall back to a remote or paid profile when the endpoint is unavailable. The failure remains visible and attributable to the selected profile.

Embedded Profile Policy

Each embedded profile must be a complete, valid Promptkit profile and must be usable in a default installation with the documented credential mechanism. The initial embedded profiles are expected to use Promptkit's openrouter backend, allowing them to inherit its endpoint and OPENROUTER_API_KEY environment variable without embedding credentials.

Embedded definitions should include only settings that are intentional for the selected model and supported by its backend. Avoid incidental generation parameters that reduce portability or trigger provider-specific request failures without a demonstrated quality benefit.

The initial profile definitions are:

Profile ID	OpenRouter model	Reasoning effort	Timeout	Service tier
weather-light	deepseek/deepseek-v4-flash	Provider default	180 seconds	flex
weather-balanced	~google/gemini-flash-latest	high	240 seconds	flex
weather-deep	~anthropic/claude-sonnet-latest	high	240 seconds	flex

These settings deliberately match the corresponding Promptkit v0.5.0 built-ins while exposing Weatherreporter-owned logical IDs. The ~ prefix is part of each OpenRouter rolling-alias identifier. The profiles do not set temperature, top_p, or output-token limits; omission preserves provider defaults and avoids unsupported incidental parameters.

Prompt And Active Execution Contract

Changing a prompt's default_profile is a material prompt-definition change. The four prompt definitions should advance from 1.0.1 to 1.1.0 when the new defaults are introduced. Prompt content and generated-text schemas need not change solely for this feature.

Prompt inspection must continue to occur before weather collection. It should report the selected logical profile ID and the resolved backend and model without exposing endpoints or credentials.

The active execution contract should retain both the selected logical profile identity and the resolved backend and model through inspection, preparation, execution, errors, and command results where those values are already exposed. This feature must not add a new durable-provenance or cross-version artifact contract.

The accepted ephemeral-state roadmap makes historical prompt provenance a non-goal. Existing workspace persistence may remain while this feature lands, but it is transitional behavior and must not be expanded or treated as part of the profile feature's desired end state. Prompt preparation and execution artifacts written at 1.0.1 are not required to remain readable after the prompt definitions advance to 1.1.0.

Evaluation Policy

Concrete model assignments should be evaluated with representative, secret-free Daily, Today, Tomorrow, and Hourly data packages. Evaluation should consider:

• strict-schema success rate;
• unsupported or invented weather claims;
• precipitation-timing accuracy and empty-string behavior;
• correct use of deterministic hazards, periods, and uncertainty;
• summary and forecast-discussion usefulness;
• generation latency;
• token use and provider cost; and
• behavior through a representative local OpenAI-compatible endpoint.

The purpose is to choose an appropriate default for each tier, not to add a permanent benchmark framework or live-provider requirement to the ordinary test suite. Repository tests remain offline and deterministic.

Scope

The completed feature includes:

• Weatherreporter-owned embedded profile assets for all three logical IDs;
• Promptkit adapter wiring that supplies those assets as the application fallback profile source;
• per-prompt default-profile assignments matching the catalog above;
• an exact prompt-version update for the changed definitions;
• preservation of the global profile override;
• same-ID override behavior through both supported external profile-source forms;
• local-backend and endpoint-only override coverage;
• fail-fast inspection of missing, malformed, or unusable selected profiles;
• offline tests for selection, source precedence, effective model inspection, batch reuse, and active execution behavior;
• maintained operator examples for overriding weather-light locally; and
• updates to the canonical configuration, Promptkit integration, report registry, operations, troubleshooting, internal adapter, and release documentation as applicable when implementation lands.

Non-Goals

The feature does not include:

• automatic discovery, health checking, or benchmarking of local endpoints;
• implicit failover between local and remote profiles;
• retries with a more expensive tier after provider or validation failure;
• per-report profile configuration fields outside prompt defaults;
• profile inheritance, aliases, or field-level merging;
• runtime model selection based on weather severity, token count, or report content;
• moving Weatherreporter profile policy into Promptkit's built-in catalog;
• exposing Promptkit types outside the adapter boundary; or
• making live provider calls part of the default repository test suite.

Compatibility And Operational Policy

Existing configurations with a nonblank promptkit.profile retain their all-report behavior. Existing profile_file, profile_dir, local-backend, and credential configuration fields retain their meanings.

Configurations that rely on the omitted profile setting will intentionally observe new per-report defaults. This is a user-visible model-selection and cost change and must be called out in release notes. Operators who require the old all-report model can preserve it by setting an explicit global profile.

The prompt-version transition does not provide backward compatibility for historical prompt preparation or execution artifacts. This is consistent with the accepted ephemeral-state direction; the profile feature does not otherwise redesign or remove the current workspace layout.

An external same-ID override is an operator-owned compatibility commitment. Weatherreporter may evolve its embedded definitions, but it must not rewrite or silently merge an operator file.

Completion Criteria

The roadmap is complete when:

• a tagged Promptkit dependency supports the required fallback layer;
• every operational prompt selects its assigned logical profile at exact version 1.1.0;
• all three embedded profiles inspect successfully without an external profile source;
• configured same-ID definitions override embedded definitions through both profile_file and profile_dir;
• an invalid matching external definition fails without fallback;
• weather-light can resolve through an endpoint-only or configured-local override without requiring code or prompt changes;
• global promptkit.profile still overrides every report in an invocation;
• active inspection and execution preserve the selected logical profile and effective model through the project-owned execution contract;
• morning and evening batch preflight deduplicates inspection of shared effective profile IDs as it does today;
• the default test suite remains offline and deterministic; and
• implemented behavior is documented by its canonical current-state owners.