weatherreporter/docs/roadmap/domain-profiles.md

# Domain-Specific Prompt Profiles Roadmap

Status: Accepted feature direction; implementation has not started.

## Purpose

Weatherreporter should provide stable, domain-specific Promptkit profile IDs
that express the relative resource and analysis needs of its report products.
These logical profiles should give each report an appropriate default while
allowing operators to replace any definition through the existing configured
profile source.

This roadmap defines the scope, policy, and intended end state. A later staged
implementation plan will translate it into implementation work. This document
does not prescribe implementation stages.

## User Intent

The feature is intended to provide three related benefits:

- frequent reports can use a cost-effective model by default;
- reports needing broader synthesis can select a stronger default without
  forcing the same cost on every invocation; and
- an installation can map a stable Weatherreporter profile ID to a model on a
  local network endpoint without modifying embedded prompts or application
  code.

`weather-light` describes the profile's intended resource tier, not a latency
guarantee. A locally hosted lightweight model may still generate slowly on the
available hardware.

## Current State

Daily, Today, Tomorrow, and Hourly each declare Promptkit's
`gemini-flash-latest` profile as their prompt default. The optional
`promptkit.profile` setting overrides that default for every selected report in
an invocation.

Weatherreporter accepts either `promptkit.profile_file` or
`promptkit.profile_dir` and passes that source to Promptkit. A matching external
profile can override a Promptkit built-in profile, and the configured local
backend can support profiles that select `backend: local`. Endpoint-only
OpenAI-compatible profiles can also provide their own endpoint.

Weatherreporter does not currently own or embed execution profiles. Promptkit's
current public source precedence also has no layer where application-owned
fallback profiles can remain overridable by the ordinary configured source.

## Prerequisite

Promptkit must first provide the application fallback profile capability
defined in the companion
[upstream feature request](promptkit-fallback-profiles-feature-request.md).

Weatherreporter should adopt a tagged Promptkit release containing that public
capability. It should not depend on Promptkit internal packages, reproduce
Promptkit's profile repository behavior, or commit a local module replacement
as the production dependency.

## Desired End State

Weatherreporter embeds usable definitions for these exact logical profile IDs:

- `weather-light`
- `weather-balanced`
- `weather-deep`

The profiles are Weatherreporter-owned assets and remain behind the existing
Promptkit adapter boundary. Prompt definitions select the logical IDs, while
Promptkit resolves the effective backend, endpoint, model, and generation
settings.

An operator can place a profile with the same ID in `profile_file` or
`profile_dir`. The operator definition completely replaces the embedded
Weatherreporter definition for that ID. If the external source does not contain
the selected ID, lookup falls through to Weatherreporter's embedded profile and
then to Promptkit's built-in catalog.

The existing global `promptkit.profile` setting remains available as an
explicit all-report override. No new configuration field is required for the
initial feature.

## Profile Catalog And Report Assignment

| Profile | Meaning | Initial default reports |
| --- | --- | --- |
| `weather-light` | Lowest-cost supported tier for frequent, bounded synthesis. It makes no latency promise. | Hourly |
| `weather-balanced` | General-purpose tier for broader day-scale synthesis and forecast discussion. | Daily, Today, Tomorrow |
| `weather-deep` | Highest-capability tier for explicit operator use and future products whose measured quality benefit warrants the cost. | None initially |

The initial assignment recognizes that Weatherreporter's deterministic modules
already perform most weather selection and calculation. A higher-capability
model should not become a default merely because it is available. Moving an
existing report to `weather-deep` requires evidence that the stronger tier
materially improves supported reasoning or output quality.

The three profile IDs are capability policies, not permanent aliases for one
provider or model family. Their embedded definitions may change in a future
Weatherreporter release, with the change disclosed through normal release and
compatibility documentation.

## Selection And Definition Precedence

Profile ID selection and profile definition lookup are separate decisions.

Weatherreporter selects the profile ID in this order:

1. nonblank `promptkit.profile`; or
2. the exact prompt version's `default_profile`.

Promptkit then resolves the selected profile definition in this order:

1. explicit in-memory profiles, when used by an embedding consumer or test;
2. Weatherreporter's configured `profile_file` or `profile_dir` source;
3. Weatherreporter's embedded fallback profiles; and
4. Promptkit's embedded built-in profiles.

A higher-precedence source falls through only when the selected ID is absent.
A matching but malformed operator profile fails before weather collection and
must not silently use the embedded definition.

## Local Endpoint Experience

An operator should be able to override `weather-light` with an endpoint-only
profile whose model name is understood by the local OpenAI-compatible server.
This path does not require a separate Weatherreporter local-backend setting.

Alternatively, an override may select `backend: local`; in that case the
existing `promptkit.local.endpoint` and concurrency settings continue to own
the shared local backend definition.

The selected local profile is deterministic configuration, not a preference
hint. Weatherreporter will not probe for a local model and will not
automatically fall back to a remote or paid profile when the endpoint is
unavailable. The failure remains visible and attributable to the selected
profile.

## Embedded Profile Policy

Each embedded profile must be a complete, valid Promptkit profile and must be
usable in a default installation with the documented credential mechanism. The
initial embedded profiles are expected to use Promptkit's `openrouter` backend,
allowing them to inherit its endpoint and `OPENROUTER_API_KEY` environment
variable without embedding credentials.

Embedded definitions should include only settings that are intentional for the
selected model and supported by its backend. Avoid incidental generation
parameters that reduce portability or trigger provider-specific request
failures without a demonstrated quality benefit.

The initial model candidates are:

- `weather-light`: the maintained OpenRouter identifier for the Gemma 4 31B
  instruction model;
- `weather-balanced`: Promptkit's maintained Gemini Flash family target; and
- `weather-deep`: Promptkit's maintained Gemini Pro family target.

The exact identifiers and generation settings must be finalized against the
available tagged Promptkit catalog and a representative Weatherreporter
evaluation corpus before the staged implementation plan is approved. The
logical profile IDs and report assignments do not depend on those concrete
choices.

## Prompt And Provenance Contract

Changing a prompt's `default_profile` is a material prompt-definition change.
The four prompt definitions should advance from `1.0.1` to `1.1.0` when the new
defaults are introduced. Prompt content and generated-text schemas need not
change solely for this feature.

Prompt inspection must continue to occur before weather collection. It should
report the selected logical profile ID and the resolved backend and model
without exposing endpoints or credentials.

Preparation, execution, and failure artifacts should retain both the logical
profile identity and effective model provenance already available through the
project-owned execution contract. This feature should not replace a logical
profile ID with only the concrete model name in durable records.

## Evaluation Policy

Concrete model assignments should be evaluated with representative,
secret-free Daily, Today, Tomorrow, and Hourly data packages. Evaluation should
consider:

- strict-schema success rate;
- unsupported or invented weather claims;
- precipitation-timing accuracy and empty-string behavior;
- correct use of deterministic hazards, periods, and uncertainty;
- summary and forecast-discussion usefulness;
- generation latency;
- token use and provider cost; and
- behavior through a representative local OpenAI-compatible endpoint.

The purpose is to choose an appropriate default for each tier, not to add a
permanent benchmark framework or live-provider requirement to the ordinary
test suite. Repository tests remain offline and deterministic.

## Scope

The completed feature includes:

- Weatherreporter-owned embedded profile assets for all three logical IDs;
- Promptkit adapter wiring that supplies those assets as the application
  fallback profile source;
- per-prompt default-profile assignments matching the catalog above;
- an exact prompt-version update for the changed definitions;
- preservation of the global profile override;
- same-ID override behavior through both supported external profile-source
  forms;
- local-backend and endpoint-only override coverage;
- fail-fast inspection of missing, malformed, or unusable selected profiles;
- offline tests for selection, source precedence, effective model inspection,
  batch reuse, and durable provenance;
- maintained operator examples for overriding `weather-light` locally; and
- updates to the canonical configuration, Promptkit integration, report
  registry, operations, troubleshooting, internal adapter, and release
  documentation as applicable when implementation lands.

## Non-Goals

The feature does not include:

- automatic discovery, health checking, or benchmarking of local endpoints;
- implicit failover between local and remote profiles;
- retries with a more expensive tier after provider or validation failure;
- per-report profile configuration fields outside prompt defaults;
- profile inheritance, aliases, or field-level merging;
- runtime model selection based on weather severity, token count, or report
  content;
- moving Weatherreporter profile policy into Promptkit's built-in catalog;
- exposing Promptkit types outside the adapter boundary; or
- making live provider calls part of the default repository test suite.

## Compatibility And Operational Policy

Existing configurations with a nonblank `promptkit.profile` retain their
all-report behavior. Existing `profile_file`, `profile_dir`, local-backend, and
credential configuration fields retain their meanings.

Configurations that rely on the omitted profile setting will intentionally
observe new per-report defaults. This is a user-visible model-selection and
cost change and must be called out in release notes. Operators who require the
old all-report model can preserve it by setting an explicit global profile.

An external same-ID override is an operator-owned compatibility commitment.
Weatherreporter may evolve its embedded definitions, but it must not rewrite or
silently merge an operator file.

## Completion Criteria

The roadmap is complete when:

- a tagged Promptkit dependency supports the required fallback layer;
- every operational prompt selects its assigned logical profile at exact
  version `1.1.0`;
- all three embedded profiles inspect successfully without an external profile
  source;
- configured same-ID definitions override embedded definitions through both
  `profile_file` and `profile_dir`;
- an invalid matching external definition fails without fallback;
- `weather-light` can resolve through an endpoint-only or configured-local
  override without requiring code or prompt changes;
- global `promptkit.profile` still overrides every report in an invocation;
- prompt preparation and execution artifacts record the selected logical
  profile and effective model;
- morning and evening batch preflight deduplicates inspection of shared
  effective profile IDs as it does today;
- the default test suite remains offline and deterministic; and
- implemented behavior is documented by its canonical current-state owners.

## Open Questions

### Exact initial model identifiers and settings

Recommendation: evaluate the current Gemma 4 31B instruction target for
`weather-light`, the maintained Gemini Flash target for `weather-balanced`, and
the maintained Gemini Pro target for `weather-deep`. Pin the exact profile
contents in this roadmap before deriving the staged implementation plan.

Alternative: duplicate three current Promptkit built-in definitions without a
Weatherreporter-specific evaluation. This is faster, but it provides less
evidence that their cost, schema reliability, and weather reasoning match the
tier policy.

### Future default use of `weather-deep`

Recommendation: ship the profile as an explicit override with no default
report assignment. Assign it later only when a report or evaluation shows a
material benefit over `weather-balanced`.

Alternative: assign Daily or Tomorrow to `weather-deep` immediately. This may
improve difficult synthesis, but it increases routine cost before the benefit
has been measured.