weatherreporter/docs/roadmap/promptkit-v0.8.0.md

# PromptKit v0.8.0 Upgrade Roadmap

Status: Implemented.

## Purpose

WeatherReporter should upgrade its PromptKit dependency from `v0.5.0` to
`v0.8.0` and deliberately adopt the useful consumer-facing capabilities added
in `v0.6.0`, `v0.7.0`, and `v0.8.0`. The upgrade should improve output-contract
reliability, profile composition, local and alternate endpoint support, and
provider-failure diagnosis without moving PromptKit responsibilities into
WeatherReporter or weakening the application's stateless and security
boundaries.

This roadmap defines the intended scope, policy, and end state. The
[implementation plan](implementation.md) owns the procedure for reaching that
state.

## User Intent

The upgrade is intended to:

- use PromptKit's bounded output repair to recover from occasional malformed
  structured weather prose;
- keep WeatherReporter's domain profile IDs stable while inheriting maintained
  PromptKit model definitions;
- make PromptKit's additional built-in backend and profile available for
  explicit generation and profile comparisons;
- support unauthenticated or optionally authenticated OpenAI-compatible
  endpoints without inventing a WeatherReporter transport layer;
- make provider HTTP failures more actionable under an explicit
  WeatherReporter disclosure policy; and
- receive PromptKit's intervening correctness, safety, cancellation, resource,
  and efficiency improvements as part of one tested dependency upgrade.

The model ladder and report assignments do not change as part of this work:
Hourly continues to select `weather-light`; Daily, Today, and Tomorrow continue
to select `weather-balanced`; and `weather-deep` remains available for explicit
selection. This upgrade does not promote the new Rakestrawhome profile into
that default ladder.

## Current State

WeatherReporter currently depends on
`gitea.maximumdirect.net/eric/promptkit` at `v0.5.0`. The PromptKit adapter
supplies embedded prompts, JSON Schemas, and
application-fallback profiles, plus an optional configured profile source and
the conventional local backend.

The four generated-text prompts are exact version `2.0.0` JSON Schema prompts.
They omit `repair_attempts`, so execution is single-pass. The project-owned
`promptexec.OutputContract` and validation result also omit repair budgets and
actual repair counts.

The three embedded WeatherReporter profiles duplicate the effective fields of
these PromptKit built-ins:

| WeatherReporter profile | PromptKit built-in with the same target |
| --- | --- |
| `weather-light` | `deepseek-4-flash` |
| `weather-balanced` | `gemini-flash-latest` |
| `weather-deep` | `claude-sonnet-latest` |

WeatherReporter preflights any nonblank `api_key_env` as a required credential,
even though PromptKit v0.7.0 distinguishes an optional environment source from
an explicit `APIKeyRequired` target. Provider generation failures are reduced
to WeatherReporter's safe `generation` category; the PromptKit dependency error
is retained as a hidden cause, but its structured HTTP status and provider
diagnostics are not mapped into project-owned values.

The maintained `weather-light` local override is an endpoint-only profile, and
the configuration contract says endpoint-only profiles are supported. PromptKit
inspection correctly reports no backend ID for that form, but WeatherReporter
application preflight currently requires both a nonblank backend and model.
That mismatch prevents the documented example from reaching generation and
should be corrected as part of adopting the current PromptKit target contract.

## Upstream Release Assessment

### PromptKit v0.6.0

`v0.6.0` adds no public declarations, but it is a material compatibility and
safety release. It centralizes execution-setting, output-contract, endpoint,
and JSON-compatible-value validation; makes YAML metadata authoritative for
prompt and profile identity; hardens `content_file` containment and regular-file
requirements; validates OpenAI-compatible endpoints structurally; bounds JSON
trees and successful provider bodies; requires exactly one JSON value in
provider responses; preserves cancellation and transport error identities; and
reuses schema and rendered-artifact work within an operation.

WeatherReporter should receive these improvements directly from the dependency
and audit its own supported assets and configuration paths against the stricter
contracts. It should not duplicate PromptKit's internal validators or tests.
The existing embedded prompt paths, inline data-package input, generated-output
limit, and adapter boundary remain conceptually correct.

### PromptKit v0.7.0

`v0.7.0` adds four potentially useful consumer features:

- linear, cycle-safe profile inheritance through `base_profile` and
  `Profile.BaseProfileID`;
- the built-in `rakestrawhome` backend and
  `rakestrawhome-gemma-4-31b` profile;
- optional API-key environment sources, with `APIKeyRequired` reserved for an
  explicit local credential requirement; and
- bounded structured `GenerationError` details for non-2xx responses from the
  built-in OpenAI-compatible client.

WeatherReporter has no manual `rakestrawhome` registration and uses keyed
PromptKit profile literals, so the two source-compatibility hazards called out
by the release do not require migration shims. The profile, credential, and
error features do require deliberate application-policy choices described
below.

### PromptKit v0.8.0

`v0.8.0` activates the existing output-contract repair budget. A positive
`repair_attempts` value authorizes up to that many corrective model calls after
eligible `basic`, `json`, or `json_schema` validation failures. The supported
budget is zero through three. Repairs preserve the original rendered
conversation, effective target, session, structured-output contract, and
backend capacity policy. The final result reports cumulative token usage and
the number of corrective calls actually made.

Repair exhaustion is a completed generation with failed validation, not an
operational error. WeatherReporter's existing policy should continue to reject
that result and publish no report for that profile. Explicit empty provider
content now reaches output validation; for WeatherReporter's JSON Schema
prompts it is therefore eligible for repair rather than being misclassified as
a malformed provider envelope.

## Desired End State

WeatherReporter builds and tests against PromptKit `v0.8.0` with no workspace,
vendor, or module replacement dependency. Its public behavior remains
stateless, its PromptKit dependency types remain confined to the adapter, and
its ordinary summaries and logs remain safe.

The completed integration:

- benefits from the v0.6.0 safety and efficiency corrections;
- composes WeatherReporter domain profiles from PromptKit's maintained built-in
  profiles while preserving WeatherReporter-owned leaf IDs and operator
  override precedence;
- accepts a successfully inspected endpoint-only profile with a nonblank model
  even though it has no logical backend ID;
- permits explicit use of PromptKit's Rakestrawhome profile without custom
  backend wiring;
- applies an accepted bounded-repair policy to every operational structured
  prompt;
- validates repair configuration during preflight and records actual repair
  activity in project-owned result values;
- retains PromptKit's cumulative usage accounting in explicit debug output;
- distinguishes safe provider HTTP status from potentially sensitive provider
  diagnostics; and
- documents the changed profile, credential, repair, comparison, debug, and
  failure contracts in their canonical owners.

## Dependency And Compatibility Policy

The module requirement should move directly from `v0.5.0` to `v0.8.0`, followed
by a clean module tidy. WeatherReporter already requires Go 1.26 while PromptKit
`v0.8.0` requires Go 1.25.5, so no Go version change is needed for this upgrade.

Consumer validation must cover the paths called out by PromptKit v0.6.0:

- every embedded prompt, content file, schema, and fallback profile inspects
  through PromptKit `v0.8.0`;
- configured single-file and directory profile sources retain their lazy,
  metadata-authoritative identity and precedence behavior;
- malformed selected profiles and invalid local endpoints retain actionable
  WeatherReporter categories;
- the inline YAML data package and prepared-execution path remain within the
  new JSON and response bounds; and
- cancellation, deadline, and backend-capacity identities still cross the
  adapter correctly.

PromptKit owns its 16 MiB successful transport-response bound and JSON framing.
WeatherReporter retains its stricter 64 KiB generated-text acceptance bound.
The consumer suite should protect that relationship without reproducing
PromptKit's lower-level transport matrix.

## Domain Profile Composition

The embedded profiles should become application-owned aliases:

```yaml
id: weather-light
base_profile: deepseek-4-flash
```

```yaml
id: weather-balanced
base_profile: gemini-flash-latest
```

```yaml
id: weather-deep
base_profile: claude-sonnet-latest
```

The effective backend, model, timeout, service tier, and reasoning settings
must initially remain identical to the current WeatherReporter definitions.
The selected leaf remains the durable logical profile identity even though its
effective target is inherited.

Profile source precedence remains:

1. explicit in-memory profiles used by tests or embedding consumers;
2. the configured `profile_file` or `profile_dir` source;
3. WeatherReporter's embedded fallback catalog; and
4. PromptKit's built-in catalog.

Sources still do not merge definitions of the same ID. Once a selected
definition names `base_profile`, however, each parent ID is resolved through
that same precedence order and the resulting linear chain is merged from root
to leaf according to PromptKit's inheritance contract. Documentation must make
that distinction explicit. A malformed leaf, missing or malformed base, cycle,
overlong chain, or incomplete resolved target fails profile inspection before
weather collection.

An operator may continue to replace `weather-light`, `weather-balanced`, or
`weather-deep` with a standalone definition. An operator may also define a
derived replacement. The maintained local endpoint example should remain
standalone because PromptKit profile inheritance has no clearing syntax: using
an OpenRouter base would retain its backend identity and capacity policy even
when the child replaces the endpoint.

WeatherReporter should treat the adapter's successful profile inspection as
authoritative that PromptKit resolved a usable route. A nonblank model remains
required, but backend ID is optional for an endpoint-only profile and should be
omitted from safe provenance where unavailable. WeatherReporter still must not
surface the endpoint outside explicit debug capture. This aligns application
preflight with PromptKit and with the existing CLI, comparison, and debug value
shapes, all of which already permit an absent backend identity.

## Rakestrawhome Availability

The reserved `rakestrawhome` backend and built-in
`rakestrawhome-gemma-4-31b` profile should be supported automatically through
ordinary PromptKit selection. Operators may choose that profile with the
existing global profile setting or as one entry in `compare`, and PromptKit's
backend capacity policy remains authoritative.

WeatherReporter should not register, wrap, or duplicate the backend or profile,
and should not add a Rakestrawhome-specific configuration field. Its canonical
PromptKit integration documentation should link to PromptKit for the current
built-in catalog and credential contract rather than copying volatile endpoint
or capacity values. Offline inspection coverage should prove that the built-in
profile crosses the WeatherReporter adapter with the expected logical backend
identity.

## Bounded Structured-Output Repair

The accepted repair budget belongs to the exact PromptKit output contract, not
to a WeatherReporter retry loop. PromptKit alone should construct corrective
messages, perform additional calls, enforce the budget, aggregate usage, and
coordinate backend capacity. WeatherReporter must not retry provider failures,
switch profiles, or layer another repair mechanism around `RunPrepared`.

All four embedded prompt definitions should declare `repair_attempts: 1`.
Because this changes prompt
execution behavior, latency, cost, hash, and provenance, each definition and
its report-registry binding should advance from exact version `2.0.0` to
`2.1.0`. Prompt text and generated-text schemas do not need to change solely
for this feature.

The embedded prompt definition is the per-report pipeline policy owner. This
upgrade should not add a global or per-report operator configuration field for
repair attempts and should not construct a request-level replacement output
contract. A future pipeline may select another budget only through a deliberate
prompt-definition and exact-version change.

The project-owned PromptKit boundary should retain:

- the configured repair budget in prompt inspection and preparation output
  contracts;
- the number of corrective calls actually made in completed validation;
- cumulative PromptKit token usage across initial and corrective calls; and
- the final candidate and final validation result only, consistent with the
  PromptKit contract.

Prompt inspection and preparation provenance must require the repair budget to
match the exact expected prompt definition just as they currently require the
format, validation mode, and schema path to match. A zero-attempt successful
result is normal when the first candidate passes. A repair-exhausted result
continues through WeatherReporter's ordinary `validation_rejected` failure
path, and an operational or generation failure during correction remains that
profile's ordinary operational failure.

For concurrent comparison, every profile should use the same prompt repair
budget. A corrective call remains part of that profile's one prepared
execution and uses PromptKit's existing backend capacity pool. One profile's
repair or failure must not cancel independent peers.

## Repair Observability And Comparison Contract

The actual repair count is safe operational provenance and should be visible
where WeatherReporter already reports completed validation. Generation, batch,
and comparison action summaries should expose it without exposing candidates,
schemas, or diagnostics. Explicit execution debug output should add it to the
validation object alongside PromptKit's already mapped cumulative usage.

Profile comparison needs this value in `comparison.json`: a successful result
that required correction is materially different from a first-pass success
when evaluating model reliability, latency, and cost. The manifest should
therefore advance to `weatherreporter.comparison.v2` and add a non-negative
`repairAttempts` field to each result. The field is zero when no corrective
call began, including ordinary first-pass success. A failure carries the count
when PromptKit returned a completed validation result; it is omitted only when
execution failed before a completed validation result made the value known.

The v2 manifest should remain flat, strict, deterministic, and atomically
published. WeatherReporter does not need to preserve v1 replacement
compatibility: comparison bundles are operator-owned development outputs, and
the current integration contract intentionally recognizes only its current
schema. The release notes and comparison documentation must call out the
version change so an operator can remove or relocate an older bundle before
using guarded replacement at the same destination.

## Credential Semantics

PromptKit v0.7.0 treats `APIKeyEnv` as an optional lookup source. If the
environment variable is absent or blank and no direct credential is supplied,
the built-in client omits `Authorization` and lets the endpoint respond.
`APIKeyRequired` is the distinct signal that a usable credential must be
provided locally.

WeatherReporter cannot supply PromptKit's request-scoped direct API-key value,
so a profile reporting `APIKeyRequired` remains unsupported and must fail
before weather collection. WeatherReporter should not require a nonblank value
for an optional `APIKeyEnv` during application preflight. The built-in client
should omit `Authorization` when that source is unavailable and let the
endpoint return any authentication failure through the ordinary structured
generation-error path.

## Structured Generation Failures

PromptKit v0.7.0's `GenerationError` can report a provider HTTP status plus
bounded provider code, type, and message. WeatherReporter should consume that
type only inside the PromptKit adapter and map any adopted fields into a
project-owned immutable error. PromptKit dependency types must not become app
or CLI contracts.

HTTP status is safe enough for ordinary diagnostics. Provider code, type, and
message remain untrusted and may contain request or schema fragments. They
must never enter ordinary errors, action summaries, comparison manifests,
logs, generated reports, or Distributor payloads. The full structured
diagnostic belongs only in an explicitly requested secure `--llm-debug-dir`
`failure.json` artifact. That artifact may contain PromptKit's normalized
bounded fields but never the raw provider body, headers, endpoint, credentials,
or reconstructed request.

Initial-call and corrective-call non-2xx responses should follow the same
mapping. Cancellation and deadline categories continue to take precedence over
provider classification where PromptKit preserves those identities.

## Testing Policy

The default suite must remain deterministic, offline, and credential-free.
Use injected PromptKit clients and synthetic embedded or temporary assets for
consumer behavior; do not call OpenRouter, Rakestrawhome, or a local endpoint.

Risk-based coverage should include:

- all embedded prompts and inherited domain profiles inspecting successfully
  under PromptKit `v0.8.0`;
- unchanged effective targets and report-to-profile assignments after the
  alias refactor;
- external standalone and derived profile precedence, plus selected missing,
  cyclic, and malformed-base failures at the WeatherReporter boundary;
- end-to-end preflight and prepared execution through the maintained
  endpoint-only `weather-light` override without exposing its endpoint;
- offline inspection of `rakestrawhome-gemma-4-31b`;
- a first-pass valid result with zero repairs;
- an invalid structured result repaired successfully within one corrective
  call;
- one-attempt exhaustion returning failed validation and no published report;
- a corrective generation failure retaining its safe category and provider
  status policy;
- cumulative usage and actual repair-count mapping;
- comparison peers remaining independent when one profile repairs, exhausts,
  or fails;
- v2 comparison manifest validation and guarded replacement; and
- absent or blank optional `APIKeyEnv` values reaching the provider without an
  `Authorization` header, while `APIKeyRequired` profiles fail preflight.

Do not reproduce PromptKit's internal matrices for path traversal, JSON tree
bounds, response framing, inheritance depth, repair prompt construction, or
provider-detail normalization. WeatherReporter tests should protect only its
adapter mappings, application policy, provenance, publication, and public
contracts. Run ordinary and race-enabled repository tests because the repaired
execution path participates in concurrent comparisons.

## Documentation And Release Impact

Implementation must update each canonical owner whose contract changes:

- `docs/policy/architecture.md` for prompt-execution, credential, diagnostic,
  and comparison invariants;
- `docs/config.md` and the maintained local profile example for profile-source,
  inheritance, and credential semantics;
- `docs/integrations/promptkit.md` for exact prompt versions, repair policy,
  profile composition, Rakestrawhome availability, and safe errors;
- `docs/integrations/comparison-bundle.md` for the v2 manifest and repair count;
- `docs/cli.md` for repair-count fields in action summaries;
- `docs/operations.md` for changed failure behavior and any explicit provider
  diagnostic capture;
- focused internal PromptKit adapter, app orchestration, prompt-debug, and
  comparison documentation; and
- release notes for the dependency jump, prompt version change, possible
  additional model call, credential behavior, profile inheritance, diagnostic
  behavior, and comparison schema change.

Current-state documentation must not describe this behavior until the
implementation lands. PromptKit remains the canonical owner of its complete
built-in catalogs, YAML merge rules, transport limits, corrective-message
construction, and public Go API.

## Scope

The completed feature includes:

- the direct module upgrade and tidy dependency graph;
- a v0.6.0 compatibility audit of WeatherReporter's supported PromptKit paths;
- inherited WeatherReporter domain profile definitions with unchanged
  effective targets;
- correction of application preflight so PromptKit endpoint-only profiles work
  as documented while retaining a required model identity;
- ordinary access to the Rakestrawhome built-in profile;
- PromptKit's optional-credential policy, while direct-key-required profiles
  remain unsupported;
- `repair_attempts` on all operational prompts and exact prompt-version bumps;
- project-owned repair budget, actual-attempt, usage, and provenance mappings;
- repair observability in action summaries, explicit debug output, and a v2
  comparison manifest;
- safe provider HTTP status in ordinary errors and bounded provider detail only
  in explicit secure debug capture;
- focused offline and race-enabled regression coverage; and
- canonical current-state and release documentation updated with the code.

## Non-Goals

This upgrade does not include:

- application-implemented repair prompts or provider transport;
- retries for HTTP, network, timeout, capacity, or other operational failures;
- automatic profile escalation, fallback, ranking, or resampling;
- changing the weather profile ladder, default report assignments, or concrete
  model targets beyond inheriting their maintained PromptKit definitions;
- making Rakestrawhome a default or adding provider-specific configuration;
- live-provider tests or a permanent benchmark framework;
- exposing raw provider responses or sensitive diagnostics routinely;
- a general prompt-source or pipeline plugin system; or
- compatibility shims for PromptKit versions older than `v0.8.0`.

## Completion Criteria

The roadmap is complete when:

- `go.mod` and `go.sum` resolve PromptKit `v0.8.0` without a replacement,
  workspace, or vendor tree;
- all PromptKit v0.6.0 compatibility points relevant to WeatherReporter have
  been checked and valid supported inputs retain project-owned error identity;
- the three WeatherReporter profiles inherit the intended PromptKit built-ins,
  retain their logical IDs, and inspect to the intended effective targets;
- external standalone and inherited overrides obey documented precedence and
  failure behavior;
- the maintained endpoint-only local override passes application preflight,
  retains an empty backend ID, and keeps its endpoint out of ordinary values;
- `rakestrawhome-gemma-4-31b` is selectable through ordinary generation and
  comparison paths without WeatherReporter backend registration;
- every operational prompt has one bounded repair attempt at exact
  version `2.1.0`;
- inspection, preparation, execution, debug, and comparison values accurately
  preserve configured and actual repair counts;
- first-pass success, repaired success, repair exhaustion, repair generation
  failure, and explicit empty content follow the documented outcomes;
- the v2 comparison bundle distinguishes first-pass and repaired results;
- credential preflight accepts absent optional environment credentials while
  rejecting direct-key-required profiles, and provider-error disclosure does
  not leak sensitive values;
- the default test suite is offline and deterministic, ordinary and race
  validation pass, and no redundant upstream implementation suite is copied;
  and
- every implemented contract is documented by its canonical current-state
  owner and disclosed in the eventual release notes.