weatherreporter/docs/roadmap/promptkit-v0.8.0.md

PromptKit v0.8.0 Upgrade Roadmap

Status: Implemented.

Purpose

WeatherReporter should upgrade its PromptKit dependency from v0.5.0 to v0.8.0 and deliberately adopt the useful consumer-facing capabilities added in v0.6.0, v0.7.0, and v0.8.0. The upgrade should improve output-contract reliability, profile composition, local and alternate endpoint support, and provider-failure diagnosis without moving PromptKit responsibilities into WeatherReporter or weakening the application's stateless and security boundaries.

This roadmap defines the intended scope, policy, and end state. The implementation plan owns the procedure for reaching that state.

User Intent

The upgrade is intended to:

• use PromptKit's bounded output repair to recover from occasional malformed structured weather prose;
• keep WeatherReporter's domain profile IDs stable while inheriting maintained PromptKit model definitions;
• make PromptKit's additional built-in backend and profile available for explicit generation and profile comparisons;
• support unauthenticated or optionally authenticated OpenAI-compatible endpoints without inventing a WeatherReporter transport layer;
• make provider HTTP failures more actionable under an explicit WeatherReporter disclosure policy; and
• receive PromptKit's intervening correctness, safety, cancellation, resource, and efficiency improvements as part of one tested dependency upgrade.

The model ladder and report assignments do not change as part of this work: Hourly continues to select weather-light; Daily, Today, and Tomorrow continue to select weather-balanced; and weather-deep remains available for explicit selection. This upgrade does not promote the new Rakestrawhome profile into that default ladder.

Current State

WeatherReporter currently depends on gitea.maximumdirect.net/eric/promptkit at v0.5.0. The PromptKit adapter supplies embedded prompts, JSON Schemas, and application-fallback profiles, plus an optional configured profile source and the conventional local backend.

The four generated-text prompts are exact version 2.0.0 JSON Schema prompts. They omit repair_attempts, so execution is single-pass. The project-owned promptexec.OutputContract and validation result also omit repair budgets and actual repair counts.

The three embedded WeatherReporter profiles duplicate the effective fields of these PromptKit built-ins:

WeatherReporter profile	PromptKit built-in with the same target
weather-light	deepseek-4-flash
weather-balanced	gemini-flash-latest
weather-deep	claude-sonnet-latest

WeatherReporter preflights any nonblank api_key_env as a required credential, even though PromptKit v0.7.0 distinguishes an optional environment source from an explicit APIKeyRequired target. Provider generation failures are reduced to WeatherReporter's safe generation category; the PromptKit dependency error is retained as a hidden cause, but its structured HTTP status and provider diagnostics are not mapped into project-owned values.

The maintained weather-light local override is an endpoint-only profile, and the configuration contract says endpoint-only profiles are supported. PromptKit inspection correctly reports no backend ID for that form, but WeatherReporter application preflight currently requires both a nonblank backend and model. That mismatch prevents the documented example from reaching generation and should be corrected as part of adopting the current PromptKit target contract.

Upstream Release Assessment

PromptKit v0.6.0

v0.6.0 adds no public declarations, but it is a material compatibility and safety release. It centralizes execution-setting, output-contract, endpoint, and JSON-compatible-value validation; makes YAML metadata authoritative for prompt and profile identity; hardens content_file containment and regular-file requirements; validates OpenAI-compatible endpoints structurally; bounds JSON trees and successful provider bodies; requires exactly one JSON value in provider responses; preserves cancellation and transport error identities; and reuses schema and rendered-artifact work within an operation.

WeatherReporter should receive these improvements directly from the dependency and audit its own supported assets and configuration paths against the stricter contracts. It should not duplicate PromptKit's internal validators or tests. The existing embedded prompt paths, inline data-package input, generated-output limit, and adapter boundary remain conceptually correct.

PromptKit v0.7.0

v0.7.0 adds four potentially useful consumer features:

• linear, cycle-safe profile inheritance through base_profile and Profile.BaseProfileID;
• the built-in rakestrawhome backend and rakestrawhome-gemma-4-31b profile;
• optional API-key environment sources, with APIKeyRequired reserved for an explicit local credential requirement; and
• bounded structured GenerationError details for non-2xx responses from the built-in OpenAI-compatible client.

WeatherReporter has no manual rakestrawhome registration and uses keyed PromptKit profile literals, so the two source-compatibility hazards called out by the release do not require migration shims. The profile, credential, and error features do require deliberate application-policy choices described below.

PromptKit v0.8.0

v0.8.0 activates the existing output-contract repair budget. A positive repair_attempts value authorizes up to that many corrective model calls after eligible basic, json, or json_schema validation failures. The supported budget is zero through three. Repairs preserve the original rendered conversation, effective target, session, structured-output contract, and backend capacity policy. The final result reports cumulative token usage and the number of corrective calls actually made.

Repair exhaustion is a completed generation with failed validation, not an operational error. WeatherReporter's existing policy should continue to reject that result and publish no report for that profile. Explicit empty provider content now reaches output validation; for WeatherReporter's JSON Schema prompts it is therefore eligible for repair rather than being misclassified as a malformed provider envelope.

Desired End State

WeatherReporter builds and tests against PromptKit v0.8.0 with no workspace, vendor, or module replacement dependency. Its public behavior remains stateless, its PromptKit dependency types remain confined to the adapter, and its ordinary summaries and logs remain safe.

The completed integration:

• benefits from the v0.6.0 safety and efficiency corrections;
• composes WeatherReporter domain profiles from PromptKit's maintained built-in profiles while preserving WeatherReporter-owned leaf IDs and operator override precedence;
• accepts a successfully inspected endpoint-only profile with a nonblank model even though it has no logical backend ID;
• permits explicit use of PromptKit's Rakestrawhome profile without custom backend wiring;
• applies an accepted bounded-repair policy to every operational structured prompt;
• validates repair configuration during preflight and records actual repair activity in project-owned result values;
• retains PromptKit's cumulative usage accounting in explicit debug output;
• distinguishes safe provider HTTP status from potentially sensitive provider diagnostics; and
• documents the changed profile, credential, repair, comparison, debug, and failure contracts in their canonical owners.

Dependency And Compatibility Policy

The module requirement should move directly from v0.5.0 to v0.8.0, followed by a clean module tidy. WeatherReporter already requires Go 1.26 while PromptKit v0.8.0 requires Go 1.25.5, so no Go version change is needed for this upgrade.

Consumer validation must cover the paths called out by PromptKit v0.6.0:

• every embedded prompt, content file, schema, and fallback profile inspects through PromptKit v0.8.0;
• configured single-file and directory profile sources retain their lazy, metadata-authoritative identity and precedence behavior;
• malformed selected profiles and invalid local endpoints retain actionable WeatherReporter categories;
• the inline YAML data package and prepared-execution path remain within the new JSON and response bounds; and
• cancellation, deadline, and backend-capacity identities still cross the adapter correctly.

PromptKit owns its 16 MiB successful transport-response bound and JSON framing. WeatherReporter retains its stricter 64 KiB generated-text acceptance bound. The consumer suite should protect that relationship without reproducing PromptKit's lower-level transport matrix.

Domain Profile Composition

The embedded profiles should become application-owned aliases:

id: weather-light
base_profile: deepseek-4-flash

id: weather-balanced
base_profile: gemini-flash-latest

id: weather-deep
base_profile: claude-sonnet-latest

The effective backend, model, timeout, service tier, and reasoning settings must initially remain identical to the current WeatherReporter definitions. The selected leaf remains the durable logical profile identity even though its effective target is inherited.

Profile source precedence remains:

1. explicit in-memory profiles used by tests or embedding consumers;
2. the configured profile_file or profile_dir source;
3. WeatherReporter's embedded fallback catalog; and
4. PromptKit's built-in catalog.

Sources still do not merge definitions of the same ID. Once a selected definition names base_profile, however, each parent ID is resolved through that same precedence order and the resulting linear chain is merged from root to leaf according to PromptKit's inheritance contract. Documentation must make that distinction explicit. A malformed leaf, missing or malformed base, cycle, overlong chain, or incomplete resolved target fails profile inspection before weather collection.

An operator may continue to replace weather-light, weather-balanced, or weather-deep with a standalone definition. An operator may also define a derived replacement. The maintained local endpoint example should remain standalone because PromptKit profile inheritance has no clearing syntax: using an OpenRouter base would retain its backend identity and capacity policy even when the child replaces the endpoint.

WeatherReporter should treat the adapter's successful profile inspection as authoritative that PromptKit resolved a usable route. A nonblank model remains required, but backend ID is optional for an endpoint-only profile and should be omitted from safe provenance where unavailable. WeatherReporter still must not surface the endpoint outside explicit debug capture. This aligns application preflight with PromptKit and with the existing CLI, comparison, and debug value shapes, all of which already permit an absent backend identity.

Rakestrawhome Availability

The reserved rakestrawhome backend and built-in rakestrawhome-gemma-4-31b profile should be supported automatically through ordinary PromptKit selection. Operators may choose that profile with the existing global profile setting or as one entry in compare, and PromptKit's backend capacity policy remains authoritative.

WeatherReporter should not register, wrap, or duplicate the backend or profile, and should not add a Rakestrawhome-specific configuration field. Its canonical PromptKit integration documentation should link to PromptKit for the current built-in catalog and credential contract rather than copying volatile endpoint or capacity values. Offline inspection coverage should prove that the built-in profile crosses the WeatherReporter adapter with the expected logical backend identity.

Bounded Structured-Output Repair

The accepted repair budget belongs to the exact PromptKit output contract, not to a WeatherReporter retry loop. PromptKit alone should construct corrective messages, perform additional calls, enforce the budget, aggregate usage, and coordinate backend capacity. WeatherReporter must not retry provider failures, switch profiles, or layer another repair mechanism around RunPrepared.

All four embedded prompt definitions should declare repair_attempts: 1. Because this changes prompt execution behavior, latency, cost, hash, and provenance, each definition and its report-registry binding should advance from exact version 2.0.0 to 2.1.0. Prompt text and generated-text schemas do not need to change solely for this feature.

The embedded prompt definition is the per-report pipeline policy owner. This upgrade should not add a global or per-report operator configuration field for repair attempts and should not construct a request-level replacement output contract. A future pipeline may select another budget only through a deliberate prompt-definition and exact-version change.

The project-owned PromptKit boundary should retain:

• the configured repair budget in prompt inspection and preparation output contracts;
• the number of corrective calls actually made in completed validation;
• cumulative PromptKit token usage across initial and corrective calls; and
• the final candidate and final validation result only, consistent with the PromptKit contract.

Prompt inspection and preparation provenance must require the repair budget to match the exact expected prompt definition just as they currently require the format, validation mode, and schema path to match. A zero-attempt successful result is normal when the first candidate passes. A repair-exhausted result continues through WeatherReporter's ordinary validation_rejected failure path, and an operational or generation failure during correction remains that profile's ordinary operational failure.

For concurrent comparison, every profile should use the same prompt repair budget. A corrective call remains part of that profile's one prepared execution and uses PromptKit's existing backend capacity pool. One profile's repair or failure must not cancel independent peers.

Repair Observability And Comparison Contract

The actual repair count is safe operational provenance and should be visible where WeatherReporter already reports completed validation. Generation, batch, and comparison action summaries should expose it without exposing candidates, schemas, or diagnostics. Explicit execution debug output should add it to the validation object alongside PromptKit's already mapped cumulative usage.

Profile comparison needs this value in comparison.json: a successful result that required correction is materially different from a first-pass success when evaluating model reliability, latency, and cost. The manifest should therefore advance to weatherreporter.comparison.v2 and add a non-negative repairAttempts field to each result. The field is zero when no corrective call began, including ordinary first-pass success. A failure carries the count when PromptKit returned a completed validation result; it is omitted only when execution failed before a completed validation result made the value known.

The v2 manifest should remain flat, strict, deterministic, and atomically published. WeatherReporter does not need to preserve v1 replacement compatibility: comparison bundles are operator-owned development outputs, and the current integration contract intentionally recognizes only its current schema. The release notes and comparison documentation must call out the version change so an operator can remove or relocate an older bundle before using guarded replacement at the same destination.

Credential Semantics

PromptKit v0.7.0 treats APIKeyEnv as an optional lookup source. If the environment variable is absent or blank and no direct credential is supplied, the built-in client omits Authorization and lets the endpoint respond. APIKeyRequired is the distinct signal that a usable credential must be provided locally.

WeatherReporter cannot supply PromptKit's request-scoped direct API-key value, so a profile reporting APIKeyRequired remains unsupported and must fail before weather collection. WeatherReporter should not require a nonblank value for an optional APIKeyEnv during application preflight. The built-in client should omit Authorization when that source is unavailable and let the endpoint return any authentication failure through the ordinary structured generation-error path.

Structured Generation Failures

PromptKit v0.7.0's GenerationError can report a provider HTTP status plus bounded provider code, type, and message. WeatherReporter should consume that type only inside the PromptKit adapter and map any adopted fields into a project-owned immutable error. PromptKit dependency types must not become app or CLI contracts.

HTTP status is safe enough for ordinary diagnostics. Provider code, type, and message remain untrusted and may contain request or schema fragments. They must never enter ordinary errors, action summaries, comparison manifests, logs, generated reports, or Distributor payloads. The full structured diagnostic belongs only in an explicitly requested secure --llm-debug-dir failure.json artifact. That artifact may contain PromptKit's normalized bounded fields but never the raw provider body, headers, endpoint, credentials, or reconstructed request.

Initial-call and corrective-call non-2xx responses should follow the same mapping. Cancellation and deadline categories continue to take precedence over provider classification where PromptKit preserves those identities.

Testing Policy

The default suite must remain deterministic, offline, and credential-free. Use injected PromptKit clients and synthetic embedded or temporary assets for consumer behavior; do not call OpenRouter, Rakestrawhome, or a local endpoint.

Risk-based coverage should include:

• all embedded prompts and inherited domain profiles inspecting successfully under PromptKit v0.8.0;
• unchanged effective targets and report-to-profile assignments after the alias refactor;
• external standalone and derived profile precedence, plus selected missing, cyclic, and malformed-base failures at the WeatherReporter boundary;
• end-to-end preflight and prepared execution through the maintained endpoint-only weather-light override without exposing its endpoint;
• offline inspection of rakestrawhome-gemma-4-31b;
• a first-pass valid result with zero repairs;
• an invalid structured result repaired successfully within one corrective call;
• one-attempt exhaustion returning failed validation and no published report;
• a corrective generation failure retaining its safe category and provider status policy;
• cumulative usage and actual repair-count mapping;
• comparison peers remaining independent when one profile repairs, exhausts, or fails;
• v2 comparison manifest validation and guarded replacement; and
• absent or blank optional APIKeyEnv values reaching the provider without an Authorization header, while APIKeyRequired profiles fail preflight.

Do not reproduce PromptKit's internal matrices for path traversal, JSON tree bounds, response framing, inheritance depth, repair prompt construction, or provider-detail normalization. WeatherReporter tests should protect only its adapter mappings, application policy, provenance, publication, and public contracts. Run ordinary and race-enabled repository tests because the repaired execution path participates in concurrent comparisons.

Documentation And Release Impact

Implementation must update each canonical owner whose contract changes:

• docs/policy/architecture.md for prompt-execution, credential, diagnostic, and comparison invariants;
• docs/config.md and the maintained local profile example for profile-source, inheritance, and credential semantics;
• docs/integrations/promptkit.md for exact prompt versions, repair policy, profile composition, Rakestrawhome availability, and safe errors;
• docs/integrations/comparison-bundle.md for the v2 manifest and repair count;
• docs/cli.md for repair-count fields in action summaries;
• docs/operations.md for changed failure behavior and any explicit provider diagnostic capture;
• focused internal PromptKit adapter, app orchestration, prompt-debug, and comparison documentation; and
• release notes for the dependency jump, prompt version change, possible additional model call, credential behavior, profile inheritance, diagnostic behavior, and comparison schema change.

Current-state documentation must not describe this behavior until the implementation lands. PromptKit remains the canonical owner of its complete built-in catalogs, YAML merge rules, transport limits, corrective-message construction, and public Go API.

Scope

The completed feature includes:

• the direct module upgrade and tidy dependency graph;
• a v0.6.0 compatibility audit of WeatherReporter's supported PromptKit paths;
• inherited WeatherReporter domain profile definitions with unchanged effective targets;
• correction of application preflight so PromptKit endpoint-only profiles work as documented while retaining a required model identity;
• ordinary access to the Rakestrawhome built-in profile;
• PromptKit's optional-credential policy, while direct-key-required profiles remain unsupported;
• repair_attempts on all operational prompts and exact prompt-version bumps;
• project-owned repair budget, actual-attempt, usage, and provenance mappings;
• repair observability in action summaries, explicit debug output, and a v2 comparison manifest;
• safe provider HTTP status in ordinary errors and bounded provider detail only in explicit secure debug capture;
• focused offline and race-enabled regression coverage; and
• canonical current-state and release documentation updated with the code.

Non-Goals

This upgrade does not include:

• application-implemented repair prompts or provider transport;
• retries for HTTP, network, timeout, capacity, or other operational failures;
• automatic profile escalation, fallback, ranking, or resampling;
• changing the weather profile ladder, default report assignments, or concrete model targets beyond inheriting their maintained PromptKit definitions;
• making Rakestrawhome a default or adding provider-specific configuration;
• live-provider tests or a permanent benchmark framework;
• exposing raw provider responses or sensitive diagnostics routinely;
• a general prompt-source or pipeline plugin system; or
• compatibility shims for PromptKit versions older than v0.8.0.

Completion Criteria

The roadmap is complete when:

• go.mod and go.sum resolve PromptKit v0.8.0 without a replacement, workspace, or vendor tree;
• all PromptKit v0.6.0 compatibility points relevant to WeatherReporter have been checked and valid supported inputs retain project-owned error identity;
• the three WeatherReporter profiles inherit the intended PromptKit built-ins, retain their logical IDs, and inspect to the intended effective targets;
• external standalone and inherited overrides obey documented precedence and failure behavior;
• the maintained endpoint-only local override passes application preflight, retains an empty backend ID, and keeps its endpoint out of ordinary values;
• rakestrawhome-gemma-4-31b is selectable through ordinary generation and comparison paths without WeatherReporter backend registration;
• every operational prompt has one bounded repair attempt at exact version 2.1.0;
• inspection, preparation, execution, debug, and comparison values accurately preserve configured and actual repair counts;
• first-pass success, repaired success, repair exhaustion, repair generation failure, and explicit empty content follow the documented outcomes;
• the v2 comparison bundle distinguishes first-pass and repaired results;
• credential preflight accepts absent optional environment credentials while rejecting direct-key-required profiles, and provider-error disclosure does not leak sensitive values;
• the default test suite is offline and deterministic, ordinary and race validation pass, and no redundant upstream implementation suite is copied; and
• every implemented contract is documented by its canonical current-state owner and disclosed in the eventual release notes.