# PromptKit v0.8.0 Upgrade Implementation Plan Status: Complete. Completion note: Stages 1–9 upgraded PromptKit, adopted inherited profiles and current credential handling, added repair and provider-failure contracts, enabled one corrective generation for v2.1.0 prompts, exposed repair provenance in ordinary and comparison results, migrated comparison bundles to v2, and added secure failure-debug capture. ## Purpose And Authority This plan translates the accepted [PromptKit v0.8.0 upgrade roadmap](promptkit-v0.8.0.md) into an ordered, decision-complete implementation procedure. The feature roadmap owns purpose, scope, policy, and the desired end state. This document owns implementation order, concrete work allocation, stage boundaries, and verification until the upgrade is complete. The implementing agent must complete the stages in numerical order. Each stage is sized for one focused prompt handled by `gpt-5.6-terra` with high reasoning. Do not combine stages merely because adjacent work touches the same package. ## Locked Decisions The following decisions are final for this implementation: - upgrade directly from PromptKit `v0.5.0` to `v0.8.0`; - declare one corrective call in each embedded prompt through `repair_attempts: 1` rather than adding WeatherReporter repair logic; - keep the repair budget in the exact embedded prompt definition and add no global, per-report, CLI, profile, or operator configuration override; - advance all four exact prompt versions from `2.0.0` to `2.1.0`; - make `weather-light`, `weather-balanced`, and `weather-deep` minimal aliases of the corresponding PromptKit built-ins through `base_profile`; - retain the existing report-to-profile assignments and effective model ladder; - allow successfully inspected endpoint-only profiles to have an empty backend ID while continuing to require a nonblank model; - treat `APIKeyEnv` as an optional lookup source and reject only profiles that report `APIKeyRequired`, because WeatherReporter supplies no direct request credential; - support PromptKit's built-in `rakestrawhome-gemma-4-31b` profile without WeatherReporter-specific backend configuration; - expose provider HTTP status through a project-owned safe generation error, while writing provider code, type, and message only to explicit secure debug capture; - emit only `weatherreporter.comparison.v2`, with repair provenance, and do not preserve v1 guarded-replacement support; and - retain PromptKit dependency types inside the adapter and preserve all stateless execution, atomic publication, comparison independence, and disclosure invariants. ## Implementation Rules For every stage: - read `docs/development.md`, all files under `docs/policy/`, this plan, the feature roadmap, and the task-specific documents named by the stage; - inspect the current code and tests before editing; use the repository's code knowledge graph first for code discovery and fall back to text search for literals, assets, and documentation; - implement only the stage's scope and preserve unrelated user changes; - keep PromptKit/provider types, client construction, YAML parsing, repair mechanics, and provider transport inside the existing adapter boundary; - use deterministic, offline, credential-free tests and injected clients or synthetic fixtures rather than live OpenRouter, Rakestrawhome, or local endpoint calls; - add tests at the narrowest stable owner identified by the testing policy and avoid copying PromptKit's internal test matrices; - update the canonical documentation owners listed for that stage in the same change as the implemented contract; - run `gofmt` on changed Go files, the stage's focused tests, `GOWORK=off go test -count=1 ./...`, and `git diff --check`; and - leave the repository passing before proceeding to the next stage. Stages affecting concurrent comparison, cancellation, or secure debug filesystem work must also run the named focused packages with `-race`. Do not weaken an existing assertion solely to accommodate the new dependency. When a test encodes an intentionally changed contract, replace it with a behavioral assertion for the accepted policy. ## Implementation Stages ### Stage 1: Upgrade The Dependency And Establish A v0.8.0 Baseline Status: Complete. Purpose: move to the tagged dependency and isolate compatibility changes before adopting new WeatherReporter behavior. Work: 1. Update `go.mod` to require `gitea.maximumdirect.net/eric/promptkit v0.8.0` and refresh `go.sum` with `GOWORK=off go mod tidy`. Do not add `go.work`, `vendor`, or a `replace` directive and do not change WeatherReporter's Go version. 2. Resolve any compile failures using PromptKit's public root package only. Keep all `Profile` and `OpenAICompatibleProfileConfig` literals keyed. Do not register the now-reserved `rakestrawhome` backend. 3. Reconcile adapter tests that directly exercise PromptKit's changed optional credential behavior. A profile whose only credential metadata is `api_key_env` must reach an injected client when the environment value is absent; it must no longer expect PromptKit to return `ErrAPIKeyEnvMissing`. Do not change WeatherReporter's application preflight in this stage. 4. Verify that every current embedded prompt, content file, schema, and fallback profile inspects under v0.8.0. Verify selected invalid local endpoints and malformed selected profile definitions still map to project-owned configuration or profile-load categories. 5. Review the v0.6.0 compatibility corrections against supported WeatherReporter inputs: metadata-authoritative identities, exact contained `content_file` paths, regular embedded files, structurally valid endpoints, bounded JSON-compatible values, cancellation identity, and strict response framing. Add consumer tests only for a WeatherReporter boundary not already protected by PromptKit. Do not enable profile inheritance or output repair yet. The expected result is the current WeatherReporter feature set running against PromptKit v0.8.0. Focused verification: ```sh GOWORK=off go test -count=1 ./internal/adapters/promptkit ./internal/promptassets GOWORK=off go test -race -count=1 ./internal/adapters/promptkit ``` ### Stage 2: Adopt Profile Inheritance And Current Credential Routing Status: Complete. Purpose: adopt v0.7.0 profile composition, endpoint-only routing, optional credential semantics, and the Rakestrawhome built-in without changing the model ladder. Work: 1. Replace the three embedded profile bodies with these exact leaf/base relationships and no duplicated execution settings: | Leaf | Base | | --- | --- | | `weather-light` | `deepseek-4-flash` | | `weather-balanced` | `gemini-flash-latest` | | `weather-deep` | `claude-sonnet-latest` | 2. Update prompt-asset fixtures and tests to understand `base_profile`. Assert that all three leaf IDs remain selected identities and resolve to the same backend, model, timeout, service tier, and reasoning settings exposed by the current standalone definitions. Test relationships and effective behavior, not copied private constants beyond the intentional model-ladder contract. 3. Preserve source precedence. Cover a standalone same-ID operator override, a derived operator override, a configured source that shadows a base ID, and selected missing-base, cyclic, malformed-base, and incomplete-target failures. Do not implement inheritance or merging in WeatherReporter; all resolution must remain PromptKit-owned. 4. Change application profile preflight to accept a successful inspection with a nonblank model and an empty backend ID. Trust PromptKit inspection to have resolved either a backend or endpoint; do not add the endpoint to `promptexec.ProfileInspection` or ordinary provenance. 5. Remove application-level environment lookup and rejection for a nonblank `APIKeyEnv`. Remove the now-unused `LookupEnv` fields and plumbing from prompt, batch, and comparison inspection requests. Continue rejecting `CredentialRequired`/`APIKeyRequired` before weather collection with the existing missing-credential category. 6. Add an end-to-end offline regression proving the maintained endpoint-only `weather-light` example passes application inspection, retains an empty backend ID, and does not expose its endpoint. 7. Prove `rakestrawhome-gemma-4-31b` can pass ordinary and comparison profile inspection through the existing adapter and reports the PromptKit `rakestrawhome` backend ID. Do not make a provider call or add Rakestrawhome-specific configuration. Canonical documentation in this stage: - update `docs/policy/architecture.md` so only direct-key-required profiles fail credential preflight and backend identity is optional for endpoint-only profiles; - update `docs/config.md` to distinguish same-ID source replacement from `base_profile` chain inheritance and to describe optional environment credentials; - update `docs/integrations/promptkit.md` for profile composition, parent lookup precedence, endpoint-only identity, optional credentials, and Rakestrawhome availability; and - update `docs/internal/promptkit-adapter.md` and focused app internals for the implemented inspection behavior. Focused verification: ```sh GOWORK=off go test -count=1 ./internal/promptassets ./internal/adapters/promptkit ./internal/app GOWORK=off go test -race -count=1 ./internal/adapters/promptkit ./internal/app ``` ### Stage 3: Extend The Project-Owned Prompt Execution Contract Status: Complete. Purpose: establish dependency-neutral repair and structured-generation-error values before the adapter or application relies on them. Work: 1. Add `RepairAttempts int` to `promptexec.OutputContract`. It is the configured additional-call budget from the exact prompt contract. 2. Add `RepairAttempts int` to `promptexec.Validation`. It is the number of corrective calls actually started for the completed result. Update `NewValidation` and every caller so construction is explicit; reject or normalize no values here because PromptKit owns output-contract validity. 3. Update all copy helpers, equality/provenance helpers, fixtures, and tests so repair values are retained without sharing mutable state. 4. Add a project-owned immutable `promptexec.GenerationError` with unexported status and provider-detail fields plus safe accessors: - `StatusCode() int` - `ProviderCode() string` - `ProviderType() string` - `ProviderMessage() string` - `Category() ErrorCategory`, always returning `Generation` - `Error()`, exposing only the WeatherReporter generation category/message and optional HTTP status - `GoString()`, returning the same safe representation - `Unwrap()`, preserving a project-owned `*promptexec.Error` 5. Provide one constructor used by adapters. Defensively normalize valid UTF-8 and bound code/type to 256 Unicode code points and message to 4,096 Unicode code points, even though PromptKit already bounds its accessors. Do not expose fields through struct formatting, JSON tags, or exported mutable fields. Preserve the dependency cause only behind the project-owned error so `errors.Is`/`errors.As` identities remain available without entering error text. 6. Add focused tests proving nil/zero safety, category and unwrap behavior, status-only ordinary formatting, `%#v` redaction, provider-detail bounds, and repair-value copying. Do not import PromptKit from `internal/promptexec` and do not change CLI or artifact schemas in this stage. Focused verification: ```sh GOWORK=off go test -count=1 ./internal/promptexec ``` ### Stage 4: Map PromptKit v0.8.0 Repair And Generation Errors In The Adapter Status: Complete. Purpose: make the adapter faithfully translate v0.8.0 preparation, execution, validation, usage, and failure values into the Stage 3 contract. Work: 1. Map `promptkit.OutputContract.RepairAttempts` in prompt inspection and prepared-execution details. Map `promptkit.ValidationResult.RepairAttempts` in completed execution. 2. Preserve PromptKit's cumulative usage exactly as reported across the initial call and every completed correction. Continue returning only the final raw candidate and final validation result, subject to WeatherReporter's 64 KiB generated-output bound. 3. In adapter error classification, retain cancellation, deadline, and capacity precedence. Before the generic `ErrLLMGenerate` branch, use `errors.As` for `*promptkit.GenerationError` and construct the project-owned `promptexec.GenerationError` with status, code, type, message, and hidden cause. Initial and corrective generation failures use the same mapping. 4. Extend the injected adapter client used by tests so it can return an ordered sequence of responses or errors and record each request safely. 5. Use a synthetic PromptKit prompt with JSON Schema validation and `repair_attempts: 1` to cover: - first-pass valid output with zero corrections; - explicitly empty or invalid output followed by valid corrected output; - one-attempt exhaustion returning a final failed validation result rather than an operational error; - a non-2xx-style `GenerationError` during correction; - cumulative token usage and actual repair count; and - the same prepared prompt/profile identity across the corrective flow. 6. Keep these tests at the adapter boundary. Do not assert PromptKit's private corrective-message wording or reconstruct its internal repair algorithm. Canonical documentation in this stage: update `docs/internal/promptkit-adapter.md` for the repair/result/error mappings. Do not yet claim that embedded WeatherReporter prompts enable repair. Focused verification: ```sh GOWORK=off go test -count=1 ./internal/adapters/promptkit ./internal/promptexec GOWORK=off go test -race -count=1 ./internal/adapters/promptkit ``` ### Stage 5: Carry Repair Provenance Through Application Workflows Status: Complete. Purpose: make application orchestration understand configured and actual repair counts before changing the embedded prompt policy. Work: 1. Add an expected generated-text repair budget to `report.Definition` and set it explicitly to zero for all four current `2.0.0` definitions in this stage. Include it in report-definition validation and retained contract tests. 2. Extend exact prompt preflight so format, validation mode, schema path, and repair budget must all match the resolved report definition. Extend preparation and completion provenance checks to require the same repair budget across inspection and the opaque prepared snapshot. 3. Add `RepairAttempts *int` to application outcomes where execution may fail before validation exists. Set it to a fresh pointer immediately after a non-nil completed execution is returned, before WeatherReporter's secondary generated-text validation. A pointer is required so completed first-pass zero is distinguishable from unavailable provenance. 4. Carry independent copies through `ReportResult`, `BatchReportResult`, batch conversion, comparison execution's internal outcome, and relevant test fakes. Do not expose the new value in CLI or comparison JSON yet. 5. Preserve the actual count on PromptKit validation rejection and on later WeatherReporter generated-text or render failures. Leave it unavailable on preparation, capacity, cancellation, deadline, and generation errors that return no completed PromptKit result. 6. Add focused tests for provenance mismatch, completed zero, completed positive, validation rejection, later local validation failure, early operational failure, batch copying, and independent concurrent profile outcomes. Canonical documentation in this stage: update the focused prepared-report and app-orchestration internals to describe configured versus actual repair provenance. Current public documents should continue to report the embedded budget as zero until Stage 6. Focused verification: ```sh GOWORK=off go test -count=1 ./internal/report ./internal/app ./internal/cli GOWORK=off go test -race -count=1 ./internal/app ``` ### Stage 6: Activate One Repair And Expose Ordinary Result Provenance Status: Complete. Purpose: switch the operational prompts to the accepted one-correction policy and make ordinary generate/run/batch output report what occurred. Work: 1. Add `repair_attempts: 1` to the output contract of all four embedded prompt definitions and change each exact prompt version from `2.0.0` to `2.1.0`. Do not change prompt text or generated-text schemas solely for this upgrade. 2. Change all four report registry definitions to exact prompt version `2.1.0` and expected repair budget one. Update exact-version fixtures and assertions throughout adapter, app, CLI, report, and prompt-asset tests. Remove tests that classify `repair_attempts` as a retired setting and replace them with an exact one-attempt contract assertion. 3. Add `repairAttempts` to successful and failed generate and batch JSON result shapes through the Stage 5 pointers. Emit integer zero for a completed first-pass result, a positive integer for a completed repaired result, and omit the field when no completed validation made it available. 4. Keep the existing `validationStatus` and failure categories authoritative. A repaired valid result proceeds normally. Repair exhaustion remains `validation_rejected`, publishes no report for that profile, and retains the actual attempt count. 5. Add representative offline assembled tests proving first-pass success, repaired success, exhaustion, explicit empty initial content, and batch result propagation. Reuse the real PromptKit adapter with an injected sequence client for at least one end-to-end repaired execution; use the existing app fake at other boundaries where lower-level repair is already covered. 6. Confirm no application loop, provider retry, profile fallback, or request-level `OutputContract` override was introduced. Canonical documentation in this stage: - update `docs/policy/architecture.md` with PromptKit-owned bounded repair and failed-exhaustion invariants; - update `docs/integrations/promptkit.md` with exact prompt version `2.1.0`, one configured repair, actual-count semantics, cumulative usage, explicit-empty handling, and the distinction from operational retries; - update `docs/cli.md` for generate and batch `repairAttempts` fields; - update `docs/internal/report-registry.md`, prepared-report internals, and app orchestration internals for exact version and repair flow; and - keep configuration documentation unchanged because no repair setting is added. Focused verification: ```sh GOWORK=off go test -count=1 ./internal/promptassets ./internal/report ./internal/adapters/promptkit ./internal/app ./internal/cli GOWORK=off go test -race -count=1 ./internal/adapters/promptkit ./internal/app ``` ### Stage 7: Migrate Comparison Bundles To v2 Status: Complete. Purpose: preserve repair activity in the profile-evaluation artifact and make the strict durable schema change explicit. Work: 1. Change `comparison.SchemaVersion` to `weatherreporter.comparison.v2`. Emit and recognize v2 only; do not retain a v1 parser or guarded-replacement compatibility path. 2. Add `RepairAttempts *int` to each application comparison profile result, CLI comparison profile summary, and durable `comparison.Result`. Propagate a fresh copy from Stage 5's execution outcome. 3. Place `repairAttempts` immediately after `validationStatus` in the canonical result-object JSON field order. Encode zero for completed first-pass validation, a positive integer for completed correction, and omit it only when no completed validation exists. 4. Tighten manifest invariants: every non-nil repair count is non-negative; a successful result must have `validationStatus: "passed"` and a non-nil repair count; a failed result with a completed validation status must also have a non-nil count; and an early operational failure may omit both. 5. Update the strict token-level JSON recognizer to accept only the canonical `repairAttempts` field at its correct object level, reject duplicate, unknown, negative, fractional, string, overflow, and malformed values, and continue rejecting v1 as an unsupported current bundle. 6. Update manifest construction, cloning, validation, exact serialization tests, guarded replacement tests, malicious bundle tests, partial-success tests, and CLI comparison summaries. Preserve flat layout, result ordering, hashes, atomic publication, cancellation safety, and no Distributor calls. 7. Cover concurrent peers where one succeeds first-pass, one repairs, one exhausts, and one fails operationally. The counts must remain attached to the selected profile positions without races or cross-contamination. Canonical documentation in this stage: - replace the v1 contract in `docs/integrations/comparison-bundle.md` with v2, including exact field order, presence rules, and the lack of v1 replacement compatibility; - update `docs/cli.md` for comparison `repairAttempts`; - update `docs/operations.md` to tell operators to move or remove an existing v1 bundle before replacing at the same destination; and - update comparison execution/publication internals and architecture policy as needed for the current-only version invariant. Focused verification: ```sh GOWORK=off go test -count=1 ./internal/comparison ./internal/app ./internal/cli GOWORK=off go test -race -count=1 ./internal/comparison ./internal/app ``` ### Stage 8: Add Secure Provider-Failure Debug Capture Status: Complete. Purpose: expose useful PromptKit v0.7.0 provider diagnostics only through the existing explicit secure debug boundary while keeping ordinary errors safe. Work: 1. Extend prompt preparation debug output with configured `repairAttempts` and advance its schema identifier from `weatherreporter.prompt_preparation_debug.v2` to `weatherreporter.prompt_preparation_debug.v3`. 2. Extend execution validation debug output with actual `repairAttempts` and advance its schema identifier from `weatherreporter.prompt_execution_debug.v2` to `weatherreporter.prompt_execution_debug.v3`. Retain cumulative token usage. 3. Add a dedicated `failure.json` artifact with schema identifier `weatherreporter.prompt_failure_debug.v1`. Its canonical fields are: - top level: `schemaVersion`, `reportId`, `validDate`, `runId`, `failure`; - failure object: `category`, `statusCode`, `providerCode`, `providerType`, `providerMessage`; - omit absent provider fields and zero status; and - never include the raw provider body, headers, endpoint, credentials, request, schema, rendered prompt, or generated candidate. 4. Add `PromptDebugWriter.WriteFailure` using the existing handle-relative secure run directory, `0700` directory and `0600` file modes, canonical JSON encoding, and no-follow/atomic replacement behavior. Disabled writers must perform no filesystem work. 5. When execution returns an error, use `errors.As` only against the project-owned `*promptexec.GenerationError`. If explicit debug capture is enabled, write `failure.json` using that profile's existing debug reference. This applies equally to initial and corrective provider failures and keeps comparison profile directories isolated. 6. If failure-debug writing also fails, retain the generation failure as the primary categorized error and join the safe debug-write failure rather than replacing or hiding the provider failure. Never place provider code, type, or message in the joined error text. 7. Ordinary generate, batch, and comparison errors should gain only the safe HTTP status already rendered by `promptexec.GenerationError.Error`; do not add provider detail fields to CLI summaries, comparison manifests, logs, or Distributor requests. 8. Add adversarial tests for formatter redaction, malicious provider strings, JSON escaping, bounds, absent fields, file modes, symlink/path attacks, write failure, cancellation identity, initial versus corrective failures, and concurrent comparison captures. Canonical documentation in this stage: - update `docs/operations.md` with the three debug artifact versions, `failure.json`, sensitivity, permissions, and retention; - update `docs/integrations/promptkit.md` with ordinary status-only disclosure and debug-only provider detail; - update prompt-debug, PromptKit-adapter, and app-orchestration internals; and - ensure `docs/policy/architecture.md` explicitly prohibits provider-controlled diagnostics from ordinary outputs. Focused verification: ```sh GOWORK=off go test -count=1 ./internal/promptexec ./internal/promptdebug ./internal/adapters/promptkit ./internal/app ./internal/cli GOWORK=off go test -race -count=1 ./internal/promptdebug ./internal/adapters/promptkit ./internal/app ``` ### Stage 9: Reconcile Documentation And Perform The Final Upgrade Audit Status: Complete. Purpose: verify the complete end state as one coherent WeatherReporter feature and leave no stale v0.5.0, prompt v2.0.0, comparison v1, credential, profile, repair, or debug claims. Work: 1. Re-read the feature roadmap, `docs/development.md`, every policy document, and every canonical document changed by Stages 1-8. Reconcile them against executable behavior and remove duplicated or stale definitions. Keep unimplemented future ideas in `docs/roadmap/future.md`, not current-state documents. 2. Search code, embedded assets, examples, tests, and documentation for stale contractual literals and review every occurrence of: - PromptKit `v0.5.0`, `v0.6.0`, and `v0.7.0` as an active dependency claim; - prompt version `2.0.0`; - `weatherreporter.comparison.v1`; - prompt debug schema v2 identifiers; - claims that profile fields never inherit; - claims that every `APIKeyEnv` must be populated; - claims that backend ID is always required; - claims that repair is disabled or `repair_attempts` is retired; and - provider detail in ordinary output. Historical release documents may retain accurate historical literals. 3. Verify canonical ownership: - architecture owns invariants and boundaries; - config owns operator profile and credential behavior, but no repair field; - PromptKit integration owns logical prompt/profile/output contracts; - CLI owns result fields; - operations owns explicit debug handling and old comparison-bundle cleanup; - comparison integration owns the complete v2 manifest; and - internal documents own implementation flow without duplicating the public references. 4. Verify maintained examples remain valid, secret-free, and tested. The local `weather-light` example remains a standalone endpoint-only profile rather than inheriting an OpenRouter backend it cannot clear. 5. Review the complete diff for architecture leakage. Production packages outside `internal/adapters/promptkit` must not import PromptKit; no application repair loop, provider client, raw provider diagnostic, profile YAML parser, or durable application state may have appeared. 6. Review tests under the testing policy. Keep consumer contract and regression coverage, remove accidental duplication of upstream implementation tests, and ensure every default test is offline and repeatable. 7. Run the complete validation set: ```sh gofmt -w GOWORK=off go test -count=1 ./... GOWORK=off go test -race -count=1 ./... GOWORK=off go vet ./... GOWORK=off go build ./... GOWORK=off go mod tidy -diff go run ./cmd/weatherreporter --help go run ./cmd/weatherreporter compare --help test -z "$(git ls-files go.work go.work.sum)" test ! -e vendor git diff --check ``` 8. Confirm `go.mod` has no `replace`, the resolved PromptKit module is exactly v0.8.0, and no live credential or provider call occurred during validation. 9. After every check passes, update this plan's status to Completed and add a concise completion note listing the implemented stages. Do not delete either roadmap until the maintainer has reviewed the implementation. Do not create a release document or tag; release preparation remains a separate maintainer action once a version is selected. ## Completion Standard The implementation is complete only when all nine stages pass their focused and repository-wide checks, all locked decisions are observable in code and canonical documentation, and the feature roadmap's completion criteria are satisfied. Passing compilation alone is insufficient. The final state must demonstrate repaired success, repair exhaustion, comparison provenance, endpoint-only routing, optional credentials, inherited profiles, Rakestrawhome inspection, safe ordinary provider failures, secure debug-only detail, and unchanged publication and concurrency invariants.