Plan the PromptKit v0.8.0 upgrade
This commit is contained in:
488
docs/roadmap/promptkit-v0.8.0.md
Normal file
488
docs/roadmap/promptkit-v0.8.0.md
Normal file
@@ -0,0 +1,488 @@
|
||||
# PromptKit v0.8.0 Upgrade Roadmap
|
||||
|
||||
Status: Accepted feature direction; implementation has not started.
|
||||
|
||||
## Purpose
|
||||
|
||||
WeatherReporter should upgrade its PromptKit dependency from `v0.5.0` to
|
||||
`v0.8.0` and deliberately adopt the useful consumer-facing capabilities added
|
||||
in `v0.6.0`, `v0.7.0`, and `v0.8.0`. The upgrade should improve output-contract
|
||||
reliability, profile composition, local and alternate endpoint support, and
|
||||
provider-failure diagnosis without moving PromptKit responsibilities into
|
||||
WeatherReporter or weakening the application's stateless and security
|
||||
boundaries.
|
||||
|
||||
This roadmap defines the intended scope, policy, and end state. The
|
||||
[implementation plan](implementation.md) owns the procedure for reaching that
|
||||
state.
|
||||
|
||||
## User Intent
|
||||
|
||||
The upgrade is intended to:
|
||||
|
||||
- use PromptKit's bounded output repair to recover from occasional malformed
|
||||
structured weather prose;
|
||||
- keep WeatherReporter's domain profile IDs stable while inheriting maintained
|
||||
PromptKit model definitions;
|
||||
- make PromptKit's additional built-in backend and profile available for
|
||||
explicit generation and profile comparisons;
|
||||
- support unauthenticated or optionally authenticated OpenAI-compatible
|
||||
endpoints without inventing a WeatherReporter transport layer;
|
||||
- make provider HTTP failures more actionable under an explicit
|
||||
WeatherReporter disclosure policy; and
|
||||
- receive PromptKit's intervening correctness, safety, cancellation, resource,
|
||||
and efficiency improvements as part of one tested dependency upgrade.
|
||||
|
||||
The model ladder and report assignments do not change as part of this work:
|
||||
Hourly continues to select `weather-light`; Daily, Today, and Tomorrow continue
|
||||
to select `weather-balanced`; and `weather-deep` remains available for explicit
|
||||
selection. This upgrade does not promote the new Rakestrawhome profile into
|
||||
that default ladder.
|
||||
|
||||
## Current State
|
||||
|
||||
WeatherReporter currently depends on
|
||||
`gitea.maximumdirect.net/eric/promptkit` at `v0.5.0`. The PromptKit adapter
|
||||
supplies embedded prompts, JSON Schemas, and
|
||||
application-fallback profiles, plus an optional configured profile source and
|
||||
the conventional local backend.
|
||||
|
||||
The four generated-text prompts are exact version `2.0.0` JSON Schema prompts.
|
||||
They omit `repair_attempts`, so execution is single-pass. The project-owned
|
||||
`promptexec.OutputContract` and validation result also omit repair budgets and
|
||||
actual repair counts.
|
||||
|
||||
The three embedded WeatherReporter profiles duplicate the effective fields of
|
||||
these PromptKit built-ins:
|
||||
|
||||
| WeatherReporter profile | PromptKit built-in with the same target |
|
||||
| --- | --- |
|
||||
| `weather-light` | `deepseek-4-flash` |
|
||||
| `weather-balanced` | `gemini-flash-latest` |
|
||||
| `weather-deep` | `claude-sonnet-latest` |
|
||||
|
||||
WeatherReporter preflights any nonblank `api_key_env` as a required credential,
|
||||
even though PromptKit v0.7.0 distinguishes an optional environment source from
|
||||
an explicit `APIKeyRequired` target. Provider generation failures are reduced
|
||||
to WeatherReporter's safe `generation` category; the PromptKit dependency error
|
||||
is retained as a hidden cause, but its structured HTTP status and provider
|
||||
diagnostics are not mapped into project-owned values.
|
||||
|
||||
The maintained `weather-light` local override is an endpoint-only profile, and
|
||||
the configuration contract says endpoint-only profiles are supported. PromptKit
|
||||
inspection correctly reports no backend ID for that form, but WeatherReporter
|
||||
application preflight currently requires both a nonblank backend and model.
|
||||
That mismatch prevents the documented example from reaching generation and
|
||||
should be corrected as part of adopting the current PromptKit target contract.
|
||||
|
||||
## Upstream Release Assessment
|
||||
|
||||
### PromptKit v0.6.0
|
||||
|
||||
`v0.6.0` adds no public declarations, but it is a material compatibility and
|
||||
safety release. It centralizes execution-setting, output-contract, endpoint,
|
||||
and JSON-compatible-value validation; makes YAML metadata authoritative for
|
||||
prompt and profile identity; hardens `content_file` containment and regular-file
|
||||
requirements; validates OpenAI-compatible endpoints structurally; bounds JSON
|
||||
trees and successful provider bodies; requires exactly one JSON value in
|
||||
provider responses; preserves cancellation and transport error identities; and
|
||||
reuses schema and rendered-artifact work within an operation.
|
||||
|
||||
WeatherReporter should receive these improvements directly from the dependency
|
||||
and audit its own supported assets and configuration paths against the stricter
|
||||
contracts. It should not duplicate PromptKit's internal validators or tests.
|
||||
The existing embedded prompt paths, inline data-package input, generated-output
|
||||
limit, and adapter boundary remain conceptually correct.
|
||||
|
||||
### PromptKit v0.7.0
|
||||
|
||||
`v0.7.0` adds four potentially useful consumer features:
|
||||
|
||||
- linear, cycle-safe profile inheritance through `base_profile` and
|
||||
`Profile.BaseProfileID`;
|
||||
- the built-in `rakestrawhome` backend and
|
||||
`rakestrawhome-gemma-4-31b` profile;
|
||||
- optional API-key environment sources, with `APIKeyRequired` reserved for an
|
||||
explicit local credential requirement; and
|
||||
- bounded structured `GenerationError` details for non-2xx responses from the
|
||||
built-in OpenAI-compatible client.
|
||||
|
||||
WeatherReporter has no manual `rakestrawhome` registration and uses keyed
|
||||
PromptKit profile literals, so the two source-compatibility hazards called out
|
||||
by the release do not require migration shims. The profile, credential, and
|
||||
error features do require deliberate application-policy choices described
|
||||
below.
|
||||
|
||||
### PromptKit v0.8.0
|
||||
|
||||
`v0.8.0` activates the existing output-contract repair budget. A positive
|
||||
`repair_attempts` value authorizes up to that many corrective model calls after
|
||||
eligible `basic`, `json`, or `json_schema` validation failures. The supported
|
||||
budget is zero through three. Repairs preserve the original rendered
|
||||
conversation, effective target, session, structured-output contract, and
|
||||
backend capacity policy. The final result reports cumulative token usage and
|
||||
the number of corrective calls actually made.
|
||||
|
||||
Repair exhaustion is a completed generation with failed validation, not an
|
||||
operational error. WeatherReporter's existing policy should continue to reject
|
||||
that result and publish no report for that profile. Explicit empty provider
|
||||
content now reaches output validation; for WeatherReporter's JSON Schema
|
||||
prompts it is therefore eligible for repair rather than being misclassified as
|
||||
a malformed provider envelope.
|
||||
|
||||
## Desired End State
|
||||
|
||||
WeatherReporter builds and tests against PromptKit `v0.8.0` with no workspace,
|
||||
vendor, or module replacement dependency. Its public behavior remains
|
||||
stateless, its PromptKit dependency types remain confined to the adapter, and
|
||||
its ordinary summaries and logs remain safe.
|
||||
|
||||
The completed integration:
|
||||
|
||||
- benefits from the v0.6.0 safety and efficiency corrections;
|
||||
- composes WeatherReporter domain profiles from PromptKit's maintained built-in
|
||||
profiles while preserving WeatherReporter-owned leaf IDs and operator
|
||||
override precedence;
|
||||
- accepts a successfully inspected endpoint-only profile with a nonblank model
|
||||
even though it has no logical backend ID;
|
||||
- permits explicit use of PromptKit's Rakestrawhome profile without custom
|
||||
backend wiring;
|
||||
- applies an accepted bounded-repair policy to every operational structured
|
||||
prompt;
|
||||
- validates repair configuration during preflight and records actual repair
|
||||
activity in project-owned result values;
|
||||
- retains PromptKit's cumulative usage accounting in explicit debug output;
|
||||
- distinguishes safe provider HTTP status from potentially sensitive provider
|
||||
diagnostics; and
|
||||
- documents the changed profile, credential, repair, comparison, debug, and
|
||||
failure contracts in their canonical owners.
|
||||
|
||||
## Dependency And Compatibility Policy
|
||||
|
||||
The module requirement should move directly from `v0.5.0` to `v0.8.0`, followed
|
||||
by a clean module tidy. WeatherReporter already requires Go 1.26 while PromptKit
|
||||
`v0.8.0` requires Go 1.25.5, so no Go version change is needed for this upgrade.
|
||||
|
||||
Consumer validation must cover the paths called out by PromptKit v0.6.0:
|
||||
|
||||
- every embedded prompt, content file, schema, and fallback profile inspects
|
||||
through PromptKit `v0.8.0`;
|
||||
- configured single-file and directory profile sources retain their lazy,
|
||||
metadata-authoritative identity and precedence behavior;
|
||||
- malformed selected profiles and invalid local endpoints retain actionable
|
||||
WeatherReporter categories;
|
||||
- the inline YAML data package and prepared-execution path remain within the
|
||||
new JSON and response bounds; and
|
||||
- cancellation, deadline, and backend-capacity identities still cross the
|
||||
adapter correctly.
|
||||
|
||||
PromptKit owns its 16 MiB successful transport-response bound and JSON framing.
|
||||
WeatherReporter retains its stricter 64 KiB generated-text acceptance bound.
|
||||
The consumer suite should protect that relationship without reproducing
|
||||
PromptKit's lower-level transport matrix.
|
||||
|
||||
## Domain Profile Composition
|
||||
|
||||
The embedded profiles should become application-owned aliases:
|
||||
|
||||
```yaml
|
||||
id: weather-light
|
||||
base_profile: deepseek-4-flash
|
||||
```
|
||||
|
||||
```yaml
|
||||
id: weather-balanced
|
||||
base_profile: gemini-flash-latest
|
||||
```
|
||||
|
||||
```yaml
|
||||
id: weather-deep
|
||||
base_profile: claude-sonnet-latest
|
||||
```
|
||||
|
||||
The effective backend, model, timeout, service tier, and reasoning settings
|
||||
must initially remain identical to the current WeatherReporter definitions.
|
||||
The selected leaf remains the durable logical profile identity even though its
|
||||
effective target is inherited.
|
||||
|
||||
Profile source precedence remains:
|
||||
|
||||
1. explicit in-memory profiles used by tests or embedding consumers;
|
||||
2. the configured `profile_file` or `profile_dir` source;
|
||||
3. WeatherReporter's embedded fallback catalog; and
|
||||
4. PromptKit's built-in catalog.
|
||||
|
||||
Sources still do not merge definitions of the same ID. Once a selected
|
||||
definition names `base_profile`, however, each parent ID is resolved through
|
||||
that same precedence order and the resulting linear chain is merged from root
|
||||
to leaf according to PromptKit's inheritance contract. Documentation must make
|
||||
that distinction explicit. A malformed leaf, missing or malformed base, cycle,
|
||||
overlong chain, or incomplete resolved target fails profile inspection before
|
||||
weather collection.
|
||||
|
||||
An operator may continue to replace `weather-light`, `weather-balanced`, or
|
||||
`weather-deep` with a standalone definition. An operator may also define a
|
||||
derived replacement. The maintained local endpoint example should remain
|
||||
standalone because PromptKit profile inheritance has no clearing syntax: using
|
||||
an OpenRouter base would retain its backend identity and capacity policy even
|
||||
when the child replaces the endpoint.
|
||||
|
||||
WeatherReporter should treat the adapter's successful profile inspection as
|
||||
authoritative that PromptKit resolved a usable route. A nonblank model remains
|
||||
required, but backend ID is optional for an endpoint-only profile and should be
|
||||
omitted from safe provenance where unavailable. WeatherReporter still must not
|
||||
surface the endpoint outside explicit debug capture. This aligns application
|
||||
preflight with PromptKit and with the existing CLI, comparison, and debug value
|
||||
shapes, all of which already permit an absent backend identity.
|
||||
|
||||
## Rakestrawhome Availability
|
||||
|
||||
The reserved `rakestrawhome` backend and built-in
|
||||
`rakestrawhome-gemma-4-31b` profile should be supported automatically through
|
||||
ordinary PromptKit selection. Operators may choose that profile with the
|
||||
existing global profile setting or as one entry in `compare`, and PromptKit's
|
||||
backend capacity policy remains authoritative.
|
||||
|
||||
WeatherReporter should not register, wrap, or duplicate the backend or profile,
|
||||
and should not add a Rakestrawhome-specific configuration field. Its canonical
|
||||
PromptKit integration documentation should link to PromptKit for the current
|
||||
built-in catalog and credential contract rather than copying volatile endpoint
|
||||
or capacity values. Offline inspection coverage should prove that the built-in
|
||||
profile crosses the WeatherReporter adapter with the expected logical backend
|
||||
identity.
|
||||
|
||||
## Bounded Structured-Output Repair
|
||||
|
||||
The accepted repair budget belongs to the exact PromptKit output contract, not
|
||||
to a WeatherReporter retry loop. PromptKit alone should construct corrective
|
||||
messages, perform additional calls, enforce the budget, aggregate usage, and
|
||||
coordinate backend capacity. WeatherReporter must not retry provider failures,
|
||||
switch profiles, or layer another repair mechanism around `RunPrepared`.
|
||||
|
||||
All four embedded prompt definitions should declare `repair_attempts: 1`.
|
||||
Because this changes prompt
|
||||
execution behavior, latency, cost, hash, and provenance, each definition and
|
||||
its report-registry binding should advance from exact version `2.0.0` to
|
||||
`2.1.0`. Prompt text and generated-text schemas do not need to change solely
|
||||
for this feature.
|
||||
|
||||
The embedded prompt definition is the per-report pipeline policy owner. This
|
||||
upgrade should not add a global or per-report operator configuration field for
|
||||
repair attempts and should not construct a request-level replacement output
|
||||
contract. A future pipeline may select another budget only through a deliberate
|
||||
prompt-definition and exact-version change.
|
||||
|
||||
The project-owned PromptKit boundary should retain:
|
||||
|
||||
- the configured repair budget in prompt inspection and preparation output
|
||||
contracts;
|
||||
- the number of corrective calls actually made in completed validation;
|
||||
- cumulative PromptKit token usage across initial and corrective calls; and
|
||||
- the final candidate and final validation result only, consistent with the
|
||||
PromptKit contract.
|
||||
|
||||
Prompt inspection and preparation provenance must require the repair budget to
|
||||
match the exact expected prompt definition just as they currently require the
|
||||
format, validation mode, and schema path to match. A zero-attempt successful
|
||||
result is normal when the first candidate passes. A repair-exhausted result
|
||||
continues through WeatherReporter's ordinary `validation_rejected` failure
|
||||
path, and an operational or generation failure during correction remains that
|
||||
profile's ordinary operational failure.
|
||||
|
||||
For concurrent comparison, every profile should use the same prompt repair
|
||||
budget. A corrective call remains part of that profile's one prepared
|
||||
execution and uses PromptKit's existing backend capacity pool. One profile's
|
||||
repair or failure must not cancel independent peers.
|
||||
|
||||
## Repair Observability And Comparison Contract
|
||||
|
||||
The actual repair count is safe operational provenance and should be visible
|
||||
where WeatherReporter already reports completed validation. Generation, batch,
|
||||
and comparison action summaries should expose it without exposing candidates,
|
||||
schemas, or diagnostics. Explicit execution debug output should add it to the
|
||||
validation object alongside PromptKit's already mapped cumulative usage.
|
||||
|
||||
Profile comparison needs this value in `comparison.json`: a successful result
|
||||
that required correction is materially different from a first-pass success
|
||||
when evaluating model reliability, latency, and cost. The manifest should
|
||||
therefore advance to `weatherreporter.comparison.v2` and add a non-negative
|
||||
`repairAttempts` field to each result. The field is zero when no corrective
|
||||
call began, including ordinary first-pass success. A failure carries the count
|
||||
when PromptKit returned a completed validation result; it is omitted only when
|
||||
execution failed before a completed validation result made the value known.
|
||||
|
||||
The v2 manifest should remain flat, strict, deterministic, and atomically
|
||||
published. WeatherReporter does not need to preserve v1 replacement
|
||||
compatibility: comparison bundles are operator-owned development outputs, and
|
||||
the current integration contract intentionally recognizes only its current
|
||||
schema. The release notes and comparison documentation must call out the
|
||||
version change so an operator can remove or relocate an older bundle before
|
||||
using guarded replacement at the same destination.
|
||||
|
||||
## Credential Semantics
|
||||
|
||||
PromptKit v0.7.0 treats `APIKeyEnv` as an optional lookup source. If the
|
||||
environment variable is absent or blank and no direct credential is supplied,
|
||||
the built-in client omits `Authorization` and lets the endpoint respond.
|
||||
`APIKeyRequired` is the distinct signal that a usable credential must be
|
||||
provided locally.
|
||||
|
||||
WeatherReporter cannot supply PromptKit's request-scoped direct API-key value,
|
||||
so a profile reporting `APIKeyRequired` remains unsupported and must fail
|
||||
before weather collection. WeatherReporter should not require a nonblank value
|
||||
for an optional `APIKeyEnv` during application preflight. The built-in client
|
||||
should omit `Authorization` when that source is unavailable and let the
|
||||
endpoint return any authentication failure through the ordinary structured
|
||||
generation-error path.
|
||||
|
||||
## Structured Generation Failures
|
||||
|
||||
PromptKit v0.7.0's `GenerationError` can report a provider HTTP status plus
|
||||
bounded provider code, type, and message. WeatherReporter should consume that
|
||||
type only inside the PromptKit adapter and map any adopted fields into a
|
||||
project-owned immutable error. PromptKit dependency types must not become app
|
||||
or CLI contracts.
|
||||
|
||||
HTTP status is safe enough for ordinary diagnostics. Provider code, type, and
|
||||
message remain untrusted and may contain request or schema fragments. They
|
||||
must never enter ordinary errors, action summaries, comparison manifests,
|
||||
logs, generated reports, or Distributor payloads. The full structured
|
||||
diagnostic belongs only in an explicitly requested secure `--llm-debug-dir`
|
||||
`failure.json` artifact. That artifact may contain PromptKit's normalized
|
||||
bounded fields but never the raw provider body, headers, endpoint, credentials,
|
||||
or reconstructed request.
|
||||
|
||||
Initial-call and corrective-call non-2xx responses should follow the same
|
||||
mapping. Cancellation and deadline categories continue to take precedence over
|
||||
provider classification where PromptKit preserves those identities.
|
||||
|
||||
## Testing Policy
|
||||
|
||||
The default suite must remain deterministic, offline, and credential-free.
|
||||
Use injected PromptKit clients and synthetic embedded or temporary assets for
|
||||
consumer behavior; do not call OpenRouter, Rakestrawhome, or a local endpoint.
|
||||
|
||||
Risk-based coverage should include:
|
||||
|
||||
- all embedded prompts and inherited domain profiles inspecting successfully
|
||||
under PromptKit `v0.8.0`;
|
||||
- unchanged effective targets and report-to-profile assignments after the
|
||||
alias refactor;
|
||||
- external standalone and derived profile precedence, plus selected missing,
|
||||
cyclic, and malformed-base failures at the WeatherReporter boundary;
|
||||
- end-to-end preflight and prepared execution through the maintained
|
||||
endpoint-only `weather-light` override without exposing its endpoint;
|
||||
- offline inspection of `rakestrawhome-gemma-4-31b`;
|
||||
- a first-pass valid result with zero repairs;
|
||||
- an invalid structured result repaired successfully within one corrective
|
||||
call;
|
||||
- one-attempt exhaustion returning failed validation and no published report;
|
||||
- a corrective generation failure retaining its safe category and provider
|
||||
status policy;
|
||||
- cumulative usage and actual repair-count mapping;
|
||||
- comparison peers remaining independent when one profile repairs, exhausts,
|
||||
or fails;
|
||||
- v2 comparison manifest validation and guarded replacement; and
|
||||
- absent or blank optional `APIKeyEnv` values reaching the provider without an
|
||||
`Authorization` header, while `APIKeyRequired` profiles fail preflight.
|
||||
|
||||
Do not reproduce PromptKit's internal matrices for path traversal, JSON tree
|
||||
bounds, response framing, inheritance depth, repair prompt construction, or
|
||||
provider-detail normalization. WeatherReporter tests should protect only its
|
||||
adapter mappings, application policy, provenance, publication, and public
|
||||
contracts. Run ordinary and race-enabled repository tests because the repaired
|
||||
execution path participates in concurrent comparisons.
|
||||
|
||||
## Documentation And Release Impact
|
||||
|
||||
Implementation must update each canonical owner whose contract changes:
|
||||
|
||||
- `docs/policy/architecture.md` for prompt-execution, credential, diagnostic,
|
||||
and comparison invariants;
|
||||
- `docs/config.md` and the maintained local profile example for profile-source,
|
||||
inheritance, and credential semantics;
|
||||
- `docs/integrations/promptkit.md` for exact prompt versions, repair policy,
|
||||
profile composition, Rakestrawhome availability, and safe errors;
|
||||
- `docs/integrations/comparison-bundle.md` for the v2 manifest and repair count;
|
||||
- `docs/cli.md` for repair-count fields in action summaries;
|
||||
- `docs/operations.md` for changed failure behavior and any explicit provider
|
||||
diagnostic capture;
|
||||
- focused internal PromptKit adapter, app orchestration, prompt-debug, and
|
||||
comparison documentation; and
|
||||
- release notes for the dependency jump, prompt version change, possible
|
||||
additional model call, credential behavior, profile inheritance, diagnostic
|
||||
behavior, and comparison schema change.
|
||||
|
||||
Current-state documentation must not describe this behavior until the
|
||||
implementation lands. PromptKit remains the canonical owner of its complete
|
||||
built-in catalogs, YAML merge rules, transport limits, corrective-message
|
||||
construction, and public Go API.
|
||||
|
||||
## Scope
|
||||
|
||||
The completed feature includes:
|
||||
|
||||
- the direct module upgrade and tidy dependency graph;
|
||||
- a v0.6.0 compatibility audit of WeatherReporter's supported PromptKit paths;
|
||||
- inherited WeatherReporter domain profile definitions with unchanged
|
||||
effective targets;
|
||||
- correction of application preflight so PromptKit endpoint-only profiles work
|
||||
as documented while retaining a required model identity;
|
||||
- ordinary access to the Rakestrawhome built-in profile;
|
||||
- PromptKit's optional-credential policy, while direct-key-required profiles
|
||||
remain unsupported;
|
||||
- `repair_attempts` on all operational prompts and exact prompt-version bumps;
|
||||
- project-owned repair budget, actual-attempt, usage, and provenance mappings;
|
||||
- repair observability in action summaries, explicit debug output, and a v2
|
||||
comparison manifest;
|
||||
- safe provider HTTP status in ordinary errors and bounded provider detail only
|
||||
in explicit secure debug capture;
|
||||
- focused offline and race-enabled regression coverage; and
|
||||
- canonical current-state and release documentation updated with the code.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This upgrade does not include:
|
||||
|
||||
- application-implemented repair prompts or provider transport;
|
||||
- retries for HTTP, network, timeout, capacity, or other operational failures;
|
||||
- automatic profile escalation, fallback, ranking, or resampling;
|
||||
- changing the weather profile ladder, default report assignments, or concrete
|
||||
model targets beyond inheriting their maintained PromptKit definitions;
|
||||
- making Rakestrawhome a default or adding provider-specific configuration;
|
||||
- live-provider tests or a permanent benchmark framework;
|
||||
- exposing raw provider responses or sensitive diagnostics routinely;
|
||||
- a general prompt-source or pipeline plugin system; or
|
||||
- compatibility shims for PromptKit versions older than `v0.8.0`.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
The roadmap is complete when:
|
||||
|
||||
- `go.mod` and `go.sum` resolve PromptKit `v0.8.0` without a replacement,
|
||||
workspace, or vendor tree;
|
||||
- all PromptKit v0.6.0 compatibility points relevant to WeatherReporter have
|
||||
been checked and valid supported inputs retain project-owned error identity;
|
||||
- the three WeatherReporter profiles inherit the intended PromptKit built-ins,
|
||||
retain their logical IDs, and inspect to the intended effective targets;
|
||||
- external standalone and inherited overrides obey documented precedence and
|
||||
failure behavior;
|
||||
- the maintained endpoint-only local override passes application preflight,
|
||||
retains an empty backend ID, and keeps its endpoint out of ordinary values;
|
||||
- `rakestrawhome-gemma-4-31b` is selectable through ordinary generation and
|
||||
comparison paths without WeatherReporter backend registration;
|
||||
- every operational prompt has one bounded repair attempt at exact
|
||||
version `2.1.0`;
|
||||
- inspection, preparation, execution, debug, and comparison values accurately
|
||||
preserve configured and actual repair counts;
|
||||
- first-pass success, repaired success, repair exhaustion, repair generation
|
||||
failure, and explicit empty content follow the documented outcomes;
|
||||
- the v2 comparison bundle distinguishes first-pass and repaired results;
|
||||
- credential preflight accepts absent optional environment credentials while
|
||||
rejecting direct-key-required profiles, and provider-error disclosure does
|
||||
not leak sensitive values;
|
||||
- the default test suite is offline and deterministic, ordinary and race
|
||||
validation pass, and no redundant upstream implementation suite is copied;
|
||||
and
|
||||
- every implemented contract is documented by its canonical current-state
|
||||
owner and disclosed in the eventual release notes.
|
||||
Reference in New Issue
Block a user