# Explicit Smoke Test Diagnostics Status: accepted target state. Implementation has not started. This roadmap replaces the earlier capability proposal and incorporates the subsequent discussion. It defines the target feature, not a staged implementation plan or current public API. The companion [implementation plan](implementation.md) owns the ordered work and delivery mechanics. ## Purpose And Ownership Provide reusable, explicitly invoked diagnostics for endpoint reachability, authentication, plain-text inference, and structured-output inference. Consumers can build setup and diagnostic workflows without maintaining their own synthetic prompts, schemas, and provider probes. PromptKit is opinionated about what each test checks and how it operates. It owns fixed content, provider adapters, evaluation rules, and neutral factual reports. Consumers select tests, targets, and supported execution settings; they interpret the results and decide what action to take. There is no overall success, application-readiness verdict, or configurable criteria framework. For structured output, report whether structured output was requested, a response was received, generated content was valid JSON, and that JSON matched the fixed test schema. Four positive observations do not prove provider enforcement; no field attempts to infer it. For authentication, distinguish credentials supplied with the dispatched request, a reachable authentication endpoint, and credentials accepted by that operation. A reachable endpoint can reject credentials, and a transport failure can leave acceptance undetermined. Authentication is inapplicable on the maintainer's credential-free RakestrawHome deployment; that is neither an error nor a failure. ## Single-Sprint Delivery Objective The staged implementation must be completable from start to finish in one sprint, including built-in defaults, tests, and documentation. It must not require intermediate human-review approvals, upstream releases, external publication, or live-provider access to progress or establish completion. Automated checks and focused local commits remain part of ordinary implementation. Built-in diagnostic metadata belongs in the local OpenRouter and RakestrawHome catalogs, with their validated revisions identified by local candidate tags. There is no intermediate PromptKit-owned metadata underlay or requirement for separate loader/catalog publication before integration. A temporary, uncommitted Go workspace may support coordinated development. The completed integration uses normal module requirements and Go-generated checksums and is validated with `GOWORK=off`, without filesystem replacements. Exact local tag contents can be supplied through a temporary filesystem module proxy, without public HTTPS availability or upstream publication. Local tags alone do not make Go discover adjacent checkouts. Keep workspace/proxy artifacts outside committed repository contents, following the [repository-independence policy](../policy/architecture.md#repository-independence). PromptKit's completed sprint commits may reference those not-yet-published catalog versions in `go.mod` and their corresponding Go-generated checksums in `go.sum`. Implementation completion and local validation do not claim that other machines can fetch them. Public distribution is separate: consumers eventually need those exact contents available through their configured module sources. Normal release preparation and publication follow implementation separately; this objective does not bypass the repository's release process or authorize publication during the sprint. ## Current Baseline At commit `d4676bc`, PromptKit has no dedicated live diagnostic operation. [`ValidateRequest`](../consumers/pkg-promptkit.md#validate-a-concrete-request) checks a concrete application request without model generation. `Run` executes that request and may structurally validate and repair its output. These operations remain unchanged; smoke tests need not validate an intended application request. The [runner](../internal/runner.md) shares captured prompt/profile selection and execution-target resolution. The [generation coordinator](../internal/generation.md) handles attempts, retries, usage, and active backend permits; the [model client](../internal/llm.md) handles transport and decoding. Ordinary `Run` requires an application prompt and returns no partial result on operational failure, so calling it directly is not sufficient for the proposed diagnostic API. Backend configuration has no diagnostic endpoints, capabilities, or default test profile. New fields must follow [ADR 0001](../adr/0001-configuration-boundary.md): ConfigKit owns loading and composition; PromptKit interprets and validates final effective values. Provenance stays available for explicit inspection and is not an execution-time resolution or validation mechanism. ## Public Operation And Request One synchronous call runs one independently selected operation: ```go func (e *Engine) SmokeTest(ctx context.Context, req SmokeTestRequest) (*SmokeTestReport, error) type SmokeTestKind string const ( SmokeTestReachability SmokeTestKind = "reachability" SmokeTestAuthentication SmokeTestKind = "authentication" SmokeTestTextInference SmokeTestKind = "text_inference" SmokeTestStructuredOutput SmokeTestKind = "structured_output" ) type SmokeTestRequest struct { Kind SmokeTestKind BackendID string PromptID string PromptVersion string ProfileID string Execution *ExecutionTargetOverride APIKey string `json:"-"` TransientRetries *int } ``` The request supplies no variables, inputs, appended messages, schema, or output contract. An application prompt is a profile-selection aid only: its messages, session template, input requirements, and output contract are not executed. Consumers can compose several calls themselves; custom application tests remain available through ordinary `Run`. Reuse existing `ExecutionTargetOverride` semantics. Inference uses its endpoint, model, numeric, reasoning, timeout, credential-source, and provider-extra controls. Backend-only probes use endpoint, credential-source, and timeout settings as applicable; they do not require model completeness or execute generation merely because a model setting was supplied. Profile settings remain the inference baseline unless overridden; no silent cheaper-model substitution is proposed. Direct credentials take precedence over environment credentials and follow existing request-scoped ownership and redacted-formatting conventions. Optional missing/blank credentials are omitted from the upstream request. Authentication can test the resulting unauthenticated request rather than reject missing keys locally. If an explicitly required credential prevents inference dispatch, report that local diagnostic finding without pretending the provider rejected it. Ordinary execution's required-credential behavior remains unchanged. ## Selection And Routing Preserve the accepted inference selection order: | Caller selection | Execution profile | | --- | --- | | Prompt ID only | Selected prompt version's default profile | | Profile ID only | Explicit profile | | Both | Explicit profile overrides the prompt default | | Neither | Explicit backend's configured default smoke-test profile | Reuse exact identity and omitted-version semantics. An invalid explicit selection does not fall back to a backend default. Missing backend/default selection is an error when inference needs it, not a reason to substitute an arbitrary profile. Backend-only probes require neither a default profile nor unrelated prompt data. Whenever inference selects a profile, that profile supplies the resolved endpoint, credential settings, and backend/capacity identity, subject to ordinary execution overrides. This applies equally to explicit profiles, prompt defaults, and backend-default profiles. A supplied `BackendID` is a selection fallback, not a constraint or a request to rebind the selected profile's backend identity. For example, if backend A names a default profile that resolves to backend B, inference uses B's resolved target and capacity identity. If that default profile is endpoint-only, inference retains an empty backend identity. Report A as the default-selection source and B (or the empty identity) as the effective backend. No recursive backend-default lookup or mismatch rejection is introduced. For backend-only probes, an explicit or prompt-selected profile likewise supplies the target and its backend's diagnostic metadata. Otherwise `BackendID` selects the backend directly, without consulting its default inference profile. The profile wins when both selectors are supplied, even when they name different backends. An endpoint-only profile remains usable for inference and reachability without an inferred backend identity or provider-specific authentication adapter. Endpoint overrides retain the effective backend's ordinary capacity identity. Backend-only probes can resolve profile endpoint/credential metadata without requiring its model. Prompt selection uses captured identity/default-profile metadata without rendering or loading the application's artifacts or schemas. ## Report Shapes Use factual Boolean observations with presence, rather than a caller-configurable list of criteria. These Go-facing values need no stable JSON contract initially. ```go type SmokeTestReport struct { Kind SmokeTestKind Supported bool Applicable *bool Completed bool Reason SmokeTestReason Target SmokeTestTarget Facts SmokeTestFacts Timing SmokeTestTiming Attempts []SmokeTestAttempt Usage ObservedTokenUsage Failure ExecutionFailureKind } type SmokeTestTarget struct { BackendID, ProfileID, PromptID, PromptVersion string DefaultSourceBackendID string // backend whose default selected the profile, if any SelectionSource string // "backend_default", "profile", "prompt", or "backend" Endpoint string // initially resolved diagnostic destination EffectiveSettings ExecutionTarget SettingsPresence ExecutionTargetPresence } type SmokeTestFacts struct { CredentialsSupplied *bool EndpointReachable *bool CredentialsAccepted *bool ResponseReceived *bool GenerationCompleted *bool TextReturned *bool ExpectedTextMatched *bool StructuredOutputRequested *bool ValidJSON *bool SchemaMatched *bool } type SmokeTestTiming struct { StartedAt time.Time Duration time.Duration CapacityWait *time.Duration RequestDuration *time.Duration RetryDelay *time.Duration EvaluationDuration *time.Duration } type SmokeTestAttempt struct { Dispatched bool InitialEndpoint string // resolved destination before redirects Endpoint string // last observed request destination; facts refer to this destination HTTPStatus *int Facts SmokeTestFacts Timing SmokeTestTiming Failure ExecutionFailureKind Usage ObservedTokenUsage } type SmokeTestReason string const ( SmokeTestReasonNone SmokeTestReason = "" SmokeTestReasonAuthenticationNotApplicable SmokeTestReason = "authentication_not_applicable" SmokeTestReasonProbeNotConfigured SmokeTestReason = "probe_not_configured" SmokeTestReasonHTTPDiagnosticsUnavailable SmokeTestReason = "http_diagnostics_unavailable" SmokeTestReasonRequiredCredentialMissing SmokeTestReason = "required_credential_missing" SmokeTestReasonAuthenticationResponseUnrecognized SmokeTestReason = "authentication_response_unrecognized" ) ``` `Supported` records whether an implementation is available for the target and selected client. `Applicable=false` records known inapplicability; nil leaves applicability undetermined, for example when no authentication probe is configured. Missing probe metadata does not imply that a provider accepts anonymous access. `Reason` is a named factual code, not application severity or free-form diagnostic text. The constants distinguish, respectively, no additional reason, explicitly inapplicable authentication, absent probe metadata, unavailable HTTP diagnostics for the selected client, a required credential missing before inference dispatch, and an authentication response whose documented meaning cannot be established. HTTP status and `Failure` classify ordinary rejection and transport outcomes; they do not need duplicate reason codes. `Completed` means the diagnostic concluded, not that checks passed. A rejected, failed, unsupported, or inapplicable diagnostic can conclude with nil Go error. Caller cancellation before completion leaves it false and returns available facts. **A nil Go error means the diagnostic concluded, not that the tested operation succeeded.** Make this distinction prominent in GoDoc and consumer examples. A fact is true or false only when established; nil means not evaluated, undetermined, or inapplicable. Operation metadata and failure classification provide context. In particular: - `CredentialsSupplied`: a nonempty credential was attached to the observed dispatched request for the attempt's `Endpoint`, not merely named in configuration or attached to an earlier redirect hop. No observed request leaves this fact absent. For injected clients, do not claim upstream transmission PromptKit cannot observe. Observation at the transport boundary is not proof of wire transmission. - `EndpointReachable`: an HTTP response was received from the tested destination, regardless of status. HTTP 401 therefore yields true. An attempted HTTP operation ending without a response yields false. An injected generation result alone does not establish HTTP reachability. - `CredentialsAccepted`: the designated authentication adapter recognized acceptance or rejection. A rejection yields false; transport failure, server error, or an uninterpretable response can leave it nil. Acceptance does not certify balance, model permissions, or future access. Absence of credentials is not an invalid-key assertion: a documented 401 with `CredentialsSupplied=false` records rejection of an unauthenticated request, not rejection of a supplied key. - `ResponseReceived`: an HTTP response was obtained for built-in operations, or a response was returned by the injected generation boundary. A non-success or malformed HTTP response still counts as received. - `GenerationCompleted`: generation decoded/completed successfully, separately from generated-content checks. Empty decoded content can complete generation. - `TextReturned`: generated content is non-whitespace. `ExpectedTextMatched` compares against the private expected text using trimming only, without case folding or semantic matching. - `StructuredOutputRequested`: the fixed native schema constraint was supplied to the built-in provider request or injected generation boundary. This is not proof of upstream transmission by an injected client or provider enforcement. - `ValidJSON`: the generated content, not the HTTP envelope, parsed as JSON. `SchemaMatched` evaluates parsed JSON against the fixed test schema and remains absent when parsing or response acquisition prevents evaluation. Known inapplicable authentication performs no dispatch and leaves request and acceptance facts absent. Unknown measurements remain unknown, including phase measurements for unperformed work. Report-level capacity wait, request duration, evaluation duration, and retry delay sum the corresponding measured work across the diagnostic. Request duration excludes capacity waiting, evaluation, and retry backoff. Retry delay includes completed and interrupted backoff. Per-attempt timings retain their own measurements; backoff is reported at the operation level. Total duration is wall time, including selection and other orchestration overhead, so it need not equal the phase sum. Timings impose no application latency threshold or universal deadline. If retries are enabled, top-level facts describe the most recent dispatched attempt. Cancellation during backoff or a later capacity wait does not replace those facts with an undispatched attempt's empty facts. The operation's `Failure` and `Completed` still describe the interruption; earlier attempt records retain their own outcomes. A canceled capacity acquisition can add a `Dispatched=false` record, but no synthetic attempt is added for cancellation during backoff. With no dispatched attempt, request facts remain absent. Usage reuses presence-aware accounting without claiming complete provider billing. No facts are inferred from arbitrary error text. `Dispatched` means the HTTP/client boundary was invoked, as in existing attempt observations; it is not proof that packets were sent or billable work occurred. Reports omit private prompts, expected text, schemas, generated content, credentials, raw bodies, and headers. Use content-free failure classifications and HTTP status, not arbitrary provider messages, to preserve this exposure boundary. Effective settings, provider-extra values, endpoints, and identities may still be sensitive operator data; applications own their disclosure. Apply existing credential exclusion and formatting rules rather than promise that all configuration is safe for unrestricted logging. ## Diagnostic Procedures And Capability Configuration ### Reachability Perform one unauthenticated HTTP GET to the configured diagnostic endpoint, or the effective base endpoint when no separate endpoint is supplied. Any HTTP response establishes reachability, including 401, 404, or 405. It does not establish that the completion route, credentials, or a model work. Authentication is tested separately. Classify transport failure where available without promising exhaustive DNS/TLS diagnosis. ### Authentication Metadata names a supported provider adapter or explicitly marks authentication inapplicable. The first adapter is `openrouter_key`, using GET on the selected base endpoint's `key` path with ordinary bearer credential resolution. OpenRouter's [current-key API](https://openrouter.ai/docs/api/api-reference/api-keys/get-current-key) documents this authenticated operation and its 200 and 401 responses. Recognize the documented successful shape and unauthorized response; leave acceptance unknown on other outcomes instead of guessing. Returned account/key details are not exposed. Configure RakestrawHome authentication as `none`, pursuant to the maintainer's stated deployment behavior. Other backends can declare supported adapters. An arbitrary models-list or health endpoint is not credential-acceptance evidence. ### Inference Plain-text inference uses a small fixed private prompt and expected answer. Structured-output inference uses a fixed private prompt and self-contained schema, requests native JSON Schema output through the existing client path, and evaluates JSON and schema conformance locally. It imports no application output contract. Exact content is an implementation detail and does not enter public prompt listings, inspection, configuration inspection, prepared snapshots, or reports. Injected clients and providers necessarily receive the content they execute: privacy here is an API-exposure boundary, not secrecy from executing collaborators. There is no persistent diagnostic prompt or session state. ### Backend Metadata Extend optional backend configuration through the final-value loader and the `BackendConfig`/`BackendSource` convenience. Illustrative shape: ```yaml backends: openrouter: smoke_tests: default_profile: promptkit-openrouter-standard authentication: kind: openrouter_key rakestrawhome: smoke_tests: default_profile: rakestrawhome-gemma-4-31b authentication: kind: none ``` Omitted smoke-test metadata leaves ordinary execution unchanged. Omitted reachability metadata uses the effective base for the generic HTTP probe; omitted authentication metadata makes that operation unsupported, with applicability undetermined. Missing default-profile selection matters when inference selects it, not for backend-only checks or an explicit profile. Retain normal effective presence/null and layering semantics. Validate syntax and recognized configuration at the final handoff, without contacting providers or validating intermediate layers. Provider paths follow the selected endpoint override, not silently the catalog's original endpoint. Explicit absolute diagnostic endpoints are supported for separate health/auth routes and take precedence for their operation. Configuration stays limited to named adapters and endpoint settings, not arbitrary methods or response predicates. The companion implementation plan specifies convenience types and presence/null interpretation for these fields. Reuse the configured HTTP client and its redirect handling, without adding custom credential forwarding. `Target.Endpoint` records the resolved starting destination. Each HTTP attempt records `InitialEndpoint` and the last observed dispatched request destination as `Endpoint`; its credential, reachability, response, and acceptance facts describe that destination. If a redirect leads to a failed request, do not attribute an earlier hop's response or credentials to the later destination. When a client prevents observing the relevant request, leave the affected evidence unknown. Injected generation alone supplies no HTTP destination evidence. Authentication acceptance is interpreted from the response for the observed destination under the selected adapter's contract. It does not assert acceptance at the initial destination or at every redirect hop. These endpoint fields provide attribution without introducing a public redirect trace. Deliver built-in metadata in the local catalog repositories at `../promptkit-backend-openrouter` and `../promptkit-backend-rakestrawhome`. Catalog changes and PromptKit adoption are included in the sprint; publication and remote module availability are not intermediate dependencies. Caller overlays remain available through ordinary composition. Interpret and validate only the assembled final configuration. Built-in reachability follows the effective endpoint rather than pinning a separate catalog endpoint. The catalogs expose diagnostic metadata through an additive, opt-in embedded asset root. Existing `/v3` catalog roots remain usable by older strict loaders; new PromptKit explicitly selects the additional assets. Provider metadata stays owned by its catalog rather than being duplicated in PromptKit. Local candidate tags identify validated contents; they are not published releases. The implementation plan owns candidate selection and local proxy mechanics. The catalogs' publication-date versioning convention still applies to later publication. If that requires new final version names, update PromptKit's pins and checksums accordingly; retain the validated catalog contents and never move a published tag. Selecting the new metadata requires a compatible PromptKit loader. No intervening PromptKit loader release or external module-download verification is required to finish implementation. Use `promptkit-openrouter-standard` for OpenRouter's default, its `openrouter_key` adapter, and `rakestrawhome-gemma-4-31b` with RakestrawHome's `none` authentication setting. Verify each shipped default resolves in the supported assembled effective configuration, including any PromptKit-owned profiles. This is final-value integration validation, not a second validation pass over intermediate sources. A catalog default is replaceable metadata, not a promise of upstream availability, cheap inference, or a stable model choice. ## Execution And Error Boundary Reuse target resolution, generation coordination, HTTP decoding, structural validation, and usage accounting internally. A separate diagnostic orchestration entry point may retain evidence that ordinary `Run` intentionally discards. Do not duplicate inference serialization or build a second inference stack. Add a narrow internal evidence path at the transport/coordinator boundary. Existing execution events do not carry redirect destinations or dispatched credential presence, and the coordinator can discard a completed response when cancellation is observed before return. Capture available facts before that discard and retain them for diagnostics. Keep ordinary execution's error/result contracts unchanged; do not expand public execution events just to reconstruct smoke-test evidence. Inference participates in existing engine-local admission and active backend limits. HTTP-only probes need no generation permit or unrelated model selection. Caller context and configured timeouts govern operations. Inference defaults to one attempt: nil `TransientRetries` means zero for this operation rather than inheritance from the engine retry budget. Positive values reuse existing built-in inference status retries and retain attempt records; injected generation remains single-dispatch. Reachability and authentication remain single-attempt regardless of a valid inference retry setting. Structural repair is excluded from the initial procedure. Otherwise use the selected profile plus explicit overrides. Ordinary engine defaults and repair behavior do not change. Small test content does not establish small output, a price estimate, or a cost ceiling. Keep `LLMClient` unchanged. Injected clients serve inference smoke tests without new required methods. HTTP diagnostics must describe an explicitly selected transport, not secretly probe a different transport in an injected-client engine. For injected-client engines, the initial delivery reports HTTP diagnostics unavailable and adds no separate diagnostic HTTP-client option. Inference diagnostics remain available through the injected client. Known inapplicability, such as authentication configured as `none`, can be reported without any transport. Do not silently fall back to a default HTTP client to diagnose an injected generation client. A future explicit diagnostic transport option remains possible but is not needed to complete this feature. Return conventions: - Invalid kinds/settings, nil engines, or invalid target/profile/prompt selections return ordinary public errors before probe work. Unsupported capability is not the same as invalid selection. - Concluded diagnostics return reports with nil Go error, including credential rejection, provider rejection, transport or configured request-timeout failure, invalid JSON, schema mismatch, unsupported capability, or inapplicability. Local capacity exhaustion is a diagnostic execution finding, not provider failure; request facts remain absent if nothing was dispatched. - Caller cancellation or caller-context deadline expiration returns accumulated observations with an error recognizable through `errors.Is` against `context.Canceled` or `context.DeadlineExceeded`. An already-done context can yield a report with no observations. Do not dispatch again merely to fill missing facts. As elsewhere, arbitrary synchronous injected code must return before cancellation can be observed. The cancellation contract applies to this new operation, not to existing preparation APIs. Internal faults preventing meaningful diagnostic execution may still return an ordinary error; the implementation plan should identify those boundaries without promising that every conceivable failure becomes a report. ## Representative Requests And Results Request examples use ordinary exact selections, without private test language: ```go // Select the application's default profile, but do not run its prompt. req := promptkit.SmokeTestRequest{ Kind: promptkit.SmokeTestTextInference, PromptID: "meeting.summary", } // An explicit profile overrides that default. req.ProfileID = "promptkit-openrouter-standard" // Select a backend's configured default for structured inference. structured := promptkit.SmokeTestRequest{ Kind: promptkit.SmokeTestStructuredOutput, BackendID: "openrouter", } ``` Expected report examples: | Operation/outcome | Observations | Go error | | --- | --- | --- | | Authentication rejected | Supported/applicable/completed; credentials supplied true, endpoint reachable true, response received true, credentials accepted false; attempt status 401 | nil | | RakestrawHome authentication | Supported true, applicable false, completed, `authentication_not_applicable`; no dispatch/acceptance facts | nil | | Unconfigured authentication | Supported false, applicability unknown, completed, `probe_not_configured` | nil | | HTTP diagnostics unavailable for injected client | Supported false, applicability unknown, completed, `http_diagnostics_unavailable`; no dispatch | nil | | Authentication without credentials receives documented 401 | Supported/applicable/completed; credentials supplied false, endpoint reachable and response received true, credentials accepted false; rejection describes an unauthenticated request | nil | | Required credential absent before inference | Supported/applicable/completed; `required_credential_missing`, failure `invalid_request`; no dispatch and no request facts | nil | | Authentication receives uninterpretable HTTP 200 | Supported/applicable/completed; endpoint reachable and response received true, credential supply recorded, acceptance unknown; `authentication_response_unrecognized`, failure `malformed_response` | nil | | Text inference returns unexpected nonempty text | Generation completed and text returned true; expected-text match false | nil | | Structured inference matches schema | Structured output requested, response received, valid JSON, and schema matched true; no provider-enforcement assertion | nil | | Authentication transport fails | Endpoint reachable/response received false; credential supply recorded when known; acceptance nil | nil | | Configured request timeout while caller context remains active | Supported/applicable/completed; failure `deadline`; retain observed response facts, or record response/reachability false for observed HTTP dispatch without a response; injected clients do not establish HTTP facts | nil | | Local capacity exhausted | Supported/applicable/completed; failure `capacity`; no dispatch or request facts | nil | | Caller cancels during initial capacity wait | Supported/applicable, completed false; failure `canceled`, measured capacity wait, undispatched attempt; request facts absent | `context.Canceled` identity | | Caller deadline expires during a dispatched request | Supported/applicable, completed false; failure `deadline`; retain available attempt evidence | `context.DeadlineExceeded` identity | | Caller cancels during retry backoff after HTTP 503 | Completed false, failure `canceled`; top-level facts retain the last dispatched attempt, its 503 remains in attempts, interrupted retry delay recorded, no extra attempt | `context.Canceled` identity | | Caller cancels | Available observations retained; completed false when interrupted | Recognizable context error | Consumers decide whether any of these findings warrant success, a warning, or a failure in their own application. They own CLI diagnostics, exit statuses, scheduling, and any combination with `ValidateRequest` or normal execution. ## Work Needed And Target Completion - Add the public request/report operation and GoDoc, keeping business logic internal. - Extend final backend metadata interpretation and captured lookup for capabilities and defaults. Preserve explicit configuration inspection without exposing test content or using source history for validation. - Connect private inference content to shared selection, execution resolution, capacity, generation, decoding, and structural evaluation without preparing application inputs or messages. - Add narrow HTTP probes, credential handling, evidence retention, factual results, and partial cancellation reporting through the internal evidence path while preserving existing client contracts. Specify destination/credential attribution without exposing headers or a redirect trace. - Deliver the agreed metadata through both local catalogs with ordinary catalog/caller precedence and no provider-availability guarantee. Shipped defaults resolve in the assembled final configuration, and selected versions work without workspace/replacement substitution. The single-sprint objective applies to the complete integration. - Add focused offline tests with controlled transports and injected clients for selection, rejected/absent credentials, applicability/support, transport/malformed responses, content/JSON/schema facts, cancellation, shared inference capacity, and credential/private-content exposure. Assign distinct test owners; do not repeat existing transport, capacity, or validation matrices or snapshot private test language. - Include the report-state matrix, cross-backend and endpoint-only default profiles, redirect attribution, cancellation after a response, and interrupted retry waits in focused contract coverage. Reuse existing lower-level checks where sufficient. - Update canonical consumer, backend-format, integration, and internal documentation when implementation lands, with release notes explaining the optional addition. Live diagnostics are explicit consumer operations, not default tests or new release gates. Completion means consumers can select the four kinds, obtain supported factual observations, distinguish false from unknown/inapplicable, and retain evidence on cancellation. Existing preparation and execution continue unchanged. Initial scope excludes application CLI/service work, readiness policy, continuous monitoring, model-status tests, batch orchestration, customizable test language, cost estimates, and guarantees about later generation.