promptkit/docs/roadmap/implementation.md

# Smoke Test Diagnostics Implementation Plan

Status: ready for implementation.

Implement the accepted [feature roadmap](smoke-tests.md). This document owns the
implementation sequence; the feature roadmap owns the requested API, factual
reporting semantics, selection rules, and scope. Each numbered stage is a single
bounded prompt for a Luna coding agent. Execute stages in order within one sprint.
There are no intermediate human-review, external-publication, or live-provider
gates. Focused automated validation and local commits are ordinary development
steps, not pauses awaiting approval.

## Baseline And Working Rules

The reviewed PromptKit baseline is `c8f9a4def9f84301bf534a331320ea9fb798ac24`.
It selects both `/v3` catalogs at `v3.20260930.0` and ConfigKit at `v0.7.1`.
The adjacent checkouts are:

- `../promptkit-backend-openrouter`, reviewed at
  `f17d208cb7ad1fb043cda2266cd3011d11b40aa0`;
- `../promptkit-backend-rakestrawhome`, reviewed at
  `993443d88c451901f4c917fa00c204fd3b54d28d`.

Recheck repository status before editing and preserve unrelated work. Begin with
[development orientation](../development.md), then the applicable
[agent](../policy/agents.md), [architecture](../policy/architecture.md),
[testing](../policy/testing.md), [documentation](../policy/documentation.md),
[OSS](../policy/oss.md), and [release](../policy/release.md) policies. Read each
catalog's own instructions and release procedure before its stages. Reuse
unchanged policy readings during the sprint.

Read focused internal documents and owning tests before modifying a component.
Use existing code-discovery guidance. Stage and commit each completed, nonempty
unit with a short plain-English message after proportionate checks and staged-diff
review. Record stage completion and relevant validation here. Catalog stages
commit in their own repositories; PromptKit records their tag and full commit
identities here. Do not push any repository or publish a PromptKit release during
this implementation sprint.

Only final effective ConfigKit values are interpreted or validated by PromptKit,
per [ADR 0001](../adr/0001-configuration-boundary.md). Do not inspect raw catalog
documents or source history to enforce domain validity, even in PromptKit tests.
The catalog repositories may test the assets they themselves own. Operator
inspection continues to delegate to ConfigKit.

Keep existing `Run`, `Prepare`, `ValidateRequest`, repair, observation, and client
contracts unchanged. Add no application CLI, readiness verdict, custom test
criteria, batch operation, live release gate, or new public transport injection
option. Do not add catalog exclusion controls absent from the current API.

## Implementation Decisions

These decisions resolve the feature roadmap's delivery details. Internal names
below identify a concrete implementation path, not additional public contracts.

### Configuration And Catalog Assets

Add optional `smoke_tests` to final backend definitions:

```yaml
backends:
  openrouter:
    smoke_tests:
      default_profile: promptkit-openrouter-standard
      authentication:
        kind: openrouter_key
  rakestrawhome:
    smoke_tests:
      default_profile: rakestrawhome-gemma-4-31b
      authentication:
        kind: none
```

Both catalogs omit a separate reachability endpoint so ordinary endpoint overrides
remain effective. Caller definitions may set `reachability.endpoint` or
`authentication.endpoint` to an absolute diagnostic destination. Authentication
otherwise joins `key` to the effective base; reachability otherwise uses that base.

The public convenience surface adds `SmokeTests *BackendSmokeTestConfig` to
`BackendConfig`, with:

- `BackendSmokeTestConfig`: `DefaultProfile *string`,
  `Reachability *ReachabilityProbeConfig`,
  `Authentication *AuthenticationProbeConfig`;
- `ReachabilityProbeConfig`: `Endpoint *string`;
- `AuthenticationProbeConfig`: `Kind *string`, `Endpoint *string`.

`BackendSource` omits nil convenience fields, as it does today. Explicit nulls
remain expressible with ordinary ConfigKit mappings/YAML. In final values,
omitted/null `smoke_tests` disables its defaults and authentication declaration;
omitted/null `default_profile` means no inference default; omitted/null
`reachability` or its endpoint means generic reachability at the effective base;
omitted/null `authentication` means unsupported authentication; a non-null
authentication object requires `kind`. Recognize `none` and `openrouter_key`.
Normalize surrounding whitespace/case for adapter kinds and record normalization
through the existing ConfigKit resolution boundary. Preserve profile IDs exactly;
a supplied default-profile string must not be blank. Resolve its existence only
when that default is selected, not eagerly at engine construction.

Diagnostic endpoint strings are trimmed and parsed as absolute HTTP(S) URLs with
a host. Use standard URL/request validation, allowing paths and queries; do not
reuse the base-endpoint validator's query restriction for these full URLs. Null
means fallback, whereas an explicit blank endpoint is invalid. Retain existing
base-endpoint rules for ordinary target overrides. Reachability sends no
Authorization header or URL user credentials. Public endpoint fields omit URL
user information; other URL components and provider settings remain potentially
sensitive operator data, as specified in the feature roadmap.

Preserve the existing `/v3` catalog asset format: do not place new fields under
the existing `Root` where older strict loaders would reject them. In each module,
add `smoke-tests/backend.yml` and the additive exported constant
`SmokeTestsRoot = "smoke-tests"`; extend `FS()` embedding to include that root.
Keep `Root = "catalog"` and its existing assets unchanged. New PromptKit explicitly
selects both roots through `internal/catalog`; diagnostic data remains
catalog-owned. This is a small opt-in asset surface, not another runtime library
or a PromptKit-owned metadata underlay.

### Private Fixtures And Evaluation

Put fixed inference messages and a self-contained embedded schema in
`internal/usecase` alongside diagnostic orchestration. Use an ordinary user
message requesting the exact token `PROMPTKIT_OK` for text inference. Structured
inference requests `{"status":"ok"}` with an object schema requiring that single
string property with `const: "ok"` and `additionalProperties: false`.
Use native schema name `promptkit_smoke`, strict mode, and the existing structured
output serializer. These literal choices are private and may change; do not
duplicate them across public tests or documentation.

Use `internal/validate.NewFSValidator` with this private schema filesystem, not
the application's schema source. Prepare validation once per diagnostic call and
reuse its schema document for native output. Use existing JSON and schema
validation mechanisms for distinct `ValidJSON` and `SchemaMatched` facts. Do not
repair content. Text matching trims whitespace only. JSON/schema failures use
`ExecutionFailureValidation`; an unexpected text answer is expressed by the
text facts without inventing a domain-validity verdict.

### Shared Execution And Evidence

Keep source-neutral diagnostic values under `internal/domain`. Add a
`usecase.SmokeTester` assembled with the existing runner's captured selection
collaborators, the same generation coordinator, admission manager, and a narrow
built-in HTTP probe collaborator. Share existing private selection and override
helpers within `internal/usecase`; do not implement diagnostics by calling `Run`
or `Prepare`, and do not render an application's prompt.

Add a per-invocation internal evidence path to `internal/llm` and
`internal/generation`. A diagnostic `GenerateWithEvidence` entry point and
ordinary `Generate` share the same private generation implementation. Ordinary
calls need not allocate diagnostic records. Evidence retains dispatch state,
final observed destination, HTTP status, credential presence, decoded-response
availability, timings, and presence-aware usage before existing cancellation
checks can discard a result. Private response content exists only long enough
for local evaluation and is not copied into reports.

Observe HTTP requests with an invocation-local wrapper around the selected
client's RoundTripper, on a shallow client copy. Preserve its timeout, redirects,
cookie jar, and underlying transport; do not mutate a caller's client or forward
credentials specially. A nil transport uses `http.DefaultTransport`. Record only
the most recent observed request and response, resetting destination-specific
response evidence on each new hop. This reports what PromptKit can observe at
its boundary, not wire transmission or any hidden dispatch inside a custom
transport. Do not retain headers, credentials, raw bodies, or a redirect trace.

Reuse the coordinator's failure classification and extract its existing
presence-aware usage aggregation into a small internal reusable accumulator if
needed. Do not duplicate that arithmetic or expand public `ExecutionEvent`.
Mask enclosing execution observation with `observation.WithoutExecutionState`
so diagnostic traffic does not become an application's ordinary execution events.

### Result And Error Decisions

Use the request/report declarations and reason constants in the feature roadmap
without adding status/severity or a stable JSON guarantee. Reports own their
pointer values, attempt slices, effective-setting maps, and usage values. Copy
mutable request overrides using existing ownership helpers before handing them
to collaborators. Apply existing request credential redaction to formatting and
exclude `APIKey` from JSON. Do not store request credentials on an engine.

Invalid selections, kinds, overrides, or negative retry settings are public Go
errors before dispatch. Validate numeric/settings bounds even when a supplied
setting is unused by an HTTP operation; do not require a model for HTTP probes.
Concluded diagnostic execution findings become reports, not raw provider errors.
Required inference credentials missing locally produce
`required_credential_missing` and `invalid_request`. HTTP authentication may
dispatch without credentials even if inference would require them.

Reachability records any observed HTTP response as reachable and received;
non-2xx status alone is not failure of this probe. Authentication records
documented 401 rejection and recognized 200 acceptance. For its 200 recognition,
require one JSON object with a `data` object, string `label`, Boolean
`is_free_tier`, and numeric `usage`; tolerate unrelated fields and ignore account
limits. These are a minimal identifying subset of the provider's documented
[current-key response](https://github.com/OpenRouterTeam/docs/blob/main/api_reference/limits.mdx#checking-your-limits),
not a validation of the full account document. An unrecognized 200 yields the
accepted reason and `malformed_response`; other statuses retain acceptance nil
and use ordinary provider-status classification. Never infer acceptance from
arbitrary provider text.

For inference, generation/transport failures retain their existing classifications.
Set content facts only when evaluation actually occurs. Preserve decoded
generation and usage evidence even when subsequent caller cancellation interrupts
evaluation. A malformed HTTP envelope still counts as an observed response.
An injected client establishes response availability and generation facts, not
HTTP reachability, destinations, status, or upstream credential supply.

Use the configured client timeout and selected target timeout, as the existing
built-in client does. When the outer caller context remains active, an elapsed
request timeout concludes a report with `deadline` and nil Go error. If caller
cancellation/deadline interrupts the diagnostic, return the partial report with
`ctx.Err()` identity, `Completed=false`, and corresponding failure. Check context
before dispatch and between phases; synchronous injected code must return before
PromptKit can observe cancellation. Preserve a diagnostic already completed
before later cancellation; do not promise new behavior for preparation APIs.

Report-level facts come from the last dispatched attempt, then its local
evaluation. A later undispatched wait cannot erase them. An interrupted backoff
adds elapsed retry delay, not an attempt. Total timing includes orchestration;
phase sums need not equal wall time. Request time includes transport/body/envelope
work but excludes capacity wait, retry delay, and generated-content evaluation.
Absent phases stay nil; performed work can measure zero.

Internal fixture loading/compilation, request construction, or assembly faults
that prevent meaningful diagnostic execution may return a content-free Go error
through the existing facade boundary. Do not recover arbitrary panics or promise
that every internal fault becomes a report. Do not expose raw provider or injected
error text in a diagnostic report or its operational-error return.

## Stage 1: Add Internal Diagnostic Values

Scope: `internal/domain`, with focused ownership helpers only.

1. Add request, report, target, facts, timing, attempt, kind, and reason values
   corresponding to the feature roadmap. Reuse existing execution settings,
   failure kinds, and observed usage types.
2. Add source-neutral backend smoke-test metadata values to `domain.Backend`.
   Keep ConfigKit and HTTP transport types out of domain values.
3. Implement copying needed for nested optional metadata, report values, and
   provider maps with existing JSON-value helpers. Separate private decoded
   response evidence from report values so conversion cannot expose content.

Validate: `go test ./internal/domain ./internal/backend`. Add ownership tests
only for mutable structures whose accidental sharing would affect calls.
No public `SmokeTest` method or temporary not-implemented API is introduced yet.

**Completed: 2026-10-13.**

## Stage 2: Load Final Backend Diagnostic Metadata

Scope: `internal/configuration`, `internal/backend`, and `backends.go`.

1. Extend the final typed configuration with the decided fields, normalize and
   validate through existing resolution/validation callbacks, and capture them
   in the immutable backend index. Deep-copy optional nested values on lookup.
2. Implement the public convenience types and `BackendSource` translation.
   Preserve existing convenience omission rules; do not add a parallel loader.
3. Test absent/null blocks, effective nested overlays, explicit endpoint fallback,
   invalid final adapter/URL/default syntax, and owned convenience input values.
   Show that an invalid lower contribution replaced by a valid final value is
   not separately domain-validated.
4. Confirm a dangling default-profile reference is retained until selected and
   does not invalidate ordinary engine construction. No provider contact.

Validate: `go test ./internal/configuration ./internal/backend` and focused root
backend-source tests. Update backend-field GoDoc and `docs/formats.md` as these
configuration fields land; runtime smoke-test guidance waits for Stage 18.

**Completed: 2026-10-13.** Validation passed:
`go test ./internal/configuration ./internal/backend`, `go test .`, and
`git diff --check`.

## Stage 3: Resolve Diagnostic Targets Without Preparing Applications

Scope: selection helpers in `internal/usecase`; no transport.

1. Add the diagnostic selector using `resolvePromptInspection`, captured profile
   selection, execution override merging, and captured backend lookup.
2. Implement the roadmap's prompt/profile/backend-default precedence exactly.
   An explicit profile wins over a backend hint; selecting A's default may route
   to B or to an endpoint-only profile without recursive default lookup.
3. Split model completeness from shared settings/base-endpoint validation so
   HTTP probes can use profile metadata without a model. Backend-only selection
   never loads a default inference profile. Resolve exact diagnostic destinations
   and retain the base execution settings separately in the target report.
4. Validate kinds/retry overrides and selections without resolving application
   artifacts, session templates, inputs, messages, or output schemas.

Validate: focused `internal/usecase` tests for the selection table, explicit
selection failure, default-reference failure only on use, cross-backend/default
identity, endpoint-only profiles, and HTTP model incompleteness. Use real captured
configuration and failing external readers to prove no application preparation.

**Completed: 2026-10-13.** Validation passed: `go test ./...`, `go vet ./...`,
and `git diff --check`.

## Stage 4: Add OpenRouter Catalog Diagnostic Assets

Scope: `../promptkit-backend-openrouter` only, plus this plan's completion record.

1. Add the opt-in embedded root/constant and keyed backend metadata decided above.
   Leave existing `Root` assets and model inventory unchanged.
2. Add a small asset-surface test for the new root and its shipped declaration.
   Reuse existing YAML test helpers; do not add runtime adapters or PromptKit
   imports to the catalog.
3. Document the opt-in root and its relation to existing `/v3` consumers in the
   README. Clarify that selecting new metadata requires a compatible loader.
4. Run the catalog's local checks with `GOWORK=off`; review and commit. Create
   an annotated, unused local candidate tag using the catalog version-selection
   rule described under Stage 6. Record the full peeled commit and tag here.
   Do not require remote synchronization, publication, or a public fetch.

Validate: offline tests, race tests, vet, build, formatting, and local hygiene
from the catalog runbook. Its remote-publication checks belong to later release
preparation, not this local-candidate step.

**Completed: 2026-10-13.** OpenRouter candidate `v3.20261013.0` points to
peeled commit `2a69672b1bb220f1a730153341470f5ac67e15f4`. Validation passed:
`GOWORK=off GOPROXY=off go test ./...`, `go test -race ./...`, `go vet ./...`,
`go build ./...`, formatting, `git diff --check`, and module identity via
`go list -m`. The annotated tag is local only; no publication or push was made.

## Stage 5: Add RakestrawHome Catalog Diagnostic Assets

Scope: `../promptkit-backend-rakestrawhome` only, plus this plan's completion record.

1. Add the same opt-in root/constant with its existing Gemma profile as the
   diagnostic default and authentication `kind: none`.
2. Leave existing base endpoint, credential environment name, capacity limits,
   profile data, and ordinary execution credential policy unchanged. Declaring
   authentication inapplicable does not alter generation's credential settings.
3. Add the focused asset test and README explanation as in Stage 4. Validate,
   commit, and locally tag independently; record the tag and full commit here.

Validate: the same local catalog checks, without provider access or publication.
Both catalog changes must be complete before PromptKit selects their new roots.

**Completed: 2026-10-13.** RakestrawHome candidate `v3.20261013.0` points to
peeled commit `d40a8aad50111c15d279469170f7b5032ac22529`. Validation passed:
`GOWORK=off GOPROXY=off go test ./...`, `go test -race ./...`, `go vet ./...`,
`go build ./...`, formatting, `git diff --check`, and module identity via
`go list -m`. The annotated tag is local only; no publication or push was made.

## Stage 6: Adopt Exact Catalog Candidates Through Local Module Resolution

Scope: `internal/catalog`, module pins/checksums, attribution, integration tests;
temporary tooling stays outside all three repositories.

1. Choose candidate versions independently as `v3.<UTC preparation date>.<N>`:
   use today's UTC date and the first unused local revision, monotonically above
   the selected version. These are local candidate identities, not claims of
   publication. Do not overwrite existing tags. If tagged contents need changes
   later, commit them and choose a new candidate revision rather than reuse its
   already-adopted identity.
2. A `mktemp -d` workspace may contain an uncommitted `go.work` selecting all
   three checkouts for development. Set `GOWORK` explicitly when using it; never
   commit workspace files or filesystem `replace` directives.
3. Build a temporary filesystem module proxy from the exact peeled catalog tag
   commits. A small scratch Go helper can use the already-selected
   `golang.org/x/mod/module` escaping helpers and
   [`zip.CreateFromVCS`](https://pkg.go.dev/golang.org/x/mod/zip#CreateFromVCS)
   to generate compliant archives. Do not add a production proxy tool or forge
   module-cache entries. Supply each escaped module/version's `.info`, `.mod`,
   `.zip`, and version list under `@v`, following the
   [Go proxy protocol](https://go.dev/ref/mod#goproxy-protocol).
   `.mod` comes from that commit; `.info` uses its version and commit UTC time.
4. Cache existing dependencies before switching resolution. For this workflow
   only, use `GOWORK=off`, `GOPROXY=file://<temporary-proxy>`,
   `GONOPROXY=none`, and `GONOSUMDB` scoped to the two catalog module paths.
   Existing dependencies can use the existing download cache as a second local
   filesystem proxy. Do not change global Go environment configuration.
5. Update both ordinary `/v3` requirements to the exact candidates with the Go
   tool; let it generate `go.sum`. Include both exported roots in the existing
   strict catalog composition group, ahead of existing PromptKit-owned profiles
   and caller overlays. Do not hard-code provider metadata in PromptKit.
6. Update selected module/version and commit-pinned notices in attribution after
   checking each candidate's notices. Keep the existing licenses unchanged unless
   the actual upstream material changed. Update stale selected-catalog references
   in format/internal docs where needed.
7. Test shipped defaults through the final assembled configuration, including
   `promptkit-openrouter-standard`, and test caller replacement/null behavior.
   Do not separately parse or audit source contributions in PromptKit.

Validate: `GOWORK=off go test ./internal/catalog ./internal/configuration` plus
focused assembled-construction tests under the temporary proxy environment.
Verify `go list -m -json` reports the exact versions and no replacement. Record
the tag-to-commit mapping here, not temporary paths. Later publication must use
the actual UTC publication date; if new version names are needed, retain the
validated contents and update PromptKit's pins/checksums before its release.
Publication and public module availability are not sprint completion gates.

**Completed: 2026-10-13.** Adopted the local OpenRouter candidate
`v3.20261013.0` at `2a69672b1bb220f1a730153341470f5ac67e15f4` and RakestrawHome
candidate `v3.20261013.0` at `d40a8aad50111c15d279469170f7b5032ac22529`. Validation passed
through a temporary filesystem module proxy: focused catalog/configuration
tests, `go test ./...`, `go vet ./...`, `go mod verify`, exact module identity
and version checks with no replacements, and `git diff --check`. No workspace or
proxy artifacts were committed; the candidate tags remain local and unpublished.

## Stage 7: Capture Destination-Specific HTTP Evidence

Scope: shared transport helpers in `internal/llm`, independent of generation.

1. Add an invocation-local evidence collector and client-copy execution helper.
   Wrap the actual selected RoundTripper and preserve standard redirect policy.
2. Record each observed outgoing request's destination and credential presence,
   retaining only the last hop; clear previous response/status facts when a new
   request is observed. Record response availability/status before body decoding
   or caller-context handling can discard it.
3. Keep dispatch invocation distinct from observed transport evidence. Handle
   redirect-policy termination and response-plus-error according to what was
   actually observed; do not infer a later dispatch from a Location header.
4. Exclude URL user information from exposed endpoints and keep credential/body
   values out of collector records. Provide plain GET support reusing the built-in
   configured client and target timeout without introducing public methods.

Validate: focused adapter tests with `httptest` or controlled transports for a
response, no response, credential-stripping redirect, failed redirected hop, and
redirect-policy termination. Verify the supplied client is not mutated and
evidence is isolated across concurrent calls. Test facts, not wrapper choreography.

**Completed: 2026-10-13.** Added invocation-local last-hop HTTP evidence and
plain GET support over a shallow copy of the configured client. Validation
passed: `go test -race ./internal/llm`, `go vet ./internal/llm`,
`go test ./...`, and `git diff --check`.

## Stage 8: Retain Generation Evidence Before Result Discard

Scope: `PreparedGeneration` execution in `internal/llm`.

1. Add an evidence-enabled internal execution path sharing the existing frozen
   payload, credential resolution, transport, size handling, envelope decoding,
   usage decoding, and context error logic. Preserve ordinary `Execute` behavior.
2. Attach Stage 7 evidence to attempt outcomes. Retain decoded-response success
   and usage before context checks overwrite an error/result.
3. Distinguish HTTP response acquisition from successful envelope decoding and
   generated-content availability, including empty decoded content. Native-schema
   request presence is known at dispatch; no provider-enforcement inference.

Validate: focused `internal/llm` tests for valid/empty/malformed generation,
non-success status, and a decoded response followed by cancellation. Reuse
existing envelope/usage/size tests rather than reproduce their full matrices.

**Completed: 2026-10-13.** Added an evidence-enabled prepared execution path
while keeping ordinary execution on its existing HTTP path. Validation passed:
`go test -race ./internal/llm`, `go vet ./internal/llm`,
`go test ./...`, and `git diff --check`.

## Stage 9: Carry Evidence Through Built-In Retry Coordination

Scope: `internal/generation` and shared observation usage accounting.

1. Add `GenerateWithEvidence` using the same private built-in attempt/retry loop
   as `Generate`. Collect attempts before ordinary result-discard checks.
2. Preserve status retry eligibility, frozen payload/credentials, active-permit
   lifetime, jitter, and Retry-After handling. Add no network/validation retries.
3. Collect capacity waits and elapsed backoffs, including interrupted backoffs.
   A canceled acquisition can produce an undispatched record; backoff alone does
   not produce an attempt. Preserve last dispatched evidence independently.
4. Make the existing failure classifier internally reusable. Share existing
   presence-aware usage aggregation with diagnostics, extracting a small
   accumulator from `internal/observation` if needed; preserve its existing tests
   and no-dispatch known/complete-zero semantics.

Validate: coordinator tests using its controlled clock/wait seams for one
attempt, retry-success, exhausted retries, cancellation during backoff, and
canceled reacquisition. Check evidence retention and timing/accounting; existing
retry tests own unchanged retry policy. Run observation tests after extraction.

**Completed: 2026-10-13.** Added built-in generation attempt evidence and shared
presence-aware usage accumulation. Validation passed: `go test -race
./internal/generation ./internal/observation`, `go vet ./...`, `go test ./...`,
and `git diff --check`.

## Stage 10: Retain Injected-Client Evidence Without HTTP Claims

Scope: the injected branch of `internal/generation`.

1. Extend the shared evidence path to one injected dispatch using the same active
   manager. Preserve the existing no-automatic-retry rule.
2. Capture returned response availability, successful generation/usage, and
   native-schema request presence. Retain successful decoded content privately
   if cancellation is observed immediately after the client returns.
3. Keep destinations, HTTP status/reachability, and upstream credential-supply
   facts unknown. A response returned alongside an error establishes receipt,
   not successful generation or content conformance; retain existing injected
   usage policy rather than inventing billing completeness on errors.

Validate: injected coordinator tests for success, nil response, opaque failure,
response-plus-error, and success followed by caller cancellation. Verify one
dispatch despite a positive retry allowance and ownership at the existing client
adapter boundary. Do not add methods to public `LLMClient`.

**Completed: 2026-10-13.** Injected generation now records one-call response,
completion, usage, and native-schema-request evidence without HTTP claims, while
ordinary generation behavior remains unchanged. Validation passed:
`go test -race ./internal/generation .`, `go vet ./...`, `go test ./...`, and
`git diff --check`.

## Stage 11: Implement The Reachability Probe

Scope: internal HTTP probe collaborator and focused use-case support decisions.

1. Resolve the chosen full diagnostic endpoint or effective base and perform
   exactly one unauthenticated GET with the selected built-in HTTP client.
2. Report any observed HTTP response as reachable/received, independently of its
   status. Report transport failure without guessing DNS/TLS subcategories.
3. Ignore inference retries for this operation after validating the option.
   Never acquire a generation permit or require credentials/model completeness.
4. For injected-client engines, return the unsupported transport outcome without
   falling back to another client. Preserve target selection errors as Go errors.

Validate: internal probe tests for 2xx/401/404/405, transport failure, timeout,
and unsupported injected transport. Derive evidence from Stage 7 rather than
repeat redirect matrices here. Close response bodies; do not retain them.

**Completed: 2026-10-13.** Added the unauthenticated, single-GET reachability
probe over the built-in evidence-aware HTTP client. It records any response as
reachable regardless of status, closes and discards bodies, and reports missing
HTTP support without fallback. Validation passed: `go test -race
./internal/usecase ./internal/llm`, `go vet ./...`, `go test ./...`, and
`git diff --check`.

## Stage 12: Implement Authentication Applicability And OpenRouter Probe

Scope: named HTTP authentication adapter; no inference orchestration.

1. Evaluate captured metadata first: absent/null means unsupported/unknown
   applicability; `none` means supported, inapplicable, completed, no dispatch.
   Known inapplicability also works in injected-client engines.
2. For `openrouter_key`, use the selected full endpoint or base-relative `key`
   path. Resolve direct/environment credentials consistently with generation,
   permitting a missing key for this probe. Send at most one GET.
3. Implement the decided 200 recognition and 401 rejection. Preserve unknown
   acceptance for other responses/transport failure, correct reason/failure
   classification, and last-observed destination evidence. Discard account data.
4. Keep adapter choice fixed by metadata, not inferred from hostname, HTTP status,
   or a models-list response. Reuse configured timeout and redirect behavior.

Validate: focused adapter tests with synthetic current-key responses for accepted,
rejected, anonymous rejection, malformed/unrecognized 200, other status, and
transport failure. Add applicability/support tests without duplicating HTTP
client tests. Synthetic credentials must not appear in report-facing values.

**Completed: 2026-10-13.** Added metadata-driven authentication applicability
and an OpenRouter key probe with optional direct/environment credentials. It
recognizes the documented minimal 200 response, records 401 rejection, leaves
other outcomes undetermined as specified, and distinguishes interrupted body
reads from malformed JSON. Validation passed: `go test -race
./internal/usecase ./internal/llm`, `go vet ./...`, `go test ./...`, and
`git diff --check`.

## Stage 13: Build And Evaluate Private Inference Cases

Scope: private fixtures and validation collaborators in `internal/usecase`.

1. Add the fixed messages/schema described above, independent of application
   prompt/profile listings and schema options. Create ordinary generation requests
   from selected settings, using zero repairs and the diagnostic retry allowance.
2. Reuse prepared private schema validation for native schema request and local
   conformance. Evaluate JSON syntax separately before evaluating the schema.
3. Populate text/JSON/schema facts only when evaluated; whitespace text returns
   false, case changes do not match, invalid JSON leaves schema matching nil.
   Keep malformed generation separate from content validation failure.

Validate: focused case tests for match/mismatch/empty text, JSON parse failure,
schema failure, and success using real validators. Derive generated success
fixtures from the private case rather than spread private literals through tests.
No domain criteria, configurable prompts, public fixtures, or repair loop.

**Completed: 2026-10-13.** Added private text and structured inference cases,
including an embedded schema prepared through `NewFSValidator`. Generation
requests retain the selected target and retry allowance; local evaluation keeps
JSON syntax and schema matching distinct. Validation passed: `go test -race
./internal/usecase ./internal/validate`, `go vet ./...`, `go test ./...`, and
`git diff --check`.

## Stage 14: Orchestrate HTTP Diagnostic Reports

Scope: `internal/usecase.SmokeTester` and its HTTP operations.

1. Connect selection, support/applicability, and HTTP probes into a synchronous
   operation. Start total timing before selection; HTTP probes bypass generation
   capacity. Keep this internal while inference and engine wiring are completed.
2. Assemble HTTP attempts, measured phases, facts, reasons, and failure kinds.
   Preserve nil unknown measurements and no-dispatch usage semantics.
3. Apply the decided Go-error/report boundary and mask inherited observation.
4. Distinguish outer cancellation from configured request timeouts; retain
   previously established facts and finalize total timing on every report return.

Validate: use-case tests for representative HTTP report-state rows, invalid
selection before dispatch, canceled-before-dispatch, timeout with live caller,
and available evidence after cancellation. Use real internal components with
controlled HTTP effects, not a new mock framework.

**Completed: 2026-10-13.** Added the internal synchronous HTTP report
coordinator with captured selection, metadata applicability, timing, sanitized
report targets, per-attempt transport facts, and caller-cancellation handling.
HTTP probes bypass generation admission and inherited execution observation.
Validation passed: `go test -race ./internal/usecase ./internal/llm`,
`go vet ./...`, `go test ./...`, and `git diff --check`.

## Stage 15: Integrate Inference Diagnostics And Engine Assembly

Scope: inference orchestration in `internal/usecase` and `internal/engine` wiring.

1. Connect private inference cases to the shared coordinator and existing
   admission manager. Admit one logical diagnostic until it concludes; active
   permits remain coordinator-owned and release during backoff/local evaluation.
   Use the effective profile backend identity, including an empty identity.
2. Nil diagnostic retries means zero, independent of the engine default. Retain
   all actual attempts, summed measured phases, aggregate usage, and last-dispatched
   facts followed by generated-content evaluation. Keep interrupted waits from
   replacing those facts. Do not run structural repair.
3. Turn local capacity exhaustion and missing required inference credentials into
   concluded findings without dispatch. Apply the shared report/error and context
   boundary, retaining decoded evidence without evaluating canceled content.
4. Assemble the engine's ordinary runner and smoke tester with the same snapshot,
   coordinator, client, and capacity manager. Built-in engines supply the shared
   HTTP collaborator; injected engines supply none. Do not change construction
   options or public client methods.

Validate: focused use-case/engine tests for text/schema results, retry-default
isolation, injected generation, required-credential/capacity findings, and evidence
after cancellation. Stage 17 owns broader facade-level capacity/observer regressions.

**Completed: 2026-10-13.** Integrated private inference cases through the shared
generation coordinator, preserving attempts, usage, phase timings, last-dispatched
facts, and local validation findings. Required-credential and logical-capacity
findings conclude without dispatch. Built-in and injected engines share the
runner/coordinator while only built-in engines supply HTTP diagnostics.
Validation passed: `go test -race ./internal/usecase ./internal/generation
./internal/engine`, `go vet ./...`, `go test ./...`, and `git diff --check`.

## Stage 16: Expose The Public SmokeTest Facade

Scope: root declarations, GoDoc, conversions, and external-package API tests.

1. Add the exact request/report method and public values from the feature roadmap
   in a focused root file. Preserve the facade/internal ownership boundary and
   existing public error translation.
2. Translate owned request values and reports, including a non-nil partial
   report when a recognizable context error is returned. Add request formatting
   redaction using established conventions; `APIKey` stays excluded from JSON.
3. Write complete GoDoc: nil Go error means concluded, not successful; tri-state
   facts; selection precedence; HTTP transport support; retry/repair semantics;
   configured timeout versus caller cancellation; timing and disclosure boundaries.
4. Cover nil engine, invalid kind, detached mutable values, concluded diagnostic
   failure with nil Go error, and partial cancellation identity through the API.

Validate: focused root `go test` cases plus internal engine tests. Do not copy all
internal selection/HTTP matrices into external-package tests or promise stable
JSON output for the new report types.

**Completed: 2026-10-13.** Added the public `Engine.SmokeTest` facade, owned
request/report conversions, credential-safe request formatting, and GoDoc for
selection, tri-state facts, transport support, retries, timeouts, timing, and
disclosure boundaries. Validation passed: `go doc . Engine.SmokeTest`,
`go test -race . ./internal/engine`, `go vet ./...`, and `git diff --check`.

## Stage 17: Verify Assembled Capacity, Cancellation, And Privacy Boundaries

Scope: a small number of consequential integration regressions.

1. Prove ordinary `Run` and inference `SmokeTest` on the same engine share a
   test-controlled backend limit/admission bound. Verify permits release after
   failure/cancellation and a later call proceeds. Avoid scheduler-order/fairness
   guarantees and sleep-based synchronization.
2. Verify canceled retry backoff and later capacity wait preserve last-dispatched
   facts/status/usage, report interruption separately, and do not add a synthetic
   backoff attempt. Reuse lower-level tests for mechanical retry behavior.
3. Verify application prompt messages, inputs, session templates, artifacts, and
   schema sources are unused, and private cases do not enter listings/inspection.
4. Verify reports/operational-error returns omit synthetic credential values,
   generated text, provider messages, raw HTTP bodies, and private fixture content.
   Verify diagnostic traffic does not enter an inherited execution observer.
5. Test a representative cross-backend default and endpoint override through the
   facade with a controlled transport to prove selection/capacity/probe wiring.

Validate: focused race-enabled tests for these assembled paths. Keep cases small
and each risk owned once; do not introduce a second end-to-end state-matrix suite.

## Stage 18: Complete Canonical Documentation

Scope: GoDoc and existing consumer, format, integration, and internal owners.

1. Add a consumer smoke-test section with minimal request/report handling,
   explicit interpretation of nil error, optional facts, and a partial cancellation
   example. Link to GoDoc for exact declarations rather than duplicate them.
2. Finish the backend-format reference for defaults, named adapters, explicit
   endpoints, null/overlay behavior, and opt-in catalog assets. Link the selected
   catalog versions rather than promise provider availability or free tests.
3. Update integration/internal documents for observed HTTP evidence, private
   fixtures, shared coordinator/capacity, operator-only configuration inspection,
   and injected-client limitations. Update the internal inventory only for
   components actually introduced. Add a brief README navigation link if useful.
4. Describe diagnostics as explicit live consumer operations, separate from the
   offline default suite and release checks. Do not add a live example/check to
   existing release tooling or make an extra maintained executable necessary.
5. Record the later release-note need and unpublished catalog handoff here.
   Do not rewrite historical release notes or publish a release in this sprint.

Validate: local documentation paths/anchors, GoDoc against implemented behavior,
and any changed example snippets through focused offline tests. No online link
availability or human sign-off gate.

## Stage 19: Validate The Integrated Candidate And Record Handoff

Scope: final verification, completion records, and unintended-file cleanup.

1. Rebuild the temporary proxy from the recorded exact tags if necessary. From
   an isolated temporary module/cache, resolve both selected catalog versions
   with `GOWORK=off` and that proxy, verify module identities/versions and no
   replacements, and compare Go-generated sums with the adopted requirements.
   Do not use local checkout substitution for this proof. Other dependencies can
   resolve from the prepared existing local download-cache proxy.
2. Run the integrated PromptKit suite with `GOWORK=off` and local proxy settings:
   `go test -race ./...`, `go vet ./...`, and
   `python3 scripts/check-examples.py`. Check Go formatting, `git diff --check`,
   local documentation links, and absence of committed workspace/replacement or
   proxy/cache artifacts. These are sprint checks, not additions to release gates.
3. Reuse successful catalog checks for unchanged tagged candidates. Ensure all
   three repositories' intended changes are committed; preserve and report any
   unrelated work. Correct discovered defects with focused tests/commits and
   rerun only checks invalidated by those changes.
4. Mark completed stages and record final catalog versions/full commits,
   validation results, and the publication handoff below. Explain that catalog
   tags/pins have been validated locally but not necessarily published, and that
   publication-date re-pinning may be needed before a distributable release.
5. Report completion without waiting for public module fetches, provider access,
   CI, human review, or separate loader/catalog releases. Publication of catalogs
   and PromptKit follows their documented procedures under separate authority.

## Completion And Publication Handoff

To be filled as stages complete: catalog candidate tags and full peeled commits,
completed stage markers, final validation results, and any remaining publication
preparation. Temporary environment paths are not durable integration metadata.
There are no remaining product decisions required to start this plan.