eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document smoke-test diagnostics

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-13 18:32:41 +00:00
parent b7a951685b
commit f4c721d934
13 changed files with 256 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
docs/internal/llm.md
View File

@@ -59,6 +59,33 @@ This client does not schedule retries; the [generation coordinator](generation.m
does so around its prepared attempts. The client has no tool-call support,
provider catalog, inbound HTTP behavior, or durable session store.
## Diagnostic HTTP Evidence
Built-in diagnostics wrap the selected HTTP client's `RoundTripper` for one
invocation using a shallow client copy, preserving its timeout, redirect policy,
cookie jar, and underlying transport without mutating the configured client.
Evidence distinguishes a diagnostic dispatch invocation from an observed
request and response. A newly observed request replaces the previous
destination-specific response evidence, so redirects retain only the last
observed hop. A response is recorded before envelope decoding or later context
checks can discard a generation result.
The collector keeps only the last observed destination (without URL user
information), status, credential-presence, response-availability, decoded
generation, and presence-aware usage facts needed by the diagnostic. It does
not retain headers, credentials, raw request or response bodies, or a redirect
trace. These are observations at Promptkit's RoundTripper boundary, not proof
that bytes reached a peer or that a custom transport did no hidden dispatch.
Reachability and named authentication probes use the same configured built-in
client but discard their response bodies after local recognition. The
[integration contract](../integrations/openai-compatible-chat.md#explicit-smoke-diagnostics)
owns their observable HTTP semantics.
An injected `LLMClient` does not pass through this HTTP boundary. Its returned
response can establish response and inference facts, but not an HTTP
destination, status, reachability, or upstream credential-supply fact. The
facade does not silently install a built-in transport for those diagnostics.
## Prepared Generation
For [`RunPrepared`](../../engine.go), the runner supplies the model client with
@@ -137,7 +164,10 @@ own configuration, client cloning, deterministic deadline precedence,
authentication, request and response mapping, malformed data, error identity,
cancellation, endpoint selection and composition, pre-transport rejection, and
bounded single-document successful-response framing, closure, and
response-body suppression. The focused
response-body suppression. The [HTTP evidence tests](../../internal/llm/http_evidence_test.go)
own invocation-local last-hop request/response facts, while the
[prepared-generation evidence tests](../../internal/llm/prepared_generation_evidence_test.go)
own retention across decoding and cancellation. The focused
[provider HTTP error tests](../../internal/llm/provider_http_error_test.go)
own envelope parsing, normalization, and bounded-reader cases; their
[transport tests](../../internal/llm/provider_http_error_transport_test.go)