Bound and strictly decode provider responses

This commit is contained in:
2026-08-11 23:46:16 +00:00
parent 3a43550f70
commit 2b6a7f83c4
4 changed files with 348 additions and 12 deletions

View File

@@ -88,13 +88,24 @@ request fields.
## Response Handling
Any 2xx response is decoded as an OpenAI-compatible chat response. The client
returns the first choice's non-empty message content and maps prompt,
completion, total, cached, and cache-write token counts.
Any 2xx response body is limited to 16 MiB (16,777,216 bytes). A larger
declared `Content-Length` is rejected before the body is read, and streamed,
chunked, or underreported bodies are read through the same bound with at most
one additional byte used to detect overflow. A body exactly at the limit is
allowed. The body is closed on every outcome and an oversized stream is not
drained.
Invalid JSON, absent choices, and empty first-choice content are malformed
responses. For a non-2xx status, the error includes the status code but never
the provider response body.
The bounded body must contain exactly one OpenAI-compatible JSON response
object followed only by JSON whitespace and EOF. The client returns the first
choice's non-empty message content and maps prompt, completion, total, cached,
and cache-write token counts. Invalid or truncated JSON, trailing non-whitespace
data, a second JSON value, absent choices, empty first-choice content, and size
overflow are malformed responses and return no partial result.
For a non-2xx status, the error includes the status code but never the provider
response body. Promptkit does not yet parse provider error envelopes; bounded
non-success parsing belongs to the
[structured-generation-error roadmap](../roadmap/structured-generation-errors.md).
An outbound `http.Client.Do` failure retains both Promptkit's request-failure
identity and the exact transport error for `errors.Is` and `errors.As` checks.

View File

@@ -37,7 +37,8 @@ resolved request target may supply the endpoint. Generation then:
4. composes `/chat/completions` through parsed URL path operations;
5. resolves authentication;
6. performs the outbound request under the applicable deadlines; and
7. decodes the first response choice and token usage.
7. decodes one strictly framed, size-bounded response object and maps its first
choice and token usage.
`internal/llm` owns the set of reserved OpenAI-compatible request fields used
when validating extra parameters. Backend registration consumes the same rule
@@ -72,6 +73,16 @@ Invalid nonempty configured endpoints are configuration failures. A missing or
invalid final selected endpoint is an invalid generation request and is
rejected before transport.
Successful response bodies have a fixed 16 MiB limit enforced by declared
length and by reading at most one byte beyond the boundary. The decoder accepts
exactly one JSON object plus trailing whitespace and EOF. Size overflow,
truncation, malformed JSON, trailing data, and a second value are malformed
responses with no partial result or provider content in the error. Every body
is closed, and an unbounded oversized stream is not drained. Non-success
responses remain status-only; bounded provider error-envelope parsing belongs
to the
[structured-generation-error roadmap](../roadmap/structured-generation-errors.md).
An `http.Client.Do` failure is represented by a redacting multi-cause error:
the package request-failure sentinel and the exact returned transport error are
both available through `errors.Is` and `errors.As`, while the rendered text
@@ -88,7 +99,8 @@ The
own configuration, client cloning, deterministic deadline precedence,
authentication, request and response mapping, malformed data, error identity,
cancellation, endpoint selection and composition, pre-transport rejection, and
response-body suppression. The root
bounded single-document response framing, closure, and response-body
suppression. The root
transport contract tests also verify that resolved backend settings reach this
client without serializing backend identity and that ordinary-run cancellation
retains its public generation and context identities. All use local test