From c239304c2a5215a550b31225a541b53abe046bc2 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 23 Aug 2026 19:04:00 +0000 Subject: [PATCH] Document structured generation errors --- docs/consumers/pkg-promptkit.md | 18 +++++++++++ docs/integrations/openai-compatible-chat.md | 24 +++++++++++--- docs/internal/llm.md | 36 +++++++++++++-------- docs/internal/overview.md | 4 +-- docs/roadmap/implementation.md | 2 ++ 5 files changed, 65 insertions(+), 19 deletions(-) diff --git a/docs/consumers/pkg-promptkit.md b/docs/consumers/pkg-promptkit.md index 29443ec..6617841 100644 --- a/docs/consumers/pkg-promptkit.md +++ b/docs/consumers/pkg-promptkit.md @@ -450,6 +450,24 @@ status; those choices remain with the consuming application. The contract, while the [`Engine.Run` and error GoDoc](../../engine.go) owns broad error and cancellation identities. +For a non-2xx response from the built-in OpenAI-compatible client, inspect the +status and deliberately selected provider diagnostic when useful: + +```go +var generationErr *promptkit.GenerationError +if errors.As(err, &generationErr) { + status := generationErr.StatusCode() + message := generationErr.ProviderMessage() + _, _ = status, message // Apply application retry and presentation policy. +} +``` + +All provider fields are untrusted and can contain sensitive request or schema +fragments. Do not log, display, or return them without an application-specific +disclosure policy. Promptkit does not assign retry or presentation behavior. +The [`GenerationError` GoDoc](../../generation_error.go) owns the exact typed +error contract. + ## Application Boundary Promptkit is an importable library. It does not own a command, inbound HTTP diff --git a/docs/integrations/openai-compatible-chat.md b/docs/integrations/openai-compatible-chat.md index 4da1e64..a9f4fa7 100644 --- a/docs/integrations/openai-compatible-chat.md +++ b/docs/integrations/openai-compatible-chat.md @@ -102,10 +102,26 @@ and cache-write token counts. Invalid or truncated JSON, trailing non-whitespace data, a second JSON value, absent choices, empty first-choice content, and size overflow are malformed responses and return no partial result. -For a non-2xx status, the error includes the status code but never the provider -response body. Promptkit does not yet parse provider error envelopes; bounded -non-success parsing belongs to the -[structured-generation-error roadmap](../roadmap/structured-generation-errors.md). +For a non-2xx status, Promptkit recognizes one JSON document with a top-level +object-valued `error` member. Its optional `message` and `type` fields must be +strings, and `code` may be a string or JSON number. Valid supported fields are +handled independently, numeric codes retain their JSON number text, and +unknown fields are ignored. Missing, invalid, malformed, or multiply framed +envelopes contribute no provider detail. + +Non-success bodies have a 65,536-byte limit. A larger declared +`Content-Length` is not read; otherwise the client reads at most one additional +byte to detect streamed or underreported overflow. Empty, unreadable, +oversized, malformed, and unrecognized bodies retain only the received status. +The body is always closed and no oversized stream is drained beyond that probe. + +Extracted strings are made valid UTF-8, trimmed, and converted to one line by +collapsing Unicode whitespace, control, and format-character runs. Blank +values are omitted. Codes and types longer than 256 Unicode code points are +omitted; messages longer than 4,096 code points are truncated at a code-point +boundary with an ellipsis inside the limit. Promptkit never exposes raw bodies, +headers, endpoints, credentials, request data, schemas, generated content, or +unsupported provider metadata through this handling. An outbound `http.Client.Do` failure retains both Promptkit's request-failure identity and the exact transport error for `errors.Is` and `errors.As` checks. diff --git a/docs/internal/llm.md b/docs/internal/llm.md index d6ca67e..b359300 100644 --- a/docs/internal/llm.md +++ b/docs/internal/llm.md @@ -37,8 +37,9 @@ resolved request target may supply the endpoint. Generation then: 4. composes `/chat/completions` through parsed URL path operations; 5. resolves authentication; 6. performs the outbound request under the applicable deadlines; and -7. decodes one strictly framed, size-bounded response object and maps its first - choice and token usage. +7. decodes one strictly framed, size-bounded successful response object and + maps its first choice and token usage, or decodes bounded structured + non-success detail. `internal/llm` owns the set of reserved OpenAI-compatible request fields used when validating extra parameters. Backend registration consumes the same rule @@ -78,10 +79,15 @@ length and by reading at most one byte beyond the boundary. The decoder accepts exactly one JSON object plus trailing whitespace and EOF. Size overflow, truncation, malformed JSON, trailing data, and a second value are malformed responses with no partial result or provider content in the error. Every body -is closed, and an unbounded oversized stream is not drained. Non-success -responses remain status-only; bounded provider error-envelope parsing belongs -to the -[structured-generation-error roadmap](../roadmap/structured-generation-errors.md). +is closed, and an unbounded oversized stream is not drained. + +For a non-success response, `ProviderHTTPError` retains the HTTP status and +only normalized detail from the bounded recognized envelope. It retains +`ErrUnexpectedStatus` through unwrapping. The client owns response closure; +its bounded reader and parser never close or drain a body themselves. The root +facade converts this concrete internal error into the public +[`GenerationError`](../../generation_error.go), while arbitrary injected-client +errors continue through the ordinary generation-error mapping unchanged. An `http.Client.Do` failure is represented by a redacting multi-cause error: the package request-failure sentinel and the exact returned transport error are @@ -99,10 +105,14 @@ The own configuration, client cloning, deterministic deadline precedence, authentication, request and response mapping, malformed data, error identity, cancellation, endpoint selection and composition, pre-transport rejection, and -bounded single-document response framing, closure, and response-body -suppression. The root -transport contract tests also verify that resolved backend settings reach this -client without serializing backend identity and that ordinary-run cancellation -retains its public generation and context identities. All use local test -servers or controlled test transports; the default suite makes no live or paid -provider requests. +bounded single-document successful-response framing, closure, and +response-body suppression. The focused +[provider HTTP error tests](../../internal/llm/provider_http_error_test.go) +own envelope parsing, normalization, and bounded-reader cases; their +[transport tests](../../internal/llm/provider_http_error_transport_test.go) +own non-success response closure and integration. Root transport contract tests +own public `GenerationError` conversion, while also verifying that resolved +backend settings reach this client without serializing backend identity and +that ordinary-run cancellation retains its public generation and context +identities. All use local test servers or controlled test transports; the +default suite makes no live or paid provider requests. diff --git a/docs/internal/overview.md b/docs/internal/overview.md index 0e0bc05..3171984 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -11,7 +11,7 @@ contributor workflow and validation. | Component | Implemented responsibility | References | | --- | --- | --- | -| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, typed capacity errors, public error mapping, and engine-local profile-source assembly including application fallbacks. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) | +| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, typed capacity and generation error mapping, and engine-local profile-source assembly including application fallbacks. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) | | `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) | | `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) | | `internal/backend` | Constructs each engine's immutable registry from the maintained built-in definitions and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) | @@ -26,7 +26,7 @@ contributor workflow and validation. | `internal/prompt` | Renders prompt messages from Go templates with artifact, variable, session, and cache-control data. | [Go-template renderer](../../internal/prompt/go_renderer.go) | | `internal/artifact` | Resolves ordinary inline and unrestricted caller-selected file references into copied artifacts with metadata and hashes. | [Internal sources and validation](sources.md) | | `internal/validate` | Validates basic, JSON, and JSON Schema output using operating-system filesystem or `fs.FS` schema sources and creates operation-local validation plans with canonical contained schema resources. | [Framework formats](../formats.md#schemas), [internal sources and validation](sources.md) | -| `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible chat requests from resolved execution targets, including response decoding, authentication, deadline handling, and ownership of the OpenAI-compatible reserved request-field policy. | [Internal model client](llm.md) | +| `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible chat requests from resolved execution targets, including bounded structured non-success response decoding, successful-response decoding, authentication, deadline handling, and ownership of the OpenAI-compatible reserved request-field policy. | [Internal model client](llm.md) | | `internal/usecase` | Resolves prompt definitions and hashes, profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary execution and one-attempt prepared execution across internal sources, rendering, artifact loading, operation-local validation plans, generation, capacity, and optional repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) | The root package assembles these internal components without exposing their diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 0989925..012bbcd 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -427,6 +427,8 @@ Finally review the complete diff against the feature roadmap and confirm: - no provider policy entered the use-case or domain layers; and - each exact contract has one canonical documentation and test owner. +**Status:** Complete. + ## Open Questions None. The roadmap and this plan fix the public API, internal representation,