promptkit/docs/roadmap/structured-generation-errors.md

Structured Generation Errors

Purpose

Promptkit should give downstream applications actionable, machine-readable details when the built-in OpenAI-compatible client receives a non-success HTTP response. Today the client reports only the status code and discards the provider response body. This makes ordinary configuration failures, such as an unsupported strict JSON Schema keyword, unnecessarily difficult to diagnose.

This feature supplies bounded facts about the provider response. It does not make retry, presentation, or logging decisions for consumers.

Target End State

Every non-2xx response received by Promptkit's built-in OpenAI-compatible client becomes a public typed generation error. A consumer can use errors.As to obtain the HTTP status and any safely extracted provider fields, and errors.Is continues to match ErrLLMGenerate.

The typed contract is available from both Run and RunPrepared. It is not produced during preparation, which performs no model request. Successful responses, transport failures before a response is received, cancellation, capacity failures, validation failures, and nil responses from injected model clients retain their existing categories and behavior.

An unusable response body never hides the known HTTP status. Empty, malformed, unrecognized, unreadable, or oversized bodies therefore produce the same typed error with status-only detail rather than falling back to an unstructured error or becoming a malformed-success response.

Public Contract

The root package exposes an immutable GenerationError type with unexported state and these read-only accessors:

• StatusCode() int returns the received HTTP status code;
• ProviderCode() string returns a normalized provider code, when present;
• ProviderType() string returns a normalized provider error type, when present; and
• ProviderMessage() string returns the bounded normalized diagnostic message, when present.

The engine returns a *GenerationError, so the idiomatic inspection form is:

var generationErr *promptkit.GenerationError
if errors.As(err, &generationErr) {
	status := generationErr.StatusCode()
	message := generationErr.ProviderMessage()
	_, _ = status, message
}

There is no public constructor or mutation API. The type implements error, unwraps to ErrLLMGenerate, and provides safe ordinary and Go-syntax formatting. Error() and GoString() include the HTTP status but no provider- controlled code, type, or message. Consumers must use the accessors deliberately when they want provider details and must not classify failures by matching error text.

The zero value and a nil *GenerationError receiver are safe: accessors return zero or empty values, formatting returns a generic redacted generation-failure description, and unwrapping still identifies ErrLLMGenerate. Engine-produced values always have the non-2xx status received from the provider. The type has no stable JSON representation.

All provider-derived strings remain untrusted even after normalization. GoDoc must warn consumers that provider fields can contain sensitive request or schema fragments and must not be logged, displayed, or returned to another caller without an application-appropriate disclosure policy.

Recognized Provider Envelope

Promptkit recognizes only the conventional OpenAI-compatible top-level error object:

{
  "error": {
    "message": "diagnostic text",
    "type": "invalid_request_error",
    "code": "unsupported_parameter"
  }
}

The envelope must be one JSON document followed only by JSON whitespace. The top-level error value must be an object. Unknown top-level and error-object fields are ignored. The optional supported fields are interpreted independently:

• message and type must be JSON strings;
• code may be a JSON string or number and is exposed as normalized text; numeric codes retain their validated JSON number text without floating-point coercion; and
• null, booleans, arrays, objects, or otherwise invalid values are treated as absent for that field.

An invalid optional field does not discard other valid supported fields. An absent error object, malformed or multiply framed JSON, or an object with no usable supported fields simply leaves all provider accessors empty while preserving the typed status error.

Promptkit does not expose param, metadata objects, nested causes, headers, or provider-specific extensions in this feature.

Bounded Reading And Normalization

Non-success bodies have a separate fixed limit of 64 KiB (65,536 bytes). This is intentionally much smaller than the successful completion-body limit while remaining large enough for useful schema diagnostics.

• A declared Content-Length above the limit is rejected without reading the body for detail extraction.
• Otherwise Promptkit reads at most one byte beyond the limit so streamed, chunked, and underreported bodies are bounded.
• A body over the limit contributes no provider fields; Promptkit does not parse or retain a prefix as though it were a complete envelope.
• Read failures likewise discard provider fields while preserving the status.
• The response body is closed on every outcome and is not drained beyond the bounded read.

Extracted strings are converted to valid UTF-8, trimmed, and made single-line: invalid UTF-8 is replaced, and runs of Unicode whitespace, control characters, and formatting controls are replaced with one ASCII space. Empty normalized values are treated as absent.

Normalized provider codes and types are retained only when they contain at most 256 Unicode code points. Longer values are omitted rather than truncated so consumers never classify on a fabricated partial identifier. A provider message is limited to 4,096 Unicode code points; a longer normalized message is truncated at a code-point boundary with a visible ellipsis inside that limit. The raw response body and pre-normalized strings are never exposed or retained in the public error.

Error Propagation And Compatibility

• Every built-in-client non-2xx response matches ErrLLMGenerate and supports errors.As to *GenerationError, including status-only cases.
• The internal model client retains its non-success-status identity for its own package tests. The use-case layer remains provider-neutral and continues to add only its generation category.
• The root error boundary converts only the built-in transport's structured status error. It does not parse arbitrary error text, inspect consumer error fields, or fabricate HTTP details for an injected LLMClient.
• Errors returned by injected clients remain in the chain exactly as today. If an injected client deliberately returns an existing *GenerationError, its identity may pass through ordinary wrapping, but Promptkit does not construct or enrich one on that client's behalf.
• Existing cancellation and deadline identities, capacity errors, validation behavior, repair behavior, and successful response decoding remain unchanged.
• This is an additive public API. Existing consumers that use errors.Is(err, ErrLLMGenerate) continue to work; consumers should not rely on the previous rendered wording of non-success errors.

Architecture And Ownership

The provider-envelope parser and bounded body reader belong in internal/llm, which owns the OpenAI-compatible transport. The internal transport error owns only normalized status facts and continues to match the package's existing non-success-status sentinel.

The use-case package does not gain HTTP DTOs, status policy, or a provider- specific branch. Its existing wrapping carries the internal error to the root facade. The root error mapper recognizes the internal structured status error and constructs the public GenerationError without exposing an internal type or raw cause through public fields. No transport error is added to internal/domain.

The public type and its exact Go semantics are owned by its declaration and GoDoc. The OpenAI-compatible integration contract owns recognized wire shapes, limits, and observable response behavior. The internal model-client document owns implementation flow, internal failure categories, and test ownership. Architecture policy does not need a new package or dependency rule for this feature.

Documentation End State

Canonical documentation at the target state has these responsibilities:

• the GenerationError declaration and GoDoc define the exact public methods, formatting, unwrapping, zero-value behavior, and trust boundary;
• Engine.Run and Engine.RunPrepared GoDoc identify the typed error without duplicating its accessor contract;
• the consumer guide includes one short errors.As example and links to the public declaration;
• the integration document replaces its status-only description with the bounded envelope contract; and
• the internal model-client document describes parsing, conversion ownership, and narrow test owners.

The architecture policy, framework format reference, and built-in backend catalog do not duplicate this API or wire contract.

Verification Expectations

Verification protects each behavior at its narrowest stable owner:

• internal model-client tests cover recognized string and numeric codes, independent optional-field handling, unknown fields, empty and malformed envelopes, single-document framing, read failures, declared and streamed size boundaries, body closure, normalization, field limits, and absence of raw provider content from rendered errors;
• root error-boundary tests cover conversion to the immutable public type, every accessor, errors.Is, errors.As, and safe %v, %+v, and %#v formatting;
• one representative ordinary run and one prepared run prove that the built-in transport contract crosses the assembled engine boundary, without repeating the complete parser matrix;
• existing injected-client tests continue to prove preservation of consumer error identity without fabricated provider details; and
• all tests use controlled transports or local servers and never contact a live or paid provider.

Security limits and their exact boundaries are contractual enough to warrant literal boundary tests. Higher-level tests should remain representative and must not duplicate the internal transport matrix.

Acceptance Criteria

• A consumer can distinguish an HTTP 400 from other generation failures and deliberately obtain a bounded provider explanation when one is available.
• The same typed error remains available through ordinary and prepared execution and still satisfies errors.Is(err, ErrLLMGenerate).
• Default and Go-syntax error formatting cannot disclose any provider-derived string or raw response content.
• Empty, malformed, unreadable, unrecognized, and oversized bodies preserve a typed status-only error.
• No read, retained field, or formatted representation can exceed its stated bound, and the body is closed on every outcome.
• Existing success, cancellation, capacity, validation, repair, and injected- client contracts remain unchanged.
• Current-state documentation changes only when the implementation exists and follows the repository's canonical ownership policy.

Non-Goals

This feature does not add:

• retryability classification, retry loops, backoff, failover, or routing;
• parsing of success bodies as errors or changes to successful-response limits;
• provider-specific envelope variants beyond the conventional top-level error object;
• response headers such as Retry-After, raw bodies, request data, endpoints, credentials, schema documents, generated content, or provider metadata;
• logging, telemetry, redaction policy for downstream applications, HTTP status mapping for consumer servers, or user-facing presentation;
• translation or enrichment of arbitrary injected-client errors; or
• a new public package, public constructor, mutable error value, or transport type in the domain model.

Open Questions

None. The public type direction, accessor surface, formatting and error-chain behavior, envelope scope, normalization, safety limits, fallback behavior, layer ownership, compatibility boundaries, documentation ownership, and test boundaries are fixed by this roadmap.