promptkit/docs/roadmap/structured-generation-errors.md

# Structured Generation Errors

## Purpose

Promptkit should give downstream applications actionable, machine-readable
details when the built-in OpenAI-compatible client receives a non-success HTTP
response. Today the client reports only the status code and discards the
provider response body. This makes ordinary configuration failures, such as an
unsupported strict JSON Schema keyword, unnecessarily difficult to diagnose.

This feature supplies bounded facts about the provider response. It does not
make retry, presentation, or logging decisions for consumers.

## Target End State

Every non-2xx response received by Promptkit's built-in OpenAI-compatible
client becomes a public typed generation error. A consumer can use
`errors.As` to obtain the HTTP status and any safely extracted provider fields,
and `errors.Is` continues to match `ErrLLMGenerate`.

The typed contract is available from both `Run` and `RunPrepared`. It is not
produced during preparation, which performs no model request. Successful
responses, transport failures before a response is received, cancellation,
capacity failures, validation failures, and nil responses from injected model
clients retain their existing categories and behavior.

An unusable response body never hides the known HTTP status. Empty, malformed,
unrecognized, unreadable, or oversized bodies therefore produce the same typed
error with status-only detail rather than falling back to an unstructured
error or becoming a malformed-success response.

## Public Contract

The root package exposes an immutable `GenerationError` type with unexported
state and these read-only accessors:

- `StatusCode() int` returns the received HTTP status code;
- `ProviderCode() string` returns a normalized provider code, when present;
- `ProviderType() string` returns a normalized provider error type, when
  present; and
- `ProviderMessage() string` returns the bounded normalized diagnostic message,
  when present.

The engine returns a `*GenerationError`, so the idiomatic inspection form is:

```go
var generationErr *promptkit.GenerationError
if errors.As(err, &generationErr) {
	status := generationErr.StatusCode()
	message := generationErr.ProviderMessage()
	_, _ = status, message
}
```

There is no public constructor or mutation API. The type implements `error`,
unwraps to `ErrLLMGenerate`, and provides safe ordinary and Go-syntax
formatting. `Error()` and `GoString()` include the HTTP status but no provider-
controlled code, type, or message. Consumers must use the accessors
deliberately when they want provider details and must not classify failures by
matching error text.

The zero value and a nil `*GenerationError` receiver are safe: accessors return
zero or empty values, formatting returns a generic redacted generation-failure
description, and unwrapping still identifies `ErrLLMGenerate`. Engine-produced
values always have the non-2xx status received from the provider. The type has
no stable JSON representation.

All provider-derived strings remain untrusted even after normalization. GoDoc
must warn consumers that provider fields can contain sensitive request or
schema fragments and must not be logged, displayed, or returned to another
caller without an application-appropriate disclosure policy.

## Recognized Provider Envelope

Promptkit recognizes only the conventional OpenAI-compatible top-level error
object:

```json
{
  "error": {
    "message": "diagnostic text",
    "type": "invalid_request_error",
    "code": "unsupported_parameter"
  }
}
```

The envelope must be one JSON document followed only by JSON whitespace. The
top-level `error` value must be an object. Unknown top-level and error-object
fields are ignored. The optional supported fields are interpreted
independently:

- `message` and `type` must be JSON strings;
- `code` may be a JSON string or number and is exposed as normalized text;
  numeric codes retain their validated JSON number text without floating-point
  coercion; and
- `null`, booleans, arrays, objects, or otherwise invalid values are treated as
  absent for that field.

An invalid optional field does not discard other valid supported fields. An
absent `error` object, malformed or multiply framed JSON, or an object with no
usable supported fields simply leaves all provider accessors empty while
preserving the typed status error.

Promptkit does not expose `param`, metadata objects, nested causes, headers, or
provider-specific extensions in this feature.

## Bounded Reading And Normalization

Non-success bodies have a separate fixed limit of 64 KiB (65,536 bytes). This
is intentionally much smaller than the successful completion-body limit while
remaining large enough for useful schema diagnostics.

- A declared `Content-Length` above the limit is rejected without reading the
  body for detail extraction.
- Otherwise Promptkit reads at most one byte beyond the limit so streamed,
  chunked, and underreported bodies are bounded.
- A body over the limit contributes no provider fields; Promptkit does not
  parse or retain a prefix as though it were a complete envelope.
- Read failures likewise discard provider fields while preserving the status.
- The response body is closed on every outcome and is not drained beyond the
  bounded read.

Extracted strings are converted to valid UTF-8, trimmed, and made single-line:
invalid UTF-8 is replaced, and runs of Unicode whitespace, control characters,
and formatting controls are replaced with one ASCII space. Empty normalized
values are treated as absent.

Normalized provider codes and types are retained only when they contain at
most 256 Unicode code points. Longer values are omitted rather than truncated
so consumers never classify on a fabricated partial identifier. A provider
message is limited to 4,096 Unicode code points; a longer normalized message is
truncated at a code-point boundary with a visible ellipsis inside that limit.
The raw response body and pre-normalized strings are never exposed or retained
in the public error.

## Error Propagation And Compatibility

- Every built-in-client non-2xx response matches `ErrLLMGenerate` and supports
  `errors.As` to `*GenerationError`, including status-only cases.
- The internal model client retains its non-success-status identity for its
  own package tests. The use-case layer remains provider-neutral and continues
  to add only its generation category.
- The root error boundary converts only the built-in transport's structured
  status error. It does not parse arbitrary error text, inspect consumer error
  fields, or fabricate HTTP details for an injected `LLMClient`.
- Errors returned by injected clients remain in the chain exactly as today.
  If an injected client deliberately returns an existing `*GenerationError`,
  its identity may pass through ordinary wrapping, but Promptkit does not
  construct or enrich one on that client's behalf.
- Existing cancellation and deadline identities, capacity errors, validation
  behavior, repair behavior, and successful response decoding remain
  unchanged.
- This is an additive public API. Existing consumers that use
  `errors.Is(err, ErrLLMGenerate)` continue to work; consumers should not rely
  on the previous rendered wording of non-success errors.

## Architecture And Ownership

The provider-envelope parser and bounded body reader belong in `internal/llm`,
which owns the OpenAI-compatible transport. The internal transport error owns
only normalized status facts and continues to match the package's existing
non-success-status sentinel.

The use-case package does not gain HTTP DTOs, status policy, or a provider-
specific branch. Its existing wrapping carries the internal error to the root
facade. The root error mapper recognizes the internal structured status error
and constructs the public `GenerationError` without exposing an internal type
or raw cause through public fields. No transport error is added to
`internal/domain`.

The public type and its exact Go semantics are owned by its declaration and
GoDoc. The
[OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md)
owns recognized wire shapes, limits, and observable response behavior. The
[internal model-client document](../internal/llm.md) owns implementation flow,
internal failure categories, and test ownership. Architecture policy does not
need a new package or dependency rule for this feature.

## Documentation End State

Canonical documentation at the target state has these responsibilities:

- the `GenerationError` declaration and GoDoc define the exact public methods,
  formatting, unwrapping, zero-value behavior, and trust boundary;
- `Engine.Run` and `Engine.RunPrepared` GoDoc identify the typed error without
  duplicating its accessor contract;
- the consumer guide includes one short `errors.As` example and links to the
  public declaration;
- the integration document replaces its status-only description with the
  bounded envelope contract; and
- the internal model-client document describes parsing, conversion ownership,
  and narrow test owners.

The architecture policy, framework format reference, and built-in backend
catalog do not duplicate this API or wire contract.

## Verification Expectations

Verification protects each behavior at its narrowest stable owner:

- internal model-client tests cover recognized string and numeric codes,
  independent optional-field handling, unknown fields, empty and malformed
  envelopes, single-document framing, read failures, declared and streamed
  size boundaries, body closure, normalization, field limits, and absence of
  raw provider content from rendered errors;
- root error-boundary tests cover conversion to the immutable public type,
  every accessor, `errors.Is`, `errors.As`, and safe `%v`, `%+v`, and `%#v`
  formatting;
- one representative ordinary run and one prepared run prove that the built-in
  transport contract crosses the assembled engine boundary, without repeating
  the complete parser matrix;
- existing injected-client tests continue to prove preservation of consumer
  error identity without fabricated provider details; and
- all tests use controlled transports or local servers and never contact a
  live or paid provider.

Security limits and their exact boundaries are contractual enough to warrant
literal boundary tests. Higher-level tests should remain representative and
must not duplicate the internal transport matrix.

## Acceptance Criteria

- A consumer can distinguish an HTTP 400 from other generation failures and
  deliberately obtain a bounded provider explanation when one is available.
- The same typed error remains available through ordinary and prepared
  execution and still satisfies `errors.Is(err, ErrLLMGenerate)`.
- Default and Go-syntax error formatting cannot disclose any provider-derived
  string or raw response content.
- Empty, malformed, unreadable, unrecognized, and oversized bodies preserve a
  typed status-only error.
- No read, retained field, or formatted representation can exceed its stated
  bound, and the body is closed on every outcome.
- Existing success, cancellation, capacity, validation, repair, and injected-
  client contracts remain unchanged.
- Current-state documentation changes only when the implementation exists and
  follows the repository's canonical ownership policy.

## Non-Goals

This feature does not add:

- retryability classification, retry loops, backoff, failover, or routing;
- parsing of success bodies as errors or changes to successful-response limits;
- provider-specific envelope variants beyond the conventional top-level
  `error` object;
- response headers such as `Retry-After`, raw bodies, request data, endpoints,
  credentials, schema documents, generated content, or provider metadata;
- logging, telemetry, redaction policy for downstream applications, HTTP status
  mapping for consumer servers, or user-facing presentation;
- translation or enrichment of arbitrary injected-client errors; or
- a new public package, public constructor, mutable error value, or transport
  type in the domain model.

## Open Questions

None. The public type direction, accessor surface, formatting and error-chain
behavior, envelope scope, normalization, safety limits, fallback behavior,
layer ownership, compatibility boundaries, documentation ownership, and test
boundaries are fixed by this roadmap.