Add HTTP whole-run deadlines and context outcomes

This commit is contained in:
2026-09-26 15:39:17 +00:00
parent 911ca5485a
commit 7de2b02891
6 changed files with 217 additions and 18 deletions

View File

@@ -51,6 +51,7 @@ copyable shape. At the HTTP adapter boundary, the smallest valid shape is:
| `model` | no | Runtime model-override object. |
| `include_raw_output` | no | Include `raw_model_output` when true. |
| `transient_retries` | no | Nonnegative additional transient attempt count for this run. Omitted or `null` inherits the configured engine value and then Promptkit's default; `0` disables additional attempts. |
| `run_timeout_seconds` | no | Optional signed 64-bit whole-second limit for engine execution. Omitted, `null`, or `0` adds no deadline; positive values shorten the caller context deadline. |
An input reference has a required `type` of `file` or `inline`. A `file`
reference requires `uri`; an `inline` reference requires `body`.
@@ -98,6 +99,12 @@ returns `400 invalid_json`, while a negative integer returns
`400 invalid_request` before execution. A request override does not change the
shared engine setting used by other requests. Promptkit owns eligible statuses,
retry waits, and the separate transient budget for each repair generation.
`run_timeout_seconds` must be a JSON int64. A wrong type or int64 overflow
returns `400 invalid_json`; a negative value or one whose conversion to a Go
duration would overflow returns `400 invalid_request`. The deadline starts
immediately before engine execution and excludes body decoding and response
writing. An earlier caller deadline or cancellation still wins. Synchronous
work is not guaranteed to stop immediately.
### Success Response
@@ -157,6 +164,7 @@ Messages are concise and do not expose wrapped internal causes.
| `404` | `prompt_not_found` | Prompt ID or version does not exist. |
| `404` | `profile_not_found` | Profile ID does not exist. |
| `405` | `method_not_allowed` | The route does not accept the method. |
| `408` | `request_canceled` | The caller context was canceled during a failed run. |
| `413` | `request_too_large` | Encoded request exceeds its limit. |
| `413` | `artifact_too_large` | File input exceeds its limit. |
| `413` | `response_too_large` | Encoded response exceeds its limit. |
@@ -164,6 +172,12 @@ Messages are concise and do not expose wrapped internal causes.
| `500` | `internal_error` | Unclassified server failure. |
| `502` | `llm_failed` | Outbound model request failed. |
| `503` | `capacity_exceeded` | The selected model backend has no admission capacity. No retry timing is supplied. |
| `504` | `deadline_exceeded` | The execution context's deadline expired during a failed run. |
These context outcomes require the execution context to have ended. A provider
attempt timeout while it remains live retains `502 llm_failed`. A successful
run remains successful even if cancellation races with response formatting;
delivery after a client disconnect cannot be guaranteed.
## Retry And Idempotency

View File

@@ -128,6 +128,10 @@ be bounded. The existing generation timeout applies afresh to each attempt and
does not cap retry waits or the complete invocation. The whole-run interval
covers the engine call, including capacity waiting and repairs as supported by
context cancellation; it excludes parsing, engine setup, and output writes.
HTTP callers can use [`run_timeout_seconds`](api.md) to shorten their request
context for the same engine interval. The API distinguishes an expired
execution deadline from caller cancellation and from a provider-attempt
timeout. A canceled capacity waiter releases its Promptkit admission state.
Before increasing a limit:

View File

@@ -210,7 +210,7 @@ Existing per-attempt timeout tests and generated-output behavior remain intact.
## Stage 5: Add HTTP Whole-Run Deadlines And Context Outcomes
**Status: Pending.**
**Status: Complete.**
Selected interface: top-level `run_timeout_seconds`, an optional signed
64-bit integer. Omission/null/zero adds no deadline; positive whole seconds