Implement layered timeout enforcement
This commit is contained in:
@@ -68,7 +68,9 @@ The optional `model` object accepts `endpoint`, `model`, `temperature`,
|
||||
`reasoning_effort`, `api_key_env`, and `extra_params`. Numeric ranges and
|
||||
credential supply are defined by the [configuration reference](config.md).
|
||||
Explicit zero values for the numeric fields are overrides; zero
|
||||
`timeout_seconds` disables the outbound client timeout.
|
||||
`timeout_seconds` disables the per-generation deadline only, retaining the
|
||||
request context and configured transport cap. The timeout layers are defined in
|
||||
the [outbound integration contract](integrations/openai-compatible-chat.md#authentication-and-timeout).
|
||||
|
||||
Raw API-key values are not accepted. `api_key` and any other unknown model
|
||||
field cause `400 invalid_json`.
|
||||
|
||||
10
docs/cli.md
10
docs/cli.md
@@ -64,10 +64,12 @@ Deprecated aliases: `--prompt-id` for `--prompt`, and `--profile-id` for
|
||||
`--profile`.
|
||||
|
||||
Omitted numeric runtime flags preserve the selected effective value; explicit
|
||||
zero values override it. `--timeout 0s` disables the outbound HTTP-client
|
||||
timeout. CLI durations are converted to whole seconds by truncation toward
|
||||
zero, so any duration whose absolute value is below one second becomes an
|
||||
explicit zero-second override.
|
||||
zero values override it. `--timeout 0s` disables the per-generation deadline
|
||||
only; the caller context and configured transport cap remain active. CLI
|
||||
durations are converted to whole seconds by truncation toward zero, so any
|
||||
duration whose absolute value is below one second becomes an explicit
|
||||
zero-second override. The timeout layers are defined in the
|
||||
[outbound integration contract](integrations/openai-compatible-chat.md#authentication-and-timeout).
|
||||
|
||||
There is no raw API-key flag. Use `--api-key-env`.
|
||||
|
||||
|
||||
@@ -96,7 +96,7 @@ selected by YAML `id`; nested directories are organizational. See the
|
||||
| `temperature` | no | Number from `0` through `2`. |
|
||||
| `max_tokens` | no | Integer zero or greater. |
|
||||
| `top_p` | no | Number from `0` through `1`. |
|
||||
| `timeout_seconds` | no | Integer zero or greater. |
|
||||
| `timeout_seconds` | no | Per-generation-call deadline in whole seconds; integer zero or greater. |
|
||||
| `service_tier` | no | Non-empty provider-specific request tier. |
|
||||
| `reasoning_effort` | no | Non-empty provider-specific reasoning setting. |
|
||||
| `api_key_env` | no | Environment-variable name containing the API key. |
|
||||
@@ -105,7 +105,11 @@ selected by YAML `id`; nested directories are organizational. See the
|
||||
Execution defaults before profile and request overrides are `temperature: 0`,
|
||||
`max_tokens: 0`, `top_p: 1`, and `timeout_seconds: 600`. Profile numeric values
|
||||
merge by non-zero value. Request overrides preserve presence, so an explicit
|
||||
zero can override a profile value.
|
||||
zero can override a profile value. For `timeout_seconds`, explicit request zero
|
||||
disables the generation deadline while retaining the caller context and the
|
||||
built-in client's transport cap. See the
|
||||
[OpenAI-compatible integration contract](integrations/openai-compatible-chat.md#authentication-and-timeout)
|
||||
for the complete timeout interaction.
|
||||
|
||||
Custom profiles take precedence over built-ins with the same ID. Invalid custom
|
||||
profiles are errors; they do not fall back to a built-in profile. Raw `api_key`
|
||||
|
||||
@@ -20,16 +20,17 @@ fields:
|
||||
| `PromptDir` | Prompt-definition directory, required unless a prompt source option is supplied. |
|
||||
| `ProfileDir` | Optional custom profile directory over built-ins. |
|
||||
| `SchemaDir` | Schema directory; empty uses `.`. |
|
||||
| `Timeout` | Base timeout for the built-in OpenAI-compatible client when `HTTPClient` is absent or has a zero timeout. A non-positive value uses the internal default. |
|
||||
| `HTTPClient` | Optional HTTP client for that built-in client. It is cloned; a non-zero `Timeout` on it takes precedence over `Config.Timeout` as the base timeout. |
|
||||
| `Timeout` | Transport-wide safety cap for the built-in OpenAI-compatible client when `HTTPClient` is absent or has a non-positive timeout. A non-positive value uses the internal ten-minute default. |
|
||||
| `HTTPClient` | Optional HTTP client for that built-in client. It is cloned; a positive `Timeout` on it is the transport cap and takes precedence over `Config.Timeout`. A non-positive client timeout is treated as unset. |
|
||||
|
||||
Nil options are ignored. Invalid construction, including
|
||||
`WithLLMClient(nil)`, returns an error matching `ErrInvalidConfig`.
|
||||
|
||||
An effective positive `timeout_seconds` replaces the base timeout. An explicit
|
||||
request override of zero disables the HTTP-client timeout. The timeout is
|
||||
otherwise inherited from the supplied client, `Config.Timeout`, or the internal
|
||||
default in that order.
|
||||
Profile and request `timeout_seconds` values select a per-generation-call
|
||||
deadline independently of the transport cap. An explicit request override of
|
||||
zero disables that generation deadline only. The complete interaction with the
|
||||
caller context is defined in the
|
||||
[OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md#authentication-and-timeout).
|
||||
|
||||
Source options replace their matching directory source:
|
||||
|
||||
|
||||
@@ -77,12 +77,22 @@ resolves the configured non-empty `api_key_env` at request time and sends the
|
||||
same header. If neither mechanism supplies a key, it sends no
|
||||
`Authorization` header.
|
||||
|
||||
The client base timeout is chosen at construction: a non-zero timeout on a
|
||||
supplied `http.Client` takes precedence over `Config.Timeout`; otherwise a
|
||||
positive `Config.Timeout` is used, then the internal default. A positive
|
||||
effective `timeout_seconds` replaces that base. An explicit request override
|
||||
of zero disables the HTTP-client timeout; negative values are rejected before a
|
||||
request is sent.
|
||||
The transport-wide safety cap is chosen at client construction. A positive
|
||||
timeout on a supplied `http.Client` takes precedence over a positive
|
||||
`Config.Timeout`; if neither is positive, the internal ten-minute default is
|
||||
used. The supplied client is cloned, and zero or negative timeout values are
|
||||
treated as unset.
|
||||
|
||||
Separately, a positive effective `timeout_seconds` creates a deadline for each
|
||||
outbound generation call. Its value follows the execution-setting hierarchy:
|
||||
an explicit request override, then a non-zero profile value, then the
|
||||
600-second framework default. An explicit request override of zero disables
|
||||
only this generation deadline. Negative values are rejected before a request
|
||||
is sent.
|
||||
|
||||
The complete observable rule is that the earliest caller-context deadline,
|
||||
transport cap, or positive generation deadline terminates the call. Transport
|
||||
and cancellation failures retain the generation-error classification.
|
||||
|
||||
## Response Subset And Failures
|
||||
|
||||
|
||||
@@ -10,14 +10,14 @@ format and protocol behavior.
|
||||
## Construction
|
||||
|
||||
`NewOpenAICompatibleClient` validates a non-empty configured base URL, records
|
||||
an optional default model, and resolves one base timeout. A supplied client with
|
||||
a non-zero timeout supplies that base; otherwise a positive configured timeout
|
||||
is used, then the internal default.
|
||||
an optional default model, and resolves one transport cap. A supplied client
|
||||
with a positive timeout supplies that cap; otherwise a positive configured
|
||||
timeout is used, then the internal default.
|
||||
|
||||
When callers supply an `http.Client`, construction clones it rather than
|
||||
mutating the caller's instance. A supplied client with no timeout receives the
|
||||
resolved base timeout in the clone. The client stores the trimmed base URL,
|
||||
default model, and cloned client.
|
||||
mutating the caller's instance. A supplied client with a zero or negative
|
||||
timeout receives the resolved transport cap in the clone. The client stores the
|
||||
trimmed base URL, default model, and cloned client.
|
||||
|
||||
## Generate Flow
|
||||
|
||||
@@ -25,13 +25,13 @@ default model, and cloned client.
|
||||
|
||||
1. validate the effective timeout and choose the request endpoint;
|
||||
2. map the domain request to the internal wire-request representation;
|
||||
3. validate and flatten extra parameters, encode JSON, and create the HTTP
|
||||
request;
|
||||
4. prefer a direct API key, otherwise resolve the configured key environment
|
||||
3. validate and flatten extra parameters and encode JSON;
|
||||
4. derive a child context when the effective generation timeout is positive,
|
||||
then create the HTTP request with that context;
|
||||
5. prefer a direct API key, otherwise resolve the configured key environment
|
||||
variable;
|
||||
5. derive a request HTTP client only when an explicit timeout changes the base
|
||||
client;
|
||||
6. execute the request, reject non-success status responses without returning
|
||||
6. execute with the construction-time HTTP client, reject non-success status
|
||||
responses without returning
|
||||
provider response bodies; and
|
||||
7. decode the response subset into `domain.GenerateResponse`.
|
||||
|
||||
@@ -58,8 +58,8 @@ error contracts.
|
||||
## Package-Local Guarantees
|
||||
|
||||
- The default-model fallback happens before wire encoding.
|
||||
- Per-request timeout handling clones a configured HTTP client when needed; it
|
||||
does not mutate shared client state.
|
||||
- Per-generation timeout handling derives a request context; it never replaces
|
||||
or mutates the configured HTTP client's transport cap.
|
||||
- Direct API keys take precedence over environment lookup within this client.
|
||||
- Provider response bodies are discarded for non-success status responses.
|
||||
- The client does not implement retries, tool calls, or a stateful session
|
||||
|
||||
@@ -58,7 +58,7 @@ The post-completion review found:
|
||||
|
||||
1. Decide and document the precedence among:
|
||||
- `Config.Timeout`;
|
||||
- a non-zero timeout on a supplied `http.Client`; and
|
||||
- a positive timeout on a supplied `http.Client`; and
|
||||
- an explicit per-request `timeout_seconds` override.
|
||||
2. Align `internal/llm` with that decision, removing any constructor state that
|
||||
is immediately discarded during generation.
|
||||
@@ -134,8 +134,17 @@ This follow-up is complete when:
|
||||
|
||||
## Completion Record
|
||||
|
||||
Completed on 2026-07-26. The timeout contract, single-file source contract,
|
||||
strict-decoding boundary, and maintained HTTP example check were reconciled.
|
||||
Completed on 2026-07-26. The single-file source contract, strict-decoding
|
||||
boundary, and maintained HTTP example check were reconciled. A subsequent
|
||||
assembled-engine review corrected the timeout work into two independent
|
||||
layers: a construction-time transport cap and a per-generation context
|
||||
deadline, with explicit request zero disabling only the latter. Deterministic
|
||||
public `Engine.Run` coverage now protects the interaction with caller
|
||||
deadlines, profile and request values, the framework default, `Config.Timeout`,
|
||||
and supplied HTTP-client caps.
|
||||
|
||||
Final validation confirmed local links and paths, both maintained configuration
|
||||
files through the real loader, the maintained render and Go package examples,
|
||||
`go test ./...`, `go vet ./...`, and `go build ./cmd/scriptorium`.
|
||||
`go test ./...`, `go vet ./...`, and a temporary-output build. The Promptkit
|
||||
Step 1 documentation gate remains complete after this layered-timeout
|
||||
correction.
|
||||
|
||||
@@ -94,8 +94,9 @@ refresh and policy updates are merged and the repository has an agreed,
|
||||
accurate baseline.
|
||||
|
||||
**Gate status:** Complete as of 2026-07-26. The completed documentation
|
||||
refresh and follow-up verification record are in the [documentation compliance
|
||||
roadmap](documentation.md).
|
||||
refresh, follow-up verification, and layered-timeout correction record are in
|
||||
the [documentation compliance roadmap](documentation.md). Step 1 remains
|
||||
complete after that validation.
|
||||
|
||||
### Step 2: Record The Architectural Decision And Detailed Boundary
|
||||
|
||||
|
||||
Reference in New Issue
Block a user