Implement layered timeout enforcement

This commit is contained in:
2026-07-26 18:31:06 -05:00
parent a0e905ce46
commit ed0c9f6370
12 changed files with 262 additions and 137 deletions

View File

@@ -10,14 +10,14 @@ format and protocol behavior.
## Construction
`NewOpenAICompatibleClient` validates a non-empty configured base URL, records
an optional default model, and resolves one base timeout. A supplied client with
a non-zero timeout supplies that base; otherwise a positive configured timeout
is used, then the internal default.
an optional default model, and resolves one transport cap. A supplied client
with a positive timeout supplies that cap; otherwise a positive configured
timeout is used, then the internal default.
When callers supply an `http.Client`, construction clones it rather than
mutating the caller's instance. A supplied client with no timeout receives the
resolved base timeout in the clone. The client stores the trimmed base URL,
default model, and cloned client.
mutating the caller's instance. A supplied client with a zero or negative
timeout receives the resolved transport cap in the clone. The client stores the
trimmed base URL, default model, and cloned client.
## Generate Flow
@@ -25,13 +25,13 @@ default model, and cloned client.
1. validate the effective timeout and choose the request endpoint;
2. map the domain request to the internal wire-request representation;
3. validate and flatten extra parameters, encode JSON, and create the HTTP
request;
4. prefer a direct API key, otherwise resolve the configured key environment
3. validate and flatten extra parameters and encode JSON;
4. derive a child context when the effective generation timeout is positive,
then create the HTTP request with that context;
5. prefer a direct API key, otherwise resolve the configured key environment
variable;
5. derive a request HTTP client only when an explicit timeout changes the base
client;
6. execute the request, reject non-success status responses without returning
6. execute with the construction-time HTTP client, reject non-success status
responses without returning
provider response bodies; and
7. decode the response subset into `domain.GenerateResponse`.
@@ -58,8 +58,8 @@ error contracts.
## Package-Local Guarantees
- The default-model fallback happens before wire encoding.
- Per-request timeout handling clones a configured HTTP client when needed; it
does not mutate shared client state.
- Per-generation timeout handling derives a request context; it never replaces
or mutates the configured HTTP client's transport cap.
- Direct API keys take precedence over environment lookup within this client.
- Provider response bodies are discarded for non-success status responses.
- The client does not implement retries, tool calls, or a stateful session