Configure built-in retries across execution paths
This commit is contained in:
@@ -2,16 +2,17 @@
|
||||
|
||||
`internal/generation` coordinates one model-generation request through the
|
||||
existing engine-local [capacity manager](capacity.md) and the
|
||||
[built-in HTTP client](llm.md). The current engine still uses the existing
|
||||
capacity wrapper; engine assembly moves to this coordinator when public retry
|
||||
configuration is wired.
|
||||
[built-in HTTP client](llm.md). The engine supplies the same coordinator to
|
||||
initial generation and the default output repairer.
|
||||
|
||||
The coordinator has explicit built-in and injected-client constructors. The
|
||||
built-in path prepares the provider request once, then acquires and releases an
|
||||
active backend permit around each actual HTTP attempt. Only HTTP 429, 500, 502,
|
||||
503, and 504 can lead to another attempt. The injected-client path acquires a
|
||||
permit and invokes its client once, regardless of the internal retry budget.
|
||||
The runner continues to own the logical execution's admission lease.
|
||||
The runner continues to own the logical execution's admission lease. A request
|
||||
can override the engine's retry budget; the runner freezes that choice for
|
||||
prepared execution and passes it to each repair generation.
|
||||
|
||||
Retry waits hold no active permit. The local delay uses equal jitter between
|
||||
half and all of a capped exponential base, beginning at 500 milliseconds and
|
||||
|
||||
Reference in New Issue
Block a user