1.6 KiB
Internal Generation Coordination
internal/generation coordinates one model-generation request through the
existing engine-local capacity manager and the
built-in HTTP client. The current engine still uses the existing
capacity wrapper; engine assembly moves to this coordinator when public retry
configuration is wired.
The coordinator has explicit built-in and injected-client constructors. The built-in path prepares the provider request once, then acquires and releases an active backend permit around each actual HTTP attempt. Only HTTP 429, 500, 502, 503, and 504 can lead to another attempt. The injected-client path acquires a permit and invokes its client once, regardless of the internal retry budget. The runner continues to own the logical execution's admission lease.
Retry waits hold no active permit. The local delay uses equal jitter between half and all of a capped exponential base, beginning at 500 milliseconds and capping at 8 seconds. A valid provider not-before time can extend that delay; the caller context interrupts it. The next attempt reacquires the ordinary FIFO permit. Preparation failures dispatch nothing. When a positive retry budget ends with eligible failures, the internal exhaustion error retains the final cause and actual dispatched-attempt count.
Focused coordinator tests use controlled transport and wait functions. They cover eligibility, exhaustion, cancellation, backoff, request freezing, permit release, and the injected-client path. The capacity manager tests own permit acquisition and idempotent release.