2.2 KiB
Internal Generation Coordination
internal/generation coordinates one model-generation request through the
existing engine-local capacity manager and the
built-in HTTP client. The engine supplies the same coordinator to
initial generation and the default output repairer.
The coordinator has explicit built-in and injected-client constructors. The built-in path prepares the provider request once, then acquires and releases an active backend permit around each actual HTTP attempt. Only HTTP 429, 500, 502, 503, and 504 can lead to another attempt. The injected-client path acquires a permit and invokes its client once, regardless of the internal retry budget. The runner continues to own the logical execution's admission lease. A request can override the engine's retry budget; the runner freezes that choice for prepared execution and passes it to each repair generation.
Retry waits hold no active permit. The local delay uses equal jitter between half and all of a capped exponential base, beginning at 500 milliseconds and capping at 8 seconds. A valid provider not-before time can extend that delay; the caller context interrupts it. The next attempt reacquires the ordinary FIFO permit. Preparation failures dispatch nothing. When a positive retry budget ends with eligible failures, the internal exhaustion error retains the final cause and actual dispatched-attempt count.
When an invocation-local observation recorder is present in the context, each acquisition emits an attempt event, including canceled capacity waits without dispatch. Each actual retry wait emits a delay event, including interrupted waits. Active permits are released before callbacks; generation duration ends before callback delivery. Built-in successful-envelope usage retains reported-field presence, while injected clients use explicit presence when supplied and otherwise treat zero fields as unknown.
Focused coordinator tests use controlled transport and wait functions. They cover eligibility, exhaustion, cancellation, backoff, request freezing, permit release, and the injected-client path. The capacity manager tests own permit acquisition and idempotent release.