29 lines
1.6 KiB
Markdown
29 lines
1.6 KiB
Markdown
# Internal Generation Coordination
|
|
|
|
`internal/generation` coordinates one model-generation request through the
|
|
existing engine-local [capacity manager](capacity.md) and the
|
|
[built-in HTTP client](llm.md). The engine supplies the same coordinator to
|
|
initial generation and the default output repairer.
|
|
|
|
The coordinator has explicit built-in and injected-client constructors. The
|
|
built-in path prepares the provider request once, then acquires and releases an
|
|
active backend permit around each actual HTTP attempt. Only HTTP 429, 500, 502,
|
|
503, and 504 can lead to another attempt. The injected-client path acquires a
|
|
permit and invokes its client once, regardless of the internal retry budget.
|
|
The runner continues to own the logical execution's admission lease. A request
|
|
can override the engine's retry budget; the runner freezes that choice for
|
|
prepared execution and passes it to each repair generation.
|
|
|
|
Retry waits hold no active permit. The local delay uses equal jitter between
|
|
half and all of a capped exponential base, beginning at 500 milliseconds and
|
|
capping at 8 seconds. A valid provider not-before time can extend that delay;
|
|
the caller context interrupts it. The next attempt reacquires the ordinary
|
|
FIFO permit. Preparation failures dispatch nothing. When a positive retry
|
|
budget ends with eligible failures, the internal exhaustion error retains the
|
|
final cause and actual dispatched-attempt count.
|
|
|
|
Focused coordinator tests use controlled transport and wait functions. They
|
|
cover eligibility, exhaustion, cancellation, backoff, request freezing, permit
|
|
release, and the injected-client path. The capacity manager tests own permit
|
|
acquisition and idempotent release.
|