37 lines
2.2 KiB
Markdown
37 lines
2.2 KiB
Markdown
# Internal Generation Coordination
|
|
|
|
`internal/generation` coordinates one model-generation request through the
|
|
existing engine-local [capacity manager](capacity.md) and the
|
|
[built-in HTTP client](llm.md). The engine supplies the same coordinator to
|
|
initial generation and the default output repairer.
|
|
|
|
The coordinator has explicit built-in and injected-client constructors. The
|
|
built-in path prepares the provider request once, then acquires and releases an
|
|
active backend permit around each actual HTTP attempt. Only HTTP 429, 500, 502,
|
|
503, and 504 can lead to another attempt. The injected-client path acquires a
|
|
permit and invokes its client once, regardless of the internal retry budget.
|
|
The runner continues to own the logical execution's admission lease. A request
|
|
can override the engine's retry budget; the runner freezes that choice for
|
|
prepared execution and passes it to each repair generation.
|
|
|
|
Retry waits hold no active permit. The local delay uses equal jitter between
|
|
half and all of a capped exponential base, beginning at 500 milliseconds and
|
|
capping at 8 seconds. A valid provider not-before time can extend that delay;
|
|
the caller context interrupts it. The next attempt reacquires the ordinary
|
|
FIFO permit. Preparation failures dispatch nothing. When a positive retry
|
|
budget ends with eligible failures, the internal exhaustion error retains the
|
|
final cause and actual dispatched-attempt count.
|
|
|
|
When an invocation-local [observation recorder](observation.md) is present in
|
|
the context, each acquisition emits an attempt event, including canceled
|
|
capacity waits without dispatch. Each actual retry wait emits a delay event,
|
|
including interrupted waits. Active permits are released before callbacks;
|
|
generation duration ends before callback delivery. Built-in successful-envelope
|
|
usage retains reported-field presence, while injected clients use explicit
|
|
presence when supplied and otherwise treat zero fields as unknown.
|
|
|
|
Focused coordinator tests use controlled transport and wait functions. They
|
|
cover eligibility, exhaustion, cancellation, backoff, request freezing, permit
|
|
release, and the injected-client path. The capacity manager tests own permit
|
|
acquisition and idempotent release.
|