45 lines
2.7 KiB
Markdown
45 lines
2.7 KiB
Markdown
# Internal Generation Coordination
|
|
|
|
`internal/generation` coordinates one model-generation request through the
|
|
existing engine-local [capacity manager](capacity.md) and the
|
|
[built-in HTTP client](llm.md). The engine supplies the same coordinator to
|
|
initial generation and the default output repairer.
|
|
|
|
The coordinator has explicit built-in and injected-client constructors. The
|
|
built-in path prepares the provider request once, then acquires and releases an
|
|
active backend permit around each actual HTTP attempt. Only HTTP 429, 500, 502,
|
|
503, and 504 can lead to another attempt. The injected-client path acquires a
|
|
permit and invokes its client once, regardless of the internal retry budget.
|
|
The runner continues to own the logical execution's admission lease. A request
|
|
can override the engine's retry budget; the runner freezes that choice for
|
|
prepared execution and passes it to each repair generation.
|
|
|
|
Retry waits hold no active permit. The local delay uses equal jitter between
|
|
half and all of a capped exponential base, beginning at 500 milliseconds and
|
|
capping at 8 seconds. A valid provider not-before time can extend that delay;
|
|
the caller context interrupts it. The next attempt reacquires the ordinary
|
|
FIFO permit. Preparation failures dispatch nothing. When a positive retry
|
|
budget ends with eligible failures, the internal exhaustion error retains the
|
|
final cause and actual dispatched-attempt count.
|
|
|
|
When an invocation-local [observation recorder](observation.md) is present in
|
|
the context, each acquisition emits an attempt event, including canceled
|
|
capacity waits without dispatch. Each actual retry wait emits a delay event,
|
|
including interrupted waits. Active permits are released before callbacks;
|
|
generation duration ends before callback delivery. Built-in successful-envelope
|
|
usage retains reported-field presence, while injected clients use explicit
|
|
presence when supplied and otherwise treat zero fields as unknown.
|
|
Malformed successful envelopes can retain strictly decoded usage; error
|
|
response bodies are never mined for it. Arbitrary injected failures carry
|
|
unknown usage. Transport, timeout, malformed, and noneligible status failures
|
|
stop the sequence without retry exhaustion. Context is checked before each
|
|
wait, acquisition, and dispatch.
|
|
|
|
Focused coordinator tests use controlled transport and wait functions. They
|
|
cover eligibility, exhaustion, cancellation, backoff, request freezing, permit
|
|
release, and the injected-client path. The capacity manager tests own permit
|
|
acquisition and idempotent release. Public
|
|
[integration tests](../../reliable_execution_contract_test.go) cover retry,
|
|
repair, observation, frozen prepared execution, and shared-backend behavior
|
|
through the assembled facade.
|