64 lines
3.8 KiB
Markdown
64 lines
3.8 KiB
Markdown
# Internal Generation Coordination
|
|
|
|
`internal/generation` coordinates one model-generation request through the
|
|
existing engine-local [capacity manager](capacity.md) and the
|
|
[built-in HTTP client](llm.md). The engine supplies the same coordinator to
|
|
initial generation and the default output repairer.
|
|
|
|
The coordinator has explicit built-in and injected-client constructors. The
|
|
built-in path prepares the provider request once, then acquires and releases an
|
|
active backend permit around each actual HTTP attempt. Only HTTP 429, 500, 502,
|
|
503, and 504 can lead to another attempt. The injected-client path acquires a
|
|
permit and invokes its client once, regardless of the internal retry budget.
|
|
The runner continues to own the logical execution's admission lease. A request
|
|
can override the engine's retry budget; the runner freezes that choice for
|
|
prepared execution and passes it to each repair generation.
|
|
|
|
Retry waits hold no active permit. The local delay uses equal jitter between
|
|
half and all of a capped exponential base, beginning at 500 milliseconds and
|
|
capping at 8 seconds. A valid provider not-before time can extend that delay;
|
|
the caller context interrupts it. The next attempt reacquires the ordinary
|
|
FIFO permit. Preparation failures dispatch nothing. When a positive retry
|
|
budget ends with eligible failures, the internal exhaustion error retains the
|
|
final cause and actual dispatched-attempt count.
|
|
|
|
When an invocation-local [observation recorder](observation.md) is present in
|
|
the context, each acquisition emits an attempt event, including canceled
|
|
capacity waits without dispatch. Each actual retry wait emits a delay event,
|
|
including interrupted waits. Active permits are released before callbacks;
|
|
generation duration ends before callback delivery. Built-in successful-envelope
|
|
usage retains reported-field presence, while injected clients use explicit
|
|
presence when supplied and otherwise treat zero fields as unknown.
|
|
Malformed successful envelopes can retain strictly decoded usage; error
|
|
response bodies are never mined for it. Arbitrary injected failures carry
|
|
unknown usage. Transport, timeout, malformed, and noneligible status failures
|
|
stop the sequence without retry exhaustion. Context is checked before each
|
|
wait, acquisition, and dispatch.
|
|
|
|
The public client adapter translates explicitly supplied public transport and
|
|
malformed-response identities into internal categories while retaining the
|
|
original error chain. This keeps injected attempt classifications consistent
|
|
with terminal observations without parsing diagnostic text or retrying injected
|
|
clients.
|
|
|
|
The diagnostic path uses the same private attempt and retry implementation as
|
|
ordinary generation. It retains invocation-local transport, response,
|
|
generation, usage, capacity-wait, and retry-delay evidence before normal
|
|
result-discard and cancellation checks. Built-in preparation also supplies its
|
|
frozen request URL to the evidence path before attempts begin; injected
|
|
generation has no equivalent HTTP destination. This evidence is returned to
|
|
diagnostic orchestration; it is not added to public `ExecutionEvent` values or ordinary
|
|
observer accounting. The [model-client document](llm.md) describes the
|
|
transport-observation boundary, and the root
|
|
[`SmokeTestReport` GoDoc](../../smoke_diagnostics.go) owns report semantics.
|
|
|
|
Focused coordinator tests use controlled transport and wait functions. They
|
|
cover eligibility, exhaustion, cancellation, backoff, request freezing, permit
|
|
release, and the injected-client path. The
|
|
[evidence tests](../../internal/generation/evidence_test.go) own diagnostic
|
|
attempt retention and retry evidence. The capacity manager tests own permit
|
|
acquisition and idempotent release. Public
|
|
[integration tests](../../reliable_execution_contract_test.go) cover retry,
|
|
repair, observation, frozen prepared execution, and shared-backend behavior
|
|
through the assembled facade.
|