Document backend capacity management

docs/internal/runner.md

@@ -19,17 +19,20 @@ and override semantics consumed by the runner.
 `Runner` coordinates narrow internal interfaces for prompt definitions,
 profiles, backend resolution, artifacts, rendering, model generation, and
 validation. The root engine supplies one immutable registry containing the
-built-in backend and validated consumer additions. Schema documents are loaded
-through the validator's optional schema-loader interface. An output repairer
-can be injected internally, but the ordinary runner constructor does not
-enable one.
+built-in backend and validated consumer additions, one engine-local run
+admitter, and a model client wrapped by the same capacity manager. Schema
+documents are loaded through the validator's optional schema-loader interface.
+An output repairer can be injected internally, but the ordinary runner
+constructor does not enable one.
 
 Each invocation carries its state in request, prepared-run, and result values.
 The runner has no durable run or session store.
 
-## Preparation Flow
+## Shared Preparation Pipeline
 
-`Prepare` performs the reusable pre-generation workflow:
+`Prepare` and `Run` share one private preparation pipeline split at the point
+where a run can be assigned to its selected backend pool. The resolution phase
+performs only the work needed to validate routing and admission:
 
 1. validate the required prompt selection and normalize any direct session ID;
 2. load the prompt definition and hash the original definition;
@@ -38,13 +41,25 @@ The runner has no durable run or session store.
 5. resolve application-neutral defaults, backend defaults, profile values,
    and explicit request overrides in that order;
 6. validate endpoint, model, numeric overrides, and credential requirements;
-7. resolve the output contract and load a structured-output schema when
-   required;
-8. load and hash input artifacts;
-9. render messages, resolve the effective session ID, and hash the effective
-   rendered prompt; and
-10. return the effective settings, source identities, messages, hashes, and
-   preparation timing.
+7. resolve the effective output contract without loading its schema; and
+8. retain the definition, source identities, effective settings, output
+   contract, and preparation start time in invocation-local state.
+
+The completion phase consumes that state without reloading the prompt,
+profile, or backend:
+
+1. load structured-output schema metadata when required;
+2. load and hash input artifacts;
+3. render messages and the prompt-defined session;
+4. apply any direct session ID;
+5. hash the effective rendered prompt; and
+6. construct the prepared value and preparation timing.
+
+`Prepare` runs both phases consecutively and never performs capacity admission.
+`Run` performs backend admission between the phases. This structure preserves
+one execution-precedence and error-ordering implementation while allowing a
+full backend pool to reject work before expensive schema, artifact, and
+rendering operations.
 
 Pointer-based numeric overrides preserve an explicit zero. Invalid negative or
 out-of-range values fail as invalid requests. Endpoint overrides do not change
@@ -70,15 +85,28 @@ invocation state local.
 
 ## Run Flow
 
-`Run` calls `Prepare` rather than maintaining a second preparation path. It
-performs one initial generation call, builds the named output artifact, and
-validates that artifact. Invalid generated content remains a validation result;
-an inability to perform validation is an operational error.
+`Run` records its start time, performs the shared resolution phase, and asks
+its `RunAdmitter` to reserve capacity for the effective backend ID. A nil
+admitter is an internal unlimited fallback. After successful admission, `Run`
+immediately defers the returned release function, performs the completion
+phase, makes one initial generation call, builds the named output artifact,
+and validates that artifact. Invalid generated content remains a validation
+result; an inability to perform validation is an operational error.
+
+The admission lease covers completion-phase preparation, initial generation,
+validation, every repair, and every exit. It bounds accepted work without
+serializing preparation or validation behind the active-generation limit.
+The wrapped model client separately acquires a FIFO active permit only around
+each actual generation call.
 
 When an internal repairer is present, a JSON or JSON Schema content failure can
 trigger bounded repair attempts. Repair receives the effective execution
 target and session ID, validation errors, prior output, and structured-output
-specification. This capability remains internal and is not a public option.
+specification. The default repairer uses the same wrapped client as initial
+generation, so each repair reacquires the selected backend's active permit
+while remaining inside its original admission lease. Repair never performs a
+second bounded admission. This capability remains internal and is not a public
+option.
 
 A successful result includes the output artifact and raw output, validation
 state, effective session ID, prompt and rendered-prompt hashes, selected
@@ -94,8 +122,20 @@ Package errors distinguish invalid requests, required profile selection,
 credential failures, and prompt, profile, artifact, rendering, generation, and
 validation failures. Wrapping preserves the package identities mapped by the
 public facade and retains collaborator identities where they are part of the
-internal contract. Context cancellation propagates through the invoked
-collaborator and is classified by the owning operation.
+internal contract.
+
+Admission capacity exhaustion retains the internal capacity identity and adds
+the selected backend ID as context. It is not recategorized as an invalid
+request or generation failure, and no partial result is returned. A context
+already done at admission retains its context identity directly. Cancellation
+while waiting for an active generation permit prevents client invocation when
+it wins the grant race; the model-client boundary then preserves the context
+error through the generation-failure category. Deferred release restores the
+admission lease on preparation, generation, validation, repair, and
+cancellation failures.
+
+Other context cancellation propagates through the invoked collaborator and is
+classified by the owning operation.
 An overlong direct session is an invalid request before source loading, while
 an invalid or overlong prompt session template remains a prompt-render failure.
 An unknown selected backend, or a selected backend with no configured resolver,
@@ -104,12 +144,16 @@ is classified as a profile-load failure.
 ## Test Ownership And Changes
 
 The [runner tests](../../internal/usecase/runner_test.go) own preparation order,
-selection and override precedence, direct-session resolution,
-schema-before-generation behavior, hashing, generation and validation
-outcomes, backend propagation, bounded repair, credentials and redaction,
-error categories, artifact metadata, usage, and timing.
+selection and override precedence, the two-phase boundary, early admission,
+lease lifetime and release, direct-session resolution, schema-before-generation
+behavior, hashing, generation and validation outcomes, backend propagation,
+bounded repair, shared initial/repair capacity, credentials and redaction,
+error categories, artifact metadata, usage, and timing. The
+[capacity subsystem document](capacity.md) identifies the focused pool,
+waiter, and wrapped-client tests.
 
 Changes to orchestration should continue to use the existing package
-interfaces, keep request state local to an invocation, and preserve `Run`'s use
-of `Prepare`. Source, renderer, validator, or model-client contract changes
-belong first in their owning package and document.
+interfaces, keep request state local to an invocation, and preserve the shared
+resolution and completion pipeline. Source, renderer, validator, or
+model-client contract changes belong first in their owning package and
+document.