Document backend capacity management

docs/internal/capacity.md

@@ -0,0 +1,97 @@
+# Internal Capacity Management
+
+## Purpose
+
+This document describes the implemented engine-local capacity coordination in
+`internal/capacity`. The [architecture policy](../policy/architecture.md) owns
+component boundaries, the [backend GoDoc](../../backends.go) owns exact public
+configuration semantics, and the
+[internal runner document](runner.md) owns orchestration around admission.
+
+Capacity scheduling is outside the provider wire contract. It does not add
+fields to execution targets, generated requests, prompt or profile YAML, or
+stable JSON values.
+
+## Construction And Pool Lifecycle
+
+Each root `NewEngine` call obtains a normalized capacity-policy snapshot from
+its immutable backend registry and constructs a new `Manager`. The manager
+creates one pool for each limited backend ID. It has no package-global mutable
+state, background workers, shutdown protocol, or persistence, so engines with
+the same registrations still have independent capacity.
+
+Unlimited registered backends and endpoint-only profiles have no pool. Their
+admission and generation calls take the unrestricted fast path. An endpoint
+override does not change the selected backend ID and therefore does not change
+the pool.
+
+One pool owns immutable active and total limits plus mutex-protected admission
+count, active count, and ordered waiter list. Pool state exists only for the
+lifetime of its engine.
+
+## Bounded Run Admission
+
+The runner asks the manager to admit a run after resolving the prompt, profile,
+selected backend, effective execution target, credentials, and output contract,
+but before schema loading, artifact loading, or rendering. Admission is
+immediate: a limited pool either reserves a slot or returns the internal
+`ErrCapacityExceeded` identity. The root facade maps that identity to the
+public error without treating it as an invalid request or generation failure.
+
+The total admitted bound is the active-generation limit plus its configured
+waiting capacity. The returned release function is idempotent. The runner
+defers it as soon as admission succeeds and holds the lease across remaining
+preparation, initial generation, validation, every repair attempt, and all
+failure or cancellation exits. A repair is part of its original admission and
+does not reserve another bounded slot.
+
+## FIFO Generation Permits
+
+`NewClient` wraps the engine's selected internal model client after public
+client adaptation or built-in client construction. Initial generation and the
+default repairer receive the same wrapper.
+
+For each `Generate` call, the wrapper selects a pool from the request's
+effective backend ID. An unlimited call passes directly to the next client. A
+limited call acquires an active permit, invokes the next client, and defers
+permit release so ordinary returns and panic unwinding both restore capacity.
+Preparation and validation never hold an active permit.
+
+When all active permits are occupied, calls join a mutex-protected FIFO waiter
+list. Releasing a permit transfers it directly to the oldest remaining waiter
+before making it generally available. Pools do not order work relative to
+other backend IDs.
+
+The wrapper passes generation requests, responses, and collaborator errors
+through unchanged. It owns scheduling only; the concrete model client remains
+responsible for provider transport behavior.
+
+## Cancellation And Release
+
+Admission checks the caller context before reserving a slot. A call canceled
+while waiting for an active permit removes its waiter under the same pool lock
+used to grant permits. If cancellation removes the waiter first, the wrapped
+client is not invoked. If a concurrent grant wins first, the call owns the
+permit and invokes the client with the original context, allowing the client
+to observe cancellation normally.
+
+This grant-or-cancel decision prevents lost and double-released permits.
+Admission leases and active permits are released after success, collaborator
+errors, validation failures, cancellation, and panic unwinding. Canceled
+waiters are unlinked so their contexts and requests are not retained by the
+pool.
+
+## Test Ownership
+
+The [manager tests](../../internal/capacity/manager_test.go) own policy
+validation, bounded admission, idempotent release, context handling, and
+unlimited admission. The
+[client tests](../../internal/capacity/client_test.go) own peak enforcement,
+FIFO transfer, canceled-waiter removal, grant/cancel races, independent pools,
+unlimited calls, passthrough behavior, and panic release.
+
+The [runner tests](../../internal/usecase/runner_test.go) own early admission,
+lease lifetime, failure release, and shared initial/repair scheduling. The
+[external package capacity tests](../../capacity_contract_test.go) own the
+assembled public-engine behavior for configured limits, capacity errors,
+endpoint identity, engine independence, cancellation, and injected clients.