# Backend-Specific Concurrency Management Implementation Plan **Status:** Ready for implementation. ## Purpose This document is the decision-complete implementation plan for [backend-specific concurrency management](concurrency.md). It is written for a coding agent that will implement each stage in order. The feature roadmap owns the intended capability, consumer value, policy choices, compatibility decision, and target end state. This document owns the concrete API, internal representation, scheduling architecture, implementation sequence, test ownership, documentation updates, and completion gates. ## Implementation Rules - Complete the stages in order. Keep the repository compiling and the focused tests passing at every stage boundary. - Preserve unrelated working-tree changes. In particular, `concurrency.md` and the removal of its source idea from `future.md` may already be uncommitted when implementation begins; retain both. - Follow every policy under `docs/policy/`, the task-specific reading guide in `docs/development.md`, and the target behavior in `concurrency.md`. - Keep the public API in the root `promptkit` package and implementation details under `internal/`. Do not expose scheduler types or create another public package. - Use only the Go standard library for scheduling. Do not add a queue, semaphore, worker-pool, or metrics dependency. - Preserve synchronous, wait-for-result `Run`, unrestricted `Prepare`, engine-local state, endpoint-only profiles, backend-selected profiles, backend identity through endpoint overrides, and injected `LLMClient` behavior. - Do not broaden the work into asynchronous jobs, durable queues, retries, rate limiting, dynamic configuration, priorities, worker lifecycle, endpoint-keyed pools, or public queue observability. - Keep all tests deterministic, bounded, offline, and race-safe. Coordinate concurrent tests with channels and barriers rather than timing assumptions or live providers. - Update exact GoDoc with each exported declaration change. Update durable current-state documents only after the corresponding behavior is implemented. - Test configurable mechanisms with small test-owned limits. Assert the exact OpenRouter `16` and default queue `1024` values only at the registry contract that owns those operational defaults. - Do not create a release, change a module version, or tag a commit. The final implementation handoff must identify the built-in OpenRouter behavior change for the next pre-`v1` minor release. ## Fixed Design ### Public Backend Configuration Append these fields to the existing root `Backend` type in `backends.go`: ```go type Backend struct { // Existing fields remain unchanged and in their current order. ConcurrencyLimit int QueueCapacity *int } ``` Use these exact semantics: | Public values | Meaning | | --- | --- | | `ConcurrencyLimit == 0`, `QueueCapacity == nil` | Unlimited backend; preserve current behavior. | | `ConcurrencyLimit > 0`, `QueueCapacity == nil` | Limit active generations and use the default waiting capacity of 1024. | | `ConcurrencyLimit > 0`, `QueueCapacity != nil` | Limit active generations and use the pointed-to capacity exactly, including zero. | | `ConcurrencyLimit < 0` | Invalid engine configuration. | | `QueueCapacity != nil` and `*QueueCapacity < 0` | Invalid engine configuration. | | `ConcurrencyLimit == 0` and `QueueCapacity != nil` | Invalid engine configuration because a queue without an active limit has no defined consumer value. | `ConcurrencyLimit` counts simultaneous calls to the engine-owned internal model-client boundary for this backend. `QueueCapacity` controls additional accepted `Run` invocations beyond that limit. The maximum admitted runs for a limited backend is therefore: ```text ConcurrencyLimit + effective QueueCapacity ``` Guard that addition against integer overflow during backend validation. Do not impose an arbitrary upper bound beyond non-negativity and overflow safety. The `QueueCapacity` pointer exists only to distinguish omission from explicit zero. `WithBackend` and `NewEngine` must not retain the caller's pointer. `Backend` continues to have no stable JSON representation, and consumers remain directed to keyed literals. Do not add concurrency fields to `Profile`, `ExecutionTarget`, `ExecutionTargetOverride`, `RunRequest`, prompt or profile files, or stable prepared/result JSON. ### Built-In And Custom Defaults The backend registry owns these exact operational defaults: ```go const ( openRouterConcurrencyLimit = 16 defaultQueueCapacity = 1024 ) ``` The built-in `openrouter` definition has a normalized concurrency limit of 16 and queue capacity of 1024. Consumer registrations remain unlimited when concurrency is omitted. For a consumer backend with a positive limit and omitted queue capacity, normalize the queue capacity to 1024. Preserve an explicitly configured zero. Consumers still cannot replace the reserved `openrouter` registration. Endpoint-only profiles have no backend policy and remain unlimited. A selected backend retains its pool when a profile or request overrides only its endpoint. ### Internal Backend Representation Extend `internal/domain.Backend` with scalar policy values and explicit presence rather than retaining a pointer: ```go type Backend struct { // Existing fields... ConcurrencyLimit int QueueCapacity int QueueCapacitySet bool } type BackendCapacityPolicy struct { ConcurrencyLimit int QueueCapacity int } ``` `WithBackend` converts the public pointer into `QueueCapacity` plus `QueueCapacitySet`. Registry normalization validates the combinations above, fills the default, and leaves every limited stored backend with `QueueCapacitySet == true`. Unlimited stored backends retain zero values and `QueueCapacitySet == false`. Add this internal registry method: ```go func (r *Registry) CapacityPolicies() map[string]domain.BackendCapacityPolicy ``` It returns a newly allocated map containing only limited backends. Values are scalars, so callers cannot mutate registry state. The built-in OpenRouter policy is included. `GetBackend` continues returning a defensive backend copy, now including normalized scalar capacity metadata. Capacity policy is operational registry metadata. Do not merge it into an execution target or expose it to injected model clients. ### Public Capacity Error Add this root sentinel beside the other run errors in `engine.go`: ```go var ErrCapacityExceeded = errors.New("backend capacity exceeded") ``` Its GoDoc must state that it identifies a `Run` rejected because the selected backend has already admitted `ConcurrencyLimit + QueueCapacity` runs. It is not an invalid request, an LLM/provider rate-limit response, or an `ErrLLMGenerate` failure. The internal capacity component owns a corresponding internal `ErrCapacityExceeded`. Add its mapping in `publicErrorFor` before the broader generation and invalid-request cases. The public error must preserve the internal error through wrapping while matching `ErrCapacityExceeded` with `errors.Is`. A capacity rejection returns no partial result and must not invoke the artifact reader, renderer, schema loader, validator, or model client. Prompt, profile, and backend loading needed to select the pool may already have occurred. ### Internal Capacity Component Add `internal/capacity` as the single owner of engine-local run admission and active-generation permits. Use these package-level boundaries: ```go var ErrCapacityExceeded error type Manager struct { // Private immutable pool map. } func NewManager( policies map[string]domain.BackendCapacityPolicy, ) (*Manager, error) func (m *Manager) Admit( ctx context.Context, backendID string, ) (release func(), err error) func NewClient(m *Manager, next llm.Client) llm.Client ``` `NewManager` copies the supplied map and creates one independent pool per limited backend. Defensively reject blank IDs, non-positive concurrency limits, negative queue capacities, or total-capacity overflow even though the registry normally supplies normalized values. Construction creates no worker goroutines. An absent manager, blank backend ID, or ID absent from the policy map is unlimited: - `Admit` succeeds with a non-nil no-op release function; and - the client wrapper calls the next client directly. For a limited pool, `Admit` is immediate and context-aware: 1. return `ctx.Err()` if the context is already done; 2. under the pool lock, compare admitted runs with `ConcurrencyLimit + QueueCapacity`; 3. return an error matching internal `ErrCapacityExceeded` when full; or 4. increment admitted runs and return an idempotent release function. The release function decrements admission exactly once, even if accidentally called more than once. It does not release an active-generation permit; those permits have their own lifetime. ### FIFO Generation Permits `NewClient` returns an internal `llm.Client` wrapper around either the built-in client or the public-client adapter. It must preserve requests, successful responses, nil responses, and collaborator error identities exactly. `next` must be non-nil; `NewEngine` and internal runner construction maintain that invariant. A nil manager returns `next` unchanged. For a configured backend ID, the wrapper: 1. acquires one active-generation permit from the matching pool; 2. waits in FIFO order when the active count equals `ConcurrencyLimit`; 3. removes a canceled waiter and returns `ctx.Err()` when cancellation wins before the permit is granted; 4. invokes the next client only after a permit is granted; and 5. releases the permit with `defer` after every success, nil response, collaborator error, panic unwinding, or context outcome. Implement FIFO and cancellation explicitly with a mutex and an ordered waiter list. A channel used only as a counting semaphore is insufficient because it does not define FIFO ordering or safe removal of canceled waiters. Permit grant and cancellation must have one lock-protected linearization point. If cancellation removes the waiter first, do not invoke the next client. If grant wins first, invoke the next client with the caller's context; the next client may then observe cancellation normally. Never lose or double-release a permit in this race. Releasing a permit transfers it to the oldest non-canceled waiter before making it generally available. Different backend pools never share admission or active counts. The active wrapper enforces its limit even if an internal caller invokes it without a run admission lease. Bounded backlog is guaranteed for ordinary engine `Run` calls by the runner admission path; no public API exposes the wrapped internal client directly. ### Engine Assembly In `NewEngine`, after constructing the validated backend registry: 1. obtain `backendRegistry.CapacityPolicies()`; 2. construct one `capacity.Manager`; 3. construct the selected base internal LLM client exactly as today; 4. wrap that base client with `capacity.NewClient`; and 5. pass both the wrapped client and manager-as-admitter to the runner. Every `NewEngine` call constructs a distinct manager. Do not cache managers, pools, or policies in package globals. The wrapper must be applied after a public injected client is adapted to `internal/llm.Client`, so built-in and injected clients receive identical scheduling behavior. If `NewManager` reports a defensive configuration error, make `NewEngine` return an error matching `ErrInvalidConfig`. `Prepare` does not use the manager. An injected client remains required to be safe for concurrent calls because different backend pools and unlimited backends may still invoke it concurrently. ### Shared Two-Phase Preparation Refactor `internal/usecase.Runner` so `Prepare` and `Run` share one preparation pipeline with two private phases. Do not duplicate prompt/profile/backend selection or execution precedence. The first phase resolves only the state required before admission: 1. validate `PromptID`; 2. normalize the direct session ID; 3. load the prompt definition; 4. hash the original prompt definition at its existing error-order position; 5. select and load the execution profile; 6. resolve the selected backend; 7. resolve and validate the effective execution target and credentials; and 8. resolve the effective output contract without loading its schema. Return a private state value containing the loaded definition, normalized direct session, prompt-definition hash, selected profile ID, effective target, numeric-presence metadata, effective output contract, and preparation start time. Keep this value private to `internal/usecase`. The second phase consumes that state and performs: 1. structured-output schema loading; 2. artifact loading and input hashing; 3. message and prompt-session rendering; 4. direct-session application; 5. rendered-prompt hashing; and 6. `PreparedRun` construction and timing. Preserve every existing precedence rule, error identity, direct-session template bypass, hash input, selected identity, copy guarantee, and timing field. Do not reload the prompt, profile, or backend between phases. `Runner.Prepare` records its start time, runs both phases consecutively, and never calls admission. Its behavior and error ordering remain unchanged. `Runner.Run` records its existing run start time, runs the first preparation phase, and then calls: ```go release, err := r.admitter.Admit(ctx, effectiveBackendID) ``` Use a narrow use-case-owned interface with the same signature: ```go type RunAdmitter interface { Admit(context.Context, string) (func(), error) } ``` A nil admitter means unlimited behavior for internal constructors and tests. On successful admission, immediately `defer release()` around the remainder of the run. Then run the second preparation phase, initial generation, validation, and all repair attempts. If admission returns internal `capacity.ErrCapacityExceeded`, add useful backend context without changing its identity. If it returns `ctx.Err()`, preserve that identity directly rather than recategorizing it as invalid request or generation failure. This refactor intentionally replaces the current literal `Run`-calls-`Prepare` implementation with shared private phases. Update current-state documentation to describe one shared pipeline rather than retaining an inaccurate call-graph claim. ### Generation And Repair Lifetime The admission lease covers the entire accepted run: - second-phase preparation; - initial generation; - validation; - every repair; and - all failure and cancellation exits. Preparation and validation do not hold an active-generation permit. The wrapped client acquires a permit only around each actual `Generate` call. The runner's initial generation already carries the effective backend ID in `GenerateRequest.Target`. Preserve that value. `RepairRequest.Target` and the default repairer's generated request must continue carrying the same backend ID, allowing each repair to reacquire the same pool's active permit. When testing or constructing `NewRunnerWithRepairer`, pass the same wrapped client to both the runner and `NewDefaultOutputRepairer`. Do not add capacity state to `RepairRequest`, `ExecutionTarget`, or public generation values. A repair remains within its existing admission lease. It waits for a FIFO active permit but never performs a second bounded admission and therefore cannot fail merely because later runs filled the admission capacity. ### Error And Cancellation Semantics The required public outcomes are: | Situation | Required error identity | | --- | --- | | Admission capacity is full | `ErrCapacityExceeded` only; not `ErrInvalidRequest` or `ErrLLMGenerate`. | | Context is done before admission succeeds | Preserve `ctx.Err()`; do not return capacity exhaustion. | | Context cancels while waiting for an active permit | Preserve `ctx.Err()` through the existing `ErrLLMGenerate` generation category. | | Wrapped client fails after permit acquisition | Preserve existing `ErrLLMGenerate` and collaborator identities. | | Preparation or validation fails after admission | Preserve its existing category and release admission. | Maintain the existing rule that `Run` returns no partial result on any operational error. Do not add queue status to errors or results. `RunResult.Duration` continues to start at runner entry and therefore includes pre-admission resolution, accepted preparation, and active-permit waiting. `PreparedRun.DurationMS` continues to cover only its shared preparation phases; it does not include later generation waiting. Capacity-rejected calls have no result or timing value. ### Ownership And Concurrency Safety The registry, capacity policy map, pool map, and per-pool limits are immutable after engine construction. Only admission counts, active counts, and waiter lists are mutable and must be protected by the owning pool mutex. Do not retain public queue pointers, caller request values, contexts, or generation requests after their call completes. A canceled waiter must be unlinked so its context and request cannot remain reachable from the pool. Do not hold a pool mutex while: - loading or rendering prompts; - reading artifacts or schemas; - invoking a model client; - validating output; - closing a waiter notification channel if the implementation could re-enter pool code; or - calling consumer code. No scheduler operation may spawn a goroutine whose lifetime outlasts the calling `Run`. The zero steady-state goroutine count is part of the in-process/no-worker-lifecycle design. ## Test Ownership Use this ownership split and avoid repeating the full policy matrix at every layer: - `internal/backend/registry_test.go` owns normalization, validation, the exact OpenRouter policy, the custom default queue, explicit zero, unlimited omission, and policy-map copying. - `internal/capacity/manager_test.go` owns admission bounds, idempotent release, FIFO active permits, cancellation races, capacity recovery, independent pools, unlimited IDs, and observed peak concurrency. - `internal/capacity/client_test.go` owns wrapper request/response/error transparency and the rule that cancellation before grant does not invoke the next client. Combine these with manager tests if one coherent package test expresses the behavior more clearly. - `internal/usecase/runner_test.go` owns two-phase preparation parity, pool selection, admission before expensive work, admission release across run exits, `Prepare` bypass, and repair reuse of the admitted backend. - Root external-package tests own public configuration conversion, assembled engine-local behavior, endpoint-override routing, injected-client limiting, and public capacity/context error identities. - Existing model-client HTTP tests remain unchanged because scheduling does not alter the OpenAI-compatible wire contract. Concurrency tests must use test-owned limits such as one or two and channel-controlled blocking clients. Record observed active and peak counts under a mutex or atomics. Do not use `time.Sleep` to infer queue state. Package-internal tests may inspect a waiter list under its mutex through a small test helper when necessary to establish deterministic FIFO ordering; do not add production metrics or hooks solely for tests. Do not add separate tests for trivial scalar copies when registry or assembled behavior already protects them. ## Stage 1 — Backend Policy And Public Configuration **Status:** Complete. ### Goal Add the public and internal backend policy representation, normalize all configured states, and expose immutable normalized policies without changing runtime scheduling yet. ### Work 1. Add `ConcurrencyLimit` and `QueueCapacity` to `Backend` in `backends.go` with exact GoDoc for unlimited, defaulted, explicit-zero, invalid, and engine-scoped behavior. 2. Convert the public queue pointer into scalar value plus presence in `WithBackend`; do not retain the pointer. 3. Add the internal backend policy fields and `BackendCapacityPolicy` to `internal/domain/domain.go`. 4. Add the two registry-owned constants and configure the built-in OpenRouter definition with 16 and 1024. 5. Extend `normalizeBackend` with the fixed validation, defaulting, explicit zero, and overflow rules. 6. Add `Registry.CapacityPolicies`, returning only limited policies in a fresh map. 7. Update existing backend composite literals and assertions only where the new fields are relevant. Continue using keyed literals. ### Tests 1. Extend the exact built-in registry test with the OpenRouter limit and queue. 2. Add one coherent table covering unlimited omission, default queue, explicit-zero queue, negative values, queue-without-limit, and total overflow. 3. Extend the registry copy/normalization test to prove returned policy maps cannot mutate registry state. 4. Add root coverage only if needed to prove the public pointer/presence conversion; do not reproduce registry validation cases at the facade. ### Focused Validation Run: ```sh gofmt -w backends.go internal/domain/domain.go \ internal/backend/registry.go internal/backend/registry_test.go go test . ./internal/backend go vet . ./internal/backend git diff --check ``` Include another touched Go test file in `gofmt` only if it actually changed. ### Completion Gate This stage is complete when every public configuration state has one normalized internal meaning, OpenRouter exposes exactly 16/1024, custom backends remain unlimited by omission, and no runtime call is scheduled yet. ## Stage 2 — Engine-Local Capacity Manager **Status:** Pending. ### Goal Implement and prove the bounded admission mechanism and FIFO active-generation client wrapper independently of runner orchestration. ### Work 1. Add `internal/capacity/manager.go` with the manager, immutable policy copy, per-backend pools, internal error, immediate admission, idempotent release, and FIFO context-aware active permits. 2. Add `internal/capacity/client.go` with the transparent `llm.Client` wrapper. 3. Use mutex-protected waiter state and an ordered list; explicitly resolve grant-versus-cancel races. 4. Ensure unlimited and independent-pool fast paths avoid queue allocation. 5. Do not start workers, timers, cleanup goroutines, or process-global state. ### Tests 1. Add a compact constructor-validation table for blank IDs, non-positive limits, negative queues, and total-capacity overflow. 2. With a small configured policy, prove that exactly `limit + queueCapacity` admissions succeed, the next matches `ErrCapacityExceeded`, and a release permits another admission. 3. Prove release is idempotent. 4. Drive more blocked client calls than the active limit and assert observed peak concurrency never exceeds that limit. 5. Prove FIFO order with deterministic queue-entry synchronization. 6. Cancel the first and a middle waiter and prove they are removed, never call the wrapped client, and do not block later waiters. 7. Exercise the grant/cancel race repeatedly under `go test -race`, asserting no permit leak or double invocation. 8. Prove different backend IDs proceed independently and blank, unknown, or nil-manager paths remain unlimited. 9. Prove request values, successful and nil responses, and collaborator errors pass through unchanged after permit acquisition. ### Focused Validation Run: ```sh gofmt -w internal/capacity/manager.go \ internal/capacity/manager_test.go \ internal/capacity/client.go \ internal/capacity/client_test.go go test ./internal/capacity go test -race ./internal/capacity go vet ./internal/capacity git diff --check ``` If tests are combined into one file, omit the nonexistent file from `gofmt`. ### Completion Gate This stage is complete when the standalone component enforces relational admission and active limits, FIFO cancellation is race-safe, separate pools are independent, and the wrapper is transparent apart from waiting. ## Stage 3 — Shared Preparation And Early Run Admission **Status:** Pending. ### Goal Refactor runner preparation into one shared two-phase pipeline and place bounded admission after backend resolution but before schema, artifact, and rendering work. ### Work 1. Add the private pre-admission preparation state and split the existing `Prepare` logic according to the fixed design. 2. Make `Runner.Prepare` call both phases without an admitter. 3. Add the `RunAdmitter` interface and runner field. 4. Update `NewRunner` and `NewRunnerWithRepairer` to accept the optional admitter; update internal call sites with nil until root assembly is wired. 5. Change `Runner.Run` to use the first phase, admit by effective backend ID, defer the returned release, and then use the second phase. 6. Preserve all existing error precedence, target resolution, hashes, metadata, session behavior, and timing. 7. Return capacity and context errors with the fixed identities. Do not invoke later collaborators after rejection. ### Tests 1. Keep the existing `Run`/`Prepare` parity coverage passing to prove the shared phases do not drift. 2. Add a fake admitter that records backend IDs and release calls. 3. Prove a backend-selected run admits with the selected ID even when the endpoint is overridden. 4. Prove an endpoint-only run uses the unlimited/blank identity and that `Prepare` never calls admission. 5. Reject admission and assert schema, artifact, renderer, validator, repairer, and LLM collaborators are not invoked. 6. Prove admission is released after one successful run and representative second-phase, generation, and validation errors. Prefer a small table around the single `defer` invariant rather than duplicating every error test. 7. Retain direct-session, backend precedence, credential, hashing, and repair tests unchanged except for constructor arguments. ### Focused Validation Run: ```sh gofmt -w internal/usecase/runner.go \ internal/usecase/runner_test.go go test ./internal/usecase go test -race ./internal/usecase go vet ./internal/usecase git diff --check ``` ### Completion Gate This stage is complete when `Prepare` remains unrestricted, `Run` admits after one canonical routing phase and before expensive completion work, every exit releases admission, and existing preparation semantics remain unchanged. ## Stage 4 — Engine Assembly And Public Runtime Contract **Status:** Pending. ### Goal Wire one manager into each engine, schedule built-in and injected clients, expose the capacity error, and prove assembled runtime behavior. ### Work 1. Add public `ErrCapacityExceeded` and its exact GoDoc in `engine.go`. 2. Map internal capacity exhaustion in `errors.go`. 3. Construct the manager from the registry policy snapshot in `NewEngine`. 4. Wrap the selected internal client after built-in or injected-client selection and pass the manager and wrapped client to the runner. 5. Update `Engine`, `NewEngine`, `Run`, `WithLLMClient`, and `LLMClient` GoDoc only where concurrency, capacity, or cancellation statements change. 6. Ensure manager-construction errors match `ErrInvalidConfig`. 7. For internal repair coverage, construct the default repairer with the same wrapped client used by its runner and confirm repair target backend identity remains intact. ### Tests 1. Add an external-package assembled test with a small custom limit and a blocking injected client; assert peak generation equals or remains below the configured limit. 2. With queue capacity zero, block one accepted run before generation and assert the next matching-backend run returns `ErrCapacityExceeded`, does not match `ErrInvalidRequest` or `ErrLLMGenerate`, returns no result, and never reaches expensive collaborators or the client. 3. In the same or another focused workflow, prove an endpoint override remains in the selected backend's pool. 4. Prove two engines with the same backend ID have independent capacity. 5. Prove an unlimited custom backend and an endpoint-only profile preserve concurrent behavior. 6. Cancel a call waiting for an active permit; assert it matches both `context.Canceled` and `ErrLLMGenerate`, never invokes the injected client, and leaves capacity reusable. 7. Add one internal repair workflow with concurrent runs or controlled permits showing initial and repair generations never exceed the same backend limit and repairs do not perform a second admission. 8. Extend the public error sentinel contract test with `ErrCapacityExceeded`. Avoid a second HTTP-level concurrency suite: the capacity client tests and one assembled injected-client workflow already protect the shared wrapper used by the built-in client. ### Focused Validation Run: ```sh gofmt -w engine.go errors.go backends.go \ internal/usecase/runner.go internal/usecase/runner_test.go \ engine_test.go public_contract_test.go go test . ./internal/backend ./internal/capacity ./internal/usecase go test -race . ./internal/capacity ./internal/usecase go vet . ./internal/backend ./internal/capacity ./internal/usecase git diff --check ``` Add any newly created capacity files to `gofmt` when they changed in this stage. ### Completion Gate This stage is complete when every engine has independent pools, limited runs are bounded and FIFO at generation, endpoint routing is correct, capacity and context errors are stable, repairs reuse admission, and both client kinds pass through the same wrapper. ## Stage 5 — Durable Documentation And Final Validation **Status:** Pending. ### Goal Move implemented contracts into their durable owners, record compatibility impact, and validate the complete repository. ### Work 1. Review every changed exported declaration. Ensure GoDoc is the canonical owner of exact field types, nil/zero semantics, defaulting, error identity, engine scope, concurrency safety, cancellation, and source compatibility. 2. Update `doc.go` so its concurrency summary acknowledges backend scheduling while continuing to require injected collaborators to be concurrency-safe. 3. Update `docs/consumers/pkg-promptkit.md` with task-oriented examples for: - a limited local backend; - omitted queue capacity selecting 1024; - explicit zero queue capacity; and - handling `ErrCapacityExceeded`. Keep exact field semantics in GoDoc rather than duplicating a full table. 4. Add `docs/internal/capacity.md` as the durable owner of pool lifecycle, admission, FIFO active permits, cancellation, client wrapping, and test ownership. 5. Add `internal/capacity` to `docs/internal/overview.md`. 6. Update `docs/policy/architecture.md` to include the implemented component and root assembly dependency without turning policy into an API reference. 7. Update `docs/internal/runner.md` to describe the shared two-phase preparation pipeline, early bounded admission, lease lifetime, generation permits, repairs, capacity failures, and cancellation. 8. Review `docs/formats.md`; add only a concise link or clarification if needed to explain that endpoint overrides preserve backend capacity identity. Do not add concurrency fields to YAML. 9. Do not change the OpenAI-compatible integration contract or `docs/internal/llm.md` unless implementation changes their current statements; scheduling is outside the provider wire contract and concrete model-client implementation. 10. Record in the implementation handoff that built-in OpenRouter now limits active generations to 16 with queue capacity 1024 and that the release must be a pre-`v1` minor release. Do not edit the release procedure or create a tag. 11. After every check passes, set `concurrency.md`, this implementation plan, and each stage status to `Complete`. Do not remove the roadmaps in the implementation change; lifecycle retirement follows review. ### Full Validation Run the complete sequence from `docs/development.md`: ```sh go test ./... go test -race ./... go vet ./... go build ./... go run ./examples/go-library/prepare gofmt -l $(git ls-files '*.go') git diff --check ``` The formatting command must produce no paths. Follow every added or changed Markdown link and confirm its target and heading exist. Also inspect: ```sh git status --short git diff --stat git diff ``` Confirm that: - only intended backend, capacity, runner, facade, test, documentation, and roadmap files changed; - no `go.work`, `go.work.sum`, local module replacement, credential, generated binary, coverage output, or unrelated change was introduced; - the built-in OpenRouter policy is exactly 16/1024; - custom and endpoint-only backends remain unlimited by omission; - explicit queue zero is distinguishable from omission; - no capacity value enters execution targets, generated requests, stable JSON, prompt/profile YAML, or provider payloads; - every engine owns distinct pools with no package-global mutable state; - every initial and repair generation uses the active permit wrapper; - capacity and waiter state is released on success, error, panic unwinding, and cancellation; - concurrency tests use deterministic coordination rather than sleeps; - current-state documentation describes implemented behavior rather than referring readers to the roadmaps; and - the feature and implementation roadmaps contain no unresolved work marked complete. ### Completion Gate The implementation is complete only when every target-end-state item in `concurrency.md` is implemented, race-enabled tests demonstrate the configured limits and cancellation safety, durable contracts no longer depend on roadmap prose, and the OpenRouter compatibility change is clearly reported for the next minor release. ## Open Questions None. The feature roadmap and this plan fix the public representation, registry defaults, admission bound, FIFO generation behavior, early-routing refactor, cancellation races, error identities, engine and repair lifetimes, test ownership, compatibility treatment, and non-goals required for implementation.