promptkit/docs/roadmap/implementation.md

# Backend-Specific Concurrency Management Implementation Plan

**Status:** Ready for implementation.

## Purpose

This document is the decision-complete implementation plan for
[backend-specific concurrency management](concurrency.md). It is written for a
coding agent that will implement each stage in order.

The feature roadmap owns the intended capability, consumer value, policy
choices, compatibility decision, and target end state. This document owns the
concrete API, internal representation, scheduling architecture, implementation
sequence, test ownership, documentation updates, and completion gates.

## Implementation Rules

- Complete the stages in order. Keep the repository compiling and the focused
  tests passing at every stage boundary.
- Preserve unrelated working-tree changes. In particular, `concurrency.md` and
  the removal of its source idea from `future.md` may already be uncommitted
  when implementation begins; retain both.
- Follow every policy under `docs/policy/`, the task-specific reading guide in
  `docs/development.md`, and the target behavior in `concurrency.md`.
- Keep the public API in the root `promptkit` package and implementation
  details under `internal/`. Do not expose scheduler types or create another
  public package.
- Use only the Go standard library for scheduling. Do not add a queue,
  semaphore, worker-pool, or metrics dependency.
- Preserve synchronous, wait-for-result `Run`, unrestricted `Prepare`,
  engine-local state, endpoint-only profiles, backend-selected profiles,
  backend identity through endpoint overrides, and injected `LLMClient`
  behavior.
- Do not broaden the work into asynchronous jobs, durable queues, retries,
  rate limiting, dynamic configuration, priorities, worker lifecycle,
  endpoint-keyed pools, or public queue observability.
- Keep all tests deterministic, bounded, offline, and race-safe. Coordinate
  concurrent tests with channels and barriers rather than timing assumptions
  or live providers.
- Update exact GoDoc with each exported declaration change. Update durable
  current-state documents only after the corresponding behavior is
  implemented.
- Test configurable mechanisms with small test-owned limits. Assert the exact
  OpenRouter `16` and default queue `1024` values only at the registry contract
  that owns those operational defaults.
- Do not create a release, change a module version, or tag a commit. The final
  implementation handoff must identify the built-in OpenRouter behavior change
  for the next pre-`v1` minor release.

## Fixed Design

### Public Backend Configuration

Append these fields to the existing root `Backend` type in `backends.go`:

```go
type Backend struct {
	// Existing fields remain unchanged and in their current order.

	ConcurrencyLimit int
	QueueCapacity    *int
}
```

Use these exact semantics:

| Public values | Meaning |
| --- | --- |
| `ConcurrencyLimit == 0`, `QueueCapacity == nil` | Unlimited backend; preserve current behavior. |
| `ConcurrencyLimit > 0`, `QueueCapacity == nil` | Limit active generations and use the default waiting capacity of 1024. |
| `ConcurrencyLimit > 0`, `QueueCapacity != nil` | Limit active generations and use the pointed-to capacity exactly, including zero. |
| `ConcurrencyLimit < 0` | Invalid engine configuration. |
| `QueueCapacity != nil` and `*QueueCapacity < 0` | Invalid engine configuration. |
| `ConcurrencyLimit == 0` and `QueueCapacity != nil` | Invalid engine configuration because a queue without an active limit has no defined consumer value. |

`ConcurrencyLimit` counts simultaneous calls to the engine-owned internal
model-client boundary for this backend. `QueueCapacity` controls additional
accepted `Run` invocations beyond that limit. The maximum admitted runs for a
limited backend is therefore:

```text
ConcurrencyLimit + effective QueueCapacity
```

Guard that addition against integer overflow during backend validation.
Do not impose an arbitrary upper bound beyond non-negativity and overflow
safety.

The `QueueCapacity` pointer exists only to distinguish omission from explicit
zero. `WithBackend` and `NewEngine` must not retain the caller's pointer.
`Backend` continues to have no stable JSON representation, and consumers
remain directed to keyed literals.

Do not add concurrency fields to `Profile`, `ExecutionTarget`,
`ExecutionTargetOverride`, `RunRequest`, prompt or profile files, or stable
prepared/result JSON.

### Built-In And Custom Defaults

The backend registry owns these exact operational defaults:

```go
const (
	openRouterConcurrencyLimit = 16
	defaultQueueCapacity       = 1024
)
```

The built-in `openrouter` definition has a normalized concurrency limit of 16
and queue capacity of 1024.

Consumer registrations remain unlimited when concurrency is omitted. For a
consumer backend with a positive limit and omitted queue capacity, normalize
the queue capacity to 1024. Preserve an explicitly configured zero.

Consumers still cannot replace the reserved `openrouter` registration.
Endpoint-only profiles have no backend policy and remain unlimited. A selected
backend retains its pool when a profile or request overrides only its endpoint.

### Internal Backend Representation

Extend `internal/domain.Backend` with scalar policy values and explicit
presence rather than retaining a pointer:

```go
type Backend struct {
	// Existing fields...
	ConcurrencyLimit int
	QueueCapacity    int
	QueueCapacitySet bool
}

type BackendCapacityPolicy struct {
	ConcurrencyLimit int
	QueueCapacity    int
}
```

`WithBackend` converts the public pointer into `QueueCapacity` plus
`QueueCapacitySet`. Registry normalization validates the combinations above,
fills the default, and leaves every limited stored backend with
`QueueCapacitySet == true`. Unlimited stored backends retain zero values and
`QueueCapacitySet == false`.

Add this internal registry method:

```go
func (r *Registry) CapacityPolicies() map[string]domain.BackendCapacityPolicy
```

It returns a newly allocated map containing only limited backends. Values are
scalars, so callers cannot mutate registry state. The built-in OpenRouter
policy is included. `GetBackend` continues returning a defensive backend copy,
now including normalized scalar capacity metadata.

Capacity policy is operational registry metadata. Do not merge it into an
execution target or expose it to injected model clients.

### Public Capacity Error

Add this root sentinel beside the other run errors in `engine.go`:

```go
var ErrCapacityExceeded = errors.New("backend capacity exceeded")
```

Its GoDoc must state that it identifies a `Run` rejected because the selected
backend has already admitted `ConcurrencyLimit + QueueCapacity` runs. It is
not an invalid request, an LLM/provider rate-limit response, or an
`ErrLLMGenerate` failure.

The internal capacity component owns a corresponding internal
`ErrCapacityExceeded`. Add its mapping in `publicErrorFor` before the broader
generation and invalid-request cases. The public error must preserve the
internal error through wrapping while matching `ErrCapacityExceeded` with
`errors.Is`.

A capacity rejection returns no partial result and must not invoke the
artifact reader, renderer, schema loader, validator, or model client. Prompt,
profile, and backend loading needed to select the pool may already have
occurred.

### Internal Capacity Component

Add `internal/capacity` as the single owner of engine-local run admission and
active-generation permits.

Use these package-level boundaries:

```go
var ErrCapacityExceeded error

type Manager struct {
	// Private immutable pool map.
}

func NewManager(
	policies map[string]domain.BackendCapacityPolicy,
) (*Manager, error)

func (m *Manager) Admit(
	ctx context.Context,
	backendID string,
) (release func(), err error)

func NewClient(m *Manager, next llm.Client) llm.Client
```

`NewManager` copies the supplied map and creates one independent pool per
limited backend. Defensively reject blank IDs, non-positive concurrency
limits, negative queue capacities, or total-capacity overflow even though the
registry normally supplies normalized values. Construction creates no worker
goroutines.

An absent manager, blank backend ID, or ID absent from the policy map is
unlimited:

- `Admit` succeeds with a non-nil no-op release function; and
- the client wrapper calls the next client directly.

For a limited pool, `Admit` is immediate and context-aware:

1. return `ctx.Err()` if the context is already done;
2. under the pool lock, compare admitted runs with
   `ConcurrencyLimit + QueueCapacity`;
3. return an error matching internal `ErrCapacityExceeded` when full; or
4. increment admitted runs and return an idempotent release function.

The release function decrements admission exactly once, even if accidentally
called more than once. It does not release an active-generation permit; those
permits have their own lifetime.

### FIFO Generation Permits

`NewClient` returns an internal `llm.Client` wrapper around either the built-in
client or the public-client adapter. It must preserve requests, successful
responses, nil responses, and collaborator error identities exactly.
`next` must be non-nil; `NewEngine` and internal runner construction maintain
that invariant. A nil manager returns `next` unchanged.

For a configured backend ID, the wrapper:

1. acquires one active-generation permit from the matching pool;
2. waits in FIFO order when the active count equals `ConcurrencyLimit`;
3. removes a canceled waiter and returns `ctx.Err()` when cancellation wins
   before the permit is granted;
4. invokes the next client only after a permit is granted; and
5. releases the permit with `defer` after every success, nil response,
   collaborator error, panic unwinding, or context outcome.

Implement FIFO and cancellation explicitly with a mutex and an ordered waiter
list. A channel used only as a counting semaphore is insufficient because it
does not define FIFO ordering or safe removal of canceled waiters.

Permit grant and cancellation must have one lock-protected linearization
point. If cancellation removes the waiter first, do not invoke the next
client. If grant wins first, invoke the next client with the caller's context;
the next client may then observe cancellation normally. Never lose or
double-release a permit in this race.

Releasing a permit transfers it to the oldest non-canceled waiter before
making it generally available. Different backend pools never share admission
or active counts.

The active wrapper enforces its limit even if an internal caller invokes it
without a run admission lease. Bounded backlog is guaranteed for ordinary
engine `Run` calls by the runner admission path; no public API exposes the
wrapped internal client directly.

### Engine Assembly

In `NewEngine`, after constructing the validated backend registry:

1. obtain `backendRegistry.CapacityPolicies()`;
2. construct one `capacity.Manager`;
3. construct the selected base internal LLM client exactly as today;
4. wrap that base client with `capacity.NewClient`; and
5. pass both the wrapped client and manager-as-admitter to the runner.

Every `NewEngine` call constructs a distinct manager. Do not cache managers,
pools, or policies in package globals. The wrapper must be applied after a
public injected client is adapted to `internal/llm.Client`, so built-in and
injected clients receive identical scheduling behavior.

If `NewManager` reports a defensive configuration error, make `NewEngine`
return an error matching `ErrInvalidConfig`.

`Prepare` does not use the manager. An injected client remains required to be
safe for concurrent calls because different backend pools and unlimited
backends may still invoke it concurrently.

### Shared Two-Phase Preparation

Refactor `internal/usecase.Runner` so `Prepare` and `Run` share one preparation
pipeline with two private phases. Do not duplicate prompt/profile/backend
selection or execution precedence.

The first phase resolves only the state required before admission:

1. validate `PromptID`;
2. normalize the direct session ID;
3. load the prompt definition;
4. hash the original prompt definition at its existing error-order position;
5. select and load the execution profile;
6. resolve the selected backend;
7. resolve and validate the effective execution target and credentials; and
8. resolve the effective output contract without loading its schema.

Return a private state value containing the loaded definition, normalized
direct session, prompt-definition hash, selected profile ID, effective target,
numeric-presence metadata, effective output contract, and preparation start
time. Keep this value private to `internal/usecase`.

The second phase consumes that state and performs:

1. structured-output schema loading;
2. artifact loading and input hashing;
3. message and prompt-session rendering;
4. direct-session application;
5. rendered-prompt hashing; and
6. `PreparedRun` construction and timing.

Preserve every existing precedence rule, error identity, direct-session
template bypass, hash input, selected identity, copy guarantee, and timing
field. Do not reload the prompt, profile, or backend between phases.

`Runner.Prepare` records its start time, runs both phases consecutively, and
never calls admission. Its behavior and error ordering remain unchanged.

`Runner.Run` records its existing run start time, runs the first preparation
phase, and then calls:

```go
release, err := r.admitter.Admit(ctx, effectiveBackendID)
```

Use a narrow use-case-owned interface with the same signature:

```go
type RunAdmitter interface {
	Admit(context.Context, string) (func(), error)
}
```

A nil admitter means unlimited behavior for internal constructors and tests.
On successful admission, immediately `defer release()` around the remainder of
the run. Then run the second preparation phase, initial generation,
validation, and all repair attempts.

If admission returns internal `capacity.ErrCapacityExceeded`, add useful
backend context without changing its identity. If it returns `ctx.Err()`,
preserve that identity directly rather than recategorizing it as invalid
request or generation failure.

This refactor intentionally replaces the current literal `Run`-calls-`Prepare`
implementation with shared private phases. Update current-state documentation
to describe one shared pipeline rather than retaining an inaccurate call-graph
claim.

### Generation And Repair Lifetime

The admission lease covers the entire accepted run:

- second-phase preparation;
- initial generation;
- validation;
- every repair; and
- all failure and cancellation exits.

Preparation and validation do not hold an active-generation permit. The
wrapped client acquires a permit only around each actual `Generate` call.

The runner's initial generation already carries the effective backend ID in
`GenerateRequest.Target`. Preserve that value. `RepairRequest.Target` and the
default repairer's generated request must continue carrying the same backend
ID, allowing each repair to reacquire the same pool's active permit.

When testing or constructing `NewRunnerWithRepairer`, pass the same wrapped
client to both the runner and `NewDefaultOutputRepairer`. Do not add capacity
state to `RepairRequest`, `ExecutionTarget`, or public generation values.

A repair remains within its existing admission lease. It waits for a FIFO
active permit but never performs a second bounded admission and therefore
cannot fail merely because later runs filled the admission capacity.

### Error And Cancellation Semantics

The required public outcomes are:

| Situation | Required error identity |
| --- | --- |
| Admission capacity is full | `ErrCapacityExceeded` only; not `ErrInvalidRequest` or `ErrLLMGenerate`. |
| Context is done before admission succeeds | Preserve `ctx.Err()`; do not return capacity exhaustion. |
| Context cancels while waiting for an active permit | Preserve `ctx.Err()` through the existing `ErrLLMGenerate` generation category. |
| Wrapped client fails after permit acquisition | Preserve existing `ErrLLMGenerate` and collaborator identities. |
| Preparation or validation fails after admission | Preserve its existing category and release admission. |

Maintain the existing rule that `Run` returns no partial result on any
operational error. Do not add queue status to errors or results.

`RunResult.Duration` continues to start at runner entry and therefore includes
pre-admission resolution, accepted preparation, and active-permit waiting.
`PreparedRun.DurationMS` continues to cover only its shared preparation phases;
it does not include later generation waiting. Capacity-rejected calls have no
result or timing value.

### Ownership And Concurrency Safety

The registry, capacity policy map, pool map, and per-pool limits are immutable
after engine construction. Only admission counts, active counts, and waiter
lists are mutable and must be protected by the owning pool mutex.

Do not retain public queue pointers, caller request values, contexts, or
generation requests after their call completes. A canceled waiter must be
unlinked so its context and request cannot remain reachable from the pool.

Do not hold a pool mutex while:

- loading or rendering prompts;
- reading artifacts or schemas;
- invoking a model client;
- validating output;
- closing a waiter notification channel if the implementation could re-enter
  pool code; or
- calling consumer code.

No scheduler operation may spawn a goroutine whose lifetime outlasts the
calling `Run`. The zero steady-state goroutine count is part of the
in-process/no-worker-lifecycle design.

## Test Ownership

Use this ownership split and avoid repeating the full policy matrix at every
layer:

- `internal/backend/registry_test.go` owns normalization, validation, the exact
  OpenRouter policy, the custom default queue, explicit zero, unlimited
  omission, and policy-map copying.
- `internal/capacity/manager_test.go` owns admission bounds, idempotent release,
  FIFO active permits, cancellation races, capacity recovery, independent
  pools, unlimited IDs, and observed peak concurrency.
- `internal/capacity/client_test.go` owns wrapper request/response/error
  transparency and the rule that cancellation before grant does not invoke the
  next client. Combine these with manager tests if one coherent package test
  expresses the behavior more clearly.
- `internal/usecase/runner_test.go` owns two-phase preparation parity, pool
  selection, admission before expensive work, admission release across run
  exits, `Prepare` bypass, and repair reuse of the admitted backend.
- Root external-package tests own public configuration conversion, assembled
  engine-local behavior, endpoint-override routing, injected-client limiting,
  and public capacity/context error identities.
- Existing model-client HTTP tests remain unchanged because scheduling does
  not alter the OpenAI-compatible wire contract.

Concurrency tests must use test-owned limits such as one or two and
channel-controlled blocking clients. Record observed active and peak counts
under a mutex or atomics. Do not use `time.Sleep` to infer queue state.
Package-internal tests may inspect a waiter list under its mutex through a
small test helper when necessary to establish deterministic FIFO ordering; do
not add production metrics or hooks solely for tests.

Do not add separate tests for trivial scalar copies when registry or assembled
behavior already protects them.

## Stage 1 — Backend Policy And Public Configuration

**Status:** Complete.

### Goal

Add the public and internal backend policy representation, normalize all
configured states, and expose immutable normalized policies without changing
runtime scheduling yet.

### Work

1. Add `ConcurrencyLimit` and `QueueCapacity` to `Backend` in `backends.go`
   with exact GoDoc for unlimited, defaulted, explicit-zero, invalid, and
   engine-scoped behavior.
2. Convert the public queue pointer into scalar value plus presence in
   `WithBackend`; do not retain the pointer.
3. Add the internal backend policy fields and
   `BackendCapacityPolicy` to `internal/domain/domain.go`.
4. Add the two registry-owned constants and configure the built-in OpenRouter
   definition with 16 and 1024.
5. Extend `normalizeBackend` with the fixed validation, defaulting, explicit
   zero, and overflow rules.
6. Add `Registry.CapacityPolicies`, returning only limited policies in a fresh
   map.
7. Update existing backend composite literals and assertions only where the
   new fields are relevant. Continue using keyed literals.

### Tests

1. Extend the exact built-in registry test with the OpenRouter limit and queue.
2. Add one coherent table covering unlimited omission, default queue,
   explicit-zero queue, negative values, queue-without-limit, and total
   overflow.
3. Extend the registry copy/normalization test to prove returned policy maps
   cannot mutate registry state.
4. Add root coverage only if needed to prove the public pointer/presence
   conversion; do not reproduce registry validation cases at the facade.

### Focused Validation

Run:

```sh
gofmt -w backends.go internal/domain/domain.go \
  internal/backend/registry.go internal/backend/registry_test.go
go test . ./internal/backend
go vet . ./internal/backend
git diff --check
```

Include another touched Go test file in `gofmt` only if it actually changed.

### Completion Gate

This stage is complete when every public configuration state has one normalized
internal meaning, OpenRouter exposes exactly 16/1024, custom backends remain
unlimited by omission, and no runtime call is scheduled yet.

## Stage 2 — Engine-Local Capacity Manager

**Status:** Pending.

### Goal

Implement and prove the bounded admission mechanism and FIFO active-generation
client wrapper independently of runner orchestration.

### Work

1. Add `internal/capacity/manager.go` with the manager, immutable policy copy,
   per-backend pools, internal error, immediate admission, idempotent release,
   and FIFO context-aware active permits.
2. Add `internal/capacity/client.go` with the transparent `llm.Client` wrapper.
3. Use mutex-protected waiter state and an ordered list; explicitly resolve
   grant-versus-cancel races.
4. Ensure unlimited and independent-pool fast paths avoid queue allocation.
5. Do not start workers, timers, cleanup goroutines, or process-global state.

### Tests

1. Add a compact constructor-validation table for blank IDs, non-positive
   limits, negative queues, and total-capacity overflow.
2. With a small configured policy, prove that exactly
   `limit + queueCapacity` admissions succeed, the next matches
   `ErrCapacityExceeded`, and a release permits another admission.
3. Prove release is idempotent.
4. Drive more blocked client calls than the active limit and assert observed
   peak concurrency never exceeds that limit.
5. Prove FIFO order with deterministic queue-entry synchronization.
6. Cancel the first and a middle waiter and prove they are removed, never call
   the wrapped client, and do not block later waiters.
7. Exercise the grant/cancel race repeatedly under `go test -race`, asserting
   no permit leak or double invocation.
8. Prove different backend IDs proceed independently and blank, unknown, or
   nil-manager paths remain unlimited.
9. Prove request values, successful and nil responses, and collaborator errors
   pass through unchanged after permit acquisition.

### Focused Validation

Run:

```sh
gofmt -w internal/capacity/manager.go \
  internal/capacity/manager_test.go \
  internal/capacity/client.go \
  internal/capacity/client_test.go
go test ./internal/capacity
go test -race ./internal/capacity
go vet ./internal/capacity
git diff --check
```

If tests are combined into one file, omit the nonexistent file from `gofmt`.

### Completion Gate

This stage is complete when the standalone component enforces relational
admission and active limits, FIFO cancellation is race-safe, separate pools
are independent, and the wrapper is transparent apart from waiting.

## Stage 3 — Shared Preparation And Early Run Admission

**Status:** Pending.

### Goal

Refactor runner preparation into one shared two-phase pipeline and place
bounded admission after backend resolution but before schema, artifact, and
rendering work.

### Work

1. Add the private pre-admission preparation state and split the existing
   `Prepare` logic according to the fixed design.
2. Make `Runner.Prepare` call both phases without an admitter.
3. Add the `RunAdmitter` interface and runner field.
4. Update `NewRunner` and `NewRunnerWithRepairer` to accept the optional
   admitter; update internal call sites with nil until root assembly is wired.
5. Change `Runner.Run` to use the first phase, admit by effective backend ID,
   defer the returned release, and then use the second phase.
6. Preserve all existing error precedence, target resolution, hashes,
   metadata, session behavior, and timing.
7. Return capacity and context errors with the fixed identities. Do not invoke
   later collaborators after rejection.

### Tests

1. Keep the existing `Run`/`Prepare` parity coverage passing to prove the
   shared phases do not drift.
2. Add a fake admitter that records backend IDs and release calls.
3. Prove a backend-selected run admits with the selected ID even when the
   endpoint is overridden.
4. Prove an endpoint-only run uses the unlimited/blank identity and that
   `Prepare` never calls admission.
5. Reject admission and assert schema, artifact, renderer, validator, repairer,
   and LLM collaborators are not invoked.
6. Prove admission is released after one successful run and representative
   second-phase, generation, and validation errors. Prefer a small table around
   the single `defer` invariant rather than duplicating every error test.
7. Retain direct-session, backend precedence, credential, hashing, and repair
   tests unchanged except for constructor arguments.

### Focused Validation

Run:

```sh
gofmt -w internal/usecase/runner.go \
  internal/usecase/runner_test.go
go test ./internal/usecase
go test -race ./internal/usecase
go vet ./internal/usecase
git diff --check
```

### Completion Gate

This stage is complete when `Prepare` remains unrestricted, `Run` admits after
one canonical routing phase and before expensive completion work, every exit
releases admission, and existing preparation semantics remain unchanged.

## Stage 4 — Engine Assembly And Public Runtime Contract

**Status:** Pending.

### Goal

Wire one manager into each engine, schedule built-in and injected clients,
expose the capacity error, and prove assembled runtime behavior.

### Work

1. Add public `ErrCapacityExceeded` and its exact GoDoc in `engine.go`.
2. Map internal capacity exhaustion in `errors.go`.
3. Construct the manager from the registry policy snapshot in `NewEngine`.
4. Wrap the selected internal client after built-in or injected-client
   selection and pass the manager and wrapped client to the runner.
5. Update `Engine`, `NewEngine`, `Run`, `WithLLMClient`, and `LLMClient` GoDoc
   only where concurrency, capacity, or cancellation statements change.
6. Ensure manager-construction errors match `ErrInvalidConfig`.
7. For internal repair coverage, construct the default repairer with the same
   wrapped client used by its runner and confirm repair target backend identity
   remains intact.

### Tests

1. Add an external-package assembled test with a small custom limit and a
   blocking injected client; assert peak generation equals or remains below
   the configured limit.
2. With queue capacity zero, block one accepted run before generation and
   assert the next matching-backend run returns `ErrCapacityExceeded`, does not
   match `ErrInvalidRequest` or `ErrLLMGenerate`, returns no result, and never
   reaches expensive collaborators or the client.
3. In the same or another focused workflow, prove an endpoint override remains
   in the selected backend's pool.
4. Prove two engines with the same backend ID have independent capacity.
5. Prove an unlimited custom backend and an endpoint-only profile preserve
   concurrent behavior.
6. Cancel a call waiting for an active permit; assert it matches both
   `context.Canceled` and `ErrLLMGenerate`, never invokes the injected client,
   and leaves capacity reusable.
7. Add one internal repair workflow with concurrent runs or controlled permits
   showing initial and repair generations never exceed the same backend limit
   and repairs do not perform a second admission.
8. Extend the public error sentinel contract test with
   `ErrCapacityExceeded`.

Avoid a second HTTP-level concurrency suite: the capacity client tests and one
assembled injected-client workflow already protect the shared wrapper used by
the built-in client.

### Focused Validation

Run:

```sh
gofmt -w engine.go errors.go backends.go \
  internal/usecase/runner.go internal/usecase/runner_test.go \
  engine_test.go public_contract_test.go
go test . ./internal/backend ./internal/capacity ./internal/usecase
go test -race . ./internal/capacity ./internal/usecase
go vet . ./internal/backend ./internal/capacity ./internal/usecase
git diff --check
```

Add any newly created capacity files to `gofmt` when they changed in this
stage.

### Completion Gate

This stage is complete when every engine has independent pools, limited runs
are bounded and FIFO at generation, endpoint routing is correct, capacity and
context errors are stable, repairs reuse admission, and both client kinds pass
through the same wrapper.

## Stage 5 — Durable Documentation And Final Validation

**Status:** Pending.

### Goal

Move implemented contracts into their durable owners, record compatibility
impact, and validate the complete repository.

### Work

1. Review every changed exported declaration. Ensure GoDoc is the canonical
   owner of exact field types, nil/zero semantics, defaulting, error identity,
   engine scope, concurrency safety, cancellation, and source compatibility.
2. Update `doc.go` so its concurrency summary acknowledges backend scheduling
   while continuing to require injected collaborators to be concurrency-safe.
3. Update `docs/consumers/pkg-promptkit.md` with task-oriented examples for:
   - a limited local backend;
   - omitted queue capacity selecting 1024;
   - explicit zero queue capacity; and
   - handling `ErrCapacityExceeded`.
   Keep exact field semantics in GoDoc rather than duplicating a full table.
4. Add `docs/internal/capacity.md` as the durable owner of pool lifecycle,
   admission, FIFO active permits, cancellation, client wrapping, and test
   ownership.
5. Add `internal/capacity` to `docs/internal/overview.md`.
6. Update `docs/policy/architecture.md` to include the implemented component
   and root assembly dependency without turning policy into an API reference.
7. Update `docs/internal/runner.md` to describe the shared two-phase
   preparation pipeline, early bounded admission, lease lifetime, generation
   permits, repairs, capacity failures, and cancellation.
8. Review `docs/formats.md`; add only a concise link or clarification if needed
   to explain that endpoint overrides preserve backend capacity identity.
   Do not add concurrency fields to YAML.
9. Do not change the OpenAI-compatible integration contract or
   `docs/internal/llm.md` unless implementation changes their current
   statements; scheduling is outside the provider wire contract and concrete
   model-client implementation.
10. Record in the implementation handoff that built-in OpenRouter now limits
    active generations to 16 with queue capacity 1024 and that the release
    must be a pre-`v1` minor release. Do not edit the release procedure or
    create a tag.
11. After every check passes, set `concurrency.md`, this implementation plan,
    and each stage status to `Complete`. Do not remove the roadmaps in the
    implementation change; lifecycle retirement follows review.

### Full Validation

Run the complete sequence from `docs/development.md`:

```sh
go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
gofmt -l $(git ls-files '*.go')
git diff --check
```

The formatting command must produce no paths. Follow every added or changed
Markdown link and confirm its target and heading exist.

Also inspect:

```sh
git status --short
git diff --stat
git diff
```

Confirm that:

- only intended backend, capacity, runner, facade, test, documentation, and
  roadmap files changed;
- no `go.work`, `go.work.sum`, local module replacement, credential,
  generated binary, coverage output, or unrelated change was introduced;
- the built-in OpenRouter policy is exactly 16/1024;
- custom and endpoint-only backends remain unlimited by omission;
- explicit queue zero is distinguishable from omission;
- no capacity value enters execution targets, generated requests, stable JSON,
  prompt/profile YAML, or provider payloads;
- every engine owns distinct pools with no package-global mutable state;
- every initial and repair generation uses the active permit wrapper;
- capacity and waiter state is released on success, error, panic unwinding,
  and cancellation;
- concurrency tests use deterministic coordination rather than sleeps;
- current-state documentation describes implemented behavior rather than
  referring readers to the roadmaps; and
- the feature and implementation roadmaps contain no unresolved work marked
  complete.

### Completion Gate

The implementation is complete only when every target-end-state item in
`concurrency.md` is implemented, race-enabled tests demonstrate the configured
limits and cancellation safety, durable contracts no longer depend on roadmap
prose, and the OpenRouter compatibility change is clearly reported for the
next minor release.

## Open Questions

None. The feature roadmap and this plan fix the public representation,
registry defaults, admission bound, FIFO generation behavior, early-routing
refactor, cancellation races, error identities, engine and repair lifetimes,
test ownership, compatibility treatment, and non-goals required for
implementation.