Document structured capacity errors

This commit is contained in:
2026-07-30 23:26:44 +00:00
parent 7428e50c2c
commit e40c4f182b
9 changed files with 45 additions and 70 deletions

View File

@@ -377,6 +377,11 @@ When a limited backend has admitted all active and waiting calls, handle
```go ```go
result, err := engine.Run(ctx, request) result, err := engine.Run(ctx, request)
if errors.Is(err, promptkit.ErrCapacityExceeded) { if errors.Is(err, promptkit.ErrCapacityExceeded) {
var capacityErr *promptkit.CapacityError
if errors.As(err, &capacityErr) {
// Record capacityErr.BackendID using application-owned diagnostics.
}
// Apply application policy: shed work, report overload, or retry later. // Apply application policy: shed work, report overload, or retry later.
} }
``` ```
@@ -384,8 +389,9 @@ if errors.Is(err, promptkit.ErrCapacityExceeded) {
A rejected call returns no partial result and does not invoke the model A rejected call returns no partial result and does not invoke the model
client. Promptkit does not prescribe retries or map this error to an HTTP client. Promptkit does not prescribe retries or map this error to an HTTP
status; those choices remain with the consuming application. The status; those choices remain with the consuming application. The
[`Engine.Run` and error GoDoc](../../engine.go) owns exact error and [`CapacityError` GoDoc](../../capacity_error.go) owns the exact typed-error
cancellation identities. contract, while the [`Engine.Run` and error GoDoc](../../engine.go) owns broad
error and cancellation identities.
## Application Boundary ## Application Boundary

View File

@@ -38,10 +38,10 @@ output contract, but before schema loading, artifact loading, or rendering.
rechecks credential availability, and then asks the manager to admit the rechecks credential availability, and then asks the manager to admit the
frozen backend before generation. frozen backend before generation.
Admission is immediate: a limited pool either reserves a slot or returns the Admission is immediate: a limited pool either reserves a slot or returns only
internal `ErrCapacityExceeded` identity. The root facade maps that identity to the internal `ErrCapacityExceeded` identity. The runner attaches the selected
the public error without treating it as an invalid request or generation backend identity at its use-case boundary, and the root facade translates that
failure. typed value without treating it as an invalid request or generation failure.
The total admitted bound is the active-generation limit plus its configured The total admitted bound is the active-generation limit plus its configured
waiting capacity. The returned release function is idempotent. The runner waiting capacity. The returned release function is idempotent. The runner

View File

@@ -11,7 +11,7 @@ contributor workflow and validation.
| Component | Implemented responsibility | References | | Component | Implemented responsibility | References |
| --- | --- | --- | | --- | --- | --- |
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, public error mapping, and engine-local assembly. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) | | Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, typed capacity errors, public error mapping, and engine-local assembly. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) |
| `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) | | `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) | | `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
| `internal/backend` | Constructs each engine's immutable registry from the built-in OpenRouter definition and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) | | `internal/backend` | Constructs each engine's immutable registry from the built-in OpenRouter definition and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |

View File

@@ -152,15 +152,17 @@ validation failures. Wrapping preserves the package identities mapped by the
public facade and retains collaborator identities where they are part of the public facade and retains collaborator identities where they are part of the
internal contract. internal contract.
Admission capacity exhaustion retains the internal capacity identity and adds Admission capacity exhaustion retains the internal capacity identity. At the
the selected backend ID as context. It is not recategorized as an invalid use-case boundary, the runner attaches the selected backend ID in an internal
request or generation failure, and no partial result is returned. A context typed error, and the root facade copies that value into the public
already done at admission retains its context identity directly. Cancellation [`CapacityError`](../../capacity_error.go) without parsing diagnostic text. It
while waiting for an active generation permit prevents client invocation when is not recategorized as an invalid request or generation failure, and no
it wins the grant race; the model-client boundary then preserves the context partial result is returned. A context already done at admission retains its
error through the generation-failure category. Deferred release restores the context identity directly. Cancellation while waiting for an active generation
admission lease on preparation, generation, validation, repair, and permit prevents client invocation when it wins the grant race; the model-client
cancellation failures. boundary then preserves the context error through the generation-failure
category. Deferred release restores the admission lease on preparation,
generation, validation, repair, and cancellation failures.
Other context cancellation propagates through the invoked collaborator and is Other context cancellation propagates through the invoked collaborator and is
classified by the owning operation. classified by the owning operation.

View File

@@ -1,6 +1,6 @@
# Structured Capacity Errors # Structured Capacity Errors
**Status:** Accepted. **Status:** Complete.
## Purpose ## Purpose

View File

@@ -33,9 +33,7 @@ consumers.
## Ideas ## Ideas
Structured capacity errors have been selected for active planning in the No ideas currently await selection.
[focused feature roadmap](capacity-errors.md). No other ideas currently await
selection.
## Entry Format ## Entry Format

View File

@@ -1,6 +1,6 @@
# Structured Capacity Errors Implementation Plan # Structured Capacity Errors Implementation Plan
**Status:** Ready for implementation. **Status:** Complete.
## Purpose ## Purpose

View File

@@ -205,8 +205,8 @@ resolution.
## Priority 4: Structured Capacity Errors ## Priority 4: Structured Capacity Errors
**Disposition:** Accepted into the **Disposition:** Implemented behavior. See the consumer guide's
[structured capacity errors](capacity-errors.md) feature roadmap. [Handle Errors](../consumers/pkg-promptkit.md#handle-errors) section.
### Downstream need ### Downstream need
@@ -215,46 +215,16 @@ provider-neutral application error. When multiple backends are active,
operators would benefit from knowing which backend rejected admission without operators would benefit from knowing which backend rejected admission without
parsing an error string or exposing endpoint details. parsing an error string or exposing endpoint details.
### Current integration ### Implemented behavior
PromptKit provides the useful `ErrCapacityExceeded` sentinel. Notarius can PromptKit now retains the broad capacity classification while allowing
classify the failure reliably, but it retains only a sanitized diagnostic Notarius to obtain the selected backend ID without parsing diagnostic text.
string as additional context. The consumer guide owns the application workflow, including retry and backoff
policy.
### Requested capability
Add a typed error that continues to match `ErrCapacityExceeded`:
```go
type CapacityError struct {
BackendID string
}
func (e *CapacityError) Is(target error) bool {
return target == ErrCapacityExceeded
}
```
The exact implementation may use `Unwrap` or another idiomatic mechanism. The
important properties are compatibility with `errors.Is` and discoverability
through `errors.As`.
### Design considerations
- Include the stable backend ID.
- Do not expose the backend endpoint, credential environment, credential
value, request content, or other sensitive configuration.
- Consider including the configured concurrency and queue limits if they are
useful and safe, but backend identity alone provides most of the downstream
value.
- Add a retry delay only if PromptKit can provide a meaningful value. A full
queue does not necessarily imply a reliable `Retry-After` duration.
- Keep retry and backoff policy with the consuming application. PromptKit
should classify the admission failure rather than silently retry it.
### Value to Notarius ### Value to Notarius
This would improve operational diagnostics and future metrics while preserving This improves operational diagnostics and future metrics while preserving
the provider-neutral error boundary used by Notarius. the provider-neutral error boundary used by Notarius.
## Capabilities PromptKit Already Provides Well ## Capabilities PromptKit Already Provides Well
@@ -311,7 +281,8 @@ Notarius would be:
1. Add atomic execution that returns prepared details and the completed result. 1. Add atomic execution that returns prepared details and the completed result.
2. Add a semantic execution-target digest, preferably alongside profile 2. Add a semantic execution-target digest, preferably alongside profile
inspection. inspection.
3. Add a typed capacity error carrying backend identity. 3. Use the implemented typed capacity error where backend admission diagnostics
are needed.
The first addresses a concrete execution workaround. The second would improve The first addresses a concrete execution workaround. The second would improve
checkpoint correctness and reduce coupling. The third is operational polish. checkpoint correctness and reduce coupling. The third is operational polish.

View File

@@ -282,18 +282,16 @@ the existing sentinel remain available.
### Structured Capacity Errors ### Structured Capacity Errors
**Disposition:** Accepted into the **Disposition:** Implemented behavior. See the consumer guide's
[structured capacity errors](capacity-errors.md) feature roadmap. [Handle Errors](../consumers/pkg-promptkit.md#handle-errors) section.
The typed capacity error proposed by the PromptKit now exposes the stable backend ID for capacity rejection without
[Notarius wishlist](notarius-promptkit-wishlist.md#priority-4-structured-capacity-errors) requiring Weatherreporter to parse error text.
would improve Weatherreporter diagnostics by exposing the stable backend ID
without parsing error text.
Weatherreporter currently generates batch reports sequentially and constructs Weatherreporter currently generates batch reports sequentially and constructs
one engine per invocation, so engine-local capacity exhaustion is unlikely in one engine per invocation, so engine-local capacity exhaustion is unlikely in
the initial design. The feature would become more valuable if report the initial design. The typed error becomes more valuable if report generation
generation later becomes concurrent or PromptKit engines become longer-lived. later becomes concurrent or PromptKit engines become longer-lived.
It should not block adoption. It should not block adoption.
### Semantic Execution-Target Fingerprints ### Semantic Execution-Target Fingerprints
@@ -364,8 +362,8 @@ Weatherreporter would be:
2. Consider eager source validation after evaluating whether the two exact 2. Consider eager source validation after evaluating whether the two exact
inspection APIs are sufficient. inspection APIs are sufficient.
3. Add structured generation errors. 3. Add structured generation errors.
4. Add structured capacity errors and semantic execution-target fingerprints 4. Use structured capacity errors and consider semantic execution-target
as lower-priority operational improvements. fingerprints as lower-priority operational improvements.
The first item removes the only material integration workaround. Prompt and The first item removes the only material integration workaround. Prompt and
profile inspection improve fail-fast validation. The remaining items improve profile inspection improve fail-fast validation. The remaining items improve