Document structured capacity errors
This commit is contained in:
@@ -377,6 +377,11 @@ When a limited backend has admitted all active and waiting calls, handle
|
|||||||
```go
|
```go
|
||||||
result, err := engine.Run(ctx, request)
|
result, err := engine.Run(ctx, request)
|
||||||
if errors.Is(err, promptkit.ErrCapacityExceeded) {
|
if errors.Is(err, promptkit.ErrCapacityExceeded) {
|
||||||
|
var capacityErr *promptkit.CapacityError
|
||||||
|
if errors.As(err, &capacityErr) {
|
||||||
|
// Record capacityErr.BackendID using application-owned diagnostics.
|
||||||
|
}
|
||||||
|
|
||||||
// Apply application policy: shed work, report overload, or retry later.
|
// Apply application policy: shed work, report overload, or retry later.
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@@ -384,8 +389,9 @@ if errors.Is(err, promptkit.ErrCapacityExceeded) {
|
|||||||
A rejected call returns no partial result and does not invoke the model
|
A rejected call returns no partial result and does not invoke the model
|
||||||
client. Promptkit does not prescribe retries or map this error to an HTTP
|
client. Promptkit does not prescribe retries or map this error to an HTTP
|
||||||
status; those choices remain with the consuming application. The
|
status; those choices remain with the consuming application. The
|
||||||
[`Engine.Run` and error GoDoc](../../engine.go) owns exact error and
|
[`CapacityError` GoDoc](../../capacity_error.go) owns the exact typed-error
|
||||||
cancellation identities.
|
contract, while the [`Engine.Run` and error GoDoc](../../engine.go) owns broad
|
||||||
|
error and cancellation identities.
|
||||||
|
|
||||||
## Application Boundary
|
## Application Boundary
|
||||||
|
|
||||||
|
|||||||
@@ -38,10 +38,10 @@ output contract, but before schema loading, artifact loading, or rendering.
|
|||||||
rechecks credential availability, and then asks the manager to admit the
|
rechecks credential availability, and then asks the manager to admit the
|
||||||
frozen backend before generation.
|
frozen backend before generation.
|
||||||
|
|
||||||
Admission is immediate: a limited pool either reserves a slot or returns the
|
Admission is immediate: a limited pool either reserves a slot or returns only
|
||||||
internal `ErrCapacityExceeded` identity. The root facade maps that identity to
|
the internal `ErrCapacityExceeded` identity. The runner attaches the selected
|
||||||
the public error without treating it as an invalid request or generation
|
backend identity at its use-case boundary, and the root facade translates that
|
||||||
failure.
|
typed value without treating it as an invalid request or generation failure.
|
||||||
|
|
||||||
The total admitted bound is the active-generation limit plus its configured
|
The total admitted bound is the active-generation limit plus its configured
|
||||||
waiting capacity. The returned release function is idempotent. The runner
|
waiting capacity. The returned release function is idempotent. The runner
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ contributor workflow and validation.
|
|||||||
|
|
||||||
| Component | Implemented responsibility | References |
|
| Component | Implemented responsibility | References |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, public error mapping, and engine-local assembly. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) |
|
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, typed capacity errors, public error mapping, and engine-local assembly. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) |
|
||||||
| `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
|
| `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
|
||||||
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
|
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
|
||||||
| `internal/backend` | Constructs each engine's immutable registry from the built-in OpenRouter definition and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |
|
| `internal/backend` | Constructs each engine's immutable registry from the built-in OpenRouter definition and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |
|
||||||
|
|||||||
@@ -152,15 +152,17 @@ validation failures. Wrapping preserves the package identities mapped by the
|
|||||||
public facade and retains collaborator identities where they are part of the
|
public facade and retains collaborator identities where they are part of the
|
||||||
internal contract.
|
internal contract.
|
||||||
|
|
||||||
Admission capacity exhaustion retains the internal capacity identity and adds
|
Admission capacity exhaustion retains the internal capacity identity. At the
|
||||||
the selected backend ID as context. It is not recategorized as an invalid
|
use-case boundary, the runner attaches the selected backend ID in an internal
|
||||||
request or generation failure, and no partial result is returned. A context
|
typed error, and the root facade copies that value into the public
|
||||||
already done at admission retains its context identity directly. Cancellation
|
[`CapacityError`](../../capacity_error.go) without parsing diagnostic text. It
|
||||||
while waiting for an active generation permit prevents client invocation when
|
is not recategorized as an invalid request or generation failure, and no
|
||||||
it wins the grant race; the model-client boundary then preserves the context
|
partial result is returned. A context already done at admission retains its
|
||||||
error through the generation-failure category. Deferred release restores the
|
context identity directly. Cancellation while waiting for an active generation
|
||||||
admission lease on preparation, generation, validation, repair, and
|
permit prevents client invocation when it wins the grant race; the model-client
|
||||||
cancellation failures.
|
boundary then preserves the context error through the generation-failure
|
||||||
|
category. Deferred release restores the admission lease on preparation,
|
||||||
|
generation, validation, repair, and cancellation failures.
|
||||||
|
|
||||||
Other context cancellation propagates through the invoked collaborator and is
|
Other context cancellation propagates through the invoked collaborator and is
|
||||||
classified by the owning operation.
|
classified by the owning operation.
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Structured Capacity Errors
|
# Structured Capacity Errors
|
||||||
|
|
||||||
**Status:** Accepted.
|
**Status:** Complete.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
|
|||||||
@@ -33,9 +33,7 @@ consumers.
|
|||||||
|
|
||||||
## Ideas
|
## Ideas
|
||||||
|
|
||||||
Structured capacity errors have been selected for active planning in the
|
No ideas currently await selection.
|
||||||
[focused feature roadmap](capacity-errors.md). No other ideas currently await
|
|
||||||
selection.
|
|
||||||
|
|
||||||
## Entry Format
|
## Entry Format
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Structured Capacity Errors Implementation Plan
|
# Structured Capacity Errors Implementation Plan
|
||||||
|
|
||||||
**Status:** Ready for implementation.
|
**Status:** Complete.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
|
|||||||
@@ -205,8 +205,8 @@ resolution.
|
|||||||
|
|
||||||
## Priority 4: Structured Capacity Errors
|
## Priority 4: Structured Capacity Errors
|
||||||
|
|
||||||
**Disposition:** Accepted into the
|
**Disposition:** Implemented behavior. See the consumer guide's
|
||||||
[structured capacity errors](capacity-errors.md) feature roadmap.
|
[Handle Errors](../consumers/pkg-promptkit.md#handle-errors) section.
|
||||||
|
|
||||||
### Downstream need
|
### Downstream need
|
||||||
|
|
||||||
@@ -215,46 +215,16 @@ provider-neutral application error. When multiple backends are active,
|
|||||||
operators would benefit from knowing which backend rejected admission without
|
operators would benefit from knowing which backend rejected admission without
|
||||||
parsing an error string or exposing endpoint details.
|
parsing an error string or exposing endpoint details.
|
||||||
|
|
||||||
### Current integration
|
### Implemented behavior
|
||||||
|
|
||||||
PromptKit provides the useful `ErrCapacityExceeded` sentinel. Notarius can
|
PromptKit now retains the broad capacity classification while allowing
|
||||||
classify the failure reliably, but it retains only a sanitized diagnostic
|
Notarius to obtain the selected backend ID without parsing diagnostic text.
|
||||||
string as additional context.
|
The consumer guide owns the application workflow, including retry and backoff
|
||||||
|
policy.
|
||||||
### Requested capability
|
|
||||||
|
|
||||||
Add a typed error that continues to match `ErrCapacityExceeded`:
|
|
||||||
|
|
||||||
```go
|
|
||||||
type CapacityError struct {
|
|
||||||
BackendID string
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *CapacityError) Is(target error) bool {
|
|
||||||
return target == ErrCapacityExceeded
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The exact implementation may use `Unwrap` or another idiomatic mechanism. The
|
|
||||||
important properties are compatibility with `errors.Is` and discoverability
|
|
||||||
through `errors.As`.
|
|
||||||
|
|
||||||
### Design considerations
|
|
||||||
|
|
||||||
- Include the stable backend ID.
|
|
||||||
- Do not expose the backend endpoint, credential environment, credential
|
|
||||||
value, request content, or other sensitive configuration.
|
|
||||||
- Consider including the configured concurrency and queue limits if they are
|
|
||||||
useful and safe, but backend identity alone provides most of the downstream
|
|
||||||
value.
|
|
||||||
- Add a retry delay only if PromptKit can provide a meaningful value. A full
|
|
||||||
queue does not necessarily imply a reliable `Retry-After` duration.
|
|
||||||
- Keep retry and backoff policy with the consuming application. PromptKit
|
|
||||||
should classify the admission failure rather than silently retry it.
|
|
||||||
|
|
||||||
### Value to Notarius
|
### Value to Notarius
|
||||||
|
|
||||||
This would improve operational diagnostics and future metrics while preserving
|
This improves operational diagnostics and future metrics while preserving
|
||||||
the provider-neutral error boundary used by Notarius.
|
the provider-neutral error boundary used by Notarius.
|
||||||
|
|
||||||
## Capabilities PromptKit Already Provides Well
|
## Capabilities PromptKit Already Provides Well
|
||||||
@@ -311,7 +281,8 @@ Notarius would be:
|
|||||||
1. Add atomic execution that returns prepared details and the completed result.
|
1. Add atomic execution that returns prepared details and the completed result.
|
||||||
2. Add a semantic execution-target digest, preferably alongside profile
|
2. Add a semantic execution-target digest, preferably alongside profile
|
||||||
inspection.
|
inspection.
|
||||||
3. Add a typed capacity error carrying backend identity.
|
3. Use the implemented typed capacity error where backend admission diagnostics
|
||||||
|
are needed.
|
||||||
|
|
||||||
The first addresses a concrete execution workaround. The second would improve
|
The first addresses a concrete execution workaround. The second would improve
|
||||||
checkpoint correctness and reduce coupling. The third is operational polish.
|
checkpoint correctness and reduce coupling. The third is operational polish.
|
||||||
|
|||||||
@@ -282,18 +282,16 @@ the existing sentinel remain available.
|
|||||||
|
|
||||||
### Structured Capacity Errors
|
### Structured Capacity Errors
|
||||||
|
|
||||||
**Disposition:** Accepted into the
|
**Disposition:** Implemented behavior. See the consumer guide's
|
||||||
[structured capacity errors](capacity-errors.md) feature roadmap.
|
[Handle Errors](../consumers/pkg-promptkit.md#handle-errors) section.
|
||||||
|
|
||||||
The typed capacity error proposed by the
|
PromptKit now exposes the stable backend ID for capacity rejection without
|
||||||
[Notarius wishlist](notarius-promptkit-wishlist.md#priority-4-structured-capacity-errors)
|
requiring Weatherreporter to parse error text.
|
||||||
would improve Weatherreporter diagnostics by exposing the stable backend ID
|
|
||||||
without parsing error text.
|
|
||||||
|
|
||||||
Weatherreporter currently generates batch reports sequentially and constructs
|
Weatherreporter currently generates batch reports sequentially and constructs
|
||||||
one engine per invocation, so engine-local capacity exhaustion is unlikely in
|
one engine per invocation, so engine-local capacity exhaustion is unlikely in
|
||||||
the initial design. The feature would become more valuable if report
|
the initial design. The typed error becomes more valuable if report generation
|
||||||
generation later becomes concurrent or PromptKit engines become longer-lived.
|
later becomes concurrent or PromptKit engines become longer-lived.
|
||||||
It should not block adoption.
|
It should not block adoption.
|
||||||
|
|
||||||
### Semantic Execution-Target Fingerprints
|
### Semantic Execution-Target Fingerprints
|
||||||
@@ -364,8 +362,8 @@ Weatherreporter would be:
|
|||||||
2. Consider eager source validation after evaluating whether the two exact
|
2. Consider eager source validation after evaluating whether the two exact
|
||||||
inspection APIs are sufficient.
|
inspection APIs are sufficient.
|
||||||
3. Add structured generation errors.
|
3. Add structured generation errors.
|
||||||
4. Add structured capacity errors and semantic execution-target fingerprints
|
4. Use structured capacity errors and consider semantic execution-target
|
||||||
as lower-priority operational improvements.
|
fingerprints as lower-priority operational improvements.
|
||||||
|
|
||||||
The first item removes the only material integration workaround. Prompt and
|
The first item removes the only material integration workaround. Prompt and
|
||||||
profile inspection improve fail-fast validation. The remaining items improve
|
profile inspection improve fail-fast validation. The remaining items improve
|
||||||
|
|||||||
Reference in New Issue
Block a user