eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Preserve repair settings and cumulative usage

Browse Source
This commit is contained in:
Eric Rakestraw
2026-08-11 23:21:34 +00:00
parent e43350fd0d
commit 350b0e76d9
7 changed files with 364 additions and 161 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
docs/internal/capacity.md
View File

@@ -56,7 +56,10 @@ reserve another bounded slot.
`NewClient` wraps the engine's selected internal model client after public
client adaptation or built-in client construction. Initial generation and the
default repairer receive the same wrapper.
default repairer receive the same wrapper. Their requests retain the same
effective backend, credential, numeric-presence metadata, and structured-output
settings, so scheduling does not change provider omission semantics between
calls.
For each `Generate` call, the wrapper selects a pool from the request's
effective backend ID. An unlimited call passes directly to the next client. A
@@ -71,7 +74,9 @@ other backend IDs.
The wrapper passes generation requests, responses, and collaborator errors
through unchanged. It owns scheduling only; the concrete model client remains
responsible for provider transport behavior.
responsible for provider transport behavior. The runner, rather than the
capacity layer, sums all five usage fields from the initial response and every
completed repair response into the successful run result.
## Cancellation And Release