eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document bounded output repair

Browse Source
This commit is contained in:
Eric Rakestraw
2026-08-25 09:58:10 +00:00
parent ae6f1a9865
commit 465dc1389d
7 changed files with 76 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
docs/internal/runner.md
View File

@@ -23,8 +23,10 @@ built-in backend and validated consumer additions, one engine-local run
admitter, and a model client wrapped by the same capacity manager. Validation
plans and provider-facing schema metadata come from the validator's preparation
interface.
An output repairer can be injected internally, but the ordinary runner
constructor does not enable one.
The root engine supplies one default output repairer through the explicit
runner constructor, using the same capacity-wrapped client as initial
generation. The no-repair runner constructor remains available for focused
internal callers and tests.
Each invocation carries its state in request, prepared-run, and result values.
The runner has no durable run or session store.
@@ -125,8 +127,8 @@ admitter is an internal unlimited fallback. After successful admission, `Run`
immediately defers the returned release function, performs the completion
phase, makes one initial generation call, builds the named output artifact,
and validates that artifact with the plan compiled during completion. Invalid
generated content remains a validation result; an inability to perform
validation is an operational error.
generated content remains a validation result; an inability to generate or
perform validation is an operational error.
Validation preparation and execution honor cancellation at every
Promptkit-controlled boundary and do not publish a partial plan or result.
@@ -142,17 +144,25 @@ serializing preparation or validation behind the active-generation limit.
The wrapped model client separately acquires a FIFO active permit only around
each actual generation call.
When an internal repairer is present, a JSON or JSON Schema content failure can
trigger bounded repair attempts. Repair receives the effective execution
target, explicit numeric-presence bits, credential, backend identity, session
ID, validation errors, prior output, and structured-output specification. One
request constructor supplies those common fields to initial and repair
generation while their rendered prompts remain intentionally distinct. The
default repairer uses the same wrapped client as initial generation, so each
repair reacquires the selected backend's active permit while remaining inside
its original admission lease. Repair never performs a second bounded
admission, and repaired outputs use the operation's existing validation plan.
This capability remains internal and is not a public option.
After a failed `basic`, JSON, or JSON Schema validation with a positive frozen
budget, the installed repairer can make a bounded corrective call. Each request
starts with a fresh copy of the complete original rendered messages, includes
only the latest nonempty candidate as an assistant message, and appends one
corrective user message. Empty candidates omit that assistant message. The
correction carries validation diagnostics as JSON data bounded to 64 KiB; the
full diagnostics remain in the validation result.
Repair receives the effective execution target, explicit numeric-presence bits,
credential, backend identity, session ID, and structured-output specification.
The same request constructor supplies those common fields to initial and repair
generation. The default repairer uses the same wrapped client as initial
generation, so each repair reacquires the selected backend's active permit
while remaining inside its original admission lease. Repair never performs a
second bounded admission, and repaired outputs use the operation's existing
validation plan. The runner stops at the first valid candidate, sums completed
generation usage, reports calls actually started, and returns the final failed
validation result on exhaustion. A repair generation failure follows the
ordinary generation-error category rather than becoming a validation error.
A successful result includes the output artifact and raw output, validation
state, effective session ID, prompt and rendered-prompt hashes, selected