Document feedback-aware validation retries

This commit is contained in:
2026-08-27 01:35:23 +00:00
parent 7f01c3e79e
commit b2e83bd6e7
12 changed files with 229 additions and 38 deletions

View File

@@ -109,6 +109,33 @@ On success, the command reports the output bundle path. A warning-bearing run
still succeeds and reports its warning count on standard error. Errors and
their exit classes are defined in the [CLI reference](cli.md#output-streams-and-exit-statuses).
## Validation Retries And Terminal Outcomes
Each producer binding has one outer **retries** budget. It covers complete
producer attempts for operational failures, invalid structured output,
normalizer fallback retry, and semantic correction. It is independent from
PromptKit's structural-repair calls inside one completion and from an
LLM-backed validator's own retry budget. A semantic correction rebuilds the
ordinary producer request and supplies only the latest rejected model response
plus aggregated validator guidance; it is not a conversation replay.
After the applicable budgets are exhausted, the resolved
[`validation_policy`](config.md#pipelines) determines the result. Structural
failure and semantic rejection normally fail the run; an explicit
`reject_output` records a rejection and allows unrelated work to finish. A
validator execution failure normally uses `warn_continue`, which keeps an
otherwise accepted result in the current run with `incomplete` validation
provenance. It emits one bounded warning for every validator whose execution
budget was exhausted. A corrected result that later passes validation does not
retain abandoned-attempt warnings.
Treat a successful process exit as a completed run, not as proof that every
candidate was fully validated. Inspect the receipt's `validation_status`,
`validation_summaries`, rejection count, and warning count when an orchestrator
requires complete validation. The durable fields and their meanings are owned
by the [run-result receipt](integrations/run-result.md) and
[published JSON output contract](integrations/json-output.md).
## Output Bundles
Each successful run receives a generated safe run identifier and writes beneath:
@@ -158,7 +185,9 @@ The configured cache mode controls one invocation:
A reused plan is still materialized and validated against the current source.
If a prior plan no longer gives acceptable results, use a refresh run rather
than editing cache files. Deleting a plan is recoverable but can repeat costly
chunking work.
chunking work. A plan accepted only under incomplete validation is not
published, and a rejected cache hit falls through to ordinary generation rather
than becoming a correction candidate.
## Checkpoint Recording, Resume, And Recompute
@@ -183,6 +212,12 @@ Reasoning-effort inheritance, replacement, and explicit clearing are distinct
runtime identities, so checkpoints created under one state are not reused by
either of the others.
Only accepted, completely validated chunk, extract, merge, and normalize
results are checkpointed for reuse. Rejected, structurally invalid, and
validation-incomplete producer results remain non-reusable, even when a
`warn_continue` result advanced during its original run. A resumed invocation
therefore reruns that producer rather than treating degraded state as accepted.
Checkpoint state is confined below an identity-specific path:
~~~
@@ -237,13 +272,16 @@ Only a [debug-enabled run](cli.md#run) creates a bundle:
~~~
The summary contains redacted invocation and resolution information plus run,
warning, checkpoint, chunk-plan, and terminal reporting artifacts. The trace
contains allowlisted application diagnostic records and can include source or
derived application data. Neither surface is a cache input. Do not treat a
debug bundle as safe to share merely because its configuration summary is
redacted. Invocation metadata omits reasoning effort when it is inherited,
records the replacement value when one is supplied, and records an empty value
when inherited reasoning was explicitly cleared.
warning, checkpoint, chunk-plan, and terminal reporting artifacts. Attempt
terminal records contain bounded attempt kinds, validator outcomes, policy,
decision, PromptKit repair count, and usage; they do not contain assistant
responses or complete correction messages. The trace contains allowlisted
application diagnostic records and can include source, model, and correction
content. Neither surface is a cache input. Do not treat a debug bundle as safe
to share merely because its configuration summary is redacted. Invocation
metadata omits reasoning effort when it is inherited, records the replacement
value when one is supplied, and records an empty value when inherited reasoning
was explicitly cleared.
Notarius never creates debug state without an explicit request and never
automatically deletes a requested bundle. If allocation succeeds, the command