60 lines
3.0 KiB
Markdown
60 lines
3.0 KiB
Markdown
# ADR-0014: Use feedback-aware validation retries
|
|
|
|
**Status:** Accepted
|
|
**Date:** 2026-08-26
|
|
|
|
## Context
|
|
|
|
Validation can identify a candidate defect after a producer has returned an
|
|
otherwise well-formed result. Retrying without the validator's deterministic,
|
|
bounded feedback wastes the useful diagnosis, while treating validator
|
|
execution failures as defects would ask a producer to repair conditions it
|
|
cannot control. The mechanism must preserve typed producer ownership,
|
|
checkpoint safety, and the repository's sensitive-data boundaries.
|
|
|
|
## Decision
|
|
|
|
The implementation will keep three independent budgets: the producer binding's
|
|
outer `retries` budget, PromptKit's structured-output repair budget, and each
|
|
validator's execution-retry budget. Validators will run sequentially in their
|
|
configured order and aggregate both rejections and execution failures before a
|
|
candidate disposition is selected.
|
|
|
|
A correction-capable producer will provide the exact single LLM response that
|
|
controlled its candidate using the `single_response_v1` protocol. A correction
|
|
attempt will reconstruct the ordinary request and append exactly two fresh
|
|
messages: that latest response as `assistant`, followed by one deterministic
|
|
aggregate correction request as `user`. Earlier turns will not accumulate.
|
|
|
|
Validator failures will not recurse into correction. Pipeline policy owns
|
|
terminal disposition, with field-by-field producer overrides over pipeline
|
|
defaults: structural failure and semantic rejection default to `fail_run`, and
|
|
validator execution failure defaults to `warn_continue`. Validators can report
|
|
facts and bounded corrective guidance, but never decide disposition.
|
|
|
|
Rejected and structurally invalid candidates will not advance. A candidate
|
|
allowed through after a validator execution failure will retain explicit
|
|
incomplete-validation provenance and will not be checkpointed. Exact response
|
|
and correction text remain attempt-local: they are excluded from ordinary
|
|
errors, warnings, manifests, receipts, caches, checkpoints, and default debug
|
|
summaries.
|
|
|
|
## Alternatives considered
|
|
|
|
- Retry every producer after any validation outcome. This conflates producer
|
|
defects with validator operational failures and wastes retry budget.
|
|
- Let validators decide whether to continue. This would distribute pipeline
|
|
disposition policy across validators and undermine consistent defaults.
|
|
- Reuse the full prior conversation. Accumulated turns introduce unbounded
|
|
prompt growth and make correction behavior depend on incidental history.
|
|
- Persist raw responses to simplify diagnosis. Raw model output and correction
|
|
guidance may be sensitive and do not belong in durable pipeline records.
|
|
|
|
## Consequences
|
|
|
|
The framework gains transport-neutral correction and candidate contracts,
|
|
producer capability checks, policy resolution, aggregated validation outcomes,
|
|
and conservative checkpoint handling. Prompt construction remains inside the
|
|
LLM adapter, while modules remain responsible for accurately exposing the
|
|
single response that directly controlled a candidate.
|