3.0 KiB
ADR-0014: Use feedback-aware validation retries
Status: Accepted Date: 2026-08-26
Context
Validation can identify a candidate defect after a producer has returned an otherwise well-formed result. Retrying without the validator's deterministic, bounded feedback wastes the useful diagnosis, while treating validator execution failures as defects would ask a producer to repair conditions it cannot control. The mechanism must preserve typed producer ownership, checkpoint safety, and the repository's sensitive-data boundaries.
Decision
The implementation will keep three independent budgets: the producer binding's
outer retries budget, PromptKit's structured-output repair budget, and each
validator's execution-retry budget. Validators will run sequentially in their
configured order and aggregate both rejections and execution failures before a
candidate disposition is selected.
A correction-capable producer will provide the exact single LLM response that
controlled its candidate using the single_response_v1 protocol. A correction
attempt will reconstruct the ordinary request and append exactly two fresh
messages: that latest response as assistant, followed by one deterministic
aggregate correction request as user. Earlier turns will not accumulate.
Validator failures will not recurse into correction. Pipeline policy owns
terminal disposition, with field-by-field producer overrides over pipeline
defaults: structural failure and semantic rejection default to fail_run, and
validator execution failure defaults to warn_continue. Validators can report
facts and bounded corrective guidance, but never decide disposition.
Rejected and structurally invalid candidates will not advance. A candidate allowed through after a validator execution failure will retain explicit incomplete-validation provenance and will not be checkpointed. Exact response and correction text remain attempt-local: they are excluded from ordinary errors, warnings, manifests, receipts, caches, checkpoints, and default debug summaries.
Alternatives considered
- Retry every producer after any validation outcome. This conflates producer defects with validator operational failures and wastes retry budget.
- Let validators decide whether to continue. This would distribute pipeline disposition policy across validators and undermine consistent defaults.
- Reuse the full prior conversation. Accumulated turns introduce unbounded prompt growth and make correction behavior depend on incidental history.
- Persist raw responses to simplify diagnosis. Raw model output and correction guidance may be sensitive and do not belong in durable pipeline records.
Consequences
The framework gains transport-neutral correction and candidate contracts, producer capability checks, policy resolution, aggregated validation outcomes, and conservative checkpoint handling. Prompt construction remains inside the LLM adapter, while modules remain responsible for accurately exposing the single response that directly controlled a candidate.