eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Verify partial validator decisions through CLI workflows

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-25 11:34:26 +00:00
parent 22f0932389
commit b8089f4a50
6 changed files with 361 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
docs/roadmap/future.md
View File

@@ -9,22 +9,20 @@ intent; inclusion does not commit us to implementing every capability.
## Domain-Aware Response Recovery
First implement [partial success for validator decision sets](decision-set-reconciliation.md),
preserving usable decisions without another model call. Reconsider guided recovery
after that simpler change, based on the failures that remain. General per-entry
The [partial-success decision reconciliation](decision-set-reconciliation.md)
preserves usable decisions without another model call. Guided recovery remains
deferred pending evidence from failures that remain. General per-entry
salvage from structurally invalid responses is also a separate design question;
the accepted reconciliation feature retains whole-response structural validation.
Add bounded feedback and regeneration for model responses that satisfy the
structural schema but violate Audita's domain response contract, such as missing,
duplicate, or unexpected validator decision indices. The goal is to recover more
usable work before excluding affected candidates, reducing the impact of an
isolated malformed decision set.
Consider bounded feedback and regeneration for model responses that satisfy the
structural schema but leave requested decisions unavailable after reconciliation.
The goal would be to recover more usable work before excluding affected
candidates, when doing so materially improves outcomes.
This capability is desired but explicitly deferred from the current resilience
work. That work retains existing PromptKit structural repair and safe filtering;
unresolved domain-invalid decision sets produce errors and exclude their affected
candidates while other candidates can succeed.
This capability is explicitly deferred. Current behavior retains PromptKit
structural repair and per-candidate reconciliation; unavailable requested
decisions produce errors while other candidates can succeed.
Before implementation, define the Audita/PromptKit ownership boundary, which
checks are eligible for feedback, how feedback identifies defects without exposing