74 lines
4.1 KiB
Markdown
74 lines
4.1 KiB
Markdown
# Deferred Feature Work
|
|
|
|
**Last updated:** 2026-09-24
|
|
|
|
**Status:** Intended future work; not implemented or scheduled. These entries
|
|
preserve scope deferred from the [resilience roadmap](resilience.md). They are
|
|
not prerequisites for completing that feature or staged implementation plans.
|
|
|
|
## Domain-Aware Response Recovery
|
|
|
|
Add bounded feedback and regeneration for model responses that satisfy the
|
|
structural schema but violate Audita's domain response contract, such as missing,
|
|
duplicate, or unexpected validator decision indices. The goal is to recover more
|
|
usable work before excluding affected candidates, reducing the impact of an
|
|
isolated malformed decision set.
|
|
|
|
This capability is desired but explicitly deferred from the current resilience
|
|
work. That work retains existing PromptKit structural repair and safe filtering;
|
|
unresolved domain-invalid decision sets produce errors and exclude their affected
|
|
candidates while other candidates can succeed.
|
|
|
|
Before implementation, define the Audita/PromptKit ownership boundary, which
|
|
checks are eligible for feedback, how feedback identifies defects without exposing
|
|
secrets, and how regeneration fits bounded budgets and operation deadlines. Reuse
|
|
shared library capabilities where appropriate; do not introduce an uncoordinated
|
|
outer retry loop that multiplies existing retry/repair budgets.
|
|
|
|
Ordinary editorial rejection is a valid decision, not a reason to regenerate until
|
|
an edit is approved. Recovery must preserve required validation, candidate identity,
|
|
partial-success safety, and truthful error/recovery reporting. Validate with
|
|
controlled responses that distinguish recoverable protocol defects from valid
|
|
negative decisions. Exact APIs, budgets, and sequencing remain undecided.
|
|
|
|
## Independent Whole-Pipeline Traversal by Chunk
|
|
|
|
Explore allowing each fixed chunk to advance through its ordered stages without
|
|
waiting for other chunks to finish the current stage. This could reduce waiting
|
|
behind a slow chunk and improve utilization while preserving stage order within
|
|
each chunk and deterministic final recombination.
|
|
|
|
The resilience feature deliberately retains stage barriers: finish a stage across
|
|
chunks before advancing to the next stage. Fixed boundaries alone do not authorize
|
|
independent traversal.
|
|
|
|
A future design must establish that stages do not depend on other chunks' evolving
|
|
state, preserve stable identities and partial-success rules, and continue to share
|
|
PromptKit's per-backend capacity across all roles and stages. Reporting must remain
|
|
deterministic despite different completion orders, and fatal cancellation must
|
|
still stop and join active work. Assess whether the scheduling change materially
|
|
improves real workloads before introducing additional orchestration machinery.
|
|
No new scheduler, global limiter, or execution mode is selected by this entry.
|
|
|
|
## Empty Replacement Text and Segment Removal
|
|
|
|
Address a frequent proposal failure reported in production: the LLM supplies an
|
|
empty string as `corrected_text`, possibly because it intends to remove the
|
|
entire segment. That interpretation is a hypothesis, not a reliable signal of
|
|
intent. Improve both prompting and handling so this case has a clear, consistent
|
|
outcome rather than repeatedly producing unusable proposals.
|
|
|
|
Review representative examples and distinguish deletion of a text span from
|
|
removal of an entire segment. Define whether either operation is supported, how
|
|
it is represented, and what validation is required. Align proposal prompts,
|
|
response contracts, validators, application behavior, and error/recovery reporting
|
|
with the selected semantics. If an operation is unsupported, prompts should give
|
|
clear guidance on what the model should return instead.
|
|
|
|
Do not infer authorization to delete a segment from an empty replacement alone.
|
|
Any future support for segment removal must address meaning preservation and the
|
|
effect on segment identity, timestamps, fixed chunks, later stages, and output.
|
|
Coordinate with domain-aware response recovery where feedback could resolve an
|
|
invalid proposal. Exact deletion semantics and recovery behavior remain undecided;
|
|
this entry does not relax current validation or expand the resilience workstream.
|