Finalize resilience decisions and record deferred work

This commit is contained in:
2026-09-24 16:34:35 +00:00
parent aa2f486845
commit 053b714ce6
2 changed files with 267 additions and 233 deletions

51
docs/roadmap/future.md Normal file
View File

@@ -0,0 +1,51 @@
# Deferred Feature Work
**Last updated:** 2026-09-24
**Status:** Intended future work; not implemented or scheduled. These entries
preserve scope deferred from the [resilience roadmap](resilience.md). They are
not prerequisites for completing that feature or staged implementation plans.
## Domain-Aware Response Recovery
Add bounded feedback and regeneration for model responses that satisfy the
structural schema but violate Audita's domain response contract, such as missing,
duplicate, or unexpected validator decision indices. The goal is to recover more
usable work before excluding affected candidates, reducing the impact of an
isolated malformed decision set.
This capability is desired but explicitly deferred from the current resilience
work. That work retains existing PromptKit structural repair and safe filtering;
unresolved domain-invalid decision sets produce errors and exclude their affected
candidates while other candidates can succeed.
Before implementation, define the Audita/PromptKit ownership boundary, which
checks are eligible for feedback, how feedback identifies defects without exposing
secrets, and how regeneration fits bounded budgets and operation deadlines. Reuse
shared library capabilities where appropriate; do not introduce an uncoordinated
outer retry loop that multiplies existing retry/repair budgets.
Ordinary editorial rejection is a valid decision, not a reason to regenerate until
an edit is approved. Recovery must preserve required validation, candidate identity,
partial-success safety, and truthful error/recovery reporting. Validate with
controlled responses that distinguish recoverable protocol defects from valid
negative decisions. Exact APIs, budgets, and sequencing remain undecided.
## Independent Whole-Pipeline Traversal by Chunk
Explore allowing each fixed chunk to advance through its ordered stages without
waiting for other chunks to finish the current stage. This could reduce waiting
behind a slow chunk and improve utilization while preserving stage order within
each chunk and deterministic final recombination.
The resilience feature deliberately retains stage barriers: finish a stage across
chunks before advancing to the next stage. Fixed boundaries alone do not authorize
independent traversal.
A future design must establish that stages do not depend on other chunks' evolving
state, preserve stable identities and partial-success rules, and continue to share
PromptKit's per-backend capacity across all roles and stages. Reporting must remain
deterministic despite different completion orders, and fatal cancellation must
still stop and join active work. Assess whether the scheduling change materially
improves real workloads before introducing additional orchestration machinery.
No new scheduler, global limiter, or execution mode is selected by this entry.

View File

@@ -2,9 +2,9 @@
**Last updated:** 2026-09-24 **Last updated:** 2026-09-24
**Status:** Feature direction agreed; not implemented. This document records the **Status:** Feature direction agreed; not implemented. This document defines the
intended end state and remaining product decisions. It is not a staged target behavior and user intent, not a staged implementation plan. Remaining
implementation plan. Recommendations under Open Questions are not adopted policy. interface details are listed under Open Questions.
## Purpose and User Intent ## Purpose and User Intent
@@ -12,23 +12,26 @@ Audita should reliably produce a useful transcript even when its LLM services
cannot improve it. Exhausted retries or unusable model responses must not turn cannot improve it. Exhausted retries or unusable model responses must not turn
an otherwise processable transcript into a failed production run. an otherwise processable transcript into a failed production run.
The production guarantee is:
> Failure of every LLM operation across every stage and chunk still produces a > Failure of every LLM operation across every stage and chunk still produces a
> successful output containing the deterministically normalized transcript, with > successful output containing the deterministically normalized transcript, with
> no LLM corrections applied. Partial LLM success preserves corrections that > no LLM corrections applied. Partial success preserves as many fully validated,
> completed successfully, subject to the selected work-unit commit boundary. > applicable corrections as possible.
This does not promise the original input byte-for-byte. Deterministic Deterministic normalization remains intentional and may change segment layout,
normalization remains intentional and may change segment layout, identifiers, identifiers, and text before LLM work begins. Output uses the selected transcript
and text before LLM work begins. Output still uses the selected transcript encoder; byte-for-byte preservation of the original input and mapping corrections
encoder. There is no requirement to map corrections back to the original, back to pre-normalization segmentation are not requirements.
pre-normalization segmentation.
Existing reports contain useful information. Preserve meaningful identities, The overriding goal is to minimize how much useful processing is lost because of
reason codes, correction/decision evidence, and execution measurements where one problematic transcript segment. A stage/chunk work unit is an attribution
appropriate; reorganize their classification and audience rather than replace boundary, not an all-or-nothing transaction.
everything with an unrelated reporting system.
Existing reports contain useful information, but there are no compatibility
obligations for reporting or debug output. Reuse existing machinery only when it
makes the implementation simpler and clearer; preservation is not itself a goal.
Internal structures and measurements must serve Audita's processing workflow.
The reporting package adapts those outcomes to structured and human-facing
presentations, rather than reshaping processing around a reporting convenience.
## Current Baseline and Intended Change ## Current Baseline and Intended Change
@@ -44,268 +47,248 @@ Detailed artifacts are written during ordinary runs and may subsequently be
removed by retention. The working transcript is re-chunked between modules. removed by retention. The working transcript is re-chunked between modules.
This feature deliberately changes those behaviors. It builds on the completed This feature deliberately changes those behaviors. It builds on the completed
[library adoption](library-adoption.md) and [review remediation](remediation.md); [library adoption](library-adoption.md) and [review remediation](remediation.md).
it does not undo their separation of library and application responsibilities. Fatal cancellation remains necessary; unsuccessful model work must be separated
The existing fatal-cancellation mechanism remains useful, but model work failures from failures that actually require cancellation of the run.
must be separated from failures that actually require cancellation of the run.
## Fixed Chunks and Ordered Stages ## Fixed Chunks and Ordered Stages
Normalize first, then partition the transcript once. Maintain the same chunk Normalize first, then partition the transcript once. Preserve chunk boundaries,
boundaries, segment membership, segment identity, and ordering throughout LLM segment membership, segment identity, and ordering throughout LLM processing.
processing. Text inside a chunk evolves as accepted corrections are applied. Text evolves as accepted corrections are applied. Recombine chunks in original
Recombine the chunks in their original order after the LLM stages finish. order after the LLM stages finish.
For this feature, a stage is a configured module instance. Repeated instances of A stage is a configured module instance. Repeated instances of the same module
the same module have distinct identities. Proposal generation and each validator have distinct identities. Proposal generation and each validator are operations
are operations within a stage; validation may use multiple batches within a chunk. within a stage; validation may use multiple batches within a chunk. A work unit
A work unit is one stage instance applied to one chunk. is one stage instance applied to one chunk.
Preserve the overall stage sequence: finish a stage across its chunks before Preserve the overall stage sequence: finish a stage across its chunks before
advancing to the next stage. Concurrent chunk work within a stage continues to advancing to the next stage. Concurrent chunk work shares PromptKit's backend
share PromptKit's backend capacity. A skipped work unit does not prevent other capacity. Failure in one unit does not prevent other chunks or later stages
chunks or later stages from executing. from executing.
Fixed chunking makes failure isolation and context stable, as well as making Retain operator control over chunk size and the current defaults. Production
cross-stage reporting meaningful. It trades away adaptive repartitioning after experience indicates that chunk size affects model latency nonlinearly and that
edits and task-specific chunk sizes. Initial sizing should allow reasonable room useful settings are well below backend context limits. Tuning is an empirical
for prompt overhead and text growth, without claiming that every later request operator decision. Do not add automatic headroom calculations, cross-profile
will fit. An oversized later model request is an explicit skipped-work outcome; sizing, adaptive repartitioning, or automatic tuning in this feature. A later
do not silently change chunk boundaries. Exact sizing policy remains open. operation that cannot process its input follows the same safe error handling;
it does not silently alter chunk boundaries. Validation batching within a chunk
remains supported.
Whole-pipeline traversal by independently advancing chunks is explicitly deferred Independent whole-pipeline traversal by chunks and domain-aware response repair
to a future roadmap. Fixed chunking does not authorize changing the stage barrier. are explicitly deferred in [future work](future.md). Read-only neighboring
Read-only neighboring context, cross-chunk edits, and transcript-wide restructuring context, cross-chunk edits, and transcript-wide restructuring are outside this
are also outside this unit; fixed chunks are not a permanent product prohibition unit, without becoming permanent prohibitions on future capabilities.
on such future capabilities.
## Resilient Execution Contract ## Partial Success and Failure Isolation
- Keep bounded transport retries, structural repairs, operation deadlines, and - Keep bounded transport retries, structural repairs, operation deadlines, and
shared backend capacity. Completion comes from safe fallback, not unlimited shared backend capacity. Completion comes from safe fallback, not unlimited
attempts or an Audita retry loop around PromptKit's existing recovery. attempts or an outer retry loop around PromptKit.
- Generate proposals, enforce deterministic/domain checks, and complete required - Apply a correction only after it passes every required validator and the
LLM validation before applying a correction. Failure must never bypass validation application checks. Failed validation must never be treated as approval.
or cause uncertain corrections to be accepted. - If a validation batch fails, exclude its affected candidates while allowing
- Unsuccessful model work preserves the affected work unit's incoming text and other candidates to continue through the remaining validators. Preserve fully
records an error. Preserve corrections committed by earlier stages and other validated, applicable corrections from successful batches in the same unit.
successful work units. The handling of partially successful batches within a - Track candidates by stable proposal identity. Batch membership can change
failing work unit is an explicit open question below. between validators; equal batch numbers do not establish candidate identity.
- A failed proposal-generation operation leaves its chunk unchanged for that
stage because there are no reliable proposals to salvage. Failures later in
validation need not discard unrelated successful candidates.
- Preserve earlier stages' corrections and other chunks' successful work.
Record partial success explicitly; a unit can apply corrections and have errors.
- Continue despite model-operation failures, including exhausted repairs/retries, - Continue despite model-operation failures, including exhausted repairs/retries,
unusable model decisions, provider/transport failures, deadlines, and admission unusable model decisions, provider/transport failures, local operation deadlines,
failures. An operation can fail without consuming its entire attempt budget; and admission failures. An operation need not exhaust its attempt budget to fail.
budget exhaustion is not the sole definition of an error. - A valid empty proposal list or valid assessments rejecting all corrections is
- A valid empty proposal list or valid assessments rejecting every correction is successful processing. Correction count alone does not determine failure.
a successful outcome. Correction count does not determine success or failure. - Keep safe filtering of defective proposals. Do not retry ordinary editorial
rejections. For now, an unusable validator decision set excludes the affected
candidates and records an error after existing permitted recovery. New bounded
feedback/regeneration for domain-invalid responses belongs to future work.
- Retain fatal treatment for invalid input/configuration, internal defects, - Retain fatal treatment for invalid input/configuration, internal defects,
explicit run cancellation, and inability to produce/write the transcript. explicit run cancellation, and inability to produce/write the transcript.
Distinguish caller cancellation from a local model-operation timeout. Caller cancellation is distinct from a local model-operation timeout.
- A completed run with skipped LLM work returns success and makes that degradation - Completed runs with model errors return success. Required report-write failures
visible in its summary and error records. Fatal run failures remain nonzero. are an additional fatal output condition when reporting is enabled, as below.
PromptKit continues to own model transport, transient retries, structural repair, PromptKit owns transport, transient retries, structural repair, and admission.
and admission. Audita owns work-unit outcomes, domain acceptance, fallback, and Audita owns domain acceptance, partial outcomes, fallback, and classification.
report classification. ConfigKit continues to own configuration resolution. ConfigKit owns configuration resolution. Debug and reporting choices must not
Debugging must not change correction decisions, retry budgets, or continuation. change correction decisions, retry budgets, or continuation.
## Errors, Recoveries, and Debug ## Errors, Recoveries, and Debug
Classify by eventual outcome and impact, not exception type or retry count alone. Classify by eventual outcome and impact. These are three reporting categories;
These are three reporting categories; debug is evidence, not a failure severity. debug is evidence, not a failure severity. Whether execution continues is a
Whether execution can continue is separate from the category. separate property.
| Category | Meaning | Representative cases | | Category | Meaning | Representative cases |
| --- | --- | --- | | --- | --- | --- |
| Errors | Intended work could not complete reliably and was skipped, or a fatal run-level operation failed. | Exhausted output repair; unavailable provider; unusable validation decisions; operation deadline. | | Errors | Intended work could not complete reliably, or a fatal run-level operation failed. Affected candidates are excluded; other corrections can succeed. | Exhausted repair; unavailable provider; unusable validation decisions; operation deadline. |
| Recoveries | A problem was handled successfully through retries, repair, or safe filtering. No operator intervention is implied. | Invalid output repaired; transient provider failure recovered; defective proposals safely discarded. | | Recoveries | A problem was handled successfully through retries, repair, or safe filtering. No operator intervention is implied. | Invalid output repaired; transient service failure recovered; defective proposals safely discarded. |
| Debug | Detailed evidence of processing and ordinary decisions. | Proposals, applied corrections, validator decisions, prompt/response content, individual attempts, detailed execution traces. | | Debug | Detailed evidence of processing and ordinary decisions. | Proposals, applied corrections, validator decisions, prompts/responses, individual attempts, execution traces. |
Normal editorial disagreement belongs in debug: a validator correctly rejecting Normal editorial disagreement belongs in debug. A validator correctly rejecting
a meaning-changing edit is expected behavior. A proposal referring to nonexistent a meaning-changing edit is expected behavior. A proposal referring to nonexistent
original text is defective; safely filtering it is a recovery. A schema-valid original text is defective; safely filtering it is a recovery. An unresolved
validation response with missing or duplicate decision indices is an operation missing/duplicate decision-index response is an error, not a recovery.
failure when it cannot be recovered. Do not label unresolved output as recovered.
Record the eventual outcome of an operation once. Several unsuccessful attempts Record an operation's eventual outcome once. Several failed attempts followed by
followed by success yield a consolidated recovery. Attempts ending in failure success produce a consolidated recovery. Attempts ending in failure contribute
contribute to the error, not additional recoveries. Earlier operations that did to error details, not additional recoveries. Earlier operations that recovered
recover remain recorded if a later operation fails the work unit. remain recorded if a later operation introduces an error in the same unit.
Separate output-quality recovery, defective-proposal filtering, and service Separate output-quality recovery, defective-proposal filtering, and service
recovery using structured reasons. Network instability should not be mistaken recovery with structured reasons. Network instability must not look like a prompt
for a prompt deficiency. Preserve attempt counts and affected-proposal counts defect. Do not introduce an arbitrary warning threshold or a fourth category.
without counting each attempt or rejected proposal as an independent incident.
Recovery frequency needs processing-volume context. Seven recoveries need not ### Counting and Processing Volume
warrant attention; seventy may or may not, depending on the amount and kind of
work. Do not introduce an arbitrary warning threshold or restore “warnings” as a
fourth category. Final counting/denominator details remain open below.
## Production Outputs and Debug Boundary - Count error-bearing work units once per stage/chunk, consolidating underlying
operation failures as details. A partially successful unit still counts once.
- Count recoveries once per recovered operation. Aggregate defective-proposal
filtering per proposal response, retaining affected-proposal counts as detail.
Attempts and individual discarded proposals are not independent incidents.
- Keep work-unit coverage separate from operation/recovery counts. Show processing
volume with explicitly labeled denominators; do not divide recovered-operation
counts by work-unit counts and call that a recovery rate.
- Report fatal run-level errors separately from the stage/chunk grid when they
cannot be attributed to a work unit. Do not invent a chunk to accommodate them.
- Derive YAML records, Markdown summaries, and CLI counts from actual processing
outcomes with deterministic ordering, rather than independently calculating them.
The selected ordinary report set is: ## Production Outputs, Placement, and Lifecycle
- `errors.yml`: structured records of unsuccessful work and applicable run failures. Reporting is enabled by default. The ordinary bundle is:
- `recoveries.yml`: structured records of successfully handled problems.
- `report.md`: a human-facing summary of outcome, processing volume, applied
corrections, errors, recoveries, and their locations in the pipeline.
Use Audita-owned structured records as the common source for YAML, Markdown, and - `errors.yml`: structured error-bearing units, their underlying failures, and
CLI summaries. Reuse useful existing report concepts rather than expose raw applicable run-level failures.
library structures or maintain independently calculated totals. Exact schemas - `recoveries.yml`: structured recovered operations and pertinent counts.
are not finalized. They should identify the run, stage instance, stable chunk, - `report.md`: a concise human-facing summary of outcome, corrections, processing
operation, validator/batch where relevant, reason, outcome, and pertinent counts. volume, errors, recoveries, and their locations in the pipeline.
Keep safe human-readable explanations alongside machine-readable reason codes.
Production issue records should be compact. Full responses, transcript excerpts, Write all three documents, including empty event lists for uneventful completed
proposed/applied correction lists, and complete validation decisions belong in runs. Use Audita-owned schemas with safe explanations and stable reason codes;
opt-in debug artifacts. Debug output may continue using JSON where suitable. retain run, stage-instance, chunk, operation, and validator/batch identities where
Preserve configured-secret redaction across every presentation and artifact. applicable. Exact schemas are detailed-design work, not a reason to preserve the
old report formats or expose raw library structures.
Provide an explicit debug opt-in, with `--debug` as the proposed CLI spelling. Place reports beside the output transcript. Retain transcript-to-stdout support
A configuration counterpart is desirable but its exact interface remains open. with an explicit report-directory selection; its default is a remaining interface
Debug-off must prevent detailed artifact persistence in the first place; writing detail. Preserve the existing overwrite convention: a new run can replace files
those files and deleting them later does not satisfy the target. Temporary at its selected destinations. Do not add a mandatory fresh-directory restriction
in-memory processing data needed for correct execution is not debug output. or compatibility exports.
Ordinary reports must survive independently of debug retention. The current Before model work, create initial versions of all three reports carrying a common
`--report-json`, run-local report, ledger, execution/utilization artifacts, and run identifier and explicit `in_progress` status (rendered as prose in Markdown).
retention settings need a deliberate disposition. Do not silently retain all Fail immediately if initialization fails. Also check the transcript destination;
existing verbose output as additional production defaults. Coordinate the creating neighboring reports does not prove that an existing transcript file is
retention decision with the existing [runtime follow-up](runtime-behavior.md). writable. Preflight must not prematurely destroy an existing transcript.
## Human-Facing Presentation Finalize all required outputs and check every final write. Initial write success
cannot guarantee later success if space or permissions change. Failure to write
the transcript or finalize a required report returns nonzero and must not erase
an already produced transcript. Where possible, finalize available reports with
an explicit failed status on fatal termination. Abrupt termination may leave
`in_progress`; elaborate crash-recovery machinery is not required.
Provide a concise completion message showing corrections applied, work units Provide an option to disable reporting entirely. It suppresses all three files
skipped due to errors, and recoveries. For example, the intended style is: (`errors.yml`, `recoveries.yml`, and `report.md`), including initialization and
final writes, while retaining the brief CLI summary and independently selectable
debug output. Disabled reporting must not create or require a writable report
destination, and must not remove reports from earlier runs.
Transcript output failures remain fatal regardless of reporting selection.
> Completed: 18 corrections applied; 3 work units skipped due to errors; 7 recoveries. Ordinary reports survive independently of debug cleanup. Reporting configuration
and finalization must not depend on retaining a debug run directory.
Include report locations when available. Send the summary to stderr so stdout ## Debug and Internal Boundaries
remains available exclusively for transcript output. Successful completion must
not imply that every stage/chunk succeeded.
`report.md` should help the reader locate concentrations of failures and recoveries Production issue records are compact. Full responses, transcript excerpts,
without opening debug data. Stable chunks support the proposed presentation of correction lists, and complete validation decisions are opt-in debug evidence.
separate error and recovery matrices: chunks as rows, configured stage instances Use a single debug boolean, with `--debug` as the proposed CLI spelling and a
as columns, and counts in each cell. Preserve execution order and distinguish ConfigKit-backed setting/override design to be finalized. Debug artifacts may use
repeated stages. Include totals and a short breakdown by reason. JSON where useful. Preserve configured-secret redaction in every presentation.
Distinguish completed work with zero events from work that was not attempted. Debug-off prevents detailed artifact persistence; writing and later deleting
Do not present canceled or unavailable work as cleanly completed. Exact table those files is insufficient. In-memory domain state needed to process candidates
counting, omission of uneventful rows, and large-report layout remain open below. correctly is not debug output. Debug and ordinary reporting are independent
selections and do not alter processing behavior.
There is no obligation to retain `--report-json`, old artifact layouts, parallel
exports, or old retention settings. Reuse code only where useful to the new
workflow. Resolve the disposition of debug retention alongside the existing
[runtime follow-up](runtime-behavior.md); the legacy `never` behavior must not
quietly become the meaning of the new reporting-disable option.
## Deterministic Human-Facing Presentation
Use the CLI style:
> Completed: 18 corrections applied; 3 work units with errors; 7 recoveries.
Do not say that all error-bearing units were skipped: some apply valid corrections.
Include report locations when reporting is enabled and the files exist. Send the
summary to stderr so stdout remains exclusively available for transcript output.
Disabled reporting does not suppress the summary or fatal-error messages.
Keep `report.md` concise and high-level. Use separate error and recovery matrices
with stages as columns and chunks as rows. Include every chunk in fixed order;
retain configured stage order and distinct labels for repeated module instances.
No automatic width detection, thresholds, adaptive omission, or table splitting.
A wide or long table is acceptable.
The error matrix counts error-bearing work units: each attempted cell is 0 or 1.
Recovery cells count recovered operations and can exceed 1. An error cell may
coexist with recoveries and successfully applied corrections in the same unit.
Distinguish completed work with zero events from unattempted work; do not portray
canceled/unavailable processing as clean completion. Include totals, processing
volume, and a short reason breakdown. Detailed inspection belongs in YAML or debug.
## Target Acceptance Criteria ## Target Acceptance Criteria
- A controlled run in which every LLM operation fails still returns success and - Total LLM failure still returns success and the encoded normalized transcript,
the normally encoded normalized transcript, with no LLM corrections applied. provided required output writes succeed.
- Mixed-success runs preserve committed corrections, skip unsuccessful work - Mixed-success runs preserve independently validated, applicable corrections
safely, and attempt subsequent stages and unaffected chunks. within the same stage/chunk as a failed batch. No candidate bypasses a required
- Chunks retain their identity and boundaries across all stages; validation validator; later validators handle survivors by stable proposal identity.
batching does not repartition them. Final recombination preserves ordering. - Subsequent stages and unaffected chunks execute after recoverable failures.
- Recoverable model failures do not cancel unrelated work. Fatal failures and Fatal errors and explicit cancellation stop and join active work appropriately.
explicit cancellation still stop and join active work appropriately. - Chunk boundaries and identity remain fixed, with operator-selected sizing and
- Valid no-change and ordinary editorial-rejection outcomes remain successful. unchanged defaults; final recombination preserves ordering.
Repaired/filtered output and unrecovered failures receive distinct classifications. - Valid no-change/editorial-rejection outcomes are successful. Repaired/filtered
- YAML records, Markdown tables, and CLI totals agree, with deterministic ordering output and unresolved failures receive distinct classifications and counts.
and no duplicate incident counts from retries or repairs. - YAML, Markdown, and CLI agree on partial outcomes and totals; error matrices
- Debug-off produces the selected ordinary output set without detailed debugging count affected work units, recovery matrices count recovered operations.
artifacts; debug-on adds evidence without changing processing outcomes. - With reporting enabled, all three reports are initialized before model work,
- Tests exercise real runner/runtime/reporting paths with controlled providers, finalized and retained, including empty lists. Initial or final required-output
including exhausted recovery, partial success, and total LLM failure. Existing write failures produce nonzero exits without destroying completed transcript output.
domain safeguards, capacity limits, redaction, and output contracts remain - Reporting can be disabled without requiring its destination. Debug can be enabled
covered where they are retained. independently; neither selection changes processing outcomes. Debug-off writes
- On implementation, update the canonical CLI, configuration, operations, no detailed debug artifacts, and stdout transcript output remains supported.
integration, internal, and architecture documents to describe the new behavior. - Reports use the fixed deterministic layout without adaptive presentation logic.
This roadmap alone does not change their current contracts. - Meaningful controlled-provider tests cover exhausted recovery, failed validation
batches with successful survivors, total failure, fatal cancellation, output
failures, and reporting/debug selections. Preserve domain safeguards, capacity
limits, and redaction.
- On implementation, update canonical CLI, configuration, operations, integration,
internal, and architecture documentation. This roadmap does not change their
current implemented contracts.
## Open Questions ## Open Questions
### Work-unit atomicity when only some validation batches succeed The product choices above replace the earlier alternatives and user comments.
Remaining details belong in the eventual decision-complete implementation plan:
**Recommendation:** Commit a stage/chunk unit only after all required operations - Select exact CLI/configuration names and precedence for reporting enablement,
complete reliably. If a required batch fails, discard that unit's pending edits, debug, and the report directory for stdout runs. Reports for file-output runs
record one work-unit error with the underlying operation failures, and preserve belong beside the transcript; the stdout report-directory default is not yet set.
its incoming text. Normal valid rejections are not operation failures. This gives - Finalize YAML versions/fields, machine identifiers versus human chunk labels,
predictable fallback and straightforward error counting. operation-volume denominators, deterministic record order, and failed/not-attempted
presentation without changing the agreed counting or fixed table layout.
**Alternative:** Preserve fully validated corrections from successful batches in - Specify how destinations are checked without destructive transcript truncation,
an otherwise failing unit. This retains more useful work but requires explicit finalization order, path-collision handling, and debug placement/retention. Keep
partial-success outcomes and clearer counts than “work unit skipped.” ordinary filesystem behavior proportionate; do not add a transaction framework.
### Domain-invalid responses and recovery opportunities
**Recommendation:** Continue safely filtering independently defective proposals;
do not retry ordinary editorial rejections. Treat an unusable validator decision
set as an error when existing permitted recovery cannot resolve it. Decide
explicitly whether dynamic decision-index/cardinality checks should become part
of the repair contract; this roadmap does not silently add semantic retry loops.
**Alternative:** Extend recovery to regenerate domain-invalid decision sets with
bounded feedback. This may improve completion quality, but its ownership,
feedback contract, and cost budget need a separate concrete design.
### Initial chunk sizing and later prompt growth
**Recommendation:** Retain operator control over chunk size and choose reasonable
headroom for known stage prompts. Keep validation batching within fixed chunks;
report a later operation that cannot fit as skipped work. Establish the concrete
sizing rule during detailed design rather than invent a universal safety margin.
**Alternative:** Size initial chunks against conservative estimates for all
selected stage/profile combinations. This reduces later size failures but may
increase call volume and still cannot guarantee fit after arbitrary text growth.
### Exact event counts and recovery-rate denominators
**Recommendation:** Count errors once per failed stage/chunk unit, linking multiple
underlying failures as details. Count recoveries once per recovered operation;
aggregate defective-proposal filtering per proposal response with an affected
count. Report work-unit coverage separately from logical-call recovery counts,
with explicit denominators. Earlier recoveries may coexist with a later error in
the same unit.
**Alternative:** Count affected work units for both matrices, placing operation
counts only in details. This makes the two tables directly comparable but hides
multiple distinct recoveries within a single unit unless the reader drills down.
### Output placement, empty files, and report-write failures
**Recommendation:** Produce all three ordinary documents for every completed run,
using empty event lists where appropriate. Select a durable report directory
independent of debug cleanup, including when transcript output goes to stdout.
Specify its CLI/default path and overwrite policy before implementation. Attempt
partial reports on fatal failures when possible. Report-writing failures should
be explicit and should not erase an already produced transcript; whether they
make the invocation nonzero must be settled as part of the output contract.
**Alternative:** Write event files only when nonempty and identify their absence
explicitly in `report.md`. This reduces empty files but requires consumers to
understand missing-file semantics. Best-effort ancillary reporting could preserve
a zero exit status after transcript success, at the cost of weaker evidence that
the promised output bundle is complete.
### Existing report interfaces and debug configuration
**Recommendation:** Keep useful internal structures and measurements, adapting the
ordinary schema to its audience and moving verbose evidence behind debug.
Finalize a versioned YAML envelope, identifiers/index display, and the disposition
of `--report-json` and retention settings in detailed design. Use one debug boolean
with a CLI override and optional ConfigKit-backed setting; avoid a matrix of
independent artifact switches.
**Alternative:** Retain explicit machine-oriented JSON export alongside the new
ordinary outputs for consumers that need it. This preserves an existing interface
but adds another maintained presentation. Compatibility/export requirements have
not been selected for this feature; do not infer them from prior migration policy.
### Large Markdown reports
**Recommendation:** Use separate matrices with totals and reason summaries; omit
uneventful chunk rows when that improves readability, explicitly reporting the
omitted count. Show zero separately from unattempted work. Keep full incident
details in YAML and identify chunks by stable labels and segment ranges.
**Alternative:** Always include every chunk row, splitting wide tables into stage
groups. This gives an exhaustive visual grid but can make long transcripts harder
to review. Exact layout and any row-selection threshold remain undecided.