audita/docs/roadmap/implementation.md

Implementation Plan: Resilient Processing and Focused Reports

Last updated: 2026-09-24

Status: Ready for implementation; all stages below are pending. Planning only: no implementation is claimed by this document.

Scope and Execution Instructions

Implement the target in resilience.md. User intent and product policy belong there; the concrete choices and ordered steps below implement them. Everything in future.md is excluded, especially domain-aware regeneration, independent whole-pipeline traversal, and new empty-replacement or segment-removal semantics. Keep the existing libraries, prompt text, retry/repair budgets, chunk-size defaults, transcript formats, and domain acceptance rules.

This plan targets a gpt-5.6-terra coding agent. Execute Stage 1 through Stage 14 in order. Each stage is a bounded implementation prompt. If assigned the full plan, continue without intermediate permission rounds. If assigned one stage, complete its scope and report the next stage. Record completion and actual validation beneath each stage; do not mark deferred integration checks as passing.

Read AGENTS.md, development guidance, and every policy under docs/policy/. Use graph-first discovery. Observe the agent, architecture, testing, documentation, and release policies. The selected resilience behavior deliberately replaces the current blanket runtime-failure cancellation and reporting contracts; update their canonical descriptions as implementation lands.

These stages partition one refactor, not supported intermediate releases. Intermediate commits need not build or run. Do not add temporary adapters, compatibility shims, duplicate pipelines, or fake success results to keep them working. Use real final types, reconnect their consumers in subsequent stages, and record unavailable checks honestly. Commit coherent stages with plain-English messages after appropriate checks under the standing agent policy.

Tests are deterministic, offline, and credential-independent. Prefer real internal collaborators and controlled HTTP/transport/filesystem boundaries. Run focused checks as their owning stage becomes testable, then the complete validation in Stage 14. Do not duplicate a test matrix at every layer. No dependency upgrades, upstream edits, paid calls, release publication, CI waiting, or unrelated timestamp or normalization changes are authorized by this plan.

This replaces the completed library-adoption plan. Its historical completion record is available with git show cb405f6:docs/roadmap/implementation.md. Legacy references to that completion evidence do not imply this new plan is done.

Concrete Design Decisions

A. Settings, CLI, and Compatibility

Retain configuration version 2 and ConfigKit's file/overlay/visited-override resolution. Add these settings:

Setting	Default	CLI override	Meaning
reporting.enabled	true	--reporting[=true|false]	Write the ordinary three-file report bundle.
reporting.stdout_dir	.	--report-dir DIR	Report destination when the transcript goes to stdout; relative to invocation working directory.
diagnostics.debug	false	--debug[=true|false]	Persist detailed diagnostic artifacts.
diagnostics.work_dir	Existing default	Existing --work-dir	Parent of unique debug directories; used only with debug enabled.

Register the new setting overrides through the same parser used by configuration inspection commands. Only explicitly visited flags override resolved settings; explicit false must override true in a file or overlay. Boolean overrides must reach ConfigKit as booleans, not strings. --reporting=false is the sole reporting disable spelling; do not add a parallel negative alias.

For file transcript output, always place reports in filepath.Dir(outputPath). reporting.stdout_dir/--report-dir have no effect in that mode, or when reporting is disabled; help must say so. With stdout output, use the selected stdout report directory, defaulting to the invocation working directory. Retain stdout support. Configuration inspection performs no output-directory creation or permission probes. At process destination selection, require a nonblank stdout report directory only for reporting-enabled stdout runs, and a nonblank debug parent only when debug is enabled. Perform filesystem checks only for destinations actually used.

Remove --report-json, diagnostics.retention, --work-dir-retention, their enum, and old automatic retention logic. Debug directories are retained whenever debug is enabled, on success or failure; otherwise none is created. No compatibility aliases or legacy ordinary JSON report are required. Keep transcript/schema, profile selection, and other existing settings unchanged. Do not reinterpret --output -; existing omission of --output continues to select stdout.

B. Processing Identity and State

Partition the normalized transcript exactly once in the runner, before entering its module loop. Return that initial section inventory/summary to CLI reporting; remove the separate pre-run CLI chunking pass. Preserve existing nil-config test behavior and the chunker's handling of empty input and impossible constraints.

Keep an ordered slice of chunk states with immutable section index, segment IDs, and initial section metadata, plus mutable segment text. A stage finishes across chunks before the next starts. Apply approved corrections within their owning chunk using the existing replacement policy and application checks, then carry that text to the next stage. Recombine only for the final transcript. Chunking estimates in the inventory describe the initial partition; later validation budgets still estimate the current payload, not stale initial text.

Retain ResolveModuleRunSpecs instance names. Internal chunk, batch, and proposal indices stay zero-based. Preserve the existing stage-wide proposal-index ordering: chunk order followed by response proposal order, assigned independently of worker completion order. A proposal's identity is its module instance plus proposal index, with section index retained for attribution. Never join decisions by batch number or provider call ID. Batch numbering is local to one validator in one chunk.

Represent each stage/chunk outcome as not_started, completed, or interrupted, with applied-correction count, compact operation outcomes, and optional debug records. completed means the unit reached a terminal processing outcome, not that all model operations succeeded. Keep known inventory for unattempted units on fatal failure. A model error can coexist with applied corrections and recoveries.

C. Domain Outcomes and Model Failures

Introduce a small internal/framework/outcomes package for processing-owned operation references, execution counts, reason codes, and outcomes. It must not import reporting, CLI, providers, or the runner. Do not create a generic event bus, workflow framework, or persistence interface in the processing model.

An operation reference contains module instance/key, section index, operation kind (proposal, validation, or validation_input), optional validator name/order, and optional batch index. Proposal operations occur once per work unit. An oversized-input failure uses validation_input without a batch; consolidate such items per validator invocation and record affected proposal indices. Ordinary validation operations represent actual structured-client invocations, not retries.

Compact operation outcomes carry execution counts when available, terminal success or failure, safe failure reason, and filtering details keyed by proposal identity. Add them to module/validator results and preserve them on fatal returns. Keep provider attempts, transient retries, and structural repairs distinct. Use a presence/availability flag for execution counts rather than inventing totals if an external client supplies none.

Keep typed Go errors for failures that require aborting the run. Extend structuredoutput.Failure to distinguish a recoverable model-operation failure from a fatal adapter/application failure and parent cancellation, with a stable safe reason code and preserved Unwrap cause. Domain callers consume recoverable failures into unsuccessful operation outcomes; they do not return them as fatal runner errors. Bare/unclassified internal errors remain fatal.

Use this classification order at the adapter boundary:

Condition	Result
Parent processing context is canceled or expired	Fatal cancellation; preserve parent cause.
Invalid output target, unknown embedded schema, prepared-target mismatch, missing/broken prompt/profile/schema asset, invalid request/configuration, or nil result without error	Fatal invariant/setup failure.
Local operation/attempt deadline while the parent is live	Recoverable operation_timeout.
Capacity rejection	Recoverable capacity_exceeded.
Transport error	Recoverable transport_failed.
Malformed provider HTTP response/envelope	Recoverable provider_response_invalid.
Provider HTTP failure, including non-retryable statuses	Recoverable provider_failed, with numeric status when available.
Retry exhaustion	Recoverable failure using its known underlying category; retain exhaustion and attempt counts as detail.
Returned output remains structurally invalid after permitted repairs	Recoverable invalid_output_exhausted.
Structurally accepted output has a model value that cannot fit its Go domain target, e.g. integer overflow (json.UnmarshalTypeError)	Recoverable invalid_output.
Other decoding/invariant failures or unknown library errors	Fatal; do not convert every exception into a successful no-op.

Use errors.Is/errors.As, PromptKit's typed errors and terminal metadata, and the phase of failure; never parse provider error text. Check known setup/invariant causes before broad generation/validation wrappers. ErrValidation by itself is not proof of repair exhaustion. A non-nil invalid result and known provider causes must be distinguished from an embedded validation-contract defect. PromptKit v0.11.0 exposes the required provider/transport/capacity and retry information; no upstream change is part of this work.

Do not expose raw causes, provider messages, URLs, or prompt fragments in ordinary reports. Use fixed safe explanations plus structured codes/status/counts. Redact all configured backend credentials, including unselected backends, at output.

D. Partial Validation and Safe Application

Every submitted candidate still receives exactly one decision per validator. Represent model-unavailable candidates as explicit non-approved decisions with an operation-failure disposition; distinguish them from valid editorial rejection. This preserves cardinality and prevents accidental approval while allowing other batches to return their genuine decisions. Keep decision-level details in debug; ordinary errors group their operation failures by work unit.

On a recoverable batch failure, reject only that batch's candidates, record the failed operation, and continue remaining batches and later validators for survivors. Missing/duplicate/unknown decision indices invalidate that batch as a whole; do not salvage a subset of an untrustworthy decision set or add semantic retries. Reject oversized individual validation inputs without sending them, record a validation_input_too_large failure, and still batch/process the other candidates. A fatal error or parent cancellation instead stops the validator, retaining its partial observations for diagnosis but never presenting unchecked candidates as approved.

Retain every required validator in its current order. Only candidates approved by all required validators reach ApplyProposals. Application stays deterministic and may reject proposals made stale by earlier accepted corrections. A failed batch does not roll back successful unrelated corrections. Fatal runs do not publish a completed transcript; already committed earlier-stage state remains available for partial diagnostic evidence.

Preserve current empty-replacement semantics: an empty replacement can remove a span only if existing preview/application rules allow it and the segment remains nonempty. Do not alter prompt/schema/deletion behavior under this plan.

E. Recoveries and Counts

The adapter returns per-call execution counts in structuredoutput.Response, including on error, using the terminal PromptKit observation. Callers determine whether the final domain result was usable. The adapter alone must not declare recovery merely because HTTP/schema handling succeeded.

Create at most one recovery record per successful logical model operation. Its reasons list can contain output_repaired, service_recovered, and, for a proposal response, proposals_filtered. If repair succeeded but its final domain decision set is unusable, that operation is an error, with its repair/retry counts in error details; it is not also a recovery. Earlier successful operations retain their recoveries even if later operations in the unit fail.

Consolidate defective proposals filtered anywhere in that unit into the originating proposal operation's recovery. Deduplicate affected proposal indices and retain reason/count details; never add one recovery per rejected proposal or validator. Recoveries from retries/repair and filtering of the same proposal response merge.

Classify existing rejection reasons deliberately:

• Filtering invalid target IDs, absent target/original text, empty original text, invalid confidence, no-effect proposals, ambiguous original spans, and proposals that would empty a segment contributes to proposals_filtered.
• Low confidence within the valid range, protected-term policy rejection, and valid LLM editorial rejection are ordinary debug decisions, not recoveries.
• A proposal becoming stale or redundant because an earlier accepted proposal changed the text is normal apply-time conflict resolution, not a model defect. Compare applicability to the stage/chunk input snapshot when necessary; do not relabel every apply-time skip as a recovery.
• Failure to validate because of size, invalid decision sets, or model availability is an error, never a filtering recovery. Unexpected missing candidates or broken cardinality in Audita's own assembled result is fatal internal failure.

Counts are derived once from processing outcomes:

• work_units_total: stage instances times initial chunks, zero for either empty set.
• work_units_started, work_units_completed, work_units_interrupted, work_units_not_started: explicit states, not inferred from absent events.
• work_units_with_errors: distinct stage/chunk units with recoverable operation failures; a partially successful unit counts once.
• run_errors: fatal errors outside that recoverable-unit count. A fatal error may carry an origin operation reference without creating a duplicate unit error.
• corrections_applied: applied proposals, not replacement occurrences.
• llm_operations: actual structured-client invocations including pre-dispatch failures; proposal_responses: structurally usable proposal responses.
• recoveries: unique successful operations with recovery reasons. Show counts alongside llm_operations; for filtering-specific context show affected proposal responses alongside proposal_responses. Do not create a single ambiguous percentage or count repair generations as independent operations.

Reason summaries count records containing each reason. Explain that one recovery can have multiple reasons, so those category counts need not sum to its total. Knownness/completeness of usage and existing indexed execution aggregation remain intact. Full token/timing/attempt evidence belongs in debug, not an expanded ordinary report.

F. Ordinary Output Schema and Markdown

Use explicit Audita DTOs in core/reporting, separate from runner/domain types; framework/processreport maps processing state to those DTOs. Render YAML with yaml.v3 and explicit YAML tags. Do not export raw PromptKit types or serialize errors with their causes. Schema version is integer 1 for these new document names; this is not the former JSON process-report schema.

Both YAML documents have these envelope fields, in this order:

Field	Definition
schema	audita-errors or audita-recoveries
version	1
run_id	Same run ID across reports/debug; generated once independently of debug directory creation.
status	in_progress, completed, or failed; model errors alone still yield completed.
started_at, completed_at	UTC RFC3339 strings; completion is null initially.
summary	Counts named in E, in that order. Unknown pre-processing counts are null; completed empty counts are zero.
errors or recoveries	Always an ordered sequence, including [] when empty.

errors contains two record variants with explicit scope:

• work_unit: stage_index, module_key, module_instance, chunk_index, start_segment_id, end_segment_id, corrections_applied, and failures. Each failure contains operation, optional validator/batch_index, reason, fixed safe message, sorted proposal_indices when known, and optional execution counts (provider_attempts, transient_retries, structural_repairs, retries_exhausted, http_status). Do not duplicate transcript text.
• run: phase, reason, safe message, and optional originating stage/chunk/ operation identity. Include primary fatal and secondary output-write failures without pretending these are skipped LLM work units.

Each recovery contains stage/module/chunk identity as above, operation, optional validator/batch_index, ordered reasons, fixed safe message, execution counts when known, and filtered_proposals with sorted indices and reason/count entries when filtering occurred. One record can describe both repairs and filtering.

Indices in YAML are zero-based. Display chunk and batch numbers as index + 1 in Markdown/CLI, while retaining exact module-instance names. Order unit records by configured stage order, then chunk index. Within units order proposal first, then validators by chain order, input preparation before batches, then batch index. Sort reason lists by code and proposal indices numerically. Render run errors in lifecycle phase order with output paths sorted for same-phase secondary failures. Use semantic identity for order; concurrent call IDs/timestamps do not determine report row or event order.

report.md always has a title/run ID, status, concise correction/coverage/error/ recovery totals, input/output paths, and the two matrices. Stage columns follow configured order; all chunks appear as rows, labeled with 1-based chunk number and initial segment range. Error cells are 0 or 1 for completed units; recovery cells are counts. Use — for not-started units and n* for interrupted units, where n is the available count; include a fixed legend. Add totals and short reason breakdowns and refer to the YAML files. An empty stage or chunk set gets an explicit “No work units” sentence instead of an invalid empty table.

No adaptive width checks, row omission, table splitting, transcript excerpts, or correction lists. Escape Markdown paths/labels, especially pipes, newlines, and backticks. Timing values may differ between runs; ordering/layout given the same snapshot are deterministic. Redact strings before presentation without corrupting numeric/boolean fields or resurrecting the old unredacted fallback.

G. Destination Lifecycle and Failure Semantics

An output session in core/diagnostics owns filesystem state; CLI composes it. The runner and renderers perform no filesystem operations. Report directories must exist, matching the existing transcript-output parent requirement; do not silently create ordinary output parents. Debug parents may still be created by the debug-directory writer.

After valid configuration/profile inspection, resolve paths and guard collisions before any report truncation. Compare cleaned absolute paths, resolved existing symlink targets/parents, and os.SameFile for existing files. Reject collisions among the transcript output and the three enabled reports, and with transcript, glossary, and known loaded configuration/profile source files. These checks prevent self-overwrite, not hostile filesystem races; allow ordinary symlink use and existing-output replacement. Return usage/configuration classification 2 for conflicting destinations. Disabled report paths are not resolved, probed, created, truncated, removed, or included in collision checks.

For file transcript output, verify an existing target can be opened for writing without O_TRUNC; close it. Probe creation in its parent with a standard temporary file and remove that probe. For an absent target probe the parent without leaving an empty purported transcript. Validate target type and collisions first; do not truncate an input or existing transcript during preflight. Stdout cannot be preflighted this way; its eventual write error is fatal.

With reporting enabled, initialize all three final report paths with the same in_progress run ID and empty event lists before input processing/model dispatch. Return 1 immediately if initialization fails; do not invoke an LLM. Attempt to mark already created reports failed when possible. A readable but invalid transcript may then produce failed reports. Earlier configuration/profile failures may occur before report initialization and need not produce a bundle.

At successful processing completion, fully encode output before writing it. Use standard temporary-file/write/close/rename replacement within the destination parent for each final file, cleaning owned temporary files on failure. This is per-file replacement, not a multi-file atomic transaction or durability guarantee; no fsync protocol, locks, journal, or automatic rollback system. Preserve existing file-output permission behavior where applicable and never erase a successfully published transcript because a later report fails.

Write the transcript first, then errors.yml, recoveries.yml, and report.md last. Final write/close/rename errors are fatal even if preflight succeeded. Attempt the other enabled report finalizations, collect safe secondary failures, and make one bounded best-effort pass to write failed-status reports if finalization failed. Never retry indefinitely or print the success completion message after a required-output failure. Some files may retain an earlier/in-progress snapshot; the nonzero exit and stderr identify incomplete finalization.

On a processing fatal error or cancellation, do not publish a completed transcript; finalize partial reports as failed where possible. Root cancellation must not prevent local report cleanup/finalization after workers have joined. Uncatchable termination may leave in_progress; no resume machinery is added.

Debug remains optional evidence, not a condition for transcript success: failure to create/write debug artifacts emits safe stderr diagnostics but does not change processing or the exit status. Do not invent an error-bearing work unit for a debug write failure. Required ordinary reports and transcript writes retain their fatal semantics. Debug and reporting selections are independent.

H. Debug Artifacts and CLI Completion

With debug enabled, create one unique child under diagnostics.work_dir, using the run ID. Retain it on every outcome; no automatic cleanup/retention modes. Gate the writers and directory creation, not merely cleanup. Empty diagnostics paths/nil writers must propagate through module factory, proposals, and validators when off.

Keep useful current artifacts under debug: source/normalized snapshots, invocation, redacted effective settings, config origins, selected profiles, chunking/normalization summaries, correction/decision ledger, indexed execution/utilization data, and model interaction payloads. Ensure all proposed candidates, including those later rejected or unavailable, can be inspected when debug is on. Keep JSON where suitable; do not retain the old ordinary process report as a compatibility output. Remove obsolete models/helpers/tests rather than keeping dead serialization paths.

Full payload capture/persistence is not required for production summaries. Compact operation state and necessary processing data remain available with both reporting and debug off, so classification and CLI totals do not change. Use all configured secrets for every writer, including debug snapshots, YAML, Markdown, and surfaced messages. Do not assume metadata identifiers are safe merely because payloads are absent.

Successful CLI stderr begins with the selected style, using exact semantic counts: Completed: N corrections applied; E work units with errors; R recoveries. Report locations and the debug location follow only when actually created. There are no ordinary warning messages. Suppressing files does not suppress this brief summary or fatal errors. Stdout remains transcript-only; with --output, stdout is empty. Exit 0 means processing and all enabled required outputs completed; exit 1 covers runtime/cancellation/output failure; exit 2 retains usage/configuration classification. Reporting disabled removes only its file requirements.

Stage 1 — Define Processing Outcomes and Contracts

Completed 2026-09-24. Added processing-owned operation identities, execution counts with availability, safe outcome/recovery reasons, work states, explicit decision dispositions, and typed recoverable/fatal/canceled failure classes. The proposal, validator, and runner result boundaries now carry operation outcomes instead of warning fields. go test ./internal/framework/outcomes ./internal/framework/structuredoutput and git diff --check passed. Tests of contracts, validators, and runner are temporarily unavailable because existing validator/runner consumers still use the removed warning fields; Stages 3, 4, and 6 reconnect those consumers.

Work: Add the small outcome/reference/count types from B–E. Extend structuredoutput.Response and Failure, module proposal results, validator results/decisions, and runner output to carry actual processing outcomes. Replace StageWarning at these boundaries with the final outcome model; do not alias the old category to recoveries. Keep Go causes nonserializable and operation references independent of report DTOs. Make invalid/unavailable decision disposition explicit.

Touchpoints: internal/framework/structuredoutput, contracts, validators/models.go, runner/runner.go, and new framework/outcomes. Keep dependencies acyclic; shared outcome types cannot import their consumers.

Validation: Focused tests for typed error matching and decision partition/cardinality where behavior is introduced. Compile affected leaf packages. Record temporary consumer compile failures for subsequent stages; do not add compatibility fields.

Exit: Contracts can express success, recoverable model failure, fatal failure, partial validation, and recovery detail without any filesystem/report dependency.

Stage 2 — Classify Model Failures and Return Execution Facts

Completed 2026-09-24. The PromptKit adapter now classifies parent cancellation, setup defects, local deadlines, capacity, transport, provider envelopes/statuses, exhausted invalid output, and domain decode overflow with typed safe failures. It returns terminal attempt/retry/repair/status counts on successful and failed calls, while preserving observation hooks and typed causes. go test ./internal/framework/llm -count=1, its race suite, and git diff --check passed. Whole-repository compilation remains deferred while the warning consumers are reconnected in later stages.

Work: Implement C in llm/adapter.go using the pinned library's public types. Capture terminal counts locally for each call and return them even on failure; forward existing observation hooks without deriving recovery classifications in the adapter. Preserve prepared-target checks, deadline-before-preparation, same shared engine/capacity, typed causes, and safe messages. Handle parent cancellation before local timeout classification. Keep metadata collection concurrency-safe.

Validation: Extend adapter behavioral tests for repaired success/counts, exhausted invalid output, provider/transport/envelope/capacity failure, local versus parent deadline/cancellation, preparation defects, and decode overflow. Reuse existing retry/capacity tests rather than retesting PromptKit. Assert no additional outer attempts and no provider text in safe errors.

Exit: Domain callers can distinguish skippable model work from fatal errors without examining error strings; counts are available independently of debug.

Stage 3 — Convert Proposal Failures and Classify Safe Filtering

Completed 2026-09-24. Proposal generation and the shared path used by all four built-in modules now preserve one identified operation with terminal counts on success, recoverable failure, and fatal return. Recoverable failures yield no candidates; a valid empty response succeeds. A focused domain classifier separates defective proposals from editorial rejection without changing deterministic validator policy. Updated proposal, wrapper, and filtering tests are present. git diff --check passed. Those tests and the existing four-module routing tests cannot compile until the validator warning consumers are converted in Stage 4; the attempted go test reports those exact stale-field errors.

Work: Update proposal_generation.GenerateCandidates and all four module wrappers to preserve operation outcomes. A recoverable proposal call returns zero candidates plus a failed operation, not a warning or fatal Go error. A valid empty response is successful. Fatal errors retain their partial call facts and propagate. Carry successful-call counts until domain filtering can consolidate its recovery.

Implement the explicit filtering/editorial distinction in E at the domain decision boundary (a small reason classifier is sufficient). Preserve current deterministic validators, thresholds, glossary protection, and empty-replacement behavior. Keep payload diagnostics conditional on a supplied writer/path.

Validation: Proposal integration tests distinguish malformed exhaustion from fatal setup and valid empty output; deterministic filtering tests cover a defective proposal versus low-confidence/protected-term rejection. Reuse current four-module routing tests to prove wrappers preserve outcomes and identities.

Exit: Proposal callers no longer abort for model failures, and filtering can be attributed to the originating proposal response without treating normal rejection as recovery.

Stage 4 — Preserve Successful Validation Batches

Completed 2026-09-24. LLM validation now keeps successful batch decisions around recoverable failed batches, emits explicit unavailable decisions for failed or oversized inputs, and records one compact operation per client call plus one consolidated oversized-input operation. Invalid decision sets fail their own batch, and retry/repair recovery is recorded only after usable domain mapping. Fatal failures retain earlier facts and stop later batches. Focused validator, proposal, and four-module tests pass, as do validator/proposal race tests. git diff --check passed. Runner compilation is still deferred because its old warning fields are reconnected in Stage 6.

Work: Refactor LLMBackedValidator.Validate, input preparation, and response mapping per D. Collect all successful batch decisions; on a recoverable failure add non-approved unavailable decisions for that batch and continue. Keep invalid index sets batch-scoped, oversized candidates excluded individually, and errors separate from valid negative decisions. Record final domain outcome before declaring service/repair recovery. Stop promptly on fatal errors/parent cancellation and retain completed operation facts. Do not synthesize approval for remaining items.

Validation: One controlled multi-batch workflow proves successes before and after a failed batch survive, and a later validator receives only survivors under new batch numbering. Cover oversized individual input, missing/duplicate/unknown indices, repaired-but-domain-invalid output, ordinary rejection, and fatal stop. Use the existing estimator seam to force batches without brittle default budgets.

Exit: Every normally returned result has exactly one decision for each submitted candidate; failed model batches do not discard unrelated decisions or bypass later required validation.

Stage 5 — Fix Chunk Boundaries for the Entire Run

Completed 2026-09-24. The runner now partitions once before the module loop, returns the initial section inventory, applies approved corrections only inside their identified sections, carries current text forward, and recombines the transcript after processing. Tests cover substantial text growth, stable segment membership, empty work, and impossible chunk constraints. go test ./internal/core/chunking and git diff --check passed. Runner tests remain unavailable because warning-field consumers still fail compilation; Stage 6 replaces them with operation outcomes.

Work: Move the runner's chunking call before its stage loop. Own fixed chunk states and initial inventory in runner output. Execute each module using the current text of those same chunks, and apply fully approved candidates only within their chunk. Keep stage barriers, initial chunk metadata, deterministic proposal indexing, and final ordered recombination. Remove the redundant CLI chunk computation when wiring its consumer, not by adding an interim second inventory. Preserve within-stage proposal/validation concurrency and shared capacity; no new scheduler is required.

Validation: A multi-stage test deliberately grows text enough that old re-chunking would alter membership; assert stable chunk/segment identity and next-stage current text. Cover repeated modules, target-section selection, empty transcript/pipeline, and application ordering with unchanged output-schema behavior.

Exit: The normalized transcript is partitioned once and domain processing does not rely on report tables to maintain chunk state.

Stage 6 — Integrate Partial Outcomes and Fatal Cancellation

Completed 2026-09-24. The runner now preserves per-section operation facts, continues after recoverable model failures, applies only candidates approved by every required validator, and retains completed observations on fatal cancellation. It tracks explicit work-unit states and merges defective-proposal filtering with an existing proposal recovery. Runner tests cover total proposal failure, mixed validation outcomes, later modules, multiple failures in one unit, merged recovery, and fatal cancellation. go test -race ./internal/framework/runner ./internal/framework/llm and git diff --check passed. Integrated reporting checks remain deferred until the reporting stages.

Work: Update collectSectionProposals, runModulePipeline, and validateSectionCandidates to consume the outcomes from Stages 3–4. Recoverable failures never enter shared fatal cancellation. Preserve candidates through every required validator by proposal index and apply successful survivors even when the unit has errors. Never return unchecked eligible candidates as approved on an error path. Propagate completed outcomes even if another worker cancels the stage; do not drop buffered terminal evidence merely because the context is canceled.

Retain synchronized first-fatal-error ownership, immediate peer cancellation, worker joining, and deterministic collection. Track not-started/interrupted/completed units and the counts in E. On fatal stage exit do not apply incomplete validation results; retain prior committed stages and honest partial observations. Aggregate proposal filtering once, merge its reasons with any proposal retry/repair recovery, and retain earlier recoveries when later operations fail.

Validation: Runner tests prove total model failure returns normalized output; mixed batches apply only fully validated candidates; later stages still execute; multiple operation failures count one affected unit; repaired-plus-filtered proposals count once. Adapt the existing cancellation regression to a genuine fatal error: model failures now continue, while fatal failure cancels active/pending calls, retains the originating cause, joins workers, and preserves observations. Run focused runner/runtime race tests when those packages are testable.

Exit: Processing fulfills the resilience contract independently of file output.

Stage 7 — Add Reporting and Debug Settings and CLI Selection

Completed 2026-09-24. Version-2 settings now resolve reporting and debug independently, with typed visited boolean overrides and destination selection only for enabled reports. The old JSON-report and retention flags/settings were removed; CLI entry points now carry a context from interrupt/SIGTERM handling. ConfigKit tests and focused CLI parser tests cover defaults, overlay false overrides, bare booleans, destination selection, and removed flags. go test ./internal/core/config, go test internal/cli/config_input.go internal/cli/config_input_test.go, and git diff --check passed. Whole CLI tests remain unavailable until Stage 8 replaces the report builder's warning fields.

Work: Implement A in core/config/config.go, cli/config_input.go, invocation state, and usage text. Add boolean flags with visited-value overrides and update safe effective-setting/provenance output. Remove the old external JSON report flag and retention selection, including tests for their former semantics. Keep path selection in CLI composition, not domain configuration resolution. Add context plumbing through cli.Run(ctx, args, stdout, stderr) and process/runtime calls; update direct callers and tests. In cmd/audita, use signal.NotifyContext for interrupt/SIGTERM, with a small run function that defers stop before os.Exit. Configuration help/inspection still creates no output files.

Validation: Parser/ConfigKit tests cover defaults, overlay true overridden by CLI false, reporting/debug independence, stdout directory selection, ignored report directory in file/disabled mode, removed flags/keys, and side-effect-free inspection. No execution scheduler or output files are added in this stage.

Exit: Final settings/CLI contract is resolved without legacy aliases or accidental filesystem requirements for disabled features.

Stage 8 — Map Outcomes to Versioned YAML Reports

Completed 2026-09-24. The old ordinary JSON report DTO was replaced with versioned errors/recoveries YAML DTOs and one processing snapshot. Mapping derives nullable summary counts, grouped recoverable work-unit failures, run failures, combined recovery reasons, and deterministic semantic order without raw causes or transcript payloads. Tests cover partial and empty work, multiple failures, initialization unknowns, ordered run errors, safe redaction, and YAML scalar types. go test ./internal/core/reporting ./internal/framework/processreport and git diff --check passed. CLI and diagnostics consumers of the removed JSON schema remain to be reconnected in later stages.

Work: Replace ordinary report DTOs and assembly in core/reporting and framework/processreport with F. Map actual runner state once into an immutable report snapshot consumed by all presentations. Preserve debug-specific ledger and indexed execution mapping only where useful; remove superseded ordinary JSON models rather than maintaining two authoritative schemas. Implement YAML rendering for initial, completed, and failed snapshots, explicit empty lists/null unavailable counts, safe messages, stable ordering, and the two error record scopes.

Use existing shared redaction at the serialization boundary. Never marshal a raw error cause or depend on typed decode of already-redacted timestamps/identifiers. YAML numbers and booleans must retain their types even for numeric credentials.

Validation: Report tests use partial units, multiple underlying failures, combined recovery reasons, no-work input, run-level failure, unavailable initialization counts, and deterministic ordering despite reordered observations. Parse rendered YAML to verify types/counts and redaction, including unselected/escaped credentials. Preserve indexed aggregation and usage semantics in retained debug builders.

Exit: Both YAML documents truthfully describe the same processing snapshot with no filesystem effects or verbose payload leakage.

Stage 9 — Render the Fixed Markdown and CLI Summaries

Work: Render F/H from the Stage 8 snapshot. Produce both full matrices, totals, fixed legends, short reason breakdowns, and safe file references. Display 1-based chunk labels without changing machine identity. Include interrupted and unattempted states and the no-work sentence. Provide a small shared summary formatter for CLI counts; CLI stream/file wiring remains Stage 12. No width heuristics, percentage thresholds, row omission, or extra export formats.

Validation: One representative Markdown golden fixture with explicit update opt-in can protect the chosen layout; supplement with semantic checks for repeated stages, partial/error-and-recovery cells, empty work, and escaped labels. Assert CLI counts agree with YAML and that secrets never enter Markdown. Avoid snapshotting wall-clock values or every incidental sentence.

Exit: Human-facing output is concise, deterministic, and generated from the same truth as the structured output.

Stage 10 — Implement Required Output Initialization and Finalization

Work: Add the output-session filesystem adapter described in G. Resolve active paths, protect collisions before truncation, preflight transcript writes without destroying existing content, initialize all enabled reports, and publish final files using standard per-file replacement. Return structured primary/secondary errors for CLI reporting. Track only files/probes created by this invocation for cleanup; do not delete previous report files when reporting is disabled. Use ordinary file permissions and no multi-file transaction or crash-recovery framework.

Validation: Real temporary-directory tests cover initialization failure before a fake model is invoked, output/report/input collisions including existing aliases, existing transcript preservation, overwrite behavior, absent parent, disabled reports with an unusable report directory, stdout mode, final transcript/report failures, and partial failed-status finalization. For permission failures use a narrow injected file-operation seam when tests may run as root; do not rely solely on chmod.

Exit: Required output failures are detected early where possible and always reported at finalization without erasing a completed transcript.

Stage 11 — Make Debug Persistence Explicit and Independent

Work: Refactor core/diagnostics run-directory creation and all CLI/module/ proposal/validator writer wiring for H. Only debug creates the unique retained run directory. Gate snapshots, interaction files, ledger, and full execution data at the write source. Keep required ordinary reporting in its separate session. Remove automatic retention and the old never semantics rather than leaving a disabled cleanup path. Ensure debug contains all proposals and decisions needed to diagnose filtering/unavailable results, not just applied changes.

All debug writers use the shared configured-secret redactor, including original input snapshots; a raw source copy must not bypass it. Keep safe stderr reporting of optional-debug write failures separate from model incidents and fatal output errors. Debug write failures must not change correction decisions or abort the run.

Validation: Tests prove no debug directory/payload files with debug off, useful retained evidence with debug on, redaction across snapshots/interactions, and no outcome/exit changes caused by optional debug-write failure. Existing diagnostic writer tests are retained or replaced according to behavior, not old filenames.

Exit: Debug-off never means “write and later delete”; reporting-off/debug-on and reporting-on/debug-off both work without coupling their destinations.

Stage 12 — Compose the Final CLI Run Lifecycle

Work: Replace the current monolithic process/report/retention flow with the final composition: load/inspect settings, resolve and initialize required outputs, optionally initialize debug, load/normalize input, run fixed-chunk processing, render/finalize, and emit completion/failure output. Thread the caller context all the way to runtime/runner. Keep phase labels and root versus local cancellation clear. After joining workers, use available partial state to finalize failed reports without treating cancellation as an instruction to abandon local cleanup.

Remove the old pre-run chunking pass and obtain chunk summary/inventory from the runner. Honor G's finalization order and bounded failed-status rewrite. Do not print success after required-output failure, invent report locations, or serialize full corrections into production summaries. Exit 0 with normalized output when all LLM operations fail; exit nonzero for genuine fatal/output failures. Both reporting and debug disabled still produce the same transcript and CLI counts.

Validation: Extend actual subprocess fixtures for default file output, stdout plus selected report directory, reports disabled, debug-only, all-model failure, partial validation success, invalid input after initialization, and output failure before dispatch. Inspect created files and streams, not only mocked return values. Keep deeper batch/classifier matrices in their owning package tests.

Exit: The complete application implements the selected production behavior and has no obsolete report/retention path keeping debug enabled implicitly.

Stage 13 — Update Canonical Guidance, Examples, and Remove Obsolete Material

Work: Update docs/cli.md, config.md, operations.md, integration output/subprocess/ LLM contracts, internal pipeline/validators/LLM/diagnostics/overview guidance, README, and maintained examples/migration guidance. Architecture policy must distinguish recoverable model failures from fatal cancellation and acknowledge the intentional reporting-contract replacement. Describe implemented behavior only outside roadmap.

Document the three ordinary files, explicit disable/debug controls, current-directory stdout default, all-row matrices, partial successes, finalization failures, exit semantics, and retained debug placement. Remove references to --report-json, old retention modes, ordinary warning reports, and mandatory verbose artifacts. Resolve the obsolete retention item in runtime-behavior.md without changing unrelated input-timestamp policy. Correct older library-adoption links to completion evidence so they identify their historical plan instead of claiming this plan is complete. Leave future.md entries deferred. Do not turn ordinary zero-correction results into an operator warning.

Remove dead warning types/serialization helpers, obsolete full-report fixtures, and unused retention/report wiring identified by the graph and references. Keep meaningful domain tests and useful debug evidence; do not delete coverage just to hide a still-failing selected invariant.

Validation: Run maintained example/config inspection checks, affected-link and whitespace checks, and focused tests only where cleanup changes behavior. Confirm examples contain no credentials and configuration inspection produces no files.

Exit: Canonical docs and examples describe the final application, with no competing current contract or implicit legacy compatibility promise.

Stage 14 — Verify the Integrated Resilience Contract

Work: Audit every acceptance criterion in resilience.md against final code and retained tests. Add only missing integration coverage. In particular verify the following cross-component outcomes rather than relying solely on green leaf tests:

• Every proposal fails; separately proposals succeed but every required validation fails. Both return encoded normalized output, completed status, and accurate errors with no unsafe corrections. Subsequent stages still attempt their work.
• A failed batch between successful batches leaves valid survivors; later validation re-batches those survivors, and repeated module instances keep distinct identities.
• Fixed chunks survive substantial text growth; corrections never cross chunk ownership and final segment ordering/metadata are preserved.
• A recovered transport/repair operation plus filtering counts once; a repaired call with invalid decision indices counts as an error; a later failed operation does not erase earlier recoveries. Ordinary rejection is debug-only.
• Fatal cancellation reaches active and queued calls, joins workers, and produces honest partial/failed reports without a purported complete transcript.
• All four reporting/debug combinations have equal transcript/summary outcomes; inactive destinations are unused. No ordinary full payloads or raw secrets leak.
• Initial/final write failures produce the selected nonzero result, clean owned temporary files, and do not destroy a previously produced transcript. Existing outputs can be replaced intentionally; incompatible destination aliases fail before truncation.

Run go test ./..., go vet ./..., and focused race tests for runner, LLM, validators, reporting/output adapters, and subprocess packages. Build required Linux amd64/arm64 binaries using scripts/build-release.sh with a valid placeholder version and temporary output directory, without tagging or publishing. macOS arm64 is best effort and not a gate. No live provider, CI polling, or release-distribution review belongs in this implementation sprint.

Record actual evidence, remaining limitations, and completed stage states here. Only mark the feature implemented after the full final state is demonstrated. Commit the completed unit under the agent policy; do not push or release.

Exit: Resilience, partial-success safety, classifications, output lifecycle, and debug isolation are proven together in the assembled application.

Open Questions

None required to begin implementation. The earlier interface questions are resolved by A–H; internal helper names and file splits may vary without changing those contracts. If inspection reveals a concrete contradiction in the pinned libraries or an agreed requirement, document the question here with a recommended approach and reasoning plus at least one viable alternative and reasoning before changing that requirement. Do not reopen deferred features or add compatibility machinery merely to avoid the selected refactoring.