From f21930961716f79ee8e13f9270ccd9a1199b0322 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 25 Sep 2026 01:21:25 +0000 Subject: [PATCH] Plan the span deletion implementation --- docs/roadmap/implementation.md | 1163 ++++++++------------------------ docs/roadmap/span-deletion.md | 43 +- 2 files changed, 296 insertions(+), 910 deletions(-) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 56a263b..cc7d4f9 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,898 +1,303 @@ -# Implementation Plan: Resilient Processing and Focused Reports +# Implementation Plan: Consistent Span Deletion -**Last updated:** 2026-09-24 +**Last updated:** 2026-09-25 -**Status:** Implemented. All stages below are complete; completion notes record -the checks performed for each unit of work. +**Status:** Ready for implementation; no stages completed. +**Source baseline:** `a83a518` (feature roadmap); application baseline `0b6cd98`. +**Audience:** gpt-5.6-terra coding agent. -**Post-implementation revisions:** [Remediation R11–R14](remediation.md) supersedes -the original blanket transcript/input collision prohibition, inferred profile -filenames, regular-file-only transcript handling, and whole-document Markdown -golden requirements below. These stages and completion notes are historical; -[operations](../operations.md) describes current output behavior. +## Scope and Execution Rules -## Scope and Execution Instructions +Implement [Consistent Span Deletion](span-deletion.md). That document owns the +product intent, target semantics, and acceptance criteria. This document owns the +concrete decisions, ordered work, and validation. Execute Stage 1 through Stage 4 +in order. Record actual completion and checks beneath each stage when implemented. +Do not interpret this ready status as implemented behavior. -Implement the target in [resilience.md](resilience.md). User intent and product -policy belong there; the concrete choices and ordered steps below implement them. -Everything in [future.md](future.md) is excluded, especially domain-aware -regeneration, independent whole-pipeline traversal, and new empty-replacement or -segment-removal semantics. Keep the existing libraries, prompt text, retry/repair -budgets, chunk-size defaults, transcript formats, and domain acceptance rules. +Read [development guidance](../development.md) and all [agent](../policy/agents.md), +[architecture](../policy/architecture.md), [documentation](../policy/documentation.md), +[testing](../policy/testing.md), and [release](../policy/release.md) policies. +Use graph-first code discovery. Inspect the current workspace before editing; +retain unrelated user work. Commit coherent completed units with plain English +messages, following the agent policy. Do not add permission rounds between stages. -This plan targets a gpt-5.6-terra coding agent. Execute Stage 1 through Stage 14 in -order. Each stage is a bounded implementation prompt. If assigned the full plan, -continue without intermediate permission rounds. If assigned one stage, complete -its scope and report the next stage. Record completion and actual validation -beneath each stage; do not mark deferred integration checks as passing. +Aim directly at the final state. Do not add temporary compatibility aliases, +feature flags, parallel implementations, or scaffolding for intermediate commits. +Keep validators in their current order and preserve module scope, literal matching, +replacement policies, fixed chunks, required validation, failure isolation, +reporting/debug independence, and existing recovery budgets. Do not implement any +of the [deferred features](future.md), automatic whitespace cleanup, a new deletion +operation type, segment tombstones, or a second normalization pass. -Read `AGENTS.md`, [development guidance](../development.md), and every policy under -`docs/policy/`. Use graph-first discovery. Observe the [agent](../policy/agents.md), -[architecture](../policy/architecture.md), [testing](../policy/testing.md), -[documentation](../policy/documentation.md), and [release](../policy/release.md) -policies. The selected resilience behavior deliberately replaces the current -blanket runtime-failure cancellation and reporting contracts; update their -canonical descriptions as implementation lands. +This replaces the completed resilience implementation plan, which remains in Git +history at `a83a518:docs/roadmap/implementation.md`. Historical resilience decisions +and their remediation are not instructions to rebuild or reverse that work. +During final documentation updates, repair references that describe this filename +as the resilience plan; use current canonical references or the historical revision. +No release, version tag, push, upstream change, or new dependency is part of this +implementation sprint. -These stages partition one refactor, not supported intermediate releases. -Intermediate commits need not build or run. Do not add temporary adapters, -compatibility shims, duplicate pipelines, or fake success results to keep them -working. Use real final types, reconnect their consumers in subsequent stages, -and record unavailable checks honestly. Commit coherent stages with plain-English -messages after appropriate checks under the standing agent policy. +## Selected Implementation Decisions -Tests are deterministic, offline, and credential-independent. Prefer real internal -collaborators and controlled HTTP/transport/filesystem boundaries. Run focused -checks as their owning stage becomes testable, then the complete validation in -Stage 14. Do not duplicate a test matrix at every layer. No dependency upgrades, -upstream edits, paid calls, release publication, CI waiting, or unrelated timestamp -or normalization changes are authorized by this plan. +### Schema and library boundary -This replaces the completed library-adoption plan. Its historical completion -record is available with `git show cb405f6:docs/roadmap/implementation.md`. -Legacy references to that completion evidence do not imply this new plan is done. +Remove only `minLength: 1` from the `corrected_text` property of +`internal/framework/responseschema/assets/correction_set.v1.json`. Keep the required +field, string type, and all unrelated schema constraints. Empty and whitespace-only +strings are structurally valid; missing, null, and non-string replacements are not. +Nonblank source-span and resulting-segment rules stay in domain validation. -## Concrete Design Decisions +Keep `CorrectionSetKey`, the schema asset path, ID/name/version, and existing prompt +IDs/`v1` selections. This widens an internal model-response constraint without +changing the correction object shape or creating a second contract to maintain. +The schema hash and PromptKit's rendered/prepared hashes reflect changed content; +no hard-coded hash is a compatibility promise. Ordinary report schema versions, +transcript encoders, configuration, and CLI selections do not change. -### A. Settings, CLI, and Compatibility +PromptKit continues to validate the whole response and own structural repairs. +Do not introduce partial JSON salvage or Audita-side regeneration. Once the schema +accepts empty replacements, normal per-candidate domain filtering handles proposals +that would blank a segment. -Retain configuration version 2 and ConfigKit's file/overlay/visited-override -resolution. Add these settings: +### Validator identity -| Setting | Default | CLI override | Meaning | -| --- | --- | --- | --- | -| `reporting.enabled` | `true` | `--reporting[=true\|false]` | Write the ordinary three-file report bundle. | -| `reporting.stdout_dir` | `.` | `--report-dir DIR` | Report destination when the transcript goes to stdout; relative to invocation working directory. | -| `diagnostics.debug` | `false` | `--debug[=true\|false]` | Persist detailed diagnostic artifacts. | -| `diagnostics.work_dir` | Existing default | Existing `--work-dir` | Parent of unique debug directories; used only with debug enabled. | +Rename the existing validator consistently: -Register the new setting overrides through the same parser used by configuration -inspection commands. Only explicitly visited flags override resolved settings; -explicit false must override true in a file or overlay. Boolean overrides must -reach ConfigKit as booleans, not strings. `--reporting=false` is the sole reporting -disable spelling; do not add a parallel negative alias. - -For file transcript output, always place reports in `filepath.Dir(outputPath)`. -`reporting.stdout_dir`/`--report-dir` have no effect in that mode, or when reporting -is disabled; help must say so. With stdout output, use the selected stdout report -directory, defaulting to the invocation working directory. Retain stdout support. -Configuration inspection performs no output-directory creation or permission -probes. At process destination selection, require a nonblank stdout report directory -only for reporting-enabled stdout runs, and a nonblank debug parent only when debug -is enabled. Perform filesystem checks only for destinations actually used. - -Remove `--report-json`, `diagnostics.retention`, `--work-dir-retention`, their enum, -and old automatic retention logic. Debug directories are retained whenever debug -is enabled, on success or failure; otherwise none is created. No compatibility -aliases or legacy ordinary JSON report are required. Keep transcript/schema, -profile selection, and other existing settings unchanged. Do not reinterpret -`--output -`; existing omission of `--output` continues to select stdout. - -### B. Processing Identity and State - -Partition the normalized transcript exactly once in the runner, before entering -its module loop. Return that initial section inventory/summary to CLI reporting; -remove the separate pre-run CLI chunking pass. Preserve existing nil-config test -behavior and the chunker's handling of empty input and impossible constraints. - -Keep an ordered slice of chunk states with immutable section index, segment IDs, -and initial section metadata, plus mutable segment text. A stage finishes across -chunks before the next starts. Apply approved corrections within their owning -chunk using the existing replacement policy and application checks, then carry -that text to the next stage. Recombine only for the final transcript. Chunking -estimates in the inventory describe the initial partition; later validation -budgets still estimate the current payload, not stale initial text. - -Retain `ResolveModuleRunSpecs` instance names. Internal chunk, batch, and proposal -indices stay zero-based. Preserve the existing stage-wide proposal-index ordering: -chunk order followed by response proposal order, assigned independently of worker -completion order. A proposal's identity is its module instance plus proposal index, -with section index retained for attribution. Never join decisions by batch number -or provider call ID. Batch numbering is local to one validator in one chunk. - -Represent each stage/chunk outcome as `not_started`, `completed`, or `interrupted`, -with applied-correction count, compact operation outcomes, and optional debug -records. `completed` means the unit reached a terminal processing outcome, not -that all model operations succeeded. Keep known inventory for unattempted units -on fatal failure. A model error can coexist with applied corrections and recoveries. - -### C. Domain Outcomes and Model Failures - -Introduce a small `internal/framework/outcomes` package for processing-owned -operation references, execution counts, reason codes, and outcomes. It must not -import reporting, CLI, providers, or the runner. Do not create a generic event bus, -workflow framework, or persistence interface in the processing model. - -An operation reference contains module instance/key, section index, operation kind -(`proposal`, `validation`, or `validation_input`), optional validator name/order, -and optional batch index. Proposal operations occur once per work unit. An -oversized-input failure uses `validation_input` without a batch; consolidate such -items per validator invocation and record affected proposal indices. Ordinary -validation operations represent actual structured-client invocations, not retries. - -Compact operation outcomes carry execution counts when available, terminal success -or failure, safe failure reason, and filtering details keyed by proposal identity. -Add them to module/validator results and preserve them on fatal returns. Keep -provider attempts, transient retries, and structural repairs distinct. Use a -presence/availability flag for execution counts rather than inventing totals if -an external client supplies none. - -Keep typed Go errors for failures that require aborting the run. Extend -`structuredoutput.Failure` to distinguish a recoverable model-operation failure -from a fatal adapter/application failure and parent cancellation, with a stable -safe reason code and preserved `Unwrap` cause. Domain callers consume recoverable -failures into unsuccessful operation outcomes; they do not return them as fatal -runner errors. Bare/unclassified internal errors remain fatal. - -Use this classification order at the adapter boundary: - -| Condition | Result | +| Current | Target | | --- | --- | -| Parent processing context is canceled or expired | Fatal cancellation; preserve parent cause. | -| Invalid output target, unknown embedded schema, prepared-target mismatch, missing/broken prompt/profile/schema asset, invalid request/configuration, or nil result without error | Fatal invariant/setup failure. | -| Local operation/attempt deadline while the parent is live | Recoverable `operation_timeout`. | -| Capacity rejection | Recoverable `capacity_exceeded`. | -| Transport error | Recoverable `transport_failed`. | -| Malformed provider HTTP response/envelope | Recoverable `provider_response_invalid`. | -| Provider HTTP failure, including non-retryable statuses | Recoverable `provider_failed`, with numeric status when available. | -| Retry exhaustion | Recoverable failure using its known underlying category; retain exhaustion and attempt counts as detail. | -| Returned output remains structurally invalid after permitted repairs | Recoverable `invalid_output_exhausted`. | -| Structurally accepted output has a model value that cannot fit its Go domain target, e.g. integer overflow (`json.UnmarshalTypeError`) | Recoverable `invalid_output`. | -| Other decoding/invariant failures or unknown library errors | Fatal; do not convert every exception into a successful no-op. | - -Use `errors.Is`/`errors.As`, PromptKit's typed errors and terminal metadata, and the -phase of failure; never parse provider error text. Check known setup/invariant -causes before broad generation/validation wrappers. `ErrValidation` by itself is -not proof of repair exhaustion. A non-nil invalid result and known provider causes -must be distinguished from an embedded validation-contract defect. PromptKit -v0.11.0 exposes the required provider/transport/capacity and retry information; -no upstream change is part of this work. - -Do not expose raw causes, provider messages, URLs, or prompt fragments in ordinary -reports. Use fixed safe explanations plus structured codes/status/counts. Redact -all configured backend credentials, including unselected backends, at output. - -### D. Partial Validation and Safe Application - -Every submitted candidate still receives exactly one decision per validator. -Represent model-unavailable candidates as explicit non-approved decisions with an -operation-failure disposition; distinguish them from valid editorial rejection. -This preserves cardinality and prevents accidental approval while allowing other -batches to return their genuine decisions. Keep decision-level details in debug; -ordinary errors group their operation failures by work unit. - -On a recoverable batch failure, reject only that batch's candidates, record the -failed operation, and continue remaining batches and later validators for survivors. -Missing/duplicate/unknown decision indices invalidate that batch as a whole; do -not salvage a subset of an untrustworthy decision set or add semantic retries. -Reject oversized individual validation inputs without sending them, record a -`validation_input_too_large` failure, and still batch/process the other candidates. -A fatal error or parent cancellation instead stops the validator, retaining its -partial observations for diagnosis but never presenting unchecked candidates as -approved. - -Retain every required validator in its current order. Only candidates approved by -all required validators reach `ApplyProposals`. Application stays deterministic -and may reject proposals made stale by earlier accepted corrections. A failed -batch does not roll back successful unrelated corrections. Fatal runs do not -publish a completed transcript; already committed earlier-stage state remains -available for partial diagnostic evidence. - -Preserve current empty-replacement semantics: an empty replacement can remove a -span only if existing preview/application rules allow it and the segment remains -nonempty. Do not alter prompt/schema/deletion behavior under this plan. - -### E. Recoveries and Counts - -The adapter returns per-call execution counts in `structuredoutput.Response`, -including on error, using the terminal PromptKit observation. Callers determine -whether the final domain result was usable. The adapter alone must not declare -recovery merely because HTTP/schema handling succeeded. - -Create at most one recovery record per successful logical model operation. Its -`reasons` list can contain `output_repaired`, `service_recovered`, and, for a -proposal response, `proposals_filtered`. If repair succeeded but its final domain -decision set is unusable, that operation is an error, with its repair/retry counts -in error details; it is not also a recovery. Earlier successful operations retain -their recoveries even if later operations in the unit fail. - -Consolidate defective proposals filtered anywhere in that unit into the originating -proposal operation's recovery. Deduplicate affected proposal indices and retain -reason/count details; never add one recovery per rejected proposal or validator. -Recoveries from retries/repair and filtering of the same proposal response merge. - -Classify existing rejection reasons deliberately: - -- Filtering invalid target IDs, absent target/original text, empty original text, - invalid confidence, no-effect proposals, ambiguous original spans, and proposals - that would empty a segment contributes to `proposals_filtered`. -- Low confidence within the valid range, protected-term policy rejection, and - valid LLM editorial rejection are ordinary debug decisions, not recoveries. -- A proposal becoming stale or redundant because an earlier accepted proposal - changed the text is normal apply-time conflict resolution, not a model defect. - Compare applicability to the stage/chunk input snapshot when necessary; do not - relabel every apply-time skip as a recovery. -- Failure to validate because of size, invalid decision sets, or model availability - is an error, never a filtering recovery. Unexpected missing candidates or broken - cardinality in Audita's own assembled result is fatal internal failure. - -Counts are derived once from processing outcomes: - -- `work_units_total`: stage instances times initial chunks, zero for either empty set. -- `work_units_started`, `work_units_completed`, `work_units_interrupted`, - `work_units_not_started`: explicit states, not inferred from absent events. -- `work_units_with_errors`: distinct stage/chunk units with recoverable operation - failures; a partially successful unit counts once. -- `run_errors`: fatal errors outside that recoverable-unit count. A fatal error may - carry an origin operation reference without creating a duplicate unit error. -- `corrections_applied`: applied proposals, not replacement occurrences. -- `llm_operations`: actual structured-client invocations including pre-dispatch - failures; `proposal_responses`: structurally usable proposal responses. -- `recoveries`: unique successful operations with recovery reasons. Show counts - alongside `llm_operations`; for filtering-specific context show affected proposal - responses alongside `proposal_responses`. Do not create a single ambiguous - percentage or count repair generations as independent operations. - -Reason summaries count records containing each reason. Explain that one recovery -can have multiple reasons, so those category counts need not sum to its total. -Knownness/completeness of usage and existing indexed execution aggregation remain -intact. Full token/timing/attempt evidence belongs in debug, not an expanded -ordinary report. - -### F. Ordinary Output Schema and Markdown - -Use explicit Audita DTOs in `core/reporting`, separate from runner/domain types; -`framework/processreport` maps processing state to those DTOs. Render YAML with -`yaml.v3` and explicit YAML tags. Do not export raw PromptKit types or serialize -errors with their causes. Schema version is integer `1` for these new document -names; this is not the former JSON process-report schema. - -Both YAML documents have these envelope fields, in this order: - -| Field | Definition | -| --- | --- | -| `schema` | `audita-errors` or `audita-recoveries` | -| `version` | `1` | -| `run_id` | Same run ID across reports/debug; generated once independently of debug directory creation. | -| `status` | `in_progress`, `completed`, or `failed`; model errors alone still yield `completed`. | -| `started_at`, `completed_at` | UTC RFC3339 strings; completion is null initially. | -| `summary` | Counts named in E, in that order. Unknown pre-processing counts are null; completed empty counts are zero. | -| `errors` or `recoveries` | Always an ordered sequence, including `[]` when empty. | - -`errors` contains two record variants with explicit `scope`: - -- `work_unit`: `stage_index`, `module_key`, `module_instance`, `chunk_index`, - `start_segment_id`, `end_segment_id`, `corrections_applied`, and `failures`. - Each failure contains `operation`, optional `validator`/`batch_index`, `reason`, - fixed safe `message`, sorted `proposal_indices` when known, and optional execution - counts (`provider_attempts`, `transient_retries`, `structural_repairs`, - `retries_exhausted`, `http_status`). Do not duplicate transcript text. -- `run`: `phase`, `reason`, safe `message`, and optional originating stage/chunk/ - operation identity. Include primary fatal and secondary output-write failures - without pretending these are skipped LLM work units. - -Each recovery contains stage/module/chunk identity as above, `operation`, optional -`validator`/`batch_index`, ordered `reasons`, fixed safe `message`, execution counts -when known, and `filtered_proposals` with sorted indices and reason/count entries -when filtering occurred. One record can describe both repairs and filtering. - -Indices in YAML are zero-based. Display chunk and batch numbers as index + 1 in -Markdown/CLI, while retaining exact module-instance names. Order unit records by -configured stage order, then chunk index. Within units order proposal first, then -validators by chain order, input preparation before batches, then batch index. -Sort reason lists by code and proposal indices numerically. Render run errors in -lifecycle phase order with output paths sorted for same-phase secondary failures. -Use semantic identity for order; concurrent call IDs/timestamps do not determine -report row or event order. - -`report.md` always has a title/run ID, status, concise correction/coverage/error/ -recovery totals, input/output paths, and the two matrices. Stage columns follow -configured order; all chunks appear as rows, labeled with 1-based chunk number -and initial segment range. Error cells are 0 or 1 for completed units; recovery -cells are counts. Use `—` for not-started units and `n*` for interrupted units, -where n is the available count; include a fixed legend. Add totals and short -reason breakdowns and refer to the YAML files. An empty stage or chunk set gets -an explicit “No work units” sentence instead of an invalid empty table. - -No adaptive width checks, row omission, table splitting, transcript excerpts, or -correction lists. Escape Markdown paths/labels, especially pipes, newlines, and -backticks. Timing values may differ between runs; ordering/layout given the same -snapshot are deterministic. Redact strings before presentation without corrupting -numeric/boolean fields or resurrecting the old unredacted fallback. - -### G. Destination Lifecycle and Failure Semantics - -An output session in `core/diagnostics` owns filesystem state; CLI composes it. -The runner and renderers perform no filesystem operations. Report directories -must exist, matching the existing transcript-output parent requirement; do not -silently create ordinary output parents. Debug parents may still be created by -the debug-directory writer. - -After valid configuration/profile inspection, resolve paths and guard collisions -before any report truncation. Compare cleaned absolute paths, resolved existing -symlink targets/parents, and `os.SameFile` for existing files. Reject collisions -among the transcript output and the three enabled reports, and with transcript, -glossary, and known loaded configuration/profile source files. These checks -prevent self-overwrite, not hostile filesystem races; allow ordinary symlink use -and existing-output replacement. Return usage/configuration classification 2 for -conflicting destinations. Disabled report paths are not resolved, probed, created, -truncated, removed, or included in collision checks. - -For file transcript output, verify an existing target can be opened for writing -without `O_TRUNC`; close it. Probe creation in its parent with a standard temporary -file and remove that probe. For an absent target probe the parent without leaving -an empty purported transcript. Validate target type and collisions first; do not -truncate an input or existing transcript during preflight. Stdout cannot be -preflighted this way; its eventual write error is fatal. - -With reporting enabled, initialize all three final report paths with the same -`in_progress` run ID and empty event lists before input processing/model dispatch. -Return 1 immediately if initialization fails; do not invoke an LLM. Attempt to mark -already created reports failed when possible. A readable but invalid transcript -may then produce failed reports. Earlier configuration/profile failures may occur -before report initialization and need not produce a bundle. - -At successful processing completion, fully encode output before writing it. Use -standard temporary-file/write/close/rename replacement within the destination -parent for each final file, cleaning owned temporary files on failure. This is -per-file replacement, not a multi-file atomic transaction or durability guarantee; -no fsync protocol, locks, journal, or automatic rollback system. Preserve existing -file-output permission behavior where applicable and never erase a successfully -published transcript because a later report fails. - -Write the transcript first, then `errors.yml`, `recoveries.yml`, and `report.md` -last. Final write/close/rename errors are fatal even if preflight succeeded. -Attempt the other enabled report finalizations, collect safe secondary failures, -and make one bounded best-effort pass to write failed-status reports if finalization -failed. Never retry indefinitely or print the success completion message after a -required-output failure. Some files may retain an earlier/in-progress snapshot; -the nonzero exit and stderr identify incomplete finalization. - -On a processing fatal error or cancellation, do not publish a completed transcript; -finalize partial reports as failed where possible. Root cancellation must not -prevent local report cleanup/finalization after workers have joined. Uncatchable -termination may leave `in_progress`; no resume machinery is added. - -Debug remains optional evidence, not a condition for transcript success: failure -to create/write debug artifacts emits safe stderr diagnostics but does not change -processing or the exit status. Do not invent an error-bearing work unit for a debug -write failure. Required ordinary reports and transcript writes retain their fatal -semantics. Debug and reporting selections are independent. - -### H. Debug Artifacts and CLI Completion - -With debug enabled, create one unique child under `diagnostics.work_dir`, using the -run ID. Retain it on every outcome; no automatic cleanup/retention modes. Gate the -writers and directory creation, not merely cleanup. Empty diagnostics paths/nil -writers must propagate through module factory, proposals, and validators when off. - -Keep useful current artifacts under debug: source/normalized snapshots, invocation, -redacted effective settings, config origins, selected profiles, chunking/normalization -summaries, correction/decision ledger, indexed execution/utilization data, and model -interaction payloads. Ensure all proposed candidates, including those later rejected -or unavailable, can be inspected when debug is on. Keep JSON where suitable; do not -retain the old ordinary process report as a compatibility output. Remove obsolete -models/helpers/tests rather than keeping dead serialization paths. - -Full payload capture/persistence is not required for production summaries. Compact -operation state and necessary processing data remain available with both reporting -and debug off, so classification and CLI totals do not change. Use all configured -secrets for every writer, including debug snapshots, YAML, Markdown, and surfaced -messages. Do not assume metadata identifiers are safe merely because payloads -are absent. - -Successful CLI stderr begins with the selected style, using exact semantic counts: -`Completed: N corrections applied; E work units with errors; R recoveries.` -Report locations and the debug location follow only when actually created. There -are no ordinary warning messages. Suppressing files does not suppress this brief -summary or fatal errors. Stdout remains transcript-only; with `--output`, stdout -is empty. Exit 0 means processing and all enabled required outputs completed; -exit 1 covers runtime/cancellation/output failure; exit 2 retains usage/configuration -classification. Reporting disabled removes only its file requirements. - -## Stage 1 — Define Processing Outcomes and Contracts - -**Completed 2026-09-24.** Added processing-owned operation identities, execution -counts with availability, safe outcome/recovery reasons, work states, explicit -decision dispositions, and typed recoverable/fatal/canceled failure classes. The -proposal, validator, and runner result boundaries now carry operation outcomes -instead of warning fields. `go test ./internal/framework/outcomes -./internal/framework/structuredoutput` and `git diff --check` passed. Tests of -contracts, validators, and runner are temporarily unavailable because existing -validator/runner consumers still use the removed warning fields; Stages 3, 4, -and 6 reconnect those consumers. - -**Work:** Add the small outcome/reference/count types from B–E. Extend -`structuredoutput.Response` and `Failure`, module proposal results, validator -results/decisions, and runner output to carry actual processing outcomes. Replace -`StageWarning` at these boundaries with the final outcome model; do not alias the -old category to recoveries. Keep Go causes nonserializable and operation references -independent of report DTOs. Make invalid/unavailable decision disposition explicit. - -**Touchpoints:** `internal/framework/structuredoutput`, `contracts`, `validators/models.go`, -`runner/runner.go`, and new `framework/outcomes`. Keep dependencies acyclic; shared -outcome types cannot import their consumers. - -**Validation:** Focused tests for typed error matching and decision partition/cardinality -where behavior is introduced. Compile affected leaf packages. Record temporary -consumer compile failures for subsequent stages; do not add compatibility fields. - -**Exit:** Contracts can express success, recoverable model failure, fatal failure, -partial validation, and recovery detail without any filesystem/report dependency. - -## Stage 2 — Classify Model Failures and Return Execution Facts - -**Completed 2026-09-24.** The PromptKit adapter now classifies parent -cancellation, setup defects, local deadlines, capacity, transport, provider -envelopes/statuses, exhausted invalid output, and domain decode overflow with -typed safe failures. It returns terminal attempt/retry/repair/status counts on -successful and failed calls, while preserving observation hooks and typed causes. -`go test ./internal/framework/llm -count=1`, its race suite, and -`git diff --check` passed. Whole-repository compilation remains deferred while -the warning consumers are reconnected in later stages. - -**Work:** Implement C in `llm/adapter.go` using the pinned library's public types. -Capture terminal counts locally for each call and return them even on failure; -forward existing observation hooks without deriving recovery classifications in -the adapter. Preserve prepared-target checks, deadline-before-preparation, same -shared engine/capacity, typed causes, and safe messages. Handle parent cancellation -before local timeout classification. Keep metadata collection concurrency-safe. - -**Validation:** Extend adapter behavioral tests for repaired success/counts, -exhausted invalid output, provider/transport/envelope/capacity failure, local versus -parent deadline/cancellation, preparation defects, and decode overflow. Reuse -existing retry/capacity tests rather than retesting PromptKit. Assert no additional -outer attempts and no provider text in safe errors. - -**Exit:** Domain callers can distinguish skippable model work from fatal errors -without examining error strings; counts are available independently of debug. - -## Stage 3 — Convert Proposal Failures and Classify Safe Filtering - -**Completed 2026-09-24.** Proposal generation and the shared path used by all -four built-in modules now preserve one identified operation with terminal counts -on success, recoverable failure, and fatal return. Recoverable failures yield no -candidates; a valid empty response succeeds. A focused domain classifier separates -defective proposals from editorial rejection without changing deterministic -validator policy. Updated proposal, wrapper, and filtering tests are present. -`git diff --check` passed. Those tests and the existing four-module routing tests -cannot compile until the validator warning consumers are converted in Stage 4; -the attempted `go test` reports those exact stale-field errors. - -**Work:** Update `proposal_generation.GenerateCandidates` and all four module -wrappers to preserve operation outcomes. A recoverable proposal call returns zero -candidates plus a failed operation, not a warning or fatal Go error. A valid empty -response is successful. Fatal errors retain their partial call facts and propagate. -Carry successful-call counts until domain filtering can consolidate its recovery. - -Implement the explicit filtering/editorial distinction in E at the domain decision -boundary (a small reason classifier is sufficient). Preserve current deterministic -validators, thresholds, glossary protection, and empty-replacement behavior. Keep -payload diagnostics conditional on a supplied writer/path. - -**Validation:** Proposal integration tests distinguish malformed exhaustion from -fatal setup and valid empty output; deterministic filtering tests cover a defective -proposal versus low-confidence/protected-term rejection. Reuse current four-module -routing tests to prove wrappers preserve outcomes and identities. - -**Exit:** Proposal callers no longer abort for model failures, and filtering can be -attributed to the originating proposal response without treating normal rejection -as recovery. - -## Stage 4 — Preserve Successful Validation Batches - -**Completed 2026-09-24.** LLM validation now keeps successful batch decisions -around recoverable failed batches, emits explicit unavailable decisions for failed -or oversized inputs, and records one compact operation per client call plus one -consolidated oversized-input operation. Invalid decision sets fail their own -batch, and retry/repair recovery is recorded only after usable domain mapping. -Fatal failures retain earlier facts and stop later batches. Focused validator, -proposal, and four-module tests pass, as do validator/proposal race tests. -`git diff --check` passed. Runner compilation is still deferred because its old -warning fields are reconnected in Stage 6. - -**Work:** Refactor `LLMBackedValidator.Validate`, input preparation, and response -mapping per D. Collect all successful batch decisions; on a recoverable failure -add non-approved unavailable decisions for that batch and continue. Keep invalid -index sets batch-scoped, oversized candidates excluded individually, and errors -separate from valid negative decisions. Record final domain outcome before declaring -service/repair recovery. Stop promptly on fatal errors/parent cancellation and -retain completed operation facts. Do not synthesize approval for remaining items. - -**Validation:** One controlled multi-batch workflow proves successes before and -after a failed batch survive, and a later validator receives only survivors under -new batch numbering. Cover oversized individual input, missing/duplicate/unknown -indices, repaired-but-domain-invalid output, ordinary rejection, and fatal stop. -Use the existing estimator seam to force batches without brittle default budgets. - -**Exit:** Every normally returned result has exactly one decision for each submitted -candidate; failed model batches do not discard unrelated decisions or bypass later -required validation. - -## Stage 5 — Fix Chunk Boundaries for the Entire Run - -**Completed 2026-09-24.** The runner now partitions once before the module loop, -returns the initial section inventory, applies approved corrections only inside -their identified sections, carries current text forward, and recombines the -transcript after processing. Tests cover substantial text growth, stable segment -membership, empty work, and impossible chunk constraints. `go test -./internal/core/chunking` and `git diff --check` passed. Runner tests remain -unavailable because warning-field consumers still fail compilation; Stage 6 -replaces them with operation outcomes. - -**Work:** Move the runner's chunking call before its stage loop. Own fixed chunk -states and initial inventory in runner output. Execute each module using the current -text of those same chunks, and apply fully approved candidates only within their -chunk. Keep stage barriers, initial chunk metadata, deterministic proposal indexing, -and final ordered recombination. Remove the redundant CLI chunk computation when -wiring its consumer, not by adding an interim second inventory. Preserve within-stage -proposal/validation concurrency and shared capacity; no new scheduler is required. - -**Validation:** A multi-stage test deliberately grows text enough that old re-chunking -would alter membership; assert stable chunk/segment identity and next-stage current -text. Cover repeated modules, target-section selection, empty transcript/pipeline, -and application ordering with unchanged output-schema behavior. - -**Exit:** The normalized transcript is partitioned once and domain processing does -not rely on report tables to maintain chunk state. - -## Stage 6 — Integrate Partial Outcomes and Fatal Cancellation - -**Completed 2026-09-24.** The runner now preserves per-section operation facts, -continues after recoverable model failures, applies only candidates approved by -every required validator, and retains completed observations on fatal cancellation. -It tracks explicit work-unit states and merges defective-proposal filtering with -an existing proposal recovery. Runner tests cover total proposal failure, mixed -validation outcomes, later modules, multiple failures in one unit, merged recovery, -and fatal cancellation. `go test -race ./internal/framework/runner -./internal/framework/llm` and `git diff --check` passed. Integrated reporting -checks remain deferred until the reporting stages. - -**Work:** Update `collectSectionProposals`, `runModulePipeline`, and -`validateSectionCandidates` to consume the outcomes from Stages 3–4. Recoverable -failures never enter shared fatal cancellation. Preserve candidates through every -required validator by proposal index and apply successful survivors even when the -unit has errors. Never return unchecked `eligible` candidates as approved on an -error path. Propagate completed outcomes even if another worker cancels the stage; -do not drop buffered terminal evidence merely because the context is canceled. - -Retain synchronized first-fatal-error ownership, immediate peer cancellation, worker -joining, and deterministic collection. Track not-started/interrupted/completed units -and the counts in E. On fatal stage exit do not apply incomplete validation results; -retain prior committed stages and honest partial observations. Aggregate proposal -filtering once, merge its reasons with any proposal retry/repair recovery, and -retain earlier recoveries when later operations fail. - -**Validation:** Runner tests prove total model failure returns normalized output; -mixed batches apply only fully validated candidates; later stages still execute; -multiple operation failures count one affected unit; repaired-plus-filtered proposals -count once. Adapt the existing cancellation regression to a genuine fatal error: -model failures now continue, while fatal failure cancels active/pending calls, -retains the originating cause, joins workers, and preserves observations. Run -focused runner/runtime race tests when those packages are testable. - -**Exit:** Processing fulfills the resilience contract independently of file output. - -## Stage 7 — Add Reporting and Debug Settings and CLI Selection - -**Completed 2026-09-24.** Version-2 settings now resolve reporting and debug -independently, with typed visited boolean overrides and destination selection -only for enabled reports. The old JSON-report and retention flags/settings were -removed; CLI entry points now carry a context from interrupt/SIGTERM handling. -ConfigKit tests and focused CLI parser tests cover defaults, overlay false -overrides, bare booleans, destination selection, and removed flags. `go test -./internal/core/config`, `go test internal/cli/config_input.go -internal/cli/config_input_test.go`, and `git diff --check` passed. Whole CLI tests -remain unavailable until Stage 8 replaces the report builder's warning fields. - -**Work:** Implement A in `core/config/config.go`, `cli/config_input.go`, invocation -state, and usage text. Add boolean flags with visited-value overrides and update -safe effective-setting/provenance output. Remove the old external JSON report flag -and retention selection, including tests for their former semantics. Keep path -selection in CLI composition, not domain configuration resolution. Add context -plumbing through `cli.Run(ctx, args, stdout, stderr)` and process/runtime calls; -update direct callers and tests. In `cmd/audita`, use `signal.NotifyContext` for -interrupt/SIGTERM, with a small run function that defers stop before `os.Exit`. -Configuration help/inspection still creates no output files. - -**Validation:** Parser/ConfigKit tests cover defaults, overlay true overridden by -CLI false, reporting/debug independence, stdout directory selection, ignored report -directory in file/disabled mode, removed flags/keys, and side-effect-free inspection. -No execution scheduler or output files are added in this stage. - -**Exit:** Final settings/CLI contract is resolved without legacy aliases or accidental -filesystem requirements for disabled features. - -## Stage 8 — Map Outcomes to Versioned YAML Reports - -**Completed 2026-09-24.** The old ordinary JSON report DTO was replaced with -versioned errors/recoveries YAML DTOs and one processing snapshot. Mapping derives -nullable summary counts, grouped recoverable work-unit failures, run failures, -combined recovery reasons, and deterministic semantic order without raw causes or -transcript payloads. Tests cover partial and empty work, multiple failures, -initialization unknowns, ordered run errors, safe redaction, and YAML scalar -types. `go test ./internal/core/reporting ./internal/framework/processreport` -and `git diff --check` passed. CLI and diagnostics consumers of the removed JSON -schema remain to be reconnected in later stages. - -**Work:** Replace ordinary report DTOs and assembly in `core/reporting` and -`framework/processreport` with F. Map actual runner state once into an immutable -report snapshot consumed by all presentations. Preserve debug-specific ledger and -indexed execution mapping only where useful; remove superseded ordinary JSON -models rather than maintaining two authoritative schemas. Implement YAML rendering -for initial, completed, and failed snapshots, explicit empty lists/null unavailable -counts, safe messages, stable ordering, and the two error record scopes. - -Use existing shared redaction at the serialization boundary. Never marshal a raw -error cause or depend on typed decode of already-redacted timestamps/identifiers. -YAML numbers and booleans must retain their types even for numeric credentials. - -**Validation:** Report tests use partial units, multiple underlying failures, -combined recovery reasons, no-work input, run-level failure, unavailable initialization -counts, and deterministic ordering despite reordered observations. Parse rendered -YAML to verify types/counts and redaction, including unselected/escaped credentials. -Preserve indexed aggregation and usage semantics in retained debug builders. - -**Exit:** Both YAML documents truthfully describe the same processing snapshot with -no filesystem effects or verbose payload leakage. - -## Stage 9 — Render the Fixed Markdown and CLI Summaries - -**Completed 2026-09-24.** Markdown now renders the fixed error and recovery -matrices, all known chunks and configured stage columns, state legends, totals, -short reason counts, safe paths, and YAML references. The CLI completion line -uses the same snapshot counts. A representative golden fixture (updated only -with `UPDATE_REPORT_GOLDEN=1`) and semantic tests cover repeated stages, -partial/unattempted cells, empty work, escaped labels, redaction, and YAML count -agreement. `go test ./internal/core/reporting ./internal/framework/processreport` -and `git diff --check` passed. CLI stream/file wiring remains for Stage 12. - -**Work:** Render F/H from the Stage 8 snapshot. Produce both full matrices, totals, -fixed legends, short reason breakdowns, and safe file references. Display 1-based -chunk labels without changing machine identity. Include interrupted and unattempted -states and the no-work sentence. Provide a small shared summary formatter for CLI -counts; CLI stream/file wiring remains Stage 12. No width heuristics, percentage -thresholds, row omission, or extra export formats. - -**Validation:** One representative Markdown golden fixture with explicit update -opt-in can protect the chosen layout; supplement with semantic checks for repeated -stages, partial/error-and-recovery cells, empty work, and escaped labels. Assert CLI -counts agree with YAML and that secrets never enter Markdown. Avoid snapshotting -wall-clock values or every incidental sentence. - -**Exit:** Human-facing output is concise, deterministic, and generated from the same -truth as the structured output. - -## Stage 10 — Implement Required Output Initialization and Finalization - -**Completed 2026-09-24.** A filesystem output session now checks active -destination collisions (including symlink and hardlink aliases), preflights the -transcript without truncation, initializes all enabled report files, and replaces -final files through owned temporary files. It attempts every report after a -finalization error and makes one failed-status pass while retaining any published -transcript. Temporary-directory tests cover collisions, unavailable parents, -disabled reporting, stdout mode, overwrite/permissions, initialization failure, -and partial finalization. `go test ./internal/core/diagnostics` and -`git diff --check` passed. CLI sequencing remains for Stage 12. - -**Work:** Add the output-session filesystem adapter described in G. Resolve active -paths, protect collisions before truncation, preflight transcript writes without -destroying existing content, initialize all enabled reports, and publish final files -using standard per-file replacement. Return structured primary/secondary errors -for CLI reporting. Track only files/probes created by this invocation for cleanup; -do not delete previous report files when reporting is disabled. Use ordinary file -permissions and no multi-file transaction or crash-recovery framework. - -**Validation:** Real temporary-directory tests cover initialization failure before -a fake model is invoked, output/report/input collisions including existing aliases, -existing transcript preservation, overwrite behavior, absent parent, disabled reports -with an unusable report directory, stdout mode, final transcript/report failures, -and partial failed-status finalization. For permission failures use a narrow injected -file-operation seam when tests may run as root; do not rely solely on chmod. - -**Exit:** Required output failures are detected early where possible and always -reported at finalization without erasing a completed transcript. - -## Stage 11 — Make Debug Persistence Explicit and Independent - -**Completed 2026-09-24.** Debug directory creation and all current debug writer -paths are gated by `diagnostics.debug`; failures are reported safely to stderr -without aborting processing. The retained directory uses the supplied run ID, -old retention modes are gone, raw and normalized snapshots are redacted, and the -candidate ledger includes proposals not applied after filtering or interruption. -Tests cover explicit candidate capture, retained debug evidence, secret redaction, -and writer behavior. `go test ./internal/core/redaction -./internal/core/diagnostics ./internal/framework/runner -./internal/framework/processreport` and `git diff --check` passed. CLI package -compilation is deferred to Stage 12's replacement of the old report lifecycle. - -**Work:** Refactor `core/diagnostics` run-directory creation and all CLI/module/ -proposal/validator writer wiring for H. Only debug creates the unique retained -run directory. Gate snapshots, interaction files, ledger, and full execution data -at the write source. Keep required ordinary reporting in its separate session. -Remove automatic retention and the old `never` semantics rather than leaving a -disabled cleanup path. Ensure debug contains all proposals and decisions needed -to diagnose filtering/unavailable results, not just applied changes. - -All debug writers use the shared configured-secret redactor, including original -input snapshots; a raw source copy must not bypass it. Keep safe stderr reporting -of optional-debug write failures separate from model incidents and fatal output -errors. Debug write failures must not change correction decisions or abort the run. - -**Validation:** Tests prove no debug directory/payload files with debug off, useful -retained evidence with debug on, redaction across snapshots/interactions, and no -outcome/exit changes caused by optional debug-write failure. Existing diagnostic -writer tests are retained or replaced according to behavior, not old filenames. - -**Exit:** Debug-off never means “write and later delete”; reporting-off/debug-on and -reporting-on/debug-off both work without coupling their destinations. - -## Stage 12 — Compose the Final CLI Run Lifecycle - -**Completed 2026-09-24.** CLI processing now inspects settings and profiles, -preflights and initializes required outputs before input/model work, creates -debug evidence only when selected, runs the fixed-chunk runner with caller -context, and finalizes transcript/reports in order. Failed processing and output -writes produce failed-status reports from available partial state; successful -stderr begins with the shared correction/error/recovery counts. The duplicate -CLI chunking pass and old JSON report flow are removed. Subprocess tests cover -file/stdout destinations, disabled reports, debug-only runs and optional debug -failure, total model failure, partial validation and later work, invalid input -after initialization, and pre-dispatch output failures. Focused package tests -and `git diff --check` passed; `go test ./...` passed before the last small -integration assertions were added and will be rerun in Stage 14. - -**Work:** Replace the current monolithic process/report/retention flow with the -final composition: load/inspect settings, resolve and initialize required outputs, -optionally initialize debug, load/normalize input, run fixed-chunk processing, -render/finalize, and emit completion/failure output. Thread the caller context all -the way to runtime/runner. Keep phase labels and root versus local cancellation -clear. After joining workers, use available partial state to finalize failed reports -without treating cancellation as an instruction to abandon local cleanup. - -Remove the old pre-run chunking pass and obtain chunk summary/inventory from the -runner. Honor G's finalization order and bounded failed-status rewrite. Do not -print success after required-output failure, invent report locations, or serialize -full corrections into production summaries. Exit 0 with normalized output when -all LLM operations fail; exit nonzero for genuine fatal/output failures. Both -reporting and debug disabled still produce the same transcript and CLI counts. - -**Validation:** Extend actual subprocess fixtures for default file output, stdout -plus selected report directory, reports disabled, debug-only, all-model failure, -partial validation success, invalid input after initialization, and output failure -before dispatch. Inspect created files and streams, not only mocked return values. -Keep deeper batch/classifier matrices in their owning package tests. - -**Exit:** The complete application implements the selected production behavior and -has no obsolete report/retention path keeping debug enabled implicitly. - -## Stage 13 — Update Canonical Guidance, Examples, and Remove Obsolete Material - -**Completed 2026-09-24.** Canonical CLI, configuration, operations, integration, -internal, architecture, README, example, and migration guidance now describes -three ordinary reports, independent debug selection, partial outcomes, fixed -matrices, and finalization/exit behavior. The obsolete retention follow-up is -resolved while timestamp policy remains open. Historical library-adoption -links point to the prior plan's completion evidence. The unreferenced warning -package and retention-only helper/tests were removed. Maintained config and -profile inspection examples passed offline, relative file and heading links -resolved, and `go test ./internal/framework/processreport -./internal/core/config ./internal/cli` plus `git diff --check` passed. - -**Work:** Update `docs/cli.md`, `config.md`, `operations.md`, integration output/subprocess/ -LLM contracts, internal pipeline/validators/LLM/diagnostics/overview guidance, README, -and maintained examples/migration guidance. Architecture policy must distinguish -recoverable model failures from fatal cancellation and acknowledge the intentional -reporting-contract replacement. Describe implemented behavior only outside roadmap. - -Document the three ordinary files, explicit disable/debug controls, current-directory -stdout default, all-row matrices, partial successes, finalization failures, exit -semantics, and retained debug placement. Remove references to `--report-json`, old -retention modes, ordinary warning reports, and mandatory verbose artifacts. Resolve -the obsolete retention item in `runtime-behavior.md` without changing unrelated -input-timestamp policy. Correct older library-adoption links to completion evidence -so they identify their historical plan instead of claiming this plan is complete. -Leave `future.md` entries deferred. Do not turn ordinary zero-correction results -into an operator warning. - -Remove dead warning types/serialization helpers, obsolete full-report fixtures, -and unused retention/report wiring identified by the graph and references. Keep -meaningful domain tests and useful debug evidence; do not delete coverage just to -hide a still-failing selected invariant. - -**Validation:** Run maintained example/config inspection checks, affected-link and -whitespace checks, and focused tests only where cleanup changes behavior. Confirm -examples contain no credentials and configuration inspection produces no files. - -**Exit:** Canonical docs and examples describe the final application, with no -competing current contract or implicit legacy compatibility promise. - -## Stage 14 — Verify the Integrated Resilience Contract - -**Completed 2026-09-24.** Audited the target acceptance criteria against the -assembled CLI, runner, validator, reporting, and output-session tests. Added -controlled-provider subprocess coverage for all-proposal and all-validation -failure across repeated modules, all reporting/debug selections, a repaired and -filtered proposal response followed by a later failure, a repaired response with -invalid validation decisions, and signal cancellation with failed reports and no -transcript. Added an atomic-replacement cleanup check for a failed rename and -asserted ordinary reports omit transcript text and the configured secret. Existing -runner/validator tests cover survivor re-batching, fixed chunk membership and -cross-chunk rejection after text growth, segment metadata and ordering, and -active/queued cancellation; output-session tests cover initial -and final write failures, intentional replacement, and destination alias checks. - -**Final validation:** `go test ./...` and `go vet ./...` passed. Focused -`go test -race` passed for runner, LLM, validators, process-report assembly, -reporting, diagnostics/output, and the subprocess package. The release build -script produced valid Linux amd64 and arm64 binaries using `v0.0.0` in a temporary -directory; a separate macOS arm64 cross-build also passed. No live provider or -released artifact was used; provider behavior is covered by deterministic local -HTTP fixtures. Builds were cross-compiled and not executed on target platforms. - -**Work:** Audit every acceptance criterion in `resilience.md` against final code and -retained tests. Add only missing integration coverage. In particular verify the -following cross-component outcomes rather than relying solely on green leaf tests: - -- Every proposal fails; separately proposals succeed but every required validation - fails. Both return encoded normalized output, completed status, and accurate - errors with no unsafe corrections. Subsequent stages still attempt their work. -- A failed batch between successful batches leaves valid survivors; later validation - re-batches those survivors, and repeated module instances keep distinct identities. -- Fixed chunks survive substantial text growth; corrections never cross chunk - ownership and final segment ordering/metadata are preserved. -- A recovered transport/repair operation plus filtering counts once; a repaired call - with invalid decision indices counts as an error; a later failed operation does - not erase earlier recoveries. Ordinary rejection is debug-only. -- Fatal cancellation reaches active and queued calls, joins workers, and produces - honest partial/failed reports without a purported complete transcript. -- All four reporting/debug combinations have equal transcript/summary outcomes; - inactive destinations are unused. No ordinary full payloads or raw secrets leak. -- Initial/final write failures produce the selected nonzero result, clean owned - temporary files, and do not destroy a previously produced transcript. Existing - outputs can be replaced intentionally; incompatible destination aliases fail - before truncation. - -Run `go test ./...`, `go vet ./...`, and focused race tests for runner, LLM, -validators, reporting/output adapters, and subprocess packages. Build required -Linux amd64/arm64 binaries using `scripts/build-release.sh` with a valid placeholder -version and temporary output directory, without tagging or publishing. macOS arm64 -is best effort and not a gate. No live provider, CI polling, or release-distribution -review belongs in this implementation sprint. - -Record actual evidence, remaining limitations, and completed stage states here. -Only mark the feature implemented after the full final state is demonstrated. -Commit the completed unit under the agent policy; do not push or release. - -**Exit:** Resilience, partial-success safety, classifications, output lifecycle, -and debug isolation are proven together in the assembled application. +| `non_empty_corrected_text` key and package directory | `non_empty_resulting_segment` | +| `KeyNonEmptyCorrectedText` in registry and metadata | `KeyNonEmptyResultingSegment` | +| `NonEmptyCorrectionValidator` framework type | `NonEmptyResultingSegmentValidator` | + +Retain the `empty_resulting_segment` rejection/skip reason, deterministic execution +class, chain position, and preview-based behavior. Do not retain an old-key alias. +This is an intentional validator identity change in diagnostic/decision records; +document it, including any affected ordinary filtered-proposal metadata. It is not +a new operator-selectable setting. Historical release notes remain untouched. + +### Prompt and payload semantics + +All four proposal prompts must explain that an empty replacement deletes the +identified span, within the module's existing task, and that a correction leaving +a blank segment should be omitted. Prompts must not ask the model to invent padding +or replacement content to avoid an empty result. + +Use a spoken-word example deleting `"uh "` from `"uh hello"` with `""`, producing +`"hello"`. A grammar example may delete the uniquely occurring extra comma from +`"Hello, world."` when appropriate; the representation is illustrative, not a +claim that the edit is always editorially justified. Explain that selection of +surrounding whitespace/punctuation must produce the intended literal result. +Keep the existing uniqueness requirement and module responsibilities. + +Update the three active validator prompt definitions (`editorial_review`, +`meaning_reversal_review`, `spoken_form_plausibility`) to interpret an empty +replacement in the context of the full before/after segment. Empty syntax alone +must not decide approval. Meaning-reversal review remains a narrow check for +antonyms/reversals; do not turn it into a general semantic-equivalence validator. +Spoken-form plausibility retains its existing phonetic/mistranscription remit. +The registered `grammar_review` and `spoken_word_review` definitions should receive +the same deletion interpretation as their editorial-review counterpart, without +adding them to active chains or deleting them as unrelated cleanup. + +`BuildLLMValidationRequest` already supplies `corrected_text`, +`original_segment_text`, and `corrected_segment_text`; the JSON fields are not +omitted when empty. Reuse this representation. Verify it behaviorally rather than +adding a new payload shape, synthetic sentinel, or special deletion flag. + +### Debug representation and outcomes + +`CorrectionLedgerEntry` currently uses strings with `omitempty` for both replacement +fields, losing explicit empty values. Change `ProposedCorrectedText` and +`AppliedCorrectedText` to `*string` with their existing JSON names and `omitempty`: + +- Every candidate-backed entry has a non-nil proposed value, including `""`. +- Applied entries also have a non-nil applied value, including `""`. +- Rejected, skipped, and interrupted entries omit the applied value. +- Module-level failure entries without a candidate omit both values. + +Construct the pointers directly when mapping the corresponding records; use the +same simple convention across all dispositions. Do not build custom marshaling +or a generic optional-value framework. These values remain replacement spans, +not full resulting-segment text. Preserve redaction and the rest of the ledger. + +Keep existing operation accounting. A valid approved deletion is an ordinary +applied correction. A candidate that would blank the segment is filtered with the +existing reason and contributes to the proposal operation's consolidated filtering +recovery. Ordinary editorial/protected-term rejection does not become a recovery. +Existing apply-time conflict handling and attribution remain unchanged. Structural +repairs required for unrelated defects still count normally. + +## Stage 1 — Align Validator Identity and Confirm Domain Semantics + +**Work** + +- Rename the validator package/type/key described above. Update `New()` wiring, + `internal/validators/registry.go`, `chains.go`, metadata, framework validators, + consumers, and tests in one coherent change. Use graph discovery for symbols and + string searches for serialized keys. Do not globally rewrite historical docs. +- Preserve existing `CorrectionProposal.Validate`, `PreviewProposalForSegment`, + `ApplyProposals`, and deterministic validation behavior wherever already correct. + Verify that there is no additional nonempty-replacement restriction downstream. +- Extend the existing proposal and validator tests only for missing meaningful + coverage: empty replacement leaving a nonblank segment, whitespace-only resulting + text rejection, literal spacing, and apply-time reevaluation after an earlier edit. + For the last case, use segment `"a b"` and ordered deletions `"a" -> ""`, + `"b" -> ""`: each is previewable initially, but after the first edit the second + must be skipped as `empty_resulting_segment`. The surviving text is `" b"`; + no automatic trim or segment deletion is expected. +- Ensure protected-term validation still rejects removal of a protected term with + an empty replacement. Prefer one case in its existing test table. Retain existing + metadata-preservation tests rather than duplicating all metadata assertions. +- Update `docs/internal/validators.md` for the renamed implemented validator and + explain its resulting-segment responsibility; do not yet claim schema alignment. + +**Validation** + +Run `go test ./internal/framework/proposals ./internal/framework/validators ./internal/validators/... ./internal/framework/runner ./internal/framework/processreport`. +Existing registry/chain tests must use the new identity and preserve order/class. +Inspect remaining old-name references, retaining only deliberately historical ones. + +**End state:** The accurately named domain validator and application preserve the +agreed behavior. No schema or retry workaround has been introduced. + +## Stage 2 — Align Response Schema, Prompts, and LLM Validation Inputs + +**Work** + +- Apply the selected schema relaxation and prompt changes. Keep unrelated prompt + instructions, hardening, variables, and response associations intact. Review the + rendered definitions, not just YAML source, using existing PromptKit inspection + and prompt tests. +- In `internal/framework/responseschema/registry_test.go`, retain the check that + reported hashes equal hashes of the actual embedded bytes. Remove fixed digest + expectations that merely freeze asset content; do not replace them with a new + golden digest for this change. +- Extend the existing real-adapter tests in `internal/framework/llm/adapter_test.go` + with a bounded table for replacement values: empty string, spaces, ordinary text, + missing field, null, and a non-string value. Use the embedded schema and local + HTTP fixture. Give accepted cases a positive repair allowance and assert one + provider generation, zero repairs, and preservation of the actual string. For + invalid cases, use zero repair attempts to prove rejection without redundant + model calls. Existing tests already own successful repair/exhaustion behavior. +- Extend the validation-request tests in + `internal/framework/validators/llm_validators_test.go` to marshal a real request + for deleting `"uh "` from `"uh hello"`. Assert that JSON contains the explicit + empty replacement plus full before/after text, correct identity, and no immediate + rejection. A field-presence assertion is required; decoding into a plain string + alone cannot distinguish omitted from empty. +- Update canonical prompt/module guidance for the implemented semantics. Do not + add exact-prose prompt snapshots or assertions for every new sentence; existing + parsing/rendering/schema-association tests plus editorial review are sufficient. + +**Validation** + +Run `go test ./internal/framework/responseschema ./internal/prompts ./internal/framework/llm ./internal/framework/proposal_generation ./internal/framework/validators ./internal/modules/...`. +Confirm missing/non-string replacements still fail structurally and domain-invalid +but schema-valid proposals still reach ordinary filtering without an outer retry. + +**End state:** A literal empty replacement can cross the real library/schema +boundary, and the model instructions and validator inputs describe its meaning. + +## Stage 3 — Preserve Empty Replacements in Debug Evidence + +**Work** + +- Implement the pointer representation selected above in + `internal/framework/processreport/correction_ledger.go`, updating every mapping + branch and affected test consumer. Leave other debug structures unchanged where + their string fields already serialize empty values explicitly. +- Extend `correction_ledger_test.go` with a small serialized-disposition table: + applied deletion, rejected deletion, skipped deletion, interrupted deletion, + and a module failure without a candidate. Decode to a map or raw JSON fields to + check presence, `""` values, and absence of the applied value on unapplied entries. + Preserve existing decision classification/order tests using the renamed key. +- Update `docs/integrations/output-files.md` to explain the two debug replacement + fields and the deliberate validator identity change. Keep ordinary schemas and + report counts unchanged. Update the diagnostics/reporting internal reference + for the implemented mapping, linking to the public artifact contract. + +**Validation** + +Run `go test ./internal/framework/processreport ./internal/core/diagnostics ./internal/cli ./cmd/audita`. +Tests must inspect serialized output, not just non-nil in-memory values. Retain +existing configured-secret redaction and debug-off persistence coverage. + +**End state:** Debug evidence distinguishes a proposed/applied deletion from an +absent proposal or an edit that was never applied. + +## Stage 4 — Verify the Integrated Contract and Finish Documentation + +**Work** + +Extend `cmd/audita/main_integration_test.go` with representative offline subprocess +cases using its controlled HTTP provider, real configuration/profiles, embedded +schemas, built-in module/validator chains, CLI, and output/report/debug writers. +Keep fixture additions local and small; do not build a new integration framework. +Use explicit fixture sizing so relevant validator candidates fit one batch, and +use stable response indices. Do not rely on production chunk/token defaults or +on concurrent request order to choose responses. + +Cover these distinct outcomes: + +- **Clean deletion plus another correction:** A spoken-word stage proposes + `"uh " -> ""` in `"uh hello"` and an independent repetition collapse. Controlled + validators approve both. Assert corrected transcript, two applied corrections, + zero model errors/recoveries, zero structural repairs with a positive repair + allowance, and explicit empty proposed/applied fields in the debug ledger. +- **Partial success with filtering:** Add a proposal to delete all text from a + separate filler-only segment, preserving that separate segment through fixture + normalization (use sufficient time separation or explicit normalization settings). + Assert that this candidate is filtered with `empty_resulting_segment`, the other + corrections still apply, the blanked segment is instead retained unchanged, and + the proposal response contributes one consolidated filtering recovery and no + work-unit error. The renamed deterministic validator must appear in evidence. +- **Ordinary negative decision:** Return a schema-valid deletion proposal, then a + valid negative decision from meaning-reversal review after earlier checks approve + it. Assert unchanged affected text, no applied deletion, no error/recovery solely + from the rejection, and a rejected debug entry. The controlled decision proves + the chain respects a negative decision, not that an arbitrary real model will + make the same editorial judgment. + +Reuse existing YAML/Markdown/CLI count assertions rather than snapshotting entire +reports. Verify that the clean case's report totals agree with the transcript and +ledger. Compare resulting metadata to the normalized input, since initial +normalization may legitimately merge segments or reassign IDs. Confirm retained +section membership/order in the existing runner boundary tests; add a targeted +assertion only if that property is not already covered for the exercised path. +Do not repeat all cases for all modules, encoders, debug/report combinations, or +platforms: those unchanged mechanisms already have coverage. + +Finish the documentation and status work: + +- Review current module, validator, prompt, diagnostics, and output references for + consistency. Add a concise note to `docs/migrations/v2.md` describing the renamed + validator identity and explicit empty debug replacement fields; no alias or + migration tool is required. No release note/version is selected in this sprint. +- Review links from other roadmaps to `implementation.md`. References to completed + resilience stages should point to their historical plan at the recorded revision + or current canonical behavior, not silently identify this plan as that work. + Update those references only as needed; do not rewrite historical findings. +- Mark this plan and `span-deletion.md` implemented only after the criteria pass. + Keep the baseline description historical and retain future.md's exploratory + whole-segment-removal status. Record concise evidence under completed stages. + +**Final validation** + +Run `go test ./...` once after the integrated changes. Run `git diff --check` and +validate affected documentation links/examples. Focused stage checks need not be +repeated solely as ceremony. Additional race/platform/live-provider checks are +not automatic gates for this feature; run additional checks only for a concrete +risk introduced by actual changes. This is not a release invocation. + +**End state:** The feature roadmap's acceptance criteria are evidenced by real +schema/adapter tests, domain/application tests, serialized debug tests, and +representative complete CLI workflows. The workspace is committed and the docs +describe implemented behavior without claiming whole-segment removal. ## Open Questions -None required to begin implementation. The earlier interface questions are resolved -by A–H; internal helper names and file splits may vary without changing those -contracts. If inspection reveals a concrete contradiction in the pinned libraries -or an agreed requirement, document the question here with a recommended approach -and reasoning plus at least one viable alternative and reasoning before changing -that requirement. Do not reopen deferred features or add compatibility machinery -merely to avoid the selected refactoring. +None requiring a product decision remain. Validator names, schema/prompt identity, +debug representation, module scope, test boundaries, and deferred behavior are +selected above. Ordinary implementation details may be resolved idiomatically +within these decisions; a newly discovered conflict with the agreed contract +should be recorded rather than resolved by silently strengthening restrictions. diff --git a/docs/roadmap/span-deletion.md b/docs/roadmap/span-deletion.md index 3023be5..e614d8b 100644 --- a/docs/roadmap/span-deletion.md +++ b/docs/roadmap/span-deletion.md @@ -2,10 +2,9 @@ **Last updated:** 2026-09-25 -**Status:** Accepted feature direction; not implemented. This document defines -the target behavior and work required. A staged implementation plan will be -prepared separately; the existing implementation plan covers the completed -resilience work. +**Status:** Accepted target; not implemented. This document owns the desired +behavior and user intent. [implementation.md](implementation.md) owns the concrete +steps, selected implementation details, and completion evidence. ## Purpose and User Intent @@ -109,6 +108,11 @@ proposal prompts. Validators must understand the before/after effect of deletion and assess it under their existing criteria. A correct negative editorial decision remains ordinary processing, not a defect that should be regenerated until approved. +The resulting-segment validator's name and documented identity should describe +that responsibility accurately, rather than imply that the replacement string +itself must be nonempty. Preserve the existing blank-result failure reason and +validation order while making that deliberate identity change. + ## Outcomes and Reporting A legitimate deletion is an ordinary correction. Empty replacement alone must @@ -127,28 +131,6 @@ purpose; no new report schema, incident category, or deletion-specific counter i needed. Unrelated structural defects may still exhaust PromptKit repair and produce an error for the operation, as today. -## Work Required - -This is a description of the change surface, not an ordered implementation plan. - -- Relax the shared correction schema's replacement-length constraint while - retaining its required string field and the other structural checks. -- Align proposal prompts, relevant validator prompts, and validation payloads with - the selected semantics. Reuse the existing source/result preview behavior. -- Rename the misleading resulting-segment validator and associated package, - registry, chain, metadata, test, and documentation references consistently. - Choose a name that states the resulting-segment requirement. Treat any exposed - validator identity change deliberately; do not rename the existing failure - reason merely to match the new validator name. -- Review preview and application paths for consistency, including successive edits - to the same segment. Preserve existing safeguards without adding new deletion - abstractions, configuration controls, or independent retry loops. -- Replace tests that enforce nonempty replacement strings with meaningful contract - tests, and add representative integration coverage through the real schema, - PromptKit adapter, validation chain, application, and outcome mapping. -- Update canonical module, validator, prompt, and affected output documentation - when implementation lands. Keep current behavior references accurate until then. - ## Acceptance Criteria - A controlled valid response containing an empty replacement reaches Audita's @@ -176,7 +158,7 @@ no live-provider or paid evaluation is a prerequisite. Representative production examples can inform prompt quality without expanding this into an evaluation framework or guaranteeing that models always follow the instructions. -## Deferred Work and Planning Boundary +## Deferred Work [Whole-segment removal](future.md#whole-segment-removal) is only potential future work and first requires deciding whether its benefit warrants implementation. @@ -184,7 +166,6 @@ Domain-aware regeneration and independent whole-pipeline traversal also remain in [future.md](future.md). This feature changes neither retry budgets nor the Audita/PromptKit ownership boundary and requires no upstream library feature. -The product choices above are settled. The next planning pass should select -concrete names and integration-test placement and produce a staged implementation -plan. No implementation stages or application changes are introduced by this -roadmap-writing pass. +Implementation sequencing and concrete names belong to the +[implementation plan](implementation.md). The target remains consistent deletion +of spans within surviving segments, with no new operator controls or report types.