From 8ffbf41b89874fd30fd504a61db4b6f150ac92b6 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Thu, 24 Sep 2026 16:50:21 +0000 Subject: [PATCH] Plan resilient processing and focused run reports --- docs/roadmap/implementation.md | 1247 +++++++++++++++++--------------- docs/roadmap/resilience.md | 54 +- 2 files changed, 697 insertions(+), 604 deletions(-) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index bc43b5a..9245651 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,642 +1,739 @@ -# Implementation Plan: Adopt ConfigKit and PromptKit +# Implementation Plan: Resilient Processing and Focused Reports **Last updated:** 2026-09-24 -**Status:** Implemented. All twelve stages are complete. This replaced -the completed testing-overlay prototype plan; its results remain in the -[boundary findings](library-boundary-findings.md#selected-overlay-extension-2026-09-24). +**Status:** Ready for implementation; all stages below are pending. Planning only: +no implementation is claimed by this document. -## Scope and Working Instructions +## Scope and Execution Instructions -Implement the final configuration and execution model in the -[combined feature roadmap](library-adoption.md) and [configuration roadmap](configkit.md). -The [selected boundary](library-boundary-findings.md#configuration-and-profile-contract), -[migration inventory](library-boundary-findings.md#migration-inventory), and -[failure mapping](library-boundary-findings.md#execution-and-failure-mapping) -are the detailed behavioral inputs. Do not repeat the prototype or reopen settled -choices without a concrete contradiction in the pinned libraries. +Implement the target in [resilience.md](resilience.md). User intent and product +policy belong there; the concrete choices and ordered steps below implement them. +Everything in [future.md](future.md) is excluded, especially domain-aware +regeneration, independent whole-pipeline traversal, and new empty-replacement or +segment-removal semantics. Keep the existing libraries, prompt text, retry/repair +budgets, chunk-size defaults, transcript formats, and domain acceptance rules. -This plan is for a gpt-5.6-terra coding agent. Execute Stage 1 through Stage 12 in -order. Each stage is one bounded implementation prompt. When assigned the whole -plan, continue between stages without a permission round. When assigned one stage, -finish that stage's scope and report the next stage. Mark completed stages with a -short result and actual validation evidence; do not call a pending integration -check passed. +This plan targets a gpt-5.6-terra coding agent. Execute Stage 1 through Stage 14 in +order. Each stage is a bounded implementation prompt. If assigned the full plan, +continue without intermediate permission rounds. If assigned one stage, complete +its scope and report the next stage. Record completion and actual validation +beneath each stage; do not mark deferred integration checks as passing. -Read `AGENTS.md`, [development guidance](../development.md), and all documents under -`docs/policy/`. Use graph-first code discovery. Follow the -[agent](../policy/agents.md), [architecture](../policy/architecture.md), -[testing](../policy/testing.md), [documentation](../policy/documentation.md), and -[release](../policy/release.md) policies. The intentional dependency, configuration, -and scheduler changes here supersede the corresponding old implementation choices; -update canonical descriptions as the new behavior lands. +Read `AGENTS.md`, [development guidance](../development.md), and every policy under +`docs/policy/`. Use graph-first discovery. Observe the [agent](../policy/agents.md), +[architecture](../policy/architecture.md), [testing](../policy/testing.md), +[documentation](../policy/documentation.md), and [release](../policy/release.md) +policies. The selected resilience behavior deliberately replaces the current +blanket runtime-failure cancellation and reporting contracts; update their +canonical descriptions as implementation lands. -The stages divide one refactor, not supported intermediate releases. Intermediate -commits may fail to build or run. Do not write temporary adapters, compatibility -shims, duplicate execution paths, stubs returning invented successes, or an interim -processing gate. Remove superseded code when replacing it. Run focused checks -where the affected packages are testable; record checks deferred because later -stages must reconnect callers. Such a completed stage is still work in progress -on the overall refactor. Commit coherent stages with plain-English messages under -the standing policy. Do not spend a stage making an intermediate application work. +These stages partition one refactor, not supported intermediate releases. +Intermediate commits need not build or run. Do not add temporary adapters, +compatibility shims, duplicate pipelines, or fake success results to keep them +working. Use real final types, reconnect their consumers in subsequent stages, +and record unavailable checks honestly. Commit coherent stages with plain-English +messages after appropriate checks under the standing agent policy. -Keep tests deterministic, offline, and credential-independent. Reuse prototype -scenarios at their production boundary rather than copy the nested application or -retest all library internals. Preserve domain tests unless the selected contract -actually changes. Update each affected canonical reference alongside its owning -stage; Stage 11 finishes the cross-document migration guide and examples. +Tests are deterministic, offline, and credential-independent. Prefer real internal +collaborators and controlled HTTP/transport/filesystem boundaries. Run focused +checks as their owning stage becomes testable, then the complete validation in +Stage 14. Do not duplicate a test matrix at every layer. No dependency upgrades, +upstream edits, paid calls, release publication, CI waiting, or unrelated timestamp +or normalization changes are authorized by this plan. -No upstream edits, paid calls, push, tag, publication, retention-policy change, -timestamp fix, or unrelated pipeline redesign is included. Release compliance -and publication remain a later release unit. Completion here means both libraries -are adopted, obsolete machinery is removed, and the final application is verified. +This replaces the completed library-adoption plan. Its historical completion +record is available with `git show cb405f6:docs/roadmap/implementation.md`. +Legacy references to that completion evidence do not imply this new plan is done. -## Implementation Decisions +## Concrete Design Decisions -These decisions close the remaining application-design details; background and -user-facing policy belong to the feature roadmaps. +### A. Settings, CLI, and Compatibility -- Pin the published modules used by the verified prototype: ConfigKit v0.4.0 and - PromptKit v0.11.0. Set the root minimum to Go 1.25.5. Keep yaml.v3 while glossary - parsing uses it. No sibling replacements or dependency on Audita's own old release. -- Keep application loading in `internal/core/config`, embedded prompt definitions - in `internal/prompts`, schemas in `internal/framework/responseschema`, and the - PromptKit integration in `internal/framework/llm`. Use one engine per process - invocation, shared by both role-bound clients; neither modules nor validators - construct engines. The CLI composes these pieces. Avoid a generic service layer. -- Retain the `CompleteStructured(ctx, request, out)` boundary, using Audita-owned - prompt ID/version, string variables, schema identity, and diagnostic context. - Replace the old rendered-message/model request fields. Model/profile/credential - selection belongs to the role-bound adapter, not individual callers. Consolidate - the equivalent validator types onto the framework contract where needed, without - an import cycle. No PromptKit types in correction algorithms or ledger records. -- Store PromptKit definitions for all nine existing prompt IDs with their existing - `v1` selection. Replace the custom `hardening` template function with an - application-supplied `HardeningPolicy` string variable sourced from the one - embedded hardening fragment. The adapter supplies that reserved variable after - caller variables. PromptKit renders all messages; do not retain an Audita template - engine or introduce generated prompt files requiring a second build step. -- Use one shared configuration-input parser for `process`, `config validate`, and - `config print-effective`. All three accept `--config`, `--overlay`, and the - retained setting overrides in the inventory, including the two profile flags. - Only `process` accepts transcript/glossary/output/report paths. Only explicitly - visited setting flags become ConfigKit overrides. Do not add retry/capacity/key - flags beyond the agreed inventory. -- `config validate` requires an explicit nonempty config path. The other two - commands use the selected discovery order. All validate the resolved settings - and both selected profiles without provider calls, including an empty module - pipeline. Help succeeds before file discovery or secret/profile loading. -- `config print-effective` adds `--view settings|profiles|explain`, defaulting to - `settings`. Settings uses ConfigKit `WriteEffective` YAML. Profiles emits a safe - Audita-owned YAML summary for the proposal and validation roles. Explain emits - a YAML sequence of field path, source, and derivation records from ConfigKit's - recorded resolution, with no resolved secret values. Unknown views are usage - errors. This avoids mixing multiple formats into the default output or adding - another command tree. Profiles includes selected ID, backend, model, temperature, - max tokens, top-p, per-attempt timeout, service tier, and reasoning effort when - present; omit arbitrary extra provider parameters and credentials. It is a - selected-controls summary, not a dump of every profile input. -- Keep CLI exit classes 0/1/2 and transcript/report stream behavior. Invalid user - settings/profiles are configuration errors (2); broken embedded assets and - runtime/provider failures are runtime errors (1). Serialization/write failures - remain runtime errors. Do not classify malformed HTTP envelopes as repairable - model output. -- Publish process reports as `audita-process-report` schema `v2`. Preserve existing - domain fields and ledger ordering, add a top-level `llm_summary`, and replace - obsolete scheduler utilization fields as specified in Stage 10. Transcript - encoder versions stay unchanged. No legacy report mode is needed. +Retain configuration version 2 and ConfigKit's file/overlay/visited-override +resolution. Add these settings: -## Stage 1: Replace the Configuration Model and Loader +| Setting | Default | CLI override | Meaning | +| --- | --- | --- | --- | +| `reporting.enabled` | `true` | `--reporting[=true\|false]` | Write the ordinary three-file report bundle. | +| `reporting.stdout_dir` | `.` | `--report-dir DIR` | Report destination when the transcript goes to stdout; relative to invocation working directory. | +| `diagnostics.debug` | `false` | `--debug[=true\|false]` | Persist detailed diagnostic artifacts. | +| `diagnostics.work_dir` | Existing default | Existing `--work-dir` | Parent of unique debug directories; used only with debug enabled. | -**Status:** Complete (2026-09-24). Replaced the version-1 loader with the ConfigKit -schema-2 model, selected overlays, typed secret references, final role inheritance, -and retained domain validation. Pinned both libraries and Go 1.25.5, and aligned -the release build image. `go test ./internal/core/config` and `git diff --check` -passed. `go test ./...` currently fails in old CLI, validator, and diagnostic -callers of removed fields; those connections are assigned to later stages. -**Depends on:** the completed prototype only. -**Owns:** root `go.mod`/`go.sum`, `internal/core/config`, and config tests. +Register the new setting overrides through the same parser used by configuration +inspection commands. Only explicitly visited flags override resolved settings; +explicit false must override true in a file or overlay. Boolean overrides must +reach ConfigKit as booleans, not strings. `--reporting=false` is the sole reporting +disable spelling; do not add a parallel negative alias. -1. Pin both modules and the Go minimum above. Align build/CI toolchain declarations - with that minimum; preserve the existing platform set and release gates. -2. Replace version-1 YAML/env merge and duration coercion with one ConfigKit - definition covering every retained field in the migration inventory. Use the - prototype's types/lifecycle as evidence, then add the full domain configuration. - Remove legacy environment loaders and alias resolution; do not keep two loaders. -3. Implement defaults, presence-aware backend defaults, final validation-profile - inheritance, and recorded derivations. Keep `chunking.target_sections` optional, - explicit empty module lists empty, and module order/repeats intact. Validate - domain constraints after overlay and explicit override resolution. Preserve - numeric normalization seconds; operation timeout is a duration string. -4. Use typed optional secret references. Resolve every surviving reference, including - unselected backends; missing/empty references fail, replacement resolves only the - replacement, and null permits unauthenticated use. Enable selected overlays; - leave imports and secret-directory bootstrap disabled. -5. Expose the ConfigKit result to CLI composition for effective output and provenance; - convert retained settings into existing domain inputs without transporting the - library result through correction code. Update `docs/config.md`'s implemented - loader/schema description, clearly marking unfinished CLI integration if needed. +For file transcript output, always place reports in `filepath.Dir(outputPath)`. +`reporting.stdout_dir`/`--report-dir` have no effect in that mode, or when reporting +is disabled; help must say so. With stdout output, use the selected stdout report +directory, defaulting to the invocation working directory. Retain stdout support. +Configuration inspection performs no output-directory creation or permission +probes. At process destination selection, require a nonblank stdout report directory +only for reporting-enabled stdout runs, and a nonblank debug parent only when debug +is enabled. Perform filesystem checks only for destinations actually used. -**Acceptance and validation:** focused config tests cover strict version/fields, -selected defaults, omission/null/zero, final role inheritance, empty/repeated -modules, overlay/override precedence, secret replacement/null/missing, and retained -cross-field rules. Old schema fails with migration guidance. General environment -variables have no implicit effect. Run `go test ./internal/core/config` when this -package is independently buildable; record caller compile failures for later stages. -Do not duplicate ConfigKit's exhaustive YAML/merge tests. +Remove `--report-json`, `diagnostics.retention`, `--work-dir-retention`, their enum, +and old automatic retention logic. Debug directories are retained whenever debug +is enabled, on success or failure; otherwise none is created. No compatibility +aliases or legacy ordinary JSON report are required. Keep transcript/schema, +profile selection, and other existing settings unchanged. Do not reinterpret +`--output -`; existing omission of `--output` continues to select stdout. -## Stage 2: Replace CLI Configuration Selection and Overrides +### B. Processing Identity and State -**Status:** Complete (2026-09-24). Added one parser for all three commands, -strict file discovery, optional overlay selection, and visited-only setting -overrides; removed retired flags and legacy process flag extraction. Updated CLI -and configuration references. Isolated parser/loading tests passed with -`go test internal/cli/config_input.go internal/cli/config_input_test.go`, and -`git diff --check` passed. `go test ./internal/cli` remains unbuildable because -the old runtime, validators, and diagnostics still use removed configuration -fields; assembled command checks remain deferred to later stages. -**Depends on:** Stage 1. -**Owns:** `internal/cli/run.go`, `process_flags.go`, configuration command parsing, -and corresponding CLI/subprocess tests. +Partition the normalized transcript exactly once in the runner, before entering +its module loop. Return that initial section inventory/summary to CLI reporting; +remove the separate pre-run CLI chunking pass. Preserve existing nil-config test +behavior and the chunker's handling of empty input and impossible constraints. -1. Implement the shared configuration-input parser described above. Preserve the - existing supported positional-argument arrangement for `process`, required - glossary, output/report behavior, and root/subcommand help. Remove obsolete - LLM/model/key/timeout/retry/concurrency flags and aliases outright. -2. Select exactly one file: explicit `--config`, then - `/usr/local/etc/audita/config.yml`, then `/etc/audita/config.yml`. Missing system - candidates are skipped; selected inaccessible files and explicit empty/missing - paths fail. With no file, report a configuration error; do not select a provider. - Remove `AUDITA_CONFIG`. Test discovery using an injected path list/filesystem - seam rather than modifying the host's system configuration. -3. Pass invocation overlay selection to ConfigKit and only supplied setting flags - to its overrides. `--modules ''` must clear the pipeline; explicit numeric zero - reaches validation rather than being treated as absent. Relative profile bases - derive from the selected main file, never the overlay directory. -4. Wire the parsed result into the final process/config command entry points. - Profile-dependent command completion belongs to Stage 4; do not emulate the old - runtime to make the process command work now. Update `docs/cli.md` for the final - input contract as parsing lands. +Keep an ordered slice of chunk states with immutable section index, segment IDs, +and initial section metadata, plus mutable segment text. A stage finishes across +chunks before the next starts. Apply approved corrections within their owning +chunk using the existing replacement policy and application checks, then carry +that text to the next stage. Recombine only for the final transcript. Chunking +estimates in the inventory describe the initial partition; later validation +budgets still estimate the current payload, not stale initial text. -**Acceptance and validation:** parser/loading tests prove help without usable -configuration, discovery/error cases, explicit-only overrides, retained domain -flags, removed flag rejection, and ignored legacy environment values. Table-drive -shared cases; retain representative subprocess checks rather than triplicate every -parser test across commands. Config commands reject positional arguments and -process-only flags. Defer assembled command checks until Stage 8 if imports still -prevent compilation. +Retain `ResolveModuleRunSpecs` instance names. Internal chunk, batch, and proposal +indices stay zero-based. Preserve the existing stage-wide proposal-index ordering: +chunk order followed by response proposal order, assigned independently of worker +completion order. A proposal's identity is its module instance plus proposal index, +with section index retained for attribution. Never join decisions by batch number +or provider call ID. Batch numbering is local to one validator in one chunk. -## Stage 3: Migrate Embedded Prompt Assets and Call Contracts +Represent each stage/chunk outcome as `not_started`, `completed`, or `interrupted`, +with applied-correction count, compact operation outcomes, and optional debug +records. `completed` means the unit reached a terminal processing outcome, not +that all model operations succeeded. Keep known inventory for unattempted units +on fatal failure. A model error can coexist with applied corrections and recoveries. -**Status:** Complete (2026-09-24). All nine embedded prompts are PromptKit -definitions with the shared hardening variable, existing `v1` IDs, and canonical -JSON Schema associations. Replaced the template renderer and its hash metadata; -the structured-call request now carries prompt, schema, variables, and attribution -through one framework contract. `go test ./internal/prompts -./internal/framework/responseschema ./internal/framework/structuredoutput` and -`git diff --check` passed. `go test ./...` still fails at old module, validator, -runtime, and diagnostic callers awaiting their assigned migrations. -**Depends on:** Stage 1; execute after Stage 2. -**Owns:** `internal/prompts`, `internal/framework/responseschema`, -`internal/framework/contracts`, validator contract aliases, and prompt tests. +### C. Domain Outcomes and Model Failures -1. Convert all four module and five validator prompts listed in - [the prompt registry](../internal/prompts.md) to PromptKit definitions. Preserve - instructions, variables, glossary/description context, prompt IDs, and the two - existing JSON response schemas. Embed assets once and expose their filesystem - to the runtime. Do not copy the prototype's duplicate schema directory. -2. Keep output contracts JSON Schema based, but supply the full effective output - contract and repair count on each runtime request. Do not create separate - operator-facing repair defaults per prompt. Use the hardening variable decision - above to retain a single hardening source. -3. Change the structured-call request to prompt identity, variables, response schema - identity, and diagnostic context. Keep decoded output and typed Audita metadata - as the return boundary. Include role/module instance/section/batch/validator - context where available for attribution; do not derive identities from messages. -4. Replace the old parser/renderer and hash assumptions. Keep small metadata and - schema lookup helpers where Audita needs them. Update the prompt reference. +Introduce a small `internal/framework/outcomes` package for processing-owned +operation references, execution counts, reason codes, and outcomes. It must not +import reporting, CLI, providers, or the runner. Do not create a generic event bus, +workflow framework, or persistence interface in the processing model. -**Acceptance and validation:** a table covering all nine definitions prepares -representative variable sets through PromptKit with controlled profile fixtures; -checks include hardening, section/glossary/description data, missing variables, -correct schema association, and rendered content preserving the previous task -instructions. Use reviewed expectations, not a retained legacy renderer or old -Audita module dependency. Existing schema/domain tests remain meaningful. Callers -are deliberately migrated in Stages 6–7. +An operation reference contains module instance/key, section index, operation kind +(`proposal`, `validation`, or `validation_input`), optional validator name/order, +and optional batch index. Proposal operations occur once per work unit. An +oversized-input failure uses `validation_input` without a batch; consolidate such +items per validator invocation and record affected proposal indices. Ordinary +validation operations represent actual structured-client invocations, not retries. -## Stage 4: Assemble Profiles and Finish Offline Inspection +Compact operation outcomes carry execution counts when available, terminal success +or failure, safe failure reason, and filtering details keyed by proposal identity. +Add them to module/validator results and preserve them on fatal returns. Keep +provider attempts, transient retries, and structural repairs distinct. Use a +presence/availability flag for execution counts rather than inventing totals if +an external client supplies none. -**Status:** Complete (2026-09-24). Added one engine and profile inspection path -for both roles, requiring registered matching backends and backend-owned -credentials. Config commands and process select that path; settings, safe selected -profiles, and provenance are exposed as separate YAML views with secret redaction. -Focused offline checks passed with `go test internal/framework/llm/runtime.go -internal/framework/llm/safe_view.go internal/framework/llm/runtime_inspection_test.go` -and `go test internal/cli/config_input.go internal/cli/config_input_test.go`; -`git diff --check` passed. Assembled command tests remain deferred until the -old caller packages are reconnected in later stages. -**Depends on:** Stages 1–3. -**Owns:** engine/profile assembly in `internal/framework/llm`, CLI config commands, -and inspection tests. +Keep typed Go errors for failures that require aborting the run. Extend +`structuredoutput.Failure` to distinguish a recoverable model-operation failure +from a fatal adapter/application failure and parent cancellation, with a stable +safe reason code and preserved `Unwrap` cause. Domain callers consume recoverable +failures into unsuccessful operation outcomes; they do not return them as fatal +runner errors. Bare/unclassified internal errors remain fatal. -1. Construct one PromptKit engine with the embedded assets, complete profile files, - and registered application backends. Reuse the prototype's supported filesystem, - backend, retry-config, and inspection APIs. Accept an HTTP client at the external - boundary for tests; production uses the library transport. -2. Resolve `profiles_dir` against the main config file. Inspect both selected roles, - require a registered backend, reject endpoint-only bypass and profile credential - lookup, and compare endpoints after trimming surrounding whitespace and trailing - slashes, as in the prototype. Do not invent profile inheritance/catalog wiring - or reject unrelated unselected profile content beyond library behavior. -3. Complete `config validate` and the three effective-output views. Settings, - profiles, and explanations share the same load/assembly path used by process. - Keep the `config is valid` success message. Redact configured secret values in - any surfaced library error; do not stringify raw configuration/profile objects. -4. Add the internal safe-view helpers later used for snapshots. Document the view - interface and its limits in `docs/cli.md` and `docs/config.md`. +Use this classification order at the adapter boundary: -**Acceptance and validation:** offline command tests prove all views and validation -agree for base/testing/explicit-profile selections; main-file-relative and absolute -profile paths work; missing profiles, backend mismatch, competing credentials, -missing surviving secrets, and invalid views fail with the selected exit class. -Use a transport that fails the test if called. Inspect outputs/errors for synthetic -secret absence and overlay/CLI provenance, including inherited validation selection. +| Condition | Result | +| --- | --- | +| Parent processing context is canceled or expired | Fatal cancellation; preserve parent cause. | +| Invalid output target, unknown embedded schema, prepared-target mismatch, missing/broken prompt/profile/schema asset, invalid request/configuration, or nil result without error | Fatal invariant/setup failure. | +| Local operation/attempt deadline while the parent is live | Recoverable `operation_timeout`. | +| Capacity rejection | Recoverable `capacity_exceeded`. | +| Transport error | Recoverable `transport_failed`. | +| Malformed provider HTTP response/envelope | Recoverable `provider_response_invalid`. | +| Provider HTTP failure, including non-retryable statuses | Recoverable `provider_failed`, with numeric status when available. | +| Retry exhaustion | Recoverable failure using its known underlying category; retain exhaustion and attempt counts as detail. | +| Returned output remains structurally invalid after permitted repairs | Recoverable `invalid_output_exhausted`. | +| Structurally accepted output has a model value that cannot fit its Go domain target, e.g. integer overflow (`json.UnmarshalTypeError`) | Recoverable `invalid_output`. | +| Other decoding/invariant failures or unknown library errors | Fatal; do not convert every exception into a successful no-op. | -## Stage 5: Implement the Final PromptKit Structured-Call Adapter +Use `errors.Is`/`errors.As`, PromptKit's typed errors and terminal metadata, and the +phase of failure; never parse provider error text. Check known setup/invariant +causes before broad generation/validation wrappers. `ErrValidation` by itself is +not proof of repair exhaustion. A non-nil invalid result and known provider causes +must be distinguished from an embedded validation-contract defect. PromptKit +v0.11.0 exposes the required provider/transport/capacity and retry information; +no upstream change is part of this work. -**Status:** Complete (2026-09-24). Added role-bound prepared execution with -target verification, validated-only decoding, typed safe failures, protected -prepared/final-content snapshots, and logical-call and execution-event hooks. -Removed the old direct HTTP client and role resolver; updated LLM integration -documentation. `go test ./internal/framework/llm` and -`go test ./internal/framework/structuredoutput` passed, including controlled -HTTP repair/retry, target-drift, transport/protocol, and cancellation cases; -`git diff --check` passed. Caller integration remains for Stages 6–8. -**Depends on:** Stages 3–4. -**Owns:** `internal/framework/llm`, the typed failure boundary in -`internal/framework/structuredoutput`, and adapter tests. +Do not expose raw causes, provider messages, URLs, or prompt fragments in ordinary +reports. Use fixed safe explanations plus structured codes/status/counts. Redact +all configured backend credentials, including unselected backends, at output. -1. Implement role-bound clients over the shared engine. Start the operation context - and Audita logical-call record before preparation; inherit earlier caller deadlines. - Select the role profile and backend secret, supply reserved hardening variables, - and pass an explicit full schema/repair contract. Use `PrepareExecution`, its - `Details`, then `RunPrepared` so diagnostic messages/hashes describe the actual - execution snapshot. Defer handle discard; do not call `Prepare` and then `Run` - with an independently reloaded profile. Before dispatch, verify the prepared - target still matches the role's registered backend/endpoint and has no competing - credential lookup. A changed target fails safely rather than sending a backend - key to a different target. This is the final diagnostic/security boundary, not - an intermediate adapter. -2. Check both the returned error and `result.Validation.IsValid`: exhausted repair - can return a result with failed validation and no error. Decode only validated - JSON into the requested Audita output. Use the selected failure table: - exhausted output repair becomes the Audita malformed-output classification; provider/transport/protocol/retry/admission/deadline failures - remain runtime failures. Preserve typed causes for classification and safe - messages for users. Explicit zero retries must not become retries-exhausted. -3. Preserve prepared messages/hashes and available final content and profile/model - metadata through Audita-owned types. Provide a per-call observer hook and logical - start/finish record bound to Audita diagnostic context for Stage 9; do not - create another retry loop, scheduler, provider envelope parser, or schema engine. -4. Remove the old OpenAI-compatible client and its role config resolution. Retain - only applicable redaction and diagnostics helpers. Update the LLM integration - reference to delegate wire/retry mechanics to the pinned library. +### D. Partial Validation and Safe Application -**Acceptance and validation:** controlled HTTP tests cover successful decode, -positive repair and exhaustion for both schema types, 429/5xx retry recovery, -explicit zero, exhausted positive budget, transport/malformed-envelope failure, -combined retry-plus-repair bounds, preparation failure without dispatch, exact -prepared request metadata, target-change rejection, and operation cancellation -during a long retry wait. Assert classifications and meaningful counts, not timing-sensitive sleeps -or complete error strings. Reuse the prototype sequences; no need to reproduce -PromptKit's entire status-code matrix. +Every submitted candidate still receives exactly one decision per validator. +Represent model-unavailable candidates as explicit non-approved decisions with an +operation-failure disposition; distinguish them from valid editorial rejection. +This preserves cardinality and prevents accidental approval while allowing other +batches to return their genuine decisions. Keep decision-level details in debug; +ordinary errors group their operation failures by work unit. -## Stage 6: Migrate All Proposal Callers +On a recoverable batch failure, reject only that batch's candidates, record the +failed operation, and continue remaining batches and later validators for survivors. +Missing/duplicate/unknown decision indices invalidate that batch as a whole; do +not salvage a subset of an untrustworthy decision set or add semantic retries. +Reject oversized individual validation inputs without sending them, record a +`validation_input_too_large` failure, and still batch/process the other candidates. +A fatal error or parent cancellation instead stops the validator, retaining its +partial observations for diagnosis but never presenting unchecked candidates as +approved. -**Status:** Complete (2026-09-24). All four modules select embedded prompt IDs -and share domain variable encoding; proposal generation passes the correction-set -key and diagnostic context through the prepared client, retaining ordering, -warnings, and later domain validation. Updated proposal and module tests and -module internals documentation. The isolated variable-encoding test passed with -`go test internal/framework/proposal_generation/variables.go internal/framework/proposal_generation/variables_test.go`; -`git diff --check` passed. After the validator imports were migrated, -`go test ./internal/framework/proposal_generation ./internal/modules/glossary -./internal/modules/homophones ./internal/modules/spoken_word ./internal/modules/grammar` -passed. A controlled HTTP integration test confirmed that repaired proposals -reach low-confidence and missing-original-text domain checks without an extra call. -**Depends on:** Stage 5. -**Owns:** `internal/modules`, `internal/framework/proposal_generation`, and their tests. +Retain every required validator in its current order. Only candidates approved by +all required validators reach `ApplyProposals`. Application stays deterministic +and may reject proposals made stale by earlier accepted corrections. A failed +batch does not roll back successful unrelated corrections. Fatal runs do not +publish a completed transcript; already committed earlier-stage state remains +available for partial diagnostic evidence. -1. Switch glossary, homophones, spoken_word, and grammar proposal construction to - prompt IDs and variables. Preserve section/glossary encoding, context trimming, - replacement policy, confidence filtering, and applicability checks. -2. Feed repaired structured output into the existing domain path. Exhausted - structural repair produces the existing section warning and no proposals; - runtime failures propagate. Schema-valid but inapplicable proposals do not - trigger semantic retries or additional repairs. -3. Update fake clients and module prompt tests to the final request contract. - Delete duplicate message rendering and obsolete transport/schema checks; retain - domain protection and prompt variable coverage. Update module internals docs. +Preserve current empty-replacement semantics: an empty replacement can remove a +span only if existing preview/application rules allow it and the segment remains +nonempty. Do not alter prompt/schema/deletion behavior under this plan. -**Acceptance and validation:** all four modules have an exercised variable/schema -mapping; existing domain suites retain their assertions. Representative controlled -adapter coverage proves repaired proposals reach domain checks and that a low- -confidence/inapplicable proposal remains rejected. Preserve section/proposal -ordering. Run the module and proposal-generation suites as imports become buildable. +### E. Recoveries and Counts -## Stage 7: Migrate LLM Validators Without Changing Domain Decisions +The adapter returns per-call execution counts in `structuredoutput.Response`, +including on error, using the terminal PromptKit observation. Callers determine +whether the final domain result was usable. The adapter alone must not declare +recovery merely because HTTP/schema handling succeeded. -**Status:** Complete (2026-09-24). All five LLM validator types now select -embedded prompt IDs and shared payload variables. Kept domain batching, -preview, index/cardinality, and deterministic validation; exhausted repair -rejects/warns while provider/protocol failures propagate. Removed duplicate -message builders and per-validator model selection. Updated internal validator -documentation. `go test ./internal/framework/validators -./internal/framework/proposal_generation ./internal/modules/glossary -./internal/modules/homophones ./internal/modules/spoken_word -./internal/modules/grammar ./internal/validators/... ./internal/core/config` -passed, including controlled HTTP repair and malformed-envelope cases; -`git diff --check` passed. Full runner behavior remains for Stage 8. -**Depends on:** Stage 6. -**Owns:** `internal/framework/validators`, validation prompt builders/batching, -and validator tests. +Create at most one recovery record per successful logical model operation. Its +`reasons` list can contain `output_repaired`, `service_recovered`, and, for a +proposal response, `proposals_filtered`. If repair succeeded but its final domain +decision set is unusable, that operation is an error, with its repair/retry counts +in error details; it is not also a recovery. Earlier successful operations retain +their recoveries even if later operations in the unit fail. -1. Migrate spoken_form_plausibility, meaning_reversal_review, editorial_review, - grammar_review, and spoken_word_review to prompt IDs/variables and the common - structured boundary. Keep deterministic validators independent of PromptKit. -2. Preserve batching by `validation.max_prompt_tokens`, proposal preview, stable - proposal indexes, exact decision cardinality, and unknown/duplicate index checks. - Keep the current domain token-estimation method; do not add a second template - renderer or treat this budget as a provider output-token limit. -3. Apply the failure mapping: exhausted repairs reject/warn the batch; schema-valid - bad cardinality or negative assessments reject without semantic retry. Provider, - malformed HTTP, deadline, and admission failures propagate as runtime errors. - Update any old tests treating malformed provider envelopes as warning-only. -4. Remove redundant validator client/model configuration types and keep execution - class distinctions. Update validator reference/internal documentation. +Consolidate defective proposals filtered anywhere in that unit into the originating +proposal operation's recovery. Deduplicate affected proposal indices and retain +reason/count details; never add one recovery per rejected proposal or validator. +Recoveries from retries/repair and filtering of the same proposal response merge. -**Acceptance and validation:** preserve existing batching no-drop/no-duplicate, -domain reason, glossary protection, and cardinality tests. Cover all five prompt -mappings, positive repair before domain validation, exhausted repair, and runtime -failure propagation through a representative validator. Run validator tests; -full runner behavior is Stage 8. +Classify existing rejection reasons deliberately: -## Stage 8: Connect the Runner and Remove the Old Scheduling Hierarchy +- Filtering invalid target IDs, absent target/original text, empty original text, + invalid confidence, no-effect proposals, ambiguous original spans, and proposals + that would empty a segment contributes to `proposals_filtered`. +- Low confidence within the valid range, protected-term policy rejection, and + valid LLM editorial rejection are ordinary debug decisions, not recoveries. +- A proposal becoming stale or redundant because an earlier accepted proposal + changed the text is normal apply-time conflict resolution, not a model defect. + Compare applicability to the stage/chunk input snapshot when necessary; do not + relabel every apply-time skip as a recovery. +- Failure to validate because of size, invalid decision sets, or model availability + is an error, never a filtering recovery. Unexpected missing candidates or broken + cardinality in Audita's own assembled result is fatal internal failure. -**Status:** Complete (2026-09-24). CLI processing now supplies both role clients -from one PromptKit engine; removed global/role schedulers, scheduler contracts, -the old utilization instrumentation, and retired concurrency settings. Runner -ordering, cancellation, partial failure handling, and domain checks remain. -Updated architecture and runtime guidance. `go build -o /tmp/audita-stage8-build -./cmd/audita` passed; focused diagnostics, module, proposal, validator, runner, -and LLM suites passed; `go test -race ./internal/framework/llm` passed. Local -HTTP tests covered shared backend capacity, independent backend identities, -zero-wait rejection, queued deadline cancellation, and later capacity reuse. -A real CLI-to-local-HTTP run completed with one proposal and two validation -calls and wrote the corrected transcript. The legacy CLI test file still uses -retired configuration and report contracts; assembled CLI tests remain for -Stages 10–12. `git diff --check` passed. -**Depends on:** Stages 2 and 4–7. -**Owns:** CLI process assembly, `internal/framework/runner`, old scheduler contracts, -and runner/CLI integration tests. +Counts are derived once from processing outcomes: -1. Pass the shared engine's two role clients into the runner. Remove global and - role schedulers, scheduler interfaces/fields, legacy concurrency configuration, - and measured scheduler wrappers. PromptKit backend admission is the only LLM - service-capacity mechanism. Do not remove unrelated deterministic work or alter - module sequencing merely because the scheduler disappears. -2. Preserve concurrent section work, deterministic collection/indexing, ordered - module application, validator execution classes, and cancellation of remaining - work on runtime failure. Capacity rejection fails the run as selected; do not - silently retry or add another queue to conceal it. -3. Wire final loaded domain settings, role clients, and diagnostics context into - normal processing. No old-runtime branch or interim gate remains. Keep existing - transcript encoding, output channels, partial domain results, and artifact - retention behavior. Ensure terminal failures cannot emit a purportedly complete - corrected transcript. -4. Update architecture policy for shared library dependencies and engine-local - backend admission; preserve the existing ownership, determinism, and redaction - invariants. Update pipeline/runtime internal guidance. +- `work_units_total`: stage instances times initial chunks, zero for either empty set. +- `work_units_started`, `work_units_completed`, `work_units_interrupted`, + `work_units_not_started`: explicit states, not inferred from absent events. +- `work_units_with_errors`: distinct stage/chunk units with recoverable operation + failures; a partially successful unit counts once. +- `run_errors`: fatal errors outside that recoverable-unit count. A fatal error may + carry an origin operation reference without creating a duplicate unit error. +- `corrections_applied`: applied proposals, not replacement occurrences. +- `llm_operations`: actual structured-client invocations including pre-dispatch + failures; `proposal_responses`: structurally usable proposal responses. +- `recoveries`: unique successful operations with recovery reasons. Show counts + alongside `llm_operations`; for filtering-specific context show affected proposal + responses alongside `proposal_responses`. Do not create a single ambiguous + percentage or count repair generations as independent operations. -**Acceptance and validation:** controlled concurrent requests from both roles to -one backend never exceed its configured cap, including retries and repairs; -separate backend identities have independent capacity. Queued cancellation, -deadline and zero-wait-capacity rejection do not leak capacity; later work can -succeed. Existing deterministic runner/domain tests pass after deleting scheduler- -implementation assertions where the affected packages compile. Exercise a real -CLI-to-local-HTTP workflow with both roles when reporting dependencies permit; -otherwise record that check for Stage 12. Do not restore old utilization fields -or add placeholders merely to make this intermediate commit build. +Reason summaries count records containing each reason. Explain that one recovery +can have multiple reasons, so those category counts need not sum to its total. +Knownness/completeness of usage and existing indexed execution aggregation remain +intact. Full token/timing/attempt evidence belongs in debug, not an expanded +ordinary report. -## Stage 9: Collect Execution Observations Safely +### F. Ordinary Output Schema and Markdown -**Status:** Complete (2026-09-24). Added Audita-owned logical-call and execution -event records with call IDs, diagnostic attribution, short synchronized capture, -copying snapshots, and terminal-only usage aggregation. Preserved runner wall -and per-validator timings. Tests cover preparation failure, undispatched -admission, known zero versus unknown usage, partial failed repairs, no double -counting, concurrent attribution, and the real role-client hook. `go test --race ./internal/framework/llm ./internal/framework/runner`, focused package -tests, and `go build -o /tmp/audita-stage9-build ./cmd/audita` passed; -`git diff --check` passed. Report/artifact serialization follows in Stage 10. -**Depends on:** Stage 8. -**Owns:** Audita-owned execution record types and collector in the LLM boundary, -runner attribution, and concurrency/observation tests. +Use explicit Audita DTOs in `core/reporting`, separate from runner/domain types; +`framework/processreport` maps processing state to those DTOs. Render YAML with +`yaml.v3` and explicit YAML tags. Do not export raw PromptKit types or serialize +errors with their causes. Schema version is integer `1` for these new document +names; this is not the former JSON process-report schema. -1. Translate PromptKit events into small Audita-owned records at the observer - boundary. Use the prototype's short mutex-protected collection pattern; callbacks - do no filesystem I/O, provider calls, or waiting on their own run. Freeze/copy - snapshots after active work has joined. -2. Bind observations to logical call, role (`proposal` or `validation`), module - instance, section, validator and batch where applicable, then retain library - run/backend/profile and attempt/generation identity. Keep provider attempts distinct from logical calls. -3. Record dispatch, queue/generation/retry/wall durations, outcome/failure category, - HTTP status, retry exhaustion, attempts/retries/repairs, and usage value/known/ - complete per field. Use terminal events once for provider totals; never add - attempt usage on top. `PrepareExecution` emits no events and `RunPrepared` timing - excludes preparation. Audita's surrounding call record therefore owns the logical - count and wall time, including preparation failures; do not invent library events. - A failure before dispatch has zero attempts and known-zero complete usage. - Preserve completed observations on nil-result failures and mark usage incomplete - when dispatched attempts lack measurements. Across calls, sum known values, - set known if any contributing dispatched usage is known, and set complete only - when all contributing dispatched usage is complete, independently per field. - An aggregate with no dispatched attempts is known-zero and complete. -4. Keep non-LLM runner wall/validator timing. Remove instrumentation that counts - scheduler permits as calls. Do not infer queue peaks, exact active concurrency, - billing totals, or timing fields the released event API does not provide. +Both YAML documents have these envelope fields, in this order: -**Acceptance and validation:** adapter/collector tests prove known zero differs -from unknown, partial usage survives failed repairs, undispatched admissions are -not HTTP attempts, terminal totals are not doubled, and parallel callbacks retain -attribution safely. Run targeted race tests for the affected LLM/runner packages. -No raw provider messages, URLs, keys, prompts, or payloads enter observation records. +| Field | Definition | +| --- | --- | +| `schema` | `audita-errors` or `audita-recoveries` | +| `version` | `1` | +| `run_id` | Same run ID across reports/debug; generated once independently of debug directory creation. | +| `status` | `in_progress`, `completed`, or `failed`; model errors alone still yield `completed`. | +| `started_at`, `completed_at` | UTC RFC3339 strings; completion is null initially. | +| `summary` | Counts named in E, in that order. Unknown pre-processing counts are null; completed empty counts are zero. | +| `errors` or `recoveries` | Always an ordered sequence, including `[]` when empty. | -## Stage 10: Update Reports and Diagnostic Artifacts +`errors` contains two record variants with explicit `scope`: -**Status:** Complete. Process reports use v2 LLM summaries; retained diagnostics -include backend utilization, safe call/event records, selected profiles, config -origins, and the redacted ConfigKit snapshot. Focused reporting, diagnostics, -runtime, and runner tests and a binary build passed. A controlled provider 500 -response echoing a synthetic secret produced a partial failure report and retained -artifacts without the secret in stdout, stderr, reports, or artifacts. The assembled -CLI test suite remains scheduled for Stage 12. +- `work_unit`: `stage_index`, `module_key`, `module_instance`, `chunk_index`, + `start_segment_id`, `end_segment_id`, `corrections_applied`, and `failures`. + Each failure contains `operation`, optional `validator`/`batch_index`, `reason`, + fixed safe `message`, sorted `proposal_indices` when known, and optional execution + counts (`provider_attempts`, `transient_retries`, `structural_repairs`, + `retries_exhausted`, `http_status`). Do not duplicate transcript text. +- `run`: `phase`, `reason`, safe `message`, and optional originating stage/chunk/ + operation identity. Include primary fatal and secondary output-write failures + without pretending these are skipped LLM work units. -**Depends on:** Stage 9. -**Owns:** `internal/core/reporting`, `internal/core/diagnostics`, -`internal/framework/processreport`, runtime diagnostic writers, and CLI artifact wiring. +Each recovery contains stage/module/chunk identity as above, `operation`, optional +`validator`/`batch_index`, ordered `reasons`, fixed safe `message`, execution counts +when known, and `filtered_proposals` with sorted indices and reason/count entries +when filtering occurred. One record can describe both repairs and filtering. -1. Bump the process report version to `v2`; keep the existing transcript/ledger - structures. Add `llm_summary` with `logical_calls`, `provider_attempts`, - `transient_retries`, `structural_repairs`, and `usage`. A logical call counts - once even when it fails before dispatch. Usage has `prompt`, `completion`, - `total`, `cached`, and `cache_write`, each `{value, known, complete}`. - Omit the whole summary only when execution observations are unavailable; a - successfully initialized run with no LLM calls has known-zero complete totals. -2. Replace obsolete utilization concurrency/pool fields with `backends`, keyed by - backend ID, containing configured `concurrency`/`queue_capacity` and the same - execution summary plus `capacity_wait_ms`, `generation_ms`, `retry_delay_ms`, - and `logical_wall_ms`. Sum library terminal timings once per call for capacity, - generation, and retry time; use Audita's whole-call elapsed time for logical wall - time. Attribute pre-dispatch failures to the already selected role backend. - Durations can overlap and must not be presented as a decomposition of elapsed - process time. Preserve domain module/validator timing and run wall time. - Existing role/module call counts now count logical calls; remove their obsolete - scheduler-wait and transport-execution fields rather than assign misleading new - meanings. Remove estimated active-concurrency fields throughout utilization. -3. Write `llm-execution.json` as `{calls: [...], events: [...]}` in the run directory. - Calls contain Audita logical identity, attribution, elapsed milliseconds, final - classification, summary, and available prompt/profile/model metadata. Events - contain the safe Stage 9 records, including attempt and terminal records, joined - by call ID. A preparation failure has a call record and no fabricated events. - Use millisecond integer durations in serialized records; absent attempt/section/ - batch identity is omitted, not encoded as a misleading zero. Add optional - `llm_execution_path` and `selected_profiles_path` to diagnostic path metadata. - Write `selected-profiles.yaml` from the safe view and use ConfigKit YAML for - the effective snapshot (`effective-config.redacted.yaml`). Keep invocation - overlay selection and recorded origins in safe invocation/config diagnostics. - Adjust centralized filename constants and readers together; do not emit old and - new snapshots in parallel. -4. Preserve redacted request/response/error diagnostic content from prepared - details and available final results, including prompt definition/rendered hashes - and selected profile/model. Do not promise legacy hash equality or complete - provider payloads for every retry. Use the separate observation artifact for retry/repair - attempts. Redact all resolved backend secrets, including unselected ones, at - every serialization/error boundary; observation records remain payload-free. -5. Update output-file, operations, and diagnostics/reporting contracts. Preserve - failure report/ledger behavior and the distinction between primary execution - failures and secondary diagnostic write errors. Do not change retention policy. +Indices in YAML are zero-based. Display chunk and batch numbers as index + 1 in +Markdown/CLI, while retaining exact module-instance names. Order unit records by +configured stage order, then chunk index. Within units order proposal first, then +validators by chain order, input preparation before batches, then batch index. +Sort reason lists by code and proposal indices numerically. Render run errors in +lifecycle phase order with output paths sorted for same-phase secondary failures. +Use semantic identity for order; concurrent call IDs/timestamps do not determine +report row or event order. -**Acceptance and validation:** report/diagnostic tests check v2 mapping, summary -semantics, per-backend grouping, safe serialization, nil-result partial failures, -and existing deterministic ledger order. A CLI failure fixture echoes a synthetic -secret in a provider error; stdout/stderr/report and retained artifacts must not -leak it. Inspect known-zero and unavailable summaries structurally, not by complete -JSON golden dumps. Reuse Stage 9 evidence rather than retest every event sequence. +`report.md` always has a title/run ID, status, concise correction/coverage/error/ +recovery totals, input/output paths, and the two matrices. Stage columns follow +configured order; all chunks appear as rows, labeled with 1-based chunk number +and initial segment range. Error cells are 0 or 1 for completed units; recovery +cells are counts. Use `—` for not-started units and `n*` for interrupted units, +where n is the available count; include a fixed legend. Add totals and short +reason breakdowns and refer to the YAML files. An empty stage or chunk set gets +an explicit “No work units” sentence instead of an invalid empty table. -## Stage 11: Finish Examples, Migration Guidance, and Removal +No adaptive width checks, row omission, table splitting, transcript excerpts, or +correction lists. Escape Markdown paths/labels, especially pipes, newlines, and +backticks. Timing values may differ between runs; ordering/layout given the same +snapshot are deterministic. Redact strings before presentation without corrupting +numeric/boolean fields or resurrecting the old unredacted fallback. -**Status:** Complete. Maintained schema-2 local and service examples, role profiles, -and a testing overlay validate offline with a synthetic credential. The v2 -migration guide maps retired files, flags, environment inputs, timing, secrets, -inspection, and reports. Removed the isolated prototype and obsolete CLI test -fixtures/hooks; production config, runtime, proposal, validator, and runner tests -retain their respective boundaries, with assembled subprocess coverage assigned -to Stage 12. Canonical references, Go requirements, historical links, and the -retry follow-up were updated. `go mod tidy`, `go mod verify`, focused tests, -local-link validation, and `git diff --check` passed. +### G. Destination Lifecycle and Failure Semantics -**Depends on:** Stages 1–10. -**Owns:** maintained examples, affected canonical documentation, obsolete code/tests, -prototype removal, and final dependency cleanup. +An output session in `core/diagnostics` owns filesystem state; CLI composes it. +The runner and renderers perform no filesystem operations. Report directories +must exist, matching the existing transcript-output parent requirement; do not +silently create ordinary output parents. Debug parents may still be created by +the debug-directory writer. -1. Replace maintained config examples with schema 2, complete PromptKit profile - files, and `overlays/testing.yml` selecting both roles. Use one shared backend - where roles share a service; provide secret-reference and unauthenticated local - examples without real secrets. Explain that operators choose real models and - testing cost/quality is not guaranteed. Validate examples offline with controlled - synthetic environment values; no service is required for profile inspection. -2. Write `docs/migrations/v2.md` as the one migration guide. Cover every old file/ - flag/environment item in the inventory, durations, explicit empties/null, - credentials, profiles/overlays, inspection views, finite capacity, retry/repair/ - deadline semantics, and changed report/diagnostic shapes. Do not add conversion - tooling or compatibility aliases. Link current contracts for complete references. -3. Finish README prerequisites/examples and canonical CLI/config, integration, - internal component, and architecture references. Audit build/CI toolchain images - for Go 1.25.5 or newer; keep only required Linux amd64/arm64 targets and best-effort - macOS arm64. Do not rewrite immutable historical release notes. -4. Remove `examples/library-adoption` once its unique useful scenarios live in - production tests. Remove the nested dependency on old Audita and duplicate - assets/fixtures. Update historical roadmap links to recorded evidence or Git - history rather than leaving broken links to deleted files. -5. Remove superseded loaders, client, renderers, schedulers, aliases, and tests - asserting retired behavior. Tidy/verify the root dependency graph and confirm - no local replacements. Keep yaml.v3 if glossary parsing still uses it. Mark the - retry-count follow-up in `runtime-behavior.md` resolved without including its - unrelated retention/timestamp proposals. +After valid configuration/profile inspection, resolve paths and guard collisions +before any report truncation. Compare cleaned absolute paths, resolved existing +symlink targets/parents, and `os.SameFile` for existing files. Reject collisions +among the transcript output and the three enabled reports, and with transcript, +glossary, and known loaded configuration/profile source files. These checks +prevent self-overwrite, not hostile filesystem races; allow ordinary symlink use +and existing-output replacement. Return usage/configuration classification 2 for +conflicting destinations. Disabled report paths are not resolved, probed, created, +truncated, removed, or included in collision checks. -**Acceptance and validation:** maintained examples validate and all guide links -resolve. Search for old CLI/env inputs, loader/client/scheduler names and prototype -references; remaining mentions must be intentional migration/history material. -Review deleted tests for lost domain/concurrency/redaction protection. Run affected -example/config/subprocess checks and `git diff --check`; final full-suite execution -belongs to Stage 12. +For file transcript output, verify an existing target can be opened for writing +without `O_TRUNC`; close it. Probe creation in its parent with a standard temporary +file and remove that probe. For an absent target probe the parent without leaving +an empty purported transcript. Validate target type and collisions first; do not +truncate an input or existing transcript during preflight. Stdout cannot be +preflighted this way; its eventual write error is fatal. -## Stage 12: Verify the Combined Application and Close the Refactor +With reporting enabled, initialize all three final report paths with the same +`in_progress` run ID and empty event lists before input processing/model dispatch. +Return 1 immediately if initialization fails; do not invoke an LLM. Attempt to mark +already created reports failed when possible. A readable but invalid transcript +may then produce failed reports. Earlier configuration/profile failures may occur +before report initialization and need not produce a bundle. -**Status:** Complete. Controlled HTTP subprocess tests passed for base and testing -overlay selection, explicit validation-profile and domain overrides, both outbound -models, accepted repair, domain rejection without semantic retry, warning-only -repair exhaustion, empty and repeated module pipelines, and safe partial failure -diagnostics. `go test ./...`, `go build -o ./bin/audita ./cmd/audita`, focused -race tests for runtime/runner/subprocess code, and both Linux release builds -passed. The full suite also passed under Go 1.25.5 with offline module access. -Maintained examples and local documentation links validated; `git diff --check` -passed. No native macOS or live provider call was run. Release preparation remains -separate: propose Audita v2.0.0 with [one migration guide](../migrations/v2.md), -the actual pinned production dependency/license inventory, and LGPL -notice/source/rebuild-distribution review before publication. No release was -published here. +At successful processing completion, fully encode output before writing it. Use +standard temporary-file/write/close/rename replacement within the destination +parent for each final file, cleaning owned temporary files on failure. This is +per-file replacement, not a multi-file atomic transaction or durability guarantee; +no fsync protocol, locks, journal, or automatic rollback system. Preserve existing +file-output permission behavior where applicable and never erase a successfully +published transcript because a later report fails. -**Depends on:** Stages 1–11. -**Owns:** final integration tests, final defect fixes within this scope, and status/evidence. +Write the transcript first, then `errors.yml`, `recoveries.yml`, and `report.md` +last. Final write/close/rename errors are fatal even if preflight succeeded. +Attempt the other enabled report finalizations, collect safe secondary failures, +and make one bounded best-effort pass to write failed-status reports if finalization +failed. Never retry indefinitely or print the success completion message after a +required-output failure. Some files may retain an earlier/in-progress snapshot; +the nonzero exit and stderr identify incomplete finalization. -1. Complete representative production subprocess workflows against controlled HTTP: - base selection, testing overlay, explicit profile override, and a retained - domain override. Observe actual outbound models for both roles, an accepted - correction after repair, a warning-only domain/repair rejection, and a runtime - failure with safe partial diagnostics. Cover an empty module pipeline and a - repeated-module sequence without duplicating all adapter scenarios. -2. Run root `go test ./...`, `go build -o ./bin/audita ./cmd/audita`, and focused race - checks for the final concurrent adapter/runner/collector changes. Verify both - Linux targets compile using `bash scripts/build-release.sh v2.0.0 `; - this names local build outputs and creates no tag or publication. No native Mac, paid-provider, or CI-completion gate. - If the supported minimum toolchain was not exercised earlier, verify the root - suite with installed Go 1.25.5; do not silently claim this from a newer compiler. -3. Fix actual failures and rerun affected checks; repeat the full suite when those - fixes materially change assembled behavior. Do not rerun unchanged tests solely - to mark multiple planning steps complete. Verify maintained documentation links, - examples, and whitespace once for the final content. -4. Record concise final evidence and known limits here, mark the production feature - roadmaps implemented, and commit the completed refactor. Both libraries must - own their selected responsibilities, processing must work, and no temporary - or legacy compatibility machinery may remain. -5. Hand off release preparation separately: proposed Audita v2.0.0, one migration - guide, actual pinned production dependency/license inventory, and the remaining - LGPL notice/source/rebuild-distribution review. Do not claim license compliance - from prototype tests or publish a release. The later release unit uses the - existing minimal automated procedure and ends on a successful source/tag push, - without waiting for CI. +On a processing fatal error or cancellation, do not publish a completed transcript; +finalize partial reports as failed where possible. Root cancellation must not +prevent local report cleanup/finalization after workers have joined. Uncatchable +termination may leave `in_progress`; no resume machinery is added. -**Acceptance:** the final production application and required Linux builds pass; -configuration selection, both LLM roles, domain outcomes, cancellation/capacity, -secret handling, reports and retained artifacts match the selected contract. -Recorded checks describe what actually ran. The workspace is committed and clean. -Release-specific compliance and publication are explicitly still pending. +Debug remains optional evidence, not a condition for transcript success: failure +to create/write debug artifacts emits safe stderr diagnostics but does not change +processing or the exit status. Do not invent an error-bearing work unit for a debug +write failure. Required ordinary reports and transcript writes retain their fatal +semantics. Debug and reporting selections are independent. + +### H. Debug Artifacts and CLI Completion + +With debug enabled, create one unique child under `diagnostics.work_dir`, using the +run ID. Retain it on every outcome; no automatic cleanup/retention modes. Gate the +writers and directory creation, not merely cleanup. Empty diagnostics paths/nil +writers must propagate through module factory, proposals, and validators when off. + +Keep useful current artifacts under debug: source/normalized snapshots, invocation, +redacted effective settings, config origins, selected profiles, chunking/normalization +summaries, correction/decision ledger, indexed execution/utilization data, and model +interaction payloads. Ensure all proposed candidates, including those later rejected +or unavailable, can be inspected when debug is on. Keep JSON where suitable; do not +retain the old ordinary process report as a compatibility output. Remove obsolete +models/helpers/tests rather than keeping dead serialization paths. + +Full payload capture/persistence is not required for production summaries. Compact +operation state and necessary processing data remain available with both reporting +and debug off, so classification and CLI totals do not change. Use all configured +secrets for every writer, including debug snapshots, YAML, Markdown, and surfaced +messages. Do not assume metadata identifiers are safe merely because payloads +are absent. + +Successful CLI stderr begins with the selected style, using exact semantic counts: +`Completed: N corrections applied; E work units with errors; R recoveries.` +Report locations and the debug location follow only when actually created. There +are no ordinary warning messages. Suppressing files does not suppress this brief +summary or fatal errors. Stdout remains transcript-only; with `--output`, stdout +is empty. Exit 0 means processing and all enabled required outputs completed; +exit 1 covers runtime/cancellation/output failure; exit 2 retains usage/configuration +classification. Reporting disabled removes only its file requirements. + +## Stage 1 — Define Processing Outcomes and Contracts + +**Work:** Add the small outcome/reference/count types from B–E. Extend +`structuredoutput.Response` and `Failure`, module proposal results, validator +results/decisions, and runner output to carry actual processing outcomes. Replace +`StageWarning` at these boundaries with the final outcome model; do not alias the +old category to recoveries. Keep Go causes nonserializable and operation references +independent of report DTOs. Make invalid/unavailable decision disposition explicit. + +**Touchpoints:** `internal/framework/structuredoutput`, `contracts`, `validators/models.go`, +`runner/runner.go`, and new `framework/outcomes`. Keep dependencies acyclic; shared +outcome types cannot import their consumers. + +**Validation:** Focused tests for typed error matching and decision partition/cardinality +where behavior is introduced. Compile affected leaf packages. Record temporary +consumer compile failures for subsequent stages; do not add compatibility fields. + +**Exit:** Contracts can express success, recoverable model failure, fatal failure, +partial validation, and recovery detail without any filesystem/report dependency. + +## Stage 2 — Classify Model Failures and Return Execution Facts + +**Work:** Implement C in `llm/adapter.go` using the pinned library's public types. +Capture terminal counts locally for each call and return them even on failure; +forward existing observation hooks without deriving recovery classifications in +the adapter. Preserve prepared-target checks, deadline-before-preparation, same +shared engine/capacity, typed causes, and safe messages. Handle parent cancellation +before local timeout classification. Keep metadata collection concurrency-safe. + +**Validation:** Extend adapter behavioral tests for repaired success/counts, +exhausted invalid output, provider/transport/envelope/capacity failure, local versus +parent deadline/cancellation, preparation defects, and decode overflow. Reuse +existing retry/capacity tests rather than retesting PromptKit. Assert no additional +outer attempts and no provider text in safe errors. + +**Exit:** Domain callers can distinguish skippable model work from fatal errors +without examining error strings; counts are available independently of debug. + +## Stage 3 — Convert Proposal Failures and Classify Safe Filtering + +**Work:** Update `proposal_generation.GenerateCandidates` and all four module +wrappers to preserve operation outcomes. A recoverable proposal call returns zero +candidates plus a failed operation, not a warning or fatal Go error. A valid empty +response is successful. Fatal errors retain their partial call facts and propagate. +Carry successful-call counts until domain filtering can consolidate its recovery. + +Implement the explicit filtering/editorial distinction in E at the domain decision +boundary (a small reason classifier is sufficient). Preserve current deterministic +validators, thresholds, glossary protection, and empty-replacement behavior. Keep +payload diagnostics conditional on a supplied writer/path. + +**Validation:** Proposal integration tests distinguish malformed exhaustion from +fatal setup and valid empty output; deterministic filtering tests cover a defective +proposal versus low-confidence/protected-term rejection. Reuse current four-module +routing tests to prove wrappers preserve outcomes and identities. + +**Exit:** Proposal callers no longer abort for model failures, and filtering can be +attributed to the originating proposal response without treating normal rejection +as recovery. + +## Stage 4 — Preserve Successful Validation Batches + +**Work:** Refactor `LLMBackedValidator.Validate`, input preparation, and response +mapping per D. Collect all successful batch decisions; on a recoverable failure +add non-approved unavailable decisions for that batch and continue. Keep invalid +index sets batch-scoped, oversized candidates excluded individually, and errors +separate from valid negative decisions. Record final domain outcome before declaring +service/repair recovery. Stop promptly on fatal errors/parent cancellation and +retain completed operation facts. Do not synthesize approval for remaining items. + +**Validation:** One controlled multi-batch workflow proves successes before and +after a failed batch survive, and a later validator receives only survivors under +new batch numbering. Cover oversized individual input, missing/duplicate/unknown +indices, repaired-but-domain-invalid output, ordinary rejection, and fatal stop. +Use the existing estimator seam to force batches without brittle default budgets. + +**Exit:** Every normally returned result has exactly one decision for each submitted +candidate; failed model batches do not discard unrelated decisions or bypass later +required validation. + +## Stage 5 — Fix Chunk Boundaries for the Entire Run + +**Work:** Move the runner's chunking call before its stage loop. Own fixed chunk +states and initial inventory in runner output. Execute each module using the current +text of those same chunks, and apply fully approved candidates only within their +chunk. Keep stage barriers, initial chunk metadata, deterministic proposal indexing, +and final ordered recombination. Remove the redundant CLI chunk computation when +wiring its consumer, not by adding an interim second inventory. Preserve within-stage +proposal/validation concurrency and shared capacity; no new scheduler is required. + +**Validation:** A multi-stage test deliberately grows text enough that old re-chunking +would alter membership; assert stable chunk/segment identity and next-stage current +text. Cover repeated modules, target-section selection, empty transcript/pipeline, +and application ordering with unchanged output-schema behavior. + +**Exit:** The normalized transcript is partitioned once and domain processing does +not rely on report tables to maintain chunk state. + +## Stage 6 — Integrate Partial Outcomes and Fatal Cancellation + +**Work:** Update `collectSectionProposals`, `runModulePipeline`, and +`validateSectionCandidates` to consume the outcomes from Stages 3–4. Recoverable +failures never enter shared fatal cancellation. Preserve candidates through every +required validator by proposal index and apply successful survivors even when the +unit has errors. Never return unchecked `eligible` candidates as approved on an +error path. Propagate completed outcomes even if another worker cancels the stage; +do not drop buffered terminal evidence merely because the context is canceled. + +Retain synchronized first-fatal-error ownership, immediate peer cancellation, worker +joining, and deterministic collection. Track not-started/interrupted/completed units +and the counts in E. On fatal stage exit do not apply incomplete validation results; +retain prior committed stages and honest partial observations. Aggregate proposal +filtering once, merge its reasons with any proposal retry/repair recovery, and +retain earlier recoveries when later operations fail. + +**Validation:** Runner tests prove total model failure returns normalized output; +mixed batches apply only fully validated candidates; later stages still execute; +multiple operation failures count one affected unit; repaired-plus-filtered proposals +count once. Adapt the existing cancellation regression to a genuine fatal error: +model failures now continue, while fatal failure cancels active/pending calls, +retains the originating cause, joins workers, and preserves observations. Run +focused runner/runtime race tests when those packages are testable. + +**Exit:** Processing fulfills the resilience contract independently of file output. + +## Stage 7 — Add Reporting and Debug Settings and CLI Selection + +**Work:** Implement A in `core/config/config.go`, `cli/config_input.go`, invocation +state, and usage text. Add boolean flags with visited-value overrides and update +safe effective-setting/provenance output. Remove the old external JSON report flag +and retention selection, including tests for their former semantics. Keep path +selection in CLI composition, not domain configuration resolution. Add context +plumbing through `cli.Run(ctx, args, stdout, stderr)` and process/runtime calls; +update direct callers and tests. In `cmd/audita`, use `signal.NotifyContext` for +interrupt/SIGTERM, with a small run function that defers stop before `os.Exit`. +Configuration help/inspection still creates no output files. + +**Validation:** Parser/ConfigKit tests cover defaults, overlay true overridden by +CLI false, reporting/debug independence, stdout directory selection, ignored report +directory in file/disabled mode, removed flags/keys, and side-effect-free inspection. +No execution scheduler or output files are added in this stage. + +**Exit:** Final settings/CLI contract is resolved without legacy aliases or accidental +filesystem requirements for disabled features. + +## Stage 8 — Map Outcomes to Versioned YAML Reports + +**Work:** Replace ordinary report DTOs and assembly in `core/reporting` and +`framework/processreport` with F. Map actual runner state once into an immutable +report snapshot consumed by all presentations. Preserve debug-specific ledger and +indexed execution mapping only where useful; remove superseded ordinary JSON +models rather than maintaining two authoritative schemas. Implement YAML rendering +for initial, completed, and failed snapshots, explicit empty lists/null unavailable +counts, safe messages, stable ordering, and the two error record scopes. + +Use existing shared redaction at the serialization boundary. Never marshal a raw +error cause or depend on typed decode of already-redacted timestamps/identifiers. +YAML numbers and booleans must retain their types even for numeric credentials. + +**Validation:** Report tests use partial units, multiple underlying failures, +combined recovery reasons, no-work input, run-level failure, unavailable initialization +counts, and deterministic ordering despite reordered observations. Parse rendered +YAML to verify types/counts and redaction, including unselected/escaped credentials. +Preserve indexed aggregation and usage semantics in retained debug builders. + +**Exit:** Both YAML documents truthfully describe the same processing snapshot with +no filesystem effects or verbose payload leakage. + +## Stage 9 — Render the Fixed Markdown and CLI Summaries + +**Work:** Render F/H from the Stage 8 snapshot. Produce both full matrices, totals, +fixed legends, short reason breakdowns, and safe file references. Display 1-based +chunk labels without changing machine identity. Include interrupted and unattempted +states and the no-work sentence. Provide a small shared summary formatter for CLI +counts; CLI stream/file wiring remains Stage 12. No width heuristics, percentage +thresholds, row omission, or extra export formats. + +**Validation:** One representative Markdown golden fixture with explicit update +opt-in can protect the chosen layout; supplement with semantic checks for repeated +stages, partial/error-and-recovery cells, empty work, and escaped labels. Assert CLI +counts agree with YAML and that secrets never enter Markdown. Avoid snapshotting +wall-clock values or every incidental sentence. + +**Exit:** Human-facing output is concise, deterministic, and generated from the same +truth as the structured output. + +## Stage 10 — Implement Required Output Initialization and Finalization + +**Work:** Add the output-session filesystem adapter described in G. Resolve active +paths, protect collisions before truncation, preflight transcript writes without +destroying existing content, initialize all enabled reports, and publish final files +using standard per-file replacement. Return structured primary/secondary errors +for CLI reporting. Track only files/probes created by this invocation for cleanup; +do not delete previous report files when reporting is disabled. Use ordinary file +permissions and no multi-file transaction or crash-recovery framework. + +**Validation:** Real temporary-directory tests cover initialization failure before +a fake model is invoked, output/report/input collisions including existing aliases, +existing transcript preservation, overwrite behavior, absent parent, disabled reports +with an unusable report directory, stdout mode, final transcript/report failures, +and partial failed-status finalization. For permission failures use a narrow injected +file-operation seam when tests may run as root; do not rely solely on chmod. + +**Exit:** Required output failures are detected early where possible and always +reported at finalization without erasing a completed transcript. + +## Stage 11 — Make Debug Persistence Explicit and Independent + +**Work:** Refactor `core/diagnostics` run-directory creation and all CLI/module/ +proposal/validator writer wiring for H. Only debug creates the unique retained +run directory. Gate snapshots, interaction files, ledger, and full execution data +at the write source. Keep required ordinary reporting in its separate session. +Remove automatic retention and the old `never` semantics rather than leaving a +disabled cleanup path. Ensure debug contains all proposals and decisions needed +to diagnose filtering/unavailable results, not just applied changes. + +All debug writers use the shared configured-secret redactor, including original +input snapshots; a raw source copy must not bypass it. Keep safe stderr reporting +of optional-debug write failures separate from model incidents and fatal output +errors. Debug write failures must not change correction decisions or abort the run. + +**Validation:** Tests prove no debug directory/payload files with debug off, useful +retained evidence with debug on, redaction across snapshots/interactions, and no +outcome/exit changes caused by optional debug-write failure. Existing diagnostic +writer tests are retained or replaced according to behavior, not old filenames. + +**Exit:** Debug-off never means “write and later delete”; reporting-off/debug-on and +reporting-on/debug-off both work without coupling their destinations. + +## Stage 12 — Compose the Final CLI Run Lifecycle + +**Work:** Replace the current monolithic process/report/retention flow with the +final composition: load/inspect settings, resolve and initialize required outputs, +optionally initialize debug, load/normalize input, run fixed-chunk processing, +render/finalize, and emit completion/failure output. Thread the caller context all +the way to runtime/runner. Keep phase labels and root versus local cancellation +clear. After joining workers, use available partial state to finalize failed reports +without treating cancellation as an instruction to abandon local cleanup. + +Remove the old pre-run chunking pass and obtain chunk summary/inventory from the +runner. Honor G's finalization order and bounded failed-status rewrite. Do not +print success after required-output failure, invent report locations, or serialize +full corrections into production summaries. Exit 0 with normalized output when +all LLM operations fail; exit nonzero for genuine fatal/output failures. Both +reporting and debug disabled still produce the same transcript and CLI counts. + +**Validation:** Extend actual subprocess fixtures for default file output, stdout +plus selected report directory, reports disabled, debug-only, all-model failure, +partial validation success, invalid input after initialization, and output failure +before dispatch. Inspect created files and streams, not only mocked return values. +Keep deeper batch/classifier matrices in their owning package tests. + +**Exit:** The complete application implements the selected production behavior and +has no obsolete report/retention path keeping debug enabled implicitly. + +## Stage 13 — Update Canonical Guidance, Examples, and Remove Obsolete Material + +**Work:** Update `docs/cli.md`, `config.md`, `operations.md`, integration output/subprocess/ +LLM contracts, internal pipeline/validators/LLM/diagnostics/overview guidance, README, +and maintained examples/migration guidance. Architecture policy must distinguish +recoverable model failures from fatal cancellation and acknowledge the intentional +reporting-contract replacement. Describe implemented behavior only outside roadmap. + +Document the three ordinary files, explicit disable/debug controls, current-directory +stdout default, all-row matrices, partial successes, finalization failures, exit +semantics, and retained debug placement. Remove references to `--report-json`, old +retention modes, ordinary warning reports, and mandatory verbose artifacts. Resolve +the obsolete retention item in `runtime-behavior.md` without changing unrelated +input-timestamp policy. Correct older library-adoption links to completion evidence +so they identify their historical plan instead of claiming this plan is complete. +Leave `future.md` entries deferred. Do not turn ordinary zero-correction results +into an operator warning. + +Remove dead warning types/serialization helpers, obsolete full-report fixtures, +and unused retention/report wiring identified by the graph and references. Keep +meaningful domain tests and useful debug evidence; do not delete coverage just to +hide a still-failing selected invariant. + +**Validation:** Run maintained example/config inspection checks, affected-link and +whitespace checks, and focused tests only where cleanup changes behavior. Confirm +examples contain no credentials and configuration inspection produces no files. + +**Exit:** Canonical docs and examples describe the final application, with no +competing current contract or implicit legacy compatibility promise. + +## Stage 14 — Verify the Integrated Resilience Contract + +**Work:** Audit every acceptance criterion in `resilience.md` against final code and +retained tests. Add only missing integration coverage. In particular verify the +following cross-component outcomes rather than relying solely on green leaf tests: + +- Every proposal fails; separately proposals succeed but every required validation + fails. Both return encoded normalized output, completed status, and accurate + errors with no unsafe corrections. Subsequent stages still attempt their work. +- A failed batch between successful batches leaves valid survivors; later validation + re-batches those survivors, and repeated module instances keep distinct identities. +- Fixed chunks survive substantial text growth; corrections never cross chunk + ownership and final segment ordering/metadata are preserved. +- A recovered transport/repair operation plus filtering counts once; a repaired call + with invalid decision indices counts as an error; a later failed operation does + not erase earlier recoveries. Ordinary rejection is debug-only. +- Fatal cancellation reaches active and queued calls, joins workers, and produces + honest partial/failed reports without a purported complete transcript. +- All four reporting/debug combinations have equal transcript/summary outcomes; + inactive destinations are unused. No ordinary full payloads or raw secrets leak. +- Initial/final write failures produce the selected nonzero result, clean owned + temporary files, and do not destroy a previously produced transcript. Existing + outputs can be replaced intentionally; incompatible destination aliases fail + before truncation. + +Run `go test ./...`, `go vet ./...`, and focused race tests for runner, LLM, +validators, reporting/output adapters, and subprocess packages. Build required +Linux amd64/arm64 binaries using `scripts/build-release.sh` with a valid placeholder +version and temporary output directory, without tagging or publishing. macOS arm64 +is best effort and not a gate. No live provider, CI polling, or release-distribution +review belongs in this implementation sprint. + +Record actual evidence, remaining limitations, and completed stage states here. +Only mark the feature implemented after the full final state is demonstrated. +Commit the completed unit under the agent policy; do not push or release. + +**Exit:** Resilience, partial-success safety, classifications, output lifecycle, +and debug isolation are proven together in the assembled application. ## Open Questions -None block this implementation plan. Configuration, inspection, runtime boundaries, -report representation, and stage ownership are selected above. Release distribution -compliance remains outside this implementation sprint and must be resolved against -the final production dependency graph before publishing binaries; it is not a -reason to build compatibility scaffolding or delay the refactor. +None required to begin implementation. The earlier interface questions are resolved +by A–H; internal helper names and file splits may vary without changing those +contracts. If inspection reveals a concrete contradiction in the pinned libraries +or an agreed requirement, document the question here with a recommended approach +and reasoning plus at least one viable alternative and reasoning before changing +that requirement. Do not reopen deferred features or add compatibility machinery +merely to avoid the selected refactoring. diff --git a/docs/roadmap/resilience.md b/docs/roadmap/resilience.md index 08102ce..a996f29 100644 --- a/docs/roadmap/resilience.md +++ b/docs/roadmap/resilience.md @@ -2,9 +2,9 @@ **Last updated:** 2026-09-24 -**Status:** Feature direction agreed; not implemented. This document defines the -target behavior and user intent, not a staged implementation plan. Remaining -interface details are listed under Open Questions. +**Status:** Target behavior agreed; not implemented. This document defines the +desired end state and user intent. Concrete design decisions and implementation +steps are recorded in [implementation.md](implementation.md). ## Purpose and User Intent @@ -173,12 +173,12 @@ Reporting is enabled by default. The ordinary bundle is: Write all three documents, including empty event lists for uneventful completed runs. Use Audita-owned schemas with safe explanations and stable reason codes; retain run, stage-instance, chunk, operation, and validator/batch identities where -applicable. Exact schemas are detailed-design work, not a reason to preserve the -old report formats or expose raw library structures. +applicable. The implementation plan specifies the versioned schemas; neither old +report formats nor raw library structures define the new output contract. Place reports beside the output transcript. Retain transcript-to-stdout support -with an explicit report-directory selection; its default is a remaining interface -detail. Preserve the existing overwrite convention: a new run can replace files +with an explicit report-directory selection, defaulting to the invocation working +directory. Preserve the existing overwrite convention: a new run can replace files at its selected destinations. Do not add a mandatory fresh-directory restriction or compatibility exports. @@ -209,9 +209,9 @@ and finalization must not depend on retaining a debug run directory. Production issue records are compact. Full responses, transcript excerpts, correction lists, and complete validation decisions are opt-in debug evidence. -Use a single debug boolean, with `--debug` as the proposed CLI spelling and a -ConfigKit-backed setting/override design to be finalized. Debug artifacts may use -JSON where useful. Preserve configured-secret redaction in every presentation. +Use a single ConfigKit-backed debug boolean with a `--debug` CLI override. Debug +artifacts may use JSON where useful. Preserve configured-secret redaction in every +presentation. Debug-off prevents detailed artifact persistence; writing and later deleting those files is insufficient. In-memory domain state needed to process candidates @@ -220,9 +220,12 @@ selections and do not alter processing behavior. There is no obligation to retain `--report-json`, old artifact layouts, parallel exports, or old retention settings. Reuse code only where useful to the new -workflow. Resolve the disposition of debug retention alongside the existing -[runtime follow-up](runtime-behavior.md); the legacy `never` behavior must not -quietly become the meaning of the new reporting-disable option. +workflow. Enabled debug output is retained on success or failure; disabled debug +creates no debug directory. This supersedes the retention question in the +[runtime follow-up](runtime-behavior.md). The legacy `never` behavior is not the +meaning of the new reporting-disable option. Optional debug-write failures produce +a safe stderr diagnostic without changing correction decisions or the run exit +status; ordinary report and transcript-write failures retain their fatal semantics. ## Deterministic Human-Facing Presentation @@ -274,21 +277,14 @@ volume, and a short reason breakdown. Detailed inspection belongs in YAML or deb batches with successful survivors, total failure, fatal cancellation, output failures, and reporting/debug selections. Preserve domain safeguards, capacity limits, and redaction. -- On implementation, update canonical CLI, configuration, operations, integration, - internal, and architecture documentation. This roadmap does not change their - current implemented contracts. +- Canonical CLI, configuration, operations, integration, internal, and architecture + documentation describes the implemented final behavior. Until implementation, + this roadmap does not change their current implemented contracts. -## Open Questions +## Implementation Reference -The product choices above replace the earlier alternatives and user comments. -Remaining details belong in the eventual decision-complete implementation plan: - -- Select exact CLI/configuration names and precedence for reporting enablement, - debug, and the report directory for stdout runs. Reports for file-output runs - belong beside the transcript; the stdout report-directory default is not yet set. -- Finalize YAML versions/fields, machine identifiers versus human chunk labels, - operation-volume denominators, deterministic record order, and failed/not-attempted - presentation without changing the agreed counting or fixed table layout. -- Specify how destinations are checked without destructive transcript truncation, - finalization order, path-collision handling, and debug placement/retention. Keep - ordinary filesystem behavior proportionate; do not add a transaction framework. +The product decisions are settled. [implementation.md](implementation.md) defines +the selected settings and precedence, YAML schemas, counts and ordering, interrupted +work presentation, destination checks, and output finalization. It also owns the +ordered implementation stages and validation work. This document remains the +reference for the desired behavior and its acceptance criteria.