audita/docs/roadmap/publish.md

Hard-Cutover Roadmap for Module-Stage LLM Resilience

Summary

This roadmap captures the module-stage resilience work for Audita:

• fail fast on initialization, configuration, schema, and other pre-module setup errors;
• remain resilient during module execution when LLM payloads are malformed or individual proposed corrections are invalid;
• reject bad corrections through validator/reporting paths instead of aborting the module or process;
• keep success stderr quiet; surface warnings only through report and diagnostics artifacts;
• use a hard cutover only, with no compatibility aliases or transitional code.

Locked decisions:

• keep the stable public validator key non_empty_corrected_text;
• change that validator’s behavior to mean “the resulting segment text must not be empty/whitespace-only after applying the proposal preview”;
• malformed LLM-validator batch payloads reject the entire affected batch under that validator and continue;
• proposal/validator transport failures, timeouts, and provider/runtime call failures remain fatal;
• malformed structured payloads are downgraded; non-malformed runtime call failures are not.

Stage 1: Proposal Intake Hardening

• Stop treating invalid individual structured corrections as fatal during proposal generation.
• Preserve returned correction ordering and proposal-index assignment even when individual corrections are malformed.
• Allow corrected_text == "" when the resulting segment remains non-empty after previewed application.
• Downgrade malformed proposal-generation structured payloads into section-scoped warnings with zero proposals for that section.
• Keep proposal-generation transport/provider/runtime call failures fatal.

Deterministic validation changes:

• Add proposal_shape as a built-in deterministic validator and run it first in every built-in module chain.
• Reject malformed proposal fields with stable reason codes:
  • invalid_target_segment_id
  • empty_original_text
  • invalid_confidence
• Keep non_empty_corrected_text as the stable validator key, but change its semantics to reject only empty_resulting_segment.
• Keep validator rejection and apply-time skip as distinct outcomes.

Stage 2: LLM Validator Resilience

• Keep validator transport/provider/runtime call failures fatal.
• Downgrade malformed validator structured payloads into batch-scoped validator rejections plus module warnings.
• Downgrade oversized single-proposal validator inputs into per-proposal validator rejections plus module warnings.
• Preserve decision-cardinality enforcement as an internal invariant after malformed-payload degradation has synthesized complete decision sets.

Stable reason codes introduced or relied upon by this cutover:

• empty_resulting_segment
• invalid_target_segment_id
• empty_original_text
• invalid_confidence
• validator_response_malformed
• validator_input_too_large
• proposal_response_malformed

Stage 3: Reporting, Diagnostics, and Docs

• Add module warning records to runner results and process reports.
• Keep correction-ledger entries per-correction only; do not add standalone warning rows.
• Record malformed proposal-generation and validator-batch warnings through report and diagnostics artifacts only.
• Keep successful runs quiet on stderr even when warnings are present.
• Update README and architecture/public-contract/diagnostics/validators/release-checklist docs to reflect the new behavior.

Acceptance Criteria

• Empty corrected_text may delete words, but proposals that would blank the whole segment are rejected or skipped safely.
• Invalid proposal shape is rejected by validators, not by proposal generation.
• Malformed proposal-generation payloads succeed with warnings and zero proposals for the affected section.
• Malformed validator payloads reject only the affected validator batch and do not fail the module.
• Oversized single validator inputs reject only the affected proposal.
• Transport/provider/runtime LLM failures still fail the module and process.
• Successful runs with warnings still exit 0, emit transcript output normally, keep stderr empty, and expose warnings in report/diagnostics artifacts.