Make module-stage LLM handling resilient and report warnings

This commit is contained in:
2026-05-23 10:07:06 -05:00
parent a84941d681
commit a3655f5540
43 changed files with 856 additions and 217 deletions

View File

@@ -133,11 +133,16 @@ internal/framework/validators/
llm_batching.go
llm_validators.go
internal/framework/warnings/
warnings.go
internal/validators/
metadata/
metadata.go
registry.go
chains.go
proposal_shape/
validator.go
confidence_threshold/
validator.go
original_text_presence/
@@ -242,6 +247,7 @@ Important behavior details:
- Explicit `--modules grammar`, `--modules glossary`, `--modules homophones`, and `--modules spoken_word` continue to run production module paths with LLM-backed proposal generation and validator-chain execution.
- Default runs (without explicit module selection) perform LLM calls through production module and validator paths.
- Success path is generally quiet on stderr.
- Malformed module-stage LLM payloads degrade to validator rejections and module warnings instead of aborting the run.
- Source IDs are preserved into a canonical transcript before normalization; normalization then reassigns output IDs sequentially from `1`.
## Implemented data contracts
@@ -539,20 +545,20 @@ Validator execution classification metadata:
Stable built-in validator keys:
- deterministic:
- `proposal_shape`
- `confidence_threshold`
- `original_text_presence`
- `non_empty_corrected_text`
- `non_empty_corrected_text` (historical key name; current semantics reject empty resulting segment text)
- `no_effect`
- `protected_terms`
- LLM-backed:
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `editorial_review`
- `grammar_review`
- `spoken_word_review`
Built-in module chains:
- `glossary`:
- `proposal_shape`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
@@ -561,6 +567,7 @@ Built-in module chains:
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `homophones`:
- `proposal_shape`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
@@ -569,20 +576,22 @@ Built-in module chains:
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `spoken_word`:
- `proposal_shape`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `spoken_word_review`
- `editorial_review`
- `meaning_reversal_review`
- `grammar`:
- `proposal_shape`
- `no_effect`
- `original_text_presence`
- `confidence_threshold`
- `protected_terms`
- `non_empty_corrected_text`
- `grammar_review`
- `editorial_review`
- `meaning_reversal_review`
1.0 boundary:
@@ -595,12 +604,14 @@ Built-in module chains:
- prompt builders for:
- spoken-form plausibility
- meaning reversal detection
- editorial review
- grammar review
- spoken-word review
- editorial review
- grammar review
- spoken-word review
- deterministic batching by `validation_max_prompt_tokens`;
- strict cardinality validation of structured LLM decisions (missing/duplicate/unknown indexes fail);
- safe failure behavior for malformed/invalid structured responses.
- strict cardinality validation of synthesized validator decision sets;
- malformed validator payloads reject only the affected batch with warnings;
- oversized single-proposal validator inputs reject only the affected proposal;
- transport/provider/runtime LLM call failures remain fatal.
`internal/framework/runner` wires LLM validators into existing validator chains using:
- the internal structured LLM client abstraction (`contracts.StructuredLLMClient`);
@@ -726,6 +737,7 @@ Current process reports include diagnostics metadata references for:
Current process reports also include:
- module-level results (when runner modules execute), including applied/skipped proposal changes;
- run-level module summary totals and failed module instance metadata.
- module-level warning records for malformed proposal-generation payloads and malformed validator batches.
- module-level validator decisions and validator rejections.
- optional decision-level diagnostic artifact paths for validator LLM interactions when available.
- stable validator keys in `validator_name` fields for validator decisions/rejections.