Move LLM validators to PromptKit prompt variables

This commit is contained in:
2026-09-24 02:07:17 +00:00
parent f577847ca2
commit a4a72168d5
17 changed files with 310 additions and 257 deletions

View File

@@ -51,11 +51,15 @@ Decision cardinality is enforced: each candidate proposal must receive exactly o
LLM validators:
- build canonical validation request payloads;
- batch using the [configured prompt budget](../config.md#concurrency);
- call structured LLM client using response schema registry.
- batch using `validation.max_prompt_tokens` and the existing domain token estimator;
- select an embedded prompt ID and pass payload/description variables to the
role-bound structured client with the validator-decision schema.
Oversized single proposals are rejected with `validator_input_too_large`.
Malformed LLM validator responses are downgraded to warnings and rejected batch decisions.
Exhausted structural repair becomes a warning and rejected batch decisions.
Provider, malformed HTTP envelope, deadline, and admission failures propagate
as runtime errors. Schema-valid decisions still undergo exact index/cardinality
checks and domain acceptance without a semantic retry.
## Decision and Rejection Reporting

View File

@@ -318,9 +318,11 @@ key and diagnostic context through the prepared client, retaining ordering,
warnings, and later domain validation. Updated proposal and module tests and
module internals documentation. The isolated variable-encoding test passed with
`go test internal/framework/proposal_generation/variables.go internal/framework/proposal_generation/variables_test.go`;
`git diff --check` passed. Assembled proposal/module tests are deferred because
Stage 7's validator imports still reference removed message-rendering types.
The repaired-proposal/domain integration check remains deferred with those imports.
`git diff --check` passed. After the validator imports were migrated,
`go test ./internal/framework/proposal_generation ./internal/modules/glossary
./internal/modules/homophones ./internal/modules/spoken_word ./internal/modules/grammar`
passed. A controlled HTTP integration test confirmed that repaired proposals
reach low-confidence and missing-original-text domain checks without an extra call.
**Depends on:** Stage 5.
**Owns:** `internal/modules`, `internal/framework/proposal_generation`, and their tests.
@@ -343,7 +345,18 @@ ordering. Run the module and proposal-generation suites as imports become builda
## Stage 7: Migrate LLM Validators Without Changing Domain Decisions
**Status:** Pending. **Depends on:** Stage 6.
**Status:** Complete (2026-09-24). All five LLM validator types now select
embedded prompt IDs and shared payload variables. Kept domain batching,
preview, index/cardinality, and deterministic validation; exhausted repair
rejects/warns while provider/protocol failures propagate. Removed duplicate
message builders and per-validator model selection. Updated internal validator
documentation. `go test ./internal/framework/validators
./internal/framework/proposal_generation ./internal/modules/glossary
./internal/modules/homophones ./internal/modules/spoken_word
./internal/modules/grammar ./internal/validators/... ./internal/core/config`
passed, including controlled HTTP repair and malformed-envelope cases;
`git diff --check` passed. Full runner behavior remains for Stage 8.
**Depends on:** Stage 6.
**Owns:** `internal/framework/validators`, validation prompt builders/batching,
and validator tests.