Move LLM validators to PromptKit prompt variables
This commit is contained in:
@@ -51,11 +51,15 @@ Decision cardinality is enforced: each candidate proposal must receive exactly o
|
||||
|
||||
LLM validators:
|
||||
- build canonical validation request payloads;
|
||||
- batch using the [configured prompt budget](../config.md#concurrency);
|
||||
- call structured LLM client using response schema registry.
|
||||
- batch using `validation.max_prompt_tokens` and the existing domain token estimator;
|
||||
- select an embedded prompt ID and pass payload/description variables to the
|
||||
role-bound structured client with the validator-decision schema.
|
||||
|
||||
Oversized single proposals are rejected with `validator_input_too_large`.
|
||||
Malformed LLM validator responses are downgraded to warnings and rejected batch decisions.
|
||||
Exhausted structural repair becomes a warning and rejected batch decisions.
|
||||
Provider, malformed HTTP envelope, deadline, and admission failures propagate
|
||||
as runtime errors. Schema-valid decisions still undergo exact index/cardinality
|
||||
checks and domain acceptance without a semantic retry.
|
||||
|
||||
## Decision and Rejection Reporting
|
||||
|
||||
|
||||
@@ -318,9 +318,11 @@ key and diagnostic context through the prepared client, retaining ordering,
|
||||
warnings, and later domain validation. Updated proposal and module tests and
|
||||
module internals documentation. The isolated variable-encoding test passed with
|
||||
`go test internal/framework/proposal_generation/variables.go internal/framework/proposal_generation/variables_test.go`;
|
||||
`git diff --check` passed. Assembled proposal/module tests are deferred because
|
||||
Stage 7's validator imports still reference removed message-rendering types.
|
||||
The repaired-proposal/domain integration check remains deferred with those imports.
|
||||
`git diff --check` passed. After the validator imports were migrated,
|
||||
`go test ./internal/framework/proposal_generation ./internal/modules/glossary
|
||||
./internal/modules/homophones ./internal/modules/spoken_word ./internal/modules/grammar`
|
||||
passed. A controlled HTTP integration test confirmed that repaired proposals
|
||||
reach low-confidence and missing-original-text domain checks without an extra call.
|
||||
**Depends on:** Stage 5.
|
||||
**Owns:** `internal/modules`, `internal/framework/proposal_generation`, and their tests.
|
||||
|
||||
@@ -343,7 +345,18 @@ ordering. Run the module and proposal-generation suites as imports become builda
|
||||
|
||||
## Stage 7: Migrate LLM Validators Without Changing Domain Decisions
|
||||
|
||||
**Status:** Pending. **Depends on:** Stage 6.
|
||||
**Status:** Complete (2026-09-24). All five LLM validator types now select
|
||||
embedded prompt IDs and shared payload variables. Kept domain batching,
|
||||
preview, index/cardinality, and deterministic validation; exhausted repair
|
||||
rejects/warns while provider/protocol failures propagate. Removed duplicate
|
||||
message builders and per-validator model selection. Updated internal validator
|
||||
documentation. `go test ./internal/framework/validators
|
||||
./internal/framework/proposal_generation ./internal/modules/glossary
|
||||
./internal/modules/homophones ./internal/modules/spoken_word
|
||||
./internal/modules/grammar ./internal/validators/... ./internal/core/config`
|
||||
passed, including controlled HTTP repair and malformed-envelope cases;
|
||||
`git diff --check` passed. Full runner behavior remains for Stage 8.
|
||||
**Depends on:** Stage 6.
|
||||
**Owns:** `internal/framework/validators`, validation prompt builders/batching,
|
||||
and validator tests.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user