Document combat semantics validator evaluation

This commit is contained in:
2026-08-28 00:23:50 +00:00
parent 9cb7462800
commit 22e6caa2a0
5 changed files with 92 additions and 1 deletions

View File

@@ -134,6 +134,19 @@ relatedness validators report advisory evidence concerns. The configured order
is documented in
[Configuration](../config.md#production-validator-keys-and-default-chains).
The optional `extract/dnd/scene-descriptions/combat_semantics` validator is the
D&D family's LLM-backed review of only combat versus non-combat classification.
It selects the shared combat-policy prompt fragment, receives the proposed kind
and current chunk, and maps its verdict deterministically into producer
guidance. It does not assess titles, summaries, non-combat subtype, or scene
boundaries; deferred boundary-coherence review remains separate. It is opt-in;
[Configuration](../config.md) owns selection and retry/failure behavior.
Before proposing default-chain inclusion, run a deliberate provider evaluation
against the synthetic corpus and record false acceptance, false rejection,
producer-correction success, added calls, latency, and token use. Default tests
remain offline and do not measure provider quality.
Every D&D rejection describes the correction in transcript-grounded domain
terms, using contextual names, artifact fields, and source segment ranges when
useful. The guidance must not ask the model to reproduce durable entity IDs,