Document completed spell catalog baseline

This commit is contained in:
2026-07-20 19:44:31 +00:00
parent 3d0d79360e
commit 87c57681f6
6 changed files with 57 additions and 28 deletions

View File

@@ -2,9 +2,31 @@
## Status
The immediate baseline feature is accepted and pending implementation.
Feedback-aware repair and semantic LLM validation are deferred until the
baseline has been evaluated.
The accepted baseline implementation is complete. It provides effective
SRD-plus-overlay catalog composition, canonical-name prompt grounding,
deterministic catalog validation, the maintained `retries: 2` policy, and
provenance and checkpoint identity coverage through the assembled offline
workflow.
External quality evaluation is pending. This repository contains the
maintained example and offline fake-LLM coverage, but no approved
human-reviewed transcript corpus or authorized live-model evaluation was
available for this implementation run. The maintained example can be run
from the repository root with:
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells-production.config.yml \
--input examples/seriatim-minimal-transcript.json \
--chunk_cache bypass
```
That command requires the configured Scriptorium profile credential. When an
approved profile and reviewed corpus are available, record anonymized
aggregate false positives, false negatives, unknown-name rejections,
acceptance by attempt, and model-call cost here without committing transcripts
or raw prompts. Feedback-aware repair and semantic LLM validation remain
deferred until that baseline evaluation is available.
## Objective

View File

@@ -9,14 +9,9 @@ not as committed release dates.
### Solidify Spell Extraction
- Implement the immediate overlay-capable catalog-grounding and
deterministic-validation baseline in
[D&D Spell Extraction Quality](dnd-spell-extraction.md), using ordinary
`retries: 2` behavior as the comparison point.
- After evaluating that baseline, reconsider the roadmap's deferred structured
diagnostics, retryability classification, repair-capable extractor contract,
hybrid repair/fresh retry policy, and narrowly scoped semantic LLM
validation.
- Evaluate the implemented baseline in
[D&D Spell Extraction Quality](dnd-spell-extraction.md), then reconsider its
[deferred retry and validation work](dnd-spell-extraction.md#deferred-retry-and-validation-work).
- Replace the no-op spell normalizer with a D&D-specific implementation that
canonicalizes recognized spell names, performs only high-confidence repairs,
retains uncertain values for review, and emits scoped warnings.

View File

@@ -2,9 +2,10 @@
## Status
Ready for implementation. Follow the stages in order. This plan covers the
accepted baseline in [D&D Spell Extraction Quality](dnd-spell-extraction.md),
not its deferred repair-aware retry or semantic LLM-validation work.
The accepted baseline implementation is complete through the assembled
offline workflow. External quality evaluation remains pending as recorded in
[D&D Spell Extraction Quality](dnd-spell-extraction.md). This plan covers that
baseline, not its deferred repair-aware retry or semantic LLM-validation work.
This active plan replaces the completed test-suite implementation record that
previously occupied this filename. That review remains documented in