Add NPC registry grounding for spell extraction

This commit is contained in:
2026-07-21 03:12:03 +00:00
parent fb043325e1
commit 20cfbfd311
26 changed files with 901 additions and 58 deletions

View File

@@ -39,9 +39,9 @@ without exposing Scriptorium types through stage contracts.
7. injecting that one shared client into complete pipeline preparation before
the source file is read or the runner is invoked.
The D&D scene chunker and spell extractor retain this injected client and use
it for every structured completion. Operation requests do not carry an LLM
client.
The D&D scene chunker and spell and NPC extractors retain this injected client
and use it for every structured completion. Operation requests do not carry an
LLM client.
The CLI separately gathers explicit profile IDs from resolved LLM-capable stage
and validator bindings. It prepares a small internal check prompt for each ID so
@@ -104,11 +104,15 @@ The small framework registry contains only generic test schemas; production
schemas remain package-owned.
The spell extractor's package-owned prompt declares a required
`application/json` `spell_catalog` input. The extractor generates that input
from its prepared effective catalog as `{"spell_names":[...]}` using sorted
canonical names only. Its input digest covers those generated bytes; manifests
record catalog identity and digest rather than names, aliases, overlay bytes,
or source metadata.
`application/json` `spell_catalog` input and an optional `application/json`
`npcs` input. The extractor generates the catalog input from its prepared
effective catalog as `{"spell_names":[...]}` using sorted canonical names only.
When an NPC registry is bound, it strictly decodes and identity-validates one
durable artifact, re-encodes canonical JSON, and generates a semantic digest
over those bytes. The unbound input is exactly `{"npcs":[]}`. Input digests
cover the generated bytes; manifests record catalog identity and optional NPC
registry digest/count rather than names, aliases, overlay bytes, registry
paths, or source metadata.
## Debug And Redaction Boundaries

View File

@@ -25,8 +25,9 @@ of that set; input and output builders receive no references. The spell
extractor is typed over the canonical D&D model. D&D validators, merge, and
normalize use typed variants; JSON representation validators use serialized
requests; and unconditional validators expose separate chunk and typed
variants. The D&D production registrar registers only the canonical typed spell
implementations.
variants. The D&D production registrar registers the canonical typed spell and
NPC implementations, including their kind-specific merge and normalize
behavior.
Prepared extractors, extract validators, and codecs may be reused concurrently
by the run-wide extract pool. Production implementations are immutable after
@@ -81,6 +82,15 @@ semantic digest; overlay content remains contextual reference material rather
than source evidence. Its external JSON contract is defined in the
[spell-catalog overlay contract](../integrations/dnd-spell-catalog-overlays.md).
### `internal/modules/dnd/npcs/identity` and `internal/modules/dnd/codec/npcs`
The NPC identity package owns Unicode comparison keys, deterministic
`npc:sha256:` IDs, display normalization, and whole-registry collision issues.
The NPC codec owns the strict durable `dnd/npc-list` JSON boundary and exposes
candidate versus approved encode/decode operations. NPC source references are
durable provenance and may later be consumed by another pipeline as registry
context without being treated as evidence for that pipeline.
## Input Adapter
### `internal/modules/seriatim/input/transcript`
@@ -186,9 +196,34 @@ stages, using the codec only for checkpoint, debug, and output boundaries.
Shared D&D helpers keep prompt input
names and source-unit reference conversion consistent with the scene chunker.
The extractor also declares the optional `npcs` registry slot. Preparation
requires one approved `application/json` item no larger than 1 MiB, validates
identity without relating registry source references to the current transcript,
and supplies canonical JSON to a spell-owned prompt message. A bound registry
adds only `npc_registry_digest` and `npc_count` to manifest metadata and an
`npc_registry` checkpoint fingerprint. The unbound prompt input is exactly
`{"npcs":[]}` and has no registry provenance or fingerprint.
The durable payload and manifest metadata shapes are defined in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
### `internal/modules/dnd/extract/npcs`
The NPC extractor maps private model output to the canonical `dnd.NPCList`,
assigns source identity and deterministic NPC IDs, and preserves source
references for deterministic validation. It uses the shared campaign
references only for disambiguation and does not consume the spell-owned NPC
registry slot. Its prompt and private response schema are package-owned.
### `internal/modules/dnd/normalize/npcs`
The NPC normalizer performs deterministic identity-aware consolidation in
merged input order. It unions only canonical identity or canonical/alias
matches, retains the first display record, unions exact relationships and
source references, rewrites unambiguous relationship targets, and leaves
ambiguous collisions for identity validation. It exposes the identity policy
as its local checkpoint fingerprint and emits bounded normalization warnings.
## Merger And Normalizer
### `internal/modules/generic/merge/appendorder`
@@ -272,6 +307,17 @@ are defined in
payload rules are defined in the
[artifact contract](../integrations/dnd-spell-artifacts.md).
## D&D NPC Validators
NPC shape validation checks required strings, arrays, and source-reference
shape. The source-reference validator checks current-document identity, unit
existence, and range ordering; source relatedness emits at most one bounded
warning per record when neither the canonical name nor an alias occurs near
its cited text. Normalize identity validation checks deterministic IDs,
canonical names, aliases, and cross-record ownership or canonical collisions.
All are deterministic and expose the policy fingerprints used by the
production chains.
## Production Registration
Production composition occurs through family registrars. The CLI allocates one
@@ -280,8 +326,8 @@ complete framework registry set and one LLM asset registry. It invokes
`internal/modules/seriatim/register`, and `internal/modules/dnd/register` in
that order, then exposes the matching catalog for resolution. The generic and
Seriatim registrars own their production leaf registrations. The D&D registrar
owns D&D leaf registrations, the typed spell default-validator chains, and D&D
prompt/schema asset collection.
owns D&D leaf registrations, typed spell and NPC default-validator chains,
typed append-order specializations, and D&D prompt/schema asset collection.
Concrete implementation packages do not import generic implementation
packages directly. A concrete family's `register` package is its composition

View File

@@ -65,9 +65,9 @@ run-local construction closures. Preparation injects shared dependencies and
constructs input, chunk, validators, ordered lanes, and output before source
parsing. Production modules use strict construction-time option decoding, and
LLM-backed modules retain the injected shared client. The D&D family registers
the canonical `dnd/spell-list` codec, typed spell extractor, normalizer, and
validators, plus kind-specific generic merge strategies; generic JSON validators
use the serialized-validation contract. The runner executes lanes through
the canonical `dnd/spell-list` and `dnd/npc-list` codecs, typed spell and NPC
extractors and normalizers, validators, plus kind-specific generic merge
strategies; generic JSON validators use the serialized-validation contract. The runner executes lanes through
private exact-type-checked closures, coordinates extract results independently
of completion timing, and serializes artifacts only through their codec at
checkpoint, debug, and output boundaries.
@@ -84,13 +84,17 @@ Configuration. The implemented module packages are:
| `internal/modules/seriatim/input/transcript` | Parses the supported Seriatim transcript format into the generic source model. |
| `internal/modules/generic/chunk/units` | Splits ordered source units by unit count and overlap. |
| `internal/modules/dnd/chunk/scenes` | Produces contiguous D&D scene chunks from structured model output. |
| `internal/modules/dnd` | Owns the canonical D&D spell-list and spell-cast artifact types. |
| `internal/modules/dnd` | Owns the canonical D&D spell-list, spell-cast, NPC-list, NPC, and relationship artifact types. |
| `internal/modules/dnd/codec/spells` | Strictly decodes and stably encodes the durable D&D spell-list representation. |
| `internal/modules/dnd/codec/npcs` | Strictly decodes and stably encodes the durable D&D NPC-list representation. |
| `internal/modules/dnd/extract/spells` | Maps private structured model output to canonical source-grounded D&D spell lists. |
| `internal/modules/dnd/extract/npcs` | Maps private structured model output to canonical source-grounded D&D NPC lists. |
| `internal/modules/dnd/npcs/identity` | Owns Unicode-aware NPC identity, ID derivation, and registry collision validation. |
| `internal/modules/dnd/spells/catalog` | Embeds and validates the versioned D&D 5e 2014 SRD catalog, composes optional overlays, and provides immutable effective lookup. |
| `internal/modules/generic/merge/appendorder` | Combines accepted extraction results in chunk order. |
| `internal/modules/generic/normalize/noop` | Preserves accepted merged output. |
| `internal/modules/dnd/normalize/spells` | Canonicalizes catalog-backed spell names and exact source references, conservatively collapses duplicate casts, and reports deterministic warnings and independently scoped catalog checkpoint identity. |
| `internal/modules/dnd/normalize/npcs` | Consolidates NPC records deterministically by identity and aliases, rewrites unambiguous relationship targets, and reports bounded warnings. |
| `internal/modules/generic/output/json` | Encodes manifests, lane payloads, warnings, and rejections as logical JSON files. |
`internal/modules/dnd/shared` owns reusable D&D prompt fragments,
@@ -98,6 +102,14 @@ reference declarations, prompt input assembly, and source-unit reference
helpers. Domain-neutral prompt filesystem composition lives in
`internal/framework/promptfs`.
The spell extractor owns its optional `npcs` registry boundary. Preparation
strictly decodes and identity-validates one normalized JSON artifact, emits
canonical registry JSON to the spell prompt, and records only its semantic
digest and count in prepared metadata. The raw reference remains independently
tracked by pipeline provenance. An absent registry is represented only by the
empty prompt value `{"npcs":[]}`; the shared D&D reference fragment is not
changed.
Generic validators under `internal/modules/generic/validate` provide
unconditional test decisions, JSON syntax validation, and JSON Schema
validation. D&D spell validators under `internal/modules/dnd/validate/spells`