Document extraction reference support

docs/internal/modules.md

@@ -20,6 +20,22 @@ A production module package should provide:
 Module specs should describe capabilities accurately. Resolution uses specs to
 reject incompatible pipelines before execution.
 
+Extractor modules that accept auxiliary reference material must declare slots
+through both `ReferenceSlots()` and `ModuleSpec().ReferenceSlots`. The runtime
+slot list and registry metadata should match so config validation can inspect
+slots without constructing extractor instances. A slot declaration names the
+slot, whether it is required, accepted media types, whether multiple items are
+allowed, and any byte limit.
+
+Reference content is delivered only to the lane extractor through
+`contracts.ExtractionRequest.References`. It is not source evidence and must not
+be converted into `SourceRef` values. If a module prompt uses references, load
+the prompt bundle with the same declared slots and render with
+`RenderUserSystemWithReferences`. Prompt templates may use the `reference`
+function for content and the `hasreference` function for conditional sections.
+Prompt metadata hashes remain based on template source, not rendered reference
+bytes.
+
 Chunk modules receive the structured LLM client through `contracts.ChunkRequest`
 when they need model-backed chunking. The pipeline runner validates generic
 chunk result invariants before extraction; module-owned policies may be stricter
@@ -124,12 +140,18 @@ metadata under `artifact_lanes[].metadata.extractor`. Durable artifact payload
 details belong in the
 [D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
 
+The extractor declares optional `roster` and `glossary` reference slots accepting
+UTF-8 text. Its prompt frames references as supporting disambiguation material
+only; spell-cast artifacts must still be grounded in the source transcript.
+
 ## D&D Spell Validators
 
 The spell extractor returns two built-in validators:
 
 - `dnd/spells/shape`: rejects malformed payloads and missing required fields.
 - `dnd/spells/source_refs`: rejects candidates without valid source references.
+  It also emits a warning when the extracted spell name is not found in the
+  cited source text.
 
 Reason codes include:
 
@@ -137,6 +159,7 @@ Reason codes include:
 - `missing_required_field`
 - `missing_source_ref`
 - `invalid_source_ref`
+- `spell_not_near_source`
 
 These validators are supplied by the extractor when no validators are configured
 for the lane.

docs/internal/pipeline.md

@@ -33,18 +33,23 @@ The CLI writes the resolved pipeline and digest to diagnostics.
 Pipeline profiles and artifact lanes may include reference binding maps keyed by
 extractor reference slot name. During resolution, pipeline-level bindings act as
 defaults for selected lanes whose extractor declares the slot, lane-level
-bindings override or add lane bindings, and runtime unbinds can remove bindings.
-Resolution validates bindings against extractor specs and records lane-scoped
-binding metadata. It does not read reference files or include reference bytes in
-source digests.
+bindings override or add lane bindings, runtime `--reference` requests override
+config bindings, and runtime unbinds remove optional bindings. Flat runtime slot
+names are resolved only when exactly one selected lane declares the slot;
+otherwise the CLI requires `lane.slot`. Resolution validates bindings against
+extractor specs and records lane-scoped binding metadata. It does not read
+reference files or include reference bytes in source digests.
 
 During run preparation, resolved file references are materialized before any
 LLM-backed pipeline work. Config bindings resolve relative to the config file,
 CLI bindings resolve relative to the current working directory, and materialized
-reference content is passed to extractors through `ExtractionRequest`.
-Reference content is omitted from diagnostics and manifests. The CLI writes
-provenance-only resolved reference diagnostics, and the run manifest records
-lane-scoped reference provenance separately from source digests.
+reference content is passed only to the matching lane extractor through
+`ExtractionRequest`. Materialization accepts UTF-8 text files, computes
+`sha256:` content digests, records file origins, enforces declared byte limits,
+and warns for empty bound files. Reference content is omitted from diagnostics
+and manifests. The CLI writes provenance-only resolved reference diagnostics,
+and the run manifest records lane-scoped reference provenance separately from
+source digests.
 
 Prompt bundles can declare reference slots and use `reference` and
 `hasreference` template functions. Bundle loading validates string-literal slot