From be6803ffa1dccc1468addbe413edc7c4595fb4c8 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 5 Jul 2026 14:52:52 +0000 Subject: [PATCH] Document extraction reference support --- docs/cli.md | 36 +++++++++++++++++++++ docs/config.md | 41 ++++++++++++++++++++++++ docs/integrations/dnd-spell-artifacts.md | 13 ++++++++ docs/integrations/json-output.md | 2 +- docs/internal/modules.md | 23 +++++++++++++ docs/internal/pipeline.md | 21 +++++++----- docs/operations.md | 7 +++- docs/troubleshooting.md | 28 ++++++++++++++++ examples/dnd-spells-glossary.txt | 2 ++ examples/dnd-spells-roster.txt | 3 ++ examples/dnd-spells.config.yml | 3 ++ 11 files changed, 169 insertions(+), 10 deletions(-) create mode 100644 examples/dnd-spells-glossary.txt create mode 100644 examples/dnd-spells-roster.txt diff --git a/docs/cli.md b/docs/cli.md index c7675866..92eda1d9 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -63,6 +63,42 @@ declares that slot. Bound reference files are read before extraction, validated as UTF-8 text, and passed only to the lane extractor that declares the slot. Reference content is not written to diagnostics, logs, errors, or manifests. +Reference binding precedence is: + +1. pipeline-level config `references`; +2. lane-level config `references`; +3. `--reference` run flags; +4. `--without-reference` run flags. + +`--reference` binds or replaces one slot for one selected lane. Use +`slot=path` when the selected lanes declare the slot unambiguously: + +```sh +go run ./cmd/notarius run dnd-session \ + --config examples/dnd-spells.config.yml \ + --input examples/seriatim-minimal-transcript.json \ + --reference roster=./campaign-roster.txt +``` + +Use `lane.slot=path` when multiple selected lanes declare the same slot or when +you want to target a specific lane: + +```sh +go run ./cmd/notarius run dnd-session \ + --config examples/dnd-spells.config.yml \ + --input examples/seriatim-minimal-transcript.json \ + --reference spells.glossary=./campaign-glossary.txt +``` + +Use `--without-reference` to remove a configured optional binding for a run: + +```sh +go run ./cmd/notarius run dnd-session \ + --config examples/dnd-spells.config.yml \ + --input examples/seriatim-minimal-transcript.json \ + --without-reference glossary +``` + For durable output, diagnostics, retention, and failure inspection, see [Operations](operations.md). diff --git a/docs/config.md b/docs/config.md index 202824a3..991bb2af 100644 --- a/docs/config.md +++ b/docs/config.md @@ -27,6 +27,9 @@ llm_profiles: pipelines: dnd-session: input: seriatim + references: + roster: ./dnd-spells-roster.txt + glossary: ./dnd-spells-glossary.txt chunk: module: generic options: @@ -148,6 +151,36 @@ directory. Bound files must be UTF-8 text and are passed only to lane extractors that declare the slot. Reference content is not written to diagnostics, logs, errors, or manifests. +Pipeline-level `references` are defaults. They apply only to selected lanes +whose extractor declares the slot: + +```yaml +pipelines: + dnd-session: + input: seriatim + references: + roster: ./campaign/party-roster.txt + glossary: ./campaign/glossary.txt + artifacts: + spells: + extract: dnd/spells +``` + +Lane-level `references` override or add bindings for one lane: + +```yaml +pipelines: + dnd-session: + input: seriatim + references: + glossary: ./campaign/glossary.txt + artifacts: + spells: + extract: dnd/spells + references: + roster: ./campaign/session-roster.txt +``` + ## Module Bindings Every module binding may use shorthand: @@ -196,6 +229,14 @@ The `generic` chunker accepts: The `dnd/scenes` chunker requires transcript source capabilities, calls the configured structured LLM provider, and does not accept module options. +The `dnd/spells` extractor declares optional text reference slots: + +- `roster` +- `glossary` + +The extractor uses these references only as supporting disambiguation material; +spell casts still must be present in the source transcript. + ## Diagnostics `diagnostics` fields: diff --git a/docs/integrations/dnd-spell-artifacts.md b/docs/integrations/dnd-spell-artifacts.md index 5de9e721..a8c1947f 100644 --- a/docs/integrations/dnd-spell-artifacts.md +++ b/docs/integrations/dnd-spell-artifacts.md @@ -17,6 +17,14 @@ The extractor requires source chunks and transcript source capability. It returns generic artifact candidates that are serialized by the JSON output module. +The extractor accepts optional UTF-8 text references: + +- `roster`: campaign roster or player-character notes. +- `glossary`: campaign glossary or spell/name notes. + +References are supporting disambiguation material only. They are not source +evidence and are not addressable through `source_refs`. + ## Artifact Envelope Approved artifacts use the generic artifact envelope documented in @@ -120,6 +128,11 @@ Rejection reason codes: - `invalid_source_ref`: at least one source reference fails generic source reference validation. +Warning reason codes: + +- `spell_not_near_source`: the extracted spell name was not found in the cited + source text. + Rejected candidates are written to `rejected.json` by the JSON output module. ## Manifest Metadata diff --git a/docs/integrations/json-output.md b/docs/integrations/json-output.md index a1bcfdfb..a1b38e5d 100644 --- a/docs/integrations/json-output.md +++ b/docs/integrations/json-output.md @@ -109,7 +109,7 @@ When references are bound, the manifest section has this shape: { "references": [ { - "lane_id": "events", + "lane_id": "spells", "slot_name": "roster", "origin_type": "file", "origin_uri": "file:///absolute/path/roster.txt", diff --git a/docs/internal/modules.md b/docs/internal/modules.md index 24f4d279..e0c289d5 100644 --- a/docs/internal/modules.md +++ b/docs/internal/modules.md @@ -20,6 +20,22 @@ A production module package should provide: Module specs should describe capabilities accurately. Resolution uses specs to reject incompatible pipelines before execution. +Extractor modules that accept auxiliary reference material must declare slots +through both `ReferenceSlots()` and `ModuleSpec().ReferenceSlots`. The runtime +slot list and registry metadata should match so config validation can inspect +slots without constructing extractor instances. A slot declaration names the +slot, whether it is required, accepted media types, whether multiple items are +allowed, and any byte limit. + +Reference content is delivered only to the lane extractor through +`contracts.ExtractionRequest.References`. It is not source evidence and must not +be converted into `SourceRef` values. If a module prompt uses references, load +the prompt bundle with the same declared slots and render with +`RenderUserSystemWithReferences`. Prompt templates may use the `reference` +function for content and the `hasreference` function for conditional sections. +Prompt metadata hashes remain based on template source, not rendered reference +bytes. + Chunk modules receive the structured LLM client through `contracts.ChunkRequest` when they need model-backed chunking. The pipeline runner validates generic chunk result invariants before extraction; module-owned policies may be stricter @@ -124,12 +140,18 @@ metadata under `artifact_lanes[].metadata.extractor`. Durable artifact payload details belong in the [D&D spell artifact contract](../integrations/dnd-spell-artifacts.md). +The extractor declares optional `roster` and `glossary` reference slots accepting +UTF-8 text. Its prompt frames references as supporting disambiguation material +only; spell-cast artifacts must still be grounded in the source transcript. + ## D&D Spell Validators The spell extractor returns two built-in validators: - `dnd/spells/shape`: rejects malformed payloads and missing required fields. - `dnd/spells/source_refs`: rejects candidates without valid source references. + It also emits a warning when the extracted spell name is not found in the + cited source text. Reason codes include: @@ -137,6 +159,7 @@ Reason codes include: - `missing_required_field` - `missing_source_ref` - `invalid_source_ref` +- `spell_not_near_source` These validators are supplied by the extractor when no validators are configured for the lane. diff --git a/docs/internal/pipeline.md b/docs/internal/pipeline.md index 83392539..f528a077 100644 --- a/docs/internal/pipeline.md +++ b/docs/internal/pipeline.md @@ -33,18 +33,23 @@ The CLI writes the resolved pipeline and digest to diagnostics. Pipeline profiles and artifact lanes may include reference binding maps keyed by extractor reference slot name. During resolution, pipeline-level bindings act as defaults for selected lanes whose extractor declares the slot, lane-level -bindings override or add lane bindings, and runtime unbinds can remove bindings. -Resolution validates bindings against extractor specs and records lane-scoped -binding metadata. It does not read reference files or include reference bytes in -source digests. +bindings override or add lane bindings, runtime `--reference` requests override +config bindings, and runtime unbinds remove optional bindings. Flat runtime slot +names are resolved only when exactly one selected lane declares the slot; +otherwise the CLI requires `lane.slot`. Resolution validates bindings against +extractor specs and records lane-scoped binding metadata. It does not read +reference files or include reference bytes in source digests. During run preparation, resolved file references are materialized before any LLM-backed pipeline work. Config bindings resolve relative to the config file, CLI bindings resolve relative to the current working directory, and materialized -reference content is passed to extractors through `ExtractionRequest`. -Reference content is omitted from diagnostics and manifests. The CLI writes -provenance-only resolved reference diagnostics, and the run manifest records -lane-scoped reference provenance separately from source digests. +reference content is passed only to the matching lane extractor through +`ExtractionRequest`. Materialization accepts UTF-8 text files, computes +`sha256:` content digests, records file origins, enforces declared byte limits, +and warns for empty bound files. Reference content is omitted from diagnostics +and manifests. The CLI writes provenance-only resolved reference diagnostics, +and the run manifest records lane-scoped reference provenance separately from +source digests. Prompt bundles can declare reference slots and use `reference` and `hasreference` template functions. Bundle loading validates string-literal slot diff --git a/docs/operations.md b/docs/operations.md index 6a51400b..ef6239f2 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -35,7 +35,8 @@ The `json` output module writes these files: - `index.json`: file index with paths to the manifest, artifact files, rejected artifacts, and warnings. - `manifest.json`: run manifest with resolved pipeline provenance, top-level - module metadata, module keys, validation status, and timing. + module metadata, module keys, reference provenance, validation status, and + timing. - `artifacts/.json`: approved artifacts grouped by artifact type. For the current D&D spell extractor, this includes `artifacts/dnd.spell_cast.json` when spell-cast artifacts are approved. @@ -110,6 +111,10 @@ stderr, and writes warnings to durable output and diagnostics when retained. The run manifest `validation_status` indicates whether final artifacts were approved or rejected after validation. +Reference-related warnings include empty bound reference files and D&D spell +relatedness warnings such as `spell_not_near_source`. Empty references are still +passed to extractors so optional slots can be intentionally blank. + ## Cleanup It is safe to remove specific old run directories after their output and diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 68b0015c..520ae431 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -102,6 +102,34 @@ go run ./cmd/notarius run dnd-session \ - For `config validate`, include `--pipeline` when using `--only`. - Confirm the lane ID exists under `pipelines..artifacts`. +## Reference Binding Failure + +Symptoms include: + +- `reference slot "..." is not declared` +- `reference slot "..." is declared by multiple selected lanes` +- `required reference slot "..." is not bound` +- `--reference must use slot=path or lane.slot=path` +- `--without-reference must use slot or lane.slot` +- `read "...": no such file` +- `must be UTF-8 text` +- `is ... bytes, limit ...` + +Fix: + +- Confirm the selected extractor declares the slot. The implemented + `dnd/spells` extractor declares optional `roster` and `glossary` slots. +- Use `lane.slot=path` when more than one selected lane declares the same slot. +- Use `--without-reference slot` to remove optional config bindings; do not pass + an empty `--reference slot=`. +- Check whether a path came from config or CLI. Config paths are relative to + the config file. CLI reference paths are relative to the current working + directory. +- Ensure the file is readable UTF-8 text and within any byte limit declared by + the extractor. +- If diagnostics are retained, inspect `resolved-pipeline.json`, + `resolved-references.json`, and `error.log`. + ## Seriatim Input Validation Failure Symptoms include `seriatim input`, `parse JSON`, `segments must not be empty`, diff --git a/examples/dnd-spells-glossary.txt b/examples/dnd-spells-glossary.txt new file mode 100644 index 00000000..c149892a --- /dev/null +++ b/examples/dnd-spells-glossary.txt @@ -0,0 +1,2 @@ +Cure Wounds: healing spell cast by touch. +Shield: defensive reaction spell. diff --git a/examples/dnd-spells-roster.txt b/examples/dnd-spells-roster.txt new file mode 100644 index 00000000..704346ea --- /dev/null +++ b/examples/dnd-spells-roster.txt @@ -0,0 +1,3 @@ +Aria: party cleric and recurring healer. +Borin: fighter ally. +Bandit mage: hostile spellcaster. diff --git a/examples/dnd-spells.config.yml b/examples/dnd-spells.config.yml index d66fb97c..c0d08654 100644 --- a/examples/dnd-spells.config.yml +++ b/examples/dnd-spells.config.yml @@ -7,6 +7,9 @@ llm_profiles: pipelines: dnd-session: input: seriatim + references: + roster: ./dnd-spells-roster.txt + glossary: ./dnd-spells-glossary.txt chunk: module: generic options: