notarius/docs/roadmap/references.md

Feature Roadmap: Extraction References

Status

This document defines the target state and policy choices for the planned extraction-reference feature in Notarius. It describes planned behavior, not implemented behavior. The staged implementation plan lives in implementation.md.

Goal

Extraction quality improves when an LLM-backed extractor receives stable reference material alongside the source input. For the initial D&D spell extractor, useful reference material includes a party roster, a player list, and a campaign glossary.

Notarius should support passing this material to extractors as named reference items without introducing domain-specific concepts into core or framework packages. The framework should know only that:

• extractors declare named reference slots they accept;
• pipeline config and CLI flags bind content, initially files, to those slots;
• bound content is rendered into module-owned prompt templates;
• bound content is digested and recorded as run provenance.

Only extract modules should know what a "roster" or "glossary" means. Domain semantics live in module-owned slot declarations and prompt templates.

Definitions

• Reference slot: a named, typed-by-convention input declared by an extractor, with a human-readable description, required/optional status, and optional guardrails such as accepted media types and maximum bytes.
• Reference item: resolved content bound to a slot for a given run: slot name, content bytes, media type, content digest, size, origin, and binding source.
• Reference binding: the association of a slot name to a content source, defined in pipeline config and overridable per run via CLI.
• Reference set: the lane-scoped collection of resolved reference items delivered to an extractor.

Architectural Principles

• References are opaque to the framework. Core and framework packages must not interpret reference content or recognize domain slot names.
• References are inputs. Anything that can change extraction output must be digested into the run manifest and participate in any cache or idempotency key.
• References are not evidence. SourceRef values must only ever reference source units. Reference items must not receive unit IDs and must not be addressable by source references.
• Slots are declared, not ad hoc. Binding an undeclared slot name, or omitting a required slot, should fail before any LLM call.
• Reference delivery is lane-scoped. Pipeline-level bindings may apply to multiple lanes, but each lane receives only the references declared by its extractor after pipeline, lane, CLI override, and CLI unbind rules are resolved.
• Optional slots degrade gracefully. Prompt templates should render cleanly whether or not an optional slot is bound.
• Rendering must be deterministic. Identical source input, config, prompts, and reference bytes should produce byte-identical rendered prompts. Reference slots should render in declaration order, with stable binding order within a slot.

Target Contracts

Extractors should declare accepted reference slots directly on the extractor contract. This is a first-class feature, so mechanical updates to existing extractors and test fakes are acceptable.

The target slot declaration includes:

• Name;
• Description;
• Required;
• AcceptedMediaTypes;
• Multiple;
• MaxBytes.

Extractors with no reference needs return an empty slot list.

Resolved reference items should be content-bearing values, not unresolved file paths. The MVP producer is "read this file," but the item shape should permit future producers such as prior-run artifacts, derived summaries, or entity registries without changing extractor-facing contracts.

The extraction request should carry the lane-scoped resolved reference set. Framework and core code should treat the set as opaque bytes plus metadata.

Binding Lifecycle

Reference handling should be split across existing lifecycle boundaries:

1. Config parsing records pipeline-level and lane-level reference bindings without reading files.
2. Pipeline resolution validates selected lanes, declared extractor slots, missing required slots, unknown bindings, and ambiguous flat CLI bindings.
3. Run preparation resolves paths, reads files, validates media type and size, computes digests, and materializes reference items.
4. Extraction receives the lane-specific resolved reference set.

Config-relative paths resolve relative to the config file. CLI-relative paths resolve relative to the current working directory.

Configuration and CLI

Reference bindings should live in pipeline config because the initial use cases are campaign-invariant more often than run-variant. Bindings should be supported at two levels:

• pipeline level: defaults shared by artifact lanes whose extractors declare matching slots;
• lane level: additions or overrides for a single artifact lane.

Illustrative config shape:

pipelines:
  dnd-session:
    input: seriatim
    references:
      roster: ./campaign/party_roster.md
      glossary: ./campaign/glossary.md
    artifacts:
      spells:
        extract: dnd/spells
      npcs:
        extract: dnd/npcs
        references:
          npc_registry: ./campaign/npcs.md

Per-run CLI binding overrides should be repeatable:

notarius run dnd-session --input session-014.json --reference roster=./alt_roster.md

Flat CLI slot names are allowed when unambiguous across selected lanes. Lane qualified names, such as spells.roster=./alt_roster.md, disambiguate or target a specific lane. CLI bindings override config bindings for the same lane and slot.

Users should also be able to unbind a config-bound optional slot for a run with an explicit repeatable flag:

notarius run dnd-session --input session-014.json --without-reference roster

Unbinding a required slot should fail during pipeline/reference resolution.

Prompt Template Integration

Prompt templates are module-owned. Template rendering should expose:

• {{ reference "roster" }}: renders the content of the bound item;
• {{ hasreference "glossary" }}: predicate for conditional sections, so optional slots can be included only when bound.

Rules:

• Referencing an undeclared slot from a template is a module bug and should fail at prompt registration/build time or the earliest feasible equivalent.
• Referencing a declared but unbound optional slot should render as empty; templates should use hasreference to avoid dangling section headers.
• Rendering must be deterministic and independent of map iteration order.
• Prompt identity should be computed over the template, not the rendered prompt. Reference digests are recorded separately in the manifest so a reference edit is visible as a reference change, not a prompt change.

Reference content is repeated in every per-chunk prompt in the MVP. Per-slot or per-chunk inclusion policies are deferred until cost data justifies them.

Provenance

The run manifest must record resolved references separately from source digests. For every bound lane and slot, it should record:

• lane ID;
• slot name;
• origin type and URI;
• content digest;
• media type;
• size in bytes;
• whether the binding came from config or CLI override.

Reference digests must participate in any idempotency/cache key alongside source digests, prompt hashes, schema versions, model, and parameters. Two runs that differ only in reference content must be distinguishable from the manifest alone.

Diagnostics for a run should include the resolved binding set with digests, not full reference content, consistent with the existing redacted-effective-config pattern.

Validation and Guardrails

References Are Not Evidence

The primary new failure mode is the model extracting facts from references rather than from the source input. For example, a roster may list a player character's known spells, and the model might emit a spell-cast artifact for a spell that was never cast in the session.

Defenses, in priority order:

1. Structural. SourceRef remains the only grounding mechanism and can only reference source units. No contract change should make references addressable as evidence.
2. Prompt discipline. Module templates should frame references explicitly as reference material, such as "use the roster to resolve speakers to characters; extract only events that occur in the transcript."
3. Validator support. The source-reference validator, or a sibling deterministic validator, should warn when referenced source text does not plausibly relate to the extracted fact. Severity should be warn, not fail, because transcripts can use paraphrase, nicknames, and abbreviations.
4. Regression fixtures. Tests should include a fixture in which a bound roster mentions a spell that is never cast in the transcript, asserting no artifact record is produced for it.

Size and Sanity Guardrails

• MVP accepts UTF-8 text content only. Other media types should be rejected with a clear error.
• A slot-level MaxBytes value should be enforced when declared.
• Empty bound files should produce a warning because they are likely user error.

Out of Scope

• Token budgeting and model context-window management for references.
• Non-file reference producers, including prior-run artifacts, derived summaries, and entity registries.
• Per-chunk or per-slot inclusion policies.
• Structured or parsed references such as typed roster schemas. References are opaque text handed to prompts.
• Reference caching, preprocessing, summarization, embedding, or retrieval.
• Making references addressable as evidence in any form.