eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Finish the references implementation for the extraction module and update roadmap documentation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-05 10:53:16 -05:00
parent be6803ffa1
commit a516944086
13 changed files with 280 additions and 562 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/cli.md
View File

@@ -61,7 +61,9 @@ Reference flags are resolved against selected artifact lanes before the run
starts. Flat slot names are accepted only when exactly one selected lane
declares that slot. Bound reference files are read before extraction, validated
as UTF-8 text, and passed only to the lane extractor that declares the slot.
Reference content is not written to diagnostics, logs, errors, or manifests.
Notarius infers reference media types from file extensions for provenance and
for optional slot checks. Reference content is not written to diagnostics, logs,
errors, or manifests.
Reference binding precedence is:

12
docs/config.md
View File

@@ -148,11 +148,15 @@ lane-level bindings, and run-time `--reference` or `--without-reference`
overrides are applied. Config-relative paths are resolved relative to the
config file; CLI reference paths are resolved relative to the current working
directory. Bound files must be UTF-8 text and are passed only to lane
extractors that declare the slot. Reference content is not written to
diagnostics, logs, errors, or manifests.
extractors that declare the slot. Reference media types are inferred from file
extensions, recorded as canonical base media types, and checked only when a
module declares `AcceptedMediaTypes`; unknown extensions are recorded as
`application/octet-stream`. Reference content is not written to diagnostics,
logs, errors, or manifests.
Pipeline-level `references` are defaults. They apply only to selected lanes
whose extractor declares the slot:
Pipeline-level `references` are defaults. They are valid when at least one
declared lane in the pipeline has an extractor that declares the slot. During a
run, they apply only to selected lanes whose extractor declares the slot:
```yaml
pipelines:

6
docs/integrations/json-output.md
View File

@@ -114,7 +114,7 @@ When references are bound, the manifest section has this shape:
"origin_type": "file",
"origin_uri": "file:///absolute/path/roster.txt",
"digest": "sha256:...",
"media_type": "text/plain; charset=utf-8",
"media_type": "text/plain",
"size_bytes": 123,
"binding_source": "config"
}
@@ -122,6 +122,10 @@ When references are bound, the manifest section has this shape:
}
```
Reference media types are inferred from file extensions and recorded as
canonical base media types. Unknown extensions are recorded as
`application/octet-stream`.
`module_metadata` is omitted when no singleton module provides metadata.
`validation_status` is `approved` when no candidates were rejected and

12
docs/internal/modules.md
View File

@@ -25,7 +25,10 @@ through both `ReferenceSlots()` and `ModuleSpec().ReferenceSlots`. The runtime
slot list and registry metadata should match so config validation can inspect
slots without constructing extractor instances. A slot declaration names the
slot, whether it is required, accepted media types, whether multiple items are
allowed, and any byte limit.
allowed, and any byte limit. Empty `AcceptedMediaTypes` means any inferred
media type is accepted, though the file must still be UTF-8 text. When a slot
declares accepted media types, Notarius compares the canonical base media type
inferred from the file extension, case-insensitively and without parameters.
Reference content is delivered only to the lane extractor through
`contracts.ExtractionRequest.References`. It is not source evidence and must not
@@ -140,9 +143,10 @@ metadata under `artifact_lanes[].metadata.extractor`. Durable artifact payload
details belong in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
The extractor declares optional `roster` and `glossary` reference slots accepting
UTF-8 text. Its prompt frames references as supporting disambiguation material
only; spell-cast artifacts must still be grounded in the source transcript.
The extractor declares optional `roster` and `glossary` reference slots
accepting UTF-8 text without narrowing accepted media types. Its prompt frames
references as supporting disambiguation material only; spell-cast artifacts must
still be grounded in the source transcript.
## D&D Spell Validators

13
docs/internal/pipeline.md
View File

@@ -45,11 +45,14 @@ LLM-backed pipeline work. Config bindings resolve relative to the config file,
CLI bindings resolve relative to the current working directory, and materialized
reference content is passed only to the matching lane extractor through
`ExtractionRequest`. Materialization accepts UTF-8 text files, computes
`sha256:` content digests, records file origins, enforces declared byte limits,
and warns for empty bound files. Reference content is omitted from diagnostics
and manifests. The CLI writes provenance-only resolved reference diagnostics,
and the run manifest records lane-scoped reference provenance separately from
source digests.
`sha256:` content digests, records file origins, infers canonical base media
types from file extensions, enforces declared byte limits, and warns for empty
bound files. Media-type acceptance is checked only when a slot declares
`AcceptedMediaTypes`; unknown extensions are recorded as
`application/octet-stream`. Reference content is omitted from diagnostics and
manifests. The CLI writes provenance-only resolved reference diagnostics, and
the run manifest records lane-scoped reference provenance separately from source
digests.
Prompt bundles can declare reference slots and use `reference` and
`hasreference` template functions. Bundle loading validates string-literal slot

22
docs/roadmap/future.md
View File

@@ -8,6 +8,21 @@ future work only.
- Additional input adapters, such as Markdown or note-export formats.
- Additional D&D extractors beyond spell casts.
- First-class LLM access for all LLM-eligible pipeline stages: `chunk`,
`extract`, and `normalize`.
- Expand reference support beyond extractors to `chunk` and `normalize`
modules, using the same declared-slot model and preserving stage-specific
ownership boundaries.
- Add non-file reference producers, such as prior-run artifacts, derived
summaries, or entity registries, without changing module-facing reference
item contracts.
- Add token budgeting and model context-window management for reference content.
- Add per-slot or per-chunk inclusion policies so modules can avoid repeating
large reference content in every prompt when that becomes important.
- Add structured or parsed references, such as typed roster schemas, when a
module has a clear need for more than opaque UTF-8 text.
- Add reference caching, preprocessing, summarization, embedding, or retrieval
if references become large enough to require preprocessing.
- Cross-lane entity normalization.
- Cross-chunk semantic deduplication.
- Configurable validator chains with production validator modules.
@@ -15,6 +30,13 @@ future work only.
- Parallel execution where it preserves deterministic manifests and diagnostics.
- Additional output encoders.
## Candidate Architecture Work
- Evaluate replacing the local LLM adapter with an import from
`gitea.maximumdirect.net/eric/scriptorium`, provided it preserves Notarius
boundaries around provider plumbing, prompt ownership, diagnostics, and secret
handling.
## Candidate Operational Work
- Packaged release artifacts for alpha distribution.

305
docs/roadmap/implementation.md
View File

@@ -1,305 +0,0 @@
# Extraction References Implementation Plan
## Purpose
Implement the extraction-reference feature described in
[references.md](references.md). This plan is decision-complete for an LLM coding
agent: implement each stage in order, keep the repository compiling after each
stage, and do not move planned behavior into non-roadmap docs until the relevant
behavior exists.
Core decisions to preserve:
- references are opaque framework inputs and domain semantics stay in extract
modules;
- references are not evidence and must not be addressable through `SourceRef`;
- reference binding is lane-scoped;
- extractors expose `ReferenceSlots()` directly on the first-class extractor
contract;
- token budgeting is deferred; enforce only UTF-8 text handling, empty-file
warnings, and declared `MaxBytes`;
- run manifests record references in a dedicated section, separate from
`source_digests`;
- CLI unbinding uses `--without-reference`.
## Stage 1: Contracts and Mechanical Adoption
Add the framework contracts needed to describe references without changing
runtime behavior. Slot declarations must be available without constructing
extractor modules, because pipeline/config validation should use registry
metadata rather than runtime module instances.
Implementation steps:
- In `internal/framework/contracts`, add reference model types:
`ReferenceSlot`, `ReferenceOrigin`, `ReferenceItem`,
`ResolvedReferenceSlot`, `ReferenceSet`, and a binding-source enum or string
constants for `config` and `cli`.
- Include `Name`, `Description`, `Required`, `AcceptedMediaTypes`, `Multiple`,
and `MaxBytes` on `ReferenceSlot`.
- Include slot name, media type, content bytes, digest, origin, size bytes, and
binding source on `ReferenceItem`.
- Add `References ReferenceSet` to `contracts.ExtractionRequest`.
- Add `ReferenceSlots() []ReferenceSlot` to `contracts.Extractor`.
- Extend extractor registration metadata so reference slots are also declared
through the extractor's registry spec. Prefer the smallest idiomatic change to
the existing registry model, such as adding `ReferenceSlots` to `ModuleSpec`
with validation that non-extractor modules leave it empty, unless the codebase
shape clearly supports a narrower extractor-specific spec.
- Update every concrete extractor and all extractor fakes/test doubles to
implement `ReferenceSlots()`. Existing extractors without references should
return `nil`.
- Add tests that compare a production extractor's runtime `ReferenceSlots()`
with its registered spec slots so the two declarations cannot drift.
- Add contract tests for empty reference sets, slot copying expectations if
helpers are introduced, and compile-time coverage through existing fakes.
Verification:
- `go test ./internal/framework/contracts`
- `go test ./internal/framework/pipeline`
- `go test ./...`
## Stage 2: Config Shape and Pipeline-Level Resolution
Add unresolved reference bindings to config and resolved lane bindings to the
pipeline model. Do not read reference files in this stage.
Implementation steps:
- Add `references` maps to file config parsing at both pipeline and artifact
lane level.
- Add corresponding fields to `pipeline.PipelineProfile` and
`pipeline.ArtifactLaneProfile`.
- Preserve deterministic map handling and duplicate-after-trim validation.
- Extend config cloning, effective config, redaction, validation, and tests for
the new fields.
- Add resolved reference binding structures to `internal/framework/pipeline`.
They should represent lane ID, slot name, source URI/path, and binding source,
but not file bytes.
- During `pipeline.ResolvePipeline`, collect selected lanes, read each lane
extractor's declared slots from registry metadata, and validate without
building extractor instances:
- every bound slot is declared by the lane extractor;
- required slots are bound after applying pipeline-level and lane-level config;
- required slots remain bound after any CLI unbinds supplied to resolution;
- selected lanes under `--only` are the only lanes considered.
- Apply pipeline-level bindings as defaults only to lanes whose extractor
declares the matching slot.
- Apply lane-level bindings as overrides or additions for that lane.
- Keep reference bindings out of source digests and artifact source references.
- Add tests proving reference-slot validation works through registry specs even
when extractor constructors would fail if called.
Verification:
- `go test ./internal/core/config`
- `go test ./internal/framework/pipeline`
- `go test ./...`
## Stage 3: CLI Reference Overrides and Unbinds
Add run-time CLI syntax for reference binding overrides and optional unbinding.
Implementation steps:
- Add repeatable `--reference` flags to `notarius run`.
Accepted forms:
- `slot=path` for unambiguous slot names across selected lanes;
- `lane.slot=path` for explicit lane-scoped binding.
- Add repeatable `--without-reference` flags to `notarius run`.
Accepted forms:
- `slot`;
- `lane.slot`.
- Reject empty paths for `--reference`; use `--without-reference` for unbinding.
- Reject malformed values with concise CLI errors before expensive work.
- Pass parsed override/unbind requests into config/pipeline resolution.
- Resolve flat CLI names only when exactly one selected lane declares the slot.
If multiple selected lanes declare the same slot, fail and instruct the user
to use `lane.slot`.
- Let CLI bindings override config bindings for the same lane and slot.
- Let CLI unbinds remove config-bound optional slots for the same lane and slot.
- Fail if unbinding leaves a required slot unbound.
- Add CLI tests for flat binding, lane-qualified binding, ambiguous flat
binding, malformed syntax, optional unbind, and required-slot unbind failure.
Verification:
- `go test ./internal/cli`
- `go test ./internal/core/config`
- `go test ./internal/framework/pipeline`
- `go test ./...`
## Stage 4: Run Preparation and Reference Materialization
Read, validate, digest, and materialize resolved file references before any LLM
call.
Implementation steps:
- Add a reference resolver/materializer near pipeline run preparation. Keep file
I/O out of pure config parsing.
- Ensure run preparation receives the loaded config path or config directory so
config-relative reference paths can be resolved after pure config parsing.
- Resolve config-relative paths relative to the config file path and
CLI-relative paths relative to the current working directory.
- For MVP, accept only UTF-8 text files. Reject non-UTF-8 content with an error
naming pipeline, lane, slot, and path.
- Compute `sha256:` content digests over the raw reference bytes.
- Populate `ReferenceItem` values with content bytes, media type, digest,
origin type `file`, normalized origin URI/path, size bytes, and binding source.
- Enforce declared `MaxBytes` when greater than zero. The error should name the
pipeline, lane, slot, actual size, limit, and path.
- Emit a warning for empty bound files, but do not fail.
- Add `ReferenceSet` values to the runner input or resolved pipeline path in a
way that keeps lane-scoped references available when calling each extractor.
- Pass the correct lane-specific `ReferenceSet` into
`contracts.ExtractionRequest`.
- Ensure no reference content is written to ordinary diagnostics, logs, errors,
or manifests.
Verification:
- Focused resolver/materializer tests for path resolution, digest stability,
UTF-8 rejection, empty-file warning, `MaxBytes`, and binding source.
- `go test ./internal/cli`
- `go test ./internal/framework/pipeline`
- `go test ./...`
## Stage 5: Prompt Template Reference Functions
Make references available to module-owned prompt templates.
Implementation steps:
- Extend `internal/framework/prompt` so prompt bundles can be compiled with
declared reference slots.
- Add `reference` and `hasreference` template functions.
- Validate at bundle build time, or the earliest feasible equivalent, that
templates reference only declared slots.
- Render a declared but unbound optional slot as an empty string.
- Ensure `hasreference` returns true only when the slot has at least one bound
item with content.
- Render multiple items deterministically if future `Multiple` support is
enabled; for MVP, reject multiple bindings unless the slot declares
`Multiple`.
- Keep prompt metadata hashes based on template source. Do not include rendered
reference content in prompt identity.
- Add deterministic rendering tests proving byte-identical output across runs
with the same reference bytes and config.
Verification:
- `go test ./internal/framework/prompt`
- `go test ./internal/modules/extract/dnd/spells`
- `go test ./...`
## Stage 6: Manifest and Diagnostics Provenance
Record reference provenance separately from source provenance.
Implementation steps:
- Add a dedicated references section to `artifacts.RunManifest`.
The shape should be lane-scoped and include lane ID, slot name, origin type,
origin URI/path, digest, media type, size bytes, and binding source.
- Do not add reference digests to `source_digests`.
- Include reference digests in any cache/idempotency key if such a key exists.
If no cache/idempotency key exists, add a test or comment documenting that no
additional key needs updating yet.
- Write a diagnostics artifact for resolved references that contains provenance
only, not full content, consistent with redacted effective config behavior.
- Ensure durable JSON output manifests include the new manifest section.
- Add manifest round-trip tests and a CLI/run test where two runs that differ
only in reference bytes produce distinguishable manifests.
Verification:
- `go test ./internal/core/artifacts`
- `go test ./internal/core/diagnostics`
- `go test ./internal/modules/output/json`
- `go test ./internal/cli`
- `go test ./...`
## Stage 7: D&D Spells Consumer
Use the new reference feature in the first production extractor.
Implementation steps:
- Declare optional `roster` and `glossary` slots on `dnd/spells`.
- Set accepted media type to text/UTF-8. Add conservative `MaxBytes` limits only
if a clear module-owned limit is chosen; otherwise leave `MaxBytes` unset.
- Update the D&D spells prompt bundle to include conditional reference sections
using `hasreference` and `reference`.
- Frame references as supporting material only. The prompt must instruct the
model to extract only spell-cast events present in the source transcript and
use references only for disambiguation.
- Update prompt metadata tests as needed while preserving template-hash
semantics.
- Add fixture coverage with no references, with roster/glossary references, and
with a roster that mentions a spell never cast in the transcript. The last
case must assert no spell-cast artifact is produced for the uncast spell.
- If existing deterministic source-reference validation can be extended
cleanly, add warning-level relatedness checks for spell names or close
variants near cited source text. If this becomes large, defer that validator
enhancement to a separate roadmap item and keep the prompt/regression fixture
guard in this stage.
Verification:
- `go test ./internal/modules/extract/dnd/spells`
- `go test ./internal/framework/pipeline`
- `go test ./internal/cli`
- `go test ./...`
## Stage 8: Canonical Documentation and Examples
Move implemented behavior out of roadmap-only status once code exists.
Implementation steps:
- Update `docs/cli.md` with `--reference` and `--without-reference` syntax,
precedence, ambiguity behavior, and examples.
- Update `docs/config.md` with pipeline-level and lane-level `references`
blocks.
- Update `docs/internal/modules.md` or the most appropriate internal docs with
module-author guidance for `ReferenceSlots()`, reference request delivery,
prompt functions, evidence exclusion, and provenance.
- Update `docs/internal/pipeline.md` with reference resolution lifecycle and
lane-scoped delivery.
- Update `docs/integrations/json-output.md` with the manifest reference
provenance shape.
- Update `docs/operations.md` or `docs/troubleshooting.md` for common reference
errors such as unknown slot, ambiguous flat override, missing required slot,
unreadable file, non-UTF-8 content, and `MaxBytes` failures.
- Add maintained example reference files and update `examples/dnd-spells.config.yml`
only after the CLI/config behavior is implemented and covered by tests.
- Keep future-only material in `docs/roadmap/references.md`; do not duplicate
canonical current behavior there after implementation.
Verification:
- `rg -n "references:|--reference|--without-reference|ReferenceSlots|reference \"|hasreference" docs examples`
- `go test ./...`
- `go vet ./...`
- `go build ./cmd/notarius`
## Final Acceptance Criteria
The feature is complete when:
- extractor modules can declare reference slots through the first-class
extractor contract;
- config and CLI can bind and unbind lane-scoped file references;
- selected-pipeline validation catches unknown, ambiguous, or missing required
references before any LLM call;
- run preparation materializes UTF-8 text references with digests, size checks,
and empty-file warnings;
- extractors receive lane-scoped resolved references;
- prompt templates can render `reference` and `hasreference` deterministically;
- D&D spell extraction uses optional roster and glossary references;
- run manifests and diagnostics record reference provenance without recording
full content or treating references as source evidence;
- canonical docs and maintained examples describe only implemented behavior;
- `go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` pass.

238
docs/roadmap/references.md
View File

@@ -1,238 +0,0 @@
# Feature Roadmap: Extraction References
## Status
This document defines the target state and policy choices for the planned
extraction-reference feature in Notarius. It describes planned behavior, not
implemented behavior. The staged implementation plan lives in
[implementation.md](implementation.md).
## Goal
Extraction quality improves when an LLM-backed extractor receives stable
reference material alongside the source input. For the initial D&D spell
extractor, useful reference material includes a party roster, a player list, and
a campaign glossary.
Notarius should support passing this material to extractors as **named reference
items** without introducing domain-specific concepts into core or framework
packages. The framework should know only that:
- extractors declare named reference slots they accept;
- pipeline config and CLI flags bind content, initially files, to those slots;
- bound content is rendered into module-owned prompt templates;
- bound content is digested and recorded as run provenance.
Only extract modules should know what a "roster" or "glossary" means. Domain
semantics live in module-owned slot declarations and prompt templates.
## Definitions
- **Reference slot**: a named, typed-by-convention input declared by an
extractor, with a human-readable description, required/optional status, and
optional guardrails such as accepted media types and maximum bytes.
- **Reference item**: resolved content bound to a slot for a given run: slot
name, content bytes, media type, content digest, size, origin, and binding
source.
- **Reference binding**: the association of a slot name to a content source,
defined in pipeline config and overridable per run via CLI.
- **Reference set**: the lane-scoped collection of resolved reference items
delivered to an extractor.
## Architectural Principles
- References are opaque to the framework. Core and framework packages must not
interpret reference content or recognize domain slot names.
- References are inputs. Anything that can change extraction output must be
digested into the run manifest and participate in any cache or idempotency key.
- References are not evidence. `SourceRef` values must only ever reference
source units. Reference items must not receive unit IDs and must not be
addressable by source references.
- Slots are declared, not ad hoc. Binding an undeclared slot name, or omitting a
required slot, should fail before any LLM call.
- Reference delivery is lane-scoped. Pipeline-level bindings may apply to
multiple lanes, but each lane receives only the references declared by its
extractor after pipeline, lane, CLI override, and CLI unbind rules are
resolved.
- Optional slots degrade gracefully. Prompt templates should render cleanly
whether or not an optional slot is bound.
- Rendering must be deterministic. Identical source input, config, prompts, and
reference bytes should produce byte-identical rendered prompts. Reference
slots should render in declaration order, with stable binding order within a
slot.
## Target Contracts
Extractors should declare accepted reference slots directly on the extractor
contract. This is a first-class feature, so mechanical updates to existing
extractors and test fakes are acceptable.
The target slot declaration includes:
- `Name`;
- `Description`;
- `Required`;
- `AcceptedMediaTypes`;
- `Multiple`;
- `MaxBytes`.
Extractors with no reference needs return an empty slot list.
Resolved reference items should be content-bearing values, not unresolved file
paths. The MVP producer is "read this file," but the item shape should permit
future producers such as prior-run artifacts, derived summaries, or entity
registries without changing extractor-facing contracts.
The extraction request should carry the lane-scoped resolved reference set.
Framework and core code should treat the set as opaque bytes plus metadata.
## Binding Lifecycle
Reference handling should be split across existing lifecycle boundaries:
1. Config parsing records pipeline-level and lane-level reference bindings
without reading files.
2. Pipeline resolution validates selected lanes, declared extractor slots,
missing required slots, unknown bindings, and ambiguous flat CLI bindings.
3. Run preparation resolves paths, reads files, validates media type and size,
computes digests, and materializes reference items.
4. Extraction receives the lane-specific resolved reference set.
Config-relative paths resolve relative to the config file. CLI-relative paths
resolve relative to the current working directory.
## Configuration and CLI
Reference bindings should live in pipeline config because the initial use cases
are campaign-invariant more often than run-variant. Bindings should be supported
at two levels:
- pipeline level: defaults shared by artifact lanes whose extractors declare
matching slots;
- lane level: additions or overrides for a single artifact lane.
Illustrative config shape:
```yaml
pipelines:
dnd-session:
input: seriatim
references:
roster: ./campaign/party_roster.md
glossary: ./campaign/glossary.md
artifacts:
spells:
extract: dnd/spells
npcs:
extract: dnd/npcs
references:
npc_registry: ./campaign/npcs.md
```
Per-run CLI binding overrides should be repeatable:
```text
notarius run dnd-session --input session-014.json --reference roster=./alt_roster.md
```
Flat CLI slot names are allowed when unambiguous across selected lanes. Lane
qualified names, such as `spells.roster=./alt_roster.md`, disambiguate or target
a specific lane. CLI bindings override config bindings for the same lane and
slot.
Users should also be able to unbind a config-bound optional slot for a run with
an explicit repeatable flag:
```text
notarius run dnd-session --input session-014.json --without-reference roster
```
Unbinding a required slot should fail during pipeline/reference resolution.
## Prompt Template Integration
Prompt templates are module-owned. Template rendering should expose:
- `{{ reference "roster" }}`: renders the content of the bound item;
- `{{ hasreference "glossary" }}`: predicate for conditional sections, so
optional slots can be included only when bound.
Rules:
- Referencing an undeclared slot from a template is a module bug and should fail
at prompt registration/build time or the earliest feasible equivalent.
- Referencing a declared but unbound optional slot should render as empty;
templates should use `hasreference` to avoid dangling section headers.
- Rendering must be deterministic and independent of map iteration order.
- Prompt identity should be computed over the template, not the rendered prompt.
Reference digests are recorded separately in the manifest so a reference edit
is visible as a reference change, not a prompt change.
Reference content is repeated in every per-chunk prompt in the MVP. Per-slot or
per-chunk inclusion policies are deferred until cost data justifies them.
## Provenance
The run manifest must record resolved references separately from source
digests. For every bound lane and slot, it should record:
- lane ID;
- slot name;
- origin type and URI;
- content digest;
- media type;
- size in bytes;
- whether the binding came from config or CLI override.
Reference digests must participate in any idempotency/cache key alongside source
digests, prompt hashes, schema versions, model, and parameters. Two runs that
differ only in reference content must be distinguishable from the manifest
alone.
Diagnostics for a run should include the resolved binding set with digests, not
full reference content, consistent with the existing redacted-effective-config
pattern.
## Validation and Guardrails
### References Are Not Evidence
The primary new failure mode is the model extracting facts from references
rather than from the source input. For example, a roster may list a player
character's known spells, and the model might emit a spell-cast artifact for a
spell that was never cast in the session.
Defenses, in priority order:
1. **Structural.** `SourceRef` remains the only grounding mechanism and can only
reference source units. No contract change should make references
addressable as evidence.
2. **Prompt discipline.** Module templates should frame references explicitly as
reference material, such as "use the roster to resolve speakers to
characters; extract only events that occur in the transcript."
3. **Validator support.** The source-reference validator, or a sibling
deterministic validator, should warn when referenced source text does not
plausibly relate to the extracted fact. Severity should be `warn`, not
`fail`, because transcripts can use paraphrase, nicknames, and abbreviations.
4. **Regression fixtures.** Tests should include a fixture in which a bound
roster mentions a spell that is never cast in the transcript, asserting no
artifact record is produced for it.
### Size and Sanity Guardrails
- MVP accepts UTF-8 text content only. Other media types should be rejected with
a clear error.
- A slot-level `MaxBytes` value should be enforced when declared.
- Empty bound files should produce a warning because they are likely user error.
## Out of Scope
- Token budgeting and model context-window management for references.
- Non-file reference producers, including prior-run artifacts, derived
summaries, and entity registries.
- Per-chunk or per-slot inclusion policies.
- Structured or parsed references such as typed roster schemas. References are
opaque text handed to prompts.
- Reference caching, preprocessing, summarization, embedding, or retrieval.
- Making references addressable as evidence in any form.

4
docs/troubleshooting.md
View File

@@ -113,6 +113,7 @@ Symptoms include:
- `--without-reference must use slot or lane.slot`
- `read "...": no such file`
- `must be UTF-8 text`
- `media type "..." is not accepted`
- `is ... bytes, limit ...`
Fix:
@@ -127,6 +128,9 @@ Fix:
directory.
- Ensure the file is readable UTF-8 text and within any byte limit declared by
the extractor.
- If the extractor narrows accepted media types, use a file extension that
infers an accepted type such as `text/markdown` or `application/json`.
Unknown extensions infer `application/octet-stream`.
- If diagnostics are retained, inspect `resolved-pipeline.json`,
`resolved-references.json`, and `error.log`.

53
internal/framework/pipeline/profile.go
View File

@@ -135,6 +135,9 @@ func ResolvePipeline(profile PipelineProfile, options ResolveOptions, catalog Mo
if len(selectedLaneIDs) == 0 {
return ResolvedPipeline{}, fmt.Errorf("pipeline %q must select at least one artifact lane", pipelineID)
}
if err := validatePipelineReferenceDefaults(pipelineID, profile.References, lanesByID, catalog); err != nil {
return ResolvedPipeline{}, err
}
resolved := ResolvedPipeline{
ID: pipelineID,
@@ -238,6 +241,44 @@ func resolveArtifactLane(
return lane, capabilities, nil
}
func validatePipelineReferenceDefaults(
pipelineID string,
pipelineReferences map[string]string,
lanesByID map[string]ArtifactLaneProfile,
catalog ModuleCatalog,
) error {
normalizedPipelineReferences, err := normalizedReferenceMap(pipelineReferences, fmt.Sprintf("pipeline %q reference slot", pipelineID))
if err != nil {
return err
}
if len(normalizedPipelineReferences) == 0 {
return nil
}
declaredByAnyLane := make(map[string]struct{}, len(normalizedPipelineReferences))
for _, laneID := range sortedArtifactLaneProfileKeys(lanesByID) {
laneProfile := lanesByID[laneID]
extract := resolveBinding(laneProfile.Extract, "")
if extract.Module == "" {
return fmt.Errorf("pipeline %q lane %q extract module must not be empty", pipelineID, laneID)
}
extractSpec, err := extractorSpec(catalog, extract.Module)
if err != nil {
return moduleLookupError(pipelineID, laneID, StageExtract, extract.Module, err)
}
for _, slot := range extractSpec.ReferenceSlots {
declaredByAnyLane[slot.Name] = struct{}{}
}
}
for _, slotName := range sortedStringMapKeys(normalizedPipelineReferences) {
if _, ok := declaredByAnyLane[slotName]; !ok {
return fmt.Errorf("pipeline %q reference slot %q is not declared by any artifact lane", pipelineID, slotName)
}
}
return nil
}
func resolveReferenceBindings(
pipelineID string,
laneID string,
@@ -370,6 +411,18 @@ func normalizedReferenceMap(values map[string]string, keyName string) (map[strin
return out, nil
}
func sortedArtifactLaneProfileKeys(values map[string]ArtifactLaneProfile) []string {
if len(values) == 0 {
return nil
}
keys := make([]string, 0, len(values))
for key := range values {
keys = append(keys, key)
}
sort.Strings(keys)
return keys
}
func sortedStringMapKeys(values map[string]string) []string {
if len(values) == 0 {
return nil

36
internal/framework/pipeline/profile_test.go
View File

@@ -129,8 +129,7 @@ func TestResolvePipelineSelectsOnlyRequestedLanes(t *testing.T) {
func TestResolvePipelineAppliesReferenceBindings(t *testing.T) {
profile := multiLaneProfile()
profile.References = map[string]string{
" roster ": " ./shared-roster.yml ",
"unclaimed": "./ignored.yml",
" roster ": " ./shared-roster.yml ",
}
lane := profile.Artifacts["events"]
lane.References = map[string]string{
@@ -168,6 +167,39 @@ func TestResolvePipelineAppliesReferenceBindings(t *testing.T) {
}
}
func TestResolvePipelineAllowsPipelineReferenceDeclaredOnlyByUnselectedLane(t *testing.T) {
profile := multiLaneProfile()
profile.References = map[string]string{"notes_context": "./notes.md"}
catalog := newProfileCatalogWithOverride(t, ModuleSpec{
Key: "note-extractor",
Stage: StageExtract,
Requires: []string{"chunk"},
Provides: []string{"candidate"},
ReferenceSlots: []contracts.ReferenceSlot{
{Name: "notes_context"},
},
})
resolved, err := ResolvePipeline(profile, ResolveOptions{Only: []string{"events"}}, catalog)
if err != nil {
t.Fatalf("ResolvePipeline() error = %v, want nil", err)
}
if refs := resolved.ArtifactLanes[0].References; len(refs) != 0 {
t.Fatalf("selected lane references = %#v, want none", refs)
}
}
func TestResolvePipelineRejectsPipelineReferenceNotDeclaredByAnyLane(t *testing.T) {
profile := multiLaneProfile()
profile.References = map[string]string{"missing": "./missing.md"}
_, err := ResolvePipeline(profile, ResolveOptions{Only: []string{"events"}}, newProfileCatalog(t))
if err == nil {
t.Fatal("ResolvePipeline() error = nil, want error")
}
assertErrorContains(t, err, "multi", "reference slot", "missing", "not declared")
}
func TestResolvePipelineRejectsUndeclaredReferenceSlot(t *testing.T) {
profile := baselineProfile()
lane := profile.Artifacts["events"]

47
internal/framework/pipeline/references.go
View File

@@ -4,6 +4,7 @@ import (
"crypto/sha256"
"encoding/hex"
"fmt"
"mime"
"net/url"
"os"
"path/filepath"
@@ -17,7 +18,8 @@ import (
const (
referenceOriginFile = "file"
referenceMediaType = "text/plain; charset=utf-8"
referenceMediaType = "text/plain"
unknownMediaType = "application/octet-stream"
)
type ReferenceMaterializationOptions struct {
@@ -88,6 +90,10 @@ func materializeLaneReferences(
if !utf8.Valid(content) {
return contracts.ReferenceSet{}, nil, fmt.Errorf("pipeline %q lane %q reference slot %q path %q must be UTF-8 text", pipelineID, lane.ID, slotName, path)
}
mediaType := referenceMediaTypeForPath(path)
if !referenceMediaTypeAccepted(mediaType, slot.AcceptedMediaTypes) {
return contracts.ReferenceSet{}, nil, fmt.Errorf("pipeline %q lane %q reference slot %q path %q media type %q is not accepted", pipelineID, lane.ID, slotName, path, mediaType)
}
if slot.MaxBytes > 0 && int64(len(content)) > slot.MaxBytes {
return contracts.ReferenceSet{}, nil, fmt.Errorf("pipeline %q lane %q reference slot %q path %q is %d bytes, limit %d", pipelineID, lane.ID, slotName, path, len(content), slot.MaxBytes)
}
@@ -101,7 +107,7 @@ func materializeLaneReferences(
item := contracts.ReferenceItem{
SlotName: slotName,
MediaType: referenceMediaType,
MediaType: mediaType,
Content: append([]byte(nil), content...),
Digest: referenceDigest(content),
Origin: contracts.ReferenceOrigin{Type: referenceOriginFile, URI: fileURI(path)},
@@ -116,6 +122,43 @@ func materializeLaneReferences(
return set, warnings, nil
}
func referenceMediaTypeForPath(path string) string {
extension := strings.ToLower(filepath.Ext(path))
mediaType := mime.TypeByExtension(extension)
if strings.TrimSpace(mediaType) == "" {
if extension == ".md" || extension == ".markdown" {
return "text/markdown"
}
return unknownMediaType
}
return canonicalMediaType(mediaType)
}
func referenceMediaTypeAccepted(mediaType string, accepted []string) bool {
if len(accepted) == 0 {
return true
}
mediaType = canonicalMediaType(mediaType)
for _, value := range accepted {
if strings.EqualFold(mediaType, canonicalMediaType(value)) {
return true
}
}
return false
}
func canonicalMediaType(mediaType string) string {
trimmed := strings.TrimSpace(mediaType)
if trimmed == "" {
return ""
}
parsed, _, err := mime.ParseMediaType(trimmed)
if err != nil {
return strings.ToLower(trimmed)
}
return strings.ToLower(parsed)
}
func referencePath(binding ReferenceBinding, options ReferenceMaterializationOptions) (string, error) {
source := strings.TrimSpace(binding.Source)
if source == "" {

90
internal/framework/pipeline/references_test.go
View File

@@ -117,6 +117,96 @@ func TestMaterializeReferencesRejectsNonUTF8Content(t *testing.T) {
}
}
func TestMaterializeReferencesAllowsAnyMediaTypeWhenSlotDoesNotRestrictIt(t *testing.T) {
configDir := t.TempDir()
path := filepath.Join(configDir, "roster.reference")
writeReferenceFile(t, path, []byte("plain text"))
resolved := resolvedPipelineWithReference(t, "roster", "roster.reference", contracts.ReferenceBindingSourceConfig, contracts.ReferenceSlot{Name: "roster"})
materialized, _, err := MaterializeReferences(resolved, referenceCatalog(t, []contracts.ReferenceSlot{{Name: "roster"}}), ReferenceMaterializationOptions{
ConfigPath: filepath.Join(configDir, "config.yml"),
})
if err != nil {
t.Fatalf("MaterializeReferences() error = %v, want nil", err)
}
item := materialized.ArtifactLanes[0].ReferenceSet.Slots["roster"].Items[0]
if item.MediaType != unknownMediaType {
t.Fatalf("MediaType = %q, want %q", item.MediaType, unknownMediaType)
}
}
func TestMaterializeReferencesAcceptsDeclaredMarkdownMediaType(t *testing.T) {
configDir := t.TempDir()
path := filepath.Join(configDir, "glossary.md")
writeReferenceFile(t, path, []byte("# Terms\n"))
slot := contracts.ReferenceSlot{Name: "glossary", AcceptedMediaTypes: []string{"text/markdown"}}
resolved := resolvedPipelineWithReference(t, "glossary", "glossary.md", contracts.ReferenceBindingSourceConfig, slot)
materialized, _, err := MaterializeReferences(resolved, referenceCatalog(t, []contracts.ReferenceSlot{slot}), ReferenceMaterializationOptions{
ConfigPath: filepath.Join(configDir, "config.yml"),
})
if err != nil {
t.Fatalf("MaterializeReferences() error = %v, want nil", err)
}
item := materialized.ArtifactLanes[0].ReferenceSet.Slots["glossary"].Items[0]
if item.MediaType != "text/markdown" {
t.Fatalf("MediaType = %q, want text/markdown", item.MediaType)
}
}
func TestMaterializeReferencesAcceptsDeclaredJSONMediaType(t *testing.T) {
configDir := t.TempDir()
path := filepath.Join(configDir, "roster.json")
writeReferenceFile(t, path, []byte(`{"aria":"cleric"}`))
slot := contracts.ReferenceSlot{Name: "roster", AcceptedMediaTypes: []string{"application/json"}}
resolved := resolvedPipelineWithReference(t, "roster", "roster.json", contracts.ReferenceBindingSourceConfig, slot)
materialized, _, err := MaterializeReferences(resolved, referenceCatalog(t, []contracts.ReferenceSlot{slot}), ReferenceMaterializationOptions{
ConfigPath: filepath.Join(configDir, "config.yml"),
})
if err != nil {
t.Fatalf("MaterializeReferences() error = %v, want nil", err)
}
item := materialized.ArtifactLanes[0].ReferenceSet.Slots["roster"].Items[0]
if item.MediaType != "application/json" {
t.Fatalf("MediaType = %q, want application/json", item.MediaType)
}
}
func TestMaterializeReferencesRejectsUnacceptedMediaType(t *testing.T) {
configDir := t.TempDir()
path := filepath.Join(configDir, "roster.json")
writeReferenceFile(t, path, []byte(`{"aria":"cleric"}`))
slot := contracts.ReferenceSlot{Name: "roster", AcceptedMediaTypes: []string{"text/markdown"}}
resolved := resolvedPipelineWithReference(t, "roster", "roster.json", contracts.ReferenceBindingSourceConfig, slot)
_, _, err := MaterializeReferences(resolved, referenceCatalog(t, []contracts.ReferenceSlot{slot}), ReferenceMaterializationOptions{
ConfigPath: filepath.Join(configDir, "config.yml"),
})
if err == nil || !strings.Contains(err.Error(), "media type") || !strings.Contains(err.Error(), "application/json") || !strings.Contains(err.Error(), "roster") {
t.Fatalf("error = %v, want media type rejection", err)
}
}
func TestMaterializeReferencesMatchesAcceptedMediaTypesIgnoringParameters(t *testing.T) {
configDir := t.TempDir()
path := filepath.Join(configDir, "roster.txt")
writeReferenceFile(t, path, []byte("Aria\n"))
slot := contracts.ReferenceSlot{Name: "roster", AcceptedMediaTypes: []string{"text/plain; charset=utf-8"}}
resolved := resolvedPipelineWithReference(t, "roster", "roster.txt", contracts.ReferenceBindingSourceConfig, slot)
materialized, _, err := MaterializeReferences(resolved, referenceCatalog(t, []contracts.ReferenceSlot{slot}), ReferenceMaterializationOptions{
ConfigPath: filepath.Join(configDir, "config.yml"),
})
if err != nil {
t.Fatalf("MaterializeReferences() error = %v, want nil", err)
}
item := materialized.ArtifactLanes[0].ReferenceSet.Slots["roster"].Items[0]
if item.MediaType != referenceMediaType {
t.Fatalf("MediaType = %q, want %q", item.MediaType, referenceMediaType)
}
}
func TestMaterializeReferencesWarnsForEmptyFiles(t *testing.T) {
configDir := t.TempDir()
path := filepath.Join(configDir, "empty.txt")