eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Materialize references for all eligible targets

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-05 16:35:24 +00:00
parent 8c623b7ad8
commit 4cafde2502
8 changed files with 316 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
docs/internal/modules.md
View File

@@ -31,14 +31,15 @@ must still be UTF-8 text. When a slot declares accepted media types, Notarius
compares the canonical base media type inferred from the file extension,
case-insensitively and without parameters.
The current resolver materializes reference content only for lane extractors
through `contracts.ExtractionRequest.References`. It is not source evidence and
must not be converted into `SourceRef` values. If a module prompt uses
references, load the prompt bundle with the same declared slots and render with
`RenderUserSystemWithReferences`. Prompt templates may use the `reference`
function for content and the `hasreference` function for conditional sections.
Prompt metadata hashes remain based on template source, not rendered reference
bytes.
The resolver materializes reference content for chunk, extractor, and
normalizer targets. Runtime delivery is currently implemented only for lane
extractors through `contracts.ExtractionRequest.References`. Reference material
is not source evidence and must not be converted into `SourceRef` values. If a
module prompt uses references, load the prompt bundle with the same declared
slots and render with `RenderUserSystemWithReferences`. Prompt templates may use
the `reference` function for content and the `hasreference` function for
conditional sections. Prompt metadata hashes remain based on template source,
not rendered reference bytes.
Chunk modules receive the structured LLM client through `contracts.ChunkRequest`
when they need model-backed chunking. The pipeline runner validates generic

18
docs/internal/pipeline.md
View File

@@ -42,19 +42,19 @@ specific selector such as `chunk.slot`, `lane.extract.slot`, or
target specs and stores the bindings in target-aware resolved reference holders.
It does not read reference files or include reference bytes in source digests.
During run preparation, resolved extractor file references are materialized
before any LLM-backed pipeline work. Config bindings resolve relative to the
config file, CLI bindings resolve relative to the current working directory, and
materialized reference content is passed only to the matching lane extractor
through `ExtractionRequest`. Materialization accepts UTF-8 text files, computes
`sha256:` content digests, records file origins, infers canonical base media
types from file extensions, enforces declared byte limits, and warns for empty
bound files. Media-type acceptance is checked only when a slot declares
During run preparation, resolved file references for chunk, extractor, and
normalizer targets are materialized before any LLM-backed pipeline work. Config
bindings resolve relative to the config file, and CLI bindings resolve relative
to the current working directory. Materialization accepts UTF-8 text files,
computes `sha256:` content digests, records file origins, infers canonical base
media types from file extensions, enforces declared byte limits, and warns for
empty bound files. Media-type acceptance is checked only when a slot declares
`AcceptedMediaTypes`; unknown extensions are recorded as
`application/octet-stream`. Reference content is omitted from diagnostics and
manifests. The CLI writes provenance-only resolved reference diagnostics, and
the run manifest records target-stage reference provenance separately from
source digests.
source digests. Runtime reference content is currently passed only to the
matching lane extractor through `ExtractionRequest`.
Prompt bundles can declare reference slots and use `reference` and
`hasreference` template functions. Bundle loading validates string-literal slot