eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Resolve references across eligible pipeline targets

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-05 16:24:49 +00:00
parent 51053d390d
commit 43dc954440
5 changed files with 443 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
docs/config.md
View File

@@ -124,9 +124,9 @@ Pipeline fields:
- `artifacts`: required for pipeline resolution. It maps artifact lane IDs to
lane definitions.
- `output`: optional module binding. Default module is `json`.
- `references`: optional map of extractor reference slot names to reference
paths. These bindings are defaults for artifact lanes whose extractor declares
the matching slot.
- `references`: optional map of reference slot names to reference paths. These
bindings are defaults for eligible pipeline targets that declare the matching
slot.
Artifact lane fields:
@@ -142,21 +142,23 @@ Artifact lane fields:
against the production module catalog and fail fast for unknown or incompatible
module keys.
Reference bindings are validated against extractor-declared slots during
pipeline resolution. Required slots must be bound after config defaults,
extractor binding references, lane-level compatibility bindings, and run-time
`--reference` or `--without-reference` overrides are applied. Config-relative
paths are resolved relative to the config file; CLI reference paths are resolved
relative to the current working directory. Bound files must be UTF-8 text and
are passed only to lane extractors that declare the slot. Reference media types
are inferred from file extensions, recorded as canonical base media types, and
checked only when a module declares `AcceptedMediaTypes`; unknown extensions are
recorded as `application/octet-stream`. Reference content is not written to
Reference bindings are validated against reference slots declared by eligible
chunk, extract, and normalize targets during pipeline resolution. Required slots
must be bound after config defaults, target-local references, lane-level
compatibility bindings, and extractor run-time `--reference` or
`--without-reference` overrides are applied. Config-relative paths are resolved
relative to the config file; CLI reference paths are resolved relative to the
current working directory. Materialized bound files must be UTF-8 text and are
currently passed only to lane extractors that declare the slot. Reference media
types are inferred from file extensions, recorded as canonical base media types,
and checked only when a module declares `AcceptedMediaTypes`; unknown extensions
are recorded as `application/octet-stream`. Reference content is not written to
diagnostics, logs, errors, or manifests.
Pipeline-level `references` are defaults. They are valid when at least one
declared lane in the pipeline has an extractor that declares the slot. During a
run, they apply only to selected lanes whose extractor declares the slot:
eligible target in the full configured pipeline declares the slot, including
chunk, extractor, and normalizer targets. During a run, they apply only to the
selected targets that declare the slot:
```yaml
pipelines:
@@ -191,9 +193,9 @@ pipelines:
```
`chunk.references` and `normalize.references` are accepted in object-form
bindings and preserved in the effective configuration. They are validated as
reference maps, but current reference materialization still delivers content
only to extractor bindings.
bindings. They override pipeline-level defaults for slots declared by the chunk
or normalizer module. Extractor-local references apply only to the extractor,
and normalizer-local references apply only to the normalizer.
## Module Bindings