Add merge references and retry config

This commit is contained in:
2026-07-07 19:10:55 +00:00
parent c05ecb58d8
commit bcedf19a08
25 changed files with 466 additions and 95 deletions

View File

@@ -141,13 +141,13 @@ against the production module catalog and fail fast for unknown or incompatible
module keys.
Reference bindings are validated against reference slots declared by eligible
chunk, extract, and normalize targets during pipeline resolution. Required slots
chunk, extract, merge, and normalize targets during pipeline resolution. Required slots
must be bound after config defaults, target-local references, lane-level
compatibility bindings, and run-time `--reference` or `--without-reference`
overrides are applied. Config-relative paths are resolved relative to the config
file; CLI reference paths are resolved relative to the current working
directory. Materialized bound files must be UTF-8 text. Materialized reference
provenance is recorded for chunk, extractor, and normalizer targets, and runtime
provenance is recorded for chunk, extractor, merger, and normalizer targets, and runtime
reference content is passed to the target that declares the slot. Reference
media types are inferred from file extensions, recorded as canonical base media
types, and checked only when a module declares `AcceptedMediaTypes`; unknown
@@ -156,8 +156,8 @@ written to diagnostics, logs, errors, or manifests.
Pipeline-level `references` are defaults. They are valid when at least one
eligible target in the full configured pipeline declares the slot, including
chunk, extractor, and normalizer targets. During a run, they apply only to the
selected targets that declare the slot:
chunk, extractor, merger, and normalizer targets. During a run, they apply only
to the selected targets that declare the slot:
```yaml
pipelines:
@@ -192,15 +192,17 @@ pipelines:
party: ./campaign/session-party.txt
```
`chunk.references` and `normalize.references` are accepted in object-form
bindings. They override pipeline-level defaults for slots declared by the chunk
or normalizer module. Extractor-local references apply only to the extractor,
and normalizer-local references apply only to the normalizer.
`chunk.references`, `merge.references`, and `normalize.references` are accepted
in object-form bindings. They override pipeline-level defaults for slots
declared by that target module. Extractor-local references apply only to the
extractor, merger-local references apply only to the merger, and
normalizer-local references apply only to the normalizer.
Target-local reference fields use the same map shape at:
- `pipelines.<id>.chunk.references`
- `pipelines.<id>.artifacts.<lane>.extract.references`
- `pipelines.<id>.artifacts.<lane>.merge.references`
- `pipelines.<id>.artifacts.<lane>.normalize.references`
Each binding is valid only when that target module declares the slot.
@@ -226,13 +228,16 @@ Binding fields:
- `module`: module key.
- `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the
Scriptorium prompt default select the profile.
- `retries`: non-negative retry count for runtime stages that support retries.
The current runner preserves this value in resolved config.
- `options`: optional module-specific settings.
- `references`: optional reference bindings. Supported only for `chunk`,
`extract`, and `normalize` bindings. `input`, `merge`, validator, and
`extract`, `merge`, and `normalize` bindings. `input`, validator, and
`output` bindings reject this field during validation.
The `--llm-profile` run flag overrides every effective LLM-capable module
binding to use one Scriptorium profile ID.
binding to use one Scriptorium profile ID: chunk, every selected lane extract,
merge, and normalize binding.
## Implemented Production Modules