Add merge references and retry config
This commit is contained in:
@@ -41,6 +41,10 @@ The runtime records the actual selected Scriptorium profile, provider, and model
|
||||
used during execution. Manifest population does not rely on a precomputed
|
||||
profile ID before pipeline execution.
|
||||
|
||||
Explicit profile validation and `--llm-profile` overrides apply to LLM-capable
|
||||
pipeline stages: chunk, extract, merge, and normalize. Input, output, and
|
||||
validator bindings are not part of the current production LLM profile scope.
|
||||
|
||||
## Scriptorium Adapter
|
||||
|
||||
`ScriptoriumClient` implements `contracts.StructuredLLMClient` by converting
|
||||
|
||||
@@ -20,7 +20,7 @@ A production module package should provide:
|
||||
Module specs should describe capabilities accurately. Resolution uses specs to
|
||||
reject incompatible pipelines before execution.
|
||||
|
||||
Chunk, extract, and normalize modules that accept auxiliary reference material
|
||||
Chunk, extract, merge, and normalize modules that accept auxiliary reference material
|
||||
must declare slots through both `ReferenceSlots()` and
|
||||
`ModuleSpec().ReferenceSlots`. The runtime slot list and registry metadata
|
||||
should match so config validation can inspect slots without constructing module
|
||||
@@ -31,10 +31,10 @@ must still be UTF-8 text. When a slot declares accepted media types, Notarius
|
||||
compares the canonical base media type inferred from the file extension,
|
||||
case-insensitively and without parameters.
|
||||
|
||||
The resolver materializes reference content for chunk, extractor, and
|
||||
The resolver materializes reference content for chunk, extractor, merger, and
|
||||
normalizer targets. Runtime delivery uses `contracts.ChunkRequest.References`,
|
||||
`contracts.ExtractionRequest.References`, and
|
||||
`contracts.NormalizeRequest.References`. Reference material is not source
|
||||
`contracts.ExtractionRequest.References`, `contracts.MergeRequest.References`,
|
||||
and `contracts.NormalizeRequest.References`. Reference material is not source
|
||||
evidence and must not be converted into `SourceRef` values. If a module prompt
|
||||
uses references, pass them as prompt input materials through the structured LLM
|
||||
request. Prompt metadata hashes remain based on prompt asset source, not
|
||||
@@ -48,7 +48,7 @@ Common D&D prompt fragments, reference slot helpers, prompt input assembly, and
|
||||
reference rendering live under `internal/modules/sharedassets/dnd`. Module
|
||||
contracts should expose prompt IDs, versions, input material names, and
|
||||
non-secret prompt/schema hashes through manifest metadata; they should not
|
||||
expose Scriptorium public types through chunk, extract, or normalize contracts.
|
||||
expose Scriptorium public types through chunk, extract, merge, or normalize contracts.
|
||||
|
||||
Chunk modules receive the structured LLM client, configured Scriptorium profile
|
||||
ID, prompt session ID, and raw source input material through
|
||||
@@ -60,6 +60,10 @@ Normalize modules receive the structured LLM client, configured Scriptorium
|
||||
profile ID, prompt session ID, and reference material through
|
||||
`contracts.NormalizeRequest` when they need model-backed reconciliation.
|
||||
|
||||
Merge modules receive the structured LLM client, configured Scriptorium profile
|
||||
ID, prompt session ID, raw source input material, and reference material through
|
||||
`contracts.MergeRequest` when they need model-backed merge behavior.
|
||||
|
||||
## `seriatim` Input
|
||||
|
||||
Package: `internal/modules/input/seriatim`
|
||||
|
||||
@@ -33,18 +33,19 @@ The CLI writes the resolved pipeline and digest to diagnostics.
|
||||
|
||||
Pipeline profiles and artifact lanes may include reference binding maps keyed by
|
||||
reference slot name. During resolution, pipeline-level bindings act as defaults
|
||||
for selected chunk, extractor, and normalizer targets that declare the slot;
|
||||
target-local bindings override or add bindings for that target. Runtime
|
||||
for selected chunk, extractor, merger, and normalizer targets that declare the
|
||||
slot; target-local bindings override or add bindings for that target. Runtime
|
||||
`--reference` requests override target config bindings, and runtime unbinds
|
||||
remove optional target bindings. Flat runtime slot names are resolved only when
|
||||
exactly one selected target declares the slot; otherwise the CLI requires a more
|
||||
specific selector such as `chunk.slot`, `lane.extract.slot`, or
|
||||
`lane.normalize.slot`. Resolution validates bindings against the declaring
|
||||
target specs and stores the bindings in target-aware resolved reference holders.
|
||||
It does not read reference files or include reference bytes in source digests.
|
||||
specific selector such as `chunk.slot`, `lane.extract.slot`,
|
||||
`lane.merge.slot`, or `lane.normalize.slot`. Resolution validates bindings
|
||||
against the declaring target specs and stores the bindings in target-aware
|
||||
resolved reference holders. It does not read reference files or include
|
||||
reference bytes in source digests.
|
||||
|
||||
During run preparation, resolved file references for chunk, extractor, and
|
||||
normalizer targets are materialized before any LLM-backed pipeline work. Config
|
||||
During run preparation, resolved file references for chunk, extractor, merger,
|
||||
and normalizer targets are materialized before any LLM-backed pipeline work. Config
|
||||
bindings resolve relative to the config file, and CLI bindings resolve relative
|
||||
to the current working directory. Materialization accepts UTF-8 text files,
|
||||
computes `sha256:` content digests, records file origins, infers canonical base
|
||||
@@ -55,8 +56,8 @@ empty bound files. Media-type acceptance is checked only when a slot declares
|
||||
manifests. The CLI writes provenance-only resolved reference diagnostics, and
|
||||
the run manifest records target-stage reference provenance separately from
|
||||
source digests. Runtime reference content is passed to the matching chunker,
|
||||
extractor, or normalizer request. LLM-backed modules pass that material onward
|
||||
as named Scriptorium prompt inputs.
|
||||
extractor, merger, or normalizer request. LLM-backed modules pass that material
|
||||
onward as named Scriptorium prompt inputs.
|
||||
|
||||
The CLI carries raw input bytes into `pipeline.RunInput`. Input adapters parse
|
||||
those bytes into the source document, while LLM-backed modules that need the
|
||||
@@ -65,9 +66,9 @@ origin metadata. The raw input payload is not written to manifests or default
|
||||
diagnostics.
|
||||
|
||||
The CLI also carries an optional run `session_id`. The runner makes it available
|
||||
to chunk, extract, and normalize requests; LLM-backed modules forward it through
|
||||
their structured completion requests so Scriptorium can include it in prompt
|
||||
execution metadata.
|
||||
to chunk, extract, merge, and normalize requests; LLM-backed modules forward it
|
||||
through their structured completion requests so Scriptorium can include it in
|
||||
prompt execution metadata.
|
||||
|
||||
## Registries And Module Specs
|
||||
|
||||
@@ -83,10 +84,9 @@ Every production module registers a `ModuleSpec` with:
|
||||
- `Provides`: capabilities added after that module runs;
|
||||
- `Requires`: capabilities that must already be available.
|
||||
|
||||
Chunk, extract, and normalize specs may also declare reference slots. Slot
|
||||
Chunk, extract, merge, and normalize specs may also declare reference slots. Slot
|
||||
declarations are available from registry metadata without constructing module
|
||||
instances. Input, merge, validate, and output specs must not declare reference
|
||||
slots.
|
||||
instances. Input, validate, and output specs must not declare reference slots.
|
||||
|
||||
Capability checks prevent incompatible pipeline composition before a run starts.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user