Implement runner retries and raw validation

This commit is contained in:
2026-07-07 19:19:04 +00:00
parent bcedf19a08
commit cc6b050367
8 changed files with 805 additions and 129 deletions

View File

@@ -228,8 +228,9 @@ Binding fields:
- `module`: module key.
- `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the
Scriptorium prompt default select the profile.
- `retries`: non-negative retry count for runtime stages that support retries.
The current runner preserves this value in resolved config.
- `retries`: non-negative retry count for extra runtime attempts after the
first attempt. The runner applies retries to `chunk`, `extract`, `merge`, and
`normalize` bindings.
- `options`: optional module-specific settings.
- `references`: optional reference bindings. Supported only for `chunk`,
`extract`, `merge`, and `normalize` bindings. `input`, validator, and

View File

@@ -96,8 +96,9 @@ stage, lane ID when present, slot name, origin type and URI, digest, media
type, byte size, and binding source. Reference content is not written to
durable output.
Reference `stage` is `chunk`, `extract`, or `normalize`. `lane_id` is omitted
for chunk references and present for extract and normalize references.
Reference `stage` is `chunk`, `extract`, `merge`, or `normalize`. `lane_id` is
omitted for chunk references and present for extract, merge, and normalize
references.
`validation_status` is `approved` when no raw outputs were rejected and
`rejected` when one or more raw outputs were rejected.

View File

@@ -116,8 +116,9 @@ The runner:
1. validates run input and registries;
2. builds the input adapter and parses the raw input into a source document;
3. validates the source document;
4. builds the chunker and produces source chunks;
5. validates source chunks against framework invariants;
4. builds the chunker and produces source chunks, retrying when configured;
5. validates source chunks against framework invariants and any registered raw
chunk validators;
6. runs each selected artifact lane in sorted resolved order;
7. builds the output encoder and validates logical output file names.
@@ -129,9 +130,8 @@ configured LLM profile, module options, and run metadata. Deterministic and
LLM-backed chunkers use the same contract; provider construction stays outside
chunk modules.
After `Chunk` returns, the runner appends chunker warnings before returning any
chunker error. When chunking succeeds, the runner validates generic chunk
invariants before running extractors:
When chunking succeeds, the runner validates generic chunk invariants before
running extractors:
- chunk IDs must be non-empty and unique in the chunk result;
- each chunk `SourceID` must match the source document ID;
@@ -150,6 +150,10 @@ chunk metadata. Extractors and downstream stages therefore see canonical source
units, while `SourceChunk.Metadata` remains the supported place for
chunker-owned context.
If chunk validation rejects a chunk after configured retries, the runner records
a rejected raw output and skips downstream lane execution. Framework-level
chunking or validation errors that remain after configured retries fail the run.
The framework does not require complete source-unit coverage and does not reject
overlap between different chunks. Stricter policies, such as full coverage or
non-overlap, belong to individual chunk modules when they are part of that
@@ -159,24 +163,37 @@ Within an artifact lane, the runner:
1. builds the extractor, merger, and normalizer;
2. records module manifest metadata when modules provide it;
3. extracts one raw `ExtractOutput` from each chunk;
3. extracts one raw `ExtractOutput` from each accepted chunk, retrying when
configured;
4. fills runner-owned provenance on each extract output, including lane ID,
extractor key, source ID, chunk ID, and chunk index;
5. merges ordered extract outputs into one raw `MergeOutput`;
6. normalizes the merge output into one raw `NormalizeOutput`;
7. appends the normalized raw output to `RunOutput.NormalizeOutputs`.
5. validates raw extract outputs and omits rejected outputs from merge input;
6. merges ordered accepted extract outputs into one raw `MergeOutput`, retrying
when configured;
7. validates raw merge output and skips normalization for rejected merge output;
8. normalizes the accepted merge output into one raw `NormalizeOutput`,
retrying when configured;
9. validates raw normalize output and appends accepted normalized raw output to
`RunOutput.NormalizeOutputs`.
## Validators
The current runner handoff is raw-output based. Extractors, mergers, and
normalizers do not advertise candidate validator chains through their module
interfaces. `RunOutput.Rejected` is reserved for rejected raw outputs when raw
validation is wired into the runner.
interfaces. Runner-side raw validation chains receive the raw module output plus
stage, lane, module, source, and chunk provenance. Empty raw validation chains
approve output by default.
Validator rejection is a non-fatal run outcome: the rejected output is recorded
in `RunOutput.Rejected` and does not pass to the next stage. Validator execution
errors are framework-level errors and retry according to the relevant binding.
## Warnings And Failures
Warnings from chunking, extraction, merging, normalization, and output encoding
are accumulated in `RunOutput.Warnings`.
Warnings from the successful chunking, extraction, merging, and normalization
attempts whose outputs are used are accumulated in `RunOutput.Warnings`, along
with output encoder warnings. Warnings from discarded retry attempts are not
promoted to final warnings.
Errors wrap the operation and module key or lane context. If execution fails
after a manifest exists, the returned manifest is marked `failed` and receives a

View File

@@ -133,7 +133,9 @@ There is no command to resume a failed run. Re-run `notarius run` after fixing
the cause.
Provider retries and timeouts are handled by Scriptorium according to the
selected execution profile. There is no separate CLI retry command.
selected execution profile. Pipeline module retries are controlled by module
binding `retries` values in config for chunk, extract, merge, and normalize.
There is no separate CLI retry command.
Notarius writes local files only. Remote storage and archive management are not
part of the implemented CLI.