Reuse valid workspace checkpoints on request

This commit is contained in:
2026-07-08 03:02:50 +00:00
parent 1d3a444df8
commit ae9c2e1d5e
14 changed files with 1406 additions and 290 deletions

View File

@@ -21,7 +21,7 @@ ID in config or with `--llm-profile`.
```text
notarius help
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--session-id id] [--reference selector=path] [--without-reference selector]
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--resume] [--session-id id] [--reference selector=path] [--without-reference selector]
notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list --config path/to/config.yml [--json]
```
@@ -41,6 +41,8 @@ Flags:
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
comma-separated and must be non-empty.
- `--resume`: reuse valid workspace checkpoints for this invocation. Requires
`workspace.resume.enabled: true`.
- `--output-dir path`: output root. The run writes to `<path>/<run-id>/`.
Defaults to `./notarius-output`.
- `--diagnostics-dir path`: diagnostics work directory override for this
@@ -140,6 +142,16 @@ go run ./cmd/notarius run dnd-session \
--session-id campaign-17-session-04
```
Use `--resume` to reuse valid checkpoints from a previous compatible
invocation:
```sh
go run ./cmd/notarius run dnd-session \
--config examples/dnd-spells.config.yml \
--input examples/seriatim-minimal-transcript.json \
--resume
```
For durable output, diagnostics, retention, and failure inspection, see
[Operations](operations.md).

View File

@@ -353,8 +353,8 @@ casts still must be present in the source transcript.
- `debug.enabled`: boolean debug artifact setting. Default: `false`.
When `workspace.resume.enabled` is true, runs write stage-owned checkpoint
artifacts under `<workspace.directory>/checkpoints/`. Checkpoint reads and
resume execution are not implemented.
artifacts under `<workspace.directory>/checkpoints/`. `notarius run --resume`
can reuse valid checkpoints from a compatible invocation.
Debug artifact writers are not part of the current workflow.

View File

@@ -77,6 +77,12 @@ through `pipeline.RunInput`. The runner records source, chunk, extract, merge,
and normalize outcomes through that interface. Concrete modules do not receive
workspace paths and do not write checkpoint files directly.
For `run --resume`, the CLI also passes a checkpoint loader. The runner consults
the loader in workflow order and reuses only checkpoints whose manifest schema,
status, identity digest, dependency fingerprints, payload files, and payload
digests validate for the current invocation. Missing or invalid checkpoints fall
back to normal execution and are refreshed by the recorder.
## Registries And Module Specs
`pipeline.Registries` holds concrete constructors for execution. A

View File

@@ -74,6 +74,8 @@ Implemented diagnostics artifacts:
- `resolved-references.json`: resolved reference provenance, including target
stage, lane ID when present, origin, digest, media type, byte size, and
binding source, without reference content.
- `checkpoint-events.json`: checkpoint steps that were reused or executed
during an explicit resume invocation.
- `run-manifest.json`: the same run manifest written to durable output when it
is available, including top-level module metadata when present.
- `warnings.json`: warning list.
@@ -94,14 +96,21 @@ write checkpoints under:
```
Each workflow step owns its own manifest and payload files. There is no
root-level checkpoint summary. Current runs write checkpoints for inspection and
future recovery support only; the CLI does not read checkpoints or skip work.
root-level checkpoint summary. Ordinary `notarius run` invocations execute the
pipeline normally and refresh checkpoints. `notarius run --resume` reuses valid
checkpoints and executes any missing, invalid, or incompatible step normally.
Checkpoint payloads preserve byte content with base64 envelopes, media type,
metadata, warnings, and content digests where applicable. Checkpoints do not
include raw prompts, raw reference contents, raw LLM request payloads, or debug
traces.
A checkpoint is reused only when its workspace schema version, checkpoint
identity digest, step status, dependency fingerprints, payload files, and
payload digests match the current invocation. Changes to input bytes, resolved
pipeline digest, selected lanes, runtime LLM profile override, or materialized
reference digests invalidate reuse.
## Retention
Diagnostics retention is configured with `workspace.diagnostics.retention`,
@@ -156,8 +165,8 @@ directories unless they are part of your own operational policy.
## Operational Limits
Checkpoint writing does not provide resume execution yet. Re-run `notarius run`
after fixing the cause of a failed run.
If `--resume` cannot reuse a checkpoint, Notarius executes that step and writes
a fresh checkpoint when checkpointing is enabled.
Provider retries and timeouts are handled by Scriptorium according to the
selected execution profile. Pipeline module retries are controlled by module