Reuse valid workspace checkpoints on request
This commit is contained in:
14
docs/cli.md
14
docs/cli.md
@@ -21,7 +21,7 @@ ID in config or with `--llm-profile`.
|
||||
|
||||
```text
|
||||
notarius help
|
||||
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--session-id id] [--reference selector=path] [--without-reference selector]
|
||||
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--resume] [--session-id id] [--reference selector=path] [--without-reference selector]
|
||||
notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
|
||||
notarius pipelines list --config path/to/config.yml [--json]
|
||||
```
|
||||
@@ -41,6 +41,8 @@ Flags:
|
||||
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
|
||||
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
|
||||
comma-separated and must be non-empty.
|
||||
- `--resume`: reuse valid workspace checkpoints for this invocation. Requires
|
||||
`workspace.resume.enabled: true`.
|
||||
- `--output-dir path`: output root. The run writes to `<path>/<run-id>/`.
|
||||
Defaults to `./notarius-output`.
|
||||
- `--diagnostics-dir path`: diagnostics work directory override for this
|
||||
@@ -140,6 +142,16 @@ go run ./cmd/notarius run dnd-session \
|
||||
--session-id campaign-17-session-04
|
||||
```
|
||||
|
||||
Use `--resume` to reuse valid checkpoints from a previous compatible
|
||||
invocation:
|
||||
|
||||
```sh
|
||||
go run ./cmd/notarius run dnd-session \
|
||||
--config examples/dnd-spells.config.yml \
|
||||
--input examples/seriatim-minimal-transcript.json \
|
||||
--resume
|
||||
```
|
||||
|
||||
For durable output, diagnostics, retention, and failure inspection, see
|
||||
[Operations](operations.md).
|
||||
|
||||
|
||||
@@ -353,8 +353,8 @@ casts still must be present in the source transcript.
|
||||
- `debug.enabled`: boolean debug artifact setting. Default: `false`.
|
||||
|
||||
When `workspace.resume.enabled` is true, runs write stage-owned checkpoint
|
||||
artifacts under `<workspace.directory>/checkpoints/`. Checkpoint reads and
|
||||
resume execution are not implemented.
|
||||
artifacts under `<workspace.directory>/checkpoints/`. `notarius run --resume`
|
||||
can reuse valid checkpoints from a compatible invocation.
|
||||
|
||||
Debug artifact writers are not part of the current workflow.
|
||||
|
||||
|
||||
@@ -77,6 +77,12 @@ through `pipeline.RunInput`. The runner records source, chunk, extract, merge,
|
||||
and normalize outcomes through that interface. Concrete modules do not receive
|
||||
workspace paths and do not write checkpoint files directly.
|
||||
|
||||
For `run --resume`, the CLI also passes a checkpoint loader. The runner consults
|
||||
the loader in workflow order and reuses only checkpoints whose manifest schema,
|
||||
status, identity digest, dependency fingerprints, payload files, and payload
|
||||
digests validate for the current invocation. Missing or invalid checkpoints fall
|
||||
back to normal execution and are refreshed by the recorder.
|
||||
|
||||
## Registries And Module Specs
|
||||
|
||||
`pipeline.Registries` holds concrete constructors for execution. A
|
||||
|
||||
@@ -74,6 +74,8 @@ Implemented diagnostics artifacts:
|
||||
- `resolved-references.json`: resolved reference provenance, including target
|
||||
stage, lane ID when present, origin, digest, media type, byte size, and
|
||||
binding source, without reference content.
|
||||
- `checkpoint-events.json`: checkpoint steps that were reused or executed
|
||||
during an explicit resume invocation.
|
||||
- `run-manifest.json`: the same run manifest written to durable output when it
|
||||
is available, including top-level module metadata when present.
|
||||
- `warnings.json`: warning list.
|
||||
@@ -94,14 +96,21 @@ write checkpoints under:
|
||||
```
|
||||
|
||||
Each workflow step owns its own manifest and payload files. There is no
|
||||
root-level checkpoint summary. Current runs write checkpoints for inspection and
|
||||
future recovery support only; the CLI does not read checkpoints or skip work.
|
||||
root-level checkpoint summary. Ordinary `notarius run` invocations execute the
|
||||
pipeline normally and refresh checkpoints. `notarius run --resume` reuses valid
|
||||
checkpoints and executes any missing, invalid, or incompatible step normally.
|
||||
|
||||
Checkpoint payloads preserve byte content with base64 envelopes, media type,
|
||||
metadata, warnings, and content digests where applicable. Checkpoints do not
|
||||
include raw prompts, raw reference contents, raw LLM request payloads, or debug
|
||||
traces.
|
||||
|
||||
A checkpoint is reused only when its workspace schema version, checkpoint
|
||||
identity digest, step status, dependency fingerprints, payload files, and
|
||||
payload digests match the current invocation. Changes to input bytes, resolved
|
||||
pipeline digest, selected lanes, runtime LLM profile override, or materialized
|
||||
reference digests invalidate reuse.
|
||||
|
||||
## Retention
|
||||
|
||||
Diagnostics retention is configured with `workspace.diagnostics.retention`,
|
||||
@@ -156,8 +165,8 @@ directories unless they are part of your own operational policy.
|
||||
|
||||
## Operational Limits
|
||||
|
||||
Checkpoint writing does not provide resume execution yet. Re-run `notarius run`
|
||||
after fixing the cause of a failed run.
|
||||
If `--resume` cannot reuse a checkpoint, Notarius executes that step and writes
|
||||
a fresh checkpoint when checkpointing is enabled.
|
||||
|
||||
Provider retries and timeouts are handled by Scriptorium according to the
|
||||
selected execution profile. Pipeline module retries are controlled by module
|
||||
|
||||
Reference in New Issue
Block a user