Write workspace checkpoints during runs

This commit is contained in:
2026-07-08 02:46:38 +00:00
parent f044c00a7c
commit 1d3a444df8
11 changed files with 1258 additions and 9 deletions

View File

@@ -352,8 +352,11 @@ casts still must be present in the source transcript.
- `resume.enabled`: boolean resume checkpointing setting. Default: `false`.
- `debug.enabled`: boolean debug artifact setting. Default: `false`.
The current run workflow uses workspace settings for diagnostics configuration.
Checkpoint and debug artifact writers are not part of the current workflow.
When `workspace.resume.enabled` is true, runs write stage-owned checkpoint
artifacts under `<workspace.directory>/checkpoints/`. Checkpoint reads and
resume execution are not implemented.
Debug artifact writers are not part of the current workflow.
## Diagnostics

View File

@@ -42,9 +42,11 @@ production modules.
- `internal/framework/contracts`: interfaces and request/result structs for
input adapters, chunkers, extractors, mergers, normalizers, validators, output
encoders, and structured LLM clients.
- `internal/framework/checkpoint`: workspace-backed checkpoint recorder and
checkpoint payload envelope serialization.
- `internal/framework/pipeline`: module registries, module specs, profile
resolution, capability checks, run orchestration, warnings, validation, and
manifest population.
resolution, capability checks, run orchestration, checkpoint recorder
boundaries, warnings, validation, and manifest population.
- `internal/framework/llm`: Scriptorium-backed structured-output client,
prompt/schema asset registry, scheduler, schema registry, and secret
redaction.

View File

@@ -71,6 +71,12 @@ to chunk, extract, merge, and normalize requests; LLM-backed modules forward it
through their structured completion requests so Scriptorium can include it in
prompt execution metadata.
When workspace resume checkpointing is enabled, the CLI constructs a checkpoint
recorder after pipeline resolution and reference materialization and passes it
through `pipeline.RunInput`. The runner records source, chunk, extract, merge,
and normalize outcomes through that interface. Concrete modules do not receive
workspace paths and do not write checkpoint files directly.
## Registries And Module Specs
`pipeline.Registries` holds concrete constructors for execution. A

View File

@@ -84,6 +84,24 @@ Implemented diagnostics artifacts:
`source-document.json` is supported by the diagnostics writer but is not written
by the current CLI run workflow.
## Checkpoints
When `workspace.resume.enabled: true` and `workspace.directory` is set, runs
write checkpoints under:
```text
<workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-digest>/<pipeline-digest>/
```
Each workflow step owns its own manifest and payload files. There is no
root-level checkpoint summary. Current runs write checkpoints for inspection and
future recovery support only; the CLI does not read checkpoints or skip work.
Checkpoint payloads preserve byte content with base64 envelopes, media type,
metadata, warnings, and content digests where applicable. Checkpoints do not
include raw prompts, raw reference contents, raw LLM request payloads, or debug
traces.
## Retention
Diagnostics retention is configured with `workspace.diagnostics.retention`,
@@ -138,8 +156,8 @@ directories unless they are part of your own operational policy.
## Operational Limits
There is no command to resume a failed run. Re-run `notarius run` after fixing
the cause.
Checkpoint writing does not provide resume execution yet. Re-run `notarius run`
after fixing the cause of a failed run.
Provider retries and timeouts are handled by Scriptorium according to the
selected execution profile. Pipeline module retries are controlled by module