Write workspace checkpoints during runs
This commit is contained in:
@@ -352,8 +352,11 @@ casts still must be present in the source transcript.
|
||||
- `resume.enabled`: boolean resume checkpointing setting. Default: `false`.
|
||||
- `debug.enabled`: boolean debug artifact setting. Default: `false`.
|
||||
|
||||
The current run workflow uses workspace settings for diagnostics configuration.
|
||||
Checkpoint and debug artifact writers are not part of the current workflow.
|
||||
When `workspace.resume.enabled` is true, runs write stage-owned checkpoint
|
||||
artifacts under `<workspace.directory>/checkpoints/`. Checkpoint reads and
|
||||
resume execution are not implemented.
|
||||
|
||||
Debug artifact writers are not part of the current workflow.
|
||||
|
||||
## Diagnostics
|
||||
|
||||
|
||||
@@ -42,9 +42,11 @@ production modules.
|
||||
- `internal/framework/contracts`: interfaces and request/result structs for
|
||||
input adapters, chunkers, extractors, mergers, normalizers, validators, output
|
||||
encoders, and structured LLM clients.
|
||||
- `internal/framework/checkpoint`: workspace-backed checkpoint recorder and
|
||||
checkpoint payload envelope serialization.
|
||||
- `internal/framework/pipeline`: module registries, module specs, profile
|
||||
resolution, capability checks, run orchestration, warnings, validation, and
|
||||
manifest population.
|
||||
resolution, capability checks, run orchestration, checkpoint recorder
|
||||
boundaries, warnings, validation, and manifest population.
|
||||
- `internal/framework/llm`: Scriptorium-backed structured-output client,
|
||||
prompt/schema asset registry, scheduler, schema registry, and secret
|
||||
redaction.
|
||||
|
||||
@@ -71,6 +71,12 @@ to chunk, extract, merge, and normalize requests; LLM-backed modules forward it
|
||||
through their structured completion requests so Scriptorium can include it in
|
||||
prompt execution metadata.
|
||||
|
||||
When workspace resume checkpointing is enabled, the CLI constructs a checkpoint
|
||||
recorder after pipeline resolution and reference materialization and passes it
|
||||
through `pipeline.RunInput`. The runner records source, chunk, extract, merge,
|
||||
and normalize outcomes through that interface. Concrete modules do not receive
|
||||
workspace paths and do not write checkpoint files directly.
|
||||
|
||||
## Registries And Module Specs
|
||||
|
||||
`pipeline.Registries` holds concrete constructors for execution. A
|
||||
|
||||
@@ -84,6 +84,24 @@ Implemented diagnostics artifacts:
|
||||
`source-document.json` is supported by the diagnostics writer but is not written
|
||||
by the current CLI run workflow.
|
||||
|
||||
## Checkpoints
|
||||
|
||||
When `workspace.resume.enabled: true` and `workspace.directory` is set, runs
|
||||
write checkpoints under:
|
||||
|
||||
```text
|
||||
<workspace.directory>/checkpoints/<pipeline-id>/<input-key>-<source-digest>/<pipeline-digest>/
|
||||
```
|
||||
|
||||
Each workflow step owns its own manifest and payload files. There is no
|
||||
root-level checkpoint summary. Current runs write checkpoints for inspection and
|
||||
future recovery support only; the CLI does not read checkpoints or skip work.
|
||||
|
||||
Checkpoint payloads preserve byte content with base64 envelopes, media type,
|
||||
metadata, warnings, and content digests where applicable. Checkpoints do not
|
||||
include raw prompts, raw reference contents, raw LLM request payloads, or debug
|
||||
traces.
|
||||
|
||||
## Retention
|
||||
|
||||
Diagnostics retention is configured with `workspace.diagnostics.retention`,
|
||||
@@ -138,8 +156,8 @@ directories unless they are part of your own operational policy.
|
||||
|
||||
## Operational Limits
|
||||
|
||||
There is no command to resume a failed run. Re-run `notarius run` after fixing
|
||||
the cause.
|
||||
Checkpoint writing does not provide resume execution yet. Re-run `notarius run`
|
||||
after fixing the cause of a failed run.
|
||||
|
||||
Provider retries and timeouts are handled by Scriptorium according to the
|
||||
selected execution profile. Pipeline module retries are controlled by module
|
||||
|
||||
Reference in New Issue
Block a user