Implement ADR-0007
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# ADR-0006: Separate output, cache, and debug state
|
||||
|
||||
**Status:** Accepted
|
||||
**Status:** Superseded by [ADR-0007](0007-separate-checkpoint-recording-from-reuse.md)
|
||||
**Date:** 2026-07-17
|
||||
|
||||
## Context
|
||||
|
||||
50
docs/adr/0007-separate-checkpoint-recording-from-reuse.md
Normal file
50
docs/adr/0007-separate-checkpoint-recording-from-reuse.md
Normal file
@@ -0,0 +1,50 @@
|
||||
# ADR-0007: Separate checkpoint recording from reuse
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-07-19
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0006 made checkpoint I/O conditional on an explicit `--resume` invocation.
|
||||
That policy requires an operator to anticipate the need for recovery before a
|
||||
run begins. A failed ordinary run cannot reuse completed work because it did not
|
||||
record checkpoints.
|
||||
|
||||
Recording reconstructible state and authorizing reuse are separate operational
|
||||
decisions. Recording consumes storage and retains sensitive derived application
|
||||
data, while reuse may change which module operations execute during a run.
|
||||
|
||||
## Decision
|
||||
|
||||
ADR-0006's separation of output, cache, and debug surfaces remains in effect;
|
||||
this decision supersedes only its checkpoint invocation policy.
|
||||
|
||||
Checkpoint recording is controlled by an explicit persistent Boolean
|
||||
configuration setting and remains disabled by default. When recording is
|
||||
enabled, every run records checkpoint transitions and reusable approved stage
|
||||
results.
|
||||
|
||||
Checkpoint loading remains an invocation policy. Only a run with `--resume`
|
||||
loads and reuses compatible completed work. A recording-enabled run without
|
||||
`--resume` executes every stage normally and never loads checkpoints. A resume
|
||||
request while recording is disabled is rejected.
|
||||
|
||||
The existing checkpoint identities, compatibility rules, payload format,
|
||||
filesystem root behavior, and pipeline collaborator contracts remain unchanged.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- Continue coupling reads and writes to `--resume`. This is safe by default but
|
||||
prevents recovery unless resume was anticipated on the earlier run.
|
||||
- Always record checkpoints. This maximizes recovery but creates potentially
|
||||
sensitive state without explicit operator consent.
|
||||
- Add a multi-value recording policy. This preserves the old behavior as an
|
||||
option but adds configuration complexity without a current need.
|
||||
|
||||
## Consequences
|
||||
|
||||
Operators can opt into recovery-ready runs while keeping checkpoint reuse
|
||||
explicit. Enabled successful, rejected, and failed runs may all leave sensitive
|
||||
checkpoint state, so operators remain responsible for access and retention.
|
||||
Disabled configurations perform no checkpoint I/O, and `--resume` requires the
|
||||
operator to enable recording first.
|
||||
@@ -29,7 +29,8 @@ Flags:
|
||||
rules in [Configuration](config.md#discovery).
|
||||
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
|
||||
comma-separated and must be non-empty.
|
||||
- `--resume`: request checkpoint reuse for this invocation. See
|
||||
- `--resume`: request checkpoint reuse for this invocation. Checkpoint recording
|
||||
must be enabled in configuration. See
|
||||
[Operations](operations.md#checkpoint-cache) for prerequisites and reuse
|
||||
behavior.
|
||||
- `--chunk_cache auto|bypass|refresh`: select chunk-plan reuse for this
|
||||
@@ -124,9 +125,9 @@ go run ./cmd/notarius run dnd-session \
|
||||
--session-id campaign-17-session-04
|
||||
```
|
||||
|
||||
The resume flag can be added to an otherwise identical run invocation. It both
|
||||
loads compatible checkpoints and records replacements for work executed by that
|
||||
invocation; without it, the checkpoint root is not used:
|
||||
When `cache.checkpoints.enabled` is `true`, runs record checkpoints whether or
|
||||
not `--resume` is present. Add the resume flag to load and reuse compatible
|
||||
recorded work; using it while checkpoint recording is disabled is an error:
|
||||
|
||||
```sh
|
||||
go run ./cmd/notarius run dnd-session \
|
||||
|
||||
@@ -48,6 +48,7 @@ Built-in defaults:
|
||||
- `cache.chunk_plans.mode`: `auto`
|
||||
- `cache.chunk_plans.directory`: unset, selecting
|
||||
`<os.UserCacheDir>/notarius/chunk-plans`
|
||||
- `cache.checkpoints.enabled`: `false`
|
||||
- `cache.checkpoints.directory`: unset, selecting
|
||||
`<os.UserCacheDir>/notarius/checkpoints`
|
||||
- `debug.directory`: `./notarius-debug`
|
||||
@@ -349,6 +350,7 @@ cache:
|
||||
directory: ""
|
||||
mode: auto
|
||||
checkpoints:
|
||||
enabled: false
|
||||
directory: ""
|
||||
debug:
|
||||
directory: ./notarius-debug
|
||||
@@ -372,9 +374,11 @@ no CLI cache-root override. The defaults are
|
||||
uses an absolute `$XDG_CACHE_HOME` or falls back to `$HOME/.cache`. A relative
|
||||
`XDG_CACHE_HOME` is an error.
|
||||
|
||||
Checkpoint I/O occurs only for `notarius run --resume`. That invocation loads
|
||||
compatible checkpoints and records work it executes. Without `--resume`,
|
||||
Notarius does not resolve, create, load, or record the checkpoint root.
|
||||
`cache.checkpoints.enabled` defaults to `false`. When `true`, every run records
|
||||
checkpoint transitions and reusable approved results. When `false`, Notarius
|
||||
does not resolve or create the checkpoint root, and `--resume` is rejected.
|
||||
The `--resume` flag authorizes loading compatible checkpoints; it does not
|
||||
control recording.
|
||||
|
||||
`debug.directory` chooses a root but never enables debug capture. Its precedence
|
||||
is `--debug-dir`, `NOTARIUS_DEBUG_DIR`, the file value, then the default.
|
||||
@@ -418,13 +422,15 @@ cache:
|
||||
directory: /srv/notarius/chunk-plans
|
||||
mode: auto
|
||||
checkpoints:
|
||||
enabled: true
|
||||
directory: /srv/notarius/state/checkpoints
|
||||
debug:
|
||||
directory: /srv/notarius/debug
|
||||
```
|
||||
|
||||
Run the migrated configuration with `--resume` when checkpoint reuse or
|
||||
recording is wanted, and with `--debug` when a debug bundle is wanted.
|
||||
Run the migrated configuration with `--resume` when checkpoint reuse is wanted,
|
||||
and with `--debug` when a debug bundle is wanted. Enabled checkpoint recording
|
||||
occurs with or without `--resume`.
|
||||
|
||||
The removed fields are `workspace.directory`, `workspace.resume.enabled`,
|
||||
`workspace.debug.enabled`, `workspace.chunk_cache.mode`,
|
||||
@@ -452,7 +458,8 @@ Configuration validation checks:
|
||||
- supported stage-worker keys and an effective extract worker count in the
|
||||
inclusive range `1..concurrency.total_llm`;
|
||||
- non-empty output and debug directories;
|
||||
- a supported chunk-cache mode and state-surface directories without NUL bytes;
|
||||
- a supported chunk-cache mode, Boolean checkpoint enablement, and state-surface
|
||||
directories without NUL bytes;
|
||||
- stale removed fields such as `llm_profiles`.
|
||||
|
||||
Pipeline resolution additionally checks:
|
||||
|
||||
@@ -23,8 +23,9 @@ and atomic publication. Its store is constructed only when the selected mode is
|
||||
not `bypass`.
|
||||
|
||||
`internal/framework/checkpoint` owns checkpoint identity, manifests, payload
|
||||
codecs, loader, and recorder. The CLI constructs both loader and recorder only
|
||||
for a `--resume` invocation. The serialized
|
||||
codecs, loader, and recorder. The CLI constructs a recorder whenever checkpoint
|
||||
recording is enabled and constructs a loader only for a `--resume` invocation.
|
||||
The serialized
|
||||
`workspace_schema_version` identifiers are frozen wire-compatibility fields;
|
||||
they do not describe a current public state surface.
|
||||
|
||||
|
||||
@@ -13,8 +13,9 @@ Notarius uses three independent filesystem surfaces:
|
||||
- debug is explicitly requested inspection data.
|
||||
|
||||
Choose separate roots and access controls for each surface. A normal run writes
|
||||
durable output and may use the chunk-plan cache. It does not create checkpoint
|
||||
or debug state unless its invocation includes `--resume` or `--debug`.
|
||||
durable output, may use the chunk-plan cache, and records checkpoints when
|
||||
`cache.checkpoints.enabled` is true. It does not create debug state unless its
|
||||
invocation includes `--debug`.
|
||||
|
||||
## Output
|
||||
|
||||
@@ -73,10 +74,14 @@ cache:
|
||||
|
||||
## Checkpoint Cache
|
||||
|
||||
Checkpoint state is used only by an invocation with `--resume`. That invocation
|
||||
loads compatible completed work and records checkpoints for work it executes.
|
||||
Without `--resume`, Notarius neither resolves nor creates the checkpoint root,
|
||||
and neither loads nor records checkpoints.
|
||||
Checkpoint recording is controlled by `cache.checkpoints.enabled`, which
|
||||
defaults to `false`. When enabled, every run records running, succeeded, and
|
||||
failed transitions and reusable validator-approved results. Successful,
|
||||
rejected, and failed runs may therefore all leave checkpoint state. The
|
||||
`--resume` flag additionally loads compatible completed work before executing
|
||||
missing or incompatible stages. Without `--resume`, a recording-enabled run
|
||||
never loads checkpoints. Using `--resume` while recording is disabled is an
|
||||
error.
|
||||
|
||||
Checkpoints use the selected root and the existing identity hierarchy:
|
||||
|
||||
@@ -102,6 +107,7 @@ For a Linux service account, independently provision:
|
||||
```yaml
|
||||
cache:
|
||||
checkpoints:
|
||||
enabled: true
|
||||
directory: /var/cache/notarius/checkpoints
|
||||
```
|
||||
|
||||
|
||||
@@ -198,9 +198,10 @@ lifecycle:
|
||||
- debug is explicitly requested inspection data, combining a redacted summary
|
||||
with a detailed trace.
|
||||
|
||||
Chunk plans are keyed only by canonical source digest. Checkpoints are used only
|
||||
for an invocation that explicitly requests resume. Debug is never a cache input
|
||||
and is never created without an explicit request. Pipeline modules receive
|
||||
Chunk plans are keyed only by canonical source digest. Configured checkpoint
|
||||
recording is independent of checkpoint reuse; checkpoints are loaded only for
|
||||
an invocation that explicitly requests resume. Debug is never a cache input and
|
||||
is never created without an explicit request. Pipeline modules receive
|
||||
collaborator interfaces and never physical roots.
|
||||
|
||||
Writes are atomic where practical. Paths for writes, moves, overwrites, and
|
||||
|
||||
Reference in New Issue
Block a user