Complete analyze checkpoint recovery

This commit is contained in:
2026-09-04 16:10:54 +00:00
parent 74977b4064
commit 7febccf550
11 changed files with 416 additions and 490 deletions

View File

@@ -82,11 +82,12 @@ The session-stage collection is the reconciled authority across invocations.
These records remain analyze-owned data inside the fixed stage; they are not
dynamic stages or generic subtasks.
The stage result contract has one analyze-specific projection boundary. On
success, the runner validates and deep-copies the complete reconciled session
collection. Aggregate session outputs are rebuilt in configured-key order from
current session records only. Ordinary stage outputs cannot accompany this
projection, so there is one source of artifact authority.
Analyze checkpoints one complete validated candidate collection after each
successful artifact and before exposing that artifact to later work. It saves a
detached manifest candidate and installs its version, collection, and updated
timestamp into live state only after the store reports success. Aggregate stage
outputs are ordinary result outputs rebuilt in configured-key order from current
session records only.
Successful incremental execution replaces only evaluated artifact records and
preserves valid unrelated current records. Rebuilt outputs are compared by
@@ -95,31 +96,29 @@ the same recomputed fingerprint to remain current, while a changed identity
removes output authority from every unselected transitive dependent by marking
it stale. A partial analyze invocation can therefore succeed while unrelated
configured records remain stale. Existing canonical files never create current
records without validated execution and projection.
records without validated execution and a successful checkpoint.
Aggregate analyze status is deliberately coarser than this collection. Resume
validation may skip a succeeded aggregate record when the selected artifact
closure is current even if unrelated records are stale. Conversely, a stale
aggregate record may cross the ordinary runner boundary and perform zero
Scriptorium calls when reconciliation proves every selected artifact current;
the successful projection then restores the aggregate status.
ordinary successful stage completion then restores the aggregate status.
Analyze may return a projection together with an error. That restricted result
cannot carry ordinary outputs, skip state, aggregate logs, generated configs,
or metadata. The runner persists only the validated per-artifact collections,
then marks the aggregate analyze state failed and invalidates delivery
dependents conservatively. Unrelated current records survive because the
session projection is complete. A malformed projection is not applied, and a
failed session projection save restores the prior per-artifact authority before
terminal failure persistence.
On an artifact error, analyze installs a complete retry collection on the live
manifest before returning. The active record is failed without output, current
transitive dependents are stale, unrelated current records survive, and only
earlier checkpointed completions remain current. The runner's ordinary failure
path marks aggregate analyze failed, clears aggregate outputs, invalidates
delivery dependents, and saves that collection. A terminal session-save failure
is joined with the original adapter, validation, filesystem, cancellation, or
checkpoint cause.
The incremental executor constructs this restricted projection over its
scheduled artifact work. The active record is failed without output,
current transitive dependents are stale, unrelated current records survive, and
only earlier validated and materialized completions remain current in the
reconciled session collection. A terminal session-save failure is joined with
the original adapter, validation, or filesystem cause; a failed projection save
does not turn incidental canonical bytes into manifest authority.
If an artifact checkpoint fails, analyze rolls its working collection back to
the last committed collection before constructing the active failed record.
Newly materialized canonical bytes therefore remain incidental and cannot
become current authority. A later ordinary run reconciles the durable records,
reuses earlier current artifacts, and resumes with the failed or stale work.
## Remote Commit Manifest

View File

@@ -89,9 +89,9 @@ Supported source families:
rebuilt when required, while unrelated disabled artifacts are excluded.
- the work plan carries explicit targets, prerequisite-only work, deterministic
execution and reuse lists, invalidated and removed records, and a cloned
projected record collection. Valid unrelated configured records survive the
projection, removed records are omitted, and incidental files never become
current without manifest-backed regeneration.
reconciled starting collection. Valid unrelated configured records survive
reconciliation, removed records are omitted, and incidental files never
become current without manifest-backed regeneration.
- implements aggregate resume validation by running the same read-only catalog,
fingerprint reconciliation, and work planner used by execution. A succeeded
aggregate record is reusable exactly when the selected closure schedules no
@@ -112,15 +112,16 @@ Supported source families:
- records each successful artifact's freshly computed fingerprint, canonical
relative output path, contract, checksum, size, producer run ID, bounded
Scriptorium provenance, logs, and generated configuration references in the
analyze-owned projection.
analyze-owned manifest collection.
- preserves valid unrelated current records during partial execution. If a
rebuilt output's bytes and contract are unchanged, unselected dependents may
remain current. If that semantic identity changes, unselected transitive
dependents become stale without being executed; dependents included in the
invocation are evaluated in dependency order instead.
- returns one complete reconciled session collection. The runner reconstructs
aggregate session outputs from every current session record; unrelated stale
records do not make an otherwise successful partial invocation fail.
- returns aggregate outputs from every current session record in normalized-key
order after the complete reconciled collection has been checkpointed;
unrelated stale records do not make an otherwise successful partial
invocation fail.
- resolves previous-session sources from local `previous/` cache only.
- runs optional render-debug, then artifact execution.
- validates non-empty output files and materializes canonical outputs.
@@ -135,20 +136,20 @@ Supported source families:
guidance.
- dependency cycles or unavailable required dependencies fail.
- adapter validation failures fail stage.
- a scheduled artifact failure returns the restricted analyze-state projection
- a scheduled artifact failure installs a retry collection on the live manifest
with the active artifact marked `failed`, a bounded error, and no output
authority. Current transitive dependents become stale without execution.
- earlier artifacts remain current only after their run-local output passed
validation and canonical materialization; unattempted later artifacts retain
their reconciled session state.
validation, canonical materialization, and a successful complete-collection
manifest checkpoint; unattempted later artifacts retain their reconciled
session state.
- unrelated current records survive a partial failure. Old canonical bytes for
the failed artifact and newly materialized bytes whose projection cannot be
persisted are incidental, not current evidence.
- the runner applies one valid partial projection at the existing end-of-stage
save boundary before it marks aggregate analyze failed and invalidates publish
and notify through the application dependency relation. Projection-persistence
errors retain the last durable per-artifact authority and are joined with the
original failure context. Per-artifact checkpoint persistence is future work.
the failed artifact and newly materialized bytes whose checkpoint fails are
incidental, not current evidence.
- the runner's ordinary error path marks aggregate analyze failed, clears its
aggregate outputs, invalidates publish and notify through the application
dependency relation, and persists the live retry collection. Terminal-save
errors are joined with the original artifact or checkpoint failure.
## Invariants