From a520cfdcc86bd1a45d2a629f3e9ff153ce6e98d1 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 4 Sep 2026 16:16:19 +0000 Subject: [PATCH] Close analyze checkpointing workstream --- docs/internal/manifest.md | 6 ++++++ docs/internal/stage-analyze.md | 11 +++++++++++ docs/operations.md | 6 ++++++ docs/roadmap/implementation.md | 2 +- docs/roadmap/refactoring.md | 2 +- docs/troubleshooting.md | 28 ++++++++++++++++++++++++++++ internal/stage/analyze_plan_test.go | 24 ++++++++++++++++++++++++ 7 files changed, 77 insertions(+), 2 deletions(-) diff --git a/docs/internal/manifest.md b/docs/internal/manifest.md index 5ecadbc..21a9fe7 100644 --- a/docs/internal/manifest.md +++ b/docs/internal/manifest.md @@ -119,6 +119,8 @@ the last committed collection before constructing the active failed record. Newly materialized canonical bytes therefore remain incidental and cannot become current authority. A later ordinary run reconciles the durable records, reuses earlier current artifacts, and resumes with the failed or stale work. +The same recovery applies after process interruption: the aggregate record may +be non-successful, but every completed artifact checkpoint remains reusable. ## Remote Commit Manifest @@ -151,6 +153,10 @@ establish a current immutable snapshot. Manifest replacement provides atomic visibility, not a crash- or power-loss durability guarantee. Callers reload after a returned save error before retrying. +An analyze checkpoint error leaves the prior manifest as per-artifact authority. +The runner then attempts to persist the bounded failed/stale retry collection; +if that terminal save also fails, its error is joined with the originating +failure and the next invocation must reconcile from the last readable manifest. ## Execution Semantics diff --git a/docs/internal/stage-analyze.md b/docs/internal/stage-analyze.md index 8576149..5f27f62 100644 --- a/docs/internal/stage-analyze.md +++ b/docs/internal/stage-analyze.md @@ -104,6 +104,10 @@ Supported source families: prerequisites remain available through the runtime catalog without invoking Scriptorium; newly produced prerequisites enter that catalog with the same contract, checksum, and size identity used for persisted current evidence. +- before the first adapter call, checkpoints the reconciled starting collection + when it removed configuration records or cleared obsolete authority. After + each successful artifact, checkpoints the complete collection before making + that artifact available to the next scheduled dependent. - keeps adapter output in the invocation's run-local analyze directory until it is a safe, non-empty, bounded regular file with a calculated checksum and complete output contract. Canonical replacement uses the shared atomic file @@ -146,6 +150,10 @@ Supported source families: - unrelated current records survive a partial failure. Old canonical bytes for the failed artifact and newly materialized bytes whose checkpoint fails are incidental, not current evidence. +- a checkpoint failure stops the queue immediately. The in-memory retry + collection starts from the last successful checkpoint, marks the active + artifact failed without output authority, and stales its current transitive + dependents. - the runner's ordinary error path marks aggregate analyze failed, clears its aggregate outputs, invalidates publish and notify through the application dependency relation, and persists the live retry collection. Terminal-save @@ -164,6 +172,9 @@ Supported source families: - output provenance and metadata are deterministic per execution. - a canonical file without current per-artifact manifest evidence is never promoted to current state. +- an ordinary retry after interruption or partial failure reuses every valid + earlier checkpoint and schedules only failed, stale, missing, or otherwise + non-current work in dependency order. ## Related Contracts And Tests diff --git a/docs/operations.md b/docs/operations.md index bba1b1c..cf6ae61 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -210,6 +210,12 @@ analysis evidence, regenerate the affected target instead of copying an older canonical file into place or editing the manifest. See [Troubleshooting: Analysis artifact evidence is not current](./troubleshooting.md#analysis-artifact-evidence-is-not-current). +Analyze checkpoints each completed configured artifact before starting later +artifact work. After a partial failure or interruption, retry normally: valid +earlier checkpoints are reused and execution resumes with failed, stale, or +missing work in dependency order. `--force` is unnecessary unless the operator +intends to replace otherwise-current artifacts. + ## Artifact Selection For a configured artifact family, selecting its family key expands to every diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index f18b9b4..60db712 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -383,7 +383,7 @@ validation, application, and rollback paths. ## Stage 5 — Verify Force Semantics, Align Documentation, And Close Workstream 6 -**Status:** Pending +**Status:** Completed ### Goal diff --git a/docs/roadmap/refactoring.md b/docs/roadmap/refactoring.md index 09e3456..8d60ad7 100644 --- a/docs/roadmap/refactoring.md +++ b/docs/roadmap/refactoring.md @@ -264,7 +264,7 @@ do not require process-global substitution. ### Workstream 6 — Per-artifact analyze checkpointing and recovery -**Status:** Pending +**Status:** Complete **Audit sources:** H04, H14, H15, H36, H37, C02, and C03. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 90294a8..271cc65 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -428,6 +428,34 @@ narratio analyze 2026-04-04 --artifacts session_recap Relevant references: [Operations: Artifact Selection](./operations.md#artifact-selection) and [Artifact Internals](./internal/artifacts.md#resolution-rules). +## Analysis manifest checkpoint failed + +Symptom: + +- analyze reports that it could not save the session manifest after completing + an artifact. + +Likely causes: + +- the workspace is unwritable or out of space; +- the manifest path or its parent directory has unsafe permissions or type; or +- a transient local filesystem error interrupted atomic manifest replacement. + +Safe fix: + +- correct the reported filesystem or permission problem; +- inspect `session artifacts` to identify the last durable current records; and +- rerun analyze normally. Narratio reuses earlier successful checkpoints and + retries the failed or non-current artifact and its required dependents. + +Do not treat a newly created canonical artifact file as current when its +checkpoint failed, and do not edit the manifest to advertise it. If the error +recurs, preserve the manifest and run-local diagnostics before repairing the +workspace. + +Relevant references: [Operations: Stage Execution and Continuation Behavior](./operations.md#stage-execution-and-continuation-behavior) +and [Manifest Internals](./internal/manifest.md#persistence-semantics). + ## Built-in transcript or bounds source is unavailable Symptom: diff --git a/internal/stage/analyze_plan_test.go b/internal/stage/analyze_plan_test.go index b07c164..afe6b2a 100644 --- a/internal/stage/analyze_plan_test.go +++ b/internal/stage/analyze_plan_test.go @@ -62,6 +62,30 @@ func TestPlanAnalyzeWorkSelectionAndDependencyClosure(t *testing.T) { }, wantTargets: []string{"alpha", "zeta"}, wantExecution: []string{"alpha", "zeta"}, }, + { + name: "ordinary default selection reuses every current artifact", + configured: map[string]config.ScriptoriumArtifactConfig{ + "alpha": {Enabled: true}, "zeta": {Enabled: true}, "disabled": {Enabled: false}, + }, + reasons: map[string]analyzeReconciliationReason{ + "alpha": analyzeReconciliationCurrent, "zeta": analyzeReconciliationCurrent, + "disabled": analyzeReconciliationCurrent, + }, + wantTargets: []string{"alpha", "zeta"}, wantReused: []string{"alpha", "zeta"}, + }, + { + name: "unqualified force reruns every enabled artifact", + configured: map[string]config.ScriptoriumArtifactConfig{ + "alpha": {Enabled: true}, "zeta": {Enabled: true}, "disabled": {Enabled: false}, + }, + force: true, + reasons: map[string]analyzeReconciliationReason{ + "alpha": analyzeReconciliationCurrent, "zeta": analyzeReconciliationCurrent, + "disabled": analyzeReconciliationCurrent, + }, + wantTargets: []string{"alpha", "zeta"}, wantExecution: []string{"alpha", "zeta"}, + wantForced: []string{"alpha", "zeta"}, + }, { name: "partial selection closes over nested dependencies", configured: map[string]config.ScriptoriumArtifactConfig{