From 7cb18a1a40267ff18e34b333c924612c994bb1c4 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 10 Aug 2026 03:04:20 +0000 Subject: [PATCH] Reconcile promotion and manifest documentation --- docs/internal/manifest.md | 20 +++++++++++--- docs/internal/stage-extract.md | 6 ++++ docs/internal/workspace.md | 4 ++- docs/operations.md | 6 ++++ docs/roadmap/notarius-extract-stage.md | 38 ++++++++++++++++++-------- docs/troubleshooting.md | 4 +++ 6 files changed, 62 insertions(+), 16 deletions(-) diff --git a/docs/internal/manifest.md b/docs/internal/manifest.md index 4c7d8f1..ec631ab 100644 --- a/docs/internal/manifest.md +++ b/docs/internal/manifest.md @@ -55,11 +55,21 @@ records may include optional contract and external provenance objects; old manifests remain compatible when those fields are absent. A successful forced rerun marks only succeeded downstream session-stage records stale. +Starting an execution clears the current session-stage record's prior outputs, +logs, generated configuration references, and metadata. Failed and skipped +transitions enforce the same clearing rule directly, while success repopulates +only fields returned by the new result. Marking a record stale does not clear +those details because resume validation and diagnosis may still require them +before execution begins. Invocation run manifests remain immutable audit +records of their own outcomes. + A stage may explicitly return a skipped disposition and stable reason. The runner persists that outcome in both manifests, clears older outputs for the -session-stage record, and continues. This self-skip is distinct from deciding -not to execute an already-succeeded stage and is reconsidered on later runs. -Skipped results cannot contain outputs. +session-stage record along with older logs, generated configuration references, +and metadata, then applies any bounded details from the current skip and +continues. This self-skip is distinct from deciding not to execute an +already-succeeded stage and is reconsidered on later runs. Skipped results +cannot contain outputs. When an already-succeeded stage is skipped, the invocation run manifest records the `skip` action and reason. The session manifest deliberately retains its @@ -74,7 +84,9 @@ Run manifest is invocation-scoped audit state. ## Invariants - stage resume/skip decisions are session-manifest driven. -- self-skipped stages do not retain stale outputs and are reconsidered. +- running, failed, and self-skipped stages do not retain result payloads from + an earlier success. +- stale stages retain prior details until replacement execution starts. - force reruns stale downstream succeeded stages. - run manifest does not replace session manifest as progress authority. diff --git a/docs/internal/stage-extract.md b/docs/internal/stage-extract.md index d29a43f..64e34cd 100644 --- a/docs/internal/stage-extract.md +++ b/docs/internal/stage-extract.md @@ -65,6 +65,12 @@ Stdout receipt and stderr diagnostics remain separate. Downstream stages are not given selectable extraction sources unless the complete configured result has passed validation and promotion. +When a replacement attempt begins, the current session-stage record no longer +advertises payload from the previous success. A failed replacement therefore +has no current outputs, logs, generated configuration references, or metadata, +while the earlier invocation manifest and immutable promoted bundle remain +available for audit and recovery. + ## Implementation And Focused Tests - Stage execution, selection, and resume validation: `internal/stage/extract.go`, diff --git a/docs/internal/workspace.md b/docs/internal/workspace.md index e0ac124..81697d6 100644 --- a/docs/internal/workspace.md +++ b/docs/internal/workspace.md @@ -28,7 +28,9 @@ Extraction uses run-local receipt, stderr, and output-root helpers, then promotes the validated external bundle to the unique immutable Notarius bundle path supplied by `internal/artifacts`. `internal/fileops.PromoteDirectory` copies only regular files and directories to a same-filesystem temporary -sibling and atomically renames it without following symlinks or replacing an +sibling. Source traversal uses confined directory handles and identity checks +so replacing an inspected root, directory, or file is rejected rather than +followed. The completed tree is atomically renamed without replacing an existing destination. Exact physical paths belong in [Operations](../operations.md#extraction-workflow). diff --git a/docs/operations.md b/docs/operations.md index 0516721..673f502 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -127,6 +127,12 @@ The directory is immutable once promoted. Configured lanes become the bundle and `index.json` are retained for audit and resume validation but are not selectable or published implicitly. +Starting a replacement clears the previous extraction payload from the current +session-stage record. If that replacement fails or self-skips, the current +record does not fall back to the earlier outputs. The earlier run manifest and +immutable bundle remain available for inspection, but downstream resolution +requires a new current successful extraction record. + Atomic Notarius bundle promotion is supported on Linux, macOS, and Windows. On other operating systems, extraction fails before copying the bundle into a temporary promotion tree because Narratio has no verified atomic no-replace diff --git a/docs/roadmap/notarius-extract-stage.md b/docs/roadmap/notarius-extract-stage.md index dd68b68..d0597a0 100644 --- a/docs/roadmap/notarius-extract-stage.md +++ b/docs/roadmap/notarius-extract-stage.md @@ -2,12 +2,14 @@ ## Status -Complete. +Pending native promotion validation. -The completed implementation sequence is retained in +The implementation sequence is retained in [implementation.md](implementation.md) as decision and delivery history. Current behavior is documented in the canonical configuration, CLI, operations, integration, and internal references linked from this roadmap. +The extraction feature is implemented, but completion remains pending until +atomic promotion has run successfully in native Linux, macOS, and Windows CI. ## Purpose @@ -249,9 +251,10 @@ skipped session-stage and run-stage outcome rather than a successful empty stage. Skipped stages are reconsidered on later invocations, so subsequently enabling Notarius causes extraction to run without requiring force. -A genuine self-skip clears any older outputs for that stage before persisting -the new skipped state. Downstream consumers cannot resolve artifacts retained -from an earlier extraction after the stage is disabled. +A genuine self-skip clears any older outputs, logs, generated configuration +references, and metadata before persisting the new skipped state and its own +bounded metadata. Downstream consumers cannot resolve artifacts retained from +an earlier extraction after the stage is disabled. ### Resume Validation @@ -301,6 +304,14 @@ transitive content of files owned by Notarius. - A failed, skipped, stale, or interrupted extract stage never supplies current extraction sources. +When a replacement attempt starts, the current session-stage record drops the +prior result payload before it is persisted as running. Failed and skipped +transitions enforce the same clearing invariant directly, and a later success +contains only details produced by that attempt. Merely marking a record stale +retains its prior details for resume validation and diagnosis until execution +actually begins. Immutable invocation run manifests and previously promoted +bundles preserve the historical successful result. + ## Bundle Storage And Commit ### Run-Local Execution @@ -319,9 +330,13 @@ artifacts/notarius// ``` The destination is unique and must not already exist. Promotion uses a sibling -temporary directory on the same filesystem, recursively copies only regular -files and directories, rejects symlinks and special files, preserves relative -layout, and renames the completed temporary tree into place. +temporary directory on the same filesystem, traverses the source through +confined directory handles, recursively copies only regular files and +directories, rejects symlinks and special files, and preserves relative +layout. It verifies that each opened root, directory, and file is the same +object that was inspected, rejecting path replacement during traversal. The +completed temporary tree is then renamed into place without replacing a +destination created by another writer. Atomic no-replace promotion is implemented on Linux, macOS, and Windows. On other operating systems extraction reports an unsupported-capability error @@ -558,12 +573,13 @@ for: - strict configuration, defaults, normalization, cross-source validation, and maintained examples; -- stage result dispositions and clearing of self-skipped outputs; +- stage result dispositions and clearing of superseded session-result payloads + when execution starts, fails, or skips while retaining stale diagnostics; - backward-compatible artifact metadata serialization; - exact adapter arguments, streams, cancellation, timeout, exit behavior, receipt/index compatibility, and every path-confinement boundary; -- recursive promotion safety, atomic visibility, cleanup on failure, symlink - rejection, and immutable destination behavior; +- recursive promotion safety, source-replacement detection, atomic visibility, + cleanup on failure, symlink rejection, and immutable destination behavior; - required-lane policy, descriptor compatibility, JSON syntax, checksums, and provenance; - canonical order, single-stage selection, force, staleness, resume diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 134b71e..d83fa46 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -152,6 +152,10 @@ Safe fix: - correct the reported Notarius pipeline, input, provider, or configuration failure and rerun extraction. Do not edit a staged output bundle into place. +After a failed replacement, an older immutable bundle may still exist even +though the current session manifest has no successful extraction payload. This +is expected audit state, not a signal to relink the old bundle manually. + Relevant reference: [Operations: Extraction Workflow](./operations.md#extraction-workflow). ## Atomic Notarius promotion unsupported