Make remote publish locks generation-safe

This commit is contained in:
2026-08-10 20:17:54 +00:00
parent 361dbb4ca8
commit 0cf2cbfeb3
22 changed files with 560 additions and 101 deletions

View File

@@ -9,7 +9,8 @@ Upload run/session outputs to object storage and atomically advance remote curre
- successful preceding stages from the [canonical stage set](overview.md#pipeline-stage-set)
- invocation-scoped run files
- resolved publish output rules
- effective publish locks (static + remote merged lock set)
- effective publish locks (static + remote merged lock set), revalidated at the
remote commit boundary
- durable previous-session cache files when present
## Outputs
@@ -44,6 +45,9 @@ Exact remote placement and the operator workflow belong in
- uploads and verifies every declared immutable object and the commit manifest;
- updates `current/commit-pointer.json` exactly once, last; and
- does not write the legacy `current/manifest.json` or `current/run_id.txt` pair.
- rechecks remote lock state immediately before the pointer update. A newly
committed lock aborts selection, leaving any uploaded immutable attempt
unselected.
## Metadata Signals
@@ -66,7 +70,8 @@ Includes counts/lists for:
undeclared entries are rejected or ignored before uploads begin.
- run-local diagnostics, including Notarius receipt and stderr files, are
archived only when recorded by the run manifest.
- publish locks are not overridden by `--force`.
- publish locks are not overridden by `--force`; remote locks are revalidated
immediately before current-state selection.
The commit boundary and cleanup gate are normative architecture invariants; see
[Architecture](../policy/architecture.md#publish-commit-boundary).

View File

@@ -12,8 +12,10 @@ operator-selected storage fields and credential mechanisms belong in
`storage.ObjectStore` interface:
- `List(ctx, prefix)`
- `Read(ctx, key)` returns an object body and the generation observed with it
- `Download(ctx, key, localPath)`
- `Upload(ctx, localPath, key, opts)`
- `UploadConditional(ctx, source, key, opts, condition)`
- `Exists(ctx, key)`
Key invariant:
@@ -31,8 +33,13 @@ not own discovery, defaults, or configuration validation.
- normalizes object keys.
- `List` paginates and returns normalized `ObjectInfo`.
- A truncated S3 listing must supply a new, non-empty continuation token;
otherwise listing fails with bucket and prefix context instead of looping.
- `Download` writes local files with parent directory creation.
- `Upload` streams local file and returns remote metadata.
- `Read` binds a returned body to its S3 ETag. `UploadConditional` maps an ETag
match or absence precondition directly to the provider request and reports a
failed precondition without performing a local check-then-write replacement.
- `Exists` maps not-found responses to `false`.
## Invariants

View File

@@ -236,7 +236,14 @@ Effective lock rules:
- static and remote locks are merged;
- static locks win on source collisions;
- locked outputs are intentional skips;
- lock add/remove commands mutate only remote lock state.
- lock add/remove commands mutate only remote lock state through generation-bound
conditional writes. A command retries a bounded number of concurrent
conflicts while its invocation context remains active, so it never replaces a
different lock-document generation; and
- a publish re-reads remote locks immediately before it writes the current
commit pointer. A lock committed before that recheck prevents selecting the
new snapshot, even though its already-uploaded immutable objects may remain
available for a later retry.
Examples:

View File

@@ -173,7 +173,9 @@ the immutable commit manifest, and finally the current commit pointer.
`current/commit-pointer.json` is the sole mutable selector and must be written
exactly once, last. Failed, incomplete, skipped, or uncommitted publish attempts
must not be presented as current remote state. Publish locks remain authoritative
and are not bypassed by a forced run.
and are not bypassed by a forced run. Mutable remote locks use provider-enforced
generation preconditions and are revalidated immediately before pointer
selection; loss of that check leaves the prior committed snapshot current.
Automatic local cleanup is permitted only after a successful publish commit,
only when explicitly configured, and only through the path-safety guardrails.

View File

@@ -30,7 +30,7 @@ All stages are pending when this plan is created.
| 12 | Centralize handled terminal-failure persistence | RSK-001, TST-002, SIM-001, COM-001 | Completed |
| 13 | Introduce the immutable remote-commit model and legacy boundary | ARC-003 | Completed |
| 14 | Publish through immutable commits and canonical mappings | COR-004, COR-011, DUP-002, TST-004 | Completed |
| 15 | Make remote locks generation-safe and harden pagination | RSK-005, RSK-014 | Pending |
| 15 | Make remote locks generation-safe and harden pagination | RSK-005, RSK-014 | Completed |
| 16 | Persist retryable post-commit cleanup state | COR-006, COR-007 | Pending |
| 17 | Bind restore/status to a committed snapshot and reject conflicts | COR-008, COR-009, TST-005 | Pending |
| 18 | Serialize restore transitions and make restored paths portable | RSK-006, RSK-008 | Pending |