eric/distributor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Add HTTP upload idempotency support

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-04 14:03:26 +00:00
parent 1a402e6cfa
commit a15722571f
9 changed files with 603 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
docs/internal/app.md
View File

@@ -36,17 +36,19 @@ Run workflows discover and validate source bundles through `internal/bundle`. De
HTTP uploads stage and validate archives before enqueueing a pipeline run with a local staged source root.
Upload idempotency is owned by the upload coordinator. Optional `Idempotency-Key` values are scoped to the authenticated pipeline. The coordinator reserves a key while staging is in progress, records the accepted run id with the validated source manifest identity after staging succeeds, returns the original accepted record for the same key and same manifest, and rejects the same key with a different manifest as a conflict.
## Skip And Resume Behavior
Fan-out destinations are independent. A destination failure is recorded and does not prevent later destinations from being attempted. Dry-run builds plans and reports without destination writes, destination state writes, notifier calls, or SSH known-host persistence.
HTTP upload status is in memory. Accepted jobs move through accepted, queued, running, succeeded, or failed states and expire after configured retention.
HTTP upload status is in memory. Accepted jobs move through accepted, queued, running, succeeded, or failed states and expire after configured retention. Upload idempotency records are also memory-only, expire with the completed status record for their accepted run, and are cleared by process restart.
## Failure Behavior
Runtime setup fails for config load, config validation, secret loading, or credential resolution errors. Source setup failures stop the affected run before destination planning. Destination open, planning, execution, and notification failures are recorded as destination failures where a partial result exists.
HTTP upload startup fails if upload tokens are missing, empty, or duplicated. Upload requests can fail during authentication, content-type validation, queue admission, archive staging, source validation, or later publish execution.
HTTP upload startup fails if upload tokens are missing, empty, or duplicated. Upload requests can fail during authentication, idempotency-key validation, content-type validation, idempotency conflict checks, queue admission, archive staging, source validation, or later publish execution.
## Tests To Inspect
@@ -63,4 +65,5 @@ HTTP upload startup fails if upload tokens are missing, empty, or duplicated. Up
- Fan-out destinations remain independent after a destination-scoped failure.
- Secret values are never printed; warnings may name variables only.
- Upload admission stages and validates a bundle before returning a run id.
- Idempotent upload retries compare normalized source manifest identity, not archive bytes.
- Runtime backend registration remains app-owned.