Add HTTP upload idempotency support

This commit is contained in:
2026-06-04 14:03:26 +00:00
parent 1a402e6cfa
commit a15722571f
9 changed files with 603 additions and 36 deletions

View File

@@ -138,6 +138,16 @@ curl -X POST http://127.0.0.1:8080/upload \
--data-binary @bundle.tar.gz
```
For safe producer retries, include an idempotency key that is stable for the producer operation:
```sh
curl -X POST http://127.0.0.1:8080/upload \
-H "Authorization: Bearer $DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN" \
-H "Content-Type: application/gzip" \
-H "Idempotency-Key: producer.run.20260604T120000Z" \
--data-binary @bundle.tar.gz
```
Accepted uploads return after the archive is staged and validated:
```json
@@ -154,6 +164,8 @@ Status values are `accepted`, `queued`, `running`, `succeeded`, and `failed`. Co
Upload admission is bounded by `server.http.queue_size`. Publication concurrency is bounded by `server.http.max_concurrency`, and the coordinator does not run two uploads for the same pipeline at the same time.
`Idempotency-Key` is optional for raw HTTP clients. When present, it is scoped to the authenticated pipeline. Reusing the same key with the same normalized source manifest returns the original accepted run response and does not enqueue another run. Reusing the key with a different source manifest returns `409 Conflict`. If another request with the same key is still being staged before its manifest is known, the server returns a retryable `409 Conflict`. Idempotency records are memory-only and expire with completed upload status records.
The upload server accepts `application/x-tar`, `application/gzip`, and `application/x-gzip`. Archives are extracted into a temporary staging directory, must contain exactly one root-level `manifest.json`, and must validate as one complete source bundle before a run id is issued. Per-source `max_upload_size` bounds both uploaded archive size and extracted bundle size. The implementation also caps extracted file count.
The default bind address is private loopback. Put TLS, public routing, rate limiting, and external access policy in a reverse proxy or deployment layer.