eric/distributor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Document pipeline-scoped upload behavior

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-08 04:46:22 +00:00
parent bd5892d1f2
commit 29f01da37b
7 changed files with 103 additions and 201 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
docs/troubleshooting.md
View File

@@ -400,7 +400,7 @@ Reference: [Configuration](config.md#serverhttp).
Symptom: `upload token environment variable ... is not set`, `... is empty`, or `upload token environment variables ... resolve to the same value`.
Likely cause: an `http_upload` source references a missing/empty `token_env`, or two upload pipelines resolve to the same bearer token.
Likely cause: a top-level upload token record references a missing or empty `token_env`, or two token records resolve to the same bearer token.
Diagnostic:
@@ -410,20 +410,20 @@ env | cut -d= -f1 | rg '^<token-variable>$'
ls -l <secrets-directory>/<token-variable>
```
Safe fix: provide one distinct non-empty token value per upload pipeline through the process environment or `secrets.directory`. Do not put literal tokens in YAML.
Safe fix: provide one distinct non-empty token value per upload token record through the process environment or `secrets.directory`. Do not put literal tokens in YAML.
Reference: [Configuration](config.md#http-upload-source-backend).
Reference: [Configuration](config.md#upload_tokens).
## Upload Request Is Unauthorized
Symptom: `POST /upload` returns `401`.
Symptom: `POST /v1/pipelines/<pipeline-id>/upload` returns `401`.
Likely cause: the request lacks `Authorization: Bearer <token>`, has an empty token, or uses a token that does not match any configured upload pipeline.
Likely cause: the request lacks `Authorization: Bearer <token>`, has an empty token, or uses a token that does not match any configured upload token record.
Diagnostic:
```sh
curl -i -X POST http://127.0.0.1:8080/upload \
curl -i -X POST http://127.0.0.1:8080/v1/pipelines/<pipeline-id>/upload \
-H "Authorization: Bearer $DISTRIBUTOR_UPLOAD_TOKEN" \
-H "Content-Type: application/x-tar" \
--data-binary @bundle.tar
@@ -433,11 +433,27 @@ Safe fix: use the token value resolved by the configured `token_env`. Do not inc
Reference: [Operations](operations.md#http-upload-operation).
## Upload Request Is Forbidden
Symptom: `POST /v1/pipelines/<pipeline-id>/upload` returns `403`.
Likely cause: the bearer token is valid, but its configured `allow_pipelines` list does not include the requested upload pipeline.
Diagnostic:
```sh
rg -n 'upload_tokens:|allow_pipelines:|id:' <config-path>
```
Safe fix: request the intended pipeline id, or update the token allowlist to include the configured `http_upload` pipeline that this producer may submit to.
Reference: [Configuration](config.md#upload_tokens).
## Upload Request Is Rejected Before A Run ID
Symptom: `POST /upload` returns `400`, `413`, `415`, or `503`.
Symptom: `POST /v1/pipelines/<pipeline-id>/upload` returns `400`, `413`, `415`, or `503`.
Likely cause: the request included a `pipeline` or `pipeline_id` query, archive content is malformed, the body exceeds size limits, content type is unsupported, or the in-memory upload queue is full.
Likely cause: the request path has an invalid pipeline id, included a `pipeline` or `pipeline_id` query, archive content is malformed, the body exceeds size limits, content type is unsupported, or the in-memory upload queue is full.
Diagnostic:
@@ -447,20 +463,20 @@ tar -tzf bundle.tar.gz
rg -n 'max_upload_size|queue_size|max_concurrency' <config-path>
```
Safe fix: send one valid tar or tar.gz source bundle archive with `Content-Type: application/x-tar`, `application/gzip`, or `application/x-gzip`; remove pipeline query parameters; reduce archive size or raise the configured limit; retry after queue pressure drops.
Safe fix: send one valid tar or tar.gz source bundle archive to `/v1/pipelines/<pipeline-id>/upload` with `Content-Type: application/x-tar`, `application/gzip`, or `application/x-gzip`; remove pipeline query parameters; reduce archive size or raise the configured limit; retry after queue pressure drops.
Reference: [Operations](operations.md#http-upload-operation).
## Upload Idempotency Conflict
Symptom: `POST /upload` returns `409`.
Symptom: `POST /v1/pipelines/<pipeline-id>/upload` returns `409`.
Likely cause: the request reused an `Idempotency-Key` for the same authenticated pipeline with a different source manifest, or another request with the same key is still being staged before its manifest is known.
Likely cause: the request reused an `Idempotency-Key` for the same token id and pipeline id with a different source manifest, or another request with the same key is still being staged before its manifest is known.
Diagnostic:
```sh
curl -i -X POST http://127.0.0.1:8080/upload \
curl -i -X POST http://127.0.0.1:8080/v1/pipelines/<pipeline-id>/upload \
-H "Authorization: Bearer $DISTRIBUTOR_UPLOAD_TOKEN" \
-H "Content-Type: application/gzip" \
-H "Idempotency-Key: <key>" \
@@ -469,7 +485,7 @@ curl -i -X POST http://127.0.0.1:8080/upload \
Safe fix: if the response includes `"retryable":true`, retry the same upload later with the same key. Otherwise, inspect the producer operation and use the same key only for the same source bundle.
Reference: [HTTP Upload API Contract](integrations/http-upload.md#post-upload).
Reference: [HTTP Upload API Contract](integrations/http-upload.md).
## Upload Status Is Missing