Add HTTP upload server and serve command

docs/troubleshooting.md

@@ -26,7 +26,10 @@ Safe fix: compare the file to the reference in [configuration](config.md) and re
 
 ## `validate config ... backend ... is unsupported`
 
-Likely cause: a source or destination uses a backend name other than `local`, `ssh`, or `s3`.
+Likely cause: a source or destination uses an unsupported backend name, or a
+command is trying to execute a backend that is valid only for another workflow.
+`run`, `validate`, and `inspect` execute `local`, `ssh`, and `s3` sources.
+`serve` executes `http_upload` sources.
 
 Diagnostic:
 
@@ -34,7 +37,127 @@ Diagnostic:
 rg -n "backend:" <config-path>
 ```
 
-Safe fix: use `backend: local`, `backend: ssh`, or `backend: s3` for executable workflows.
+Safe fix: use `backend: local`, `backend: ssh`, or `backend: s3` for normal
+source and destination workflows. Use `backend: http_upload` only for sources
+handled by `distributor serve`.
+
+## `bind HTTP server ... address already in use`
+
+Likely cause: another process is already listening on `server.http.bind`.
+
+Diagnostic:
+
+```sh
+ss -ltnp | rg '<port>'
+```
+
+Safe fix: stop the conflicting process or configure a different
+`server.http.bind` value. The default bind address is `127.0.0.1:8080`.
+
+## `upload token environment variable ... is not set`
+
+Likely cause: a configured `http_upload` source references `token_env`, but the
+variable is absent from both the real process environment and
+`secrets.directory`.
+
+Diagnostic:
+
+```sh
+env | cut -d= -f1 | rg '^<token-variable>$'
+ls -l <secrets-directory>/<token-variable>
+```
+
+Safe fix: set the real environment variable or create a readable
+secrets-directory file with the same name. Do not place literal token values in
+YAML.
+
+## `upload token environment variables ... resolve to the same value`
+
+Likely cause: two configured `http_upload` pipelines resolve to the same bearer
+token value.
+
+Diagnostic:
+
+```sh
+rg -n 'token_env:' <config-path>
+```
+
+Safe fix: assign a distinct non-empty token value to each `http_upload`
+pipeline. Distributor does not print the duplicate token value.
+
+## `POST /upload` returns `401`
+
+Likely cause: the request is missing `Authorization: Bearer <token>` or the
+token does not match any configured `http_upload` pipeline.
+
+Diagnostic:
+
+```sh
+curl -i -X POST http://127.0.0.1:8080/upload \
+  -H "Authorization: Bearer $DISTRIBUTOR_UPLOAD_TOKEN" \
+  -H "Content-Type: application/x-tar" \
+  --data-binary @bundle.tar
+```
+
+Safe fix: use the token value resolved by the configured `token_env`. Do not
+include token values in logs or tickets.
+
+## `POST /upload` returns `413`
+
+Likely cause: the request body exceeds the selected pipeline's
+`source.max_upload_size` or the default `server.http.max_upload_size`.
+
+Diagnostic:
+
+```sh
+ls -lh bundle.tar bundle.tar.gz
+rg -n 'max_upload_size:' <config-path>
+```
+
+Safe fix: upload a smaller archive, remove unnecessary files from the source
+bundle, or raise the configured upload size limit.
+
+## `POST /upload` returns `415`
+
+Likely cause: the upload uses an unsupported content type. The server accepts
+uncompressed tar and gzip-compressed tar archives only.
+
+Diagnostic:
+
+```sh
+file bundle.tar.gz
+```
+
+Safe fix: send `Content-Type: application/x-tar`, `application/gzip`, or
+`application/x-gzip`, matching the archive format.
+
+## `POST /upload` returns `503`
+
+Likely cause: the in-memory upload queue is full.
+
+Diagnostic:
+
+```sh
+rg -n 'queue_size|max_concurrency' <config-path>
+```
+
+Safe fix: retry after active uploads finish, or increase `server.http.queue_size`
+for the deployment.
+
+## `GET /runs/<run_id>` returns `404`
+
+Likely cause: the run id is wrong, the process restarted, or the completed
+status record expired after `server.http.retention`.
+
+Diagnostic:
+
+```sh
+curl -i http://127.0.0.1:8080/runs/<run-id>
+rg -n 'retention:' <config-path>
+```
+
+Safe fix: use the exact `run_id` returned by `POST /upload`. If status retention
+is too short for operators, increase `server.http.retention`.
 
 ## `--format: format must be text or json`