Stage uploads before accepting HTTP runs

docs/cli.md

@@ -149,7 +149,8 @@ curl -X POST http://127.0.0.1:8080/upload \
   --data-binary @bundle.tar.gz
 ```
 
-The upload response is accepted asynchronously:
+The upload response is returned after the archive is staged and validated; the
+destination fan-out continues asynchronously:
 
 ```json
 {"run_id":"reports.20260603T120000Z.abcdef12","status":"accepted"}

docs/config.md

@@ -130,12 +130,15 @@ Accepted upload content types:
 - `application/gzip`
 - `application/x-gzip`
 
-Accepted uploads return:
+Accepted uploads return after the archive is staged and validated:
 
 ```json
 {"run_id":"<id>","status":"accepted"}
 ```
 
+Malformed tar or gzip content and invalid staged bundles are rejected before a
+run id is issued.
+
 The run id can be queried through `GET /runs/<run_id>` while the status record
 is retained in memory. Completed records expire after `server.http.retention`;
 expiration also removes committed staged bundle directories for completed

docs/internal/app.md

@@ -87,10 +87,10 @@ prepared.
 ## Upload Coordination
 
 `UploadCoordinator` owns in-memory coordination for asynchronous upload
-processing. It admits uploads for configured `http_upload` pipelines, generates
-run IDs, tracks status records, stages accepted archives through
-`internal/ingest`, and executes the selected pipeline through
-`RunPipelineWithLocalSource`.
+processing. It admits uploads for configured `http_upload` pipelines, reserves
+queue capacity before request-body staging, stages and validates archives
+through `internal/ingest`, tracks accepted status records, and executes the
+selected pipeline through `RunPipelineWithLocalSource`.
 
 Upload run IDs use:
 
@@ -111,10 +111,11 @@ The coordinator records these statuses:
 - `expired`
 
 Admission is bounded by `server.http.queue_size`. Full queues are rejected
-before the upload body is staged. Execution is bounded by
+before the upload body is read. Successfully reserved uploads are staged and
+validated before an accepted run record is created. Execution is bounded by
 `server.http.max_concurrency`, and only one upload for a given pipeline may run
-at a time. Later uploads for the same pipeline remain queued until the active
-run finishes.
+at a time. Later accepted uploads for the same pipeline remain queued until the
+active run finishes.
 
 Completed records retain the final run report or error text until
 `server.http.retention` elapses. Expiration removes completed status records and
@@ -137,15 +138,16 @@ pipeline ids, but not token values.
 Routes:
 
 - `GET /healthz`: returns `200` after config, secrets, tokens, coordinator, and route setup succeed.
-- `POST /upload`: accepts authenticated tar and tar.gz archives and returns an accepted run id.
+- `POST /upload`: stages and validates an authenticated tar or tar.gz archive, then returns an accepted run id.
 - `GET /runs/<run_id>`: returns the current in-memory upload status record or `404`.
 
 The upload token maps to exactly one configured pipeline. Producers do not
 submit pipeline ids, and submitted `pipeline` or `pipeline_id` query values are
-rejected. Full queues are rejected before the request body is read. Oversized
-uploads, unsupported content types, invalid bearer tokens, full queues, and
-unknown status records are mapped to stable HTTP status codes without returning
-secret token values.
+rejected. Full queues are rejected before the request body is read. Malformed
+archives and invalid staged bundles are rejected before a run id is issued.
+Oversized uploads, unsupported content types, invalid bearer tokens, full
+queues, and unknown status records are mapped to stable HTTP status codes
+without returning secret token values.
 
 ## Coordination
 
@@ -201,7 +203,7 @@ Run helpers are grouped by responsibility:
 - `run_warnings.go`: secret and SSH warning records.
 - `run_notify.go`: notification event projection and action filtering.
 - `run_coordinator.go`: in-memory run admission, run IDs, status records, and duplicate-run errors.
-- `upload_coordinator.go`: in-memory upload admission, queueing, status tracking, staging handoff, and staged-source execution.
+- `upload_coordinator.go`: in-memory upload admission, queue reservation, staging handoff, status tracking, queueing, and staged-source execution.
 - `upload_http.go`: HTTP upload authentication, routes, JSON response projection, and HTTP error mapping.
 - `serve.go`: config/secrets loading and HTTP server startup.
 - `backends.go`: app-level backend factory wiring.

docs/internal/ingest.md

@@ -6,6 +6,9 @@
 
 ## Archive staging
 
+`ValidateContentType` owns accepted upload content-type policy for archive
+staging callers.
+
 `StageArchive` accepts one upload body, content type, pipeline staging path, run id, and explicit size and file-count limits. It writes the request body to temporary storage while enforcing the configured upload size limit, extracts the archive into temporary local storage, validates the extracted source bundle, and then commits the validated bundle to:
 
 ```text

docs/operations.md

@@ -71,9 +71,9 @@ go run ./cmd/distributor validate --config examples/local-publish.yml --pipeline
 ## HTTP Upload Workflow
 
 `distributor serve` runs the HTTP upload API for pipelines whose source backend
-is `http_upload`. Each upload token maps to one configured pipeline, and each
-accepted archive is staged, validated, and published through the same
-destination fan-out path used by local source runs.
+is `http_upload`. Each upload token maps to one configured pipeline. A valid
+archive is staged and validated before a run id is returned, then published
+through the same destination fan-out path used by local source runs.
 
 Minimal local HTTP upload configuration:
 
@@ -119,7 +119,7 @@ curl -X POST http://127.0.0.1:8080/upload \
   --data-binary @bundle.tar.gz
 ```
 
-Successful admission returns a run id:
+Successful staging and admission returns a run id:
 
 ```json
 {"run_id":"reports.20260603T120000Z.abcdef12","status":"accepted"}
@@ -136,6 +136,9 @@ or error details on failure. Status is memory-only and expires after
 `server.http.retention`; completed staged bundle directories are removed on
 expiry. Restarting the process clears upload status and queue state.
 
+Malformed archives and invalid source bundles are rejected by `POST /upload`
+before a run id is issued.
+
 Use `GET /healthz` for readiness after config and tokens load:
 
 ```sh

docs/troubleshooting.md

@@ -102,6 +102,24 @@ curl -i -X POST http://127.0.0.1:8080/upload \
 Safe fix: use the token value resolved by the configured `token_env`. Do not
 include token values in logs or tickets.
 
+## `POST /upload` returns `400`
+
+Likely cause: the archive content is malformed, the gzip body is invalid, the
+tar body cannot be extracted safely, or the extracted source bundle fails
+manifest and file validation.
+
+Diagnostic:
+
+```sh
+tar -tf bundle.tar
+tar -tzf bundle.tar.gz
+go run ./cmd/distributor validate <extracted-bundle-root>
+```
+
+Safe fix: rebuild the tar or tar.gz archive from one complete source bundle
+root. The archive must contain exactly one root-level `manifest.json`, and every
+manifest-listed file must exist as a regular file with matching size and digest.
+
 ## `POST /upload` returns `413`
 
 Likely cause: the request body exceeds the selected pipeline's