109 lines
3.4 KiB
Markdown
109 lines
3.4 KiB
Markdown
# HTTP Upload API Contract
|
|
|
|
Audience: producers, operators, and maintainers integrating with `distributor serve`.
|
|
|
|
`distributor serve` exposes a local HTTP upload API for pipelines whose source backend is `http_upload`. Each bearer token maps to exactly one configured pipeline.
|
|
|
|
## Authentication
|
|
|
|
Uploads authenticate with:
|
|
|
|
```text
|
|
Authorization: Bearer <token>
|
|
```
|
|
|
|
Token values are resolved from the configured `source.token_env` through the process environment or `secrets.directory`. Tokens are not configured as YAML literal values.
|
|
|
|
Requests that include `pipeline` or `pipeline_id` query parameters are rejected. The bearer token selects the pipeline.
|
|
|
|
## Endpoints
|
|
|
|
### `GET /healthz`
|
|
|
|
Returns `200 OK` when the server is running:
|
|
|
|
```json
|
|
{"status":"ok"}
|
|
```
|
|
|
|
### `POST /upload`
|
|
|
|
Accepts one source bundle archive and returns after the archive is staged and validated.
|
|
|
|
Accepted content types:
|
|
|
|
- `application/x-tar`
|
|
- `application/gzip`
|
|
- `application/x-gzip`
|
|
|
|
Successful admission returns `202 Accepted`:
|
|
|
|
```json
|
|
{"run_id":"reports.20260604T120000Z.abcdef12","status":"accepted"}
|
|
```
|
|
|
|
Common error responses:
|
|
|
|
- `400`: pipeline query supplied, archive rejected, malformed archive, or invalid staged source bundle.
|
|
- `401`: missing, empty, or unknown bearer token.
|
|
- `413`: upload body exceeds the selected pipeline size limit.
|
|
- `415`: unsupported content type.
|
|
- `503`: upload queue is full.
|
|
|
|
Error bodies use:
|
|
|
|
```json
|
|
{"error":"<message>"}
|
|
```
|
|
|
|
### `GET /runs/<run-id>`
|
|
|
|
Returns an in-memory status record while retained:
|
|
|
|
```json
|
|
{
|
|
"run_id": "reports.20260604T120000Z.abcdef12",
|
|
"pipeline_id": "reports",
|
|
"status": "succeeded",
|
|
"accepted_at": "2026-06-04T12:00:00Z",
|
|
"started_at": "2026-06-04T12:00:01Z",
|
|
"finished_at": "2026-06-04T12:00:02Z",
|
|
"report": {}
|
|
}
|
|
```
|
|
|
|
Status values are `accepted`, `queued`, `running`, `succeeded`, and `failed`. Failed records include `error`. Succeeded and failed records may include a run report.
|
|
|
|
Unknown, malformed, expired, or process-lost run ids return `404`.
|
|
|
|
## Archive Contract
|
|
|
|
Upload archives must be uncompressed tar or gzip-compressed tar. The archive must contain exactly one root-level `manifest.json` and all manifest-listed files.
|
|
|
|
Archive entry rules:
|
|
|
|
- Paths must be clean relative slash-separated paths.
|
|
- Absolute paths, backslashes, `.` and `..` segments, duplicate files, and nested `manifest.json` entries are rejected.
|
|
- Only directories and regular files are accepted.
|
|
- Symlinks, hardlinks, devices, FIFOs, sockets, and other entry types are rejected.
|
|
|
|
The uploaded archive size and extracted bundle size are bounded by the selected pipeline's `source.max_upload_size`. Extracted file count is also bounded by the implementation.
|
|
|
|
## Queue And Retention
|
|
|
|
`server.http.queue_size` bounds accepted-but-not-started uploads plus uploads being staged. `server.http.max_concurrency` bounds publishing concurrency. The coordinator does not run two uploads for the same pipeline concurrently.
|
|
|
|
Completed status records expire after `server.http.retention`; expiration removes committed staged bundle directories for completed uploads. Server restart clears queue state and status records.
|
|
|
|
## Boundaries
|
|
|
|
The HTTP API does not expose pipeline selection by request parameter, TLS, public routing policy, or durable status storage. Put public access controls, TLS termination, and rate limiting in deployment infrastructure.
|
|
|
|
## Tests
|
|
|
|
Before changing this contract, inspect and run:
|
|
|
|
```sh
|
|
go test ./internal/app ./internal/ingest
|
|
```
|