Document producer upload client behavior
This commit is contained in:
@@ -106,6 +106,28 @@ Archive entry rules:
|
||||
|
||||
The uploaded archive size and extracted bundle size are bounded by the selected pipeline's `source.max_upload_size`. Extracted file count is also bounded by the implementation.
|
||||
|
||||
## Go Producer Helper
|
||||
|
||||
Go producers can use `gitea.maximumdirect.net/eric/distributor/pkg/upload` to build or validate source bundles, package them as gzip-compressed tar archives, and submit them to this API:
|
||||
|
||||
```go
|
||||
client, err := upload.NewClient(upload.ClientOptions{
|
||||
Endpoint: "http://127.0.0.1:8080",
|
||||
Token: token,
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
result, err := client.UploadBundle(ctx, upload.UploadBundleOptions{
|
||||
Root: "examples/source-bundle",
|
||||
IdempotencyKey: "reports.example.20260604T120000Z",
|
||||
})
|
||||
```
|
||||
|
||||
`Endpoint` is the server base URL; the package derives `/upload` and `/runs/<run-id>`. `UploadBundle` validates a local bundle by default and uploads only `manifest.json` plus manifest-listed files. `UploadFiles` creates a temporary bundle from explicit `bundle.BundleFile` values before uploading. When `IdempotencyKey` is omitted, the package generates one random 128-bit lowercase hex key for the upload operation and reuses it across retries.
|
||||
|
||||
The helper retries only safe cases: `503 Service Unavailable`, temporary network errors, and ambiguous mid-upload failures. It does not retry after `202 Accepted` and does not retry `400`, `401`, `409`, `413`, or `415`. Bearer token values are redacted from returned errors.
|
||||
|
||||
## Queue And Retention
|
||||
|
||||
`server.http.queue_size` bounds accepted-but-not-started uploads plus uploads being staged. `server.http.max_concurrency` bounds publishing concurrency. The coordinator does not run two uploads for the same pipeline concurrently.
|
||||
@@ -123,5 +145,5 @@ The HTTP API does not expose pipeline selection by request parameter, TLS, publi
|
||||
Before changing this contract, inspect and run:
|
||||
|
||||
```sh
|
||||
go test ./internal/app ./internal/ingest
|
||||
go test ./internal/app ./internal/ingest ./pkg/upload
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user