eric/distributor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Clean up upload API documentation roadmap
All checks were successful
ci/woodpecker/tag/release Pipeline was successful
Details

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-08 04:48:06 +00:00
parent 29f01da37b
commit ee6a351960
3 changed files with 34 additions and 297 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
docs/roadmap/api.md
View File

@@ -2,34 +2,24 @@
This document records API work that is not part of the current implementation. Current HTTP upload behavior is documented in `docs/integrations/http-upload.md`, current configuration behavior is documented in `docs/config.md`, and current producer package usage is documented under `docs/consumers/`.
## Implemented HTTP Upload Behavior
## Deferred Upload API Work
The pipeline-scoped HTTP upload API is implemented. The current contract is:
- upload authentication is configured with top-level `upload_tokens`;
- token values resolve through the process environment or `secrets.directory`, never YAML literals;
- bearer tokens authenticate producers and authorize configured upload pipelines through `allow_pipelines`;
- upload requests use `POST /v1/pipelines/{pipeline_id}/upload`;
- missing, malformed, or unknown bearer tokens return `401 Unauthorized`;
- valid tokens that are not allowed for the requested pipeline return `403 Forbidden`;
- `pkg/upload` upload options require `PipelineID`;
- idempotency records are scoped by token id, pipeline id, and idempotency key;
- source manifests remain free of routing, destination, transform, and credential data.
The unscoped upload route no longer accepts uploads. Pipeline query parameters are rejected.
## Deferred API Work
The following topics remain separate roadmap items:
- durable upload status storage;
- durable idempotency records across server restarts;
- in-app public exposure policy;
- built-in TLS termination;
- built-in upload rate limiting.
- Durable upload status storage.
- Durable idempotency records across server restarts.
- Run listing, retry, and cancellation endpoints.
- Long-polling or wait-for-completion workflows.
- Multipart, resumable, or streaming upload protocols.
- Additional archive content negotiation beyond tar and gzip-compressed tar.
- URL-token authentication for constrained clients.
- Upload token lifecycle tooling.
- Mutual TLS or other in-app identity mechanisms.
- In-app TLS termination.
- In-app public exposure policy.
- In-app upload rate limiting.
## Boundaries
- Do not add destination selection to producer manifests.
- Do not let producers specify destination ids, transforms, links, publish policy, or transfer policy through the upload API.
- Keep `http_upload` source-only. A selected pipeline owns destination configuration, transforms, links, transfer policy, and publication behavior.
- Producers do not choose destination ids, destination paths, transforms, links, publish policy, transfer policy, or storage backends through upload requests.
- Source manifests remain free of routing, destination, transform, and credential data.
- `http_upload` remains source-only unless a future design changes that contract.
- Public access policy, TLS termination, and rate limiting belong in deployment infrastructure unless a future design changes that boundary.