Document pipeline-scoped upload behavior
This commit is contained in:
@@ -86,11 +86,15 @@ server:
|
||||
queue_size: 16
|
||||
max_concurrency: 1
|
||||
retention: 24h
|
||||
upload_tokens:
|
||||
- id: weather-reporter
|
||||
token_env: WEATHER_UPLOAD_TOKEN
|
||||
allow_pipelines:
|
||||
- weather-daily
|
||||
pipelines:
|
||||
- id: weather-daily
|
||||
source:
|
||||
backend: http_upload
|
||||
token_env: WEATHER_DAILY_UPLOAD_TOKEN
|
||||
staging_path: /var/spool/distributor/weather-daily
|
||||
max_upload_size: 20MB
|
||||
destinations:
|
||||
@@ -99,9 +103,9 @@ pipelines:
|
||||
path: /srv/reports/archive
|
||||
```
|
||||
|
||||
`token_env` is required for `http_upload` sources. `staging_path` defaults to `<server.http.staging_root>/<pipeline id>`. `max_upload_size` defaults to `server.http.max_upload_size`.
|
||||
`upload_tokens` is required when any pipeline source uses `http_upload`. Each token record resolves its bearer token value from the process environment or `secrets.directory`. `allow_pipelines` lists configured upload pipeline ids that the token may submit to. One token may authorize multiple upload pipelines, and multiple tokens may authorize the same upload pipeline.
|
||||
|
||||
`serve` maps each resolved bearer token to exactly one `http_upload` pipeline. Startup fails when a token is missing, empty, or duplicates another upload pipeline token.
|
||||
For `http_upload` sources, `staging_path` defaults to `<server.http.staging_root>/<pipeline id>`. `max_upload_size` defaults to `server.http.max_upload_size`.
|
||||
|
||||
## Top-Level Fields
|
||||
|
||||
@@ -124,6 +128,18 @@ Numeric server values and durations must be greater than zero after defaults are
|
||||
|
||||
See [Secrets](#secrets) for resolution rules.
|
||||
|
||||
### `upload_tokens`
|
||||
|
||||
`upload_tokens` configures bearer tokens for `distributor serve`. It is required when any pipeline source backend is `http_upload` and is invalid when no upload pipelines are configured.
|
||||
|
||||
Each token has:
|
||||
|
||||
- `id`: required unique slug-like identifier for the token record. It must start with a letter or number and may contain letters, numbers, `.`, `_`, and `-`.
|
||||
- `token_env`: required environment variable or secret-file name containing the bearer token value.
|
||||
- `allow_pipelines`: required non-empty list of configured pipeline ids whose source backend is `http_upload`.
|
||||
|
||||
Token values must resolve to non-empty strings and must be unique across token records. Every configured upload pipeline must be allowed by at least one token.
|
||||
|
||||
### `pipelines`
|
||||
|
||||
`pipelines` is required and must contain at least one pipeline.
|
||||
@@ -216,13 +232,11 @@ HTTP upload backends are valid only as pipeline sources and are served by `distr
|
||||
|
||||
```yaml
|
||||
backend: http_upload
|
||||
token_env: WEATHER_DAILY_UPLOAD_TOKEN
|
||||
staging_path: /var/spool/distributor/weather-daily
|
||||
max_upload_size: 20MB
|
||||
```
|
||||
|
||||
- `backend`: required value `http_upload`.
|
||||
- `token_env`: required environment variable or secret-file name containing the bearer token.
|
||||
- `staging_path`: optional staging path. Default: `<server.http.staging_root>/<pipeline id>`.
|
||||
- `max_upload_size`: optional per-source upload limit. Default: `server.http.max_upload_size`.
|
||||
|
||||
@@ -416,7 +430,7 @@ Fields resolved through this resolver:
|
||||
|
||||
- `credentials.access_key_id_env`
|
||||
- `credentials.secret_access_key_env`
|
||||
- `source.token_env` for `http_upload` sources
|
||||
- `upload_tokens[].token_env`
|
||||
|
||||
## Maintained Examples
|
||||
|
||||
|
||||
Reference in New Issue
Block a user