154 lines
4.3 KiB
Markdown
154 lines
4.3 KiB
Markdown
# Distributor Configuration
|
|
|
|
## Config File Location
|
|
|
|
`distributor run --config <path>` loads the YAML config at the provided path.
|
|
|
|
If `--config` is omitted, `run` uses:
|
|
|
|
```text
|
|
/usr/local/etc/distributor/config.yml
|
|
```
|
|
|
|
Config parsing rejects unknown YAML fields. The current executable backend support is local only. SSH and S3 config fields are accepted by config validation, but runtime execution for those backends is unavailable.
|
|
|
|
## Minimal Local Config
|
|
|
|
```yaml
|
|
pipelines:
|
|
- id: reports
|
|
source:
|
|
backend: local
|
|
path: /var/spool/distributor/reports
|
|
destinations:
|
|
- id: archive
|
|
backend: local
|
|
path: /srv/reports/archive
|
|
```
|
|
|
|
This publishes source files only. It uses the default validation and transfer policies.
|
|
|
|
## Production-Oriented Local Config
|
|
|
|
```yaml
|
|
pipelines:
|
|
- id: reports
|
|
source:
|
|
backend: local
|
|
path: /var/spool/distributor/reports
|
|
validation:
|
|
on_digest_mismatch: fail
|
|
destinations:
|
|
- id: archive
|
|
backend: local
|
|
path: /srv/reports/archive
|
|
publish:
|
|
source: true
|
|
html: false
|
|
transfer:
|
|
on_destination_same: skip
|
|
on_destination_older: replace
|
|
on_destination_newer: skip
|
|
on_conflict: fail
|
|
```
|
|
|
|
## HTML Publication
|
|
|
|
To publish generated HTML from Markdown files:
|
|
|
|
```yaml
|
|
publish:
|
|
source: false
|
|
html: true
|
|
transform:
|
|
markdown_to_html:
|
|
enabled: true
|
|
mode: sidecar
|
|
```
|
|
|
|
Sidecar generation writes `report.html` for `report.md`. It does not mutate the source bundle.
|
|
|
|
## Reference
|
|
|
|
Top level:
|
|
|
|
- `pipelines`: required non-empty list.
|
|
|
|
Pipeline:
|
|
|
|
- `id`: required unique slug-like identifier.
|
|
- `source`: required backend config.
|
|
- `validation.on_digest_mismatch`: optional; defaults to `fail`; only `fail` is supported.
|
|
- `destinations`: required non-empty destination list.
|
|
|
|
Source backend:
|
|
|
|
- `backend`: required.
|
|
- `path`: required for `local` and `ssh`.
|
|
- `uri`: required for `ssh`.
|
|
- `endpoint`: required for `s3`.
|
|
- `bucket`: required for `s3`.
|
|
- `prefix`: optional for `s3`.
|
|
- `region`: optional for `s3`.
|
|
- `force_path_style`: optional for `s3`.
|
|
- `credentials.access_key_id_env`: optional S3 credential environment variable name.
|
|
- `credentials.secret_access_key_env`: optional S3 credential environment variable name.
|
|
|
|
Destination:
|
|
|
|
- `id`: required unique slug-like identifier within the pipeline.
|
|
- Backend fields: same accepted shape as source backends, with destination fields at the destination level.
|
|
- `publish`: optional; defaults to source-only publication.
|
|
- `transform`: required only for generated HTML publication.
|
|
- `transfer`: optional; defaults described below.
|
|
|
|
Accepted backend names:
|
|
|
|
- `local`: executable; requires `path`.
|
|
- `ssh`: config validation only; execution is unavailable.
|
|
- `s3`: config validation only; execution is unavailable.
|
|
|
|
Publish policy:
|
|
|
|
- `publish.source`: publish source artifacts.
|
|
- `publish.html`: publish generated HTML artifacts from Markdown source files.
|
|
|
|
At least one output type must be enabled. When `publish.html` is true, `transform.markdown_to_html.enabled` must be `true` and `transform.markdown_to_html.mode` must be `sidecar`.
|
|
|
|
Transfer policy:
|
|
|
|
- `transfer.on_destination_same`: `skip` or `fail`; defaults to `skip`.
|
|
- `transfer.on_destination_older`: `replace` or `fail`; defaults to `replace`.
|
|
- `transfer.on_destination_newer`: `skip` or `fail`; defaults to `skip`.
|
|
- `transfer.on_conflict`: only `fail`; defaults to `fail`.
|
|
|
|
## Defaults
|
|
|
|
Defaults are applied after YAML decoding and before validation:
|
|
|
|
- `validation.on_digest_mismatch: fail`
|
|
- `publish.source: true`
|
|
- `publish.html: false`
|
|
- `transfer.on_destination_same: skip`
|
|
- `transfer.on_destination_older: replace`
|
|
- `transfer.on_destination_newer: skip`
|
|
- `transfer.on_conflict: fail`
|
|
|
|
## Secrets
|
|
|
|
Do not put literal secrets in config files. S3 credentials may name environment variables:
|
|
|
|
- `credentials.access_key_id_env`
|
|
- `credentials.secret_access_key_env`
|
|
|
|
S3 execution is unavailable; these fields are accepted so config shape can be validated.
|
|
|
|
## Examples
|
|
|
|
Maintained examples live under [examples](../examples/):
|
|
|
|
- `local-to-local.yml`: minimal local config.
|
|
- `local-publish.yml`: runnable local source publication.
|
|
- `local-html.yml`: runnable local HTML publication.
|
|
- `fan-out.yml`: runnable local fan-out publication to source and HTML destinations.
|