Distributor Configuration

Config file location

distributor run --config <path> loads the YAML config at the path provided by --config.

If --config is omitted during run, the built-in default path is:

/usr/local/etc/distributor/config.yml

The current implementation supports local-to-local source-file publication. Remote backends and HTML publication are not implemented yet.

Minimal config

pipelines:
  - id: reports
    source:
      backend: local
      path: /var/spool/distributor/reports
    destinations:
      - id: archive
        backend: local
        path: /srv/reports/archive

This uses the default publish policy of source files only and the default transfer policy.

Production-oriented config

pipelines:
  - id: reports
    source:
      backend: local
      path: /var/spool/distributor/reports
    validation:
      on_digest_mismatch: fail
    destinations:
      - id: archive
        backend: local
        path: /srv/reports/archive
        publish:
          source: true
          html: false
        transfer:
          on_destination_same: skip
          on_destination_older: replace
          on_destination_newer: skip
          on_conflict: fail

Reference

Top level:

pipelines: required non-empty list.

Pipeline:

id: required unique identifier.
source: required backend config.
destinations: required non-empty destination list.
validation.on_digest_mismatch: optional, defaults to fail; only fail is supported.

Backends:

local: requires path.
ssh: requires uri and path.
s3: requires endpoint and bucket; supports optional prefix, region, force_path_style, and credentials.

Destination policy:

publish.source: publish source artifacts.
publish.html: publish generated HTML artifacts.
transfer.on_destination_same: skip or fail, defaults to skip.
transfer.on_destination_older: replace or fail, defaults to replace.
transfer.on_destination_newer: skip or fail, defaults to skip.
transfer.on_conflict: only fail, defaults to fail.

When publish.html is true, transform.markdown_to_html.enabled: true and transform.markdown_to_html.mode: sidecar are required.

Secrets

Do not put literal secrets in config files. S3 credentials may refer to environment variable names with:

credentials.access_key_id_env
credentials.secret_access_key_env

Examples

Maintained examples live under examples/.

2.5 KiB Raw Blame History