Distributor Configuration

Config file location

distributor run --config <path> --dry-run loads the YAML config at the path provided by --config.

If --config is omitted during dry-run, the built-in default path is:

/usr/local/etc/distributor/config.yml

The current implementation loads and validates configuration only. Bundle discovery and publication are not implemented yet.

Minimal config

pipelines:
  - id: reports
    source:
      backend: local
      path: /var/spool/distributor/reports
    destinations:
      - id: archive
        backend: local
        path: /srv/reports/archive

This uses the default publish policy of source files only and the default transfer policy.

Production-oriented config

pipelines:
  - id: reports
    source:
      backend: local
      path: /var/spool/distributor/reports
    validation:
      on_digest_mismatch: fail
    destinations:
      - id: archive
        backend: s3
        endpoint: https://s3.example.com
        bucket: reports
        prefix: archive
        region: us-east-1
        force_path_style: true
        credentials:
          access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
          secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
        publish:
          source: true
          html: false

Reference

Top level:

pipelines: required non-empty list.

Pipeline:

id: required unique identifier.
source: required backend config.
destinations: required non-empty destination list.
validation.on_digest_mismatch: optional, defaults to fail; only fail is supported.

Backends:

local: requires path.
ssh: requires uri and path.
s3: requires endpoint and bucket; supports optional prefix, region, force_path_style, and credentials.

Destination policy:

publish.source: publish source artifacts.
publish.html: publish generated HTML artifacts.
transfer.on_destination_same: skip or fail, defaults to skip.
transfer.on_destination_older: replace or fail, defaults to replace.
transfer.on_destination_newer: skip or fail, defaults to skip.
transfer.on_conflict: only fail, defaults to fail.

When publish.html is true, transform.markdown_to_html.enabled: true and transform.markdown_to_html.mode: sidecar are required.

Secrets

Do not put literal secrets in config files. S3 credentials may refer to environment variable names with:

credentials.access_key_id_env
credentials.secret_access_key_env

Examples

Maintained examples live under examples/.

2.6 KiB Raw Blame History