Distributor Configuration

Config file location

distributor run --config <path> loads the YAML config at the path provided by --config.

If --config is omitted during run, the built-in default path is:

/usr/local/etc/distributor/config.yml

The current implementation supports local-to-local publication of source files, generated HTML files, or both. Remote backends are not implemented yet.

Minimal config

pipelines:
  - id: reports
    source:
      backend: local
      path: /var/spool/distributor/reports
    destinations:
      - id: archive
        backend: local
        path: /srv/reports/archive

This uses the default publish policy of source files only and the default transfer policy.

Production-oriented config

pipelines:
  - id: reports
    source:
      backend: local
      path: /var/spool/distributor/reports
    validation:
      on_digest_mismatch: fail
    destinations:
      - id: archive
        backend: local
        path: /srv/reports/archive
        publish:
          source: true
          html: false
        transfer:
          on_destination_same: skip
          on_destination_older: replace
          on_destination_newer: skip
          on_conflict: fail

Reference

Top level:

pipelines: required non-empty list.

Pipeline:

id: required unique identifier.
source: required backend config.
destinations: required non-empty destination list.
validation.on_digest_mismatch: optional, defaults to fail; only fail is supported.

Backends:

local: requires path.
ssh: requires uri and path.
s3: requires endpoint and bucket; supports optional prefix, region, force_path_style, and credentials.

Destination policy:

publish.source: publish source artifacts.
publish.html: publish generated HTML artifacts from Markdown source files.
transfer.on_destination_same: skip or fail, defaults to skip.
transfer.on_destination_older: replace or fail, defaults to replace.
transfer.on_destination_newer: skip or fail, defaults to skip.
transfer.on_conflict: only fail, defaults to fail.

When publish.html is true, transform.markdown_to_html.enabled: true and transform.markdown_to_html.mode: sidecar are required.

Markdown-to-HTML sidecar generation writes report.html for report.md and does not mutate the source bundle.

Secrets

Do not put literal secrets in config files. S3 credentials may refer to environment variable names with:

credentials.access_key_id_env
credentials.secret_access_key_env

Examples

Maintained examples live under examples/.

2.6 KiB Raw Blame History