distributor/docs/config.md

Distributor Configuration

Config File Location

distributor run --config <path> loads the YAML config at the provided path.

If --config is omitted, run uses:

/usr/local/etc/distributor/config.yml

Config parsing rejects unknown YAML fields. The current executable backend support is local only. SSH and S3 config fields are accepted by config validation, but runtime execution for those backends is not implemented.

Minimal Local Config

pipelines:
  - id: reports
    source:
      backend: local
      path: /var/spool/distributor/reports
    destinations:
      - id: archive
        backend: local
        path: /srv/reports/archive

This publishes source files only. It uses the default validation and transfer policies.

Production-Oriented Local Config

pipelines:
  - id: reports
    source:
      backend: local
      path: /var/spool/distributor/reports
    validation:
      on_digest_mismatch: fail
    destinations:
      - id: archive
        backend: local
        path: /srv/reports/archive
        publish:
          source: true
          html: false
        transfer:
          on_destination_same: skip
          on_destination_older: replace
          on_destination_newer: skip
          on_conflict: fail

HTML Publication

To publish generated HTML from Markdown files:

publish:
  source: false
  html: true
transform:
  markdown_to_html:
    enabled: true
    mode: sidecar

Sidecar generation writes report.html for report.md. It does not mutate the source bundle.

Reference

Top level:

• pipelines: required non-empty list.

Pipeline:

• id: required unique slug-like identifier.
• source: required backend config.
• validation.on_digest_mismatch: optional; defaults to fail; only fail is supported.
• destinations: required non-empty destination list.

Source backend:

• backend: required.
• path: required for local and ssh.
• uri: required for ssh.
• endpoint: required for s3.
• bucket: required for s3.
• prefix: optional for s3.
• region: optional for s3.
• force_path_style: optional for s3.
• credentials.access_key_id_env: optional S3 credential environment variable name.
• credentials.secret_access_key_env: optional S3 credential environment variable name.

Destination:

• id: required unique slug-like identifier within the pipeline.
• Backend fields: same accepted shape as source backends, with destination fields at the destination level.
• publish: optional; defaults to source-only publication.
• transform: required only for generated HTML publication.
• transfer: optional; defaults described below.

Accepted backend names:

• local: executable; requires path.
• ssh: config validation only; execution is not implemented.
• s3: config validation only; execution is not implemented.

Publish policy:

• publish.source: publish source artifacts.
• publish.html: publish generated HTML artifacts from Markdown source files.

At least one output type must be enabled. When publish.html is true, transform.markdown_to_html.enabled must be true and transform.markdown_to_html.mode must be sidecar.

Transfer policy:

• transfer.on_destination_same: skip or fail; defaults to skip.
• transfer.on_destination_older: replace or fail; defaults to replace.
• transfer.on_destination_newer: skip or fail; defaults to skip.
• transfer.on_conflict: only fail; defaults to fail.

Defaults

Defaults are applied after YAML decoding and before validation:

• validation.on_digest_mismatch: fail
• publish.source: true
• publish.html: false
• transfer.on_destination_same: skip
• transfer.on_destination_older: replace
• transfer.on_destination_newer: skip
• transfer.on_conflict: fail

Secrets

Do not put literal secrets in config files. S3 credentials may name environment variables:

• credentials.access_key_id_env
• credentials.secret_access_key_env

S3 execution is not implemented yet; these fields are accepted so config shape can be validated ahead of backend implementation.

Examples

Maintained examples live under examples:

• local-to-local.yml: minimal local config.
• local-publish.yml: runnable local source publication.
• local-html.yml: runnable local HTML publication.

examples/fan-out.yml currently demonstrates accepted SSH/S3 config shape but is not executable until remote backend support exists.