distributor/docs/config.md

# Distributor Configuration

## Config File Location

`distributor run --config <path>` loads the YAML config at the provided path.

If `--config` is omitted, `run` uses:

```text
/usr/local/etc/distributor/config.yml
```

Config parsing rejects unknown YAML fields. The executable backends are `local` and `ssh`. S3 config fields are accepted by config validation, but runtime execution for S3 is unavailable.

## Minimal Local Config

```yaml
pipelines:
  - id: reports
    source:
      backend: local
      path: /var/spool/distributor/reports
    destinations:
      - id: archive
        backend: local
        path: /srv/reports/archive
```

This publishes source files only. It uses the default validation and transfer policies.

## Production-Oriented Local Config

```yaml
pipelines:
  - id: reports
    source:
      backend: local
      path: /var/spool/distributor/reports
    validation:
      on_digest_mismatch: fail
    destinations:
      - id: archive
        backend: local
        path: /srv/reports/archive
        publish:
          source: true
          html: false
        transfer:
          on_destination_same: skip
          on_destination_older: replace
          on_destination_newer: skip
          on_conflict: fail
```

## HTML Publication

To publish generated HTML from Markdown files:

```yaml
publish:
  source: false
  html: true
transform:
  markdown_to_html:
    enabled: true
    mode: sidecar
```

Sidecar generation writes `report.html` for `report.md`. It does not mutate the source bundle.

## Reference

Top level:

- `pipelines`: required non-empty list.

Pipeline:

- `id`: required unique slug-like identifier.
- `source`: required backend config.
- `validation.on_digest_mismatch`: optional; defaults to `fail`; only `fail` is supported.
- `destinations`: required non-empty destination list.

Source backend:

- `backend`: required.
- `path`: required for `local` and `ssh`.
- `host`: required for `ssh`.
- `user`: optional for `ssh`; defaults to the current OS user when available.
- `port`: optional for `ssh`; defaults to `22`.
- `ssh_key_file`: optional for `ssh`.
- `known_hosts`: optional for `ssh`; defaults to the service user's OpenSSH `known_hosts` path when available.
- `host_key_policy`: optional for `ssh`; defaults to `accept-new`.
- `endpoint`: required for `s3`.
- `bucket`: required for `s3`.
- `prefix`: optional for `s3`.
- `region`: optional for `s3`.
- `force_path_style`: optional for `s3`.
- `credentials.access_key_id_env`: optional S3 credential environment variable name.
- `credentials.secret_access_key_env`: optional S3 credential environment variable name.

Destination:

- `id`: required unique slug-like identifier within the pipeline.
- Backend fields: same accepted shape as source backends, with destination fields at the destination level.
- `publish`: optional; defaults to source-only publication.
- `transform`: required only for generated HTML publication.
- `transfer`: optional; defaults described below.

Accepted backend names:

- `local`: executable; requires `path`.
- `ssh`: executable; requires `host` and `path`.
- `s3`: config validation only; execution is unavailable.

## SSH Backend

SSH uses native SFTP. It can be used for sources, destinations, or both:

```yaml
backend: ssh
host: example.com
user: distributor
port: 2222
path: /remote/root
ssh_key_file: /home/distributor/.ssh/id_ed25519
known_hosts: /home/distributor/.ssh/known_hosts
host_key_policy: accept-new
```

Authentication uses SSH agent identities first when `SSH_AUTH_SOCK` is set, then `ssh_key_file` if configured. Password authentication in YAML is not supported.

Host key policies:

- `strict`, `true`, and `"true"` require a matching known host key.
- `accept-new` accepts and persists a new host key, but fails if an existing key changed.
- `off`, `false`, and `"false"` disable host key checking and are insecure.

`accept-new` and `strict` use `known_hosts` when configured. If omitted, distributor uses the current service user's default OpenSSH `known_hosts` path where practical. `accept-new` fails when it needs to persist a new host key and no writable `known_hosts` path is available. It does not create a missing parent `.ssh` directory.

Publish policy:

- `publish.source`: publish source artifacts.
- `publish.html`: publish generated HTML artifacts from Markdown source files.

At least one output type must be enabled. When `publish.html` is true, `transform.markdown_to_html.enabled` must be `true` and `transform.markdown_to_html.mode` must be `sidecar`.

Transfer policy:

- `transfer.on_destination_same`: `skip` or `fail`; defaults to `skip`.
- `transfer.on_destination_older`: `replace` or `fail`; defaults to `replace`.
- `transfer.on_destination_newer`: `skip` or `fail`; defaults to `skip`.
- `transfer.on_conflict`: only `fail`; defaults to `fail`.

## Defaults

Defaults are applied after YAML decoding and before validation:

- `validation.on_digest_mismatch: fail`
- SSH `port: 22`
- SSH `host_key_policy: accept-new`
- `publish.source: true`
- `publish.html: false`
- `transfer.on_destination_same: skip`
- `transfer.on_destination_older: replace`
- `transfer.on_destination_newer: skip`
- `transfer.on_conflict: fail`

## Secrets

Do not put literal secrets in config files. S3 credentials may name environment variables:

- `credentials.access_key_id_env`
- `credentials.secret_access_key_env`

S3 execution is unavailable; these fields are accepted so config shape can be validated.

## Examples

Maintained examples live under [examples](../examples/):

- `local-to-local.yml`: minimal local config.
- `local-publish.yml`: runnable local source publication.
- `local-html.yml`: runnable local HTML publication.
- `fan-out.yml`: runnable local fan-out publication to source and HTML destinations.
- `ssh-destination.yml`: environment-gated local-to-SSH publication example.