From 18bba116f208a46f36a4a3dfa97ae2b19fe9051c Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Thu, 4 Jun 2026 12:07:41 +0000 Subject: [PATCH] Refresh configuration reference documentation --- docs/config.md | 490 +++++++++++++++++++++++++------------------------ 1 file changed, 246 insertions(+), 244 deletions(-) diff --git a/docs/config.md b/docs/config.md index 167a895..08fef31 100644 --- a/docs/config.md +++ b/docs/config.md @@ -1,19 +1,25 @@ # Configuration Reference -## Config File Location +Audience: administrators, operators, and advanced users who write YAML configuration for `distributor`. -`distributor run --config ` and `distributor serve --config ` load -the YAML config at the provided path. +This document is the canonical user-facing configuration reference. CLI syntax lives in [CLI](cli.md), operating procedures live in [Operations](operations.md), and symptom-oriented recovery lives in [Troubleshooting](troubleshooting.md). -If `--config` is omitted, both commands use: +## Config File Loading + +`distributor run --config ` and `distributor serve --config ` load the YAML file at ``. If `--config` is omitted, both commands use: ```text /usr/local/etc/distributor/config.yml ``` -Config parsing rejects unknown YAML fields. The executable `run` backends are -`local`, `ssh`, and `s3`. The `serve` command executes `http_upload` sources -through the HTTP upload API and normal destination fan-out. +YAML decoding rejects unknown fields. Defaults are applied after decoding and before validation. + +Runtime backend support is command-specific: + +- `run`, `validate --config`, and `inspect --config` execute `local`, `ssh`, and `s3` sources. +- `run` executes `local`, `ssh`, and `s3` destinations. +- `serve` uses `http_upload` sources through the HTTP upload API and publishes to configured `local`, `ssh`, and `s3` destinations. +- `http_upload` is valid only as a source backend. ## Minimal Local Config @@ -29,7 +35,7 @@ pipelines: path: /srv/reports/archive ``` -This publishes source files only. It uses the default validation and transfer policies. +This config publishes source files only. It uses default validation, destination path mapping, publish, transfer, and HTTP server values. ## Production-Oriented Local Config @@ -42,6 +48,8 @@ server: queue_size: 16 max_concurrency: 1 retention: 24h +secrets: + directory: /run/secrets/distributor pipelines: - id: reports source: @@ -56,6 +64,8 @@ pipelines: publish: source: true html: false + path_mapping: + mode: preserve_relative transfer: on_destination_same: skip on_destination_older: replace @@ -63,11 +73,9 @@ pipelines: on_conflict: fail ``` -## HTTP Upload Source Configuration +## HTTP Upload Source Config -HTTP upload sources are configured as pipeline sources only. They are not valid -destination backends. `distributor serve` maps each configured upload token to -exactly one `http_upload` pipeline. +HTTP upload sources are configured on pipelines and are served by `distributor serve`. Upload tokens are resolved from the process environment or `secrets.directory`; literal bearer tokens are not configured in YAML. ```yaml server: @@ -91,62 +99,188 @@ pipelines: path: /srv/reports/archive ``` -`source.token_env` is required and names the environment variable or `secrets.directory` file that provides the bearer token. Literal upload tokens are not supported in YAML. +`token_env` is required for `http_upload` sources. `staging_path` defaults to `/`. `max_upload_size` defaults to `server.http.max_upload_size`. -`source.staging_path` is optional. When omitted, it defaults to `/`. +`serve` maps each resolved bearer token to exactly one `http_upload` pipeline. Startup fails when a token is missing, empty, or duplicates another upload pipeline token. -`source.max_upload_size` is optional. When omitted, it defaults to `server.http.max_upload_size`. +## Top-Level Fields -The server resolves each `token_env` through the real process environment and -the configured `secrets.directory` resolver. Startup fails if any configured -upload token is missing, empty, or resolves to the same value as another upload -pipeline. Token values are not read from YAML and are not printed in API -responses. +### `server.http` -## HTTP Upload API +`server.http` controls the HTTP upload server used by `serve`. -`distributor serve` binds to `server.http.bind`, which defaults to -`127.0.0.1:8080`. +- `bind`: optional TCP bind address. Default: `127.0.0.1:8080`. +- `staging_root`: optional root used to default `http_upload` source staging paths. Default: `/var/spool/distributor`. +- `max_upload_size`: optional default upload limit for HTTP upload sources. Default: `20MB`. +- `queue_size`: optional upload admission queue size. Default: `16`. +- `max_concurrency`: optional upload worker concurrency. Default: `1`. +- `retention`: optional in-memory completed-run retention duration. Default: `24h`. -Routes: +Numeric server values and durations must be greater than zero after defaults are applied. -- `GET /healthz`: returns readiness status after config and upload tokens load. -- `POST /upload`: accepts one tar or tar.gz source bundle archive. -- `GET /runs/`: returns an in-memory upload status record, or `404` if the run id is unknown or expired. +### `secrets` -`POST /upload` authenticates with: +- `directory`: optional directory of secret files used by the config-owned credential resolver. -```text -Authorization: Bearer +See [Secrets](#secrets) for resolution rules. + +### `pipelines` + +`pipelines` is required and must contain at least one pipeline. + +Each pipeline has: + +- `id`: required unique slug-like identifier. It must start with a letter or number and may contain letters, numbers, `.`, `_`, and `-`. +- `source`: required source backend config. +- `validation`: optional validation policy. +- `destinations`: required non-empty destination list. + +Pipeline ids must be unique across the config. + +## Backend Reference + +### Local Backend + +Local backends can be used as sources and destinations. + +```yaml +backend: local +path: /srv/distributor/archive ``` -The token selects the configured `http_upload` pipeline. Producers do not send a -pipeline id. Requests with a submitted `pipeline` or `pipeline_id` query value -are rejected. +- `backend`: required value `local`. +- `path`: required local filesystem root for this backend. -Accepted upload content types: +### SSH/SFTP Backend -- `application/x-tar` -- `application/gzip` -- `application/x-gzip` +SSH backends use native SFTP and can be used as sources and destinations. -Accepted uploads return after the archive is staged and validated: - -```json -{"run_id":"","status":"accepted"} +```yaml +backend: ssh +host: ssh.example.com +user: distributor +port: 22 +path: /srv/distributor/archive +ssh_key_file: /home/distributor/.ssh/id_ed25519 +known_hosts: /home/distributor/.ssh/known_hosts +host_key_policy: strict ``` -Malformed tar or gzip content and invalid staged bundles are rejected before a -run id is issued. +- `backend`: required value `ssh`. +- `host`: required SSH host. +- `path`: required remote root path. +- `user`: optional SSH username. If omitted, the adapter uses the current OS user when available. +- `port`: optional TCP port. Default: `22`. +- `ssh_key_file`: optional private key path. +- `known_hosts`: optional OpenSSH `known_hosts` path. +- `host_key_policy`: optional host key policy. Default: `accept-new`. -The run id can be queried through `GET /runs/` while the status record -is retained in memory. Completed records expire after `server.http.retention`; -expiration also removes committed staged bundle directories for completed -uploads. +Accepted host key policy values: -## HTML Publication +- `strict` or boolean `true`: require a matching known host key. +- `accept-new`: accept and persist a new host key, but reject changed known keys. +- `off` or boolean `false`: disable host key checking. -To publish generated sidecar HTML from Markdown files: +Authentication uses SSH agent identities when `SSH_AUTH_SOCK` is available, then `ssh_key_file` when configured. Password authentication is not configured in YAML. + +### S3-Compatible Backend + +S3 backends can be used as sources and destinations. + +```yaml +backend: s3 +endpoint: https://s3.example.com +bucket: reports +prefix: distributor/archive +region: us-east-1 +force_path_style: true +credentials: + access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID + secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY +``` + +- `backend`: required value `s3`. +- `endpoint`: required S3-compatible endpoint URL. +- `bucket`: required bucket name. +- `prefix`: optional backend root prefix. Leading and trailing slashes are trimmed; the remaining value must be a clean relative slash-separated path. +- `region`: optional region. Default: `us-east-1`. +- `force_path_style`: optional addressing mode toggle. Default: `true`. +- `credentials.access_key_id_env`: optional environment variable or secret-file name for the access key id. +- `credentials.secret_access_key_env`: optional environment variable or secret-file name for the secret access key. + +The S3 credential variable names must either both be configured or both be omitted. When omitted, the AWS SDK default credential chain is used. When configured, both values must resolve to non-empty strings through the process environment or `secrets.directory`. + +### HTTP Upload Source Backend + +HTTP upload backends are valid only as pipeline sources and are served by `distributor serve`. + +```yaml +backend: http_upload +token_env: WEATHER_DAILY_UPLOAD_TOKEN +staging_path: /var/spool/distributor/weather-daily +max_upload_size: 20MB +``` + +- `backend`: required value `http_upload`. +- `token_env`: required environment variable or secret-file name containing the bearer token. +- `staging_path`: optional staging path. Default: `/`. +- `max_upload_size`: optional per-source upload limit. Default: `server.http.max_upload_size`. + +## Validation Policy + +```yaml +validation: + on_digest_mismatch: fail +``` + +- `validation.on_digest_mismatch`: optional. Default and only accepted value: `fail`. + +Source bundle digest mismatches fail validation before destination writes occur. + +## Destination Fields + +Each destination embeds a backend config at the destination level and may also configure publishing, transforms, path mapping, links, and transfer behavior. + +```yaml +destinations: + - id: archive + backend: local + path: /srv/reports/archive + publish: + source: true + html: false + path_mapping: + mode: preserve_relative + transfer: + on_destination_same: skip + on_destination_older: replace + on_destination_newer: skip + on_conflict: fail +``` + +- `id`: required unique slug-like identifier within the pipeline. +- Backend fields: required according to the selected destination backend. +- `publish`: optional publish policy. Default: source-only publication. +- `transform`: required only when publishing generated HTML. +- `path_mapping`: optional destination path mapping policy. +- `links`: optional public URL metadata policy. +- `transfer`: optional destination reconciliation policy. + +Destination ids must be unique within a pipeline. + +## Publish And Transform Policy + +### Source-Only Publication + +```yaml +publish: + source: true + html: false +``` + +`publish.source` controls whether source manifest files are copied to the destination. + +### Markdown-To-HTML Publication ```yaml publish: @@ -158,240 +292,106 @@ transform: mode: sidecar ``` -Sidecar generation writes `report.html` for `report.md`. It does not mutate the source bundle. +`publish.html` controls whether generated HTML outputs are published. When `publish.html` is `true`, `transform.markdown_to_html.enabled` must also be `true`. -To publish a single Markdown file as `index.html`: +Markdown transform fields: -```yaml -publish: - source: false - html: true -transform: - markdown_to_html: - enabled: true - mode: index - input: report.md -``` +- `transform.markdown_to_html.enabled`: enables Markdown-to-HTML generation for this destination. +- `transform.markdown_to_html.mode`: optional. Accepted values are `sidecar` and `index`; default is `sidecar` when a Markdown transform block is present. +- `transform.markdown_to_html.input`: optional source manifest path for `index` mode only. -When `mode: index` omits `input`, the source manifest must list exactly one Markdown file. +`sidecar` mode renders every manifest-listed `.md` file to a same-directory `.html` output. `index` mode renders one Markdown source to `index.html` at the destination bundle path. If `index` mode omits `input`, the selected source bundle must contain exactly one Markdown file. + +At least one output type must be enabled. Enabled Markdown transforms are rejected when `publish.html` is `false`, and `input` is rejected unless `mode` is `index`. ## Destination Path Mapping -Each destination chooses how source bundle paths map into that destination: - ```yaml path_mapping: mode: preserve_relative ``` -`preserve_relative` is the default. It publishes each discovered source bundle at the same path relative to the destination backend root. A source bundle at `daily/2026-06-01` publishes below `daily/2026-06-01` for that destination. +- `path_mapping.mode`: optional. Accepted values are `preserve_relative` and `fixed`; default is `preserve_relative`. -`fixed` publishes one selected source bundle directly at the destination backend root: +`preserve_relative` publishes each discovered source bundle at the same path relative to the destination backend root. -```yaml -destinations: - - id: latest-html - backend: local - path: /srv/www/reports/latest - path_mapping: - mode: fixed - publish: - source: false - html: true - transform: - markdown_to_html: - enabled: true - mode: index - input: report.md -``` +`fixed` publishes one selected source bundle directly at the destination backend root. Fixed destinations select the newest discovered source bundle by manifest `created` timestamp. If multiple bundles have the same timestamp, the source-root-relative bundle path in ascending order wins. -Fixed destinations select the newest discovered source bundle by manifest `created` timestamp. If multiple candidates have the same timestamp, the source-root-relative bundle path in ascending order wins. Older candidates are not planned or written for that destination. - -Fixed mapping is useful for stable latest-style paths. It is more destructive than archive-style publication because successive source bundles target the same destination root. Preview fixed destinations with `run --dry-run`, especially before using `--force`. +Fixed mapping is useful for stable latest-style paths. Preview fixed destinations with `run --dry-run`, especially before using `--force`. ## Destination Links -Destinations can record public URLs for published outputs: - ```yaml links: base_url: https://reports.example.com/archive primary: auto ``` -`links.base_url` is an absolute `http` or `https` URL corresponding to the destination backend root. It may include a path prefix, but it must not include a query string or fragment. Distributor does not infer public URLs from backend config. +- `links.base_url`: required when `links` is present. It must be an absolute `http` or `https` URL with a host and no query string or fragment. +- `links.primary`: optional. Accepted values are `auto`, `html`, and `source`; default is `auto` when `links` is present. -`links.primary` selects the top-level primary URL stored in destination state: +`distributor` does not infer public URLs from backend config. Output URLs are built from `links.base_url`, the destination bundle path, and output paths using URL path semantics. `index.html` outputs produce directory-style URLs that omit the filename. + +Primary URL policies: - `auto`: prefer `index.html`, then generated HTML, then source outputs. - `html`: use the first generated HTML output. - `source`: use the first copied source output. -If a destination has no `links` block, no URL metadata is generated. If a primary policy has no matching output, per-output URLs are still recorded and the top-level primary URL is omitted. +If no output matches the primary policy, per-output URLs may still be recorded and the top-level primary URL is omitted. -Output URLs are built from `links.base_url`, the destination bundle path, and the output path using URL path semantics. `index.html` outputs produce directory-style URLs that omit the filename. +## Transfer Policy -## Reference +```yaml +transfer: + on_destination_same: skip + on_destination_older: replace + on_destination_newer: skip + on_conflict: fail +``` -Top level: +Transfer fields and accepted values: -- `server.http.bind`: optional HTTP bind address; defaults to `127.0.0.1:8080`. -- `server.http.staging_root`: optional root for default HTTP upload staging paths; defaults to `/var/spool/distributor`. -- `server.http.max_upload_size`: optional default upload size limit; defaults to `20MB`. -- `server.http.queue_size`: optional HTTP upload admission queue size; defaults to `16`. -- `server.http.max_concurrency`: optional HTTP upload worker concurrency; defaults to `1`. -- `server.http.retention`: optional completed upload retention duration; defaults to `24h`. -- `secrets.directory`: optional credential secrets directory. -- `pipelines`: required non-empty list. +- `transfer.on_destination_same`: `skip` or `fail`. Default: `skip`. +- `transfer.on_destination_older`: `replace` or `fail`. Default: `replace`. +- `transfer.on_destination_newer`: `skip`, `replace`, or `fail`. Default: `skip`. +- `transfer.on_conflict`: `fail` or `replace`. Default: `fail`. -Pipeline: - -- `id`: required unique slug-like identifier. -- `source`: required backend config. -- `validation.on_digest_mismatch`: optional; defaults to `fail`; only `fail` is supported. -- `destinations`: required non-empty destination list. - -Source backend: - -- `backend`: required. -- `path`: required for `local` and `ssh`. -- `host`: required for `ssh`. -- `user`: optional for `ssh`; defaults to the current OS user when available. -- `port`: optional for `ssh`; defaults to `22`. -- `ssh_key_file`: optional for `ssh`. -- `known_hosts`: optional for `ssh`; defaults to the service user's OpenSSH `known_hosts` path when available. -- `host_key_policy`: optional for `ssh`; defaults to `accept-new`. -- `endpoint`: required for `s3`. -- `bucket`: required for `s3`. -- `prefix`: optional for `s3`; leading and trailing slashes are trimmed. -- `region`: optional for `s3`; defaults to `us-east-1`. -- `force_path_style`: optional for `s3`; defaults to `true`. Set `false` only for services that require virtual-host addressing. -- `credentials.access_key_id_env`: optional S3 credential environment variable name. -- `credentials.secret_access_key_env`: optional S3 credential environment variable name. -- `token_env`: required for `http_upload`; names the token environment variable or secret-file name. -- `staging_path`: optional for `http_upload`; defaults below `server.http.staging_root` using the pipeline id. -- `max_upload_size`: optional for `http_upload`; defaults to `server.http.max_upload_size`. - -Destination: - -- `id`: required unique slug-like identifier within the pipeline. -- Backend fields: same accepted shape as source backends, with destination fields at the destination level. -- `publish`: optional; defaults to source-only publication. -- `transform`: required only for generated HTML publication. -- `path_mapping.mode`: optional; defaults to `preserve_relative`. Accepted values are `preserve_relative` and `fixed`. -- `links.base_url`: optional links block; when present, `base_url` is required and must be an absolute HTTP or HTTPS URL without query string or fragment. -- `links.primary`: optional; defaults to `auto`. Accepted values are `auto`, `html`, and `source`. -- `transfer`: optional; defaults described below. - -Accepted backend names: - -- `local`: executable; requires `path`. -- `ssh`: executable; requires `host` and `path`. -- `s3`: executable; requires `endpoint` and `bucket`. -- `http_upload`: source-only configuration; requires `token_env`. +`replace` for `on_destination_newer` and `on_conflict` is honored only when `run --force` is supplied. There is no config field that enables forced replacement by default. ## Size And Duration Values -Upload size fields use an integer plus one of the supported binary-size suffixes: +Upload size fields must be YAML strings with an integer and one of these suffixes: - `B` - `KB` - `MB` - `GB` -Suffix multipliers use powers of 1024. Size values must be greater than zero after defaults are applied. +Suffix multipliers use powers of 1024. Values must be greater than zero after defaults are applied. -HTTP retention uses Go-style duration strings such as `24h`, `90m`, or `168h`. Retention must be greater than zero after defaults are applied. - -## SSH Backend - -SSH uses native SFTP. It can be used for sources, destinations, or both: - -```yaml -backend: ssh -host: example.com -user: distributor -port: 2222 -path: /remote/root -ssh_key_file: /home/distributor/.ssh/id_ed25519 -known_hosts: /home/distributor/.ssh/known_hosts -host_key_policy: accept-new -``` - -Authentication uses SSH agent identities first when `SSH_AUTH_SOCK` is set, then `ssh_key_file` if configured. Password authentication in YAML is not supported. - -Host key policies: - -- `strict`, `true`, and `"true"` require a matching known host key. -- `accept-new` accepts and persists a new host key, but fails if an existing key changed. During `run --dry-run`, new host keys are accepted only for the current connection and are not persisted. -- `off`, `false`, and `"false"` disable host key checking and are insecure. - -`accept-new` and `strict` use `known_hosts` when configured. If omitted, distributor uses the current service user's default OpenSSH `known_hosts` path where practical. `accept-new` fails when it needs to persist a new host key and no writable `known_hosts` path is available. It does not create a missing parent `.ssh` directory. - -## S3 Backend - -S3 uses the AWS SDK for Go v2 and supports S3-compatible endpoints: - -```yaml -backend: s3 -endpoint: https://s3.example.com -bucket: reports -prefix: archive -region: us-east-1 -force_path_style: true -credentials: - access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID - secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY -``` - -`endpoint` and `bucket` are required. `prefix` is an optional backend root; it is treated as an object-key prefix, not a real directory. Prefixes must be clean slash-separated paths after trimming leading and trailing slashes. `http://` endpoints are allowed for explicitly configured local development or local S3-compatible test services. - -If either credential environment variable name is configured, both must be configured and both referenced variables must resolve to non-empty values through the real process environment or `secrets.directory`. Explicit credentials take precedence over the AWS SDK default credential chain. If credential environment variable names are omitted, the SDK default credential chain is used and `secrets.directory` values are not injected into the process environment. - -Publish policy: - -- `publish.source`: publish source artifacts. -- `publish.html`: publish generated HTML artifacts from Markdown source files. - -At least one output type must be enabled. When `publish.html` is true, `transform.markdown_to_html.enabled` must be `true`. - -Markdown-to-HTML transform: - -- `transform.markdown_to_html.enabled`: enables Markdown-to-HTML generation for destinations with `publish.html: true`. -- `transform.markdown_to_html.mode`: optional; defaults to `sidecar`. Accepted values are `sidecar` and `index`. -- `transform.markdown_to_html.input`: optional source manifest path for `index` mode. It must identify a listed Markdown file. - -`sidecar` mode renders each manifest-listed `.md` file to a same-directory `.html` output. `index` mode renders one selected Markdown file to `index.html` at the destination bundle path. Enabled Markdown-to-HTML config is rejected when `publish.html` is false, and `input` is valid only with `mode: index`. - -Transfer policy: - -- `transfer.on_destination_same`: `skip` or `fail`; defaults to `skip`. -- `transfer.on_destination_older`: `replace` or `fail`; defaults to `replace`. -- `transfer.on_destination_newer`: `skip`, `replace`, or `fail`; defaults to `skip`. -- `transfer.on_conflict`: `fail` or `replace`; defaults to `fail`. - -`replace` for `on_destination_newer` and `on_conflict` is honored only when `run --force` is used for that invocation. Force is CLI-only; there is no persistent config field that enables forced replacement by default. +Duration fields must be YAML strings accepted by Go duration parsing, such as `24h`, `90m`, or `168h`. Values must be greater than zero after defaults are applied. ## Defaults Defaults are applied after YAML decoding and before validation: -- `validation.on_digest_mismatch: fail` -- SSH `port: 22` -- SSH `host_key_policy: accept-new` -- S3 `region: us-east-1` -- S3 `force_path_style: true` - `server.http.bind: 127.0.0.1:8080` - `server.http.staging_root: /var/spool/distributor` - `server.http.max_upload_size: 20MB` - `server.http.queue_size: 16` - `server.http.max_concurrency: 1` - `server.http.retention: 24h` -- `source.staging_path: /var/spool/distributor/` for `http_upload` -- `source.max_upload_size: server.http.max_upload_size` for `http_upload` -- `transform.markdown_to_html.mode: sidecar` when a Markdown-to-HTML transform block is present and mode is omitted -- `publish.source: true` -- `publish.html: false` +- `validation.on_digest_mismatch: fail` +- SSH `port: 22` +- SSH `host_key_policy: accept-new` +- S3 `region: us-east-1` +- S3 `prefix`: leading and trailing slashes trimmed +- S3 `force_path_style: true` +- `http_upload` source `staging_path: /` +- `http_upload` source `max_upload_size: server.http.max_upload_size` +- `publish.source: true` and `publish.html: false` +- `transform.markdown_to_html.mode: sidecar` when a Markdown transform block is present and mode is omitted - `path_mapping.mode: preserve_relative` - `links.primary: auto` when a `links` block is present and `primary` is omitted - `transfer.on_destination_same: skip` @@ -408,29 +408,31 @@ secrets: directory: /run/secrets/distributor ``` -Each regular file in the directory becomes an internal credential environment value named by the filename. Valid filenames must match `[A-Za-z_][A-Za-z0-9_]*`. Directories are ignored, and symlinks to regular files are followed. Exactly one trailing LF or CRLF is trimmed from each file; other whitespace is preserved. +Each regular file in the directory becomes an internal credential environment value named by the filename. Valid filenames match `[A-Za-z_][A-Za-z0-9_]*`. Directories are ignored. Symlinks to regular files are followed. Exactly one trailing LF or CRLF is trimmed from each file; other whitespace is preserved. -The resolver checks the real process environment first, then the secrets directory. If both define the same variable with different values, `run` prints a warning with the variable name and uses the real environment value. Secret values are not printed. The process environment is not modified, so SDK default credential chains see only real environment variables. +Credential resolution checks the real process environment first, then `secrets.directory`. If both define the same name with different values, `run` emits a warning with the variable name and uses the real environment value. Secret values are not printed. The process environment is not modified, so SDK default credential chains see only real process environment variables. -S3 credentials may name environment variables: +Fields resolved through this resolver: - `credentials.access_key_id_env` - `credentials.secret_access_key_env` +- `source.token_env` for `http_upload` sources -HTTP upload tokens name one environment variable or secret-file name: +## Maintained Examples -- `source.token_env` +Maintained examples live under [examples](../examples/). Config tests load every file listed here. -## Examples +Local examples: -Maintained examples live under [examples](../examples/): +- `local-to-local.yml`: minimal local-to-local config using absolute sample paths; load-tested, but paths should be adapted before running. +- `local-publish.yml`: runnable local source publication used by the README quickstart. +- `local-html.yml`: local sidecar HTML publication. +- `local-index.yml`: local `index.html` publication. +- `fan-out.yml`: local fan-out publication to source and HTML destinations. +- `archive-and-latest.yml`: local archive plus fixed latest publication. +- `http-upload-local.yml`: local HTTP upload server config; requires `DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN` in the process environment or as a secret-file name before running `serve`. -- `local-to-local.yml`: minimal local config. -- `local-publish.yml`: runnable local source publication. -- `local-html.yml`: runnable local HTML publication. -- `local-index.yml`: runnable local `index.html` publication. -- `fan-out.yml`: runnable local fan-out publication to source and HTML destinations. -- `archive-and-latest.yml`: runnable local fan-out publication to an archive destination and a fixed latest destination. -- `http-upload-local.yml`: local HTTP upload server example with a token environment variable reference. -- `ssh-destination.yml`: environment-gated local-to-SSH publication example. -- `s3-destination.yml`: environment-gated local-to-S3 publication example. +Environment-gated remote examples: + +- `ssh-destination.yml`: local-to-SSH publication; replace host, user, path, key, and known-host values for an SSH/SFTP endpoint you control. +- `s3-destination.yml`: local-to-S3 publication; replace endpoint, bucket, prefix, region, and credential variable names for an S3-compatible service you control.