eric/distributor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Tighten local MVP user documentation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-31 04:02:09 +00:00
parent c36217d0df
commit b3044c5b7b
5 changed files with 299 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files

102
docs/config.md
View File

@@ -1,18 +1,18 @@
# Distributor Configuration
## Config file location
## Config File Location
`distributor run --config <path>` loads the YAML config at the path provided by `--config`.
`distributor run --config <path>` loads the YAML config at the provided path.
If `--config` is omitted during run, the built-in default path is:
If `--config` is omitted, `run` uses:
```text
/usr/local/etc/distributor/config.yml
```
The current implementation supports local-to-local publication of source files, generated HTML files, or both. Remote backends are not implemented yet.
Config parsing rejects unknown YAML fields. The current executable backend support is local only. SSH and S3 config fields are accepted by config validation, but runtime execution for those backends is not implemented.
## Minimal config
## Minimal Local Config
```yaml
pipelines:
@@ -26,9 +26,9 @@ pipelines:
path: /srv/reports/archive
```
This uses the default publish policy of source files only and the default transfer policy.
This publishes source files only. It uses the default validation and transfer policies.
## Production-oriented config
## Production-Oriented Local Config
```yaml
pipelines:
@@ -52,6 +52,22 @@ pipelines:
on_conflict: fail
```
## HTML Publication
To publish generated HTML from Markdown files:
```yaml
publish:
source: false
html: true
transform:
markdown_to_html:
enabled: true
mode: sidecar
```
Sidecar generation writes `report.html` for `report.md`. It does not mutate the source bundle.
## Reference
Top level:
@@ -60,37 +76,79 @@ Top level:
Pipeline:
- `id`: required unique identifier.
- `id`: required unique slug-like identifier.
- `source`: required backend config.
- `validation.on_digest_mismatch`: optional; defaults to `fail`; only `fail` is supported.
- `destinations`: required non-empty destination list.
- `validation.on_digest_mismatch`: optional, defaults to `fail`; only `fail` is supported.
Backends:
Source backend:
- `local`: requires `path`.
- `ssh`: requires `uri` and `path`.
- `s3`: requires `endpoint` and `bucket`; supports optional `prefix`, `region`, `force_path_style`, and `credentials`.
- `backend`: required.
- `path`: required for `local` and `ssh`.
- `uri`: required for `ssh`.
- `endpoint`: required for `s3`.
- `bucket`: required for `s3`.
- `prefix`: optional for `s3`.
- `region`: optional for `s3`.
- `force_path_style`: optional for `s3`.
- `credentials.access_key_id_env`: optional S3 credential environment variable name.
- `credentials.secret_access_key_env`: optional S3 credential environment variable name.
Destination policy:
Destination:
- `id`: required unique slug-like identifier within the pipeline.
- Backend fields: same accepted shape as source backends, with destination fields at the destination level.
- `publish`: optional; defaults to source-only publication.
- `transform`: required only for generated HTML publication.
- `transfer`: optional; defaults described below.
Accepted backend names:
- `local`: executable; requires `path`.
- `ssh`: config validation only; execution is not implemented.
- `s3`: config validation only; execution is not implemented.
Publish policy:
- `publish.source`: publish source artifacts.
- `publish.html`: publish generated HTML artifacts from Markdown source files.
- `transfer.on_destination_same`: `skip` or `fail`, defaults to `skip`.
- `transfer.on_destination_older`: `replace` or `fail`, defaults to `replace`.
- `transfer.on_destination_newer`: `skip` or `fail`, defaults to `skip`.
- `transfer.on_conflict`: only `fail`, defaults to `fail`.
When `publish.html` is true, `transform.markdown_to_html.enabled: true` and `transform.markdown_to_html.mode: sidecar` are required.
At least one output type must be enabled. When `publish.html` is true, `transform.markdown_to_html.enabled` must be `true` and `transform.markdown_to_html.mode` must be `sidecar`.
Markdown-to-HTML sidecar generation writes `report.html` for `report.md` and does not mutate the source bundle.
Transfer policy:
- `transfer.on_destination_same`: `skip` or `fail`; defaults to `skip`.
- `transfer.on_destination_older`: `replace` or `fail`; defaults to `replace`.
- `transfer.on_destination_newer`: `skip` or `fail`; defaults to `skip`.
- `transfer.on_conflict`: only `fail`; defaults to `fail`.
## Defaults
Defaults are applied after YAML decoding and before validation:
- `validation.on_digest_mismatch: fail`
- `publish.source: true`
- `publish.html: false`
- `transfer.on_destination_same: skip`
- `transfer.on_destination_older: replace`
- `transfer.on_destination_newer: skip`
- `transfer.on_conflict: fail`
## Secrets
Do not put literal secrets in config files. S3 credentials may refer to environment variable names with:
Do not put literal secrets in config files. S3 credentials may name environment variables:
- `credentials.access_key_id_env`
- `credentials.secret_access_key_env`
S3 execution is not implemented yet; these fields are accepted so config shape can be validated ahead of backend implementation.
## Examples
Maintained examples live under [examples/](../examples/).
Maintained examples live under [examples](../examples/):
- `local-to-local.yml`: minimal local config.
- `local-publish.yml`: runnable local source publication.
- `local-html.yml`: runnable local HTML publication.
`examples/fan-out.yml` currently demonstrates accepted SSH/S3 config shape but is not executable until remote backend support exists.