Add config loading and dry-run validation

This commit is contained in:
2026-05-31 01:53:13 +00:00
parent 22d0424232
commit 29dbad2967
16 changed files with 928 additions and 17 deletions

View File

@@ -3,10 +3,10 @@
## Shortest useful command
```sh
go run ./cmd/distributor --help
go run ./cmd/distributor run --config examples/local-to-local.yml --dry-run
```
This prints the currently implemented command shell.
This loads and validates the example config, then prints the resolved pipeline summary without publishing files.
## Command overview
@@ -20,7 +20,9 @@ distributor inspect
`version` prints the application name and version. The default development version is `dev`; release builds may replace it at build time.
`run`, `validate`, and `inspect` are command placeholders. They intentionally fail with a clear `not implemented` error until the corresponding application behavior exists.
`run --config <path> --dry-run` loads and validates configuration, then prints a concise summary of configured pipelines and destinations. It does not discover bundles or publish files yet.
`run` without `--dry-run`, `validate`, and `inspect` intentionally fail with a clear `not implemented` error until the corresponding application behavior exists.
## Flag reference
@@ -28,6 +30,19 @@ The root command supports:
- `--help`, `-h`: print root help.
Each implemented subcommand supports:
Each subcommand supports:
- `--help`, `-h`: print command-specific help.
`run` supports:
- `--config <path>`: config file to load.
- `--dry-run`: validate config and print the resolved summary without publishing.
## Common workflows
Validate a config file without publishing:
```sh
go run ./cmd/distributor run --config examples/local-to-local.yml --dry-run
```

96
docs/config.md Normal file
View File

@@ -0,0 +1,96 @@
# Distributor Configuration
## Config file location
`distributor run --config <path> --dry-run` loads the YAML config at the path provided by `--config`.
If `--config` is omitted during dry-run, the built-in default path is:
```text
/usr/local/etc/distributor/config.yml
```
The current implementation loads and validates configuration only. Bundle discovery and publication are not implemented yet.
## Minimal config
```yaml
pipelines:
- id: reports
source:
backend: local
path: /var/spool/distributor/reports
destinations:
- id: archive
backend: local
path: /srv/reports/archive
```
This uses the default publish policy of source files only and the default transfer policy.
## Production-oriented config
```yaml
pipelines:
- id: reports
source:
backend: local
path: /var/spool/distributor/reports
validation:
on_digest_mismatch: fail
destinations:
- id: archive
backend: s3
endpoint: https://s3.example.com
bucket: reports
prefix: archive
region: us-east-1
force_path_style: true
credentials:
access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
publish:
source: true
html: false
```
## Reference
Top level:
- `pipelines`: required non-empty list.
Pipeline:
- `id`: required unique identifier.
- `source`: required backend config.
- `destinations`: required non-empty destination list.
- `validation.on_digest_mismatch`: optional, defaults to `fail`; only `fail` is supported.
Backends:
- `local`: requires `path`.
- `ssh`: requires `uri` and `path`.
- `s3`: requires `endpoint` and `bucket`; supports optional `prefix`, `region`, `force_path_style`, and `credentials`.
Destination policy:
- `publish.source`: publish source artifacts.
- `publish.html`: publish generated HTML artifacts.
- `transfer.on_destination_same`: `skip` or `fail`, defaults to `skip`.
- `transfer.on_destination_older`: `replace` or `fail`, defaults to `replace`.
- `transfer.on_destination_newer`: `skip` or `fail`, defaults to `skip`.
- `transfer.on_conflict`: only `fail`, defaults to `fail`.
When `publish.html` is true, `transform.markdown_to_html.enabled: true` and `transform.markdown_to_html.mode: sidecar` are required.
## Secrets
Do not put literal secrets in config files. S3 credentials may refer to environment variable names with:
- `credentials.access_key_id_env`
- `credentials.secret_access_key_env`
## Examples
Maintained examples live under [examples/](../examples/).