Add config loading and dry-run validation
This commit is contained in:
23
docs/cli.md
23
docs/cli.md
@@ -3,10 +3,10 @@
|
||||
## Shortest useful command
|
||||
|
||||
```sh
|
||||
go run ./cmd/distributor --help
|
||||
go run ./cmd/distributor run --config examples/local-to-local.yml --dry-run
|
||||
```
|
||||
|
||||
This prints the currently implemented command shell.
|
||||
This loads and validates the example config, then prints the resolved pipeline summary without publishing files.
|
||||
|
||||
## Command overview
|
||||
|
||||
@@ -20,7 +20,9 @@ distributor inspect
|
||||
|
||||
`version` prints the application name and version. The default development version is `dev`; release builds may replace it at build time.
|
||||
|
||||
`run`, `validate`, and `inspect` are command placeholders. They intentionally fail with a clear `not implemented` error until the corresponding application behavior exists.
|
||||
`run --config <path> --dry-run` loads and validates configuration, then prints a concise summary of configured pipelines and destinations. It does not discover bundles or publish files yet.
|
||||
|
||||
`run` without `--dry-run`, `validate`, and `inspect` intentionally fail with a clear `not implemented` error until the corresponding application behavior exists.
|
||||
|
||||
## Flag reference
|
||||
|
||||
@@ -28,6 +30,19 @@ The root command supports:
|
||||
|
||||
- `--help`, `-h`: print root help.
|
||||
|
||||
Each implemented subcommand supports:
|
||||
Each subcommand supports:
|
||||
|
||||
- `--help`, `-h`: print command-specific help.
|
||||
|
||||
`run` supports:
|
||||
|
||||
- `--config <path>`: config file to load.
|
||||
- `--dry-run`: validate config and print the resolved summary without publishing.
|
||||
|
||||
## Common workflows
|
||||
|
||||
Validate a config file without publishing:
|
||||
|
||||
```sh
|
||||
go run ./cmd/distributor run --config examples/local-to-local.yml --dry-run
|
||||
```
|
||||
|
||||
96
docs/config.md
Normal file
96
docs/config.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Distributor Configuration
|
||||
|
||||
## Config file location
|
||||
|
||||
`distributor run --config <path> --dry-run` loads the YAML config at the path provided by `--config`.
|
||||
|
||||
If `--config` is omitted during dry-run, the built-in default path is:
|
||||
|
||||
```text
|
||||
/usr/local/etc/distributor/config.yml
|
||||
```
|
||||
|
||||
The current implementation loads and validates configuration only. Bundle discovery and publication are not implemented yet.
|
||||
|
||||
## Minimal config
|
||||
|
||||
```yaml
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: /var/spool/distributor/reports
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: local
|
||||
path: /srv/reports/archive
|
||||
```
|
||||
|
||||
This uses the default publish policy of source files only and the default transfer policy.
|
||||
|
||||
## Production-oriented config
|
||||
|
||||
```yaml
|
||||
pipelines:
|
||||
- id: reports
|
||||
source:
|
||||
backend: local
|
||||
path: /var/spool/distributor/reports
|
||||
validation:
|
||||
on_digest_mismatch: fail
|
||||
destinations:
|
||||
- id: archive
|
||||
backend: s3
|
||||
endpoint: https://s3.example.com
|
||||
bucket: reports
|
||||
prefix: archive
|
||||
region: us-east-1
|
||||
force_path_style: true
|
||||
credentials:
|
||||
access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
|
||||
secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
|
||||
publish:
|
||||
source: true
|
||||
html: false
|
||||
```
|
||||
|
||||
## Reference
|
||||
|
||||
Top level:
|
||||
|
||||
- `pipelines`: required non-empty list.
|
||||
|
||||
Pipeline:
|
||||
|
||||
- `id`: required unique identifier.
|
||||
- `source`: required backend config.
|
||||
- `destinations`: required non-empty destination list.
|
||||
- `validation.on_digest_mismatch`: optional, defaults to `fail`; only `fail` is supported.
|
||||
|
||||
Backends:
|
||||
|
||||
- `local`: requires `path`.
|
||||
- `ssh`: requires `uri` and `path`.
|
||||
- `s3`: requires `endpoint` and `bucket`; supports optional `prefix`, `region`, `force_path_style`, and `credentials`.
|
||||
|
||||
Destination policy:
|
||||
|
||||
- `publish.source`: publish source artifacts.
|
||||
- `publish.html`: publish generated HTML artifacts.
|
||||
- `transfer.on_destination_same`: `skip` or `fail`, defaults to `skip`.
|
||||
- `transfer.on_destination_older`: `replace` or `fail`, defaults to `replace`.
|
||||
- `transfer.on_destination_newer`: `skip` or `fail`, defaults to `skip`.
|
||||
- `transfer.on_conflict`: only `fail`, defaults to `fail`.
|
||||
|
||||
When `publish.html` is true, `transform.markdown_to_html.enabled: true` and `transform.markdown_to_html.mode: sidecar` are required.
|
||||
|
||||
## Secrets
|
||||
|
||||
Do not put literal secrets in config files. S3 credentials may refer to environment variable names with:
|
||||
|
||||
- `credentials.access_key_id_env`
|
||||
- `credentials.secret_access_key_env`
|
||||
|
||||
## Examples
|
||||
|
||||
Maintained examples live under [examples/](../examples/).
|
||||
Reference in New Issue
Block a user