eric/distributor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Add configured source validation and inspection

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-01 21:11:35 +00:00
parent 8b1e5abf68
commit 29fd0e494c
13 changed files with 728 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
docs/cli.md
View File

@@ -15,17 +15,19 @@ distributor [--help]
distributor version [--format text|json]
distributor run [--config <path>] [--dry-run] [--force] [--format text|json]
distributor validate [--format text|json] <path>
distributor validate --config <path> --pipeline <id> [--bundle <path>] [--format text|json]
distributor inspect [--format text|json] <path>
distributor inspect --config <path> --pipeline <id> [--bundle <path>] [--format text|json]
distributor manifest create <bundle-path> --id <bundle-id> [options]
```
- `version`: prints the application name and version. Development builds print `distributor dev`.
- `run`: loads a YAML config, discovers source bundles, plans each configured destination, writes selected outputs unless `--dry-run` is set, and prints a final status summary.
- `validate`: validates a local source bundle directory or a local tree containing source bundles.
- `inspect`: validates local source bundles and prints normalized bundle metadata.
- `validate`: validates a local source bundle directory, a local source bundle tree, or one configured pipeline source.
- `inspect`: validates source bundles and prints normalized bundle metadata for a local path or one configured pipeline source.
- `manifest create`: creates `manifest.json` for a local source bundle directory.
`validate` and `inspect` accept local paths only. `run` executes `local`, `ssh`, and `s3` backends.
`validate` and `inspect` have two mutually exclusive modes: a local path shortcut, or configured source mode with `--config <path> --pipeline <id>`. Configured source mode opens only the selected pipeline source and supports configured `local`, `ssh`, and `s3` sources. It does not open destinations. `run` executes configured sources and destinations.
## Flag reference
@@ -47,6 +49,12 @@ Output-producing subcommands:
- `--dry-run`: load config, discover bundles, inspect destination state, print planned actions and final status, and do not write output files, destination state, or SSH `known_hosts` entries.
- `--force`: allow explicit destructive replacement for supported conflict cases in this run only.
`validate` and `inspect` configured source flags:
- `--config <path>`: config file to load for source validation or inspection. Required in configured source mode.
- `--pipeline <id>`: pipeline source to validate or inspect. Required in configured source mode.
- `--bundle <path>`: source-root-relative bundle directory to validate or inspect instead of discovering every bundle under the source root.
`manifest create` flags:
- `--id <bundle-id>`: source bundle id. Required.
@@ -54,7 +62,7 @@ Output-producing subcommands:
- `--created <time>`: RFC3339 source created timestamp. If omitted, the current UTC time is used.
- `--overwrite`: replace an existing `manifest.json`.
`run` does not accept positional arguments. `validate` and `inspect` accept at most one path; omitting the path returns a required-path error.
`run` does not accept positional arguments. `validate` and `inspect` accept at most one path in local mode. Local paths cannot be combined with `--config`, `--pipeline`, or `--bundle`.
## Common workflows
@@ -70,6 +78,21 @@ Inspect a source bundle:
go run ./cmd/distributor inspect examples/source-bundle
```
Validate a configured source without opening destinations:
```sh
go run ./cmd/distributor validate --config examples/local-publish.yml --pipeline example-source-bundle
```
Inspect one configured source bundle:
```sh
go run ./cmd/distributor inspect \
--config <config-path> \
--pipeline <pipeline-id> \
--bundle daily/2026-06-01
```
Create a manifest for a local producer bundle:
```sh
@@ -152,13 +175,13 @@ Warnings are objects in the top-level `warnings` array and are not printed again
Command-specific JSON results:
- `version`: application name and version.
- `validate`: bundle count and discovered bundle identifiers.
- `inspect`: bundle path, id, created timestamp, digest, file count, total size, and manifest file records.
- `validate`: bundle count and discovered bundle identifiers. Configured source results also include pipeline id and source backend.
- `inspect`: bundle path, id, created timestamp, digest, file count, total size, and manifest file records. Configured source results also include pipeline id and source backend.
- `manifest create`: manifest path, bundle root, id, created timestamp, digest, file count, and file records.
- `run`: dry-run status, pipeline summaries, destination action records, output records, final counters, warnings, and partial failure records.
## Diagnostics
Use `manifest create` when a local producer has written bundle files but not `manifest.json`. Use `validate` before publication when a producer has written a new bundle. Use `inspect` to confirm normalized ids, timestamps, digests, file paths, and file sizes.
Use `manifest create` when a local producer has written bundle files but not `manifest.json`. Use `validate` before publication when a producer has written a new bundle; use configured source mode when the bundle is already on an SSH or S3 source. Use `inspect` to confirm normalized ids, timestamps, digests, file paths, and file sizes.
For symptom-oriented recovery steps, see [troubleshooting](troubleshooting.md). For destination state and retry behavior, see [operations](operations.md). For config fields and defaults, see [configuration](config.md).