Add source bundle validation and inspection

This commit is contained in:
2026-05-31 02:07:30 +00:00
parent 3c2f36a6e5
commit 518944e601
25 changed files with 931 additions and 17 deletions

View File

@@ -3,10 +3,10 @@
## Shortest useful command
```sh
go run ./cmd/distributor run --config examples/local-to-local.yml --dry-run
go run ./cmd/distributor validate examples/source-bundle
```
This loads and validates the example config, then prints the resolved pipeline summary without publishing files.
This validates a local source bundle fixture.
## Command overview
@@ -22,7 +22,11 @@ distributor inspect
`run --config <path> --dry-run` loads and validates configuration, then prints a concise summary of configured pipelines and destinations. It does not discover bundles or publish files yet.
`run` without `--dry-run`, `validate`, and `inspect` intentionally fail with a clear `not implemented` error until the corresponding application behavior exists.
`validate <path>` validates a local source bundle directory or a local tree containing source bundles.
`inspect <path>` validates discovered local source bundles and prints a concise normalized summary.
`run` without `--dry-run` intentionally fails with a clear `not implemented` error until execution behavior exists.
## Flag reference
@@ -41,6 +45,18 @@ Each subcommand supports:
## Common workflows
Validate a source bundle:
```sh
go run ./cmd/distributor validate examples/source-bundle
```
Inspect a source bundle:
```sh
go run ./cmd/distributor inspect examples/source-bundle
```
Validate a config file without publishing:
```sh

39
docs/internal/bundle.md Normal file
View File

@@ -0,0 +1,39 @@
# Bundles
## Purpose
`internal/bundle` parses, discovers, and validates source bundles through the storage interface.
## Inputs and outputs
Input is a backend-rooted directory tree containing one or more `manifest.json` files. Output is a deterministic list of validated bundles with relative bundle paths and normalized manifest data.
## Manifest behavior
The source manifest requires:
- `schema_version: 1`
- `id`
- `digest`
- `created`
- non-empty `files`
Each file requires `path`, `sha256`, and `size`. Digests must use lowercase `sha256:<64 hex>` format. `created` must parse as RFC3339.
## Validation
Bundle validation checks source path safety, duplicate file paths, reserved paths, file existence, regular-file type, file size, per-file SHA-256, and the top-level bundle digest.
The bundle digest is SHA-256 of a deterministic JSON array of file records in manifest order with fields `path`, `sha256`, and `size`.
## Discovery
Discovery walks a storage backend beneath a source root, finds `manifest.json` files, sorts bundle paths lexically, and rejects nested manifests.
## Boundaries
Bundle code uses `internal/storage` and does not import local, SSH, or S3 adapters. CLI local path support is wired in `internal/app`.
## Tests
Before changing bundle behavior, inspect tests under `internal/bundle`.