Add public bundle manifest package
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
## Purpose
|
||||
|
||||
`internal/bundle` parses, discovers, and validates source bundles through the storage interface.
|
||||
`internal/bundle` discovers and validates source bundles through the storage interface. The source manifest model, manifest parsing, manifest validation, path rules, and digest calculation come from `pkg/bundle` so producer-facing APIs and distributor validation share one manifest contract.
|
||||
|
||||
## Inputs and outputs
|
||||
|
||||
@@ -22,9 +22,9 @@ Each file requires `path`, `sha256`, and `size`. Digests must use lowercase `sha
|
||||
|
||||
## Validation
|
||||
|
||||
`ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest.
|
||||
`pkg/bundle.ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest.
|
||||
|
||||
Storage-backed bundle validation additionally checks file existence, regular-file type, file size, and per-file SHA-256.
|
||||
Storage-backed bundle validation in `internal/bundle` additionally checks file existence, regular-file type, file size, and per-file SHA-256 for configured storage backends.
|
||||
|
||||
The bundle digest is SHA-256 of a deterministic JSON array of file records in manifest order with fields `path`, `sha256`, and `size`.
|
||||
|
||||
@@ -38,11 +38,11 @@ Manifest parsing and validation fail before destination planning. Storage-backed
|
||||
|
||||
## Boundaries
|
||||
|
||||
Bundle code uses `internal/storage` and does not import concrete adapters. CLI local path support is wired in `internal/app`.
|
||||
Internal bundle discovery uses `internal/storage` and does not import concrete adapters. Producer-side local filesystem manifest building and validation belong to `pkg/bundle`. CLI local path support is wired in `internal/app`.
|
||||
|
||||
## Tests
|
||||
|
||||
Before changing bundle behavior, inspect tests under `internal/bundle`.
|
||||
Before changing bundle behavior, inspect tests under `pkg/bundle` and `internal/bundle`.
|
||||
|
||||
## Invariants
|
||||
|
||||
|
||||
@@ -76,6 +76,28 @@ Each published destination bundle contains `.distributor.json`. This file is the
|
||||
|
||||
Do not edit `.distributor.json` by hand during normal operation. If it is missing or invalid while destination files remain, `distributor` treats the destination as unmanaged or conflicted.
|
||||
|
||||
## Go Producer Manifests
|
||||
|
||||
Go producer applications can import `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to create source manifests with the same path, digest, timestamp, and validation rules used by `distributor`.
|
||||
|
||||
Minimal producer-side manifest creation:
|
||||
|
||||
```go
|
||||
manifest, err := bundle.BuildManifest(bundle.BuildOptions{
|
||||
Root: outputDir,
|
||||
ID: "reports.example.2026-05-30",
|
||||
Files: []string{"report.md", "summary.txt"},
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := bundle.WriteManifest(outputDir, manifest, bundle.WriteManifestOptions{}); err != nil {
|
||||
return err
|
||||
}
|
||||
```
|
||||
|
||||
Use explicit `Files` to preserve caller order, or `Scan: true` to recursively include regular files under `Root` in deterministic slash-path order. Scan mode includes dotfiles, excludes files named `manifest.json` or `.distributor.json`, and rejects symlinks.
|
||||
|
||||
## Dry Runs
|
||||
|
||||
`--dry-run` loads and validates config, discovers source bundles, inspects destination state, plans outputs, and prints summary lines. It does not write output files, destination state, or SSH `known_hosts` entries.
|
||||
|
||||
@@ -28,7 +28,7 @@ The current core workflow is:
|
||||
8. optionally transform Markdown to HTML for that destination;
|
||||
9. publish selected source and generated artifacts;
|
||||
10. write `.distributor.json` as the destination sentinel/state file;
|
||||
11. run the notification hook, which is a no-op in the MVP.
|
||||
11. run the notification hook, whose default implementation is currently a no-op.
|
||||
|
||||
## Pipeline Model
|
||||
|
||||
@@ -53,7 +53,7 @@ A source bundle is a directory containing `manifest.json`.
|
||||
|
||||
`manifest.json` is the sole producer-to-`distributor` contract. `distributor` must not rely on producer-specific work directory layouts, filenames, metadata, or conventions outside the configured source root and the source manifest.
|
||||
|
||||
The MVP source manifest schema is intentionally minimal:
|
||||
The source manifest schema is intentionally minimal:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -73,7 +73,7 @@ The MVP source manifest schema is intentionally minimal:
|
||||
|
||||
Required fields:
|
||||
|
||||
- `schema_version`: source manifest schema version. MVP value: `1`.
|
||||
- `schema_version`: source manifest schema version. Current value: `1`.
|
||||
- `id`: stable bundle identifier.
|
||||
- `digest`: SHA-256 digest for the listed files.
|
||||
- `created`: RFC3339 timestamp. UTC is preferred; explicit offsets are allowed.
|
||||
@@ -187,10 +187,11 @@ Avoid dependencies for small conveniences. Do not let external dependency types
|
||||
Use this current layout unless the project has a documented reason to differ:
|
||||
|
||||
- `cmd/distributor`: application entrypoint only.
|
||||
- `pkg/bundle`: public producer-facing source manifest model, digest logic, parsing, building, manifest writing, and local validation helpers.
|
||||
- `internal/app`: application orchestration and top-level use cases.
|
||||
- `internal/cli`: CLI command definitions, flags, argument parsing, and command wiring.
|
||||
- `internal/config`: configuration structs, defaults, loading, precedence, and validation.
|
||||
- `internal/bundle`: source manifest parsing, source bundle discovery, source digest validation, and source bundle model.
|
||||
- `internal/bundle`: storage-backed source bundle discovery and validation over the public manifest contract.
|
||||
- `internal/state`: `.distributor.json` parsing, validation, comparison, and output metadata.
|
||||
- `internal/storage`: backend interfaces, shared path/resource types, backend registry, and storage errors.
|
||||
- `internal/adapters/local`: local filesystem backend.
|
||||
|
||||
@@ -6,10 +6,11 @@ Use it with `docs/policy/architecture.md` and `docs/policy/documentation.md`.
|
||||
## Repository Layout
|
||||
|
||||
- `cmd/distributor`: executable entrypoint only.
|
||||
- `pkg/bundle`: public producer-facing source manifest helpers.
|
||||
- `internal/app`: top-level use cases for `run`, `validate`, and `inspect`.
|
||||
- `internal/cli`: standard-library command parsing, flags, help text, and command wiring.
|
||||
- `internal/config`: YAML configuration structs, loading, defaults, and validation.
|
||||
- `internal/bundle`: source bundle discovery, manifest parsing, digest calculation, and validation.
|
||||
- `internal/bundle`: storage-backed source bundle discovery and validation using the public manifest contract.
|
||||
- `internal/state`: destination `.distributor.json` parsing, validation, and comparison.
|
||||
- `internal/storage`: backend interface, registry, logical path rules, typed errors, and shared storage helpers.
|
||||
- `internal/adapters/local`: local filesystem backend.
|
||||
@@ -24,7 +25,7 @@ Use it with `docs/policy/architecture.md` and `docs/policy/documentation.md`.
|
||||
- `docs`: current user, operator, policy, internal, and roadmap documentation.
|
||||
- `examples`: copyable example configs and source bundles.
|
||||
|
||||
Do not create new top-level package families such as `pkg`, `internal/stage`,
|
||||
Do not create new top-level package families such as public `pkg/...` packages beyond `pkg/bundle`, `internal/stage`,
|
||||
`internal/modules`, or service-specific adapter directories unless the
|
||||
architecture policy or a current roadmap explicitly calls for them.
|
||||
|
||||
@@ -73,7 +74,7 @@ GOCACHE=/private/tmp/distributor-gocache GOMODCACHE=/private/tmp/distributor-gom
|
||||
- Keep adapter packages thin. Backend-specific filesystem or service behavior belongs in adapters; bundle, state, transform, and publish policy belongs outside adapters.
|
||||
- Preserve public CLI behavior, config semantics, manifest schema, destination state schema, and implemented backend behavior unless the current task explicitly changes them.
|
||||
- Use `storage.DisplayPath`, `storage.StateFileName`, `storage.StatePath`, and `storage.ManagedBundleTargets` instead of duplicating those conventions.
|
||||
- Use `bundle.ValidateManifest` for normalized source manifest semantics, including embedded source manifests in destination state.
|
||||
- Use `pkg/bundle` for normalized source manifest semantics. Internal packages should reach those rules through `internal/bundle` when they also need storage-backed bundle discovery or validation.
|
||||
- Use `config.ValidatePublishTransformPolicy` for publish and transform policy combinations.
|
||||
- Do not import concrete transform implementations from `internal/publish`; app-level wiring owns transform registration.
|
||||
- Do not import `internal/testutil` from production code.
|
||||
|
||||
Reference in New Issue
Block a user