Normalize internal component documentation
This commit is contained in:
@@ -1,55 +1,53 @@
|
||||
# Bundles
|
||||
# Source Bundle Internals
|
||||
|
||||
Audience: developers and LLM coding agents changing `internal/bundle`.
|
||||
|
||||
## Purpose
|
||||
|
||||
`internal/bundle` discovers and validates source bundles through the storage interface. The source manifest model, manifest parsing, manifest validation, path rules, digest calculation, and producer-side local writer come from `pkg/bundle` so producer-facing APIs and distributor validation share one manifest contract.
|
||||
`internal/bundle` discovers and validates source bundles through the storage abstraction. It adapts the public producer-facing source manifest contract from `pkg/bundle` to local, SSH/SFTP, S3-compatible, and test storage backends.
|
||||
|
||||
## Inputs and outputs
|
||||
## Inputs And Outputs
|
||||
|
||||
Input is a backend-rooted directory tree containing one or more `manifest.json` files. Output is a deterministic list of validated bundles with relative bundle paths and normalized manifest data.
|
||||
|
||||
## Manifest behavior
|
||||
|
||||
The source manifest requires:
|
||||
|
||||
- `schema_version: 1`
|
||||
- `id`
|
||||
- `digest`
|
||||
- `created`
|
||||
- non-empty `files`
|
||||
|
||||
Each file requires `path`, `sha256`, and `size`. Digests must use lowercase `sha256:<64 hex>` format. `created` must parse as RFC3339.
|
||||
|
||||
`pkg/bundle.ValidateDigest` is the canonical digest format validator for producer-facing and internal code. `internal/bundle.ValidateDigest` delegates to that public validator so source manifests and destination state use the same digest grammar.
|
||||
|
||||
## Validation
|
||||
|
||||
`pkg/bundle.ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest.
|
||||
|
||||
Storage-backed bundle validation in `internal/bundle` additionally checks file existence, regular-file type, file size, and per-file SHA-256 for configured storage backends.
|
||||
|
||||
The bundle digest is SHA-256 of a deterministic JSON array of file records in manifest order with fields `path`, `sha256`, and `size`.
|
||||
|
||||
## Discovery
|
||||
|
||||
Discovery walks a storage backend beneath a source root, finds `manifest.json` files, sorts bundle paths lexically, and rejects nested manifests.
|
||||
|
||||
## Failure behavior
|
||||
|
||||
Manifest parsing and validation fail before destination planning. Storage-backed validation fails when listed files are missing, are not regular files, have unexpected sizes, have unexpected SHA-256 digests, or when a source bundle includes unsafe or reserved paths.
|
||||
Inputs are a context, a `storage.Backend`, and a source-root prefix or bundle root path. Outputs are sorted `Bundle` records containing the source-root-relative bundle path and validated manifest.
|
||||
|
||||
## Boundaries
|
||||
|
||||
Internal bundle discovery uses `internal/storage` and does not import concrete adapters. Producer-side local filesystem manifest building, complete bundle writing, and validation belong to `pkg/bundle`. CLI local path support is wired in `internal/app`.
|
||||
`internal/bundle` delegates manifest parsing, digest calculation, source path validation, and manifest validation to `pkg/bundle`. It does not publish files, inspect destination state, choose pipelines, or know concrete backend implementations.
|
||||
|
||||
## Tests
|
||||
The external source bundle file-format contract is documented in `docs/integrations/source-bundle.md`.
|
||||
|
||||
Before changing bundle behavior, inspect tests under `pkg/bundle` and `internal/bundle`.
|
||||
## Config Fields Used
|
||||
|
||||
## Invariants
|
||||
The package does not read config directly. App workflows pass it storage backends that were opened from configured source fields.
|
||||
|
||||
- `manifest.json` is the only source bundle contract.
|
||||
- Source file paths must stay relative to the bundle root.
|
||||
- The top-level bundle digest is derived from manifest file records in order.
|
||||
- Discovery order is lexical and deterministic.
|
||||
- Nested manifests are rejected.
|
||||
## Adapters Used
|
||||
|
||||
The package depends only on `internal/storage.Backend`. Concrete local, SSH/SFTP, S3-compatible, and fake backends are hidden behind that interface.
|
||||
|
||||
## State And Manifest Behavior
|
||||
|
||||
Discovery walks recursively under the source root, finds entries whose basename is `manifest.json`, converts each manifest path to a bundle root, sorts roots, rejects nested bundle roots, and validates each bundle.
|
||||
|
||||
Validation reads `manifest.json`, parses it, stats each manifest-listed file, requires regular files, verifies file sizes, reads file bytes, checks per-file SHA-256 digests, and recomputes the bundle digest.
|
||||
|
||||
## Skip And Resume Behavior
|
||||
|
||||
The package has no skip or resume state. Each call performs discovery or validation from the supplied backend state.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
Failures include invalid storage prefixes, missing manifests, parse errors, nested manifests, unsafe manifest paths, non-regular files, size mismatches, digest mismatches, backend stat/read errors, and no discovered bundles.
|
||||
|
||||
## Tests To Inspect
|
||||
|
||||
- `internal/bundle/*_test.go`
|
||||
- `pkg/bundle/*_test.go`
|
||||
- `internal/storage/fake/*_test.go`
|
||||
|
||||
## Architectural Invariants
|
||||
|
||||
- Source manifest semantics remain owned by `pkg/bundle`.
|
||||
- Discovery order is deterministic.
|
||||
- Nested manifests are rejected before returning bundles.
|
||||
- Source paths stay clean, relative, slash-separated, and confined to the backend root.
|
||||
- Concrete adapters never leak into bundle validation logic.
|
||||
|
||||
Reference in New Issue
Block a user