Update cleanup internals documentation
This commit is contained in:
@@ -26,9 +26,22 @@ The runner:
|
|||||||
|
|
||||||
Destination failures are collected while later destinations continue to run. Source open and source discovery failures stop the run because there are no valid bundles to fan out.
|
Destination failures are collected while later destinations continue to run. Source open and source discovery failures stop the run because there are no valid bundles to fan out.
|
||||||
|
|
||||||
|
## Run implementation
|
||||||
|
|
||||||
|
`run.go` contains the public `Run` entrypoint and the main configuration orchestration path. Package-local run helpers are grouped by responsibility:
|
||||||
|
|
||||||
|
- `run_selection.go`: destination bundle selection, path mapping decisions, and fixed-path warnings;
|
||||||
|
- `run_warnings.go`: secret and SSH warning data;
|
||||||
|
- `run_output.go`: text plan lines, JSON action records, and output projections;
|
||||||
|
- `run_summary.go`: summary counters and JSON summary records;
|
||||||
|
- `run_failures.go`: destination failure aggregation and partial-result detection;
|
||||||
|
- `run_notify.go`: notification event projection and action filtering.
|
||||||
|
|
||||||
|
These helpers remain in `internal/app` because command output, warning collection, destination failure aggregation, notifier handoff, and backend construction are app-owned orchestration concerns.
|
||||||
|
|
||||||
## Backend and transform wiring
|
## Backend and transform wiring
|
||||||
|
|
||||||
The app-level backend factory registers local, SSH, and S3 backends for execution. S3 explicit credential references are resolved through the config environment resolver before adapter construction.
|
The app-level backend factory registers local, SSH, and S3 backends for execution. Source and destination backend config is converted through a shared app-local open spec before adapter construction. S3 explicit credential references are resolved through the config environment resolver.
|
||||||
|
|
||||||
The app-level transform registry registers Markdown-to-HTML using `internal/transform/markdown`. Lower-level publish code receives a resolver and does not import concrete transform implementations.
|
The app-level transform registry registers Markdown-to-HTML using `internal/transform/markdown`. Lower-level publish code receives a resolver and does not import concrete transform implementations.
|
||||||
|
|
||||||
|
|||||||
@@ -20,6 +20,8 @@ The source manifest requires:
|
|||||||
|
|
||||||
Each file requires `path`, `sha256`, and `size`. Digests must use lowercase `sha256:<64 hex>` format. `created` must parse as RFC3339.
|
Each file requires `path`, `sha256`, and `size`. Digests must use lowercase `sha256:<64 hex>` format. `created` must parse as RFC3339.
|
||||||
|
|
||||||
|
`pkg/bundle.ValidateDigest` is the canonical digest format validator for producer-facing and internal code. `internal/bundle.ValidateDigest` delegates to that public validator so source manifests and destination state use the same digest grammar.
|
||||||
|
|
||||||
## Validation
|
## Validation
|
||||||
|
|
||||||
`pkg/bundle.ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest.
|
`pkg/bundle.ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest.
|
||||||
|
|||||||
@@ -26,6 +26,19 @@ Storage errors use typed categories such as not found, already exists, invalid p
|
|||||||
|
|
||||||
Backends may wrap implementation-specific errors, but callers should receive storage errors where practical. Traversal can stop cleanly with `ErrStopWalk`.
|
Backends may wrap implementation-specific errors, but callers should receive storage errors where practical. Traversal can stop cleanly with `ErrStopWalk`.
|
||||||
|
|
||||||
|
## Traversal helpers
|
||||||
|
|
||||||
|
Backends own their traversal mechanics. The local adapter owns filesystem walking, the SSH adapter owns SFTP directory walking, and the S3 adapter owns object listing and pagination.
|
||||||
|
|
||||||
|
`internal/storage` owns the shared callback emission rules used by backends:
|
||||||
|
|
||||||
|
- context cancellation is checked before callback emission;
|
||||||
|
- `WalkOptions.Limit` bounds the number of emitted entries;
|
||||||
|
- `ErrStopWalk` stops traversal without becoming a caller-visible error;
|
||||||
|
- callback errors are wrapped as storage walk errors.
|
||||||
|
|
||||||
|
`storage.HasAny(ctx, backend, prefix)` provides the shared destination-content check. It calls `Walk` with non-recursive, limit-one traversal and stops after the first emitted entry.
|
||||||
|
|
||||||
## Deletion
|
## Deletion
|
||||||
|
|
||||||
`DeleteManagedBundle` may delete listed managed outputs plus `.distributor.json`.
|
`DeleteManagedBundle` may delete listed managed outputs plus `.distributor.json`.
|
||||||
|
|||||||
@@ -101,7 +101,7 @@ Do not edit `.distributor.json` by hand during normal operation. If it is missin
|
|||||||
|
|
||||||
## Go Producer Bundles
|
## Go Producer Bundles
|
||||||
|
|
||||||
Go producer applications can import `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to create complete local source bundles with the same path, digest, timestamp, and validation rules used by `distributor`.
|
Go producer applications can import `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to create complete local source bundles with the same path, digest, timestamp, and validation rules used by `distributor`. The package also exposes digest helpers, including `ValidateDigest`, for producer code that needs to validate lowercase `sha256:<64 hex>` strings before writing manifests.
|
||||||
|
|
||||||
Minimal producer-side bundle creation:
|
Minimal producer-side bundle creation:
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user