Update cleanup internals documentation

This commit is contained in:
2026-06-02 19:00:09 +00:00
parent 67dc07435d
commit bb276101d2
4 changed files with 30 additions and 2 deletions

View File

@@ -26,9 +26,22 @@ The runner:
Destination failures are collected while later destinations continue to run. Source open and source discovery failures stop the run because there are no valid bundles to fan out. Destination failures are collected while later destinations continue to run. Source open and source discovery failures stop the run because there are no valid bundles to fan out.
## Run implementation
`run.go` contains the public `Run` entrypoint and the main configuration orchestration path. Package-local run helpers are grouped by responsibility:
- `run_selection.go`: destination bundle selection, path mapping decisions, and fixed-path warnings;
- `run_warnings.go`: secret and SSH warning data;
- `run_output.go`: text plan lines, JSON action records, and output projections;
- `run_summary.go`: summary counters and JSON summary records;
- `run_failures.go`: destination failure aggregation and partial-result detection;
- `run_notify.go`: notification event projection and action filtering.
These helpers remain in `internal/app` because command output, warning collection, destination failure aggregation, notifier handoff, and backend construction are app-owned orchestration concerns.
## Backend and transform wiring ## Backend and transform wiring
The app-level backend factory registers local, SSH, and S3 backends for execution. S3 explicit credential references are resolved through the config environment resolver before adapter construction. The app-level backend factory registers local, SSH, and S3 backends for execution. Source and destination backend config is converted through a shared app-local open spec before adapter construction. S3 explicit credential references are resolved through the config environment resolver.
The app-level transform registry registers Markdown-to-HTML using `internal/transform/markdown`. Lower-level publish code receives a resolver and does not import concrete transform implementations. The app-level transform registry registers Markdown-to-HTML using `internal/transform/markdown`. Lower-level publish code receives a resolver and does not import concrete transform implementations.

View File

@@ -20,6 +20,8 @@ The source manifest requires:
Each file requires `path`, `sha256`, and `size`. Digests must use lowercase `sha256:<64 hex>` format. `created` must parse as RFC3339. Each file requires `path`, `sha256`, and `size`. Digests must use lowercase `sha256:<64 hex>` format. `created` must parse as RFC3339.
`pkg/bundle.ValidateDigest` is the canonical digest format validator for producer-facing and internal code. `internal/bundle.ValidateDigest` delegates to that public validator so source manifests and destination state use the same digest grammar.
## Validation ## Validation
`pkg/bundle.ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest. `pkg/bundle.ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest.

View File

@@ -26,6 +26,19 @@ Storage errors use typed categories such as not found, already exists, invalid p
Backends may wrap implementation-specific errors, but callers should receive storage errors where practical. Traversal can stop cleanly with `ErrStopWalk`. Backends may wrap implementation-specific errors, but callers should receive storage errors where practical. Traversal can stop cleanly with `ErrStopWalk`.
## Traversal helpers
Backends own their traversal mechanics. The local adapter owns filesystem walking, the SSH adapter owns SFTP directory walking, and the S3 adapter owns object listing and pagination.
`internal/storage` owns the shared callback emission rules used by backends:
- context cancellation is checked before callback emission;
- `WalkOptions.Limit` bounds the number of emitted entries;
- `ErrStopWalk` stops traversal without becoming a caller-visible error;
- callback errors are wrapped as storage walk errors.
`storage.HasAny(ctx, backend, prefix)` provides the shared destination-content check. It calls `Walk` with non-recursive, limit-one traversal and stops after the first emitted entry.
## Deletion ## Deletion
`DeleteManagedBundle` may delete listed managed outputs plus `.distributor.json`. `DeleteManagedBundle` may delete listed managed outputs plus `.distributor.json`.

View File

@@ -101,7 +101,7 @@ Do not edit `.distributor.json` by hand during normal operation. If it is missin
## Go Producer Bundles ## Go Producer Bundles
Go producer applications can import `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to create complete local source bundles with the same path, digest, timestamp, and validation rules used by `distributor`. Go producer applications can import `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to create complete local source bundles with the same path, digest, timestamp, and validation rules used by `distributor`. The package also exposes digest helpers, including `ValidateDigest`, for producer code that needs to validate lowercase `sha256:<64 hex>` strings before writing manifests.
Minimal producer-side bundle creation: Minimal producer-side bundle creation: