Harden cross-backend run diagnostics
This commit is contained in:
@@ -82,7 +82,7 @@ go run ./cmd/distributor run --config examples/fan-out.yml --dry-run
|
||||
|
||||
## Output
|
||||
|
||||
`run` prints the number of configured pipelines, one line per pipeline, one line per planned destination action, and a final status line. Actions include:
|
||||
`run` prints the number of configured pipelines, one line per pipeline, one line per planned destination action, and a final status line. Destination action lines include the bundle path, destination id, destination backend, action, outputs, and reason. Actions include:
|
||||
|
||||
- `publish_new`: destination has no managed state and is empty.
|
||||
- `replace_older`: destination state is older than the source manifest.
|
||||
|
||||
@@ -39,6 +39,8 @@ Dry-run still loads config, opens backends, discovers bundles, inspects destinat
|
||||
|
||||
`Run` returns immediately for config loading errors, context cancellation before work starts, source open errors, and source discovery errors. Per-destination backend, planning, execution, and notification errors are aggregated into one run error after remaining destinations have been attempted.
|
||||
|
||||
Run diagnostics include pipeline id, destination id, destination backend, and bundle path for destination-scoped failures. Source open and discovery failures include the source backend.
|
||||
|
||||
Stdout write errors are returned immediately because the caller's requested output stream can no longer be trusted.
|
||||
|
||||
## Boundaries
|
||||
|
||||
@@ -76,6 +76,8 @@ Do not edit `.distributor.json` by hand during normal operation. If it is missin
|
||||
|
||||
Dry-run output is useful before publishing to confirm actions such as `publish_new`, `replace_older`, `skip_same`, and `skip_destination_newer`.
|
||||
|
||||
Destination action lines include the destination backend, so mixed local, SSH, and S3 fan-out runs can be audited before publication.
|
||||
|
||||
## Retry and Replacement Behavior
|
||||
|
||||
If a destination has matching `.distributor.json`, publication skips it as already published.
|
||||
@@ -90,6 +92,8 @@ If a destination path has files but no valid `.distributor.json`, publication fa
|
||||
|
||||
If one destination fails in a fan-out run, independent later destinations are still planned and executed. The command exits non-zero after printing the final status if any destination failed.
|
||||
|
||||
Errors include the pipeline id, destination id, destination backend, and bundle path where applicable.
|
||||
|
||||
If a write fails during publication, `distributor` attempts to remove outputs written during that failed attempt so a retry does not see those partial outputs as unmanaged destination content.
|
||||
|
||||
After a successful publish or replacement, the internal notifier hook runs. The current default notifier is a no-op. Skipped destinations do not invoke it.
|
||||
|
||||
@@ -286,7 +286,7 @@ Safe fix: adjust the source bundle contents or publish policy so source and gene
|
||||
|
||||
## A run failed after writing some files
|
||||
|
||||
Likely cause: a write failed partway through publication. Local execution attempts to clean up outputs written during the failed attempt.
|
||||
Likely cause: a write failed partway through publication. Local, SSH, and S3 execution attempt to clean up outputs written during the failed attempt.
|
||||
|
||||
Diagnostic:
|
||||
|
||||
@@ -294,4 +294,4 @@ Diagnostic:
|
||||
find <destination-path> -maxdepth 2 -print
|
||||
```
|
||||
|
||||
Safe fix: inspect the destination before retrying. If only unrelated unmanaged files remain, move them aside or choose a clean destination. Re-run with `--dry-run` before publishing again. See [operations](operations.md).
|
||||
Safe fix: use the pipeline id, destination id, backend, and bundle path printed in the run error to inspect the destination before retrying. If only unrelated unmanaged files remain, move them aside or choose a clean destination. Re-run with `--dry-run` before publishing again. See [operations](operations.md).
|
||||
|
||||
Reference in New Issue
Block a user