Implement catalog idempotent publish skips
This commit is contained in:
@@ -293,6 +293,7 @@ Text output is optimized for direct operator use. JSON output is optimized for a
|
||||
- `run --format json` emits a JSON result for partial destination failures, sets `ok` to `false`, includes result details and errors, and exits non-zero.
|
||||
- Warnings are included in JSON output and are printed in text output when relevant.
|
||||
- `run` summaries include `publish_new`, `upsert_additive`, `replace_catalog`, `skip_same`, `force_replace`, `fail_unmanaged`, and `fail_conflict` counters. Destination action records use the same stable action values and include the resolved `destination_path`.
|
||||
- `skip_same` means the planned outputs already match valid catalog metadata, so `run` does not write outputs, rewrite `.distributor.json`, delete files, or notify. The decision is based on catalog metadata and does not read destination file bytes.
|
||||
|
||||
## Diagnostics And Recovery
|
||||
|
||||
|
||||
@@ -33,6 +33,7 @@ Planning inspects destination state through `internal/state` and maps catalog co
|
||||
- `publish_new`: no valid state exists and the destination bundle path is empty.
|
||||
- `upsert_additive`: additive workflow writes planned outputs and retains unrelated catalog outputs.
|
||||
- `replace_catalog`: replacement workflow writes planned outputs and deletes omitted outputs for the current owner.
|
||||
- `skip_same`: valid catalog metadata already matches every planned output.
|
||||
- `force_replace`: explicit bounded replacement selected by `Force`.
|
||||
- `fail_unmanaged`: unmanaged destination content blocks publication.
|
||||
- `fail_conflict`: invalid or unsupported state blocks publication.
|
||||
@@ -43,9 +44,11 @@ Execution writes destination state after selected outputs are written. Catalog o
|
||||
|
||||
## Workflow Behavior
|
||||
|
||||
Additive workflow computes a write set for the planned outputs and preserves catalog outputs for unplanned paths. Existing catalog records for planned paths are replaced by the current owner and source identity.
|
||||
Additive workflow computes a write set for the planned outputs and preserves catalog outputs for unplanned paths. Existing catalog records for planned paths are replaced by the current owner and source identity. When every planned output already matches catalog metadata, planning returns `skip_same` and ignores unrelated retained catalog outputs for the no-op decision.
|
||||
|
||||
Replacement workflow computes a write set for the planned outputs, preserves other-owner outputs, and deletes omitted outputs owned by the current pipeline and destination. It does not need `Force`.
|
||||
Replacement workflow computes a write set for the planned outputs, preserves other-owner outputs, and deletes omitted outputs owned by the current pipeline and destination. It does not need `Force`. Matching planned outputs return `skip_same` only when replacement would not delete omitted outputs for the current owner.
|
||||
|
||||
Catalog skip comparison is metadata-only. It checks pipeline id, destination id, source id, source digest, source creation timestamp, output path, kind, digest, size, generated output source path, generated output transform, and output URL metadata. It does not read destination file bytes, and `skip_same` execution does not write outputs, rewrite `.distributor.json`, delete files, or notify.
|
||||
|
||||
Forced replacement is explicit per request. It deletes the bounded destination bundle path before writing planned outputs and schema version `4` catalog state. Catalog planning selects `force_replace` only when `Force` is true and normal planning would otherwise fail for a non-empty no-state destination, a planned path collision with unmanaged storage content, invalid destination state, or unsupported future destination state.
|
||||
|
||||
|
||||
@@ -144,7 +144,7 @@ Review these action labels before publishing:
|
||||
- `publish_new`: destination state is absent and the destination bundle path is empty.
|
||||
- `upsert_additive`: additive workflow will write planned outputs into valid catalog state.
|
||||
- `replace_catalog`: replacement workflow will write planned outputs and remove omitted outputs for the current owner.
|
||||
- `skip_same`: no-op action value in the run output vocabulary.
|
||||
- `skip_same`: planned outputs already match valid catalog metadata, so publication will not write outputs, rewrite `.distributor.json`, delete files, or notify.
|
||||
- `force_replace`: destructive catalog replacement selected because `--force` is present for unmanaged content, a planned unmanaged path collision, invalid state, or unsupported future state.
|
||||
- `fail_unmanaged`: unmanaged destination content prevents publication.
|
||||
- `fail_conflict`: invalid or unsupported destination state prevents publication.
|
||||
@@ -152,6 +152,8 @@ Review these action labels before publishing:
|
||||
|
||||
Text and JSON summaries count `publish_new`, `upsert_additive`, `replace_catalog`, `skip_same`, `force_replace`, `fail_unmanaged`, `fail_conflict`, and failed destinations separately. JSON output includes warnings, pipeline summaries, destination action records, output records, URLs when configured, final counters, and partial failure details. Fatal setup failures such as unreadable config or invalid secrets do not produce a JSON result document.
|
||||
|
||||
The `skip_same` optimization trusts valid catalog metadata. It compares owner identity, source identity, output path, kind, digest, size, generated output metadata, and URL metadata recorded in `.distributor.json`; it does not read destination file bytes to detect bitrot.
|
||||
|
||||
Fixed destinations add fixed-path warnings during dry runs, including the selected source bundle and replacement warnings when the destination root would be replaced. For fixed destinations, the resolved destination bundle path is the backend root.
|
||||
|
||||
## Forced Replacement Workflow
|
||||
|
||||
Reference in New Issue
Block a user