Implement single-owner managed takeover
This commit is contained in:
@@ -418,6 +418,24 @@ pipelines:
|
||||
|
||||
Every output path in a shared root belongs to exactly one `pipeline_id` and `destination_id`. A different owner planning the same path fails as a conflict.
|
||||
|
||||
## Takeover Policy
|
||||
|
||||
```yaml
|
||||
takeover:
|
||||
mode: same_pipeline
|
||||
```
|
||||
|
||||
- `takeover.mode`: optional. Accepted values are `same_pipeline`, `same_source`, `any_managed`, and `never`; default is `same_pipeline`.
|
||||
|
||||
Takeover controls when a single-owner destination may normally replace valid distributor-managed state whose pipeline, destination, or source identity differs from the current publication.
|
||||
|
||||
- `same_pipeline`: replace managed state owned by the same pipeline. The previous destination id and source id may differ.
|
||||
- `same_source`: replace managed state only when the existing source manifest id matches the current source id.
|
||||
- `any_managed`: replace any valid distributor-managed single-owner state at the selected destination bundle path.
|
||||
- `never`: do not replace identity or source conflicts without the explicit forced replacement workflow.
|
||||
|
||||
Takeover does not apply to unmanaged content, invalid destination state, same-created digest conflicts, same-source destination-newer comparisons, or shared-root output paths owned by another owner.
|
||||
|
||||
## Reconciliation Policy
|
||||
|
||||
```yaml
|
||||
@@ -529,6 +547,7 @@ Defaults are applied after YAML decoding and before validation:
|
||||
- `links.primary: auto` when a `links` block is present and `primary` is omitted
|
||||
- `state.mode: single_owner`
|
||||
- `reconciliation.mode: replace`
|
||||
- `takeover.mode: same_pipeline`
|
||||
- `retention.prune.enabled: false`
|
||||
- `transfer.on_destination_same: skip`
|
||||
- `transfer.on_destination_older: replace`
|
||||
|
||||
@@ -8,7 +8,7 @@ Audience: developers and LLM coding agents changing `internal/publish`.
|
||||
|
||||
## Inputs And Outputs
|
||||
|
||||
Inputs are a source bundle, source backend, destination backend, pipeline id, destination id, destination bundle path, path mapping mode, publish policy, transform policy, optional link policy, state policy, reconciliation policy, transformer resolver, transfer policy, distributor version, and force flag.
|
||||
Inputs are a source bundle, source backend, destination backend, pipeline id, destination id, destination bundle path, path mapping mode, publish policy, transform policy, optional link policy, state policy, reconciliation policy, takeover policy, transformer resolver, transfer policy, distributor version, and force flag.
|
||||
|
||||
Output from planning is a `Plan` with action, reason, destination identity, selected outputs, state mode, owner scope, reconciliation mode, optional existing single-owner or shared-root state, optional primary URL, and force metadata. Shared-root plans also expose other-owner outputs to preserve, current-owner outputs retained by merge, current-owner outputs deleted by replace, and current-owner outputs to write. Execution writes selected source outputs, generated outputs, and `.distributor.json` for executable publish or replacement actions.
|
||||
|
||||
@@ -20,7 +20,7 @@ External destination state semantics are documented in `docs/integrations/destin
|
||||
|
||||
## Config Fields Used
|
||||
|
||||
The package consumes already-defaulted config values for destination `publish`, `transform`, `links`, `state`, `reconciliation`, `transfer`, and path mapping mode. It uses `config.ValidatePublishTransformPolicy` for publish/transform consistency.
|
||||
The package consumes already-defaulted config values for destination `publish`, `transform`, `links`, `state`, `reconciliation`, `takeover`, `transfer`, and path mapping mode. It uses `config.ValidatePublishTransformPolicy` for publish/transform consistency.
|
||||
|
||||
## Adapters Used
|
||||
|
||||
@@ -28,15 +28,15 @@ The package depends on `internal/storage.Backend` for source and destination IO,
|
||||
|
||||
## State And Manifest Behavior
|
||||
|
||||
Planning inspects destination state through `internal/state`, compares it with the source manifest, and maps comparison outcomes plus transfer policy into actions: `publish_new`, `replace_older`, `force_replace`, `skip_same`, `skip_destination_newer`, `fail_conflict`, or `fail_unmanaged`.
|
||||
Planning inspects destination state through `internal/state`, compares it with the source manifest, and maps comparison outcomes plus transfer and takeover policy into actions: `publish_new`, `replace_older`, `replace_takeover`, `force_replace`, `skip_same`, `skip_destination_newer`, `fail_conflict`, or `fail_unmanaged`.
|
||||
|
||||
Single-owner destinations compare the whole destination state against the configured pipeline and destination ids. Shared-root destinations compare only the current owner scope, keyed by pipeline id and destination id. An absent shared-root owner is publishable for that owner unless a planned output collides with unmanaged storage content. Planned writes to a path owned by another shared-root owner fail as conflicts.
|
||||
Single-owner destinations compare the whole destination state against the configured pipeline and destination ids. Valid managed identity and source conflicts can become `replace_takeover` when `takeover.mode` allows them. Shared-root destinations compare only the current owner scope, keyed by pipeline id and destination id. An absent shared-root owner is publishable for that owner unless a planned output collides with unmanaged storage content. Planned writes to a path owned by another shared-root owner fail as conflicts.
|
||||
|
||||
Execution writes destination state after selected outputs are written. Destination state includes copied source output metadata, generated output metadata, output timestamps, embedded source manifest, reconciliation metadata, link metadata when configured, pipeline id, destination id, and publication timestamps.
|
||||
|
||||
## Skip And Resume Behavior
|
||||
|
||||
`skip_same` and `skip_destination_newer` execute as no-ops. Replacement-mode single-owner updates remove managed output paths from existing state plus `.distributor.json`, verify the destination is empty, and write state whose outputs are exactly the new plan. Replacement-mode shared-root updates remove only current-owner omitted outputs and preserve unrelated owners. Merge-mode updates retain omitted managed outputs, overwrite only paths already recorded as managed, reject unmanaged destination path collisions, and write cumulative output state. Failed writes trigger cleanup where practical; merge cleanup removes only newly created outputs from the failed attempt.
|
||||
`skip_same` and `skip_destination_newer` execute as no-ops. Replacement-mode single-owner updates remove managed output paths from existing state plus `.distributor.json`, verify the destination is empty, and write state whose outputs are exactly the new plan. Single-owner `replace_takeover` uses the same managed replacement mechanics and does not retain omitted outputs through merge reconciliation. Replacement-mode shared-root updates remove only current-owner omitted outputs and preserve unrelated owners. Merge-mode same-source updates retain omitted managed outputs, overwrite only paths already recorded as managed, reject unmanaged destination path collisions, and write cumulative output state. Failed writes trigger cleanup where practical; merge cleanup removes only newly created outputs from the failed attempt.
|
||||
|
||||
Shared-root execution writes schema version `3` state. It preserves unrelated owner records and outputs, updates only the publishing owner metadata, preserves root `created_at`, and updates root `updated_at` after successful state writes. Compatible single-owner state for the same pipeline and destination is converted to shared-root state on successful publish.
|
||||
|
||||
|
||||
@@ -67,12 +67,15 @@ Published destination bundle paths contain `.distributor.json`. See [Destination
|
||||
- No destination state and no destination content: publish new outputs.
|
||||
- Matching destination state: skip as already published.
|
||||
- Older destination state for the same source id: replace if transfer policy allows it.
|
||||
- Newer destination state: skip by default.
|
||||
- Invalid destination state, identity mismatch, different source id, or same-created digest mismatch: fail by default.
|
||||
- Newer destination state for the same source id: skip by default.
|
||||
- Valid single-owner state with an identity or source mismatch: replace only when destination `takeover.mode` allows it.
|
||||
- Invalid destination state, identity or source mismatches not allowed by `takeover.mode`, or same-created digest mismatch: fail by default.
|
||||
- Content without `.distributor.json`: fail as unmanaged content by default.
|
||||
|
||||
When destination state is older than the source, `transfer.on_destination_older` controls whether publication may proceed and `reconciliation.mode` controls how managed outputs are updated.
|
||||
|
||||
For single-owner takeover replacement, `reconciliation.mode: merge` does not retain omitted outputs from the previous source identity. The destination is rewritten as a managed replacement for the current source.
|
||||
|
||||
`reconciliation.mode: replace` is the default. It deletes only managed output paths recorded in `.distributor.json` plus the state file, verifies the destination bundle path is empty, then writes the newly planned outputs and state. The new state `outputs` array is exactly the newly planned output set.
|
||||
|
||||
`reconciliation.mode: merge` retains prior managed outputs that are omitted from the new plan. It overwrites planned paths only when those paths are already recorded in existing state as managed. If a newly planned path already exists in storage but is not recorded in state, publication fails as an unmanaged path collision. The new state `outputs` array is the cumulative managed output set.
|
||||
@@ -150,12 +153,13 @@ For single-owner state, the state owner must match the selected pipeline and des
|
||||
|
||||
`run --dry-run` loads config, resolves credentials, discovers source bundles, opens destinations, inspects destination state, builds publish plans, and prints actions. It does not write outputs, `.distributor.json`, or SSH `known_hosts` entries. For reconciliation, dry runs report the same high-level action labels as execution; inspect the configured destination's `reconciliation.mode` to determine whether `replace_older` will replace the managed set or merge into it.
|
||||
|
||||
For shared-root destinations, dry runs are owner-scoped. A `replace_older` action replaces or merges only the current owner according to `reconciliation.mode`; unrelated owners remain managed by the shared-root state.
|
||||
For shared-root destinations, dry runs are owner-scoped. A `replace_older` action replaces or merges only the current owner according to `reconciliation.mode`; unrelated owners remain managed by the shared-root state. Paths owned by another owner still fail as conflicts.
|
||||
|
||||
Review these action labels before publishing:
|
||||
|
||||
- `publish_new`: destination state is absent, or a shared-root owner is absent and planned paths are publishable.
|
||||
- `replace_older`: destination state is older than the source.
|
||||
- `replace_takeover`: single-owner destination state is valid managed state and `takeover.mode` allows replacement across an identity or source mismatch.
|
||||
- `skip_same`: destination state already matches the source.
|
||||
- `skip_destination_newer`: destination state is newer than the source and is skipped.
|
||||
- `force_replace`: destructive replacement selected because `--force` is present and policy permits it.
|
||||
|
||||
@@ -205,7 +205,7 @@ Reference: [Operations](operations.md#forced-replacement-workflow).
|
||||
|
||||
Symptom: `fail_conflict`, `destination source id differs`, `same id and created time but different digest`, `pipeline id ... does not match`, or `destination id ... does not match`.
|
||||
|
||||
Likely cause: `.distributor.json` belongs to a different pipeline, destination, source id, or same-created source with different content.
|
||||
Likely cause: `.distributor.json` belongs to a different pipeline, destination, source id, or same-created source with different content. For single-owner state, identity and source mismatches can publish as `replace_takeover` only when destination `takeover.mode` allows them.
|
||||
|
||||
Diagnostic:
|
||||
|
||||
@@ -214,7 +214,7 @@ cat <destination-path>/.distributor.json
|
||||
go run ./cmd/distributor inspect <source-root>
|
||||
```
|
||||
|
||||
Safe fix: verify the source and destination are intended to match. Use a separate destination path for unrelated content. To replace the existing state, configure `transfer.on_conflict: replace`, preview with `--dry-run --force`, then publish with `--force`.
|
||||
Safe fix: verify the source and destination are intended to match. Use a separate destination path for unrelated content. For normal single-owner managed replacement, configure destination `takeover.mode` to match the intended ownership boundary. To force exceptional replacement, configure `transfer.on_conflict: replace`, preview with `--dry-run --force`, then publish with `--force`.
|
||||
|
||||
Reference: [Operations](operations.md#destination-state-and-retry-behavior).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user