distributor/docs/roadmap/takeover.md

Managed Destination Takeover Roadmap

This roadmap defines planned replacement policy for distributor-managed destination content when the existing destination state does not describe the same source bundle as the current publication.

Current behavior is intentionally conservative. A destination may be replaced automatically when its existing state describes the same source id and an older source creation time. If the existing state belongs to a different source id, pipeline id, destination id, or shared-root owner, distributor reports a conflict unless an explicit forced replacement workflow is selected where supported.

That strict default was useful for the initial implementation, but it is too rigid for producers that intentionally publish newer generated content to the same destination path from different producer tasks or bundle ids. distributor is primarily a distribution tool: by default, a valid producer request should publish the requested bundle unless doing so would claim unmanaged content or cross an ownership boundary that is likely to surprise the operator.

Goals

• Add a destination-level policy that defines when normal, non-force publication may replace existing distributor-managed content whose source or owner identity differs from the current source.
• Default the policy to same_pipeline, so a pipeline may distribute the latest valid bundle to its configured destination paths without requiring every producer task to reuse one source manifest id.
• Preserve strong safety boundaries for unmanaged content, invalid destination state, and cross-pipeline ownership by default.
• Keep reconciliation policy focused on how managed output sets are updated after replacement is allowed.
• Keep transfer policy focused on same-source comparison outcomes and explicit forced replacement behavior.
• Preserve clear dry-run and JSON output so takeover behavior is visible before files are changed.

Non-Goals

• Do not allow automatic adoption of unmanaged destination files.
• Do not allow invalid .distributor.json state to be overwritten without the existing explicit force workflow.
• Do not make producers select destination ownership or takeover policy through source manifests or HTTP upload requests.
• Do not replace state.mode, reconciliation.mode, path_mapping.mode, or transfer policy.
• Do not introduce broad synchronization behavior outside configured destination bundle paths.

Configuration

Add destination-level takeover policy:

destinations:
  - id: latest
    backend: local
    path: /srv/reports/weather/latest/tomorrow
    path_mapping:
      mode: fixed
    takeover:
      mode: same_pipeline

Accepted takeover.mode values:

• same_pipeline: default. Normal publication may replace valid distributor-managed state or managed output paths owned by the same pipeline, even when the previous source id or destination id differs.
• same_source: normal publication may replace only when the destination state has the same source manifest id as the current source. This is closest to the current strict managed replacement behavior.
• any_managed: normal publication may replace any valid distributor-managed state or managed output path in the selected destination bundle path, regardless of pipeline id, destination id, or source id.
• never: normal publication never performs takeover replacement for identity or source conflicts. Same-source older/newer behavior remains governed by transfer, and explicit forced replacement remains governed by --force and transfer.on_conflict.

The field is destination-local because different destinations from the same pipeline can have different ownership expectations. Archive destinations often want stricter behavior, while latest-style fixed destinations often want same_pipeline or, after explicit operator review, any_managed.

Policy Semantics

Takeover policy applies only after destination state has been parsed and validated as distributor-managed state.

For single-owner state:

• same_pipeline permits replacement when the existing state pipeline_id matches the current pipeline id. The previous destination_id and source id may differ.
• same_source permits replacement only when the existing source manifest id matches the current source manifest id.
• any_managed permits replacement of any valid single-owner destination state.
• never does not permit takeover replacement.

For shared-root state:

• Takeover is evaluated per managed output path that collides with the current plan and per current owner record when one already exists.
• same_pipeline permits the current destination owner to take over output paths owned by another destination under the same pipeline.
• same_source permits takeover only when the colliding output owner records the same source manifest id as the current source.
• any_managed permits takeover of colliding managed output paths owned by any owner in the shared root.
• never preserves the current owner-scoped conflict behavior.

Takeover never applies to:

• destination content with no valid .distributor.json;
• invalid destination state;
• planned paths that exist in storage but are not recorded as managed;
• source digest mismatches or invalid source manifests;
• destination newer/same-created digest comparisons for the same source id, except where existing transfer policy already permits replacement.

Relationship To Existing Policies

takeover.mode answers this question:

May this destination normally replace valid managed content whose owner or source identity differs from the current publication?

reconciliation.mode continues to answer:

Once replacement is allowed, should omitted managed outputs be removed (replace) or retained (merge)?

transfer continues to answer:

What should happen for same-source comparisons, destination-newer state, and explicit forced conflict replacement?

--force remains an operator workflow for exceptional cases. Takeover policy is not a replacement for forced replacement of unmanaged content or invalid state.

Publish Planning

Add a new planned action for successful takeover replacement, tentatively named replace_takeover.

replace_takeover should execute through the same bounded managed replacement machinery as other managed replacements, with one additional source-identity safety rule: when takeover crosses source id or owner identity, omitted outputs from the previous source must not be retained merely because the destination is configured with reconciliation.mode: merge. Retaining omitted outputs is valid only when those outputs still belong to the same source identity being published. This keeps destination state from claiming that outputs derived from an old source belong to the new source manifest.

• single-owner replace deletes only managed outputs recorded in existing state plus .distributor.json, then writes the new outputs and state;
• single-owner merge retains omitted managed outputs only for same-source updates; cross-source takeover behaves as a managed replacement for the affected owner;
• shared-root replace deletes only output records taken over by the current owner or omitted from the current owner according to the shared-root plan;
• shared-root merge preserves unrelated non-conflicting managed outputs and updates ownership for paths explicitly taken over, but does not retain omitted outputs from a different source under the taking-over owner.

Text output, JSON output, dry-run output, and summary counters should identify takeover replacements separately from replace_older and force_replace.

Destination State Results

After a successful takeover, destination state must describe the current pipeline, destination, source manifest, outputs, links, reconciliation policy, and state mode.

For single-owner state, the destination bundle path has one current owner after publication: the publishing pipeline and destination.

For shared-root state, only the affected owner records and output records are changed. Unrelated owners and non-conflicting outputs remain recorded.

No source manifest schema change is required. No destination state schema change is expected unless implementation discovers that takeover metadata must be persisted for recovery or audit. The preferred initial design is to make takeover a planning decision, not a new persisted state concept.

Documentation Impact

When this feature is implemented, current-behavior docs should explain the new policy where operators and maintainers already look for publication safety, state comparison, and configuration behavior:

• docs/config.md: document takeover.mode, default same_pipeline, accepted values, examples for archive and latest destinations, and interaction with transfer, reconciliation, and --force.
• docs/operations.md: explain takeover dry-runs, normal replacement safety, shared-root behavior, and recovery guidance.
• docs/troubleshooting.md: update conflict guidance for destination source id differs from source, pipeline/destination mismatch, and shared-root ownership conflicts.
• docs/integrations/destination-state.md: describe how takeover affects comparison and state rewriting without changing the source manifest contract.
• docs/internal/publish.md, docs/internal/state.md, and docs/internal/config.md: document package responsibilities and invariants.
• docs/policy/architecture.md: clarify that the default posture is to distribute valid producer bundles through configured destinations while protecting unmanaged content and cross-pipeline ownership by default.
• docs/policy/development.md: add takeover.mode to the list of config changes that must be validated and documented when publish safety changes.

Safety Rules

• Takeover must be dry-runnable.
• Takeover must operate only inside the resolved destination bundle path.
• Takeover must never delete parent paths, sibling paths, or unmanaged files.
• Takeover must never claim storage content that is not recorded in valid destination state.
• any_managed should be documented as an intentional cross-pipeline ownership policy for tightly controlled destinations, not as the recommended default for archives.
• Archive-style destinations should generally use same_source if each bundle id represents a distinct immutable artifact.

Implementation Reference

The staged implementation plan for this feature lives in docs/roadmap/implementation.md. This document remains the feature contract and policy reference; implementation sequencing, package-by-package work, and test commands belong in the implementation roadmap.