distributor/docs/roadmap/implementation.md

Managed Destination Takeover Implementation Roadmap

This is the active staged implementation plan for docs/roadmap/takeover.md. The feature roadmap defines the target policy and end-state semantics; this document defines the implementation sequence for an LLM coding agent to follow stage by stage.

Future behavior must remain under docs/roadmap/ until each stage is implemented. Preparatory internal stages should not update user-facing current docs. Current-behavior docs should be updated when takeover behavior is wired for operator-facing use.

Current Baseline

distributor already supports local, SSH/SFTP, S3, and HTTP upload source workflows, destination state schemas for single-owner and shared-root state, path mapping, link generation, reconciliation, transfer policy, explicit --force, and text/JSON run output.

Current destination comparison is strict:

• same source manifest skips;
• same source id with older destination state normally replaces;
• same source id with newer destination state normally skips;
• same source id and same creation time with different digest conflicts;
• different source id, pipeline id, destination id, or shared-root output owner conflicts unless explicit force policy applies;
• unmanaged content and invalid state do not become normal managed replacement cases.

The target feature adds destination-level takeover.mode, defaulting to same_pipeline, so valid managed content can be replaced without --force when the configured policy says that ownership/source takeover is expected.

Implementation Principles

• Preserve public behavior until the stage that explicitly changes it.
• Keep source manifests unchanged; takeover is destination configuration and publish planning policy.
• Keep state comparison pure. State comparison may expose structured conflict details, but publish planning decides whether takeover is allowed.
• Keep adapters thin. No local, SSH, or S3 adapter should know takeover policy.
• Keep unmanaged content and invalid destination state outside normal takeover.
• Keep reconciliation.mode, transfer, state.mode, path_mapping.mode, and --force as separate concepts.
• Prefer narrow behavior-preserving refactors over broad publication rewrites.
• Update implemented-behavior docs in the same stage as the behavior change.

Active Implementation Stages

Stage 1: Config Model And Validation

Goal: add the destination config field, defaults, and validation without changing publish behavior.

Source roadmap reference:

• docs/roadmap/takeover.md: Configuration, Policy Semantics, Safety Rules.

Implementation scope:

• Add destination-level TakeoverPolicy to internal/config.
• Add constants for:
  • same_pipeline
  • same_source
  • any_managed
  • never
• Default takeover.mode to same_pipeline in config defaults.
• Validate accepted values with clear field context such as pipelines[0].destinations[0].takeover.mode.
• Preserve strict YAML unknown-field behavior.
• Thread the defaulted policy into existing destination config views or helper structures if those are used by app/publish request construction.
• Do not change publish planning, execution, CLI output, or docs outside roadmap files in this stage.

Tests:

• go test ./internal/config
• Add config tests for omitted takeover, each accepted mode, invalid mode, and unknown nested fields.
• Add or update example-loading tests only if examples are touched, which should not be necessary in this stage.

Completion criteria:

• Every destination has a defaulted takeover.mode.
• Invalid values fail during config validation.
• No run behavior changes because publish planning does not consume the policy yet.

Stage 2: Structured State Comparison Details

Goal: expose enough structured comparison detail for publish planning to decide takeover eligibility without parsing human-readable reason strings.

Source roadmap reference:

• docs/roadmap/takeover.md: Policy Semantics, Relationship To Existing Policies.

Implementation scope:

• Extend internal/state.Comparison or add a package-local structured detail type so callers can distinguish:
  • pipeline id mismatch;
  • destination id mismatch;
  • different source id;
  • same-created digest conflict;
  • destination newer;
  • invalid state;
  • unmanaged content;
  • absent shared-root owner;
  • shared-root managed output owner conflicts, if those are currently reported outside internal/state.
• Keep existing outcome names and reason strings stable where practical.
• Keep comparison functions pure. They should report facts about existing state, not consult takeover.mode, transfer, reconciliation, or force.
• Do not add new persisted state fields.
• Do not change publish actions in this stage.

Tests:

• go test ./internal/state ./internal/publish
• Add state tests for structured details on identity mismatch and different source id.
• Add shared-root tests for owner absence and output ownership conflict detail, either in internal/state or internal/publish depending on where the conflict is currently detected.
• Preserve existing comparison outcome tests.

Completion criteria:

• Publish planning can make takeover decisions from structured data.
• Existing behavior remains unchanged because no takeover mapping is applied yet.

Stage 3: Single-Owner Takeover Planning And Execution

Goal: implement takeover.mode for single-owner destination state.

Source roadmap reference:

• docs/roadmap/takeover.md: Policy Semantics, Publish Planning, Destination State Results, Safety Rules.

Implementation scope:

• Add TakeoverPolicy or equivalent values to internal/publish.Request.
• Pass destination takeover policy from app-level run planning into publish.
• Add publish action replace_takeover.
• Map eligible single-owner conflicts to replace_takeover:
  • same_pipeline: existing state pipeline id equals current pipeline id, regardless of destination id or source id;
  • same_source: existing state source id equals current source id;
  • any_managed: existing state is valid distributor-managed state;
  • never: no identity/source takeover.
• Keep these cases failing by default unless existing force behavior applies:
  • unmanaged content;
  • invalid state;
  • same-created digest conflict;
  • destination newer for the same source id unless transfer plus --force already permits replacement;
  • conflicts not allowed by takeover.mode.
• Execute replace_takeover through bounded managed replacement mechanics.
• For cross-source takeover, do not retain omitted outputs through reconciliation.mode: merge; treat the affected single-owner state as a managed replacement so old-source outputs are not attributed to the new source.
• Preserve existing replace_older, skip_same, skip_destination_newer, fail_conflict, fail_unmanaged, and force_replace behavior.
• Keep adapters unchanged.

Current-behavior documentation updates:

• Do not update user docs yet unless CLI output changes in this stage. Prefer deferring user docs to Stage 5 so the behavior, output, and docs land together.
• If this stage changes visible dry-run or run output enough that tests require new wording, document only the implemented single-owner behavior and clearly leave shared-root takeover out until Stage 4.

Tests:

• go test ./internal/publish ./internal/app
• Default same_pipeline replaces a different source id from the same pipeline.
• Default same_pipeline replaces state with a different destination id under the same pipeline.
• Default same_pipeline refuses a different pipeline.
• same_source allows same source id and refuses different source id.
• any_managed replaces valid state from a different pipeline.
• never refuses identity/source takeover.
• Invalid state and unmanaged content still fail without force.
• Cross-source takeover with reconciliation.mode: merge does not retain omitted outputs from the previous source.
• Dry-run plans replace_takeover without writing.
• Existing force tests still pass.

Completion criteria:

• Single-owner latest-style destinations can be updated by different bundle ids from the same configured pipeline without --force.
• No unmanaged or invalid-state path becomes a normal takeover path.

Stage 4: Shared-Root Takeover Planning And Execution

Goal: apply the same takeover vocabulary to shared-root owner and output-path conflicts.

Source roadmap reference:

• docs/roadmap/takeover.md: Policy Semantics, Publish Planning, Destination State Results, Safety Rules.

Implementation scope:

• Extend shared-root planning so planned output collisions with existing managed owners are eligible for takeover according to takeover.mode.
• Apply policy as follows:
  • same_pipeline: current owner may take over output paths owned by another destination under the same pipeline;
  • same_source: current owner may take over output paths whose owner records the same source id as the current source;
  • any_managed: current owner may take over output paths owned by any valid shared-root owner;
  • never: preserve current owner conflict behavior.
• Preserve unrelated owner records and non-conflicting output records.
• Keep unmanaged path collisions failing without force.
• Keep compatible single-owner-to-shared-root migration behavior intact.
• For cross-source takeover, do not retain omitted outputs from the previous source under the taking-over owner when reconciliation.mode: merge is set.
• Keep shared-root forced replacement behavior explicit and bounded as it is today.

Current-behavior documentation updates:

• Defer broad documentation updates to Stage 5 unless shared-root behavior must be documented immediately to keep tests or generated docs consistent.

Tests:

• go test ./internal/state ./internal/publish ./internal/app
• same_pipeline permits taking over a path owned by another destination in the same pipeline.
• same_pipeline refuses a path owned by another pipeline.
• same_source permits only matching source-id ownership transfer.
• any_managed permits cross-pipeline managed ownership transfer.
• never preserves current shared-root conflict behavior.
• Reconciliation replace and merge handle omitted outputs according to the feature roadmap, including the cross-source no-retain rule.
• Shared-root migration from compatible single-owner state still works.
• Unmanaged collisions still fail without force.

Completion criteria:

• Single-owner and shared-root destinations use one consistent takeover policy.
• Shared-root takeover changes only affected owners/outputs and preserves unrelated managed state.

Stage 5: Run Output, Summaries, And Documentation

Goal: make takeover behavior visible to operators and document the implemented feature.

Source roadmap reference:

• docs/roadmap/takeover.md: Documentation Impact, Publish Planning, Relationship To Existing Policies.

Implementation scope:

• Add replace_takeover to text run output.
• Add replace_takeover to JSON run action output.
• Add a distinct summary counter for takeover replacements in text and JSON run summaries, using the field name replace_takeover for JSON.
• Include takeover mode and conflict reason in action projection where useful and consistent with existing output style.
• Update fixed-path dry-run warnings when takeover will replace the destination root.
• Update current-behavior docs:
  • docs/config.md
  • docs/operations.md
  • docs/troubleshooting.md
  • docs/integrations/destination-state.md
  • docs/internal/publish.md
  • docs/internal/state.md
  • docs/internal/config.md
  • docs/policy/architecture.md
  • docs/policy/development.md
• Keep docs concise and link to canonical references rather than duplicating full state semantics in every file.
• Do not document any unimplemented future takeover extensions outside docs/roadmap/.

Tests:

• go test ./internal/app ./internal/cli ./internal/config
• Text dry-run output includes replace_takeover.
• JSON output records replace_takeover with stable field names.
• Summary counters are deterministic.
• Fixed-path warnings remain deterministic.
• Config docs examples, if changed or added, load through existing config tests.
• Troubleshooting text matches actual error/action wording.

Completion criteria:

• Operators can distinguish ordinary same-source replacement, takeover replacement, and explicit forced replacement.
• Current docs describe implemented takeover behavior outside roadmap files.

Stage 6: Cross-Backend Regression And Roadmap Closeout

Goal: prove takeover behavior is backend-agnostic and clean up roadmap state after implementation.

Source roadmap reference:

• docs/roadmap/takeover.md: Goals, Non-Goals, Safety Rules.

Implementation scope:

• Add app-level coverage showing takeover planning/execution works with storage abstraction rather than adapter-specific logic.
• Cover local destinations directly.
• Cover SSH and S3 through fake-backed app tests where possible; add adapter integration tests only if existing test infrastructure already supports opt-in remote credentials.
• Add tests for path_mapping.mode: fixed with different source ids under the same pipeline.
• Add tests for archive-style preserve_relative destinations using takeover.mode: same_source where strict source identity is desired.
• Run final consistency searches.
• Once behavior and docs are complete, either remove docs/roadmap/takeover.md or reduce it to future-only material according to the documentation policy. If no future takeover work remains, remove the roadmap file.

Tests:

• go test ./internal/config
• go test ./internal/state
• go test ./internal/publish
• go test ./internal/app ./internal/cli
• go test ./...

Documentation and consistency checks:

rg -n "takeover|replace_takeover|same_pipeline|same_source|any_managed" docs README.md examples
rg -n "destination source id differs from source|pipeline id .* does not match|destination id .* does not match" docs
rg -n "Managed Destination Takeover Roadmap|future takeover|planned takeover" docs README.md examples --glob '!docs/roadmap/**'

Completion criteria:

• Full test suite passes.
• The default same_pipeline behavior applies only to valid distributor-managed takeover cases.
• Unmanaged content and invalid state remain protected.
• Current docs no longer describe implemented takeover behavior as future work.

Refactors To Avoid

• Do not build a generic ownership engine outside internal/state and internal/publish.
• Do not move destination comparison policy into storage adapters.
• Do not change the source manifest schema.
• Do not add takeover fields to HTTP upload requests or producer APIs.
• Do not make --force persistent config.
• Do not merge takeover, transfer, and reconciliation into one broad policy object.
• Do not silently adopt unmanaged content.

Open Questions

No open questions are known. The feature roadmap selects the default mode (same_pipeline), the accepted modes, the safety boundary, and the relationship to existing state, reconciliation, transfer, and force policies.