376 lines
15 KiB
Markdown
376 lines
15 KiB
Markdown
# Managed Destination Takeover Implementation Roadmap
|
|
|
|
This is the active staged implementation plan for
|
|
`docs/roadmap/takeover.md`. The feature roadmap defines the target policy and
|
|
end-state semantics; this document defines the implementation sequence for an
|
|
LLM coding agent to follow stage by stage.
|
|
|
|
Future behavior must remain under `docs/roadmap/` until each stage is
|
|
implemented. Preparatory internal stages should not update user-facing current
|
|
docs. Current-behavior docs should be updated when takeover behavior is wired
|
|
for operator-facing use.
|
|
|
|
## Current Baseline
|
|
|
|
`distributor` already supports local, SSH/SFTP, S3, and HTTP upload source
|
|
workflows, destination state schemas for single-owner and shared-root state,
|
|
path mapping, link generation, reconciliation, transfer policy, explicit
|
|
`--force`, and text/JSON run output.
|
|
|
|
Current destination comparison is strict:
|
|
|
|
- same source manifest skips;
|
|
- same source id with older destination state normally replaces;
|
|
- same source id with newer destination state normally skips;
|
|
- same source id and same creation time with different digest conflicts;
|
|
- different source id, pipeline id, destination id, or shared-root output owner
|
|
conflicts unless explicit force policy applies;
|
|
- unmanaged content and invalid state do not become normal managed replacement
|
|
cases.
|
|
|
|
The target feature adds destination-level `takeover.mode`, defaulting to
|
|
`same_pipeline`, so valid managed content can be replaced without `--force` when
|
|
the configured policy says that ownership/source takeover is expected.
|
|
|
|
## Implementation Principles
|
|
|
|
- Preserve public behavior until the stage that explicitly changes it.
|
|
- Keep source manifests unchanged; takeover is destination configuration and
|
|
publish planning policy.
|
|
- Keep state comparison pure. State comparison may expose structured conflict
|
|
details, but publish planning decides whether takeover is allowed.
|
|
- Keep adapters thin. No local, SSH, or S3 adapter should know takeover policy.
|
|
- Keep unmanaged content and invalid destination state outside normal takeover.
|
|
- Keep `reconciliation.mode`, `transfer`, `state.mode`, `path_mapping.mode`, and
|
|
`--force` as separate concepts.
|
|
- Prefer narrow behavior-preserving refactors over broad publication rewrites.
|
|
- Update implemented-behavior docs in the same stage as the behavior change.
|
|
|
|
## Active Implementation Stages
|
|
|
|
## Stage 1: Config Model And Validation
|
|
|
|
Goal: add the destination config field, defaults, and validation without
|
|
changing publish behavior.
|
|
|
|
Source roadmap reference:
|
|
|
|
- `docs/roadmap/takeover.md`: Configuration, Policy Semantics, Safety Rules.
|
|
|
|
Implementation scope:
|
|
|
|
- Add destination-level `TakeoverPolicy` to `internal/config`.
|
|
- Add constants for:
|
|
- `same_pipeline`
|
|
- `same_source`
|
|
- `any_managed`
|
|
- `never`
|
|
- Default `takeover.mode` to `same_pipeline` in config defaults.
|
|
- Validate accepted values with clear field context such as
|
|
`pipelines[0].destinations[0].takeover.mode`.
|
|
- Preserve strict YAML unknown-field behavior.
|
|
- Thread the defaulted policy into existing destination config views or helper
|
|
structures if those are used by app/publish request construction.
|
|
- Do not change publish planning, execution, CLI output, or docs outside
|
|
roadmap files in this stage.
|
|
|
|
Tests:
|
|
|
|
- `go test ./internal/config`
|
|
- Add config tests for omitted `takeover`, each accepted mode, invalid mode, and
|
|
unknown nested fields.
|
|
- Add or update example-loading tests only if examples are touched, which should
|
|
not be necessary in this stage.
|
|
|
|
Completion criteria:
|
|
|
|
- Every destination has a defaulted `takeover.mode`.
|
|
- Invalid values fail during config validation.
|
|
- No run behavior changes because publish planning does not consume the policy
|
|
yet.
|
|
|
|
## Stage 2: Structured State Comparison Details
|
|
|
|
Goal: expose enough structured comparison detail for publish planning to decide
|
|
takeover eligibility without parsing human-readable reason strings.
|
|
|
|
Source roadmap reference:
|
|
|
|
- `docs/roadmap/takeover.md`: Policy Semantics, Relationship To Existing
|
|
Policies.
|
|
|
|
Implementation scope:
|
|
|
|
- Extend `internal/state.Comparison` or add a package-local structured detail
|
|
type so callers can distinguish:
|
|
- pipeline id mismatch;
|
|
- destination id mismatch;
|
|
- different source id;
|
|
- same-created digest conflict;
|
|
- destination newer;
|
|
- invalid state;
|
|
- unmanaged content;
|
|
- absent shared-root owner;
|
|
- shared-root managed output owner conflicts, if those are currently reported
|
|
outside `internal/state`.
|
|
- Keep existing outcome names and reason strings stable where practical.
|
|
- Keep comparison functions pure. They should report facts about existing state,
|
|
not consult `takeover.mode`, `transfer`, `reconciliation`, or `force`.
|
|
- Do not add new persisted state fields.
|
|
- Do not change publish actions in this stage.
|
|
|
|
Tests:
|
|
|
|
- `go test ./internal/state ./internal/publish`
|
|
- Add state tests for structured details on identity mismatch and different
|
|
source id.
|
|
- Add shared-root tests for owner absence and output ownership conflict detail,
|
|
either in `internal/state` or `internal/publish` depending on where the
|
|
conflict is currently detected.
|
|
- Preserve existing comparison outcome tests.
|
|
|
|
Completion criteria:
|
|
|
|
- Publish planning can make takeover decisions from structured data.
|
|
- Existing behavior remains unchanged because no takeover mapping is applied yet.
|
|
|
|
## Stage 3: Single-Owner Takeover Planning And Execution
|
|
|
|
Goal: implement `takeover.mode` for single-owner destination state.
|
|
|
|
Source roadmap reference:
|
|
|
|
- `docs/roadmap/takeover.md`: Policy Semantics, Publish Planning, Destination
|
|
State Results, Safety Rules.
|
|
|
|
Implementation scope:
|
|
|
|
- Add `TakeoverPolicy` or equivalent values to `internal/publish.Request`.
|
|
- Pass destination takeover policy from app-level run planning into publish.
|
|
- Add publish action `replace_takeover`.
|
|
- Map eligible single-owner conflicts to `replace_takeover`:
|
|
- `same_pipeline`: existing state pipeline id equals current pipeline id,
|
|
regardless of destination id or source id;
|
|
- `same_source`: existing state source id equals current source id;
|
|
- `any_managed`: existing state is valid distributor-managed state;
|
|
- `never`: no identity/source takeover.
|
|
- Keep these cases failing by default unless existing force behavior applies:
|
|
- unmanaged content;
|
|
- invalid state;
|
|
- same-created digest conflict;
|
|
- destination newer for the same source id unless `transfer` plus `--force`
|
|
already permits replacement;
|
|
- conflicts not allowed by `takeover.mode`.
|
|
- Execute `replace_takeover` through bounded managed replacement mechanics.
|
|
- For cross-source takeover, do not retain omitted outputs through
|
|
`reconciliation.mode: merge`; treat the affected single-owner state as a
|
|
managed replacement so old-source outputs are not attributed to the new source.
|
|
- Preserve existing `replace_older`, `skip_same`, `skip_destination_newer`,
|
|
`fail_conflict`, `fail_unmanaged`, and `force_replace` behavior.
|
|
- Keep adapters unchanged.
|
|
|
|
Current-behavior documentation updates:
|
|
|
|
- Do not update user docs yet unless CLI output changes in this stage. Prefer
|
|
deferring user docs to Stage 5 so the behavior, output, and docs land
|
|
together.
|
|
- If this stage changes visible dry-run or run output enough that tests require
|
|
new wording, document only the implemented single-owner behavior and clearly
|
|
leave shared-root takeover out until Stage 4.
|
|
|
|
Tests:
|
|
|
|
- `go test ./internal/publish ./internal/app`
|
|
- Default `same_pipeline` replaces a different source id from the same pipeline.
|
|
- Default `same_pipeline` replaces state with a different destination id under
|
|
the same pipeline.
|
|
- Default `same_pipeline` refuses a different pipeline.
|
|
- `same_source` allows same source id and refuses different source id.
|
|
- `any_managed` replaces valid state from a different pipeline.
|
|
- `never` refuses identity/source takeover.
|
|
- Invalid state and unmanaged content still fail without force.
|
|
- Cross-source takeover with `reconciliation.mode: merge` does not retain
|
|
omitted outputs from the previous source.
|
|
- Dry-run plans `replace_takeover` without writing.
|
|
- Existing force tests still pass.
|
|
|
|
Completion criteria:
|
|
|
|
- Single-owner latest-style destinations can be updated by different bundle ids
|
|
from the same configured pipeline without `--force`.
|
|
- No unmanaged or invalid-state path becomes a normal takeover path.
|
|
|
|
## Stage 4: Shared-Root Takeover Planning And Execution
|
|
|
|
Goal: apply the same takeover vocabulary to shared-root owner and output-path
|
|
conflicts.
|
|
|
|
Source roadmap reference:
|
|
|
|
- `docs/roadmap/takeover.md`: Policy Semantics, Publish Planning, Destination
|
|
State Results, Safety Rules.
|
|
|
|
Implementation scope:
|
|
|
|
- Extend shared-root planning so planned output collisions with existing managed
|
|
owners are eligible for takeover according to `takeover.mode`.
|
|
- Apply policy as follows:
|
|
- `same_pipeline`: current owner may take over output paths owned by another
|
|
destination under the same pipeline;
|
|
- `same_source`: current owner may take over output paths whose owner records
|
|
the same source id as the current source;
|
|
- `any_managed`: current owner may take over output paths owned by any valid
|
|
shared-root owner;
|
|
- `never`: preserve current owner conflict behavior.
|
|
- Preserve unrelated owner records and non-conflicting output records.
|
|
- Keep unmanaged path collisions failing without force.
|
|
- Keep compatible single-owner-to-shared-root migration behavior intact.
|
|
- For cross-source takeover, do not retain omitted outputs from the previous
|
|
source under the taking-over owner when `reconciliation.mode: merge` is set.
|
|
- Keep shared-root forced replacement behavior explicit and bounded as it is
|
|
today.
|
|
|
|
Current-behavior documentation updates:
|
|
|
|
- Defer broad documentation updates to Stage 5 unless shared-root behavior must
|
|
be documented immediately to keep tests or generated docs consistent.
|
|
|
|
Tests:
|
|
|
|
- `go test ./internal/state ./internal/publish ./internal/app`
|
|
- `same_pipeline` permits taking over a path owned by another destination in the
|
|
same pipeline.
|
|
- `same_pipeline` refuses a path owned by another pipeline.
|
|
- `same_source` permits only matching source-id ownership transfer.
|
|
- `any_managed` permits cross-pipeline managed ownership transfer.
|
|
- `never` preserves current shared-root conflict behavior.
|
|
- Reconciliation `replace` and `merge` handle omitted outputs according to the
|
|
feature roadmap, including the cross-source no-retain rule.
|
|
- Shared-root migration from compatible single-owner state still works.
|
|
- Unmanaged collisions still fail without force.
|
|
|
|
Completion criteria:
|
|
|
|
- Single-owner and shared-root destinations use one consistent takeover policy.
|
|
- Shared-root takeover changes only affected owners/outputs and preserves
|
|
unrelated managed state.
|
|
|
|
## Stage 5: Run Output, Summaries, And Documentation
|
|
|
|
Goal: make takeover behavior visible to operators and document the implemented
|
|
feature.
|
|
|
|
Source roadmap reference:
|
|
|
|
- `docs/roadmap/takeover.md`: Documentation Impact, Publish Planning,
|
|
Relationship To Existing Policies.
|
|
|
|
Implementation scope:
|
|
|
|
- Add `replace_takeover` to text run output.
|
|
- Add `replace_takeover` to JSON run action output.
|
|
- Add a distinct summary counter for takeover replacements in text and JSON run
|
|
summaries, using the field name `replace_takeover` for JSON.
|
|
- Include takeover mode and conflict reason in action projection where useful
|
|
and consistent with existing output style.
|
|
- Update fixed-path dry-run warnings when takeover will replace the destination
|
|
root.
|
|
- Update current-behavior docs:
|
|
- `docs/config.md`
|
|
- `docs/operations.md`
|
|
- `docs/troubleshooting.md`
|
|
- `docs/integrations/destination-state.md`
|
|
- `docs/internal/publish.md`
|
|
- `docs/internal/state.md`
|
|
- `docs/internal/config.md`
|
|
- `docs/policy/architecture.md`
|
|
- `docs/policy/development.md`
|
|
- Keep docs concise and link to canonical references rather than duplicating
|
|
full state semantics in every file.
|
|
- Do not document any unimplemented future takeover extensions outside
|
|
`docs/roadmap/`.
|
|
|
|
Tests:
|
|
|
|
- `go test ./internal/app ./internal/cli ./internal/config`
|
|
- Text dry-run output includes `replace_takeover`.
|
|
- JSON output records `replace_takeover` with stable field names.
|
|
- Summary counters are deterministic.
|
|
- Fixed-path warnings remain deterministic.
|
|
- Config docs examples, if changed or added, load through existing config tests.
|
|
- Troubleshooting text matches actual error/action wording.
|
|
|
|
Completion criteria:
|
|
|
|
- Operators can distinguish ordinary same-source replacement, takeover
|
|
replacement, and explicit forced replacement.
|
|
- Current docs describe implemented takeover behavior outside roadmap files.
|
|
|
|
## Stage 6: Cross-Backend Regression And Roadmap Closeout
|
|
|
|
Goal: prove takeover behavior is backend-agnostic and clean up roadmap state
|
|
after implementation.
|
|
|
|
Source roadmap reference:
|
|
|
|
- `docs/roadmap/takeover.md`: Goals, Non-Goals, Safety Rules.
|
|
|
|
Implementation scope:
|
|
|
|
- Add app-level coverage showing takeover planning/execution works with storage
|
|
abstraction rather than adapter-specific logic.
|
|
- Cover local destinations directly.
|
|
- Cover SSH and S3 through fake-backed app tests where possible; add adapter
|
|
integration tests only if existing test infrastructure already supports
|
|
opt-in remote credentials.
|
|
- Add tests for `path_mapping.mode: fixed` with different source ids under the
|
|
same pipeline.
|
|
- Add tests for archive-style `preserve_relative` destinations using
|
|
`takeover.mode: same_source` where strict source identity is desired.
|
|
- Run final consistency searches.
|
|
- Once behavior and docs are complete, either remove `docs/roadmap/takeover.md`
|
|
or reduce it to future-only material according to the documentation policy.
|
|
If no future takeover work remains, remove the roadmap file.
|
|
|
|
Tests:
|
|
|
|
- `go test ./internal/config`
|
|
- `go test ./internal/state`
|
|
- `go test ./internal/publish`
|
|
- `go test ./internal/app ./internal/cli`
|
|
- `go test ./...`
|
|
|
|
Documentation and consistency checks:
|
|
|
|
```sh
|
|
rg -n "takeover|replace_takeover|same_pipeline|same_source|any_managed" docs README.md examples
|
|
rg -n "destination source id differs from source|pipeline id .* does not match|destination id .* does not match" docs
|
|
rg -n "Managed Destination Takeover Roadmap|future takeover|planned takeover" docs README.md examples --glob '!docs/roadmap/**'
|
|
```
|
|
|
|
Completion criteria:
|
|
|
|
- Full test suite passes.
|
|
- The default `same_pipeline` behavior applies only to valid
|
|
distributor-managed takeover cases.
|
|
- Unmanaged content and invalid state remain protected.
|
|
- Current docs no longer describe implemented takeover behavior as future work.
|
|
|
|
## Refactors To Avoid
|
|
|
|
- Do not build a generic ownership engine outside `internal/state` and
|
|
`internal/publish`.
|
|
- Do not move destination comparison policy into storage adapters.
|
|
- Do not change the source manifest schema.
|
|
- Do not add takeover fields to HTTP upload requests or producer APIs.
|
|
- Do not make `--force` persistent config.
|
|
- Do not merge `takeover`, `transfer`, and `reconciliation` into one broad
|
|
policy object.
|
|
- Do not silently adopt unmanaged content.
|
|
|
|
## Open Questions
|
|
|
|
No open questions are known. The feature roadmap selects the default mode
|
|
(`same_pipeline`), the accepted modes, the safety boundary, and the relationship
|
|
to existing state, reconciliation, transfer, and force policies.
|