230 lines
11 KiB
Markdown
230 lines
11 KiB
Markdown
# Managed Destination Takeover Roadmap
|
|
|
|
This roadmap defines planned replacement policy for distributor-managed
|
|
destination content when the existing destination state does not describe the
|
|
same source bundle as the current publication.
|
|
|
|
Current behavior is intentionally conservative. A destination may be replaced
|
|
automatically when its existing state describes the same source id and an older
|
|
source creation time. If the existing state belongs to a different source id,
|
|
pipeline id, destination id, or shared-root owner, `distributor` reports a
|
|
conflict unless an explicit forced replacement workflow is selected where
|
|
supported.
|
|
|
|
That strict default was useful for the initial implementation, but it is too
|
|
rigid for producers that intentionally publish newer generated content to the
|
|
same destination path from different producer tasks or bundle ids. `distributor`
|
|
is primarily a distribution tool: by default, a valid producer request should
|
|
publish the requested bundle unless doing so would claim unmanaged content or
|
|
cross an ownership boundary that is likely to surprise the operator.
|
|
|
|
## Goals
|
|
|
|
- Add a destination-level policy that defines when normal, non-force publication
|
|
may replace existing distributor-managed content whose source or owner
|
|
identity differs from the current source.
|
|
- Default the policy to `same_pipeline`, so a pipeline may distribute the latest
|
|
valid bundle to its configured destination paths without requiring every
|
|
producer task to reuse one source manifest id.
|
|
- Preserve strong safety boundaries for unmanaged content, invalid destination
|
|
state, and cross-pipeline ownership by default.
|
|
- Keep reconciliation policy focused on how managed output sets are updated
|
|
after replacement is allowed.
|
|
- Keep transfer policy focused on same-source comparison outcomes and explicit
|
|
forced replacement behavior.
|
|
- Preserve clear dry-run and JSON output so takeover behavior is visible before
|
|
files are changed.
|
|
|
|
## Non-Goals
|
|
|
|
- Do not allow automatic adoption of unmanaged destination files.
|
|
- Do not allow invalid `.distributor.json` state to be overwritten without the
|
|
existing explicit force workflow.
|
|
- Do not make producers select destination ownership or takeover policy through
|
|
source manifests or HTTP upload requests.
|
|
- Do not replace `state.mode`, `reconciliation.mode`, `path_mapping.mode`, or
|
|
`transfer` policy.
|
|
- Do not introduce broad synchronization behavior outside configured
|
|
destination bundle paths.
|
|
|
|
## Configuration
|
|
|
|
Add destination-level takeover policy:
|
|
|
|
```yaml
|
|
destinations:
|
|
- id: latest
|
|
backend: local
|
|
path: /srv/reports/weather/latest/tomorrow
|
|
path_mapping:
|
|
mode: fixed
|
|
takeover:
|
|
mode: same_pipeline
|
|
```
|
|
|
|
Accepted `takeover.mode` values:
|
|
|
|
- `same_pipeline`: default. Normal publication may replace valid
|
|
distributor-managed state or managed output paths owned by the same pipeline,
|
|
even when the previous source id or destination id differs.
|
|
- `same_source`: normal publication may replace only when the destination state
|
|
has the same source manifest id as the current source. This is closest to the
|
|
current strict managed replacement behavior.
|
|
- `any_managed`: normal publication may replace any valid distributor-managed
|
|
state or managed output path in the selected destination bundle path,
|
|
regardless of pipeline id, destination id, or source id.
|
|
- `never`: normal publication never performs takeover replacement for identity
|
|
or source conflicts. Same-source older/newer behavior remains governed by
|
|
`transfer`, and explicit forced replacement remains governed by `--force` and
|
|
`transfer.on_conflict`.
|
|
|
|
The field is destination-local because different destinations from the same
|
|
pipeline can have different ownership expectations. Archive destinations often
|
|
want stricter behavior, while latest-style fixed destinations often want
|
|
`same_pipeline` or, after explicit operator review, `any_managed`.
|
|
|
|
## Policy Semantics
|
|
|
|
Takeover policy applies only after destination state has been parsed and
|
|
validated as distributor-managed state.
|
|
|
|
For single-owner state:
|
|
|
|
- `same_pipeline` permits replacement when the existing state `pipeline_id`
|
|
matches the current pipeline id. The previous `destination_id` and source id
|
|
may differ.
|
|
- `same_source` permits replacement only when the existing source manifest id
|
|
matches the current source manifest id.
|
|
- `any_managed` permits replacement of any valid single-owner destination state.
|
|
- `never` does not permit takeover replacement.
|
|
|
|
For shared-root state:
|
|
|
|
- Takeover is evaluated per managed output path that collides with the current
|
|
plan and per current owner record when one already exists.
|
|
- `same_pipeline` permits the current destination owner to take over output
|
|
paths owned by another destination under the same pipeline.
|
|
- `same_source` permits takeover only when the colliding output owner records
|
|
the same source manifest id as the current source.
|
|
- `any_managed` permits takeover of colliding managed output paths owned by any
|
|
owner in the shared root.
|
|
- `never` preserves the current owner-scoped conflict behavior.
|
|
|
|
Takeover never applies to:
|
|
|
|
- destination content with no valid `.distributor.json`;
|
|
- invalid destination state;
|
|
- planned paths that exist in storage but are not recorded as managed;
|
|
- source digest mismatches or invalid source manifests;
|
|
- destination newer/same-created digest comparisons for the same source id,
|
|
except where existing transfer policy already permits replacement.
|
|
|
|
## Relationship To Existing Policies
|
|
|
|
`takeover.mode` answers this question:
|
|
|
|
> May this destination normally replace valid managed content whose owner or
|
|
> source identity differs from the current publication?
|
|
|
|
`reconciliation.mode` continues to answer:
|
|
|
|
> Once replacement is allowed, should omitted managed outputs be removed
|
|
> (`replace`) or retained (`merge`)?
|
|
|
|
`transfer` continues to answer:
|
|
|
|
> What should happen for same-source comparisons, destination-newer state, and
|
|
> explicit forced conflict replacement?
|
|
|
|
`--force` remains an operator workflow for exceptional cases. Takeover policy is
|
|
not a replacement for forced replacement of unmanaged content or invalid state.
|
|
|
|
## Publish Planning
|
|
|
|
Add a new planned action for successful takeover replacement, tentatively named
|
|
`replace_takeover`.
|
|
|
|
`replace_takeover` should execute through the same bounded managed replacement
|
|
machinery as other managed replacements, with one additional source-identity
|
|
safety rule: when takeover crosses source id or owner identity, omitted outputs
|
|
from the previous source must not be retained merely because the destination is
|
|
configured with `reconciliation.mode: merge`. Retaining omitted outputs is valid
|
|
only when those outputs still belong to the same source identity being
|
|
published. This keeps destination state from claiming that outputs derived from
|
|
an old source belong to the new source manifest.
|
|
|
|
- single-owner `replace` deletes only managed outputs recorded in existing
|
|
state plus `.distributor.json`, then writes the new outputs and state;
|
|
- single-owner `merge` retains omitted managed outputs only for same-source
|
|
updates; cross-source takeover behaves as a managed replacement for the
|
|
affected owner;
|
|
- shared-root `replace` deletes only output records taken over by the current
|
|
owner or omitted from the current owner according to the shared-root plan;
|
|
- shared-root `merge` preserves unrelated non-conflicting managed outputs and
|
|
updates ownership for paths explicitly taken over, but does not retain omitted
|
|
outputs from a different source under the taking-over owner.
|
|
|
|
Text output, JSON output, dry-run output, and summary counters should identify
|
|
takeover replacements separately from `replace_older` and `force_replace`.
|
|
|
|
## Destination State Results
|
|
|
|
After a successful takeover, destination state must describe the current
|
|
pipeline, destination, source manifest, outputs, links, reconciliation policy,
|
|
and state mode.
|
|
|
|
For single-owner state, the destination bundle path has one current owner after
|
|
publication: the publishing pipeline and destination.
|
|
|
|
For shared-root state, only the affected owner records and output records are
|
|
changed. Unrelated owners and non-conflicting outputs remain recorded.
|
|
|
|
No source manifest schema change is required. No destination state schema change
|
|
is expected unless implementation discovers that takeover metadata must be
|
|
persisted for recovery or audit. The preferred initial design is to make
|
|
takeover a planning decision, not a new persisted state concept.
|
|
|
|
## Documentation Impact
|
|
|
|
When this feature is implemented, current-behavior docs should explain the new
|
|
policy where operators and maintainers already look for publication safety,
|
|
state comparison, and configuration behavior:
|
|
|
|
- `docs/config.md`: document `takeover.mode`, default `same_pipeline`, accepted
|
|
values, examples for archive and latest destinations, and interaction with
|
|
`transfer`, `reconciliation`, and `--force`.
|
|
- `docs/operations.md`: explain takeover dry-runs, normal replacement safety,
|
|
shared-root behavior, and recovery guidance.
|
|
- `docs/troubleshooting.md`: update conflict guidance for
|
|
`destination source id differs from source`, pipeline/destination mismatch,
|
|
and shared-root ownership conflicts.
|
|
- `docs/integrations/destination-state.md`: describe how takeover affects
|
|
comparison and state rewriting without changing the source manifest contract.
|
|
- `docs/internal/publish.md`, `docs/internal/state.md`, and
|
|
`docs/internal/config.md`: document package responsibilities and invariants.
|
|
- `docs/policy/architecture.md`: clarify that the default posture is to
|
|
distribute valid producer bundles through configured destinations while
|
|
protecting unmanaged content and cross-pipeline ownership by default.
|
|
- `docs/policy/development.md`: add `takeover.mode` to the list of config
|
|
changes that must be validated and documented when publish safety changes.
|
|
|
|
## Safety Rules
|
|
|
|
- Takeover must be dry-runnable.
|
|
- Takeover must operate only inside the resolved destination bundle path.
|
|
- Takeover must never delete parent paths, sibling paths, or unmanaged files.
|
|
- Takeover must never claim storage content that is not recorded in valid
|
|
destination state.
|
|
- `any_managed` should be documented as an intentional cross-pipeline ownership
|
|
policy for tightly controlled destinations, not as the recommended default for
|
|
archives.
|
|
- Archive-style destinations should generally use `same_source` if each bundle
|
|
id represents a distinct immutable artifact.
|
|
|
|
## Implementation Reference
|
|
|
|
The staged implementation plan for this feature lives in
|
|
`docs/roadmap/implementation.md`. This document remains the feature contract and
|
|
policy reference; implementation sequencing, package-by-package work, and test
|
|
commands belong in the implementation roadmap.
|