Expose takeover actions in run output
This commit is contained in:
@@ -292,6 +292,7 @@ Text output is optimized for direct operator use. JSON output is optimized for a
|
||||
- Usage errors and fatal setup errors exit non-zero and do not emit a JSON result document.
|
||||
- `run --format json` emits a JSON result for partial destination failures, sets `ok` to `false`, includes result details and errors, and exits non-zero.
|
||||
- Warnings are included in JSON output and are printed in text output when relevant.
|
||||
- `run` summaries include separate `replace_older`, `replace_takeover`, and `force_replace` counters. Takeover destination actions include `takeover_mode` in JSON and text output.
|
||||
|
||||
## Diagnostics And Recovery
|
||||
|
||||
|
||||
@@ -96,9 +96,11 @@ Destination reconciliation applies when destination state is older than the sour
|
||||
- `replace`: for single-owner state, delete managed output paths recorded in `outputs` plus `.distributor.json`, require the destination bundle path to be empty afterward, write the newly planned outputs, and write state whose `outputs` are exactly that new planned set. For shared-root state, delete only the current owner's omitted outputs and preserve unrelated owners.
|
||||
- `merge`: retain managed output paths omitted from the new plan, overwrite planned paths only when they are already recorded in existing state, fail when a newly planned path exists in storage but is not recorded as managed, and write state whose `outputs` are the cumulative managed set.
|
||||
|
||||
Takeover replacement applies when valid managed state has an identity, source, or shared-root output-owner mismatch and destination `takeover.mode` permits the current publication to take ownership. It uses bounded managed replacement behavior. For shared-root state, only taken-over output records and the current owner records are rewritten; unrelated owner records and non-conflicting outputs remain managed by their existing owners.
|
||||
|
||||
Top-level `links.primary_url` is selected from the newly planned outputs for the current publication. Retained outputs keep their existing per-output URL metadata.
|
||||
|
||||
If a merge publication fails after writing outputs, cleanup removes only newly created outputs from that failed attempt. Previously managed overwritten paths remain managed and are not removed by failed-attempt cleanup.
|
||||
If a same-source merge publication fails after writing outputs, cleanup removes only newly created outputs from that failed attempt. Previously managed overwritten paths remain managed and are not removed by failed-attempt cleanup. Takeover replacement does not retain omitted outputs through merge reconciliation.
|
||||
|
||||
## Comparison Semantics
|
||||
|
||||
@@ -106,15 +108,16 @@ If a merge publication fails after writing outputs, cleanup removes only newly c
|
||||
|
||||
- No state and no content: publish new outputs.
|
||||
- No state and existing content: treat the destination as unmanaged.
|
||||
- Shared-root state without the current owner: publish new outputs for that owner if planned paths do not collide with other owners or unmanaged content.
|
||||
- Shared-root state without the current owner: publish new outputs for that owner if planned paths do not collide with unmanaged content or with other owners that `takeover.mode` does not permit.
|
||||
- Matching embedded source manifest: skip.
|
||||
- Same source id with older `created`: replace if policy allows.
|
||||
- Same source id with newer `created`: skip by default.
|
||||
- Same source id and same `created` with different digest: conflict.
|
||||
- Different source id, pipeline id, or destination id: conflict.
|
||||
- Different source id, pipeline id, or destination id in single-owner state: conflict unless `takeover.mode` permits managed ownership transfer.
|
||||
- Shared-root output path owned by a different owner: conflict unless `takeover.mode` permits managed ownership transfer.
|
||||
- Invalid state JSON or invalid state fields: conflict.
|
||||
|
||||
Normal single-owner replacement deletes only managed output paths recorded in `outputs` plus `.distributor.json`. Shared-root replacement deletes only omitted outputs for the current owner. Merge publication retains omitted managed outputs. Forced replacement deletes the bounded destination bundle path.
|
||||
Normal single-owner replacement deletes only managed output paths recorded in `outputs` plus `.distributor.json`. Shared-root replacement deletes only omitted outputs for the current owner. Merge publication retains omitted managed outputs for same-source replacement. Forced replacement deletes the bounded destination bundle path.
|
||||
|
||||
## State Repair Semantics
|
||||
|
||||
@@ -238,7 +241,7 @@ For shared-root state:
|
||||
|
||||
If `state.mode: shared_root` is configured and existing state is a compatible single-owner `.distributor.json` for the same pipeline id and destination id, the next successful publish writes schema version `3` shared-root state for that owner.
|
||||
|
||||
If existing single-owner state belongs to a different pipeline or destination, publish fails as a conflict. `distributor` does not implicitly take over unrelated state or unmanaged files.
|
||||
If `state.mode: shared_root` is configured and existing single-owner state belongs to a different pipeline or destination, publish fails as a conflict. `distributor` does not implicitly convert unrelated single-owner state or take over unmanaged files during shared-root migration.
|
||||
|
||||
## Boundaries
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ The adapter uses these S3 operations:
|
||||
|
||||
Writes buffer the input and set `ContentLength`. If no content type is supplied by the caller, the adapter infers a content type from the logical path.
|
||||
|
||||
Normal replacement and failed-write cleanup delete only managed output objects plus `.distributor.json`. Merge publication retains omitted managed objects and may overwrite existing managed objects. Forced replacement deletes objects under the bounded destination bundle prefix. The backend does not manage bucket versioning, lifecycle rules, object lock, or delete markers.
|
||||
Normal replacement and failed-write cleanup delete only managed output objects plus `.distributor.json`. Same-source merge publication retains omitted managed objects and may overwrite existing managed objects. Takeover replacement does not retain omitted outputs through merge reconciliation. Forced replacement deletes objects under the bounded destination bundle prefix. The backend does not manage bucket versioning, lifecycle rules, object lock, or delete markers.
|
||||
|
||||
## Error Mapping
|
||||
|
||||
|
||||
@@ -54,7 +54,7 @@ The configured `path` is the backend root. All source discovery, destination pat
|
||||
|
||||
The adapter rejects symlink ancestors for reads and writes. Reads require regular files. Writes create parent directories and prefer atomic temp-file-plus-rename writes when requested. Walk output is sorted through the shared storage walker.
|
||||
|
||||
Managed cleanup and normal replacement delete only managed output paths plus `.distributor.json`. Merge publication retains omitted managed paths and may overwrite existing managed paths. Forced replacement deletes the bounded destination bundle path.
|
||||
Managed cleanup and normal replacement delete only managed output paths plus `.distributor.json`. Same-source merge publication retains omitted managed paths and may overwrite existing managed paths. Takeover replacement does not retain omitted outputs through merge reconciliation. Forced replacement deletes the bounded destination bundle path.
|
||||
|
||||
## Boundaries
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ User-facing command parsing stays in `internal/cli`, including `reconcile-state`
|
||||
|
||||
## Config Fields Used
|
||||
|
||||
The package consumes the loaded `config.Config`: `server.http`, `secrets.directory`, pipeline ids, source and destination backend fields, validation policy, publish policy, transform policy, path mapping, links, state policy, reconciliation policy, retention policy, and transfer policy.
|
||||
The package consumes the loaded `config.Config`: `server.http`, `secrets.directory`, pipeline ids, source and destination backend fields, validation policy, publish policy, transform policy, path mapping, links, state policy, reconciliation policy, takeover policy, retention policy, and transfer policy.
|
||||
|
||||
Config fields are validated and defaulted by `internal/config` before app workflows use them.
|
||||
|
||||
@@ -32,7 +32,7 @@ The app layer registers default transforms, including Markdown-to-HTML, and supp
|
||||
|
||||
## State And Manifest Behavior
|
||||
|
||||
Run workflows discover and validate source bundles through `internal/bundle`. Destination state actions are prepared and written through `internal/publish` and `internal/state`; the app layer records report projections of those actions and results.
|
||||
Run workflows discover and validate source bundles through `internal/bundle`. Destination state actions are prepared and written through `internal/publish` and `internal/state`; the app layer records report projections of those actions and results. Run summaries count takeover replacements separately from ordinary older-state replacement and explicit forced replacement.
|
||||
|
||||
Reconcile-state workflows load one configured pipeline/destination selector, open that destination root, parse the root `.distributor.json`, and report missing managed output records plus unmanaged storage entries. Managed output existence checks use storage `Stat`; unmanaged reporting uses bounded storage `Walk` and excludes `.distributor.json` plus all paths already recorded as managed. Apply mode removes missing managed output records from state and rewrites valid state only; dry-run reports the same repair without writing. Text output reports `changed`, `would_change`, or `unchanged`; JSON output uses the shared app envelope. It does not validate output digests, delete destination files, adopt unmanaged files, or rewrite invalid or mismatched state.
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ The canonical user-facing config reference is `docs/config.md`.
|
||||
|
||||
## Config Fields Used
|
||||
|
||||
The package defines all user-visible config fields: `server.http`, `secrets`, `pipelines`, source and destination backend fields, validation policy, publish policy, transform policy, path mapping, links, state policy, reconciliation policy, retention policy, and transfer policy.
|
||||
The package defines all user-visible config fields: `server.http`, `secrets`, `pipelines`, source and destination backend fields, validation policy, publish policy, transform policy, path mapping, links, state policy, reconciliation policy, takeover policy, retention policy, and transfer policy.
|
||||
|
||||
## Adapters Used
|
||||
|
||||
@@ -26,7 +26,7 @@ No external storage adapters are used directly. The package exposes normalized c
|
||||
|
||||
## State And Manifest Behavior
|
||||
|
||||
The package does not parse source manifests or destination state. It validates config values that later affect manifest validation and destination state, such as publish/transform combinations, links, state policy, reconciliation policy, retention policy, transfer policy, backend roots, S3 prefix shape, and HTTP upload source settings.
|
||||
The package does not parse source manifests or destination state. It validates config values that later affect manifest validation and destination state, such as publish/transform combinations, links, state policy, reconciliation policy, takeover policy, retention policy, transfer policy, backend roots, S3 prefix shape, and HTTP upload source settings.
|
||||
|
||||
## Skip And Resume Behavior
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ Audience: developers and LLM coding agents changing `internal/publish`.
|
||||
|
||||
Inputs are a source bundle, source backend, destination backend, pipeline id, destination id, destination bundle path, path mapping mode, publish policy, transform policy, optional link policy, state policy, reconciliation policy, takeover policy, transformer resolver, transfer policy, distributor version, and force flag.
|
||||
|
||||
Output from planning is a `Plan` with action, reason, destination identity, selected outputs, state mode, owner scope, reconciliation mode, optional existing single-owner or shared-root state, optional primary URL, and force metadata. Shared-root plans also expose other-owner outputs to preserve, current-owner outputs retained by merge, current-owner outputs deleted by replace, and current-owner outputs to write. Execution writes selected source outputs, generated outputs, and `.distributor.json` for executable publish or replacement actions.
|
||||
Output from planning is a `Plan` with action, reason, destination identity, selected outputs, state mode, owner scope, reconciliation mode, takeover mode, optional existing single-owner or shared-root state, optional primary URL, and force metadata. Shared-root plans also expose other-owner outputs to preserve, taken-over outputs, current-owner outputs retained by merge, current-owner outputs deleted by replace or takeover, and current-owner outputs to write. Execution writes selected source outputs, generated outputs, and `.distributor.json` for executable publish or replacement actions.
|
||||
|
||||
## Boundaries
|
||||
|
||||
@@ -65,7 +65,7 @@ Execution fails on delete, read, transform output, unmanaged merge path collisio
|
||||
- Replacement reconciliation deletes only managed paths recorded in existing state plus `.distributor.json` for single-owner state, and only current-owner omitted outputs for shared-root state.
|
||||
- Merge reconciliation never adopts unmanaged content.
|
||||
- Merge state output records are cumulative for the single owner.
|
||||
- Shared-root planning is owner-scoped and preserves unrelated owner outputs.
|
||||
- Shared-root planning is owner-scoped, preserves unrelated owner outputs, and records taken-over managed output paths separately from unrelated owners.
|
||||
- Shared-root execution writes owner-scoped changes without deleting unrelated owners.
|
||||
- Forced replacement deletes only within the supplied destination bundle path.
|
||||
- Destination state is written after selected outputs are written.
|
||||
|
||||
@@ -44,7 +44,7 @@ Shared-root helper projections preserve output `created_at` for existing managed
|
||||
|
||||
## Skip And Resume Behavior
|
||||
|
||||
Comparison is pure. It returns outcomes for absent state, unmanaged content, invalid state, pipeline/destination mismatch, same source manifest, older destination, newer destination, same-created digest conflict, and different source id conflict. It does not decide whether to skip, replace, force, or fail; publish planning maps outcomes to actions.
|
||||
Comparison is pure. It returns outcomes for absent state, unmanaged content, invalid state, pipeline/destination mismatch, same source manifest, older destination, newer destination, same-created digest conflict, and different source id conflict. Shared-root helpers expose structured output ownership conflicts. State code does not decide whether to skip, replace, take over, force, or fail; publish planning maps outcomes and conflicts to actions.
|
||||
|
||||
Shared-root comparison is owner-scoped. It compares only the owner keyed by the current pipeline id and destination id, treats an absent owner as absent destination state for that owner, and can compare compatible single-owner state for the current owner without converting unrelated single-owner state.
|
||||
|
||||
@@ -74,7 +74,7 @@ Parsing rejects invalid JSON, trailing data, missing required fields, invalid ti
|
||||
- Newly written single-owner state uses schema version `2`.
|
||||
- Schema version `1` state remains readable as replacement-mode single-owner state.
|
||||
- Schema version `3` shared-root state is parsed and validated without converting unrelated single-owner state.
|
||||
- Shared-root owner updates preserve unrelated owners and reject planned path collisions with other owners.
|
||||
- Shared-root owner updates preserve unrelated owners. Publish planning removes taken-over output records before owner replacement when takeover policy allows a managed path to move owners.
|
||||
- Missing-output repair helpers preserve unrelated owner records and outputs.
|
||||
- Prune planning uses output `updated_at` and preserves unrelated shared-root owners.
|
||||
- Generated outputs always record a transform id.
|
||||
|
||||
@@ -167,7 +167,7 @@ Review these action labels before publishing:
|
||||
|
||||
Fixed destinations add fixed-path warnings during dry runs, including the selected source bundle and replacement warnings when the destination root would be replaced.
|
||||
|
||||
JSON output includes warnings, pipeline summaries, destination action records, output records, URLs when configured, final counters, and partial failure details. Fatal setup failures such as unreadable config or invalid secrets do not produce a JSON result document.
|
||||
Text and JSON summaries count `publish_new`, `replace_older`, `replace_takeover`, `force_replace`, skipped, and failed destinations separately. Takeover action records include the configured takeover mode and the conflict reason. JSON output includes warnings, pipeline summaries, destination action records, output records, URLs when configured, final counters, and partial failure details. Fatal setup failures such as unreadable config or invalid secrets do not produce a JSON result document.
|
||||
|
||||
## Forced Replacement Workflow
|
||||
|
||||
@@ -271,7 +271,7 @@ S3 execution uses the AWS SDK for Go v2. See [S3-Compatible Storage Integration]
|
||||
|
||||
When explicit S3 credential variable names are configured, both must resolve to non-empty values through the process environment or `secrets.directory`. When omitted, the AWS SDK default credential chain is used as-is.
|
||||
|
||||
Normal single-owner replacement and failed-write cleanup delete only managed output objects recorded in `.distributor.json` plus the state object. Shared-root replacement deletes only current-owner omitted output objects and rewrites the shared state object. Merge publication retains omitted managed objects and may overwrite existing managed objects. Forced replacement deletes objects under the bounded destination bundle prefix. Distributor does not manage bucket versioning or delete markers.
|
||||
Normal single-owner replacement and failed-write cleanup delete only managed output objects recorded in `.distributor.json` plus the state object. Shared-root replacement deletes only current-owner omitted output objects and rewrites the shared state object. Same-source merge publication retains omitted managed objects and may overwrite existing managed objects. Takeover replacement does not retain omitted outputs through merge reconciliation. Forced replacement deletes objects under the bounded destination bundle prefix. Distributor does not manage bucket versioning or delete markers.
|
||||
|
||||
## Secrets Operation
|
||||
|
||||
|
||||
@@ -171,7 +171,7 @@ Destination comparison rules are based on `.distributor.json`:
|
||||
- Existing state has the same source id and an older source `created`: replace, subject to destructive-operation safety rules.
|
||||
- Existing state has the same source id and a newer source `created`: skip because the destination is newer than the source.
|
||||
- Existing state has the same source id and same `created` but different digest: fail as a conflict.
|
||||
- Existing state has a different source id: fail as a conflict.
|
||||
- Existing valid managed state has a different source id, pipeline id, destination id, or shared-root output owner: replace only when destination `takeover.mode` permits that ownership transfer; otherwise fail as a conflict.
|
||||
|
||||
For older destination state, destination `reconciliation.mode` controls output updates. `replace` rewrites the managed output set to match the new plan. `merge` retains omitted managed outputs, overwrites only existing managed paths, and rejects planned paths that collide with unmanaged storage content.
|
||||
|
||||
@@ -264,6 +264,7 @@ Pipeline configuration should express:
|
||||
- per-destination transform policy;
|
||||
- per-destination public link policy;
|
||||
- validation behavior;
|
||||
- per-destination takeover behavior;
|
||||
- destination conflict/replacement behavior.
|
||||
|
||||
## Modules and Registries
|
||||
|
||||
@@ -18,7 +18,7 @@ Use it with `docs/policy/architecture.md` and `docs/policy/documentation.md`.
|
||||
- `internal/adapters/ssh`: SSH/SFTP backend.
|
||||
- `internal/adapters/s3`: S3-compatible object storage backend.
|
||||
- `internal/storage/fake`: in-memory backend for tests.
|
||||
- `internal/publish`: destination inspection, output planning, reconciliation, execution, managed cleanup, and explicit forced replacement.
|
||||
- `internal/publish`: destination inspection, output planning, takeover planning, reconciliation, execution, managed cleanup, and explicit forced replacement.
|
||||
- `internal/transform`: transform interface and registry.
|
||||
- `internal/transform/markdown`: Markdown-to-HTML transform.
|
||||
- `internal/notify`: notification interface and current no-op notifier.
|
||||
|
||||
@@ -354,7 +354,7 @@ Diagnostic:
|
||||
find <destination-path> -maxdepth 2 -print
|
||||
```
|
||||
|
||||
Safe fix: inspect the destination bundle path printed in the error. `distributor` attempts to remove outputs from the failed attempt, but operators should verify the destination before retrying. In merge mode, previously managed retained or overwritten outputs may remain intentionally. Rerun `--dry-run` before publishing again.
|
||||
Safe fix: inspect the destination bundle path printed in the error. `distributor` attempts to remove outputs from the failed attempt, but operators should verify the destination before retrying. In same-source merge mode, previously managed retained or overwritten outputs may remain intentionally. Rerun `--dry-run` before publishing again.
|
||||
|
||||
Reference: [Operations](operations.md#destination-state-and-retry-behavior).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user