Document reconciliation modes

This commit is contained in:
2026-06-08 18:36:07 +00:00
parent 93821ea6f9
commit ef0b6c1056
12 changed files with 161 additions and 35 deletions

View File

@@ -6,15 +6,23 @@ Each managed destination bundle path contains `.distributor.json`. This file is
## State Schema
Current schema version: `1`.
Current schema version: `2`.
```json
{
"schema_version": 1,
"schema_version": 2,
"distributor_version": "dev",
"pipeline_id": "reports",
"destination_id": "archive",
"published_at": "2026-06-04T12:00:00Z",
"created_at": "2026-06-04T12:00:00Z",
"updated_at": "2026-06-04T12:00:00Z",
"state": {
"mode": "single_owner"
},
"reconciliation": {
"mode": "replace"
},
"source": {
"manifest": {
"schema_version": 1,
@@ -37,7 +45,9 @@ Current schema version: `1`.
"transform": "markdown_to_html",
"url": "https://reports.example.com/archive/report.html",
"sha256": "sha256:...",
"size": 2345
"size": 2345,
"created_at": "2026-06-04T12:00:00Z",
"updated_at": "2026-06-04T12:00:00Z"
}
]
}
@@ -45,10 +55,14 @@ Current schema version: `1`.
Required fields:
- `schema_version`: must be `1`.
- `schema_version`: must be `2` for newly written state.
- `pipeline_id`: configured pipeline id that wrote the state.
- `destination_id`: configured destination id that wrote the state.
- `published_at`: RFC3339 publication timestamp.
- `created_at`: RFC3339 timestamp for when this destination state record was first created.
- `updated_at`: RFC3339 timestamp for the latest state update.
- `state.mode`: must be `single_owner`.
- `reconciliation.mode`: `replace` or `merge`.
- `source.manifest`: embedded source bundle manifest.
- `outputs`: output records array; empty is allowed, but the field is required.
@@ -68,9 +82,24 @@ Each output record has:
- `url`: optional absolute HTTP or HTTPS URL for the output.
- `sha256`: lowercase `sha256:<64 hex>` digest of the output bytes.
- `size`: output byte size, zero or greater.
- `created_at`: RFC3339 timestamp for when this output path was first recorded as managed.
- `updated_at`: RFC3339 timestamp for when this output path was last written or updated in state.
Output paths must be unique and use clean relative slash-separated path rules.
For replacement updates, newly planned outputs are written into state. For merge updates, retained output records preserve both timestamps, overwritten managed output records preserve `created_at` and receive a new `updated_at`, and new output records receive the current publication time for both fields.
## Reconciliation Semantics
Destination reconciliation applies when destination state is older than the source and transfer policy permits replacement.
- `replace`: delete managed output paths recorded in `outputs` plus `.distributor.json`, require the destination bundle path to be empty afterward, write the newly planned outputs, and write state whose `outputs` are exactly that new planned set.
- `merge`: retain managed output paths omitted from the new plan, overwrite planned paths only when they are already recorded in existing state, fail when a newly planned path exists in storage but is not recorded as managed, and write state whose `outputs` are the cumulative managed set.
Top-level `links.primary_url` is selected from the newly planned outputs for the current publication. Retained outputs keep their existing per-output URL metadata.
If a merge publication fails after writing outputs, cleanup removes only newly created outputs from that failed attempt. Previously managed overwritten paths remain managed and are not removed by failed-attempt cleanup.
## Comparison Semantics
`distributor` compares the current source manifest to destination state before writing:
@@ -84,7 +113,13 @@ Output paths must be unique and use clean relative slash-separated path rules.
- Different source id, pipeline id, or destination id: conflict.
- Invalid state JSON or invalid state fields: conflict.
Normal replacement deletes only managed output paths recorded in `outputs` plus `.distributor.json`. Forced replacement deletes the bounded destination bundle path.
Normal replacement deletes only managed output paths recorded in `outputs` plus `.distributor.json`. Merge publication retains omitted managed outputs. Forced replacement deletes the bounded destination bundle path.
## Compatibility
`distributor` can read schema version `1` destination state for compatibility. When v1 state is read, it is treated as single-owner state with `reconciliation.mode: replace`. Missing top-level `created_at` and `updated_at` are inferred from `published_at`, and missing per-output timestamps are also inferred from `published_at`.
Newly written destination state uses schema version `2`.
## Boundaries