Document reconciliation modes

This commit is contained in:
2026-06-08 18:36:07 +00:00
parent 93821ea6f9
commit ef0b6c1056
12 changed files with 161 additions and 35 deletions

View File

@@ -20,7 +20,7 @@ User-facing command parsing stays in `internal/cli`. User-facing config referenc
## Config Fields Used
The package consumes the loaded `config.Config`: `server.http`, `secrets.directory`, pipeline ids, source and destination backend fields, validation policy, publish policy, transform policy, path mapping, links, and transfer policy.
The package consumes the loaded `config.Config`: `server.http`, `secrets.directory`, pipeline ids, source and destination backend fields, validation policy, publish policy, transform policy, path mapping, links, reconciliation policy, and transfer policy.
Config fields are validated and defaulted by `internal/config` before app workflows use them.

View File

@@ -8,9 +8,9 @@ Audience: developers and LLM coding agents changing `internal/publish`.
## Inputs And Outputs
Inputs are a source bundle, source backend, destination backend, pipeline id, destination id, destination bundle path, path mapping mode, publish policy, transform policy, optional link policy, transformer resolver, transfer policy, distributor version, and force flag.
Inputs are a source bundle, source backend, destination backend, pipeline id, destination id, destination bundle path, path mapping mode, publish policy, transform policy, optional link policy, reconciliation policy, transformer resolver, transfer policy, distributor version, and force flag.
Output from planning is a `Plan` with action, reason, destination identity, selected outputs, optional existing state, optional primary URL, and force metadata. Execution writes selected source outputs, generated outputs, and `.distributor.json` for executable publish or replacement actions.
Output from planning is a `Plan` with action, reason, destination identity, selected outputs, reconciliation mode, optional existing state, optional primary URL, and force metadata. Execution writes selected source outputs, generated outputs, and `.distributor.json` for executable publish or replacement actions.
## Boundaries
@@ -20,7 +20,7 @@ External destination state semantics are documented in `docs/integrations/destin
## Config Fields Used
The package consumes already-defaulted config values for destination `publish`, `transform`, `links`, `transfer`, and path mapping mode. It uses `config.ValidatePublishTransformPolicy` for publish/transform consistency.
The package consumes already-defaulted config values for destination `publish`, `transform`, `links`, `reconciliation`, `transfer`, and path mapping mode. It uses `config.ValidatePublishTransformPolicy` for publish/transform consistency.
## Adapters Used
@@ -30,19 +30,19 @@ The package depends on `internal/storage.Backend` for source and destination IO,
Planning inspects destination state through `internal/state`, compares it with the source manifest, and maps comparison outcomes plus transfer policy into actions: `publish_new`, `replace_older`, `force_replace`, `skip_same`, `skip_destination_newer`, `fail_conflict`, or `fail_unmanaged`.
Execution writes destination state after selected outputs are written. Destination state includes copied source output metadata, generated output metadata, embedded source manifest, link metadata when configured, pipeline id, destination id, and publication timestamp.
Execution writes destination state after selected outputs are written. Destination state includes copied source output metadata, generated output metadata, output timestamps, embedded source manifest, reconciliation metadata, link metadata when configured, pipeline id, destination id, and publication timestamps.
## Skip And Resume Behavior
`skip_same` and `skip_destination_newer` execute as no-ops. Normal replacement removes only managed output paths from existing state plus `.distributor.json`; this allows retries without broad deletion. Failed writes trigger cleanup of outputs written during that failed attempt where practical.
`skip_same` and `skip_destination_newer` execute as no-ops. Replacement-mode updates remove only managed output paths from existing state plus `.distributor.json`, verify the destination is empty, and write state whose outputs are exactly the new plan. Merge-mode updates retain omitted managed outputs, overwrite only paths already recorded as managed, reject unmanaged destination path collisions, and write cumulative output state. Failed writes trigger cleanup where practical; merge cleanup removes only newly created outputs from the failed attempt.
Forced replacement is explicit per request and deletes the bounded destination bundle path before writing new outputs and state.
## Failure Behavior
Planning fails for incomplete requests, invalid publish/transform policy, output path collisions, invalid destination state, unmanaged destination content without force, conflict outcomes not allowed by transfer policy, unresolved transforms, invalid Markdown output selection, and invalid link URL planning.
Planning fails for incomplete requests, invalid publish/transform policy, invalid reconciliation mode, output path collisions, invalid destination state, unmanaged destination content without force, conflict outcomes not allowed by transfer policy, unresolved transforms, invalid Markdown output selection, and invalid link URL planning.
Execution fails on delete, read, transform output, write, state validation, state serialization, or context errors. Execution refuses actions that are not executable publish or replacement actions.
Execution fails on delete, read, transform output, unmanaged merge path collision, write, state validation, state serialization, or context errors. Execution refuses actions that are not executable publish or replacement actions.
## Tests To Inspect
@@ -56,7 +56,9 @@ Execution fails on delete, read, transform output, write, state validation, stat
- Planning is deterministic for the same request and destination state.
- Destination bundle paths are caller-supplied and backend-root-relative.
- URL generation uses URL path semantics and never infers public URLs from backend config.
- Normal replacement deletes only managed paths recorded in existing state plus `.distributor.json`.
- Replacement reconciliation deletes only managed paths recorded in existing state plus `.distributor.json`.
- Merge reconciliation never adopts unmanaged content.
- Merge state output records are cumulative for the single owner.
- Forced replacement deletes only within the supplied destination bundle path.
- Destination state is written after selected outputs are written.
- Transform resolution stays behind a caller-supplied interface.

View File

@@ -18,7 +18,7 @@ The external destination state contract is documented in `docs/integrations/dest
## Config Fields Used
None directly. Destination ids, pipeline ids, and link URLs originate from config but are supplied as values by callers.
`internal/state` uses config reconciliation mode constants for destination state validation and legacy state normalization. Destination ids, pipeline ids, and link URLs originate from config but are supplied as values by callers.
## Adapters Used
@@ -26,9 +26,13 @@ None.
## State And Manifest Behavior
`.distributor.json` schema version is `1`. Required fields are `pipeline_id`, `destination_id`, `published_at`, `source.manifest`, and `outputs`. `distributor_version` and `links` are optional.
`.distributor.json` schema version is `2` for newly written single-owner state. Required fields are `pipeline_id`, `destination_id`, `published_at`, `created_at`, `updated_at`, `state.mode`, `reconciliation.mode`, `source.manifest`, and `outputs`. `distributor_version` and `links` are optional.
Embedded source manifests are parsed and validated through `internal/bundle`, which delegates source manifest semantics to `pkg/bundle`. Output records require clean paths, `source` or `generated` kind, valid source paths, lowercase SHA-256 digests, non-negative sizes, and transform ids for generated outputs. Stored URLs must pass `internal/link` validation.
Schema version `1` state remains readable. Parsing infers `state.mode: single_owner`, `reconciliation.mode: replace`, top-level `created_at` and `updated_at` from `published_at`, and per-output timestamps from `published_at`.
Embedded source manifests are parsed and validated through `internal/bundle`, which delegates source manifest semantics to `pkg/bundle`. Output records require clean paths, `source` or `generated` kind, valid source paths, lowercase SHA-256 digests, non-negative sizes, created and updated timestamps, and transform ids for generated outputs. Stored URLs must pass `internal/link` validation.
The package also provides helpers for finding output records by path, projecting planned publish outputs into timestamped state outputs, merging retained and newly planned output records, and computing managed output paths from single-owner state.
## Skip And Resume Behavior
@@ -36,7 +40,7 @@ Comparison is pure. It returns outcomes for absent state, unmanaged content, inv
## Failure Behavior
Parsing rejects invalid JSON, trailing data, missing required fields, invalid timestamps, invalid embedded manifests, duplicate outputs, invalid output paths, unsupported output kinds, missing generated transforms, invalid URLs, invalid digests, and negative sizes.
Parsing rejects invalid JSON, trailing data, missing required fields, invalid timestamps, invalid state mode, invalid reconciliation mode, invalid embedded manifests, duplicate outputs, invalid output paths, unsupported output kinds, missing generated transforms, invalid URLs, invalid digests, and negative sizes.
## Tests To Inspect
@@ -49,6 +53,9 @@ Parsing rejects invalid JSON, trailing data, missing required fields, invalid ti
- `.distributor.json` is the destination sentinel and state record.
- Comparison does not mutate storage.
- Embedded source manifests use the source bundle contract.
- Newly written single-owner state uses schema version `2`.
- Schema version `1` state remains readable as replacement-mode single-owner state.
- Generated outputs always record a transform id.
- Output records always carry created and updated timestamps after parsing.
- Stored URLs are optional and must be absolute HTTP or HTTPS URLs when present.
- `distributor_version` is diagnostic metadata, not a comparison key.