Add shared-root destination state model

2026-06-08 18:45:16 +00:00
parent ef0b6c1056
commit eb86cf9ab6
13 changed files with 1165 additions and 10 deletions
--- a/docs/config.md
+++ b/docs/config.md
@@ -255,7 +255,7 @@ Source bundle digest mismatches fail validation before destination writes occur.

 ## Destination Fields

-Each destination embeds a backend config at the destination level and may also configure publishing, transforms, path mapping, links, reconciliation, and transfer behavior.
+Each destination embeds a backend config at the destination level and may also configure publishing, transforms, path mapping, links, state, reconciliation, and transfer behavior.

 ```yaml
 destinations:
@@ -267,6 +267,8 @@ destinations:
      html: false
    path_mapping:
      mode: preserve_relative
+    state:
+      mode: single_owner
    reconciliation:
      mode: replace
    transfer:
@@ -282,6 +284,7 @@ destinations:
 - `transform`: required only when publishing generated HTML.
 - `path_mapping`: optional destination path mapping policy.
 - `links`: optional public URL metadata policy.
+- `state`: optional destination state ownership policy.
 - `reconciliation`: optional managed-output reconciliation policy.
 - `transfer`: optional destination comparison action policy.

@@ -359,6 +362,19 @@ Primary URL policies:

 If no output matches the primary policy, per-output URLs may still be recorded and the top-level primary URL is omitted.

+## Destination State Policy
+
+```yaml
+state:
+  mode: single_owner
+```
+
+- `state.mode`: optional. Accepted values are `single_owner` and `shared_root`; default is `single_owner`.
+
+`single_owner` state records one pipeline/destination owner for each destination bundle path and is the state mode written by `run`.
+
+`shared_root` is accepted by configuration validation and by destination state parsing for shared-root `.distributor.json` files. Current publish execution writes single-owner destination state.
+
 ## Reconciliation Policy

 ```yaml
@@ -448,6 +464,7 @@ Defaults are applied after YAML decoding and before validation:
 - `transform.markdown_to_html.mode: sidecar` when a Markdown transform block is present and mode is omitted
 - `path_mapping.mode: preserve_relative`
 - `links.primary: auto` when a `links` block is present and `primary` is omitted
+- `state.mode: single_owner`
 - `reconciliation.mode: replace`
 - `transfer.on_destination_same: skip`
 - `transfer.on_destination_older: replace`
--- a/docs/integrations/destination-state.md
+++ b/docs/integrations/destination-state.md
@@ -4,9 +4,9 @@ Audience: operators, integrators, and maintainers who inspect or reason about de

 Each managed destination bundle path contains `.distributor.json`. This file is the destination sentinel and state record used for comparison, skip, replacement, and recovery decisions.

-## State Schema
+## Single-Owner State Schema

-Current schema version: `2`.
+Current state written by `run` uses schema version `2`.

 ```json
 {
@@ -119,7 +119,80 @@ Normal replacement deletes only managed output paths recorded in `outputs` plus

 `distributor` can read schema version `1` destination state for compatibility. When v1 state is read, it is treated as single-owner state with `reconciliation.mode: replace`. Missing top-level `created_at` and `updated_at` are inferred from `published_at`, and missing per-output timestamps are also inferred from `published_at`.

-Newly written destination state uses schema version `2`.
+Newly written destination state from publish execution uses schema version `2`.
+
+## Shared-Root State Schema
+
+`distributor` can parse and validate shared-root destination state with schema version `3`. Publish execution currently writes single-owner state.
+
+```json
+{
+  "schema_version": 3,
+  "distributor_version": "dev",
+  "created_at": "2026-06-04T12:00:00Z",
+  "updated_at": "2026-06-04T12:10:00Z",
+  "state": {
+    "mode": "shared_root"
+  },
+  "owners": [
+    {
+      "pipeline_id": "reports",
+      "destination_id": "archive",
+      "reconciliation": {
+        "mode": "merge"
+      },
+      "source": {
+        "manifest": {
+          "schema_version": 1,
+          "id": "reports.example.2026-06-04",
+          "digest": "sha256:...",
+          "created": "2026-06-04T11:55:00Z",
+          "files": [
+            {"path": "report.md", "sha256": "sha256:...", "size": 1234}
+          ]
+        }
+      },
+      "links": {
+        "primary_url": "https://reports.example.com/archive/report.html"
+      }
+    }
+  ],
+  "outputs": [
+    {
+      "path": "report.html",
+      "kind": "generated",
+      "source_path": "report.md",
+      "transform": "markdown_to_html",
+      "url": "https://reports.example.com/archive/report.html",
+      "sha256": "sha256:...",
+      "size": 2345,
+      "pipeline_id": "reports",
+      "destination_id": "archive",
+      "source_id": "reports.example.2026-06-04",
+      "source_digest": "sha256:...",
+      "source_created": "2026-06-04T11:55:00Z",
+      "created_at": "2026-06-04T12:00:00Z",
+      "updated_at": "2026-06-04T12:10:00Z"
+    }
+  ]
+}
+```
+
+Shared-root required fields:
+
+- `schema_version`: must be `3`.
+- `created_at`: RFC3339 timestamp for when this shared-root state record was first created.
+- `updated_at`: RFC3339 timestamp for the latest shared-root state update.
+- `state.mode`: must be `shared_root`.
+- `owners`: owner records keyed by `pipeline_id` and `destination_id`; each owner records its latest source manifest and reconciliation mode.
+- `outputs`: output records for every managed path under the shared destination root.
+
+Shared-root optional fields:
+
+- `distributor_version`: application version string when available.
+- `owners[].links.primary_url`: absolute HTTP or HTTPS URL selected by that owner's destination link policy.
+
+Shared-root output records carry the same `path`, `kind`, `source_path`, `transform`, `url`, `sha256`, `size`, `created_at`, and `updated_at` fields as single-owner outputs. They also include the owner `pipeline_id` and `destination_id`, plus compact source identity fields `source_id`, `source_digest`, and `source_created`.

 ## Boundaries

--- a/docs/internal/state.md
+++ b/docs/internal/state.md
@@ -18,7 +18,7 @@ The external destination state contract is documented in `docs/integrations/dest

 ## Config Fields Used

-`internal/state` uses config reconciliation mode constants for destination state validation and legacy state normalization. Destination ids, pipeline ids, and link URLs originate from config but are supplied as values by callers.
+`internal/state` uses config state mode and reconciliation mode constants for destination state validation and legacy state normalization. Destination ids, pipeline ids, and link URLs originate from config but are supplied as values by callers.

 ## Adapters Used

@@ -30,21 +30,26 @@ None.

 Schema version `1` state remains readable. Parsing infers `state.mode: single_owner`, `reconciliation.mode: replace`, top-level `created_at` and `updated_at` from `published_at`, and per-output timestamps from `published_at`.

+Schema version `3` is shared-root state. It records `state.mode: shared_root`, shared state timestamps, owner records keyed by pipeline id and destination id, each owner's latest source manifest and reconciliation metadata, optional owner primary links, and output records for every managed path. Shared-root output records include owner ids and compact source identity fields for source id, digest, and creation time.
+
 Embedded source manifests are parsed and validated through `internal/bundle`, which delegates source manifest semantics to `pkg/bundle`. Output records require clean paths, `source` or `generated` kind, valid source paths, lowercase SHA-256 digests, non-negative sizes, created and updated timestamps, and transform ids for generated outputs. Stored URLs must pass `internal/link` validation.

 The package also provides helpers for finding output records by path, projecting planned publish outputs into timestamped state outputs, merging retained and newly planned output records, and computing managed output paths from single-owner state.

+For shared-root state, helpers parse either state shape, identify the current owner scope, return an owner's latest source manifest, list managed paths for one owner or all owners, detect path ownership conflicts, project planned owner outputs, merge one owner's planned outputs while preserving unrelated owners, and replace one owner's outputs by removing that owner's omitted outputs.
+
 ## Skip And Resume Behavior

 Comparison is pure. It returns outcomes for absent state, unmanaged content, invalid state, pipeline/destination mismatch, same source manifest, older destination, newer destination, same-created digest conflict, and different source id conflict. It does not decide whether to skip, replace, force, or fail; publish planning maps outcomes to actions.

 ## Failure Behavior

-Parsing rejects invalid JSON, trailing data, missing required fields, invalid timestamps, invalid state mode, invalid reconciliation mode, invalid embedded manifests, duplicate outputs, invalid output paths, unsupported output kinds, missing generated transforms, invalid URLs, invalid digests, and negative sizes.
+Parsing rejects invalid JSON, trailing data, missing required fields, invalid timestamps, invalid state mode, invalid reconciliation mode, invalid embedded manifests, duplicate owners, duplicate outputs, invalid output paths, unsupported output kinds, missing generated transforms, invalid URLs, invalid digests, negative sizes, and shared-root outputs whose owner is not registered.

 ## Tests To Inspect

 - `internal/state/distributor_test.go`
+- `internal/state/shared_root_test.go`
 - `internal/state/compare_test.go`
 - `internal/publish/*_test.go`

@@ -55,6 +60,8 @@ Parsing rejects invalid JSON, trailing data, missing required fields, invalid ti
 - Embedded source manifests use the source bundle contract.
 - Newly written single-owner state uses schema version `2`.
 - Schema version `1` state remains readable as replacement-mode single-owner state.
+- Schema version `3` shared-root state is parsed and validated without converting unrelated single-owner state.
+- Shared-root owner updates preserve unrelated owners and reject planned path collisions with other owners.
 - Generated outputs always record a transform id.
 - Output records always carry created and updated timestamps after parsing.
 - Stored URLs are optional and must be absolute HTTP or HTTPS URLs when present.
--- a/docs/policy/architecture.md
+++ b/docs/policy/architecture.md
@@ -94,7 +94,11 @@ The source manifest should remain minimal. Routing, destination selection, publi

 `manifest.json` from the source bundle is not copied to destinations as destination state.

-Each destination bundle path is managed by `.distributor.json`. This file is both the destination sentinel and the destination state record. It records:
+Each destination bundle path is managed by `.distributor.json`. This file is both the destination sentinel and the destination state record.
+
+Publish execution currently writes single-owner destination state. `internal/state` also parses and validates shared-root destination state, where one `.distributor.json` records multiple pipeline/destination owners and every managed output carries its owner identity.
+
+Single-owner state records:

 - `distributor` state schema version;
 - pipeline id;