distributor/docs/integrations/destination-state.md

Destination State Contract

Audience: operators, integrators, and maintainers who inspect or reason about destination .distributor.json files.

Each managed destination bundle path contains .distributor.json. This file is the destination sentinel and state record used for comparison, skip, replacement, and recovery decisions.

Single-Owner State Schema

State written by run for state.mode: single_owner uses schema version 2.

{
  "schema_version": 2,
  "distributor_version": "dev",
  "pipeline_id": "reports",
  "destination_id": "archive",
  "published_at": "2026-06-04T12:00:00Z",
  "created_at": "2026-06-04T12:00:00Z",
  "updated_at": "2026-06-04T12:00:00Z",
  "state": {
    "mode": "single_owner"
  },
  "reconciliation": {
    "mode": "replace"
  },
  "source": {
    "manifest": {
      "schema_version": 1,
      "id": "reports.example.2026-06-04",
      "digest": "sha256:...",
      "created": "2026-06-04T11:55:00Z",
      "files": [
        {"path": "report.md", "sha256": "sha256:...", "size": 1234}
      ]
    }
  },
  "links": {
    "primary_url": "https://reports.example.com/archive/report.html"
  },
  "outputs": [
    {
      "path": "report.html",
      "kind": "generated",
      "source_path": "report.md",
      "transform": "markdown_to_html",
      "url": "https://reports.example.com/archive/report.html",
      "sha256": "sha256:...",
      "size": 2345,
      "created_at": "2026-06-04T12:00:00Z",
      "updated_at": "2026-06-04T12:00:00Z"
    }
  ]
}

Required fields:

• schema_version: must be 2 for newly written state.
• pipeline_id: configured pipeline id that wrote the state.
• destination_id: configured destination id that wrote the state.
• published_at: RFC3339 publication timestamp.
• created_at: RFC3339 timestamp for when this destination state record was first created.
• updated_at: RFC3339 timestamp for the latest state update.
• state.mode: must be single_owner.
• reconciliation.mode: replace or merge.
• source.manifest: embedded source bundle manifest.
• outputs: output records array; empty is allowed, but the field is required.

Optional fields:

• distributor_version: application version string when available.
• links.primary_url: absolute HTTP or HTTPS URL selected by destination link policy.

Output Records

Each output record has:

• path: destination-relative output path.
• kind: source or generated.
• source_path: source manifest path used for the output.
• transform: required for generated outputs; omitted for copied source outputs.
• url: optional absolute HTTP or HTTPS URL for the output.
• sha256: lowercase sha256:<64 hex> digest of the output bytes.
• size: output byte size, zero or greater.
• created_at: RFC3339 timestamp for when this output path was first recorded as managed.
• updated_at: RFC3339 timestamp for when this output path was last written or updated in state.

Output paths must be unique and use clean relative slash-separated path rules.

For replacement updates, newly planned outputs are written into state. For merge updates, retained output records preserve both timestamps, overwritten managed output records preserve created_at and receive a new updated_at, and new output records receive the current publication time for both fields.

Reconciliation Semantics

Destination reconciliation applies when destination state is older than the source and transfer policy permits replacement.

• replace: for single-owner state, delete managed output paths recorded in outputs plus .distributor.json, require the destination bundle path to be empty afterward, write the newly planned outputs, and write state whose outputs are exactly that new planned set. For shared-root state, delete only the current owner's omitted outputs and preserve unrelated owners.
• merge: retain managed output paths omitted from the new plan, overwrite planned paths only when they are already recorded in existing state, fail when a newly planned path exists in storage but is not recorded as managed, and write state whose outputs are the cumulative managed set.

Takeover replacement applies when valid managed state has an identity, source, or shared-root output-owner mismatch and destination takeover.mode permits the current publication to take ownership. It uses bounded managed replacement behavior. For shared-root state, only taken-over output records and the current owner records are rewritten; unrelated owner records and non-conflicting outputs remain managed by their existing owners.

Top-level links.primary_url is selected from the newly planned outputs for the current publication. Retained outputs keep their existing per-output URL metadata.

If a same-source merge publication fails after writing outputs, cleanup removes only newly created outputs from that failed attempt. Previously managed overwritten paths remain managed and are not removed by failed-attempt cleanup. Takeover replacement does not retain omitted outputs through merge reconciliation.

Comparison Semantics

distributor compares the current source manifest to destination state before writing:

• No state and no content: publish new outputs.
• No state and existing content: treat the destination as unmanaged.
• Shared-root state without the current owner: publish new outputs for that owner if planned paths do not collide with unmanaged content or with other owners that takeover.mode does not permit.
• Matching embedded source manifest: skip.
• Same source id with older created: replace if policy allows.
• Same source id with newer created: skip by default, or replace when transfer.on_destination_newer: replace is configured.
• Same source id and same created with different digest: conflict by default, or replace when transfer.on_conflict: replace is configured.
• Different source id, pipeline id, or destination id in single-owner state: conflict unless takeover.mode permits managed ownership transfer or transfer.on_conflict: replace is configured.
• Shared-root output path owned by a different owner: conflict unless takeover.mode permits managed ownership transfer or transfer.on_conflict: replace is configured.
• Invalid state JSON or invalid state fields: conflict.

Normal single-owner replacement deletes only managed output paths recorded in outputs plus .distributor.json. Shared-root replacement deletes only omitted outputs for the current owner. Merge publication retains omitted managed outputs for same-source replacement. Forced replacement deletes the bounded destination bundle path.

State Repair Semantics

distributor reconcile-state can remove managed output records for files that no longer exist in destination storage. It uses the configured pipeline and destination selector to open one destination root and reads that root's .distributor.json.

For single-owner state, the state pipeline_id and destination_id must match the selected pipeline and destination. The command checks output paths recorded in outputs, reports missing managed outputs, reports unmanaged entries under the destination root, and removes missing output records from valid state unless --dry-run is set.

For shared-root state, repair is scoped to the selected owner by default. With --all-owners, it checks and repairs missing output records for every owner in the selected shared-root state file.

State repair does not validate output digests, delete destination files, adopt unmanaged entries, or rewrite invalid or mismatched state.

Prune Semantics

distributor prune can delete managed outputs selected by the configured destination retention policy. It uses the configured pipeline and destination selector to open one destination root and reads that root's .distributor.json.

For single-owner state, the state pipeline_id and destination_id must match the selected pipeline and destination. For shared-root state, pruning is scoped to the selected owner and preserves other owners.

Prune planning uses output updated_at timestamps. prune --dry-run reports planned managed-output deletes without deleting files or rewriting state. prune --apply deletes only planned managed output paths, removes confirmed deleted records from valid state, and updates the state timestamp. It does not delete unmanaged files or .distributor.json.

Compatibility

distributor can read schema version 1 destination state for compatibility. When v1 state is read, it is treated as single-owner state with reconciliation.mode: replace. Missing top-level created_at and updated_at are inferred from published_at, and missing per-output timestamps are also inferred from published_at.

Newly written single-owner destination state from publish execution uses schema version 2.

Shared-Root State Schema

State written by run for state.mode: shared_root uses schema version 3.

{
  "schema_version": 3,
  "distributor_version": "dev",
  "created_at": "2026-06-04T12:00:00Z",
  "updated_at": "2026-06-04T12:10:00Z",
  "state": {
    "mode": "shared_root"
  },
  "owners": [
    {
      "pipeline_id": "reports",
      "destination_id": "archive",
      "reconciliation": {
        "mode": "merge"
      },
      "source": {
        "manifest": {
          "schema_version": 1,
          "id": "reports.example.2026-06-04",
          "digest": "sha256:...",
          "created": "2026-06-04T11:55:00Z",
          "files": [
            {"path": "report.md", "sha256": "sha256:...", "size": 1234}
          ]
        }
      },
      "links": {
        "primary_url": "https://reports.example.com/archive/report.html"
      }
    }
  ],
  "outputs": [
    {
      "path": "report.html",
      "kind": "generated",
      "source_path": "report.md",
      "transform": "markdown_to_html",
      "url": "https://reports.example.com/archive/report.html",
      "sha256": "sha256:...",
      "size": 2345,
      "pipeline_id": "reports",
      "destination_id": "archive",
      "source_id": "reports.example.2026-06-04",
      "source_digest": "sha256:...",
      "source_created": "2026-06-04T11:55:00Z",
      "created_at": "2026-06-04T12:00:00Z",
      "updated_at": "2026-06-04T12:10:00Z"
    }
  ]
}

Shared-root required fields:

• schema_version: must be 3.
• created_at: RFC3339 timestamp for when this shared-root state record was first created.
• updated_at: RFC3339 timestamp for the latest shared-root state update.
• state.mode: must be shared_root.
• owners: owner records keyed by pipeline_id and destination_id; each owner records its latest source manifest and reconciliation mode.
• outputs: output records for every managed path under the shared destination root.

Shared-root optional fields:

• distributor_version: application version string when available.
• owners[].links.primary_url: absolute HTTP or HTTPS URL selected by that owner's destination link policy.

Shared-root output records carry the same path, kind, source_path, transform, url, sha256, size, created_at, and updated_at fields as single-owner outputs. They also include the owner pipeline_id and destination_id, plus compact source identity fields source_id, source_digest, and source_created.

Shared-Root Ownership

Shared-root state is owner-scoped by pipeline_id and destination_id.

• One output path may be owned by only one owner.
• The same owner may overwrite its own managed paths.
• A different owner planning an already owned path fails as a conflict.
• A planned path that exists in storage but is not recorded in state fails as unmanaged content unless forced replacement is explicitly selected.

When an owner publishes, unrelated owner records and output records are preserved. The publishing owner's record is updated with the latest source manifest, reconciliation mode, and latest primary URL when present.

Shared-Root Timestamps

For shared-root state:

• top-level created_at remains the original shared-root state creation time;
• top-level updated_at changes after a successful state write;
• output created_at remains stable for an existing managed path;
• output updated_at changes only when that path is rewritten;
• newly managed output paths receive the publication time for both output timestamps.

Shared-Root Migration

If state.mode: shared_root is configured and existing state is a compatible single-owner .distributor.json for the same pipeline id and destination id, the next successful publish writes schema version 3 shared-root state for that owner.

If state.mode: shared_root is configured and existing single-owner state belongs to a different pipeline or destination, publish fails as a conflict. distributor does not implicitly convert unrelated single-owner state or take over unmanaged files during shared-root migration.

Boundaries

Destination state is internal managed state written by distributor. Operators may inspect it during recovery, but normal workflows should not edit it by hand. Source manifest.json is not copied as destination state.

Tests

Before changing this contract, inspect and run:

go test ./internal/state ./internal/publish