Destination State Contract

Audience: operators, integrators, and maintainers who inspect or reason about destination .distributor.json files.

Each managed destination bundle path contains .distributor.json. This file is the destination sentinel and state record used for catalog planning, managed replacement, retention pruning, repair, and recovery.

Catalog State Schema

Publish execution writes catalog state with schema_version 4.

{
  "schema_version": 4,
  "distributor_version": "dev",
  "created_at": "2026-06-04T12:00:00Z",
  "updated_at": "2026-06-04T12:10:00Z",
  "state": {
    "mode": "catalog"
  },
  "outputs": [
    {
      "path": "report.html",
      "pipeline_id": "reports",
      "destination_id": "static-site",
      "source": {
        "id": "reports.example.2026-06-04",
        "digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
        "created": "2026-06-04T11:55:00Z"
      },
      "kind": "generated",
      "source_path": "report.md",
      "transform": "markdown_to_html",
      "url": "https://reports.example.com/archive/report.html",
      "sha256": "sha256:abcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcd",
      "size": 2345,
      "created_at": "2026-06-04T12:00:00Z",
      "updated_at": "2026-06-04T12:10:00Z"
    }
  ]
}

Top-level fields:

schema_version: required value 4.
distributor_version: optional application version string.
created_at: RFC3339 timestamp for when this catalog record was first created.
updated_at: RFC3339 timestamp for the latest catalog update.
state.mode: required value catalog.
outputs: required array of catalog output records. Empty is valid.

Output Records

Each output record has:

path: destination-bundle-relative output path.
pipeline_id: configured pipeline id that manages the output path.
destination_id: configured destination id that manages the output path.
source: compact source identity for the output.
kind: source or generated.
source_path: generated outputs only; source manifest path used to derive the output.
transform: generated outputs only; transform id, currently markdown_to_html.
url: optional absolute HTTP or HTTPS URL for the output.
sha256: lowercase sha256:<64 hex> digest of the output bytes.
size: output byte size, zero or greater.
created_at: RFC3339 timestamp for when this output path was first recorded as managed.
updated_at: RFC3339 timestamp for when this output path was last written or updated in state.

Output paths must be unique and use clean relative slash-separated path rules. pipeline_id and destination_id must be slug-like identifiers.

For copied source outputs, source_path and transform are omitted. For generated outputs, both fields are required.

Source Identity

Each output records the source identity that produced it:

source.id: source manifest id.
source.digest: source manifest digest.
source.created: source manifest creation timestamp.

The full source manifest is not embedded in catalog state. The source bundle's manifest.json remains the producer contract, and .distributor.json records only the source identity needed for catalog ownership and later maintenance workflows.

Workflow Semantics

Destination workflow is runtime configuration and is not persisted in .distributor.json.

workflow: additive writes planned outputs and retains unrelated catalog-managed outputs. If a planned path already has a catalog output record, the current publication replaces that record and overwrites the file. If a planned path exists in storage but is not recorded in valid catalog state, planning fails as unmanaged unless run --force selects force_replace.

workflow: replacement writes planned outputs for the current pipeline and destination and removes omitted outputs owned by that same pipeline and destination. Outputs owned by other pipeline/destination pairs remain catalog-managed. This is normal managed replacement and does not require --force.

force_replace is an explicit per-run recovery path. It deletes only the resolved destination bundle path, then writes planned outputs and fresh catalog state. It can replace unmanaged content, planned unmanaged path collisions, invalid destination state, and unsupported future destination state after dry-run review.

Publish Planning Outcomes

Current run reports use these destination action labels:

publish_new: no valid state exists and the destination bundle path is empty.
upsert_additive: valid catalog state exists and additive workflow will write the planned outputs.
replace_catalog: valid catalog state exists and replacement workflow will write the planned outputs and remove omitted outputs for the current owner.
skip_same: no-op action value in the run output vocabulary.
force_replace: explicit bounded destructive replacement selected by --force.
fail_unmanaged: unmanaged destination content prevents publication.
fail_conflict: invalid state or unsupported state prevents publication without explicit force.

Schema versions older than 4 are superseded legacy state for publish planning. Normal catalog planning may publish over superseded legacy state according to the configured workflow, while invalid state and unsupported future schema versions fail unless --force is explicitly selected.

Repair Semantics

distributor reconcile-state removes catalog output records for files that no longer exist in destination storage. It uses the configured pipeline and destination selector to open one destination root and reads that root's .distributor.json.

By default, repair is scoped to output records whose pipeline_id and destination_id match the selected pipeline and destination. With --all-owners, it checks every catalog output record in the selected root.

The command reports missing managed outputs and unmanaged storage entries. Without --dry-run, it removes missing managed output records from valid catalog state and rewrites .distributor.json. It does not delete destination files, adopt unmanaged entries, validate output digests, or rewrite invalid state.

Prune Semantics

distributor prune deletes catalog output paths selected by the configured destination retention.prune policy. It uses the configured pipeline and destination selector to open one destination root and reads that root's .distributor.json.

Prune planning is scoped to output records whose pipeline_id and destination_id match the selected pipeline and destination. It uses output updated_at timestamps. prune --dry-run reports planned managed-output deletes without deleting files or rewriting state. prune --apply deletes only planned managed output paths, removes confirmed deleted records from valid catalog state, and updates the state timestamp. It does not delete unmanaged files or .distributor.json.

Boundaries

Destination state is internal managed state written by distributor. Operators may inspect it during recovery, but normal workflows should not edit it by hand. Source manifest.json is not copied as destination state.

Before changing this contract, inspect and run:

go test ./internal/state ./internal/publish

7.2 KiB Raw Blame History