Destination State Internals

Audience: developers and LLM coding agents changing internal/state.

Purpose

internal/state parses, validates, serializes, and compares .distributor.json destination state.

Inputs And Outputs

Inputs are destination state JSON, constructed state values, current source manifest, pipeline id, destination id, and whether the destination path has content without state. Outputs are validated state values, JSON bytes, comparison outcomes, and human-readable reasons.

Boundaries

The package does not inspect storage backends, mutate files, choose transfer policy, build publish outputs, generate URLs, or parse config. Publish planning consumes state comparison outcomes.

The external destination state contract is documented in docs/integrations/destination-state.md.

Config Fields Used

internal/state uses config state mode and reconciliation mode constants for destination state validation and legacy state normalization. Destination ids, pipeline ids, and link URLs originate from config but are supplied as values by callers.

Adapters Used

None.

State And Manifest Behavior

.distributor.json schema version is 2 for newly written single-owner state. Required fields are pipeline_id, destination_id, published_at, created_at, updated_at, state.mode, reconciliation.mode, source.manifest, and outputs. distributor_version and links are optional.

Schema version 1 state remains readable. Parsing infers state.mode: single_owner, reconciliation.mode: replace, top-level created_at and updated_at from published_at, and per-output timestamps from published_at.

Schema version 3 is shared-root state. It records state.mode: shared_root, shared state timestamps, owner records keyed by pipeline id and destination id, each owner's latest source manifest and reconciliation metadata, optional owner primary links, and output records for every managed path. Shared-root output records include owner ids and compact source identity fields for source id, digest, and creation time.

Shared-root publish conversion is explicit. Compatible single-owner state for the same pipeline and destination can be projected into the current owner scope by publish execution. Unrelated single-owner state remains a conflict.

Embedded source manifests are parsed and validated through internal/bundle, which delegates source manifest semantics to pkg/bundle. Output records require clean paths, source or generated kind, valid source paths, lowercase SHA-256 digests, non-negative sizes, created and updated timestamps, and transform ids for generated outputs. Stored URLs must pass internal/link validation.

The package also provides helpers for finding output records by path, projecting planned publish outputs into timestamped state outputs, merging retained and newly planned output records, computing managed output paths from single-owner state, and removing missing managed output records from single-owner state.

For shared-root state, helpers parse either state shape, identify the current owner scope, return an owner's latest source manifest, list managed paths for one owner or all owners, detect path ownership conflicts, project planned owner outputs, merge one owner's planned outputs while preserving unrelated owners, replace one owner's outputs by removing that owner's omitted outputs, and remove missing managed output records for either the current owner or every owner.

Shared-root helper projections preserve output created_at for existing managed paths and use the current publication time for rewritten updated_at. Root-level created_at preservation is owned by publish execution.

Skip And Resume Behavior

Comparison is pure. It returns outcomes for absent state, unmanaged content, invalid state, pipeline/destination mismatch, same source manifest, older destination, newer destination, same-created digest conflict, and different source id conflict. It does not decide whether to skip, replace, force, or fail; publish planning maps outcomes to actions.

Shared-root comparison is owner-scoped. It compares only the owner keyed by the current pipeline id and destination id, treats an absent owner as absent destination state for that owner, and can compare compatible single-owner state for the current owner without converting unrelated single-owner state.

Missing-output removal helpers are pure state transformations used by app-level state repair. They remove matching output records only and leave storage inspection, timestamp updates, validation, and state rewrites to callers.

Failure Behavior

Parsing rejects invalid JSON, trailing data, missing required fields, invalid timestamps, invalid state mode, invalid reconciliation mode, invalid embedded manifests, duplicate owners, duplicate outputs, invalid output paths, unsupported output kinds, missing generated transforms, invalid URLs, invalid digests, negative sizes, and shared-root outputs whose owner is not registered.

Tests To Inspect

internal/state/distributor_test.go
internal/state/shared_root_test.go
internal/state/compare_test.go
internal/app/reconcile_state_test.go
internal/cli/reconcile_state_test.go
internal/publish/*_test.go

Architectural Invariants

.distributor.json is the destination sentinel and state record.
Comparison does not mutate storage.
Embedded source manifests use the source bundle contract.
Newly written single-owner state uses schema version 2.
Schema version 1 state remains readable as replacement-mode single-owner state.
Schema version 3 shared-root state is parsed and validated without converting unrelated single-owner state.
Shared-root owner updates preserve unrelated owners and reject planned path collisions with other owners.
Missing-output repair helpers preserve unrelated owner records and outputs.
Generated outputs always record a transform id.
Output records always carry created and updated timestamps after parsing.
Stored URLs are optional and must be absolute HTTP or HTTPS URLs when present.
distributor_version is diagnostic metadata, not a comparison key.

6.1 KiB Raw Blame History