distributor/docs/internal/publish.md

Publish Internals

Audience: developers and LLM coding agents changing internal/publish.

Purpose

internal/publish plans and executes publication for one validated source bundle and one destination bundle path. It owns destination comparison mapping, output selection, URL planning, managed cleanup selection, replacement safety, and destination state projection.

Inputs And Outputs

Inputs are a source bundle, source backend, destination backend, pipeline id, destination id, destination bundle path, path mapping mode, publish policy, transform policy, optional link policy, state policy, reconciliation policy, transformer resolver, transfer policy, distributor version, and force flag.

Output from planning is a Plan with action, reason, destination identity, selected outputs, state mode, owner scope, reconciliation mode, optional existing single-owner or shared-root state, optional primary URL, and force metadata. Shared-root plans also expose other-owner outputs to preserve, current-owner outputs retained by merge, current-owner outputs deleted by replace, and current-owner outputs to write. Execution writes selected source outputs, generated outputs, and .distributor.json for executable publish or replacement actions.

Boundaries

The package does not parse CLI flags, load config files, open concrete adapters, discover source bundles, select fixed-path bundle candidates, register transforms, prune retained outputs, or render command output. The app layer supplies validated request data and concrete dependencies.

External destination state semantics are documented in docs/integrations/destination-state.md.

Config Fields Used

The package consumes already-defaulted config values for destination publish, transform, links, state, reconciliation, transfer, and path mapping mode. It uses config.ValidatePublishTransformPolicy for publish/transform consistency.

Adapters Used

The package depends on internal/storage.Backend for source and destination IO, and on a narrow transformer resolver interface for generated outputs. It does not import concrete storage adapters or concrete transform implementations.

State And Manifest Behavior

Planning inspects destination state through internal/state, compares it with the source manifest, and maps comparison outcomes plus transfer policy into actions: publish_new, replace_older, force_replace, skip_same, skip_destination_newer, fail_conflict, or fail_unmanaged.

Single-owner destinations compare the whole destination state against the configured pipeline and destination ids. Shared-root destinations compare only the current owner scope, keyed by pipeline id and destination id. An absent shared-root owner is publishable for that owner unless a planned output collides with unmanaged storage content. Planned writes to a path owned by another shared-root owner fail as conflicts.

Execution writes destination state after selected outputs are written. Destination state includes copied source output metadata, generated output metadata, output timestamps, embedded source manifest, reconciliation metadata, link metadata when configured, pipeline id, destination id, and publication timestamps.

Skip And Resume Behavior

skip_same and skip_destination_newer execute as no-ops. Replacement-mode single-owner updates remove managed output paths from existing state plus .distributor.json, verify the destination is empty, and write state whose outputs are exactly the new plan. Replacement-mode shared-root updates remove only current-owner omitted outputs and preserve unrelated owners. Merge-mode updates retain omitted managed outputs, overwrite only paths already recorded as managed, reject unmanaged destination path collisions, and write cumulative output state. Failed writes trigger cleanup where practical; merge cleanup removes only newly created outputs from the failed attempt.

Shared-root execution writes schema version 3 state. It preserves unrelated owner records and outputs, updates only the publishing owner metadata, preserves root created_at, and updates root updated_at after successful state writes. Compatible single-owner state for the same pipeline and destination is converted to shared-root state on successful publish.

Forced replacement is explicit per request and deletes the bounded destination bundle path before writing new outputs and state.

Retention pruning is not part of publish execution and does not run automatically after a successful publish. The app-level prune workflow uses destination state after publication to select managed outputs for deletion.

Failure Behavior

Planning fails for incomplete requests, invalid publish/transform policy, invalid state mode, invalid reconciliation mode, output path collisions, invalid destination state, unmanaged destination content without force, shared-root owner path conflicts, conflict outcomes not allowed by transfer policy, unresolved transforms, invalid Markdown output selection, and invalid link URL planning.

Execution fails on delete, read, transform output, unmanaged merge path collision, shared-root ownership conflict, write, state validation, state serialization, or context errors. Execution refuses actions that are not executable publish or replacement actions.

Tests To Inspect

• internal/publish/*_test.go
• internal/app/run_test.go
• internal/state/*_test.go
• internal/transform/markdown/*_test.go

Architectural Invariants

• Planning is deterministic for the same request and destination state.
• Destination bundle paths are caller-supplied and backend-root-relative.
• URL generation uses URL path semantics and never infers public URLs from backend config.
• Replacement reconciliation deletes only managed paths recorded in existing state plus .distributor.json for single-owner state, and only current-owner omitted outputs for shared-root state.
• Merge reconciliation never adopts unmanaged content.
• Merge state output records are cumulative for the single owner.
• Shared-root planning is owner-scoped and preserves unrelated owner outputs.
• Shared-root execution writes owner-scoped changes without deleting unrelated owners.
• Forced replacement deletes only within the supplied destination bundle path.
• Destination state is written after selected outputs are written.
• Transform resolution stays behind a caller-supplied interface.
• Unmanaged content is claimed only by explicit force.