Close out HTTP upload documentation

README.md

@@ -2,7 +2,12 @@
 
 `distributor` validates manifested report bundles and publishes selected source or generated artifacts to configured destinations.
 
-It is currently a local-first CLI: source bundles are read from local storage, destinations are local directories, and Markdown files can be rendered to HTML sidecars.
+It is a local-first CLI with SSH/SFTP, S3-compatible storage, and HTTP upload
+support: source bundles can be read from local or remote storage, pushed to the
+upload API, published to local directories or remote paths, and rendered from
+Markdown to HTML sidecars or `index.html`.
+
+Go producers can use `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to build, write, parse, and validate complete local source bundles with the same manifest contract used by `distributor`.
 
 Run the local example pipeline:
 
@@ -10,4 +15,4 @@ Run the local example pipeline:
 go run ./cmd/distributor run --config examples/local-publish.yml
 ```
 
-See [docs/cli.md](docs/cli.md), [docs/config.md](docs/config.md), [docs/operations.md](docs/operations.md), and [docs/troubleshooting.md](docs/troubleshooting.md) for the implemented CLI, configuration, operating notes, and common failure modes. Planning material lives under `docs/roadmap/`.
+See [docs/cli.md](docs/cli.md), [docs/config.md](docs/config.md), [docs/operations.md](docs/operations.md), and [docs/troubleshooting.md](docs/troubleshooting.md) for the implemented CLI, configuration, operating notes, and common failure modes. Future and deferred work lives under `docs/roadmap/`.

docs/cli.md

@@ -12,18 +12,24 @@ This discovers the example source bundle and publishes source files to `workspac
 
 ```sh
 distributor [--help]
-distributor version
-distributor run [--config <path>] [--dry-run]
-distributor validate <path>
-distributor inspect <path>
+distributor version [--format text|json]
+distributor run [--config <path>] [--dry-run] [--force] [--format text|json]
+distributor serve [--config <path>]
+distributor validate [--format text|json] <path>
+distributor validate --config <path> --pipeline <id> [--bundle <path>] [--format text|json]
+distributor inspect [--format text|json] <path>
+distributor inspect --config <path> --pipeline <id> [--bundle <path>] [--format text|json]
+distributor manifest create <bundle-path> --id <bundle-id> [options]
 ```
 
 - `version`: prints the application name and version. Development builds print `distributor dev`.
-- `run`: loads a YAML config, discovers local source bundles, plans each configured destination, writes selected outputs unless `--dry-run` is set, and prints a final status summary.
-- `validate`: validates a local source bundle directory or a local tree containing source bundles.
-- `inspect`: validates local source bundles and prints normalized bundle metadata.
+- `run`: loads a YAML config, discovers source bundles, plans each configured destination, writes selected outputs unless `--dry-run` is set, and prints a final status summary.
+- `serve`: loads a YAML config, resolves HTTP upload bearer tokens, and runs the HTTP upload API.
+- `validate`: validates a local source bundle directory, a local source bundle tree, or one configured pipeline source.
+- `inspect`: validates source bundles and prints normalized bundle metadata for a local path or one configured pipeline source.
+- `manifest create`: creates `manifest.json` for a local source bundle directory.
 
-`validate` and `inspect` accept local paths only. `run` currently executes local backends only. SSH and S3 config can be parsed and validated, but configured SSH or S3 execution fails with a clear unsupported-execution error.
+`validate` and `inspect` have two mutually exclusive modes: a local path shortcut, or configured source mode with `--config <path> --pipeline <id>`. Configured source mode opens only the selected pipeline source and supports configured `local`, `ssh`, and `s3` sources. It does not open destinations. `run` executes configured `local`, `ssh`, and `s3` sources and destinations. `serve` executes configured `http_upload` sources through the upload API and normal destination fan-out.
 
 ## Flag reference
 
@@ -35,12 +41,34 @@ All subcommands:
 
 - `--help`, `-h`: print command-specific help.
 
+Output-producing subcommands:
+
+- `--format text|json`: output format. `text` is the default. Help and usage output are always text.
+
 `run` flags:
 
 - `--config <path>`: config file to load. If omitted, `run` uses `/usr/local/etc/distributor/config.yml`.
-- `--dry-run`: load config, discover bundles, inspect destination state, print planned actions and final status, and do not write files.
+- `--dry-run`: load config, discover bundles, inspect destination state, print planned actions and final status, and do not write output files, destination state, or SSH `known_hosts` entries.
+- `--force`: allow explicit destructive replacement for supported conflict cases in this run only.
 
-`run` does not accept positional arguments. `validate` and `inspect` accept at most one path; omitting the path returns a required-path error.
+`serve` flags:
+
+- `--config <path>`: config file to load. If omitted, `serve` uses `/usr/local/etc/distributor/config.yml`.
+
+`validate` and `inspect` configured source flags:
+
+- `--config <path>`: config file to load for source validation or inspection. Required in configured source mode.
+- `--pipeline <id>`: pipeline source to validate or inspect. Required in configured source mode.
+- `--bundle <path>`: source-root-relative bundle directory to validate or inspect instead of discovering every bundle under the source root.
+
+`manifest create` flags:
+
+- `--id <bundle-id>`: source bundle id. Required.
+- `--file <path>`: bundle-relative file to include. Repeatable. If omitted, files are scanned recursively.
+- `--created <time>`: RFC3339 source created timestamp. If omitted, the current UTC time is used.
+- `--overwrite`: replace an existing `manifest.json`.
+
+`run` does not accept positional arguments. `validate` and `inspect` accept at most one path in local mode. Local paths cannot be combined with `--config`, `--pipeline`, or `--bundle`.
 
 ## Common workflows
 
@@ -56,6 +84,37 @@ Inspect a source bundle:
 go run ./cmd/distributor inspect examples/source-bundle
 ```
 
+Validate a configured source without opening destinations:
+
+```sh
+go run ./cmd/distributor validate --config examples/local-publish.yml --pipeline example-source-bundle
+```
+
+Inspect one configured source bundle:
+
+```sh
+go run ./cmd/distributor inspect \
+  --config <config-path> \
+  --pipeline <pipeline-id> \
+  --bundle daily/2026-06-01
+```
+
+Create a manifest for a local producer bundle:
+
+```sh
+go run ./cmd/distributor manifest create <bundle-path> --id <bundle-id>
+```
+
+Create a manifest with explicit file order:
+
+```sh
+go run ./cmd/distributor manifest create <bundle-path> \
+  --id <bundle-id> \
+  --created 2026-06-01T11:00:00Z \
+  --file report.md \
+  --file summary.txt
+```
+
 Preview local publication without writing:
 
 ```sh
@@ -74,26 +133,103 @@ Publish the local HTML example:
 go run ./cmd/distributor run --config examples/local-html.yml
 ```
 
+Start the HTTP upload API:
+
+```sh
+DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN=<token> \
+  go run ./cmd/distributor serve --config examples/http-upload-local.yml
+```
+
+Upload an archive to the configured `http_upload` pipeline associated with a bearer token:
+
+```sh
+curl -X POST http://127.0.0.1:8080/upload \
+  -H "Authorization: Bearer $DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN" \
+  -H "Content-Type: application/gzip" \
+  --data-binary @bundle.tar.gz
+```
+
+The upload response is accepted asynchronously:
+
+```json
+{"run_id":"reports.20260603T120000Z.abcdef12","status":"accepted"}
+```
+
+Check upload status:
+
+```sh
+curl http://127.0.0.1:8080/runs/<run-id>
+```
+
+Check server readiness:
+
+```sh
+curl http://127.0.0.1:8080/healthz
+```
+
 Preview local fan-out publication:
 
 ```sh
 go run ./cmd/distributor run --config examples/fan-out.yml --dry-run
 ```
 
+Preview local archive-plus-latest publication:
+
+```sh
+go run ./cmd/distributor run --config examples/archive-and-latest.yml --dry-run
+```
+
+Preview a forced replacement before publishing:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --dry-run --force
+```
+
 ## Output
 
-`run` prints the number of configured pipelines, one line per pipeline, one line per planned destination action, and a final status line. Actions include:
+Text output is the default and is intended for humans.
+
+`run` text output prints the number of configured pipelines, one line per pipeline, one line per planned destination action, and a final status line. Destination action lines include the source bundle path, destination id, destination backend, action, outputs, and reason. Fixed path destinations also print `path_mapping=fixed target=.` to show that the selected bundle targets the destination backend root. Actions include:
 
 - `publish_new`: destination has no managed state and is empty.
 - `replace_older`: destination state is older than the source manifest.
+- `force_replace`: `--force` requested a supported destructive replacement.
 - `skip_same`: destination state already matches the source manifest.
 - `skip_destination_newer`: destination state is newer than the source manifest.
 - `error`: planning or execution failed for that destination.
 
 The command exits non-zero if any destination fails. Independent later destinations are still attempted.
 
+Dry-run output for fixed path destinations prints a warning with the candidate count and selected source bundle. If a fixed path dry run plans a destructive replacement, it prints an additional warning that the destination root would be replaced.
+
+The final status line includes counters for `publish_new`, `replace_older`, `force_replace`, skipped destinations, failures, whether the run was a dry run, and fixed path destinations.
+
+JSON output writes exactly one JSON document to stdout:
+
+```json
+{
+  "schema_version": 1,
+  "command": "inspect",
+  "ok": true,
+  "warnings": [],
+  "result": {}
+}
+```
+
+Warnings are objects in the top-level `warnings` array and are not printed again as text. Fatal setup errors, such as a missing config file or invalid arguments, write no JSON document and return a non-zero exit code with a text error on stderr.
+
+`run --format json` returns partial results when destination failures occur after planning or execution begins. In that case stdout contains `ok: false`, a `result` with pipeline summaries, destination actions, final counters, and a top-level `errors` array; the process still exits non-zero.
+
+Command-specific JSON results:
+
+- `version`: application name and version.
+- `validate`: bundle count and discovered bundle identifiers. Configured source results also include pipeline id and source backend.
+- `inspect`: bundle path, id, created timestamp, digest, file count, total size, and manifest file records. Configured source results also include pipeline id and source backend.
+- `manifest create`: manifest path, bundle root, id, created timestamp, digest, file count, and file records.
+- `run`: dry-run status, pipeline summaries, destination action records, destination bundle paths, path mapping markers, optional primary URLs, output records with optional URLs, final counters, warnings, and partial failure records.
+
 ## Diagnostics
 
-Use `validate` before publication when a producer has written a new bundle. Use `inspect` to confirm normalized ids, timestamps, digests, file paths, and file sizes.
+Use `manifest create` when a local producer has written bundle files but not `manifest.json`. Use `validate` before publication when a producer has written a new bundle; use configured source mode when the bundle is already on an SSH or S3 source. Use `inspect` to confirm normalized ids, timestamps, digests, file paths, and file sizes.
 
 For symptom-oriented recovery steps, see [troubleshooting](troubleshooting.md). For destination state and retry behavior, see [operations](operations.md). For config fields and defaults, see [configuration](config.md).

docs/config.md

@@ -1,16 +1,19 @@
-# Distributor Configuration
+# Configuration Reference
 
 ## Config File Location
 
-`distributor run --config <path>` loads the YAML config at the provided path.
+`distributor run --config <path>` and `distributor serve --config <path>` load
+the YAML config at the provided path.
 
-If `--config` is omitted, `run` uses:
+If `--config` is omitted, both commands use:
 
 ```text
 /usr/local/etc/distributor/config.yml
 ```
 
-Config parsing rejects unknown YAML fields. The current executable backend support is local only. SSH and S3 config fields are accepted by config validation, but runtime execution for those backends is unavailable.
+Config parsing rejects unknown YAML fields. The executable `run` backends are
+`local`, `ssh`, and `s3`. The `serve` command executes `http_upload` sources
+through the HTTP upload API and normal destination fan-out.
 
 ## Minimal Local Config
 
@@ -31,6 +34,14 @@ This publishes source files only. It uses the default validation and transfer po
 ## Production-Oriented Local Config
 
 ```yaml
+server:
+  http:
+    bind: 127.0.0.1:8080
+    staging_root: /var/spool/distributor
+    max_upload_size: 20MB
+    queue_size: 16
+    max_concurrency: 1
+    retention: 24h
 pipelines:
   - id: reports
     source:
@@ -52,9 +63,87 @@ pipelines:
           on_conflict: fail
 ```
 
+## HTTP Upload Source Configuration
+
+HTTP upload sources are configured as pipeline sources only. They are not valid
+destination backends. `distributor serve` maps each configured upload token to
+exactly one `http_upload` pipeline.
+
+```yaml
+server:
+  http:
+    bind: 127.0.0.1:8080
+    staging_root: /var/spool/distributor
+    max_upload_size: 20MB
+    queue_size: 16
+    max_concurrency: 1
+    retention: 24h
+pipelines:
+  - id: weather-daily
+    source:
+      backend: http_upload
+      token_env: WEATHER_DAILY_UPLOAD_TOKEN
+      staging_path: /var/spool/distributor/weather-daily
+      max_upload_size: 20MB
+    destinations:
+      - id: archive
+        backend: local
+        path: /srv/reports/archive
+```
+
+`source.token_env` is required and names the environment variable or `secrets.directory` file that provides the bearer token. Literal upload tokens are not supported in YAML.
+
+`source.staging_path` is optional. When omitted, it defaults to `<server.http.staging_root>/<pipeline id>`.
+
+`source.max_upload_size` is optional. When omitted, it defaults to `server.http.max_upload_size`.
+
+The server resolves each `token_env` through the real process environment and
+the configured `secrets.directory` resolver. Startup fails if any configured
+upload token is missing, empty, or resolves to the same value as another upload
+pipeline. Token values are not read from YAML and are not printed in API
+responses.
+
+## HTTP Upload API
+
+`distributor serve` binds to `server.http.bind`, which defaults to
+`127.0.0.1:8080`.
+
+Routes:
+
+- `GET /healthz`: returns readiness status after config and upload tokens load.
+- `POST /upload`: accepts one tar or tar.gz source bundle archive.
+- `GET /runs/<run_id>`: returns an in-memory upload status record, or `404` if the run id is unknown or expired.
+
+`POST /upload` authenticates with:
+
+```text
+Authorization: Bearer <token>
+```
+
+The token selects the configured `http_upload` pipeline. Producers do not send a
+pipeline id. Requests with a submitted `pipeline` or `pipeline_id` query value
+are rejected.
+
+Accepted upload content types:
+
+- `application/x-tar`
+- `application/gzip`
+- `application/x-gzip`
+
+Accepted uploads return:
+
+```json
+{"run_id":"<id>","status":"accepted"}
+```
+
+The run id can be queried through `GET /runs/<run_id>` while the status record
+is retained in memory. Completed records expire after `server.http.retention`;
+expiration also removes committed staged bundle directories for completed
+uploads.
+
 ## HTML Publication
 
-To publish generated HTML from Markdown files:
+To publish generated sidecar HTML from Markdown files:
 
 ```yaml
 publish:
@@ -68,10 +157,88 @@ transform:
 
 Sidecar generation writes `report.html` for `report.md`. It does not mutate the source bundle.
 
+To publish a single Markdown file as `index.html`:
+
+```yaml
+publish:
+  source: false
+  html: true
+transform:
+  markdown_to_html:
+    enabled: true
+    mode: index
+    input: report.md
+```
+
+When `mode: index` omits `input`, the source manifest must list exactly one Markdown file.
+
+## Destination Path Mapping
+
+Each destination chooses how source bundle paths map into that destination:
+
+```yaml
+path_mapping:
+  mode: preserve_relative
+```
+
+`preserve_relative` is the default. It publishes each discovered source bundle at the same path relative to the destination backend root. A source bundle at `daily/2026-06-01` publishes below `daily/2026-06-01` for that destination.
+
+`fixed` publishes one selected source bundle directly at the destination backend root:
+
+```yaml
+destinations:
+  - id: latest-html
+    backend: local
+    path: /srv/www/reports/latest
+    path_mapping:
+      mode: fixed
+    publish:
+      source: false
+      html: true
+    transform:
+      markdown_to_html:
+        enabled: true
+        mode: index
+        input: report.md
+```
+
+Fixed destinations select the newest discovered source bundle by manifest `created` timestamp. If multiple candidates have the same timestamp, the source-root-relative bundle path in ascending order wins. Older candidates are not planned or written for that destination.
+
+Fixed mapping is useful for stable latest-style paths. It is more destructive than archive-style publication because successive source bundles target the same destination root. Preview fixed destinations with `run --dry-run`, especially before using `--force`.
+
+## Destination Links
+
+Destinations can record public URLs for published outputs:
+
+```yaml
+links:
+  base_url: https://reports.example.com/archive
+  primary: auto
+```
+
+`links.base_url` is an absolute `http` or `https` URL corresponding to the destination backend root. It may include a path prefix, but it must not include a query string or fragment. Distributor does not infer public URLs from backend config.
+
+`links.primary` selects the top-level primary URL stored in destination state:
+
+- `auto`: prefer `index.html`, then generated HTML, then source outputs.
+- `html`: use the first generated HTML output.
+- `source`: use the first copied source output.
+
+If a destination has no `links` block, no URL metadata is generated. If a primary policy has no matching output, per-output URLs are still recorded and the top-level primary URL is omitted.
+
+Output URLs are built from `links.base_url`, the destination bundle path, and the output path using URL path semantics. `index.html` outputs produce directory-style URLs that omit the filename.
+
 ## Reference
 
 Top level:
 
+- `server.http.bind`: optional HTTP bind address; defaults to `127.0.0.1:8080`.
+- `server.http.staging_root`: optional root for default HTTP upload staging paths; defaults to `/var/spool/distributor`.
+- `server.http.max_upload_size`: optional default upload size limit; defaults to `20MB`.
+- `server.http.queue_size`: optional HTTP upload admission queue size; defaults to `16`.
+- `server.http.max_concurrency`: optional HTTP upload worker concurrency; defaults to `1`.
+- `server.http.retention`: optional completed upload retention duration; defaults to `24h`.
+- `secrets.directory`: optional credential secrets directory.
 - `pipelines`: required non-empty list.
 
 Pipeline:
@@ -85,14 +252,22 @@ Source backend:
 
 - `backend`: required.
 - `path`: required for `local` and `ssh`.
-- `uri`: required for `ssh`.
+- `host`: required for `ssh`.
+- `user`: optional for `ssh`; defaults to the current OS user when available.
+- `port`: optional for `ssh`; defaults to `22`.
+- `ssh_key_file`: optional for `ssh`.
+- `known_hosts`: optional for `ssh`; defaults to the service user's OpenSSH `known_hosts` path when available.
+- `host_key_policy`: optional for `ssh`; defaults to `accept-new`.
 - `endpoint`: required for `s3`.
 - `bucket`: required for `s3`.
-- `prefix`: optional for `s3`.
-- `region`: optional for `s3`.
-- `force_path_style`: optional for `s3`.
+- `prefix`: optional for `s3`; leading and trailing slashes are trimmed.
+- `region`: optional for `s3`; defaults to `us-east-1`.
+- `force_path_style`: optional for `s3`; defaults to `true`. Set `false` only for services that require virtual-host addressing.
 - `credentials.access_key_id_env`: optional S3 credential environment variable name.
 - `credentials.secret_access_key_env`: optional S3 credential environment variable name.
+- `token_env`: required for `http_upload`; names the token environment variable or secret-file name.
+- `staging_path`: optional for `http_upload`; defaults below `server.http.staging_root` using the pipeline id.
+- `max_upload_size`: optional for `http_upload`; defaults to `server.http.max_upload_size`.
 
 Destination:
 
@@ -100,35 +275,122 @@ Destination:
 - Backend fields: same accepted shape as source backends, with destination fields at the destination level.
 - `publish`: optional; defaults to source-only publication.
 - `transform`: required only for generated HTML publication.
+- `path_mapping.mode`: optional; defaults to `preserve_relative`. Accepted values are `preserve_relative` and `fixed`.
+- `links.base_url`: optional links block; when present, `base_url` is required and must be an absolute HTTP or HTTPS URL without query string or fragment.
+- `links.primary`: optional; defaults to `auto`. Accepted values are `auto`, `html`, and `source`.
 - `transfer`: optional; defaults described below.
 
 Accepted backend names:
 
 - `local`: executable; requires `path`.
-- `ssh`: config validation only; execution is unavailable.
-- `s3`: config validation only; execution is unavailable.
+- `ssh`: executable; requires `host` and `path`.
+- `s3`: executable; requires `endpoint` and `bucket`.
+- `http_upload`: source-only configuration; requires `token_env`.
+
+## Size And Duration Values
+
+Upload size fields use an integer plus one of the supported binary-size suffixes:
+
+- `B`
+- `KB`
+- `MB`
+- `GB`
+
+Suffix multipliers use powers of 1024. Size values must be greater than zero after defaults are applied.
+
+HTTP retention uses Go-style duration strings such as `24h`, `90m`, or `168h`. Retention must be greater than zero after defaults are applied.
+
+## SSH Backend
+
+SSH uses native SFTP. It can be used for sources, destinations, or both:
+
+```yaml
+backend: ssh
+host: example.com
+user: distributor
+port: 2222
+path: /remote/root
+ssh_key_file: /home/distributor/.ssh/id_ed25519
+known_hosts: /home/distributor/.ssh/known_hosts
+host_key_policy: accept-new
+```
+
+Authentication uses SSH agent identities first when `SSH_AUTH_SOCK` is set, then `ssh_key_file` if configured. Password authentication in YAML is not supported.
+
+Host key policies:
+
+- `strict`, `true`, and `"true"` require a matching known host key.
+- `accept-new` accepts and persists a new host key, but fails if an existing key changed. During `run --dry-run`, new host keys are accepted only for the current connection and are not persisted.
+- `off`, `false`, and `"false"` disable host key checking and are insecure.
+
+`accept-new` and `strict` use `known_hosts` when configured. If omitted, distributor uses the current service user's default OpenSSH `known_hosts` path where practical. `accept-new` fails when it needs to persist a new host key and no writable `known_hosts` path is available. It does not create a missing parent `.ssh` directory.
+
+## S3 Backend
+
+S3 uses the AWS SDK for Go v2 and supports S3-compatible endpoints:
+
+```yaml
+backend: s3
+endpoint: https://s3.example.com
+bucket: reports
+prefix: archive
+region: us-east-1
+force_path_style: true
+credentials:
+  access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
+  secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
+```
+
+`endpoint` and `bucket` are required. `prefix` is an optional backend root; it is treated as an object-key prefix, not a real directory. Prefixes must be clean slash-separated paths after trimming leading and trailing slashes. `http://` endpoints are allowed for explicitly configured local development or local S3-compatible test services.
+
+If either credential environment variable name is configured, both must be configured and both referenced variables must resolve to non-empty values through the real process environment or `secrets.directory`. Explicit credentials take precedence over the AWS SDK default credential chain. If credential environment variable names are omitted, the SDK default credential chain is used and `secrets.directory` values are not injected into the process environment.
 
 Publish policy:
 
 - `publish.source`: publish source artifacts.
 - `publish.html`: publish generated HTML artifacts from Markdown source files.
 
-At least one output type must be enabled. When `publish.html` is true, `transform.markdown_to_html.enabled` must be `true` and `transform.markdown_to_html.mode` must be `sidecar`.
+At least one output type must be enabled. When `publish.html` is true, `transform.markdown_to_html.enabled` must be `true`.
+
+Markdown-to-HTML transform:
+
+- `transform.markdown_to_html.enabled`: enables Markdown-to-HTML generation for destinations with `publish.html: true`.
+- `transform.markdown_to_html.mode`: optional; defaults to `sidecar`. Accepted values are `sidecar` and `index`.
+- `transform.markdown_to_html.input`: optional source manifest path for `index` mode. It must identify a listed Markdown file.
+
+`sidecar` mode renders each manifest-listed `.md` file to a same-directory `.html` output. `index` mode renders one selected Markdown file to `index.html` at the destination bundle path. Enabled Markdown-to-HTML config is rejected when `publish.html` is false, and `input` is valid only with `mode: index`.
 
 Transfer policy:
 
 - `transfer.on_destination_same`: `skip` or `fail`; defaults to `skip`.
 - `transfer.on_destination_older`: `replace` or `fail`; defaults to `replace`.
-- `transfer.on_destination_newer`: `skip` or `fail`; defaults to `skip`.
-- `transfer.on_conflict`: only `fail`; defaults to `fail`.
+- `transfer.on_destination_newer`: `skip`, `replace`, or `fail`; defaults to `skip`.
+- `transfer.on_conflict`: `fail` or `replace`; defaults to `fail`.
+
+`replace` for `on_destination_newer` and `on_conflict` is honored only when `run --force` is used for that invocation. Force is CLI-only; there is no persistent config field that enables forced replacement by default.
 
 ## Defaults
 
 Defaults are applied after YAML decoding and before validation:
 
 - `validation.on_digest_mismatch: fail`
+- SSH `port: 22`
+- SSH `host_key_policy: accept-new`
+- S3 `region: us-east-1`
+- S3 `force_path_style: true`
+- `server.http.bind: 127.0.0.1:8080`
+- `server.http.staging_root: /var/spool/distributor`
+- `server.http.max_upload_size: 20MB`
+- `server.http.queue_size: 16`
+- `server.http.max_concurrency: 1`
+- `server.http.retention: 24h`
+- `source.staging_path: /var/spool/distributor/<pipeline id>` for `http_upload`
+- `source.max_upload_size: server.http.max_upload_size` for `http_upload`
+- `transform.markdown_to_html.mode: sidecar` when a Markdown-to-HTML transform block is present and mode is omitted
 - `publish.source: true`
 - `publish.html: false`
+- `path_mapping.mode: preserve_relative`
+- `links.primary: auto` when a `links` block is present and `primary` is omitted
 - `transfer.on_destination_same: skip`
 - `transfer.on_destination_older: replace`
 - `transfer.on_destination_newer: skip`
@@ -136,12 +398,25 @@ Defaults are applied after YAML decoding and before validation:
 
 ## Secrets
 
-Do not put literal secrets in config files. S3 credentials may name environment variables:
+Do not put literal secrets in config files. `secrets.directory` lets deployments provide credential values as files:
+
+```yaml
+secrets:
+  directory: /run/secrets/distributor
+```
+
+Each regular file in the directory becomes an internal credential environment value named by the filename. Valid filenames must match `[A-Za-z_][A-Za-z0-9_]*`. Directories are ignored, and symlinks to regular files are followed. Exactly one trailing LF or CRLF is trimmed from each file; other whitespace is preserved.
+
+The resolver checks the real process environment first, then the secrets directory. If both define the same variable with different values, `run` prints a warning with the variable name and uses the real environment value. Secret values are not printed. The process environment is not modified, so SDK default credential chains see only real environment variables.
+
+S3 credentials may name environment variables:
 
 - `credentials.access_key_id_env`
 - `credentials.secret_access_key_env`
 
-S3 execution is unavailable; these fields are accepted so config shape can be validated.
+HTTP upload tokens name one environment variable or secret-file name:
+
+- `source.token_env`
 
 ## Examples
 
@@ -150,4 +425,9 @@ Maintained examples live under [examples](../examples/):
 - `local-to-local.yml`: minimal local config.
 - `local-publish.yml`: runnable local source publication.
 - `local-html.yml`: runnable local HTML publication.
+- `local-index.yml`: runnable local `index.html` publication.
 - `fan-out.yml`: runnable local fan-out publication to source and HTML destinations.
+- `archive-and-latest.yml`: runnable local fan-out publication to an archive destination and a fixed latest destination.
+- `http-upload-local.yml`: local HTTP upload server example with a token environment variable reference.
+- `ssh-destination.yml`: environment-gated local-to-SSH publication example.
+- `s3-destination.yml`: environment-gated local-to-S3 publication example.

docs/integrations/markdown.md

@@ -12,7 +12,12 @@ Rendering uses `github.com/yuin/goldmark`. The exact dependency version is pinne
 
 `internal/transform/markdown.New` constructs the renderer with `goldmark.New()` and no project-specific extensions or renderer options.
 
-For each source bundle file ending in `.md`, the transform reads the Markdown source and generates an HTML sidecar in the same logical directory. The output path replaces the `.md` suffix with `.html`, so `report.md` produces `report.html`. Non-Markdown source files produce no Markdown outputs.
+The transform supports two output modes:
+
+- `sidecar`: reads each source bundle file ending in `.md` and generates an HTML sidecar in the same logical directory. The output path replaces the `.md` suffix with `.html`, so `report.md` produces `report.html`. Non-Markdown source files produce no Markdown outputs.
+- `index`: renders one selected Markdown source to `index.html` at the destination bundle path.
+
+In `index` mode, `transform.markdown_to_html.input` can name the source manifest path to render. If `input` is omitted, the manifest must list exactly one Markdown file. The selected input must be a safe relative source path, must be listed in the source manifest, and must end in `.md`.
 
 Raw HTML embedded in Markdown is not passed through by the current renderer behavior. Tests allow Goldmark's disabled-or-escaped raw HTML output forms and reject literal script tags in generated HTML.
 
@@ -42,7 +47,7 @@ Generated outputs record:
 
 Markdown rendering does not mutate source bundles, publish files, write `.distributor.json`, select outputs, or choose transfer actions. Publish planning decides whether generated HTML is selected for a destination.
 
-Only sidecar output mode is supported for current behavior.
+Publish planning chooses the configured mode and input for each destination. Markdown rendering does not inspect destinations, publish files, write `.distributor.json`, or choose transfer actions.
 
 ## Tests
 
@@ -52,4 +57,4 @@ Before changing Markdown renderer behavior, inspect and run:
 go test ./internal/transform/markdown
 ```
 
-The tests cover sidecar naming, ignored non-Markdown files, raw HTML handling, deterministic output, digest metadata, and size metadata.
+The tests cover sidecar naming, index input selection, ignored non-Markdown files, raw HTML handling, deterministic output, digest metadata, and size metadata.

docs/internal/app.md

@@ -2,50 +2,232 @@
 
 ## Purpose
 
-`internal/app` owns top-level use cases for `run`, `validate`, and `inspect`. It wires configuration, storage backends, transforms, publish planning, execution, summaries, and notification handoff.
+`internal/app` owns the top-level application use cases. It coordinates
+configuration loading, secret resolution, backend construction, source bundle
+discovery, destination selection, publish planning, publish execution,
+notification handoff, run reporting, and in-memory run coordination.
 
-## Inputs and outputs
+The package is the boundary between callers and lower-level domain packages. It
+does not own manifest validation rules, destination state comparison, storage
+path rules, output planning, transform rendering, or backend-specific behavior.
 
-`Run` accepts a context, optional config path, dry-run flag, stdout writer, and optional notifier. It loads YAML config, discovers source bundles for each configured pipeline, plans each destination independently, optionally executes publish plans, writes summary output when stdout is supplied, and returns an aggregated error if any destination fails.
+## Use Cases
 
-`Validate` and `Inspect` accept a local path. `Validate` discovers and validates bundles. `Inspect` writes bundle metadata and manifest file entries to stdout when provided.
+`Run` is the CLI-facing all-pipeline entrypoint. It accepts a context, optional
+config path, dry-run flag, force flag, stdout writer, output format, and
+optional notifier. It runs every configured pipeline, builds a `RunReport`, and
+projects the report to text or JSON when stdout is supplied.
 
-## Run flow
+`RunPipeline` is the app-layer single-pipeline entrypoint. It accepts a context,
+config path, pipeline ID, dry-run flag, force flag, and optional notifier. It
+loads the same config as `Run`, narrows execution to exactly one configured
+pipeline, and returns a `RunReport` without writing command output.
 
-The runner:
+`RunPipelineWithLocalSource` is the app-layer single-pipeline entrypoint for an
+already prepared local source bundle root. It accepts the same pipeline
+selection and execution options as `RunPipeline` plus a local source root path.
+It loads config, selects one configured pipeline, opens the supplied source
+root as a local backend, validates exactly that root bundle, and then uses the
+same destination fan-out path as normal runs.
+
+`Validate` and `Inspect` accept either a local path or one configured pipeline
+source. They share source backend construction with run workflows and never open
+destination backends.
+
+`Serve` is the CLI-facing HTTP upload server entrypoint. It loads config,
+loads the configured secrets directory, resolves upload bearer tokens for
+configured `http_upload` sources, creates an `UploadCoordinator`, binds
+`server.http.bind`, and serves the upload API until its context is cancelled.
+
+## Run Reports
+
+`RunReport` is the structured result model for run workflows. It includes
+dry-run state, pipeline summaries, action records, output metadata, summary
+counters, warnings, and destination-scoped output errors.
+
+Text and JSON run output are projections of `RunReport`. JSON tags on report
+records match the CLI JSON output contract. Text output preserves the CLI
+summary shape while keeping output rendering outside the core planning and
+execution loop.
+
+Destination-scoped failures produce a report plus an aggregated error. Fatal
+setup failures, such as config loading, source open, or source discovery
+failures, return before a complete run report is available.
+
+## Run Flow
+
+The app runner:
 
 1. loads config from the supplied path or `config.DefaultConfigPath`;
-2. opens the configured source backend;
-3. discovers validated bundles from the source root;
-4. opens each destination backend independently;
-5. builds a publish plan for each bundle and destination;
-6. prints plan lines and records summary counters;
-7. executes publish or replacement plans unless dry-run is enabled;
-8. invokes the notifier after successful publish or replacement actions.
+2. loads configured secret files into a config-owned environment resolver;
+3. builds the app-level backend factory and transform registry;
+4. opens each selected pipeline source backend;
+5. discovers validated source bundles from the source root;
+6. selects source bundles for each destination according to path mapping;
+7. opens destination backends independently;
+8. builds publish plans for selected bundle and destination combinations;
+9. records warnings, action records, output metadata, and summary counters;
+10. executes publish or replacement plans unless dry-run is enabled;
+11. invokes the notifier after successful publish or replacement actions;
+12. returns the structured report and any aggregated destination failures.
 
-Destination failures are collected while later destinations continue to run. Source open and source discovery failures stop the run because there are no valid bundles to fan out.
+`RunPipeline` follows the same flow after selecting a single configured
+pipeline. It uses the same backend factory, secret loading, transform registry,
+warning generation, destination planning, publish execution, notification
+behavior, and failure aggregation as `Run`.
 
-## Backend and transform wiring
+`RunPipelineWithLocalSource` follows the same flow after pipeline selection
+except for source opening and source discovery. It opens the supplied local
+source root directly, validates the root bundle before opening any destinations,
+and passes the resulting local source backend and bundle into the same
+destination planning and execution loop. Destination code receives the normal
+storage backend and bundle values and does not depend on how the source root was
+prepared.
 
-The app-level backend factory registers only the local backend for execution. Config validation accepts other backend shapes, but `Run` can execute only local sources and local destinations.
+## Upload Coordination
 
-The app-level transform registry registers Markdown-to-HTML using `internal/transform/markdown`. Lower-level publish code receives a resolver and does not import concrete transform implementations.
+`UploadCoordinator` owns in-memory coordination for asynchronous upload
+processing. It admits uploads for configured `http_upload` pipelines, generates
+run IDs, tracks status records, stages accepted archives through
+`internal/ingest`, and executes the selected pipeline through
+`RunPipelineWithLocalSource`.
 
-## Dry-run behavior
+Upload run IDs use:
 
-Dry-run still loads config, opens backends, discovers bundles, inspects destinations, resolves transforms, and builds publish plans. It does not write destination outputs, write `.distributor.json`, delete managed outputs, or notify.
+```text
+<pipeline id>.<UTC timestamp>.<random suffix>
+```
 
-## Failure behavior
+The timestamp uses `YYYYMMDDThhmmssZ` UTC format and the suffix is filesystem
+safe.
 
-`Run` returns immediately for config loading errors, context cancellation before work starts, source open errors, and source discovery errors. Per-destination backend, planning, execution, and notification errors are aggregated into one run error after remaining destinations have been attempted.
+The coordinator records these statuses:
 
-Stdout write errors are returned immediately because the caller's requested output stream can no longer be trusted.
+- `accepted`
+- `queued`
+- `running`
+- `succeeded`
+- `failed`
+- `expired`
 
-## Boundaries
+Admission is bounded by `server.http.queue_size`. Full queues are rejected
+before the upload body is staged. Execution is bounded by
+`server.http.max_concurrency`, and only one upload for a given pipeline may run
+at a time. Later uploads for the same pipeline remain queued until the active
+run finishes.
 
-`internal/app` coordinates packages but does not own manifest validation rules, destination state comparison, storage path rules, output planning, transform rendering, or backend-specific filesystem behavior.
+Completed records retain the final run report or error text until
+`server.http.retention` elapses. Expiration removes completed status records and
+their committed staged bundle directories. The coordinator is memory-only and
+does not persist queue state, status records, or run reports.
 
-`Validate` and `Inspect` are local path commands. Remote execution wiring is outside current behavior.
+## HTTP Upload Server
+
+The HTTP upload server is app-layer transport wiring around
+`UploadCoordinator`. It owns request authentication, route dispatch, HTTP status
+mapping, and JSON response projection. Bundle staging and publication remain in
+the coordinator and staged-source run path.
+
+Server startup resolves every configured `http_upload` source `token_env`
+through the config-owned environment resolver after `secrets.directory` has
+been loaded. Startup fails when a token is missing, empty, or duplicates another
+upload pipeline token. Error messages identify environment variable names and
+pipeline ids, but not token values.
+
+Routes:
+
+- `GET /healthz`: returns `200` after config, secrets, tokens, coordinator, and route setup succeed.
+- `POST /upload`: accepts authenticated tar and tar.gz archives and returns an accepted run id.
+- `GET /runs/<run_id>`: returns the current in-memory upload status record or `404`.
+
+The upload token maps to exactly one configured pipeline. Producers do not
+submit pipeline ids, and submitted `pipeline` or `pipeline_id` query values are
+rejected. Full queues are rejected before the request body is read. Oversized
+uploads, unsupported content types, invalid bearer tokens, full queues, and
+unknown status records are mapped to stable HTTP status codes without returning
+secret token values.
+
+## Coordination
+
+`PipelineRunCoordinator` wraps `RunPipeline` with in-memory admission control.
+It allows different pipeline IDs to run concurrently and rejects a second active
+run for the same pipeline ID.
+
+Coordinator records contain a run ID, pipeline ID, status, timestamps, completed
+report, and error text when applicable. Active state is memory-only and is
+cleared after success, failure, unknown pipeline ID, or context cancellation.
+
+The admission context is checked before a run is accepted. Once accepted, the
+run uses the coordinator lifetime context, so caller cancellation can stop
+waiting for admission without owning the actual run lifetime.
+
+The coordinator does not queue duplicate runs, persist run records, or define
+transport endpoints.
+
+## Errors
+
+`Run` returns immediately for config loading errors, context cancellation before
+work starts, source open errors, and source discovery errors.
+
+`RunPipeline` returns `PipelineNotFoundError` when the requested pipeline ID is
+not configured. Callers can detect that condition with `IsPipelineNotFound`.
+
+`RunPipelineWithLocalSource` also returns `PipelineNotFoundError` for an unknown
+pipeline ID. It returns before destination opening when the supplied local
+source root is missing, cannot be opened, or does not validate as one complete
+source bundle.
+
+Per-destination backend, planning, execution, and notification errors are
+aggregated into one run error after remaining destinations have been attempted.
+Destination diagnostics include pipeline ID, destination ID, backend, and
+bundle path.
+
+`PipelineRunCoordinator` returns `DuplicatePipelineRunError` when the same
+pipeline already has an active run. Callers can detect that condition with
+`IsDuplicatePipelineRun`.
+
+Stdout write errors are returned immediately because the caller's requested
+output stream can no longer be trusted.
+
+## Package Layout
+
+Run helpers are grouped by responsibility:
+
+- `run.go`: `Run`, `RunPipeline`, and shared run orchestration.
+- `run_output.go`: `RunReport`, action/output records, and text/JSON report projection.
+- `run_summary.go`: summary counters.
+- `run_failures.go`: destination failure aggregation and partial-result detection.
+- `run_selection.go`: destination bundle selection, path mapping decisions, and fixed-path warnings.
+- `run_warnings.go`: secret and SSH warning records.
+- `run_notify.go`: notification event projection and action filtering.
+- `run_coordinator.go`: in-memory run admission, run IDs, status records, and duplicate-run errors.
+- `upload_coordinator.go`: in-memory upload admission, queueing, status tracking, staging handoff, and staged-source execution.
+- `upload_http.go`: HTTP upload authentication, routes, JSON response projection, and HTTP error mapping.
+- `serve.go`: config/secrets loading and HTTP server startup.
+- `backends.go`: app-level backend factory wiring.
+- `transforms.go`: app-level transform registry wiring.
+- `source_select.go`: configured-source selection shared by `validate` and `inspect`.
+
+## Backend And Transform Wiring
+
+The app-level backend factory registers local, SSH, and S3 backends for runtime
+execution. Source and destination backend config is converted through a shared
+app-local open spec before adapter construction.
+
+Credential references are resolved through the config environment resolver.
+Production app code must not read backend credential environment variables
+directly.
+
+The app-level transform registry registers Markdown-to-HTML through
+`internal/transform/markdown`. Lower-level publish code receives a resolver and
+does not import concrete transform implementations.
+
+## Dry-Run Behavior
+
+Dry-run loads config, opens backends, discovers bundles, inspects destinations,
+resolves transforms, and builds publish plans. It does not write destination
+outputs, write `.distributor.json`, delete managed outputs, perform forced
+prefix deletion, or invoke notifications.
 
 ## Tests
 
@@ -55,10 +237,18 @@ Before changing app orchestration, inspect tests under:
 - `internal/cli`
 - `internal/publish`
 
+Use focused app tests for report structure, single-pipeline execution,
+coordinator admission, warning generation, notification behavior, and
+partial-result aggregation.
+
 ## Invariants
 
 - One source fans out to each destination independently.
 - Destination failures do not prevent later destinations from being planned.
+- Destination-scoped failures still produce a structured report plus an aggregated error.
 - Dry-run must not mutate destination storage or invoke notifications.
+- `RunPipeline` must use the same run path as `Run` after pipeline selection.
+- Duplicate in-flight runs are rejected only for the same pipeline ID.
+- Different pipeline IDs may run concurrently.
 - Concrete backend and transform registration stays at the app layer.
 - The default notifier is `notify.Noop`.

docs/internal/bundle.md

@@ -2,7 +2,7 @@
 
 ## Purpose
 
-`internal/bundle` parses, discovers, and validates source bundles through the storage interface.
+`internal/bundle` discovers and validates source bundles through the storage interface. The source manifest model, manifest parsing, manifest validation, path rules, digest calculation, and producer-side local writer come from `pkg/bundle` so producer-facing APIs and distributor validation share one manifest contract.
 
 ## Inputs and outputs
 
@@ -20,11 +20,13 @@ The source manifest requires:
 
 Each file requires `path`, `sha256`, and `size`. Digests must use lowercase `sha256:<64 hex>` format. `created` must parse as RFC3339.
 
+`pkg/bundle.ValidateDigest` is the canonical digest format validator for producer-facing and internal code. `internal/bundle.ValidateDigest` delegates to that public validator so source manifests and destination state use the same digest grammar.
+
 ## Validation
 
-`ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest.
+`pkg/bundle.ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest.
 
-Storage-backed bundle validation additionally checks file existence, regular-file type, file size, and per-file SHA-256.
+Storage-backed bundle validation in `internal/bundle` additionally checks file existence, regular-file type, file size, and per-file SHA-256 for configured storage backends.
 
 The bundle digest is SHA-256 of a deterministic JSON array of file records in manifest order with fields `path`, `sha256`, and `size`.
 
@@ -38,11 +40,11 @@ Manifest parsing and validation fail before destination planning. Storage-backed
 
 ## Boundaries
 
-Bundle code uses `internal/storage` and does not import concrete adapters. CLI local path support is wired in `internal/app`.
+Internal bundle discovery uses `internal/storage` and does not import concrete adapters. Producer-side local filesystem manifest building, complete bundle writing, and validation belong to `pkg/bundle`. CLI local path support is wired in `internal/app`.
 
 ## Tests
 
-Before changing bundle behavior, inspect tests under `internal/bundle`.
+Before changing bundle behavior, inspect tests under `pkg/bundle` and `internal/bundle`.
 
 ## Invariants
 

docs/internal/config.md

@@ -6,7 +6,7 @@
 
 ## Inputs and outputs
 
-Input is a YAML file containing `pipelines`. Output is a `Config` value with defaults applied and validation completed. Load failures include the config path and whether the failure occurred during file loading, YAML parsing, or validation.
+Input is a YAML file containing optional `server`, optional `secrets`, and required `pipelines`. Output is a `Config` value with defaults applied and validation completed. Load failures include the config path and whether the failure occurred during file loading, YAML parsing, or validation.
 
 ## Loading flow
 
@@ -14,12 +14,28 @@ Input is a YAML file containing `pipelines`. Output is a `Config` value with def
 
 Known-field checking rejects misspelled or unknown YAML keys before defaults and validation run.
 
+`LoadFile` does not read secret files. App entrypoints load the configured
+secrets directory after config validation and before credential-consuming work.
+
 ## Defaults
 
 Defaults are applied in `ApplyDefaults`:
 
+- HTTP server `bind` defaults to `127.0.0.1:8080`;
+- HTTP server `staging_root` defaults to `/var/spool/distributor`;
+- HTTP server `max_upload_size` defaults to `20MB`;
+- HTTP server `queue_size` defaults to `16`;
+- HTTP server `max_concurrency` defaults to `1`;
+- HTTP server `retention` defaults to `24h`;
+- `http_upload` source `staging_path` defaults to `<server.http.staging_root>/<pipeline id>`;
+- `http_upload` source `max_upload_size` defaults to `server.http.max_upload_size`;
 - pipeline validation defaults `on_digest_mismatch` to `fail`;
+- SSH backend `port` defaults to `22`;
+- SSH backend `host_key_policy` defaults to `accept-new`;
 - destination publish policy defaults to source output only;
+- Markdown-to-HTML mode defaults to `sidecar` when a transform block is present and mode is omitted;
+- destination path mapping defaults to `preserve_relative`;
+- destination link primary policy defaults to `auto` when a `links` block is present;
 - `transfer.on_destination_same` defaults to `skip`;
 - `transfer.on_destination_older` defaults to `replace`;
 - `transfer.on_destination_newer` defaults to `skip`;
@@ -27,13 +43,37 @@ Defaults are applied in `ApplyDefaults`:
 
 ## Validation responsibilities
 
-Validation requires at least one pipeline, slug-like unique pipeline ids, one source per pipeline, at least one destination, slug-like unique destination ids within each pipeline, backend-specific required fields, valid validation policy, valid publish and transform combinations, and valid transfer actions.
+Validation requires positive HTTP server limits and retention, at least one pipeline, slug-like unique pipeline ids, one source per pipeline, at least one destination, slug-like unique destination ids within each pipeline, backend-specific required fields, valid validation policy, valid publish and transform combinations, valid destination path mapping mode, valid destination link config, and valid transfer actions.
 
-`ValidatePublishTransformPolicy` is shared with publish planning so destination policy combinations are checked consistently. Publishing HTML requires an enabled Markdown-to-HTML transform in `sidecar` mode. A publish policy must select source output, HTML output, or both.
+HTTP upload sources require `token_env`, a staging path after defaults, and a positive maximum upload size. Literal token fields are not part of the YAML schema. The `http_upload` backend is accepted only for sources and rejected for destinations.
+
+Upload size values are parsed from strings with `B`, `KB`, `MB`, or `GB` suffixes using 1024 multipliers. Retention values are parsed with `time.ParseDuration`. Explicit zero values fail validation; omitted values receive defaults before validation.
+
+Transfer validation accepts `replace` for `on_destination_newer` and `on_conflict`, but publish planning honors those destructive actions only when the current run explicitly requests force.
+
+`ValidatePublishTransformPolicy` is shared with publish planning so destination policy combinations are checked consistently. Publishing HTML requires an enabled Markdown-to-HTML transform in `sidecar` or `index` mode. Enabled Markdown-to-HTML config is rejected when `publish.html` is false. `input` is accepted only for enabled `index` mode. A publish policy must select source output, HTML output, or both.
+
+Destination path mapping accepts `preserve_relative` and `fixed`. The app layer applies the mapping when it selects destination bundle paths; config owns only YAML shape, defaulting, and validation.
+
+Destination links are optional. When a `links` block is present, `base_url` is required, must use `http` or `https`, and must not include a query string or fragment. `primary` accepts `auto`, `html`, and `source`.
 
 ## Executable support boundary
 
-Config validation accepts `local`, `ssh`, and `s3` backend shapes so config files can be validated as schemas. Runtime execution currently opens only local backends through `internal/app`.
+Config validation accepts `local`, `ssh`, `s3`, and source-only `http_upload` backend shapes. Runtime `run`, `validate`, and `inspect` workflows open `local`, `ssh`, and `s3` through `internal/app`. Runtime `serve` workflows execute `http_upload` sources through the app upload coordinator and HTTP server.
+
+SSH config uses structured fields: `host`, optional `user`, optional `port`, `path`, optional `ssh_key_file`, optional `known_hosts`, and optional `host_key_policy`. `host_key_policy` accepts YAML booleans and strings and normalizes `true`/`strict`, `accept-new`, and `false`/`off`.
+
+S3 config requires `endpoint` and `bucket`, normalizes optional `prefix`, defaults `region` to `us-east-1`, and defaults omitted `force_path_style` to `true` while preserving explicit `false`.
+
+HTTP upload config is source-only. Config owns its YAML shape, defaulting, size and duration parsing, and validation. The config package does not authenticate requests, stage uploads, or execute HTTP upload sources. The app layer resolves `token_env` through the config-owned environment resolver before starting the HTTP server.
+
+## Secrets and credential resolution
+
+`secrets.directory` points to a directory of credential files. `LoadSecretEnvironment` reads regular files and symlinks to regular files, rejects invalid filenames, trims exactly one trailing LF or CRLF, and returns an `Environment` resolver plus conflict metadata.
+
+The resolver checks the real process environment first and loaded secret values second. Differing process/secret conflicts are reported by variable name only. The resolver does not mutate `os.Environ`; default SDK credential chains continue to see only real process environment values.
+
+Credential-consuming backend wiring should resolve explicit credential environment variable references through `Environment.ResolveCredentials` or the same resolver pattern instead of calling `os.Getenv` directly.
 
 The user-facing configuration reference is `docs/config.md`; this file documents package behavior for maintainers.
 

docs/internal/ingest.md

@@ -0,0 +1,43 @@
+# Ingestion Internals
+
+## Purpose
+
+`internal/ingest` stages uploaded source bundle archives into local per-run directories. It does not authenticate requests, manage upload queues, publish destinations, or start an HTTP server.
+
+## Archive staging
+
+`StageArchive` accepts one upload body, content type, pipeline staging path, run id, and explicit size and file-count limits. It writes the request body to temporary storage while enforcing the configured upload size limit, extracts the archive into temporary local storage, validates the extracted source bundle, and then commits the validated bundle to:
+
+```text
+<pipeline staging path>/<run id>
+```
+
+The returned `StagedBundle.Root` is a local filesystem path to the validated source bundle root.
+
+## Accepted archive formats
+
+The package accepts only:
+
+- `application/x-tar`
+- `application/gzip`
+- `application/x-gzip`
+
+Gzip uploads must contain a tar archive.
+
+## Extraction rules
+
+Archive entry paths must be clean relative slash-separated paths. Extraction rejects absolute paths, path traversal, backslash paths, duplicate files, symlinks, hardlinks, devices, sockets, and other special entries.
+
+The archive must contain exactly one root-level `manifest.json`. Nested manifests are rejected.
+
+Regular files and directories are the only accepted tar entries. Regular file extraction enforces the explicit maximum extracted byte count and maximum file count supplied by the caller.
+
+## Bundle validation
+
+After extraction, the package loads and validates the staged bundle through `pkg/bundle`. Manifest parsing, source path validation, file existence checks, regular-file checks, file sizes, file SHA-256 digests, and bundle digest validation use the existing source bundle contract.
+
+Validation happens before the staged bundle is committed to its final per-run path.
+
+## Failure behavior
+
+Failed staging removes temporary archive and extraction data created by the package. A failed call does not publish anything and does not leave a committed per-run bundle directory.

docs/internal/link.md

@@ -0,0 +1,27 @@
+# Link URL Policy
+
+## Purpose
+
+`internal/link` defines shared validation for configured and persisted HTTP link URLs.
+
+## Inputs and outputs
+
+Input is a URL string. Output is either nil for an accepted URL or a concise validation error that callers wrap with field context.
+
+## Validation behavior
+
+Accepted URLs must parse successfully, use `http` or `https`, include a host, and omit query strings and fragments.
+
+## Boundaries
+
+This package validates URL shape only. It does not construct destination output URLs, choose primary URLs, infer public URLs from backend configuration, or read configuration files.
+
+## Tests
+
+Before changing link URL policy, inspect tests under `internal/link` and callers in `internal/config`, `internal/state`, and `internal/publish`.
+
+## Invariants
+
+- Configured `links.base_url`, persisted `links.primary_url`, persisted output `url`, and publish link planning use the same URL policy.
+- Callers own field-specific error context.
+- URL path construction remains in `internal/publish`.

docs/internal/publish.md

@@ -6,38 +6,47 @@
 
 ## Inputs and outputs
 
-Inputs are a source bundle, source backend, destination backend, pipeline id, destination id, publish policy, transform policy, transformer resolver, transfer policy, destination bundle path, and existing destination state.
+Inputs are a source bundle, source backend, destination backend, pipeline id, destination id, publish policy, transform policy, optional link policy, transformer resolver, transfer policy, path mapping mode, destination bundle path, existing destination state, and whether explicit force was requested for the current run.
 
-Output is a plan with an action, reason, and selected source or generated outputs. Execution writes selected source files, generated files, and `.distributor.json` for publish or replacement actions.
+Output is a plan with an action, reason, optional primary URL, and selected source or generated outputs. Execution writes selected source files, generated files, and `.distributor.json` for publish or replacement actions.
 
 ## Actions
 
-Supported actions are `publish_new`, `replace_older`, `skip_same`, `skip_destination_newer`, `fail_conflict`, and `fail_unmanaged`.
+Supported actions are `publish_new`, `replace_older`, `force_replace`, `skip_same`, `skip_destination_newer`, `fail_conflict`, and `fail_unmanaged`.
 
 ## Failure behavior
 
-Planning fails when request fields are incomplete, publish and transform policies are invalid, selected outputs collide, HTML output is requested without Markdown inputs, destination state is invalid, destination content is unmanaged, or transfer policy maps the comparison outcome to failure.
+Planning fails when request fields are incomplete, publish and transform policies are invalid, selected outputs collide, HTML output is requested without Markdown inputs, destination state is invalid, destination content is unmanaged without force, or transfer policy maps the comparison outcome to failure.
 
 Execution fails if a write, delete, state serialization, or context check fails. Outputs written during a failed publish attempt are cleaned up through managed deletion where possible.
 
 ## Boundaries
 
-The current implementation publishes source files and Markdown-to-HTML sidecar outputs. Backend behavior is supplied through `internal/storage`; app runtime currently supplies local backends.
+The package publishes source files and Markdown-to-HTML outputs. Markdown sidecar mode writes same-directory `.html` outputs, and Markdown index mode writes `index.html`. Backend behavior is supplied through `internal/storage`; app runtime supplies local, SSH, and S3 backends.
 
-The package uses `internal/state` for destination comparison, `internal/storage` for IO, and the shared `internal/config` publish/transform policy helper for request validation. It resolves transforms through a narrow resolver supplied by the caller; concrete transform registration is owned by the app layer. It does not parse CLI flags or load config files.
+The package uses `internal/state` for destination comparison, `internal/storage` for IO, and the shared `internal/config` publish/transform policy helper for request validation. It resolves transforms through a narrow resolver supplied by the caller; concrete transform registration is owned by the app layer. It does not parse CLI flags, load config files, or choose which source bundles a destination receives.
+
+The package owns projection from planned publish outputs to destination state output records and managed destination output paths. App JSON results and notification events keep their own schemas, but may use the publish output projection to avoid field-mapping drift.
+
+The app layer computes the destination bundle path before planning. `preserve_relative` destinations pass the source-root-relative bundle path. `fixed` destinations pass an empty destination bundle path, which means the destination backend root, and pass only the newest selected source bundle for that destination.
+
+When link config is present, publish planning builds per-output URLs from `links.base_url`, the destination bundle path, and each output path. `index.html` outputs use directory-style URLs. The primary URL is selected from planned outputs according to the destination primary policy.
 
 ## Safety
 
-Replacement deletes only outputs recorded in existing destination state plus `.distributor.json`. Failed local writes trigger cleanup of outputs written during the failed attempt.
+Normal replacement deletes only outputs recorded in existing destination state plus `.distributor.json`. Forced replacement deletes the bounded destination bundle path before writing outputs and state. Failed writes trigger cleanup of outputs written during the failed attempt where practical.
 
 ## Tests
 
-Before changing publish behavior, inspect tests under `internal/publish` and local run tests under `internal/app`.
+Before changing publish behavior, inspect tests under `internal/publish` and run tests under `internal/app`.
 
 ## Invariants
 
 - Publish planning is deterministic for the same source, destination state, policies, and transform outputs.
-- Replacement deletes only managed paths recorded in existing state plus `.distributor.json`.
+- Destination bundle paths are caller-supplied and are interpreted relative to the destination backend root.
+- URL generation uses URL path semantics and does not infer public URLs from backend configuration.
+- Normal replacement deletes only managed paths recorded in existing state plus `.distributor.json`.
+- Forced replacement is explicit per run and deletes only within the destination bundle path.
 - Publish execution writes destination state after selected outputs are written.
 - Transform implementations are resolved through an interface supplied by the caller.
-- Unmanaged destination content is never overwritten.
+- Unmanaged destination content is overwritten only by explicit forced replacement.

docs/internal/state.md

@@ -19,13 +19,15 @@ Input is JSON destination state plus the current source manifest, pipeline id, d
 - `source.manifest`
 - `outputs`
 
-`distributor_version` is optional diagnostic metadata. `published_at` parses as RFC3339 and distributor-written state serializes it as RFC3339 UTC.
+`distributor_version` is optional diagnostic metadata. `links` is optional URL metadata. `published_at` parses as RFC3339 and distributor-written state serializes it as RFC3339 UTC.
 
 The embedded `source.manifest` is validated with the same source manifest rules used by `internal/bundle`.
 
 ## Outputs
 
-Each output records `path`, `kind`, `source_path`, `sha256`, and `size`. Supported output kinds are `source` and `generated`. Generated outputs require `transform`.
+Each output records `path`, `kind`, `source_path`, `sha256`, and `size`. Supported output kinds are `source` and `generated`. Generated outputs require `transform`. Outputs may record `url` when the destination has link generation configured.
+
+The optional top-level `links.primary_url` records the selected primary URL for the published destination bundle. It is omitted when link generation is not configured or when the destination primary policy has no matching output.
 
 ## Comparison
 
@@ -33,7 +35,7 @@ Comparison outcomes cover absent destination state, unmanaged destination conten
 
 ## Failure behavior
 
-Invalid JSON, invalid state schema, invalid embedded source manifests, unsafe output paths, unsupported output kinds, missing generated-output transform names, and mismatched pipeline or destination ids produce comparison outcomes that publish planning can turn into fail actions.
+Invalid JSON, invalid state schema, invalid embedded source manifests, unsafe output paths, invalid stored URLs, unsupported output kinds, missing generated-output transform names, and mismatched pipeline or destination ids produce comparison outcomes that publish planning can turn into fail actions. Supported identity and source-manifest conflicts can become forced replacement only when publish planning receives explicit force and compatible transfer policy.
 
 ## Boundaries
 
@@ -48,5 +50,6 @@ Before changing destination state behavior, inspect tests under `internal/state`
 - `.distributor.json` is the destination sentinel and state record.
 - Embedded source manifests use the same validation rules as source bundles.
 - Generated outputs always record a transform id.
+- Stored URLs are optional and must be absolute HTTP or HTTPS URLs when present.
 - Comparison returns outcomes and reasons; it does not mutate storage.
 - `distributor_version` is diagnostic metadata, not a comparison key.

docs/internal/storage.md

@@ -6,7 +6,7 @@
 
 ## Inputs and outputs
 
-The storage interface supports byte reads, stream reads, byte writes, stream writes, exact metadata lookup, traversal, destination emptiness checks, and guarded managed deletion.
+The storage interface supports byte reads, stream reads, byte writes, stream writes, exact metadata lookup, traversal, destination emptiness checks, guarded managed deletion, and bounded prefix deletion for explicit forced replacement.
 
 Entries report a logical path, type, and size when available. Entry types are `file`, `directory`, `symlink`, and `other`.
 
@@ -14,7 +14,7 @@ Entries report a logical path, type, and size when available. Entry types are `f
 
 Core packages should depend on `internal/storage`, not adapter packages. Adapter-specific path handling stays behind backend implementations.
 
-The local adapter lives in `internal/adapters/local`. Runtime backend construction is wired through the app-level backend factory and storage registry. The fake backend lives in `internal/storage/fake` for tests and is not registered for runtime use.
+The local adapter lives in `internal/adapters/local`. The SSH/SFTP adapter lives in `internal/adapters/ssh`. The S3-compatible adapter lives in `internal/adapters/s3`. Runtime backend construction is wired through the app-level backend factory and storage registry. The fake backend lives in `internal/storage/fake` for tests and is not registered for runtime use.
 
 ## Paths
 
@@ -26,14 +26,33 @@ Storage errors use typed categories such as not found, already exists, invalid p
 
 Backends may wrap implementation-specific errors, but callers should receive storage errors where practical. Traversal can stop cleanly with `ErrStopWalk`.
 
+## Traversal helpers
+
+Backends own their traversal mechanics. The local adapter owns filesystem walking, the SSH adapter owns SFTP directory walking, and the S3 adapter owns object listing and pagination.
+
+`internal/storage` owns the shared callback emission rules used by backends:
+
+- context cancellation is checked before callback emission;
+- `WalkOptions.Limit` bounds the number of emitted entries;
+- `ErrStopWalk` stops traversal without becoming a caller-visible error;
+- callback errors are wrapped as storage walk errors.
+
+`storage.HasAny(ctx, backend, prefix)` provides the shared destination-content check. It calls `Walk` with non-recursive, limit-one traversal and stops after the first emitted entry.
+
 ## Deletion
 
-Backends expose guarded managed deletion only. `DeleteManagedBundle` may delete listed managed outputs plus `.distributor.json`; it does not provide broad recursive deletion.
+`DeleteManagedBundle` may delete listed managed outputs plus `.distributor.json`.
 
-## Local and fake backends
+`DeletePrefix` removes content at and below a logical prefix for explicit forced replacement. It must not delete above the requested prefix or above the configured backend root.
+
+## Local, SSH, S3, and fake backends
 
 The local adapter maps logical paths to a configured filesystem root and keeps adapter-specific path handling behind the storage interface.
 
+The SSH adapter maps logical paths to a configured remote SFTP root. It uses native SSH and SFTP libraries, supports SSH agent and key-file authentication, applies host-key policies, rejects unsafe logical paths, reports symlink entries from `Lstat`, and limits deletion to managed targets or explicit bounded prefixes.
+
+The S3 adapter maps logical paths to object keys below a configured bucket and optional prefix. It uses the AWS SDK for Go v2, treats prefixes as object trees, requires exact objects for `Stat`, paginates traversal, applies conservative overwrite checks with `HeadObject`, infers basic content types, and limits deletion to managed target objects or explicit bounded object-key prefixes.
+
 The fake backend is an in-memory implementation for package tests. It is not registered for runtime use.
 
 ## Tests
@@ -43,6 +62,8 @@ Before changing storage behavior, inspect tests under:
 - `internal/storage`
 - `internal/storage/fake`
 - `internal/adapters/local`
+- `internal/adapters/ssh`
+- `internal/adapters/s3`
 
 ## Invariants
 
@@ -50,4 +71,5 @@ Before changing storage behavior, inspect tests under:
 - Logical paths are slash-separated and confined to the backend root.
 - `storage.List` uses backend traversal and returns deterministic entries.
 - Managed deletion is limited to recorded outputs plus `.distributor.json`.
+- Prefix deletion is limited to the requested logical prefix.
 - Runtime backend registration is owned by `internal/app`.

docs/internal/transform.md

@@ -2,11 +2,11 @@
 
 ## Purpose
 
-`internal/transform` defines generated publication artifacts. `internal/transform/markdown` implements Markdown-to-HTML sidecar generation.
+`internal/transform` defines generated publication artifacts. `internal/transform/markdown` implements Markdown-to-HTML generation.
 
 ## Inputs and outputs
 
-Inputs are a validated source bundle and source backend. Outputs include destination path, source path, transform id, generated bytes, SHA-256, and size.
+Inputs are a validated source bundle, source backend, and transform options supplied by publish planning. Outputs include destination path, source path, transform id, generated bytes, SHA-256, and size.
 
 ## Registry
 
@@ -14,7 +14,7 @@ Inputs are a validated source bundle and source backend. Outputs include destina
 
 ## Markdown behavior
 
-Markdown files ending in `.md` generate `.html` files in the same logical directory. Non-Markdown files do not generate outputs. Raw HTML embedded in Markdown is not passed through by the renderer.
+Markdown sidecar mode renders files ending in `.md` to `.html` files in the same logical directory. Markdown index mode renders one selected manifest-listed Markdown file to `index.html`. Non-Markdown files do not generate sidecar outputs. Raw HTML embedded in Markdown is not passed through by the renderer.
 
 Generated HTML is deterministic for the same source content and transform configuration.
 
@@ -22,7 +22,7 @@ See `docs/integrations/markdown.md` for the Goldmark integration contract.
 
 ## Failure behavior
 
-Transform resolution fails when a requested transform id is not registered. Markdown rendering fails when the source file cannot be read or rendered. Publish planning fails when HTML output is requested and the selected transform produces no outputs for a bundle.
+Transform resolution fails when a requested transform id is not registered. Markdown rendering fails when the source file cannot be read or rendered. Index input selection fails when the configured input is unsafe, not listed, not Markdown, or when no configured input can be inferred from exactly one manifest-listed Markdown file. Publish planning fails when HTML output is requested and the selected transform produces no outputs for a bundle.
 
 ## Boundaries
 
@@ -42,5 +42,6 @@ Before changing transform behavior, inspect tests under:
 - Source bundle files are never mutated by transforms.
 - Generated outputs record destination path, source path, transform id, SHA-256, and size.
 - Markdown sidecar naming changes only the `.md` extension to `.html`.
+- Markdown index mode always writes `index.html`.
 - Non-Markdown source files do not generate Markdown outputs.
 - Transform registration stays outside publish planning.

docs/operations.md

@@ -26,19 +26,141 @@ Run the local HTML publication:
 go run ./cmd/distributor run --config examples/local-html.yml
 ```
 
+Run the local `index.html` publication:
+
+```sh
+go run ./cmd/distributor run --config examples/local-index.yml
+```
+
 Preview local fan-out publication:
 
 ```sh
 go run ./cmd/distributor run --config examples/fan-out.yml --dry-run
 ```
 
+Preview local archive-plus-latest publication:
+
+```sh
+go run ./cmd/distributor run --config examples/archive-and-latest.yml --dry-run
+```
+
+Preview a run for automation:
+
+```sh
+go run ./cmd/distributor run --config examples/fan-out.yml --dry-run --format json
+```
+
+Preview an environment-gated SSH destination config after editing it for an SSH/SFTP endpoint you control:
+
+```sh
+go run ./cmd/distributor run --config examples/ssh-destination.yml --dry-run
+```
+
+Preview an environment-gated S3 destination config after editing it for an S3-compatible endpoint and bucket you control:
+
+```sh
+go run ./cmd/distributor run --config examples/s3-destination.yml --dry-run
+```
+
+Validate one configured source without opening destinations:
+
+```sh
+go run ./cmd/distributor validate --config examples/local-publish.yml --pipeline example-source-bundle
+```
+
+## HTTP Upload Workflow
+
+`distributor serve` runs the HTTP upload API for pipelines whose source backend
+is `http_upload`. Each upload token maps to one configured pipeline, and each
+accepted archive is staged, validated, and published through the same
+destination fan-out path used by local source runs.
+
+Minimal local HTTP upload configuration:
+
+```yaml
+server:
+  http:
+    bind: 127.0.0.1:8080
+    staging_root: /var/spool/distributor
+    max_upload_size: 20MB
+    queue_size: 16
+    max_concurrency: 1
+    retention: 24h
+secrets:
+  directory: /run/secrets/distributor
+pipelines:
+  - id: reports
+    source:
+      backend: http_upload
+      token_env: DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN
+    destinations:
+      - id: archive
+        backend: local
+        path: /srv/reports/archive
+```
+
+Create `/run/secrets/distributor/DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN` or set the
+real process environment variable before starting the server. Distributor does
+not read literal upload tokens from YAML.
+
+Start the maintained local example:
+
+```sh
+DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN=<token> \
+  go run ./cmd/distributor serve --config examples/http-upload-local.yml
+```
+
+Submit a tar or tar.gz source bundle:
+
+```sh
+curl -X POST http://127.0.0.1:8080/upload \
+  -H "Authorization: Bearer $DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN" \
+  -H "Content-Type: application/gzip" \
+  --data-binary @bundle.tar.gz
+```
+
+Successful admission returns a run id:
+
+```json
+{"run_id":"reports.20260603T120000Z.abcdef12","status":"accepted"}
+```
+
+Poll status until it reaches `succeeded` or `failed`:
+
+```sh
+curl http://127.0.0.1:8080/runs/<run-id>
+```
+
+The status record includes the completed run report on successful publication
+or error details on failure. Status is memory-only and expires after
+`server.http.retention`; completed staged bundle directories are removed on
+expiry. Restarting the process clears upload status and queue state.
+
+Use `GET /healthz` for readiness after config and tokens load:
+
+```sh
+curl http://127.0.0.1:8080/healthz
+```
+
+The default bind address is private loopback. Put TLS, public routing,
+rate-limiting, and external access policy in a reverse proxy or deployment
+layer.
+
 ## Filesystem Layout
 
-Source bundles are discovered beneath the configured local source root. Each bundle is a directory containing `manifest.json`.
+Source bundles are discovered beneath the configured source root. Each bundle is a directory containing `manifest.json`.
 
-Destination bundle paths preserve the source bundle path relative to the source root. A source bundle at the source root publishes to the destination root. A source bundle under `daily/` publishes under `daily/` at each destination.
+Destination bundle paths are configured per destination with `path_mapping.mode`.
 
-The maintained examples write under `workspace/`, which is ignored by Git.
+The default mode, `preserve_relative`, preserves the source bundle path relative to the source root. A source bundle at the source root publishes to the destination root. A source bundle under `daily/` publishes under `daily/` at that destination.
+
+The `fixed` mode publishes one selected source bundle at the destination backend root. A fixed destination with local `path: /srv/www/reports/latest` writes outputs and `.distributor.json` directly under `/srv/www/reports/latest`. Fixed destinations select the newest discovered source bundle by manifest `created` timestamp, with the source-root-relative bundle path as the deterministic tie-breaker.
+
+The maintained local examples write under `workspace/`, which is ignored by Git.
+
+SSH backends use the configured remote `path` as the backend root. Source bundle discovery and destination bundle paths are relative to that root, using the same logical path rules as local storage.
+
+S3 backends use the configured bucket plus optional `prefix` as the backend root. Source bundle discovery and destination bundle paths are relative to that object-key prefix. Prefixes are object-key prefixes, not real directories.
 
 ## Destination State
 
@@ -48,17 +170,173 @@ Each published destination bundle contains `.distributor.json`. This file is the
 - publication timestamp;
 - source manifest used for publication;
 - copied source output metadata;
-- generated output metadata.
+- generated output metadata;
+- optional public URL metadata when destination links are configured.
 
 `manifest.json` from the source bundle is not copied as destination state.
 
 Do not edit `.distributor.json` by hand during normal operation. If it is missing or invalid while destination files remain, `distributor` treats the destination as unmanaged or conflicted.
 
+## Go Producer Bundles
+
+Go producer applications can import `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to create complete local source bundles with the same path, digest, timestamp, and validation rules used by `distributor`. The package also exposes digest helpers, including `ValidateDigest`, for producer code that needs to validate lowercase `sha256:<64 hex>` strings before writing manifests.
+
+Minimal producer-side bundle creation:
+
+```go
+manifest, err := bundle.WriteBundle(bundle.WriteBundleOptions{
+    Root:  outputDir,
+    ID:    "reports.example.2026-05-30",
+    Files: []bundle.BundleFile{
+        {SourcePath: reportPath, Path: "report.md"},
+        {SourcePath: summaryPath, Path: "summary.txt"},
+    },
+})
+if err != nil {
+    return err
+}
+```
+
+`WriteBundle` copies local producer files into a sibling temporary directory, writes `manifest.json`, validates the result, and promotes the completed bundle into place. It fails if `Root` already exists unless `Overwrite` is true. With overwrite enabled, it builds and validates the replacement before moving the existing root aside.
+
+Use `BuildManifest` and `WriteManifest` when a producer already wrote all bundle files into the final root. `BuildManifest` can preserve an explicit file order, or `Scan: true` can recursively include regular files under `Root` in deterministic slash-path order. Scan mode includes dotfiles, excludes files named `manifest.json` or `.distributor.json`, and rejects symlinks.
+
+Shell producers can create the same manifest through the CLI after writing bundle files:
+
+```sh
+go run ./cmd/distributor manifest create <bundle-path> --id reports.example.2026-05-30
+go run ./cmd/distributor validate <bundle-path>
+```
+
+Use repeated `--file` flags to preserve a specific file order. If no `--file` flags are provided, the command scans the bundle directory recursively using the same filtering rules as `pkg/bundle.BuildManifest`.
+
+## Static HTML Publication
+
+Markdown-to-HTML publication can write sidecar files or a fixed `index.html`.
+
+Use sidecar mode when each Markdown source should keep a matching HTML filename:
+
+```yaml
+publish:
+  source: false
+  html: true
+transform:
+  markdown_to_html:
+    enabled: true
+    mode: sidecar
+```
+
+Use index mode for static-site destinations that should serve a bundle through `index.html`:
+
+```yaml
+publish:
+  source: false
+  html: true
+transform:
+  markdown_to_html:
+    enabled: true
+    mode: index
+    input: report.md
+```
+
+If `input` is omitted in index mode, the source manifest must list exactly one Markdown file. Generated HTML is recorded in `.distributor.json` with `kind: generated`, `source_path`, `transform: markdown_to_html`, digest, and size metadata.
+
+## Archive And Latest Fan-Out
+
+A pipeline can publish the same source to an archive destination and a stable latest destination:
+
+```yaml
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: /var/spool/distributor/reports
+    destinations:
+      - id: archive
+        backend: local
+        path: /srv/reports/archive
+        path_mapping:
+          mode: preserve_relative
+        publish:
+          source: true
+          html: false
+      - id: latest-html
+        backend: local
+        path: /srv/www/reports/latest
+        path_mapping:
+          mode: fixed
+        links:
+          base_url: https://reports.example.com/latest
+          primary: auto
+        publish:
+          source: false
+          html: true
+        transform:
+          markdown_to_html:
+            enabled: true
+            mode: index
+            input: report.md
+```
+
+The archive destination plans every discovered source bundle at its source-relative path. The fixed latest destination plans only the newest discovered bundle and writes `index.html` plus `.distributor.json` at its backend root.
+
+## Static Site URLs
+
+Use destination `links` when a destination backend root corresponds to a public HTTP or HTTPS URL:
+
+```yaml
+links:
+  base_url: https://reports.example.com/archive
+  primary: auto
+```
+
+Distributor records URLs in `.distributor.json`; it does not publish notifications or infer URLs from local, SSH, or S3 backend fields.
+
+For archive-style destinations, URLs include the destination bundle path. A source bundle under `daily/brentwood/2026-06-01` with `base_url: https://reports.example.com/archive` can produce:
+
+```text
+https://reports.example.com/archive/daily/brentwood/2026-06-01/report.html
+```
+
+For fixed destinations, URLs are rooted at `links.base_url`. A fixed HTML index destination with `base_url: https://reports.example.com/latest` records:
+
+```text
+https://reports.example.com/latest/
+```
+
+`index.html` outputs use directory-style URLs. Other outputs include their filename. The primary URL is selected from the published outputs using the destination `links.primary` policy.
+
+## Source Validation and Inspection
+
+`validate` and `inspect` can operate on a local path or on one configured pipeline source. Configured source mode requires both `--config` and `--pipeline`; it loads the normal config, resolves `secrets.directory`, opens only the selected source backend, and does not open any destinations.
+
+Configured source validation is useful when producers write directly to SSH or S3 storage:
+
+```sh
+go run ./cmd/distributor validate --config <config-path> --pipeline <pipeline-id>
+go run ./cmd/distributor inspect --config <config-path> --pipeline <pipeline-id>
+```
+
+Use `--bundle <path>` to validate or inspect one source-root-relative bundle directory:
+
+```sh
+go run ./cmd/distributor validate \
+  --config <config-path> \
+  --pipeline <pipeline-id> \
+  --bundle daily/2026-06-01
+```
+
+For configured SSH sources, host key and authentication behavior matches `run`. For configured S3 sources, endpoint, bucket, prefix, region, path-style, explicit credential environment variables, and `secrets.directory` handling match `run`.
+
 ## Dry Runs
 
-`--dry-run` loads and validates config, discovers source bundles, inspects destination state, plans outputs, and prints summary lines. It does not write output files or destination state.
+`--dry-run` loads and validates config, discovers source bundles, inspects destination state, plans outputs, and prints summary lines. It does not write output files, destination state, or SSH `known_hosts` entries.
 
-Dry-run output is useful before publishing to confirm actions such as `publish_new`, `replace_older`, `skip_same`, and `skip_destination_newer`.
+Dry-run output is useful before publishing to confirm actions such as `publish_new`, `replace_older`, `force_replace`, `skip_same`, and `skip_destination_newer`.
+
+Destination action lines include the destination backend, so mixed local, SSH, and S3 fan-out runs can be audited before publication. Fixed path destinations add `path_mapping=fixed target=.` to planned action lines. Dry-run also prints a warning with the fixed destination candidate count and selected source bundle; destructive fixed replacements print an additional warning that the destination root would be replaced.
+
+Use `--format json` when another process needs stable run data. JSON output includes warnings, pipeline summaries, destination actions, destination bundle paths, path mapping modes, optional link URLs, output records, final counters, and partial failure records. The summary includes `fixed_path`. If one destination fails after planning or execution begins, JSON output still contains the successful and failed destination records with `ok: false`, and the command exits non-zero.
 
 ## Retry and Replacement Behavior
 
@@ -68,18 +346,69 @@ If destination state is older than the source manifest and transfer policy allow
 
 If destination state is newer than the source manifest, the default behavior is to skip. If destination state has the same source id and created timestamp but a different digest, publication fails as a conflict.
 
-If a destination path has files but no valid `.distributor.json`, publication fails as unmanaged content. There is no force overwrite option.
+If a destination path has files but no valid `.distributor.json`, publication fails as unmanaged content unless the current run explicitly uses `--force`.
+
+## Force Workflow
+
+Use `--force` only after a dry run shows the intended `force_replace` action:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --dry-run --force
+go run ./cmd/distributor run --config <config-path> --force
+```
+
+Forced replacement can overwrite unmanaged non-empty destination paths. Destination state conflicts require `transfer.on_conflict: replace` plus `--force`. Newer destination state requires `transfer.on_destination_newer: replace` plus `--force`.
+
+Forced replacement deletes the current destination bundle path before writing outputs and state. It does not delete above that bundle path. For fixed destinations, the destination bundle path is the backend root, so forced replacement may clear that configured root but not its parent path, sibling directories, or anything outside the configured S3 bucket and prefix. Force is per run only and has no config default.
 
 ## Failure Handling
 
 If one destination fails in a fan-out run, independent later destinations are still planned and executed. The command exits non-zero after printing the final status if any destination failed.
 
-If a write fails during local publication, `distributor` attempts to remove outputs written during that failed attempt so a retry does not see those partial outputs as unmanaged destination content.
+Errors include the pipeline id, destination id, destination backend, and bundle path where applicable.
+
+In JSON mode, destination failures after planning or execution begins are reported in the top-level `errors` array and in the run result while preserving a non-zero exit code. Fatal setup errors such as an unreadable config or invalid secrets directory write no JSON document.
+
+If a write fails during publication, `distributor` attempts to remove outputs written during that failed attempt so a retry does not see those partial outputs as unmanaged destination content.
 
 After a successful publish or replacement, the internal notifier hook runs. The current default notifier is a no-op. Skipped destinations do not invoke it.
 
+## SSH Operation Notes
+
+SSH execution uses SFTP over `golang.org/x/crypto/ssh` and `github.com/pkg/sftp`. It does not shell out to `ssh`, `scp`, or `rsync`.
+
+Configure `ssh_key_file`, an SSH agent, or both. Agent identities are attempted first, followed by the configured key file. YAML password authentication is not supported.
+
+The default host key policy is `accept-new`. New host keys are written to `known_hosts` when the file path is writable. During `--dry-run`, unknown host keys may be accepted for the current connection but are not written to `known_hosts`; a later non-dry-run may persist the same key. Changed host keys are fatal for both `strict` and `accept-new`. The `off` policy disables host key checking and `run` prints a warning when stdout is enabled.
+
+Recovery boundaries are the same as local storage: replacement deletes only managed output paths recorded in `.distributor.json` plus the state file, and failed writes are cleaned up where practical. Distributor never performs broad recursive remote deletion.
+
+## S3 Operation Notes
+
+S3 execution uses the AWS SDK for Go v2. Configure `endpoint`, `bucket`, optional `prefix`, optional `region`, and optional explicit credential environment variable names.
+
+When explicit credential env names are configured, both variables must resolve to non-empty values through the real process environment or `secrets.directory`. When they are omitted, the AWS SDK default credential chain is used as-is.
+
+Normal replacement and failed-write cleanup delete only managed output objects recorded in `.distributor.json` plus the state object. Forced replacement deletes objects under the bounded destination bundle prefix. For fixed destinations, that prefix is the configured bucket plus optional `prefix`. Distributor does not manage bucket versioning or delete markers.
+
+## Secrets Directory
+
+Configure `secrets.directory` when credential values should come from mounted files, such as deployment secrets:
+
+```yaml
+secrets:
+  directory: /run/secrets/distributor
+```
+
+The directory is loaded during `run`, `serve`, and configured-source `validate`
+or `inspect` before credential-consuming work starts. If the directory is
+missing, unreadable, or contains an invalid secret filename, the command fails
+before storage work starts.
+
+Real process environment values take precedence over files with the same name. If the values differ and stdout is enabled, `run` and configured-source diagnostics print a warning naming the ignored secret file variable without printing either value. The process environment is not changed.
+
 ## Caveats
 
-Only local-to-local execution is available. SSH execution, S3 execution, external notification adapters, and force overwrite behavior are unavailable.
+External notification adapters are unavailable. Force overwrite behavior is available only through the explicit `run --force` workflow.
 
 For symptom-oriented fixes, see [troubleshooting](troubleshooting.md). For config details, see [configuration](config.md). For command syntax, see [CLI](cli.md).

docs/policy/architecture.md

@@ -22,19 +22,21 @@ The current core workflow is:
 2. open the source backend;
 3. discover source bundles beneath the source root;
 4. validate each source bundle and its `manifest.json`;
-5. for each configured destination, inspect destination state;
-6. compare source state to destination state;
-7. build a publish plan;
-8. optionally transform Markdown to HTML for that destination;
-9. publish selected source and generated artifacts;
-10. write `.distributor.json` as the destination sentinel/state file;
-11. run the notification hook, which is a no-op in the MVP.
+5. select the source bundle or bundles for each destination according to that destination's path mapping policy;
+6. open each destination backend independently;
+7. inspect destination state at the resolved destination bundle path;
+8. compare source state to destination state;
+9. build a publish plan that selects source files, generated files, destination paths, and optional public URLs;
+10. optionally transform Markdown to HTML for that destination;
+11. publish selected source and generated artifacts;
+12. write `.distributor.json` as the destination sentinel/state file;
+13. run the notification hook, whose default implementation is currently a no-op.
 
 ## Pipeline Model
 
 A pipeline has exactly one source and one or more destinations.
 
-The source is discovered and validated once. Each destination has independent backend configuration, publication policy, transform policy, replacement behavior, state, and notification behavior.
+The source is discovered and validated once. Each destination has independent backend configuration, path mapping, publication policy, transform policy, public link policy, replacement behavior, state, and notification behavior.
 
 The pipeline model is fan-out by design:
 
@@ -53,7 +55,7 @@ A source bundle is a directory containing `manifest.json`.
 
 `manifest.json` is the sole producer-to-`distributor` contract. `distributor` must not rely on producer-specific work directory layouts, filenames, metadata, or conventions outside the configured source root and the source manifest.
 
-The MVP source manifest schema is intentionally minimal:
+The source manifest schema is intentionally minimal:
 
 ```json
 {
@@ -73,7 +75,7 @@ The MVP source manifest schema is intentionally minimal:
 
 Required fields:
 
-- `schema_version`: source manifest schema version. MVP value: `1`.
+- `schema_version`: source manifest schema version. Current value: `1`.
 - `id`: stable bundle identifier.
 - `digest`: SHA-256 digest for the listed files.
 - `created`: RFC3339 timestamp. UTC is preferred; explicit offsets are allowed.
@@ -101,6 +103,7 @@ Each destination bundle path is managed by `.distributor.json`. This file is bot
 - the normalized source manifest used for publication;
 - metadata for copied source outputs;
 - metadata for generated outputs, such as HTML files;
+- optional URL metadata for published outputs;
 - any additional metadata required by `distributor`.
 
 A representative destination state file is:
@@ -127,14 +130,18 @@ A representative destination state file is:
       ]
     }
   },
+  "links": {
+    "primary_url": "https://reports.example.com/weather-daily/"
+  },
   "outputs": [
     {
-      "path": "report.html",
+      "path": "index.html",
       "kind": "generated",
       "source_path": "report.md",
       "transform": "markdown_to_html",
       "sha256": "sha256:...",
-      "size": 23456
+      "size": 23456,
+      "url": "https://reports.example.com/weather-daily/"
     }
   ]
 }
@@ -157,6 +164,8 @@ Transforms are configured per destination. A destination may receive source file
 
 The MVP supports only Markdown-to-HTML transformation. HTML generation must not mutate the source bundle. Generated outputs must be deterministic from the source bundle and destination transform configuration, and must be recorded in `.distributor.json`.
 
+Destination path mapping and public link generation are destination behavior. Source manifests do not declare where a bundle is published or which public URLs are recorded.
+
 The application should distinguish:
 
 - transform policy: how derived files are generated;
@@ -166,7 +175,7 @@ For example, one destination may publish source files only as a long-term archiv
 
 ## Backend Abstraction
 
-Sources and destinations use the same storage abstraction. Current runtime execution uses the local filesystem backend. Additional storage backends should be peer implementations behind the same interface, and any backend-specific execution limitation must be documented.
+Sources and destinations use the same storage abstraction. Current runtime execution uses the local filesystem, SSH/SFTP, and S3-compatible backends. Additional storage backends should be peer implementations behind the same interface, and any backend-specific execution limitation must be documented.
 
 Application logic must interact with storage through internal backend interfaces. Backend-specific behavior belongs in adapter packages. Pipeline, bundle, state, publish, and transform packages must not import service-specific or filesystem adapter implementation details.
 
@@ -187,13 +196,17 @@ Avoid dependencies for small conveniences. Do not let external dependency types
 Use this current layout unless the project has a documented reason to differ:
 
 - `cmd/distributor`: application entrypoint only.
+- `pkg/bundle`: public producer-facing source manifest model, digest logic, parsing, manifest building, complete local bundle writing, and local validation helpers.
 - `internal/app`: application orchestration and top-level use cases.
 - `internal/cli`: CLI command definitions, flags, argument parsing, and command wiring.
 - `internal/config`: configuration structs, defaults, loading, precedence, and validation.
-- `internal/bundle`: source manifest parsing, source bundle discovery, source digest validation, and source bundle model.
+- `internal/bundle`: storage-backed source bundle discovery and validation over the public manifest contract.
 - `internal/state`: `.distributor.json` parsing, validation, comparison, and output metadata.
+- `internal/link`: shared HTTP URL validation for configured and persisted link metadata.
 - `internal/storage`: backend interfaces, shared path/resource types, backend registry, and storage errors.
 - `internal/adapters/local`: local filesystem backend.
+- `internal/adapters/ssh`: SSH/SFTP backend.
+- `internal/adapters/s3`: S3-compatible object storage backend.
 - `internal/transform`: transform interfaces, registry, planning, and shared transform models.
 - `internal/transform/markdown`: Markdown-to-HTML implementation.
 - `internal/publish`: destination planning, reconciliation, safety checks, and publish execution.
@@ -226,17 +239,20 @@ Pipeline configuration should express:
 - pipeline id;
 - one source backend;
 - one or more destinations;
+- per-destination path mapping;
 - per-destination publish policy;
 - per-destination transform policy;
+- per-destination public link policy;
 - validation behavior;
 - destination conflict/replacement behavior.
 
-## Modules, Stages, and Registries
+## Modules and Registries
 
-Each major stage should have an explicit input/output contract:
+Each major workflow step should have an explicit input/output contract:
 
 - source discovery;
 - source validation;
+- destination bundle selection;
 - destination state inspection;
 - destination comparison;
 - transform planning/execution;
@@ -272,9 +288,11 @@ If the application writes durable state, writes should be atomic where practical
 
 Code that deletes, moves, or overwrites files must use narrow, explicit paths. Avoid broad parent-directory operations. Cleanup that can cause data loss must be opt-in.
 
-`distributor` must never perform broad deletion against a configured source root or destination root. Destructive replacement may occur only inside a resolved destination bundle path when a valid `.distributor.json` confirms that the path is distributor-managed.
+`distributor` must never perform broad deletion against a configured source root. Destination deletion must be bounded to the resolved destination bundle path for the current source bundle and backend root.
 
-Replacement must be narrow, logged, test-covered, and configurable. Prefer deleting files recorded in `.distributor.json` and known generated outputs rather than blindly deleting parent directories. Backend implementations must guard against path traversal, prefix confusion, and accidental root deletion.
+Normal destructive replacement may occur only when a valid `.distributor.json` confirms that the destination bundle path is distributor-managed. Explicit forced replacement is a per-run CLI workflow for supported conflict and unmanaged-content cases; it must be dry-runnable, clearly reported, and constrained to the destination bundle path.
+
+Replacement must be narrow, reported, test-covered, and configurable. Prefer normal replacement that deletes files recorded in `.distributor.json` and known generated outputs. Forced replacement may delete a bounded destination bundle prefix only when the operator explicitly requests it. Backend implementations must guard against path traversal, prefix confusion, and accidental deletion above the configured backend root.
 
 Where practical, publish operations should use staging paths or temporary objects and promote them into place only after validation and transform steps succeed.
 
@@ -293,8 +311,10 @@ Important tests include:
 - relative path safety and path traversal rejection;
 - destination `.distributor.json` parsing and comparison;
 - same/older/newer/conflict publish decisions;
+- destination bundle path mapping;
 - destructive replacement safety checks;
 - transform output planning and metadata recording;
+- public URL planning and state metadata;
 - dry-run output;
 - local backend behavior with temporary directories;
 - fake backend behavior for storage-facing core logic.

docs/policy/development.md

@@ -6,25 +6,29 @@ Use it with `docs/policy/architecture.md` and `docs/policy/documentation.md`.
 ## Repository Layout
 
 - `cmd/distributor`: executable entrypoint only.
+- `pkg/bundle`: public producer-facing source manifest and local bundle writer helpers.
 - `internal/app`: top-level use cases for `run`, `validate`, and `inspect`.
 - `internal/cli`: standard-library command parsing, flags, help text, and command wiring.
 - `internal/config`: YAML configuration structs, loading, defaults, and validation.
-- `internal/bundle`: source bundle discovery, manifest parsing, digest calculation, and validation.
+- `internal/bundle`: storage-backed source bundle discovery and validation using the public manifest contract.
 - `internal/state`: destination `.distributor.json` parsing, validation, and comparison.
 - `internal/storage`: backend interface, registry, logical path rules, typed errors, and shared storage helpers.
 - `internal/adapters/local`: local filesystem backend.
+- `internal/adapters/ssh`: SSH/SFTP backend.
+- `internal/adapters/s3`: S3-compatible object storage backend.
 - `internal/storage/fake`: in-memory backend for tests.
-- `internal/publish`: destination inspection, output planning, reconciliation, execution, and managed cleanup.
+- `internal/publish`: destination inspection, output planning, reconciliation, execution, managed cleanup, and explicit forced replacement.
 - `internal/transform`: transform interface and registry.
-- `internal/transform/markdown`: Markdown-to-HTML sidecar transform.
+- `internal/transform/markdown`: Markdown-to-HTML transform.
 - `internal/notify`: notification interface and current no-op notifier.
 - `internal/testutil`: shared test fixtures. Production code must not import this package.
 - `docs`: current user, operator, policy, internal, and roadmap documentation.
 - `examples`: copyable example configs and source bundles.
 
-Do not create new top-level package families such as `pkg`, `internal/stage`,
-`internal/modules`, or service-specific adapter directories unless the
-architecture policy or a current roadmap explicitly calls for them.
+Do not create new top-level package families such as public `pkg/...` packages
+beyond `pkg/bundle`, generic workflow containers, or service-specific adapter
+directories unless the architecture policy or a current roadmap explicitly
+calls for them.
 
 ## Common Commands
 
@@ -69,9 +73,9 @@ GOCACHE=/private/tmp/distributor-gocache GOMODCACHE=/private/tmp/distributor-gom
 - Prefer package-local helpers over broad abstractions until behavior is shared by multiple packages.
 - Keep CLI parsing in `internal/cli`; business decisions belong in `internal/app`, `internal/bundle`, `internal/publish`, `internal/state`, and related core packages.
 - Keep adapter packages thin. Backend-specific filesystem or service behavior belongs in adapters; bundle, state, transform, and publish policy belongs outside adapters.
-- Preserve public CLI behavior, config semantics, manifest schema, destination state schema, and local MVP behavior unless the current task explicitly changes them.
+- Preserve public CLI behavior, config semantics, manifest schema, destination state schema, and implemented backend behavior unless the current task explicitly changes them.
 - Use `storage.DisplayPath`, `storage.StateFileName`, `storage.StatePath`, and `storage.ManagedBundleTargets` instead of duplicating those conventions.
-- Use `bundle.ValidateManifest` for normalized source manifest semantics, including embedded source manifests in destination state.
+- Use `pkg/bundle` for normalized source manifest semantics. Internal packages should reach those rules through `internal/bundle` when they also need storage-backed bundle discovery or validation.
 - Use `config.ValidatePublishTransformPolicy` for publish and transform policy combinations.
 - Do not import concrete transform implementations from `internal/publish`; app-level wiring owns transform registration.
 - Do not import `internal/testutil` from production code.
@@ -82,6 +86,9 @@ The project currently depends on:
 
 - `gopkg.in/yaml.v3` for YAML configuration loading.
 - `github.com/yuin/goldmark` for Markdown rendering.
+- `golang.org/x/crypto/ssh`, `golang.org/x/crypto/ssh/agent`, and `golang.org/x/crypto/ssh/knownhosts` for native SSH support.
+- `github.com/pkg/sftp` for native SFTP support.
+- `github.com/aws/aws-sdk-go-v2/...` packages for S3-compatible storage support.
 
 Add external dependencies only when they materially improve correctness,
 security, interoperability, or implementation complexity. Avoid dependencies
@@ -100,9 +107,15 @@ When adding or changing configuration:
 5. Update `docs/config.md` in the same change if current user-visible config behavior changes.
 6. Update examples only with configs that are valid and executable for implemented behavior.
 
-Config validation may accept fields for backends that are not executable yet,
-but user-facing docs and examples must clearly state execution support. At the
-time of this policy, only the local backend is executable.
+Config validation may accept fields for roadmap backends before execution
+support exists, but user-facing docs and examples must clearly state execution
+support. Runtime executable backends are local, SSH, and S3.
+
+Credential-consuming code must use the config-owned environment resolver for
+explicit credential environment variable references. Do not call `os.Getenv`
+directly for backend credentials, because `secrets.directory` values are
+intentionally available through the resolver without mutating the process
+environment.
 
 ## CLI Changes
 
@@ -116,8 +129,9 @@ When adding or changing commands or flags:
 3. Add or update CLI tests in `internal/cli`.
 4. Update `docs/cli.md` if syntax, flags, output expectations, or workflows change.
 
-`validate` and `inspect` are local path commands. `run` loads configured
-pipelines and currently executes local backends only.
+`validate` and `inspect` support a local path shortcut and configured
+source-only diagnostics. `run` loads configured pipelines and executes local,
+SSH, and S3 backends.
 
 ## Storage Backends
 
@@ -133,8 +147,8 @@ When adding a backend:
 5. Add focused adapter tests and app-level wiring tests.
 6. Update user docs, operations docs, examples, and internal docs only for behavior that is actually implemented.
 
-Do not document SSH/SFTP or S3 execution as available until corresponding
-adapter packages and app wiring exist.
+Do not document future backend execution as available until the corresponding
+adapter package and app wiring exist.
 
 ## Transforms
 
@@ -158,16 +172,29 @@ Test close to the behavior being changed:
 - Use `internal/testutil` for shared valid fixtures only; keep edge cases near the package under test.
 - Run `go test ./...` after cross-package changes or documentation/example changes tied to tests.
 
+Live integration tests must be opt-in and skipped during normal `go test ./...`
+unless their required environment variables are set. Test-only environment
+variables must use this prefix shape:
+
+```text
+DISTRIBUTOR_TEST_<BACKEND>_*
+```
+
+Examples include `DISTRIBUTOR_TEST_SSH_HOST` and
+`DISTRIBUTOR_TEST_S3_ENDPOINT`. Do not use production credential variable names
+for test-only controls.
+
 ## Examples
 
 Examples under `examples/` must be valid, maintained, and free of secrets.
-They should be copyable for implemented behavior. Do not leave examples that
-look runnable but require unsupported backend execution.
+They should be copyable for implemented behavior. Remote examples must use
+placeholders or environment variables for endpoint and credential material.
 
 When changing examples:
 
 1. Keep paths relative to the repository where practical.
-2. Prefer local examples until remote backend support exists.
+2. Keep local examples runnable without external services; gate remote examples
+   behind user-provided endpoints and credentials.
 3. Run `go test ./internal/config` because config tests load examples.
 4. Update README, CLI, or config docs if links or recommended workflows change.
 

docs/policy/documentation.md

@@ -106,7 +106,7 @@ Recommended:
 - `examples/`
 - `docs/policy/development.md`
 
-### Modular, staged, service-oriented, or orchestration application
+### Modular, service-oriented, or orchestration application
 
 Required:
 - `docs/cli.md`, if CLI-based
@@ -175,7 +175,7 @@ It should include:
 - dependency policy;
 - how to add config fields;
 - how to add CLI flags;
-- how to add stages/modules/adapters, if applicable;
+- how to add modules or adapters, if applicable;
 - how to update examples;
 - documentation update expectations.
 
@@ -216,7 +216,7 @@ Explain when commands are useful, not just their syntax.
 
 **Audience:** administrators, operators
 
-Required for applications that maintain state, support resume behavior, run multiple stages, write durable artifacts, use remote storage, or require recovery procedures.
+Required for applications that maintain state, support resume behavior, run multi-step workflows, write durable artifacts, use remote storage, or require recovery procedures.
 
 It should cover:
 
@@ -248,7 +248,7 @@ Each entry should include:
 
 **Audience:** developers, LLM coding agents
 
-Required for modular, staged, service-oriented, or orchestration projects.
+Required for modular, service-oriented, or orchestration projects.
 
 This directory describes implemented internal components. It is not the roadmap.
 

docs/roadmap/http.md

@@ -0,0 +1,60 @@
+# Roadmap: HTTP Upload Extensions
+
+## Purpose
+
+The HTTP upload API is implemented. Current behavior is documented in:
+
+- [CLI](../cli.md)
+- [Configuration](../config.md)
+- [Operations](../operations.md)
+- [Troubleshooting](../troubleshooting.md)
+- [Application internals](../internal/app.md)
+- [Ingestion internals](../internal/ingest.md)
+
+This roadmap records HTTP upload extensions that are intentionally not part of
+the current implementation.
+
+## Deferred Extensions
+
+### Authentication
+
+- URL-token authentication for constrained clients.
+- Additional token lifecycle tooling.
+- Mutual TLS or other in-app identity mechanisms.
+
+### Archive Formats
+
+- Zstandard-compressed tar archives.
+- Additional content negotiation rules for future archive formats.
+
+### Status And Queue Durability
+
+- Durable status persistence across process restarts.
+- Database-backed queueing.
+- Recovery semantics for queued or running uploads after a restart.
+
+### Producer Coordination
+
+- Producer-supplied idempotency keys.
+- Run retry endpoints.
+- Run cancellation endpoints.
+- Run listing endpoints.
+
+### Deployment Surface
+
+- In-app TLS.
+- Public exposure defaults.
+- Browser UI.
+
+## Boundaries
+
+Current HTTP upload behavior remains intentionally small:
+
+- `http_upload` is source-only and is not a durable storage backend.
+- Upload status is memory-only.
+- Producers submit complete tar or gzip-compressed tar source bundles.
+- Producers authenticate with `Authorization: Bearer <token>`.
+- Public access policy, TLS termination, and rate limiting belong outside
+  `distributor` unless a future roadmap explicitly changes that boundary.
+
+Do not document deferred extensions as available outside `docs/roadmap/`.

docs/roadmap/implementation.md

@@ -1,463 +1,34 @@
-# Post-Local-MVP Implementation Roadmap
+# HTTP Upload Deferred Work
 
-This is the canonical active roadmap for `distributor` after the local MVP checkpoint.
+## Purpose
 
-The original MVP stages 1-8 are complete and are no longer listed as pending implementation work. Current behavior is documented outside the roadmap in `README.md`, `docs/cli.md`, `docs/config.md`, `docs/operations.md`, `docs/troubleshooting.md`, `docs/internal/`, `docs/integrations/markdown.md`, and `docs/policy/development.md`.
+HTTP upload behavior is implemented and documented in the current-behavior
+manuals:
 
-Future, planned, or aspirational behavior should remain under `docs/roadmap/` until implemented.
+- [CLI](../cli.md)
+- [Configuration](../config.md)
+- [Operations](../operations.md)
+- [Troubleshooting](../troubleshooting.md)
+- [Application internals](../internal/app.md)
+- [Configuration internals](../internal/config.md)
+- [Ingestion internals](../internal/ingest.md)
 
-## Current Baseline
-
-The implemented local MVP includes:
-
-- standard-library CLI commands for `version`, `run`, `validate`, and `inspect`;
-- YAML config loading, defaults, known-field rejection, and validation;
-- accepted config shapes for `local`, `ssh`, and `s3`, with executable backend support currently limited to `local`;
-- backend-rooted storage interface with typed errors, safe logical paths, traversal, `HasAny`, managed deletion, local backend, and fake backend;
-- source bundle discovery, manifest parsing, RFC3339 timestamp handling, duplicate path checks, path safety checks, symlink rejection, per-file digest validation, and bundle digest validation;
-- destination `.distributor.json` state parsing, validation, output metadata, and source comparison;
-- local publication of source files, Markdown sidecar HTML, or both;
-- destination output collision detection before writes;
-- managed replacement for older destination state;
-- unmanaged destination and conflict failures by default;
-- deterministic dry-run output and final run summaries;
-- deterministic sequential fan-out with aggregated failures;
-- cleanup of outputs written during failed local publish attempts where practical;
-- no-op notification hook after successful publish or replacement;
-- current user, operator, internal, integration, and development documentation for implemented behavior.
-
-The local MVP intentionally does not include executable SSH/SFTP backends, executable S3-compatible backends, force overwrite behavior, external notification adapters, warning-only digest mismatch behavior, or broad recursive destination deletion.
-
-## Active Roadmap Stages
-
-Implement each stage independently. Unless a stage explicitly says otherwise:
-
-1. read `docs/policy/architecture.md`, `docs/policy/documentation.md`, `docs/policy/development.md`, and this roadmap before editing;
-2. preserve current local MVP behavior;
-3. keep user-facing docs limited to implemented behavior;
-4. add or update focused tests for the behavior changed;
-5. run the relevant package tests and `go test ./...` for cross-package changes;
-6. avoid implementing later stages early.
-
-## Stage 1: SSH/SFTP Backend
-
-### Goal
-
-Implement native SSH/SFTP storage backend support for sources and destinations through the existing storage interface and app-level backend factory.
-
-### Implementation Scope
-
-Add an SSH/SFTP adapter package under `internal/adapters/ssh`.
-
-The backend must implement the current `internal/storage.Backend` contract:
-
-- `ReadFile` and `OpenReader`;
-- `WriteFile` and `WriteFrom`;
-- `Stat`;
-- `Walk`;
-- `HasAny`;
-- `DeleteManagedBundle`.
-
-Use native SFTP operations rather than shelling out to `ssh`, `scp`, or `rsync`.
-
-Authentication behavior:
-
-- prefer SSH agent by default;
-- use `known_hosts` validation by default where practical;
-- support optional key-file configuration only if it can be added cleanly;
-- do not support passwords in YAML in this stage.
-
-Config execution behavior:
-
-- use the existing accepted config shape:
-
-```yaml
-backend: ssh
-uri: ssh://user@example.com:22
-path: /remote/root
-```
-
-- keep secrets out of config files;
-- keep config loading and validation centralized in `internal/config`;
-- wire runtime construction through app-level backend factory and storage registry patterns.
-
-Supported pipeline combinations:
-
-- local source to SSH destination;
-- SSH source to local destination;
-- SSH source to SSH destination where feasible through streaming or backend-owned staging.
-
-Safety requirements:
-
-- enforce the same backend-rooted logical path rules as local storage;
-- reject path traversal, absolute logical paths, dot segments, and backslashes;
-- report or reject symlinks according to storage and bundle validation rules;
-- keep deletion limited to managed output paths and `.distributor.json`;
-- never delete a configured backend root;
-- preserve conservative non-force conflict behavior.
-
-### Documentation Updates
-
-After implementation, update only current-behavior docs:
-
-- `docs/config.md`: mark SSH as executable and document any implemented SSH-only fields.
-- `docs/operations.md`: add SSH source/destination operating notes and recovery boundaries.
-- `docs/troubleshooting.md`: add common SSH authentication, known-hosts, and remote path failures.
-- `docs/internal/storage.md`: add implemented SSH adapter behavior and tests.
-- `docs/policy/development.md`: update backend addition guidance if implementation changes the pattern.
-- `examples/`: add only runnable or clearly environment-gated SSH examples.
-
-Do not document S3 or force overwrite as implemented in this stage.
-
-### Tests
-
-Add unit tests for:
-
-- SSH config execution wiring;
-- URI and path handling;
-- logical path validation;
-- storage error translation where practical;
-- `Walk` and `HasAny` behavior through test doubles or controlled fixtures;
-- managed deletion boundaries;
-- app-level local-to-SSH and SSH-to-local planning or wiring using fakes/mocks where possible.
-
-Add integration tests only if they are skipped unless explicit SSH test endpoint environment variables are configured. Normal `go test ./...` must not require a live SSH server.
-
-### Completion Criteria
-
-- SSH/SFTP backend compiles and satisfies `storage.Backend`.
-- Runtime `run` can execute supported SSH source and destination flows.
-- Local MVP tests still pass.
-- Normal test runs do not require a live SSH server.
-- User docs accurately describe implemented SSH behavior and boundaries.
-
-## Stage 2: S3-Compatible Backend
-
-### Goal
-
-Implement S3-compatible object storage backend support for sources and destinations through the existing storage interface and app-level backend factory.
-
-### Implementation Scope
-
-Add an S3-compatible adapter package under `internal/adapters/s3`.
-
-The backend must implement the current `internal/storage.Backend` contract:
-
-- `ReadFile` and `OpenReader`;
-- `WriteFile` and `WriteFrom`;
-- `Stat`;
-- `Walk`;
-- `HasAny`;
-- `DeleteManagedBundle`.
-
-Use the existing accepted config shape:
-
-```yaml
-backend: s3
-endpoint: https://s3.example.com
-bucket: reports
-prefix: some/prefix
-region: us-east-1
-force_path_style: true
-credentials:
-  access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
-  secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
-```
-
-Credential behavior:
-
-- read access key and secret key from the named environment variables when configured;
-- support standard SDK credential discovery only if it does not weaken explicit config behavior;
-- do not put literal secrets in YAML.
-
-Object semantics:
-
-- treat prefixes as object trees, not real directories;
-- normalize configured prefix plus logical path with exact path-boundary matching;
-- `Stat` must not synthesize directory metadata only because objects exist below a prefix;
-- `Walk` should use object-list pagination and should not load an entire prefix into memory;
-- `HasAny` should stop after the first matching object;
-- `DeleteManagedBundle` must delete only listed managed output objects plus `.distributor.json`.
-
-Write behavior:
-
-- treat successful object PUT as publish-on-success;
-- set content type from `storage.WriteOptions` where available;
-- spool or buffer `WriteFrom` only when required by the SDK or backend;
-- preserve overwrite checks and conservative conflict behavior.
-
-Content type behavior should cover at least:
-
-- `.md`: `text/markdown; charset=utf-8`;
-- `.html`: `text/html; charset=utf-8`;
-- `.json`: `application/json`;
-- `.txt`: `text/plain; charset=utf-8`.
-
-Supported pipeline combinations:
-
-- local source to S3 destination;
-- S3 source to local destination;
-- S3 source to S3 destination where feasible through streaming or backend-owned staging.
-
-### Documentation Updates
-
-After implementation, update only current-behavior docs:
-
-- `docs/config.md`: mark S3 as executable and document implemented credential behavior.
-- `docs/operations.md`: add S3 source/destination layout, prefix, and recovery notes.
-- `docs/troubleshooting.md`: add common S3 credential, bucket, endpoint, prefix, and permission failures.
-- `docs/internal/storage.md`: add implemented S3 adapter behavior and tests.
-- `docs/policy/development.md`: update backend addition guidance if implementation changes the pattern.
-- `examples/`: add only safe S3 examples that use placeholder endpoints and environment variable names.
-
-Do not document force overwrite or notification adapters as implemented in this stage.
-
-### Tests
-
-Add unit tests for:
-
-- S3 config execution wiring;
-- credential environment variable handling;
-- key and prefix normalization;
-- exact prefix boundary behavior;
-- path traversal rejection;
-- content type selection;
-- paginated `Walk` behavior through mocks/fakes;
-- early-stop `HasAny`;
-- managed deletion boundaries;
-- publish planning with S3 destination state fixtures.
-
-Add integration tests only if they are skipped unless explicit S3-compatible endpoint credentials are configured. Normal `go test ./...` must not require live S3 credentials.
-
-### Completion Criteria
-
-- S3-compatible backend compiles and satisfies `storage.Backend`.
-- Runtime `run` can execute supported S3 source and destination flows.
-- Local and SSH behavior, if implemented, remain unchanged.
-- Normal test runs do not require live S3.
-- User docs accurately describe implemented S3 behavior and boundaries.
-
-## Stage 3: Cross-Backend Hardening and Documentation
-
-### Goal
-
-Harden behavior across implemented backend combinations, improve operator-facing failures, and synchronize current-behavior documentation and examples after remote backend support exists.
-
-### Implementation Scope
-
-Exercise and harden representative flows across all implemented backend types:
-
-- local source to local archive destination;
-- local source to local HTML destination;
-- local source to multiple destinations with different publish policies;
-- local source to SSH destination, when SSH is implemented and test credentials exist;
-- SSH source to local destination, when SSH is implemented and test credentials exist;
-- local source to S3 destination, when S3 is implemented and test credentials exist;
-- S3 source to local destination, when S3 is implemented and test credentials exist.
-
-Improve error context where practical for:
-
-- invalid config;
-- invalid source manifest;
-- digest mismatch;
-- destination conflict;
-- unmanaged destination path;
-- backend read, write, stat, walk, and delete failures;
-- transform failures;
-- partial fan-out failures.
-
-Ensure errors and logs identify pipeline id, destination id, bundle path or id, backend type, and logical path where useful without exposing secrets.
-
-Do not add force overwrite behavior in this stage.
-
-### Documentation Updates
-
-Update current-behavior docs after hardening:
-
-- `README.md`: keep the quickstart local unless remote examples become safe and concise.
-- `docs/cli.md`: document any changed output or diagnostics.
-- `docs/config.md`: ensure backend support status and config reference match implementation.
-- `docs/operations.md`: document cross-backend state layout, retry behavior, and recovery caveats.
-- `docs/troubleshooting.md`: add recurring SSH/S3 failure modes discovered during hardening.
-- `docs/internal/`: update storage, publish, app, and config internals where behavior changed.
-- `examples/`: keep examples copyable and free of secrets; remote examples must rely on placeholders and environment variables.
-
-### Tests
-
-Add or expand tests for:
-
-- dry-run across multiple destinations and backend types;
-- partial failure behavior;
-- repeated run idempotency;
-- older/newer destination state behavior across backends;
-- destination state output metadata accuracy;
-- generated HTML output metadata accuracy;
-- destructive replacement safety across implemented backends;
-- error context for common failures.
-
-Integration tests for SSH or S3 must remain opt-in through environment variables.
-
-### Completion Criteria
-
-- Implemented backend combinations behave consistently through the common pipeline path.
-- Repeated runs are idempotent.
-- Destructive paths remain bounded to managed destination bundle paths.
-- Operator-facing errors are actionable.
-- Current-behavior docs and examples match implemented backend support.
-
-## Stage 4: Explicit Force Overwrite
-
-### Goal
-
-Introduce explicit operator-requested force behavior for controlled overwrite cases that remain intentionally unsupported by default.
-
-### Implementation Scope
-
-Add a CLI-only force option:
-
-```bash
-distributor run --config config.yml --force
-```
-
-Force must be explicit per run. Do not add a persistent config default for force behavior.
-
-Define and implement force planning for:
-
-- unmanaged non-empty destination paths;
-- destination state with a different source id;
-- destination state with matching source id and matching `created` timestamp but different digest;
-- destination state with mismatched `pipeline_id` or `destination_id`;
-- destination newer than source when transfer policy explicitly allows replacement.
-
-Once force behavior exists, update transfer policy validation only for values supported by implemented force behavior:
-
-- `on_destination_newer: replace`;
-- `on_conflict: replace`.
-
-Safety requirements:
-
-- non-force behavior remains unchanged and conservative;
-- dry-run must show destructive force actions before any forced run;
-- force must never delete above the resolved destination bundle path or configured destination prefix;
-- local replacement should remain staged where practical;
-- S3 replacement must remain constrained to the destination bundle prefix;
-- managed state should still be written only after successful output writes;
-- logs and output must clearly mark force decisions.
-
-### Documentation Updates
-
-After implementation, update:
-
-- `docs/cli.md`: document `--force` syntax and dry-run workflow.
-- `docs/config.md`: document newly accepted transfer policy values and note force is CLI-only.
-- `docs/operations.md`: document safe force workflow and recovery boundaries.
-- `docs/troubleshooting.md`: describe when force may be appropriate and when it remains unsafe.
-- `docs/internal/publish.md` and `docs/internal/state.md`: document force planning and comparison handling.
-
-Do not document force as a default or config-only behavior.
-
-### Tests
-
-Add tests for:
-
-- force rejected or unavailable when the flag is absent;
-- unmanaged non-empty destination overwritten only with force;
-- different source id overwritten only with force and allowed policy;
-- same id and created timestamp with different digest overwritten only with force and allowed policy;
-- destination newer replaced only with force and allowed policy;
-- pipeline or destination id mismatch overwritten only with force and allowed policy;
-- dry-run reports destructive force actions without writing;
-- force deletes only bounded destination bundle paths;
-- local, SSH, and S3 backends, where implemented, preserve deletion boundaries.
-
-### Completion Criteria
-
-- Force overwrite behavior is explicit, logged, dry-runnable, and test-covered.
-- Default non-force behavior remains unchanged.
-- User docs clearly describe force risks and safe workflow.
-
-## Stage 5: Release Readiness
-
-### Goal
-
-Perform a final quality pass before treating `distributor` as ready for routine use against real producer pipelines and implemented destination backends.
-
-### Implementation Scope
-
-Review:
-
-- package boundaries against `docs/policy/architecture.md`;
-- contributor workflow against `docs/policy/development.md`;
-- user docs against `docs/policy/documentation.md`;
-- CLI UX and command output;
-- config validation and examples;
-- manifest and state compatibility;
-- destructive operation safety;
-- backend error handling;
-- logging and diagnostics for unattended operation;
-- test coverage for core invariants.
-
-Do not add new product features in this stage.
-
-### Documentation Updates
-
-Update current-behavior docs only for issues found during the readiness review.
-
-If release packaging, version injection, or installation workflow is added, document it in the appropriate current-behavior user or development docs.
-
-### Tests
-
-Run:
-
-```bash
-go test ./...
-```
-
-Also verify representative CLI examples that are documented as runnable.
-
-### Completion Criteria
-
-- A dry-run can be performed safely against real configured sources and destinations.
-- Repeated runs are idempotent.
-- Destructive replacement cannot occur outside managed destination bundle paths.
-- Current docs accurately reflect the application.
-- The project is ready to deploy against one real producer pipeline.
+This file tracks only HTTP upload work that is not implemented.
 
 ## Deferred Work
 
-The following work remains intentionally deferred unless a future roadmap promotes it:
+- URL-token authentication.
+- Zstandard-compressed archive support.
+- Durable status persistence across process restarts.
+- Database-backed queueing.
+- Producer-supplied idempotency keys.
+- Run listing, cancellation, and retry endpoints.
+- In-app TLS.
+- Public network exposure defaults.
+- Browser UI.
 
-- external notification adapters such as email, ntfy, Gotify, or Pushover;
-- RSS or Atom feed generation;
-- static site index pages beyond sidecar HTML output;
-- destination path remapping rules;
-- HTML themes beyond the minimal deterministic template;
-- full plugin architecture;
-- web UI;
-- report editing;
-- producer pipeline execution;
-- database-backed state;
-- complex retry queues;
-- concurrent publication workers;
-- symlink support;
-- warning-only digest mismatch handling;
-- password-based SSH authentication in YAML;
-- broad recursive or prefix deletion outside explicitly bounded force behavior.
+## Documentation Rule
 
-## Validation
-
-For roadmap-only edits:
-
-```bash
-git status --short
-git diff -- docs/roadmap
-rg -n "docs/roadmap/(packages|contracts|storage|config|documentation)\\.md" README.md docs examples
-rg -n "docs/roadmap/(packages|contracts|storage|config|documentation)\\.md" .
-rg -n "SSH|S3|--force|force overwrite|notification adapter|future|planned" README.md docs/*.md docs/internal docs/policy examples
-```
-
-Also search `docs/roadmap` for old MVP stage headings and titles from deleted roadmap files. That check should return no matches.
-
-The final SSH/S3/force/future-work search is not expected to return zero results. Review matches and confirm they are either under roadmap material or clearly marked as unsupported current behavior.
-
-Go tests are not required for documentation-only roadmap rationalization unless examples, behavior docs, or code change.
+Deferred behavior belongs under `docs/roadmap/` until implemented. Current
+behavior docs must describe only the active HTTP upload API, configuration,
+operation, troubleshooting, and internal package contracts.

docs/troubleshooting.md

@@ -26,7 +26,10 @@ Safe fix: compare the file to the reference in [configuration](config.md) and re
 
 ## `validate config ... backend ... is unsupported`
 
-Likely cause: a source or destination uses a backend name other than `local`, `ssh`, or `s3`.
+Likely cause: a source or destination uses an unsupported backend name, or a
+command is trying to execute a backend that is valid only for another workflow.
+`run`, `validate`, and `inspect` execute `local`, `ssh`, and `s3` sources.
+`serve` executes `http_upload` sources.
 
 Diagnostic:
 
@@ -34,23 +37,155 @@ Diagnostic:
 rg -n "backend:" <config-path>
 ```
 
-Safe fix: use `backend: local` for executable workflows. SSH and S3 config shapes are accepted only for validation; runtime execution is unavailable.
+Safe fix: use `backend: local`, `backend: ssh`, or `backend: s3` for normal
+source and destination workflows. Use `backend: http_upload` only for sources
+handled by `distributor serve`.
 
-## `backend ssh is not implemented for execution` or `backend s3 is not implemented for execution`
+## `bind HTTP server ... address already in use`
 
-Likely cause: the config validates but `run` tried to execute a remote backend.
+Likely cause: another process is already listening on `server.http.bind`.
 
 Diagnostic:
 
 ```sh
-go run ./cmd/distributor run --config <config-path> --dry-run
+ss -ltnp | rg '<port>'
 ```
 
-Safe fix: use local destinations for current executable workflows, or keep remote backend configs under roadmap material unless those adapters are added. See [configuration](config.md).
+Safe fix: stop the conflicting process or configure a different
+`server.http.bind` value. The default bind address is `127.0.0.1:8080`.
 
-## `validate command requires a path` or `inspect command requires a path`
+## `upload token environment variable ... is not set`
 
-Likely cause: `validate` or `inspect` was run without a path.
+Likely cause: a configured `http_upload` source references `token_env`, but the
+variable is absent from both the real process environment and
+`secrets.directory`.
+
+Diagnostic:
+
+```sh
+env | cut -d= -f1 | rg '^<token-variable>$'
+ls -l <secrets-directory>/<token-variable>
+```
+
+Safe fix: set the real environment variable or create a readable
+secrets-directory file with the same name. Do not place literal token values in
+YAML.
+
+## `upload token environment variables ... resolve to the same value`
+
+Likely cause: two configured `http_upload` pipelines resolve to the same bearer
+token value.
+
+Diagnostic:
+
+```sh
+rg -n 'token_env:' <config-path>
+```
+
+Safe fix: assign a distinct non-empty token value to each `http_upload`
+pipeline. Distributor does not print the duplicate token value.
+
+## `POST /upload` returns `401`
+
+Likely cause: the request is missing `Authorization: Bearer <token>` or the
+token does not match any configured `http_upload` pipeline.
+
+Diagnostic:
+
+```sh
+curl -i -X POST http://127.0.0.1:8080/upload \
+  -H "Authorization: Bearer $DISTRIBUTOR_UPLOAD_TOKEN" \
+  -H "Content-Type: application/x-tar" \
+  --data-binary @bundle.tar
+```
+
+Safe fix: use the token value resolved by the configured `token_env`. Do not
+include token values in logs or tickets.
+
+## `POST /upload` returns `413`
+
+Likely cause: the request body exceeds the selected pipeline's
+`source.max_upload_size` or the default `server.http.max_upload_size`.
+
+Diagnostic:
+
+```sh
+ls -lh bundle.tar bundle.tar.gz
+rg -n 'max_upload_size:' <config-path>
+```
+
+Safe fix: upload a smaller archive, remove unnecessary files from the source
+bundle, or raise the configured upload size limit.
+
+## `POST /upload` returns `415`
+
+Likely cause: the upload uses an unsupported content type. The server accepts
+uncompressed tar and gzip-compressed tar archives only.
+
+Diagnostic:
+
+```sh
+file bundle.tar.gz
+```
+
+Safe fix: send `Content-Type: application/x-tar`, `application/gzip`, or
+`application/x-gzip`, matching the archive format.
+
+## `POST /upload` returns `503`
+
+Likely cause: the in-memory upload queue is full.
+
+Diagnostic:
+
+```sh
+rg -n 'queue_size|max_concurrency' <config-path>
+```
+
+Safe fix: retry after active uploads finish, or increase `server.http.queue_size`
+for the deployment.
+
+## `GET /runs/<run_id>` returns `404`
+
+Likely cause: the run id is wrong, the process restarted, or the completed
+status record expired after `server.http.retention`.
+
+Diagnostic:
+
+```sh
+curl -i http://127.0.0.1:8080/runs/<run-id>
+rg -n 'retention:' <config-path>
+```
+
+Safe fix: use the exact `run_id` returned by `POST /upload`. If status retention
+is too short for operators, increase `server.http.retention`.
+
+## `--format: format must be text or json`
+
+Likely cause: a command was run with an unsupported output format.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --help
+```
+
+Safe fix: use `--format text` or `--format json`. Help and usage output are always text.
+
+## `--format json` wrote no JSON output
+
+Likely cause: the command failed before it could construct a result, such as a missing config file, invalid arguments, unreadable secrets directory, or source setup failure.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --format json
+```
+
+Safe fix: read the stderr error and fix the setup problem. JSON mode writes a document only after the command has enough information to construct a result.
+
+## `configured source mode requires --pipeline`
+
+Likely cause: `validate` or `inspect` was run with `--config` but without an explicit pipeline id.
 
 Diagnostic:
 
@@ -59,7 +194,230 @@ go run ./cmd/distributor validate --help
 go run ./cmd/distributor inspect --help
 ```
 
-Safe fix: pass a local source bundle directory or a local tree containing source bundles.
+Safe fix: add `--pipeline <pipeline-id>`. Configured source diagnostics require an explicit pipeline even when the config contains one pipeline.
+
+## `does not accept a local path with --config, --pipeline, or --bundle`
+
+Likely cause: local-path mode and configured source mode were mixed in one `validate` or `inspect` command.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor inspect --help
+```
+
+Safe fix: use either `distributor inspect <local-path>` or `distributor inspect --config <path> --pipeline <id>`, not both.
+
+## `--format json` exited non-zero with `ok: false`
+
+Likely cause: `run` began planning or executing destinations, and at least one destination failed while other destination results were still available.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --format json
+```
+
+Safe fix: inspect the top-level `errors` array, `result.actions`, and `result.summary`. Fix the failed destination, then preview with `--dry-run --format json` before retrying.
+
+## `prefix must be a clean relative slash-separated path`
+
+Likely cause: S3 `prefix` contains traversal, dot segments, empty segments, or backslashes after leading and trailing slashes are trimmed.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --dry-run
+```
+
+Safe fix: use a clean relative prefix such as `reports/archive`, or omit `prefix`.
+
+## `NoSuchBucket`, `InvalidBucketName`, or `not_found`
+
+Likely cause: the S3 bucket, endpoint, or prefix is wrong, or the configured credentials cannot see the requested object.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --dry-run
+```
+
+Safe fix: verify `endpoint`, `bucket`, `region`, `force_path_style`, and `prefix`. For S3-compatible services, keep `force_path_style: true` unless the service requires virtual-host addressing.
+
+## `AccessDenied`, `InvalidAccessKeyId`, or `SignatureDoesNotMatch`
+
+Likely cause: S3 credentials are missing, wrong, empty, or lack permission for the bucket or prefix.
+
+Diagnostic:
+
+```sh
+env | cut -d= -f1 | rg '^(<access-key-variable>|<secret-key-variable>)$'
+ls -l <secrets-directory>
+```
+
+Safe fix: provide both configured credential environment variables through the real environment or `secrets.directory`, or omit explicit credential fields to use the AWS SDK default credential chain.
+
+## S3 endpoint connection failures
+
+Likely cause: the endpoint URL is unreachable, uses the wrong scheme, or does not match the configured path-style mode.
+
+Diagnostic:
+
+```sh
+curl -I <endpoint>
+```
+
+Safe fix: correct `endpoint`, network routing, TLS settings outside distributor, or `force_path_style`. Distributor does not provide insecure TLS bypass configuration.
+
+## `load secrets directory ... no such file or directory`
+
+Likely cause: `secrets.directory` points to a missing directory.
+
+Diagnostic:
+
+```sh
+ls -ld <secrets-directory>
+```
+
+Safe fix: create or mount the directory before running, or remove `secrets.directory` if no credential files are needed.
+
+## `load secrets directory ... permission denied`
+
+Likely cause: the service user cannot read the configured secrets directory.
+
+Diagnostic:
+
+```sh
+ls -ld <secrets-directory>
+namei -l <secrets-directory>
+```
+
+Safe fix: adjust the directory path or deployment permissions so the service user can read the directory. Distributor does not enforce owner, group, or mode policy beyond OS read access.
+
+## `secret filename ... is invalid`
+
+Likely cause: a regular file in `secrets.directory` does not match `[A-Za-z_][A-Za-z0-9_]*`.
+
+Diagnostic:
+
+```sh
+find <secrets-directory> -maxdepth 1 -type f -printf '%f\n'
+```
+
+Safe fix: rename the file to a valid credential environment variable name, or remove it from the secrets directory.
+
+## `credential environment variable ... is not set`
+
+Likely cause: a backend credential field references an environment variable that is absent from both the real process environment and the configured secrets directory.
+
+Diagnostic:
+
+```sh
+env | cut -d= -f1 | rg '^<variable-name>$'
+ls -l <secrets-directory>/<variable-name>
+```
+
+Safe fix: set the real environment variable or create a readable secrets-directory file with the same name.
+
+## `secret ... ignored because the real environment already has that variable`
+
+Likely cause: the real process environment and secrets directory both define the variable with different values.
+
+Diagnostic:
+
+```sh
+env | cut -d= -f1 | rg '^<variable-name>$'
+ls -l <secrets-directory>/<variable-name>
+```
+
+Safe fix: remove one source of the credential or make the deployment intentionally prefer the real environment value. Distributor does not print either value.
+
+## `host is required for ssh backend`
+
+Likely cause: SSH config is missing the structured `host` field, or an old URL-style SSH config is still in use.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --dry-run
+```
+
+Safe fix: configure SSH with `host`, optional `user` and `port`, and `path`. SSH URLs are not part of the active config schema.
+
+## `no SSH auth methods configured`
+
+Likely cause: neither an SSH agent nor `ssh_key_file` is available.
+
+Diagnostic:
+
+```sh
+test -n "$SSH_AUTH_SOCK" && ssh-add -l
+ls -l <ssh-key-file>
+```
+
+Safe fix: start an SSH agent with an appropriate key loaded, or configure `ssh_key_file` with a readable private key.
+
+## `host key ... is unknown` or `known_hosts is required`
+
+Likely cause: strict host key checking has no known host key, or `accept-new` cannot persist a new key.
+
+Diagnostic:
+
+```sh
+ls -l <known-hosts-path>
+ssh-keygen -F <host> -f <known-hosts-path>
+```
+
+Safe fix: configure a writable `known_hosts` path for `accept-new`, pre-populate `known_hosts` for `strict`, or explicitly use `host_key_policy: off` only for insecure test environments.
+
+## `host key ... has changed`
+
+Likely cause: the remote server presented a different host key than the one recorded in `known_hosts`.
+
+Diagnostic:
+
+```sh
+ssh-keygen -F <host> -f <known-hosts-path>
+```
+
+Safe fix: verify the server identity out of band before updating `known_hosts`. Do not switch to `host_key_policy: off` to bypass an unexpected changed key.
+
+## `pipeline "<id>" not found`
+
+Likely cause: configured source validation or inspection requested a pipeline id that is not present in the config file.
+
+Diagnostic:
+
+```sh
+rg -n "id:" <config-path>
+```
+
+Safe fix: pass an existing pipeline id with `--pipeline`, or update the config.
+
+## `stat ssh ... not_found`, `stat s3 ... not_found`, or `no bundles found`
+
+Likely cause: the configured source root is wrong, unreadable, or does not contain source bundles.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor validate --config <config-path> --pipeline <pipeline-id>
+```
+
+Safe fix: correct the configured source root, S3 prefix, permissions, or source bundle location. Use `--bundle <path>` only with a source-root-relative bundle directory that contains `manifest.json`.
+
+## `validate command requires a path` or `inspect command requires a path`
+
+Likely cause: `validate` or `inspect` was run without a local path and without configured source mode.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor validate --help
+go run ./cmd/distributor inspect --help
+```
+
+Safe fix: pass a local source bundle directory or local tree, or pass both `--config <path>` and `--pipeline <id>`.
 
 ## `no bundles found under "."`
 
@@ -95,7 +453,7 @@ Diagnostic:
 find <destination-path> -maxdepth 2 -print
 ```
 
-Safe fix: choose an empty destination path or move existing files aside after confirming they are not needed. There is no force overwrite option.
+Safe fix: choose an empty destination path or move existing files aside after confirming they are not needed. If the destination should be claimed by distributor, preview with `run --dry-run --force` and publish with `run --force` only after confirming the reported `force_replace` action is bounded to the intended bundle path.
 
 ## `fail_conflict`
 
@@ -108,7 +466,31 @@ cat <destination-path>/.distributor.json
 go run ./cmd/distributor inspect <source-root>
 ```
 
-Safe fix: verify you are publishing the intended source to the intended destination. Use a separate destination path for unrelated content.
+Safe fix: verify you are publishing the intended source to the intended destination. Use a separate destination path for unrelated content. If the existing state should be replaced, configure `transfer.on_conflict: replace`, preview with `run --dry-run --force`, then publish with `run --force`.
+
+## `destination is newer and replacement requires --force`
+
+Likely cause: config explicitly allows newer-destination replacement, but the current run did not include `--force`.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --dry-run --force
+```
+
+Safe fix: prefer the default `transfer.on_destination_newer: skip` unless replacing newer destination state is intentional. To replace it, keep `transfer.on_destination_newer: replace`, confirm the dry-run output shows `force_replace`, then run with `--force`.
+
+## `force_replace`
+
+Likely cause: the current run used `--force` and publish planning selected a supported destructive replacement.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --dry-run --force
+```
+
+Safe fix: inspect the printed pipeline id, destination id, backend, and bundle path. Proceed only if deleting all content within that destination bundle path is intended.
 
 ## `destination output path collision`
 
@@ -124,7 +506,7 @@ Safe fix: adjust the source bundle contents or publish policy so source and gene
 
 ## A run failed after writing some files
 
-Likely cause: a write failed partway through publication. Local execution attempts to clean up outputs written during the failed attempt.
+Likely cause: a write failed partway through publication. Local, SSH, and S3 execution attempt to clean up outputs written during the failed attempt.
 
 Diagnostic:
 
@@ -132,4 +514,4 @@ Diagnostic:
 find <destination-path> -maxdepth 2 -print
 ```
 
-Safe fix: inspect the destination before retrying. If only unrelated unmanaged files remain, move them aside or choose a clean destination. Re-run with `--dry-run` before publishing again. See [operations](operations.md).
+Safe fix: use the pipeline id, destination id, backend, and bundle path printed in the run error to inspect the destination before retrying. If only unrelated unmanaged files remain, move them aside or choose a clean destination. Re-run with `--dry-run` before publishing again. See [operations](operations.md).

examples/archive-and-latest.yml

@@ -0,0 +1,29 @@
+pipelines:
+  - id: example-archive-and-latest
+    source:
+      backend: local
+      path: examples/source-bundle
+    destinations:
+      - id: local-source-archive
+        backend: local
+        path: workspace/published/archive-and-latest/archive
+        path_mapping:
+          mode: preserve_relative
+        publish:
+          source: true
+          html: false
+      - id: local-html-latest
+        backend: local
+        path: workspace/published/archive-and-latest/latest
+        path_mapping:
+          mode: fixed
+        links:
+          base_url: https://reports.example.com/latest
+          primary: auto
+        publish:
+          source: false
+          html: true
+        transform:
+          markdown_to_html:
+            enabled: true
+            mode: index

examples/http-upload-local.yml

@@ -0,0 +1,24 @@
+# Local HTTP upload example.
+# Set DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN in the process environment or provide a
+# secrets-directory file with that name before running `distributor serve`.
+server:
+  http:
+    bind: 127.0.0.1:8080
+    staging_root: workspace/http-upload/staging
+    max_upload_size: 20MB
+    queue_size: 16
+    max_concurrency: 1
+    retention: 24h
+pipelines:
+  - id: example-http-upload
+    source:
+      backend: http_upload
+      token_env: DISTRIBUTOR_EXAMPLE_UPLOAD_TOKEN
+    destinations:
+      - id: local-archive
+        backend: local
+        path: workspace/published/http-upload
+        publish:
+          source: true
+          html: false
+

examples/local-index.yml

@@ -0,0 +1,16 @@
+pipelines:
+  - id: example-index-bundle
+    source:
+      backend: local
+      path: examples/source-bundle
+    destinations:
+      - id: local-index
+        backend: local
+        path: workspace/published/index-bundle
+        publish:
+          source: false
+          html: true
+        transform:
+          markdown_to_html:
+            enabled: true
+            mode: index

examples/s3-destination.yml

@@ -0,0 +1,22 @@
+# Environment-gated example.
+# Replace endpoint, bucket, prefix, and credential environment variable names
+# with values for an S3-compatible service you control before running this config.
+pipelines:
+  - id: example-s3-destination
+    source:
+      backend: local
+      path: examples/source-bundle
+    destinations:
+      - id: s3-archive
+        backend: s3
+        endpoint: https://s3.example.com
+        bucket: reports
+        prefix: distributor/archive
+        region: us-east-1
+        force_path_style: true
+        credentials:
+          access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
+          secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
+        publish:
+          source: true
+          html: false

examples/ssh-destination.yml

@@ -0,0 +1,21 @@
+# Environment-gated example.
+# Replace host, user, path, ssh_key_file, and known_hosts with values for an
+# SSH/SFTP endpoint you control before running this config.
+pipelines:
+  - id: example-ssh-destination
+    source:
+      backend: local
+      path: examples/source-bundle
+    destinations:
+      - id: ssh-archive
+        backend: ssh
+        host: ssh.example.com
+        user: distributor
+        port: 22
+        path: /srv/distributor/archive
+        ssh_key_file: /home/distributor/.ssh/id_ed25519
+        known_hosts: /home/distributor/.ssh/known_hosts
+        host_key_policy: strict
+        publish:
+          source: true
+          html: false

go.mod

@@ -3,6 +3,31 @@ module gitea.maximumdirect.net/eric/distributor
 go 1.26
 
 require (
+	github.com/aws/aws-sdk-go-v2 v1.41.9
+	github.com/aws/aws-sdk-go-v2/config v1.32.20
+	github.com/aws/aws-sdk-go-v2/credentials v1.19.19
+	github.com/aws/aws-sdk-go-v2/service/s3 v1.102.2
+	github.com/aws/smithy-go v1.26.0
+	github.com/pkg/sftp v1.13.10
 	github.com/yuin/goldmark v1.8.2
+	golang.org/x/crypto v0.52.0
 	gopkg.in/yaml.v3 v3.0.1
 )
+
+require (
+	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.25 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.25 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.25 // indirect
+	github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.26 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.10 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.18 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.25 // indirect
+	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.25 // indirect
+	github.com/aws/aws-sdk-go-v2/service/signin v1.1.1 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sso v1.30.19 // indirect
+	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.2 // indirect
+	github.com/aws/aws-sdk-go-v2/service/sts v1.42.3 // indirect
+	github.com/kr/fs v0.1.0 // indirect
+	golang.org/x/sys v0.45.0 // indirect
+)

go.sum

@@ -1,5 +1,57 @@
+github.com/aws/aws-sdk-go-v2 v1.41.9 h1:/rYeyO2+HrMztAmxAq9++XJtFMqSIpSsNA0yDGALYq4=
+github.com/aws/aws-sdk-go-v2 v1.41.9/go.mod h1:+HsoOEX80qAVUitj1A2DhCNTjmb3edVyuDypb6LNEeo=
+github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11 h1:h5+3VT69KUBK24grGuuA5saDJTj2IIjLb9au668Fo5I=
+github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.11/go.mod h1:dnakxebH6UwFvcvujL0LVggYQ8nEvBGjU4G/V79Nv94=
+github.com/aws/aws-sdk-go-v2/config v1.32.20 h1:8VMDnWc/kEzxsI/1ngGM9mG81a8IGmIHD8KLcYGwagc=
+github.com/aws/aws-sdk-go-v2/config v1.32.20/go.mod h1:PuwEpciweIXGULWeOeSTXtSbH4CW9mWdWrhdCKQI1sM=
+github.com/aws/aws-sdk-go-v2/credentials v1.19.19 h1:yuFzSV1U0aRNYCQGVaTY2zW2M/L93pYHnXnrJUphYhU=
+github.com/aws/aws-sdk-go-v2/credentials v1.19.19/go.mod h1:7y63L1kGzeoDlJaQ3Z578KrnmfBut96JjvJUzGwR+YE=
+github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.25 h1:0w6dCiO8iez+YKwRhRBlL1CH/E3GTfdkuzrwj1by8vo=
+github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.25/go.mod h1:9FDWUothyr5RCRAHc45XOiVCzUR8n/IhCYX+uVqw6vk=
+github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.25 h1:Uii3frf9ztec/ABM2/FSH9/z7PLzxfpG8h4RpkUFflQ=
+github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.25/go.mod h1:G6kntsA2GorAxDPbap6xgB2F+amSLUF8GJTi7PUoX44=
+github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.25 h1:r1+/l6m+WaUJF9HISEsNOLHSNj5EXYQxK8VX6Cz9NlA=
+github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.25/go.mod h1:cKf+D+NMDK1LndD7BowHbBZPgR9V0/5HubH0PFWvA+c=
+github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.26 h1:A1PmWU2zfkIm9EyFlJncFXL4W4phML+h8KjltUsCvNQ=
+github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.26/go.mod h1:dY4MRzXEizrD4hqtpKvWVGPX7QleSGGVY+EBolo1RmM=
+github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.10 h1:d5/908OJ4bXg8lyjeMPvXetEKqoDoLi5Owy1zNue3yg=
+github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.10/go.mod h1:a57l7Hwh+FWI+we50g5NPJHYUKeJKfXbc4w8SyXu8Ig=
+github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.18 h1:W/EyPFl9A5rXrtoilfwHYEvzHER+K4SpBPtMXi24Mos=
+github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.18/go.mod h1:UG50K+pvd/uy6xExbobg0rjqFBFZe6I3l75EPDZw4tg=
+github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.25 h1:dD3dhHNglpd98gs72my22Ndqi1hqQGllFFg1F+twfxg=
+github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.25/go.mod h1:0yAbjPfd64gG7mj85RW+fMEYdfBgCRZw8g/oWcL1pjc=
+github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.25 h1:2pQEbwf+/6EDbiit/GcBE2K4IUpMZymaA0kOz3xK978=
+github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.25/go.mod h1:KvT6NCcQ0EZ+ZkVRrlBMt04Po3ok23YELEp7WimhLhM=
+github.com/aws/aws-sdk-go-v2/service/s3 v1.102.2 h1:ie4ElCmUKS26pzrZcIk/lmt4yWjAqLLcawstyQCh298=
+github.com/aws/aws-sdk-go-v2/service/s3 v1.102.2/go.mod h1:zjsomFeX5duj+4PlMB+o4JoWTIx+G0XMyzjYrUbQkN0=
+github.com/aws/aws-sdk-go-v2/service/signin v1.1.1 h1:1VwbP3qMNfxUDEXWki4rCE5iA+44VA1lokTz9HasGzw=
+github.com/aws/aws-sdk-go-v2/service/signin v1.1.1/go.mod h1:vUtyoSj0OPji3kjIVSc/GlKuWEiL33f/WFxl6dmpy/A=
+github.com/aws/aws-sdk-go-v2/service/sso v1.30.19 h1:N6pIsdFOW1Kd9S4KyFKXdGRBojPPxkP32+uHFWLv4Hc=
+github.com/aws/aws-sdk-go-v2/service/sso v1.30.19/go.mod h1:3gt5WJArFooNmyLONS+h/R4J+o86II8du38IgCwj9dE=
+github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.2 h1:hc+lBYiiTr8Zk4MTzIsQ92MeDWCIDvWGmzKUWOaBcOg=
+github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.2/go.mod h1:hU6fqB3OJA6/ePheD47LQnxvjYk6br6PtQxs+Q9ojvk=
+github.com/aws/aws-sdk-go-v2/service/sts v1.42.3 h1:ErklX/7uhSbkAAeyQD/Y1OoQ9hO3SJXQNEgksORW3Js=
+github.com/aws/aws-sdk-go-v2/service/sts v1.42.3/go.mod h1:ULe4HCzfKPiR6R3HEurE3b1upEkuk8AkMrOKtaOxKO8=
+github.com/aws/smithy-go v1.26.0 h1:9ouqbi+NyKP7fV3Te7UElCwdAb6Y8uk7LGwPE5tVe/s=
+github.com/aws/smithy-go v1.26.0/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8=
+github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
+github.com/pkg/sftp v1.13.10 h1:+5FbKNTe5Z9aspU88DPIKJ9z2KZoaGCu6Sr6kKR/5mU=
+github.com/pkg/sftp v1.13.10/go.mod h1:bJ1a7uDhrX/4OII+agvy28lzRvQrmIQuaHrcI1HbeGA=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
+github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
 github.com/yuin/goldmark v1.8.2 h1:kEGpgqJXdgbkhcOgBxkC0X0PmoPG1ZyoZ117rDVp4zE=
 github.com/yuin/goldmark v1.8.2/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
+golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988=
+golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc=
+golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
+golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
+golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4=
+golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=

internal/adapters/local/backend.go

@@ -161,31 +161,10 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
 		}
 		return b.translateError(storage.OpWalk, prefix, err)
 	}
-	visited := 0
-	emit := func(entry storage.Entry) error {
-		if err := ctx.Err(); err != nil {
-			return err
-		}
-		if opts.Limit > 0 && visited >= opts.Limit {
-			return storage.ErrStopWalk
-		}
-		visited++
-		if err := fn(entry); err != nil {
-			if errors.Is(err, storage.ErrStopWalk) {
-				return storage.ErrStopWalk
-			}
-			return storage.NewError(storage.OpWalk, backendName, entry.Path, storage.ErrUnknown, err)
-		}
-		return nil
-	}
+	emitter := storage.NewWalkEmitter(ctx, backendName, opts, fn)
 
 	if !info.IsDir() {
-		if err := emit(entryFromInfo(prefix, info)); errors.Is(err, storage.ErrStopWalk) {
-			return nil
-		} else if err != nil {
-			return err
-		}
-		return nil
+		return storage.FinishWalk(emitter.Emit(entryFromInfo(prefix, info)))
 	}
 
 	walkErr := filepath.WalkDir(nativePrefix, func(nativePath string, dirEntry fs.DirEntry, err error) error {
@@ -210,24 +189,13 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
 		if err != nil {
 			return b.translateError(storage.OpWalk, logicalPath, err)
 		}
-		return emit(entryFromInfo(logicalPath, info))
+		return emitter.Emit(entryFromInfo(logicalPath, info))
 	})
-	if errors.Is(walkErr, storage.ErrStopWalk) {
-		return nil
-	}
-	return walkErr
+	return storage.FinishWalk(walkErr)
 }
 
 func (b *Backend) HasAny(ctx context.Context, prefix string) (bool, error) {
-	found := false
-	err := b.Walk(ctx, prefix, storage.WalkOptions{Recursive: false, Limit: 1}, func(storage.Entry) error {
-		found = true
-		return storage.ErrStopWalk
-	})
-	if err != nil {
-		return false, err
-	}
-	return found, nil
+	return storage.HasAny(ctx, b, prefix)
 }
 
 func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {
@@ -267,6 +235,51 @@ func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, ma
 	return nil
 }
 
+func (b *Backend) DeletePrefix(ctx context.Context, prefix string, opts storage.DeleteOptions) error {
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	nativePrefix, err := b.nativePath(prefix, true)
+	if err != nil {
+		return err
+	}
+	if err := b.rejectSymlinkAncestors(nativePrefix, false); err != nil {
+		return err
+	}
+	if prefix == "" {
+		entries, err := os.ReadDir(nativePrefix)
+		if err != nil {
+			if opts.IgnoreMissing && errors.Is(err, fs.ErrNotExist) {
+				return nil
+			}
+			return b.translateError(storage.OpDeletePrefix, prefix, err)
+		}
+		for _, entry := range entries {
+			if err := ctx.Err(); err != nil {
+				return err
+			}
+			child := filepath.Join(nativePrefix, entry.Name())
+			if err := os.RemoveAll(child); err != nil {
+				return b.translateError(storage.OpDeletePrefix, entry.Name(), err)
+			}
+		}
+		return nil
+	}
+	if _, err := os.Lstat(nativePrefix); err != nil {
+		if opts.IgnoreMissing && errors.Is(err, fs.ErrNotExist) {
+			return nil
+		}
+		return b.translateError(storage.OpDeletePrefix, prefix, err)
+	}
+	if err := os.RemoveAll(nativePrefix); err != nil {
+		return b.translateError(storage.OpDeletePrefix, prefix, err)
+	}
+	if opts.PruneEmptyDirs {
+		b.pruneEmptyParents(filepath.Dir(nativePrefix))
+	}
+	return nil
+}
+
 func (b *Backend) nativePath(logicalPath string, allowEmpty bool) (string, error) {
 	if logicalPath == "" {
 		if !allowEmpty {

internal/adapters/local/backend_test.go

@@ -169,6 +169,30 @@ func TestBackendManagedDeletion(t *testing.T) {
 	}
 }
 
+func TestBackendDeletePrefixStaysWithinPrefix(t *testing.T) {
+	backend := newBackend(t)
+	mustWrite(t, backend, "bundle/report.md", "report")
+	mustWrite(t, backend, "bundle/nested/old.txt", "old")
+	mustWrite(t, backend, "bundle-sibling/keep.txt", "keep")
+	mustWrite(t, backend, "outside.txt", "outside")
+
+	if err := backend.DeletePrefix(context.Background(), "bundle", storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true}); err != nil {
+		t.Fatalf("DeletePrefix() error = %v", err)
+	}
+	if _, err := backend.Stat(context.Background(), "bundle/report.md"); !storage.IsNotFound(err) {
+		t.Fatalf("deleted file stat error = %v, want not found", err)
+	}
+	if _, err := backend.Stat(context.Background(), "bundle/nested/old.txt"); !storage.IsNotFound(err) {
+		t.Fatalf("deleted nested file stat error = %v, want not found", err)
+	}
+	if _, err := backend.Stat(context.Background(), "bundle-sibling/keep.txt"); err != nil {
+		t.Fatalf("sibling stat error = %v", err)
+	}
+	if _, err := backend.Stat(context.Background(), "outside.txt"); err != nil {
+		t.Fatalf("outside stat error = %v", err)
+	}
+}
+
 func TestBackendHasAny(t *testing.T) {
 	backend := newBackend(t)
 	found, err := backend.HasAny(context.Background(), "missing")

internal/adapters/s3/backend.go

@@ -0,0 +1,455 @@
+package s3
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"path"
+	"sort"
+	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+	"github.com/aws/aws-sdk-go-v2/aws"
+	awscfg "github.com/aws/aws-sdk-go-v2/config"
+	"github.com/aws/aws-sdk-go-v2/credentials"
+	awss3 "github.com/aws/aws-sdk-go-v2/service/s3"
+	"github.com/aws/aws-sdk-go-v2/service/s3/types"
+	"github.com/aws/smithy-go"
+)
+
+type Client interface {
+	HeadObject(ctx context.Context, input *awss3.HeadObjectInput, optFns ...func(*awss3.Options)) (*awss3.HeadObjectOutput, error)
+	GetObject(ctx context.Context, input *awss3.GetObjectInput, optFns ...func(*awss3.Options)) (*awss3.GetObjectOutput, error)
+	PutObject(ctx context.Context, input *awss3.PutObjectInput, optFns ...func(*awss3.Options)) (*awss3.PutObjectOutput, error)
+	ListObjectsV2(ctx context.Context, input *awss3.ListObjectsV2Input, optFns ...func(*awss3.Options)) (*awss3.ListObjectsV2Output, error)
+	DeleteObject(ctx context.Context, input *awss3.DeleteObjectInput, optFns ...func(*awss3.Options)) (*awss3.DeleteObjectOutput, error)
+}
+
+type Backend struct {
+	client Client
+	bucket string
+	prefix string
+}
+
+func New(ctx context.Context, options Options) (*Backend, error) {
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+	options, err := options.normalized()
+	if err != nil {
+		return nil, storage.NewError(storage.OpOpenBackend, BackendName, options.Prefix, storage.ErrInvalidPath, err)
+	}
+	loadOptions := []func(*awscfg.LoadOptions) error{
+		awscfg.WithRegion(options.Region),
+	}
+	if options.AccessKeyID != "" {
+		loadOptions = append(loadOptions, awscfg.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(options.AccessKeyID, options.SecretAccessKey, "")))
+	}
+	cfg, err := awscfg.LoadDefaultConfig(ctx, loadOptions...)
+	if err != nil {
+		return nil, storage.NewError(storage.OpOpenBackend, BackendName, options.Endpoint, storage.ErrUnknown, err)
+	}
+	client := awss3.NewFromConfig(cfg, func(o *awss3.Options) {
+		o.BaseEndpoint = aws.String(options.Endpoint)
+		o.UsePathStyle = options.ForcePathStyle
+	})
+	return NewWithClient(client, options)
+}
+
+func NewWithClient(client Client, options Options) (*Backend, error) {
+	options, err := options.normalized()
+	if err != nil {
+		return nil, storage.NewError(storage.OpOpenBackend, BackendName, options.Prefix, storage.ErrInvalidPath, err)
+	}
+	if client == nil {
+		return nil, storage.NewError(storage.OpOpenBackend, BackendName, options.Bucket, storage.ErrInvalidPath, fmt.Errorf("client is required"))
+	}
+	return &Backend{
+		client: client,
+		bucket: options.Bucket,
+		prefix: options.Prefix,
+	}, nil
+}
+
+func (b *Backend) ReadFile(ctx context.Context, logicalPath string) ([]byte, error) {
+	reader, err := b.OpenReader(ctx, logicalPath)
+	if err != nil {
+		return nil, err
+	}
+	defer reader.Close()
+	data, err := io.ReadAll(reader)
+	if err != nil {
+		return nil, storage.NewError(storage.OpReadFile, BackendName, logicalPath, storage.ErrUnknown, err)
+	}
+	return data, nil
+}
+
+func (b *Backend) OpenReader(ctx context.Context, logicalPath string) (io.ReadCloser, error) {
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+	key, err := b.objectKey(logicalPath, false)
+	if err != nil {
+		return nil, err
+	}
+	output, err := b.client.GetObject(ctx, &awss3.GetObjectInput{
+		Bucket: aws.String(b.bucket),
+		Key:    aws.String(key),
+	})
+	if err != nil {
+		return nil, b.translateError(storage.OpOpenReader, logicalPath, err)
+	}
+	return output.Body, nil
+}
+
+func (b *Backend) WriteFile(ctx context.Context, logicalPath string, data []byte, opts storage.WriteOptions) (storage.Entry, error) {
+	opts.Size = int64(len(data))
+	opts.SizeKnown = true
+	return b.WriteFrom(ctx, logicalPath, bytes.NewReader(data), opts)
+}
+
+func (b *Backend) WriteFrom(ctx context.Context, logicalPath string, r io.Reader, opts storage.WriteOptions) (storage.Entry, error) {
+	if err := ctx.Err(); err != nil {
+		return storage.Entry{}, err
+	}
+	key, err := b.objectKey(logicalPath, false)
+	if err != nil {
+		return storage.Entry{}, err
+	}
+	if !opts.Overwrite {
+		_, err := b.client.HeadObject(ctx, &awss3.HeadObjectInput{
+			Bucket: aws.String(b.bucket),
+			Key:    aws.String(key),
+		})
+		if err == nil {
+			return storage.Entry{}, storage.NewError(storage.OpWriteFrom, BackendName, logicalPath, storage.ErrAlreadyExist, nil)
+		}
+		if !isNotFound(err) {
+			return storage.Entry{}, b.translateError(storage.OpWriteFrom, logicalPath, err)
+		}
+	}
+	data, err := io.ReadAll(r)
+	if err != nil {
+		return storage.Entry{}, storage.NewError(storage.OpWriteFrom, BackendName, logicalPath, storage.ErrUnknown, err)
+	}
+	if opts.SizeKnown && int64(len(data)) != opts.Size {
+		return storage.Entry{}, storage.NewError(storage.OpWriteFrom, BackendName, logicalPath, storage.ErrConflict, fmt.Errorf("stream size %d does not match expected size %d", len(data), opts.Size))
+	}
+	contentType := opts.ContentType
+	if contentType == "" {
+		contentType = ContentType(logicalPath)
+	}
+	_, err = b.client.PutObject(ctx, &awss3.PutObjectInput{
+		Bucket:        aws.String(b.bucket),
+		Key:           aws.String(key),
+		Body:          bytes.NewReader(data),
+		ContentLength: aws.Int64(int64(len(data))),
+		ContentType:   aws.String(contentType),
+	})
+	if err != nil {
+		return storage.Entry{}, b.translateError(storage.OpWriteFrom, logicalPath, err)
+	}
+	return storage.Entry{Path: logicalPath, Type: storage.EntryTypeFile, Size: int64(len(data))}, nil
+}
+
+func (b *Backend) Stat(ctx context.Context, logicalPath string) (storage.Entry, error) {
+	if err := ctx.Err(); err != nil {
+		return storage.Entry{}, err
+	}
+	key, err := b.objectKey(logicalPath, false)
+	if err != nil {
+		return storage.Entry{}, err
+	}
+	output, err := b.client.HeadObject(ctx, &awss3.HeadObjectInput{
+		Bucket: aws.String(b.bucket),
+		Key:    aws.String(key),
+	})
+	if err != nil {
+		return storage.Entry{}, b.translateError(storage.OpStat, logicalPath, err)
+	}
+	size := int64(0)
+	if output.ContentLength != nil {
+		size = *output.ContentLength
+	}
+	return storage.Entry{Path: logicalPath, Type: storage.EntryTypeFile, Size: size}, nil
+}
+
+func (b *Backend) Walk(ctx context.Context, logicalPrefix string, opts storage.WalkOptions, fn storage.WalkFunc) error {
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	if err := storage.ValidatePrefix(logicalPrefix); err != nil {
+		return err
+	}
+	emitter := storage.NewWalkEmitter(ctx, BackendName, opts, fn)
+	if logicalPrefix != "" {
+		entry, err := b.Stat(ctx, logicalPrefix)
+		if err == nil {
+			if err := emitter.Emit(entry); err != nil {
+				return storage.FinishWalk(err)
+			}
+			if emitter.LimitReached() {
+				return nil
+			}
+		} else if !storage.IsNotFound(err) {
+			return err
+		}
+	}
+	return storage.FinishWalk(b.walkObjects(ctx, logicalPrefix, opts, emitter.Emit))
+}
+
+func (b *Backend) HasAny(ctx context.Context, logicalPrefix string) (bool, error) {
+	return storage.HasAny(ctx, b, logicalPrefix)
+}
+
+func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	targets, err := storage.ManagedBundleTargets(bundlePath, managedOutputPaths)
+	if err != nil {
+		return err
+	}
+	for _, target := range targets {
+		if err := b.deleteObject(ctx, storage.OpDeleteManagedBundle, target, opts); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func (b *Backend) DeletePrefix(ctx context.Context, logicalPrefix string, opts storage.DeleteOptions) error {
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	if err := storage.ValidatePrefix(logicalPrefix); err != nil {
+		return err
+	}
+	found := false
+	if logicalPrefix != "" {
+		key, err := b.objectKey(logicalPrefix, false)
+		if err != nil {
+			return err
+		}
+		_, err = b.client.HeadObject(ctx, &awss3.HeadObjectInput{
+			Bucket: aws.String(b.bucket),
+			Key:    aws.String(key),
+		})
+		if err == nil {
+			found = true
+			if err := b.deleteObject(ctx, storage.OpDeletePrefix, logicalPrefix, storage.DeleteOptions{IgnoreMissing: false}); err != nil {
+				return err
+			}
+		} else if !isNotFound(err) {
+			return b.translateError(storage.OpDeletePrefix, logicalPrefix, err)
+		}
+	}
+	var entries []storage.Entry
+	if err := b.walkObjects(ctx, logicalPrefix, storage.WalkOptions{Recursive: true}, func(entry storage.Entry) error {
+		if entry.Type == storage.EntryTypeFile {
+			entries = append(entries, entry)
+		}
+		return nil
+	}); err != nil {
+		return err
+	}
+	for _, entry := range entries {
+		found = true
+		if err := b.deleteObject(ctx, storage.OpDeletePrefix, entry.Path, storage.DeleteOptions{IgnoreMissing: true}); err != nil {
+			return err
+		}
+	}
+	if !found && !opts.IgnoreMissing {
+		return storage.NewError(storage.OpDeletePrefix, BackendName, logicalPrefix, storage.ErrNotFound, nil)
+	}
+	return nil
+}
+
+func (b *Backend) deleteObject(ctx context.Context, op, logicalPath string, opts storage.DeleteOptions) error {
+	key, err := b.objectKey(logicalPath, false)
+	if err != nil {
+		return err
+	}
+	if key == b.prefix {
+		return storage.NewError(op, BackendName, logicalPath, storage.ErrInvalidPath, nil)
+	}
+	if !opts.IgnoreMissing {
+		_, err := b.client.HeadObject(ctx, &awss3.HeadObjectInput{
+			Bucket: aws.String(b.bucket),
+			Key:    aws.String(key),
+		})
+		if err != nil {
+			return b.translateError(op, logicalPath, err)
+		}
+	}
+	_, err = b.client.DeleteObject(ctx, &awss3.DeleteObjectInput{
+		Bucket: aws.String(b.bucket),
+		Key:    aws.String(key),
+	})
+	if err != nil {
+		if opts.IgnoreMissing && isNotFound(err) {
+			return nil
+		}
+		return b.translateError(op, logicalPath, err)
+	}
+	return nil
+}
+
+func (b *Backend) walkObjects(ctx context.Context, logicalPrefix string, opts storage.WalkOptions, emit func(storage.Entry) error) error {
+	listPrefix, err := b.listPrefix(logicalPrefix)
+	if err != nil {
+		return err
+	}
+	delimiter := ""
+	if !opts.Recursive {
+		delimiter = "/"
+	}
+	var token *string
+	for {
+		output, err := b.client.ListObjectsV2(ctx, &awss3.ListObjectsV2Input{
+			Bucket:            aws.String(b.bucket),
+			Prefix:            aws.String(listPrefix),
+			Delimiter:         aws.String(delimiter),
+			ContinuationToken: token,
+		})
+		if err != nil {
+			return b.translateError(storage.OpWalk, logicalPrefix, err)
+		}
+		entries := entriesFromList(logicalPrefix, b.prefix, output)
+		sort.Slice(entries, func(i, j int) bool { return entries[i].Path < entries[j].Path })
+		for _, entry := range entries {
+			if entry.Path == "" {
+				continue
+			}
+			if err := emit(entry); err != nil {
+				return err
+			}
+		}
+		if output.IsTruncated == nil || !*output.IsTruncated {
+			return nil
+		}
+		token = output.NextContinuationToken
+	}
+}
+
+func entriesFromList(logicalPrefix, rootPrefix string, output *awss3.ListObjectsV2Output) []storage.Entry {
+	seen := make(map[string]storage.Entry)
+	for _, object := range output.Contents {
+		if object.Key == nil {
+			continue
+		}
+		logicalPath := logicalPathFromKey(rootPrefix, *object.Key)
+		if logicalPath == "" || logicalPath == logicalPrefix {
+			continue
+		}
+		size := int64(0)
+		if object.Size != nil {
+			size = *object.Size
+		}
+		seen[logicalPath] = storage.Entry{Path: logicalPath, Type: storage.EntryTypeFile, Size: size}
+	}
+	for _, commonPrefix := range output.CommonPrefixes {
+		if commonPrefix.Prefix == nil {
+			continue
+		}
+		logicalPath := strings.TrimSuffix(logicalPathFromKey(rootPrefix, *commonPrefix.Prefix), "/")
+		if logicalPath == "" || logicalPath == logicalPrefix {
+			continue
+		}
+		seen[logicalPath] = storage.Entry{Path: logicalPath, Type: storage.EntryTypeDirectory}
+	}
+	entries := make([]storage.Entry, 0, len(seen))
+	for _, entry := range seen {
+		entries = append(entries, entry)
+	}
+	return entries
+}
+
+func (b *Backend) objectKey(logicalPath string, allowEmpty bool) (string, error) {
+	if logicalPath == "" {
+		if !allowEmpty {
+			return "", storage.NewError(storage.OpValidatePath, BackendName, logicalPath, storage.ErrInvalidPath, nil)
+		}
+		return b.prefix, nil
+	}
+	if err := storage.ValidatePath(logicalPath); err != nil {
+		return "", err
+	}
+	if b.prefix == "" {
+		return logicalPath, nil
+	}
+	return b.prefix + "/" + logicalPath, nil
+}
+
+func (b *Backend) listPrefix(logicalPrefix string) (string, error) {
+	key, err := b.objectKey(logicalPrefix, true)
+	if err != nil {
+		return "", err
+	}
+	if key != "" {
+		key = strings.TrimSuffix(key, "/") + "/"
+	}
+	return key, nil
+}
+
+func logicalPathFromKey(rootPrefix, key string) string {
+	if rootPrefix == "" {
+		return key
+	}
+	if key == rootPrefix {
+		return ""
+	}
+	return strings.TrimPrefix(key, rootPrefix+"/")
+}
+
+func ContentType(logicalPath string) string {
+	switch strings.ToLower(path.Ext(logicalPath)) {
+	case ".md":
+		return "text/markdown; charset=utf-8"
+	case ".html":
+		return "text/html; charset=utf-8"
+	case ".json":
+		return "application/json"
+	case ".txt":
+		return "text/plain; charset=utf-8"
+	default:
+		return "application/octet-stream"
+	}
+}
+
+func isNotFound(err error) bool {
+	var notFound *types.NotFound
+	if errors.As(err, &notFound) {
+		return true
+	}
+	var apiErr smithy.APIError
+	if errors.As(err, &apiErr) {
+		switch apiErr.ErrorCode() {
+		case "NotFound", "NoSuchKey", "404":
+			return true
+		}
+	}
+	return false
+}
+
+func (b *Backend) translateError(op, logicalPath string, err error) error {
+	kind := storage.ErrUnknown
+	if isNotFound(err) {
+		kind = storage.ErrNotFound
+	} else {
+		var apiErr smithy.APIError
+		if errors.As(err, &apiErr) {
+			switch apiErr.ErrorCode() {
+			case "AccessDenied", "InvalidAccessKeyId", "SignatureDoesNotMatch":
+				kind = storage.ErrPermission
+			case "SlowDown", "RequestTimeout", "ServiceUnavailable", "InternalError":
+				kind = storage.ErrTemporary
+			case "InvalidBucketName", "NoSuchBucket":
+				kind = storage.ErrInvalidPath
+			}
+		}
+	}
+	return storage.NewError(op, BackendName, logicalPath, kind, err)
+}

internal/adapters/s3/backend_test.go

@@ -0,0 +1,442 @@
+package s3
+
+import (
+	"context"
+	"io"
+	"sort"
+	"strings"
+	"testing"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+	"github.com/aws/aws-sdk-go-v2/aws"
+	awss3 "github.com/aws/aws-sdk-go-v2/service/s3"
+	"github.com/aws/aws-sdk-go-v2/service/s3/types"
+)
+
+func TestKeyAndPrefixNormalization(t *testing.T) {
+	backend := newTestBackend(t, "root/prefix", nil)
+	key, err := backend.objectKey("bundle/report.md", false)
+	if err != nil {
+		t.Fatalf("objectKey() error = %v", err)
+	}
+	if got, want := key, "root/prefix/bundle/report.md"; got != want {
+		t.Fatalf("objectKey() = %q, want %q", got, want)
+	}
+	listPrefix, err := backend.listPrefix("bundle")
+	if err != nil {
+		t.Fatalf("listPrefix() error = %v", err)
+	}
+	if got, want := listPrefix, "root/prefix/bundle/"; got != want {
+		t.Fatalf("listPrefix() = %q, want %q", got, want)
+	}
+}
+
+func TestPathTraversalRejected(t *testing.T) {
+	backend := newTestBackend(t, "", nil)
+	for _, logicalPath := range []string{"/absolute", "../escape", "a/../b", `a\b`} {
+		t.Run(logicalPath, func(t *testing.T) {
+			if _, err := backend.objectKey(logicalPath, false); err == nil || !storage.IsInvalidPath(err) {
+				t.Fatalf("objectKey() error = %v, want invalid path", err)
+			}
+		})
+	}
+}
+
+func TestContentType(t *testing.T) {
+	tests := map[string]string{
+		"report.md":   "text/markdown; charset=utf-8",
+		"report.html": "text/html; charset=utf-8",
+		"state.json":  "application/json",
+		"summary.txt": "text/plain; charset=utf-8",
+		"data.bin":    "application/octet-stream",
+	}
+	for path, want := range tests {
+		if got := ContentType(path); got != want {
+			t.Fatalf("ContentType(%q) = %q, want %q", path, got, want)
+		}
+	}
+}
+
+func TestStatRequiresExactObject(t *testing.T) {
+	client := newFakeClient(map[string]string{"root/dir/file.txt": "data"})
+	backend := newTestBackend(t, "root", client)
+	_, err := backend.Stat(context.Background(), "dir")
+	if err == nil || !storage.IsNotFound(err) {
+		t.Fatalf("Stat() error = %v, want not found", err)
+	}
+}
+
+func TestWalkIncludesExactObjectAndDescendants(t *testing.T) {
+	client := newFakeClient(map[string]string{
+		"root/bundle":           "marker",
+		"root/bundle/report.md": "report",
+	})
+	backend := newTestBackend(t, "root", client)
+	entries, err := storage.List(context.Background(), backend, "bundle", storage.WalkOptions{Recursive: true})
+	if err != nil {
+		t.Fatalf("List() error = %v", err)
+	}
+	paths := entryPaths(entries)
+	if got, want := paths, []string{"bundle", "bundle/report.md"}; !equalStrings(got, want) {
+		t.Fatalf("paths = %v, want %v", got, want)
+	}
+}
+
+func TestWalkExactObjectHonorsLimitBeforeListingDescendants(t *testing.T) {
+	client := newFakeClient(map[string]string{
+		"root/bundle":           "marker",
+		"root/bundle/report.md": "report",
+	})
+	backend := newTestBackend(t, "root", client)
+	var entries []storage.Entry
+	err := backend.Walk(context.Background(), "bundle", storage.WalkOptions{Recursive: true, Limit: 1}, func(entry storage.Entry) error {
+		entries = append(entries, entry)
+		return nil
+	})
+	if err != nil {
+		t.Fatalf("Walk() error = %v", err)
+	}
+	paths := entryPaths(entries)
+	if got, want := paths, []string{"bundle"}; !equalStrings(got, want) {
+		t.Fatalf("paths = %v, want %v", got, want)
+	}
+	if len(client.tokens) != 0 {
+		t.Fatalf("list calls = %d, want none", len(client.tokens))
+	}
+}
+
+func TestHasAnyWithExactObjectStopsBeforeListingDescendants(t *testing.T) {
+	client := newFakeClient(map[string]string{
+		"root/bundle":           "marker",
+		"root/bundle/report.md": "report",
+	})
+	backend := newTestBackend(t, "root", client)
+	found, err := backend.HasAny(context.Background(), "bundle")
+	if err != nil {
+		t.Fatalf("HasAny() error = %v", err)
+	}
+	if !found {
+		t.Fatal("HasAny() = false, want true")
+	}
+	if len(client.tokens) != 0 {
+		t.Fatalf("list calls = %d, want none", len(client.tokens))
+	}
+}
+
+func TestWriteFromChecksOverwriteBeforePut(t *testing.T) {
+	client := newFakeClient(map[string]string{"root/report.md": "old"})
+	backend := newTestBackend(t, "root", client)
+	_, err := backend.WriteFile(context.Background(), "report.md", []byte("new"), storage.WriteOptions{})
+	if err == nil || !storage.IsAlreadyExists(err) {
+		t.Fatalf("WriteFile() error = %v, want already exists", err)
+	}
+	if len(client.putKeys) != 0 {
+		t.Fatalf("put keys = %v, want none", client.putKeys)
+	}
+}
+
+func TestWriteFromPutsNewObjectWithContentType(t *testing.T) {
+	client := newFakeClient(nil)
+	backend := newTestBackend(t, "root", client)
+	entry, err := backend.WriteFile(context.Background(), "report.html", []byte("<p>ok</p>"), storage.WriteOptions{})
+	if err != nil {
+		t.Fatalf("WriteFile() error = %v", err)
+	}
+	if entry.Path != "report.html" || entry.Size != 9 || entry.Type != storage.EntryTypeFile {
+		t.Fatalf("entry = %#v", entry)
+	}
+	if got, want := client.objects["root/report.html"], "<p>ok</p>"; got != want {
+		t.Fatalf("object = %q, want %q", got, want)
+	}
+	if got, want := client.contentTypes["root/report.html"], "text/html; charset=utf-8"; got != want {
+		t.Fatalf("content type = %q, want %q", got, want)
+	}
+}
+
+func TestWalkUsesPagination(t *testing.T) {
+	client := newFakeClient(nil)
+	client.listPages = []awss3.ListObjectsV2Output{
+		{
+			Contents:              []types.Object{{Key: aws.String("root/a.txt"), Size: aws.Int64(1)}},
+			IsTruncated:           aws.Bool(true),
+			NextContinuationToken: aws.String("next"),
+		},
+		{
+			Contents:    []types.Object{{Key: aws.String("root/b.txt"), Size: aws.Int64(2)}},
+			IsTruncated: aws.Bool(false),
+		},
+	}
+	backend := newTestBackend(t, "root", client)
+	entries, err := storage.List(context.Background(), backend, "", storage.WalkOptions{Recursive: true})
+	if err != nil {
+		t.Fatalf("List() error = %v", err)
+	}
+	paths := entryPaths(entries)
+	if got, want := paths, []string{"a.txt", "b.txt"}; !equalStrings(got, want) {
+		t.Fatalf("paths = %v, want %v", got, want)
+	}
+	if got, want := client.tokens, []string{"", "next"}; !equalStrings(got, want) {
+		t.Fatalf("tokens = %v, want %v", got, want)
+	}
+}
+
+func TestHasAnyStopsAfterFirstPage(t *testing.T) {
+	client := newFakeClient(nil)
+	client.listPages = []awss3.ListObjectsV2Output{
+		{
+			Contents:              []types.Object{{Key: aws.String("root/a.txt"), Size: aws.Int64(1)}},
+			IsTruncated:           aws.Bool(true),
+			NextContinuationToken: aws.String("next"),
+		},
+		{
+			Contents:    []types.Object{{Key: aws.String("root/b.txt"), Size: aws.Int64(2)}},
+			IsTruncated: aws.Bool(false),
+		},
+	}
+	backend := newTestBackend(t, "root", client)
+	found, err := backend.HasAny(context.Background(), "")
+	if err != nil {
+		t.Fatalf("HasAny() error = %v", err)
+	}
+	if !found {
+		t.Fatal("HasAny() = false, want true")
+	}
+	if got, want := len(client.tokens), 1; got != want {
+		t.Fatalf("list calls = %d, want %d", got, want)
+	}
+}
+
+func TestWalkNonRecursiveUsesPrefixBoundary(t *testing.T) {
+	client := newFakeClient(map[string]string{
+		"base/dir/file.txt": "nested",
+		"base/file.txt":     "file",
+		"baseball/file.txt": "wrong",
+	})
+	backend := newTestBackend(t, "base", client)
+	entries, err := storage.List(context.Background(), backend, "", storage.WalkOptions{})
+	if err != nil {
+		t.Fatalf("List() error = %v", err)
+	}
+	paths := entryPaths(entries)
+	if got, want := paths, []string{"dir", "file.txt"}; !equalStrings(got, want) {
+		t.Fatalf("paths = %v, want %v", got, want)
+	}
+}
+
+func TestDeleteManagedBundleDeletesOnlyManagedTargets(t *testing.T) {
+	client := newFakeClient(map[string]string{
+		"root/report.md":         "report",
+		"root/.distributor.json": "state",
+		"root/keep.txt":          "keep",
+	})
+	backend := newTestBackend(t, "root", client)
+	err := backend.DeleteManagedBundle(context.Background(), "", []string{"report.md"}, storage.DeleteOptions{IgnoreMissing: true})
+	if err != nil {
+		t.Fatalf("DeleteManagedBundle() error = %v", err)
+	}
+	if _, ok := client.objects["root/report.md"]; ok {
+		t.Fatal("managed output still exists")
+	}
+	if _, ok := client.objects["root/.distributor.json"]; ok {
+		t.Fatal("state file still exists")
+	}
+	if _, ok := client.objects["root/keep.txt"]; !ok {
+		t.Fatal("unmanaged object was deleted")
+	}
+	if got, want := sortedStrings(client.deleteKeys), []string{"root/.distributor.json", "root/report.md"}; !equalStrings(got, want) {
+		t.Fatalf("deleted keys = %v, want %v", got, want)
+	}
+}
+
+func TestDeletePrefixStaysWithinPrefix(t *testing.T) {
+	client := newFakeClient(map[string]string{
+		"root/bundle/report.md":           "report",
+		"root/bundle/nested/old.txt":      "old",
+		"root/bundle-sibling/keep.txt":    "keep",
+		"root/outside.txt":                "outside",
+		"other-root/bundle/report.md":     "other",
+		"root/.distributor-prefix-marker": "marker",
+	})
+	backend := newTestBackend(t, "root", client)
+	if err := backend.DeletePrefix(context.Background(), "bundle", storage.DeleteOptions{IgnoreMissing: true}); err != nil {
+		t.Fatalf("DeletePrefix() error = %v", err)
+	}
+	for _, deleted := range []string{"root/bundle/report.md", "root/bundle/nested/old.txt"} {
+		if _, ok := client.objects[deleted]; ok {
+			t.Fatalf("%s still exists", deleted)
+		}
+	}
+	for _, kept := range []string{"root/bundle-sibling/keep.txt", "root/outside.txt", "other-root/bundle/report.md", "root/.distributor-prefix-marker"} {
+		if _, ok := client.objects[kept]; !ok {
+			t.Fatalf("%s was deleted", kept)
+		}
+	}
+	if got, want := sortedStrings(client.deleteKeys), []string{"root/bundle/nested/old.txt", "root/bundle/report.md"}; !equalStrings(got, want) {
+		t.Fatalf("deleted keys = %v, want %v", got, want)
+	}
+}
+
+func newTestBackend(t *testing.T, prefix string, client *fakeClient) *Backend {
+	t.Helper()
+	if client == nil {
+		client = newFakeClient(nil)
+	}
+	backend, err := NewWithClient(client, Options{
+		Endpoint:       "https://s3.example.com",
+		Bucket:         "reports",
+		Prefix:         prefix,
+		Region:         DefaultRegion,
+		ForcePathStyle: true,
+	})
+	if err != nil {
+		t.Fatalf("NewWithClient() error = %v", err)
+	}
+	return backend
+}
+
+type fakeClient struct {
+	objects      map[string]string
+	contentTypes map[string]string
+	listPages    []awss3.ListObjectsV2Output
+	tokens       []string
+	putKeys      []string
+	deleteKeys   []string
+}
+
+func newFakeClient(objects map[string]string) *fakeClient {
+	copied := make(map[string]string)
+	for key, value := range objects {
+		copied[key] = value
+	}
+	return &fakeClient{
+		objects:      copied,
+		contentTypes: make(map[string]string),
+	}
+}
+
+func (c *fakeClient) HeadObject(ctx context.Context, input *awss3.HeadObjectInput, optFns ...func(*awss3.Options)) (*awss3.HeadObjectOutput, error) {
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+	value, ok := c.objects[aws.ToString(input.Key)]
+	if !ok {
+		return nil, &types.NotFound{}
+	}
+	return &awss3.HeadObjectOutput{ContentLength: aws.Int64(int64(len(value)))}, nil
+}
+
+func (c *fakeClient) GetObject(ctx context.Context, input *awss3.GetObjectInput, optFns ...func(*awss3.Options)) (*awss3.GetObjectOutput, error) {
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+	value, ok := c.objects[aws.ToString(input.Key)]
+	if !ok {
+		return nil, &types.NotFound{}
+	}
+	return &awss3.GetObjectOutput{
+		Body:          io.NopCloser(stringsReader(value)),
+		ContentLength: aws.Int64(int64(len(value))),
+	}, nil
+}
+
+func (c *fakeClient) PutObject(ctx context.Context, input *awss3.PutObjectInput, optFns ...func(*awss3.Options)) (*awss3.PutObjectOutput, error) {
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+	data, err := io.ReadAll(input.Body)
+	if err != nil {
+		return nil, err
+	}
+	key := aws.ToString(input.Key)
+	c.objects[key] = string(data)
+	c.contentTypes[key] = aws.ToString(input.ContentType)
+	c.putKeys = append(c.putKeys, key)
+	return &awss3.PutObjectOutput{}, nil
+}
+
+func (c *fakeClient) ListObjectsV2(ctx context.Context, input *awss3.ListObjectsV2Input, optFns ...func(*awss3.Options)) (*awss3.ListObjectsV2Output, error) {
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+	c.tokens = append(c.tokens, aws.ToString(input.ContinuationToken))
+	if len(c.listPages) > 0 {
+		index := len(c.tokens) - 1
+		if index >= len(c.listPages) {
+			return &awss3.ListObjectsV2Output{IsTruncated: aws.Bool(false)}, nil
+		}
+		page := c.listPages[index]
+		return &page, nil
+	}
+	return c.dynamicList(input), nil
+}
+
+func (c *fakeClient) DeleteObject(ctx context.Context, input *awss3.DeleteObjectInput, optFns ...func(*awss3.Options)) (*awss3.DeleteObjectOutput, error) {
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+	key := aws.ToString(input.Key)
+	delete(c.objects, key)
+	c.deleteKeys = append(c.deleteKeys, key)
+	return &awss3.DeleteObjectOutput{}, nil
+}
+
+func (c *fakeClient) dynamicList(input *awss3.ListObjectsV2Input) *awss3.ListObjectsV2Output {
+	prefix := aws.ToString(input.Prefix)
+	delimiter := aws.ToString(input.Delimiter)
+	var contents []types.Object
+	commonPrefixes := make(map[string]struct{})
+	for key, value := range c.objects {
+		if !strings.HasPrefix(key, prefix) {
+			continue
+		}
+		remainder := strings.TrimPrefix(key, prefix)
+		if delimiter != "" {
+			if index := strings.Index(remainder, delimiter); index >= 0 {
+				commonPrefixes[prefix+remainder[:index+1]] = struct{}{}
+				continue
+			}
+		}
+		contents = append(contents, types.Object{Key: aws.String(key), Size: aws.Int64(int64(len(value)))})
+	}
+	sort.Slice(contents, func(i, j int) bool { return aws.ToString(contents[i].Key) < aws.ToString(contents[j].Key) })
+	prefixes := make([]types.CommonPrefix, 0, len(commonPrefixes))
+	for prefix := range commonPrefixes {
+		prefixes = append(prefixes, types.CommonPrefix{Prefix: aws.String(prefix)})
+	}
+	sort.Slice(prefixes, func(i, j int) bool { return aws.ToString(prefixes[i].Prefix) < aws.ToString(prefixes[j].Prefix) })
+	return &awss3.ListObjectsV2Output{
+		Contents:       contents,
+		CommonPrefixes: prefixes,
+		IsTruncated:    aws.Bool(false),
+	}
+}
+
+func stringsReader(value string) io.Reader {
+	return strings.NewReader(value)
+}
+
+func entryPaths(entries []storage.Entry) []string {
+	paths := make([]string, 0, len(entries))
+	for _, entry := range entries {
+		paths = append(paths, entry.Path)
+	}
+	return paths
+}
+
+func sortedStrings(values []string) []string {
+	copied := append([]string(nil), values...)
+	sort.Strings(copied)
+	return copied
+}
+
+func equalStrings(a, b []string) bool {
+	if len(a) != len(b) {
+		return false
+	}
+	for index := range a {
+		if a[index] != b[index] {
+			return false
+		}
+	}
+	return true
+}

internal/adapters/s3/integration_test.go

@@ -0,0 +1,39 @@
+package s3
+
+import (
+	"context"
+	"os"
+	"strconv"
+	"testing"
+)
+
+func TestIntegrationS3BackendHasAny(t *testing.T) {
+	endpoint := os.Getenv("DISTRIBUTOR_TEST_S3_ENDPOINT")
+	bucket := os.Getenv("DISTRIBUTOR_TEST_S3_BUCKET")
+	if endpoint == "" || bucket == "" {
+		t.Skip("DISTRIBUTOR_TEST_S3_ENDPOINT and DISTRIBUTOR_TEST_S3_BUCKET are not set")
+	}
+	forcePathStyle := true
+	if raw := os.Getenv("DISTRIBUTOR_TEST_S3_FORCE_PATH_STYLE"); raw != "" {
+		parsed, err := strconv.ParseBool(raw)
+		if err != nil {
+			t.Fatalf("parse DISTRIBUTOR_TEST_S3_FORCE_PATH_STYLE: %v", err)
+		}
+		forcePathStyle = parsed
+	}
+	backend, err := New(context.Background(), Options{
+		Endpoint:        endpoint,
+		Bucket:          bucket,
+		Prefix:          os.Getenv("DISTRIBUTOR_TEST_S3_PREFIX"),
+		Region:          os.Getenv("DISTRIBUTOR_TEST_S3_REGION"),
+		ForcePathStyle:  forcePathStyle,
+		AccessKeyID:     os.Getenv("DISTRIBUTOR_TEST_S3_ACCESS_KEY_ID"),
+		SecretAccessKey: os.Getenv("DISTRIBUTOR_TEST_S3_SECRET_ACCESS_KEY"),
+	})
+	if err != nil {
+		t.Fatalf("New() error = %v", err)
+	}
+	if _, err := backend.HasAny(context.Background(), ""); err != nil {
+		t.Fatalf("HasAny(root) error = %v", err)
+	}
+}

internal/adapters/s3/options.go

@@ -0,0 +1,42 @@
+package s3
+
+import (
+	"fmt"
+	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+)
+
+const BackendName = "s3"
+
+const DefaultRegion = "us-east-1"
+
+type Options struct {
+	Endpoint        string
+	Bucket          string
+	Prefix          string
+	Region          string
+	ForcePathStyle  bool
+	AccessKeyID     string
+	SecretAccessKey string
+}
+
+func (o Options) normalized() (Options, error) {
+	if o.Endpoint == "" {
+		return Options{}, fmt.Errorf("endpoint is required")
+	}
+	if o.Bucket == "" {
+		return Options{}, fmt.Errorf("bucket is required")
+	}
+	if o.Region == "" {
+		o.Region = DefaultRegion
+	}
+	o.Prefix = strings.Trim(o.Prefix, "/")
+	if err := storage.ValidatePrefix(o.Prefix); err != nil {
+		return Options{}, fmt.Errorf("prefix: %w", err)
+	}
+	if (o.AccessKeyID == "") != (o.SecretAccessKey == "") {
+		return Options{}, fmt.Errorf("access key id and secret access key must be configured together")
+	}
+	return o, nil
+}

internal/adapters/ssh/auth.go

@@ -0,0 +1,61 @@
+package ssh
+
+import (
+	"fmt"
+	"io"
+	"net"
+	"os"
+
+	cryptossh "golang.org/x/crypto/ssh"
+	"golang.org/x/crypto/ssh/agent"
+)
+
+type agentDialer func(network, address string) (net.Conn, error)
+
+func authMethods(keyFile string) ([]cryptossh.AuthMethod, func(), error) {
+	return authMethodsWithAgent(os.Getenv("SSH_AUTH_SOCK"), net.Dial, keyFile)
+}
+
+func authMethodsWithAgent(agentSocket string, dial agentDialer, keyFile string) ([]cryptossh.AuthMethod, func(), error) {
+	var methods []cryptossh.AuthMethod
+	var closers []io.Closer
+	if agentSocket != "" {
+		methods = append(methods, cryptossh.PublicKeysCallback(func() ([]cryptossh.Signer, error) {
+			conn, err := dial("unix", agentSocket)
+			if err != nil {
+				return nil, err
+			}
+			closers = append(closers, conn)
+			return agent.NewClient(conn).Signers()
+		}))
+	}
+	if keyFile != "" {
+		signer, err := signerFromKeyFile(keyFile)
+		if err != nil {
+			return nil, nil, err
+		}
+		methods = append(methods, cryptossh.PublicKeys(signer))
+	}
+	if len(methods) == 0 {
+		return nil, nil, fmt.Errorf("no SSH auth methods configured; set SSH_AUTH_SOCK or ssh_key_file")
+	}
+	return methods, func() { closeAll(closers) }, nil
+}
+
+func signerFromKeyFile(path string) (cryptossh.Signer, error) {
+	key, err := os.ReadFile(path)
+	if err != nil {
+		return nil, fmt.Errorf("read ssh_key_file %q: %w", path, err)
+	}
+	signer, err := cryptossh.ParsePrivateKey(key)
+	if err != nil {
+		return nil, fmt.Errorf("parse ssh_key_file %q: %w", path, err)
+	}
+	return signer, nil
+}
+
+func closeAll(closers []io.Closer) {
+	for _, closer := range closers {
+		_ = closer.Close()
+	}
+}

internal/adapters/ssh/auth_test.go

@@ -0,0 +1,59 @@
+package ssh
+
+import (
+	"crypto/rand"
+	"crypto/rsa"
+	"crypto/x509"
+	"encoding/pem"
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+func TestAuthMethodsPreferAgentBeforeKeyFile(t *testing.T) {
+	keyFile := writePrivateKey(t)
+	methods, cleanup, err := authMethodsWithAgent("/tmp/ssh-agent.sock", nil, keyFile)
+	if err != nil {
+		t.Fatalf("authMethodsWithAgent() error = %v", err)
+	}
+	defer cleanup()
+	if got, want := len(methods), 2; got != want {
+		t.Fatalf("auth method count = %d, want %d", got, want)
+	}
+}
+
+func TestAuthMethodsLoadsKeyFile(t *testing.T) {
+	keyFile := writePrivateKey(t)
+	methods, cleanup, err := authMethodsWithAgent("", nil, keyFile)
+	if err != nil {
+		t.Fatalf("authMethodsWithAgent() error = %v", err)
+	}
+	defer cleanup()
+	if got, want := len(methods), 1; got != want {
+		t.Fatalf("auth method count = %d, want %d", got, want)
+	}
+}
+
+func TestAuthMethodsRejectsMissingAuth(t *testing.T) {
+	_, _, err := authMethodsWithAgent("", nil, "")
+	if err == nil {
+		t.Fatal("authMethodsWithAgent() error = nil, want error")
+	}
+}
+
+func writePrivateKey(t *testing.T) string {
+	t.Helper()
+	privateKey, err := rsa.GenerateKey(rand.Reader, 2048)
+	if err != nil {
+		t.Fatalf("generate key: %v", err)
+	}
+	data := pem.EncodeToMemory(&pem.Block{
+		Type:  "RSA PRIVATE KEY",
+		Bytes: x509.MarshalPKCS1PrivateKey(privateKey),
+	})
+	path := filepath.Join(t.TempDir(), "id_rsa")
+	if err := os.WriteFile(path, data, 0o600); err != nil {
+		t.Fatalf("write private key: %v", err)
+	}
+	return path
+}

internal/adapters/ssh/backend.go

@@ -0,0 +1,487 @@
+package ssh
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"io/fs"
+	"os"
+	"path"
+	"sort"
+	"strings"
+	"time"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+	"github.com/pkg/sftp"
+	cryptossh "golang.org/x/crypto/ssh"
+)
+
+type Backend struct {
+	client    *sftp.Client
+	sshClient *cryptossh.Client
+	root      string
+}
+
+func New(ctx context.Context, options Options) (*Backend, error) {
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+	options, err := options.normalized()
+	if err != nil {
+		return nil, storage.NewError(storage.OpOpenBackend, BackendName, options.Root, storage.ErrInvalidPath, err)
+	}
+	hostKeyCallback, err := hostKeyCallback(options)
+	if err != nil {
+		return nil, storage.NewError(storage.OpOpenBackend, BackendName, options.KnownHosts, storage.ErrInvalidPath, err)
+	}
+	auth, cleanupAuth, err := authMethods(options.KeyFile)
+	if err != nil {
+		return nil, storage.NewError(storage.OpOpenBackend, BackendName, options.KeyFile, storage.ErrPermission, err)
+	}
+	defer cleanupAuth()
+
+	sshClient, err := cryptossh.Dial("tcp", options.address(), &cryptossh.ClientConfig{
+		User:            options.User,
+		Auth:            auth,
+		HostKeyCallback: hostKeyCallback,
+		Timeout:         30 * time.Second,
+	})
+	if err != nil {
+		return nil, storage.NewError(storage.OpOpenBackend, BackendName, options.address(), storage.ErrUnknown, err)
+	}
+	client, err := sftp.NewClient(sshClient)
+	if err != nil {
+		_ = sshClient.Close()
+		return nil, storage.NewError(storage.OpOpenBackend, BackendName, options.address(), storage.ErrUnknown, err)
+	}
+	return &Backend{client: client, sshClient: sshClient, root: options.Root}, nil
+}
+
+func (b *Backend) Close() error {
+	var err error
+	if b.client != nil {
+		err = b.client.Close()
+	}
+	if b.sshClient != nil {
+		if closeErr := b.sshClient.Close(); err == nil {
+			err = closeErr
+		}
+	}
+	return err
+}
+
+func (b *Backend) ReadFile(ctx context.Context, logicalPath string) ([]byte, error) {
+	reader, err := b.OpenReader(ctx, logicalPath)
+	if err != nil {
+		return nil, err
+	}
+	defer reader.Close()
+	data, err := io.ReadAll(reader)
+	if err != nil {
+		return nil, storage.NewError(storage.OpReadFile, BackendName, logicalPath, storage.ErrUnknown, err)
+	}
+	return data, nil
+}
+
+func (b *Backend) OpenReader(ctx context.Context, logicalPath string) (io.ReadCloser, error) {
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+	nativePath, err := b.nativePath(logicalPath, false)
+	if err != nil {
+		return nil, err
+	}
+	if err := b.rejectSymlinkAncestors(ctx, logicalPath, true); err != nil {
+		return nil, err
+	}
+	info, err := b.client.Lstat(nativePath)
+	if err != nil {
+		return nil, b.translateError(storage.OpOpenReader, logicalPath, err)
+	}
+	if !info.Mode().IsRegular() {
+		return nil, storage.NewError(storage.OpOpenReader, BackendName, logicalPath, storage.ErrUnsupported, nil)
+	}
+	file, err := b.client.Open(nativePath)
+	if err != nil {
+		return nil, b.translateError(storage.OpOpenReader, logicalPath, err)
+	}
+	return file, nil
+}
+
+func (b *Backend) WriteFile(ctx context.Context, logicalPath string, data []byte, opts storage.WriteOptions) (storage.Entry, error) {
+	opts.Size = int64(len(data))
+	opts.SizeKnown = true
+	return b.WriteFrom(ctx, logicalPath, bytes.NewReader(data), opts)
+}
+
+func (b *Backend) WriteFrom(ctx context.Context, logicalPath string, r io.Reader, opts storage.WriteOptions) (storage.Entry, error) {
+	if err := ctx.Err(); err != nil {
+		return storage.Entry{}, err
+	}
+	nativePath, err := b.nativePath(logicalPath, false)
+	if err != nil {
+		return storage.Entry{}, err
+	}
+	if err := b.rejectSymlinkAncestors(ctx, parentOf(logicalPath), true); err != nil {
+		return storage.Entry{}, err
+	}
+	if info, err := b.client.Lstat(nativePath); err == nil {
+		if !opts.Overwrite {
+			return storage.Entry{}, storage.NewError(storage.OpWriteFrom, BackendName, logicalPath, storage.ErrAlreadyExist, nil)
+		}
+		if !info.Mode().IsRegular() {
+			return storage.Entry{}, storage.NewError(storage.OpWriteFrom, BackendName, logicalPath, storage.ErrConflict, nil)
+		}
+	} else if !isNotExist(err) {
+		return storage.Entry{}, b.translateError(storage.OpWriteFrom, logicalPath, err)
+	}
+
+	parentNative := path.Dir(nativePath)
+	if err := b.client.MkdirAll(parentNative); err != nil {
+		return storage.Entry{}, b.translateError(storage.OpWriteFrom, logicalPath, err)
+	}
+
+	writePath := nativePath
+	if opts.PreferAtomic {
+		writePath = path.Join(parentNative, fmt.Sprintf(".distributor-write-%d", time.Now().UnixNano()))
+	}
+	file, err := b.client.Create(writePath)
+	if err != nil {
+		return storage.Entry{}, b.translateError(storage.OpWriteFrom, logicalPath, err)
+	}
+	cleanup := opts.PreferAtomic
+	defer func() {
+		if cleanup {
+			_ = b.client.Remove(writePath)
+		}
+	}()
+
+	written, copyErr := io.Copy(file, r)
+	closeErr := file.Close()
+	if copyErr != nil {
+		return storage.Entry{}, storage.NewError(storage.OpWriteFrom, BackendName, logicalPath, storage.ErrUnknown, copyErr)
+	}
+	if closeErr != nil {
+		return storage.Entry{}, storage.NewError(storage.OpWriteFrom, BackendName, logicalPath, storage.ErrUnknown, closeErr)
+	}
+	if opts.SizeKnown && written != opts.Size {
+		return storage.Entry{}, storage.NewError(storage.OpWriteFrom, BackendName, logicalPath, storage.ErrConflict, fmt.Errorf("stream size %d does not match expected size %d", written, opts.Size))
+	}
+	if opts.PreferAtomic {
+		if err := b.client.Rename(writePath, nativePath); err != nil {
+			return storage.Entry{}, b.translateError(storage.OpWriteFrom, logicalPath, err)
+		}
+		cleanup = false
+	}
+	return b.Stat(ctx, logicalPath)
+}
+
+func (b *Backend) Stat(ctx context.Context, logicalPath string) (storage.Entry, error) {
+	if err := ctx.Err(); err != nil {
+		return storage.Entry{}, err
+	}
+	nativePath, err := b.nativePath(logicalPath, true)
+	if err != nil {
+		return storage.Entry{}, err
+	}
+	info, err := b.client.Lstat(nativePath)
+	if err != nil {
+		return storage.Entry{}, b.translateError(storage.OpStat, logicalPath, err)
+	}
+	return entryFromInfo(logicalPath, info), nil
+}
+
+func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOptions, fn storage.WalkFunc) error {
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	nativePrefix, err := b.nativePath(prefix, true)
+	if err != nil {
+		return err
+	}
+	info, err := b.client.Lstat(nativePrefix)
+	if err != nil {
+		if isNotExist(err) {
+			return nil
+		}
+		return b.translateError(storage.OpWalk, prefix, err)
+	}
+
+	emitter := storage.NewWalkEmitter(ctx, BackendName, opts, fn)
+
+	if !info.IsDir() {
+		return storage.FinishWalk(emitter.Emit(entryFromInfo(prefix, info)))
+	}
+
+	return storage.FinishWalk(b.walkDirectory(ctx, prefix, nativePrefix, opts, emitter.Emit))
+}
+
+func (b *Backend) HasAny(ctx context.Context, prefix string) (bool, error) {
+	return storage.HasAny(ctx, b, prefix)
+}
+
+func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	targets, err := storage.ManagedBundleTargets(bundlePath, managedOutputPaths)
+	if err != nil {
+		return err
+	}
+	for _, target := range targets {
+		nativePath, err := b.nativePath(target, false)
+		if err != nil {
+			return err
+		}
+		if nativePath == b.root {
+			return storage.NewError(storage.OpDeleteManagedBundle, BackendName, target, storage.ErrInvalidPath, nil)
+		}
+		info, err := b.client.Lstat(nativePath)
+		if err != nil {
+			if opts.IgnoreMissing && isNotExist(err) {
+				continue
+			}
+			return b.translateError(storage.OpDeleteManagedBundle, target, err)
+		}
+		if info.IsDir() {
+			return storage.NewError(storage.OpDeleteManagedBundle, BackendName, target, storage.ErrUnsupported, nil)
+		}
+		if err := b.client.Remove(nativePath); err != nil {
+			return b.translateError(storage.OpDeleteManagedBundle, target, err)
+		}
+		if opts.PruneEmptyDirs {
+			b.pruneEmptyParents(parentOf(target))
+		}
+	}
+	return nil
+}
+
+func (b *Backend) DeletePrefix(ctx context.Context, prefix string, opts storage.DeleteOptions) error {
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	if err := storage.ValidatePrefix(prefix); err != nil {
+		return err
+	}
+	var entries []storage.Entry
+	if prefix != "" {
+		entry, err := b.Stat(ctx, prefix)
+		if err != nil {
+			if opts.IgnoreMissing && storage.IsNotFound(err) {
+				return nil
+			}
+			return err
+		}
+		if entry.Type != storage.EntryTypeDirectory {
+			return b.deleteEntry(ctx, entry, opts)
+		}
+		entries = append(entries, entry)
+	}
+	if err := b.Walk(ctx, prefix, storage.WalkOptions{Recursive: true}, func(entry storage.Entry) error {
+		entries = append(entries, entry)
+		return nil
+	}); err != nil {
+		return err
+	}
+	if prefix != "" && len(entries) == 1 {
+		if err := b.deleteEntry(ctx, entries[0], opts); err != nil {
+			return err
+		}
+		if opts.PruneEmptyDirs {
+			b.pruneEmptyParents(parentOf(prefix))
+		}
+		return nil
+	}
+	sort.Slice(entries, func(i, j int) bool {
+		return strings.Count(entries[i].Path, "/") > strings.Count(entries[j].Path, "/")
+	})
+	for _, entry := range entries {
+		if entry.Path == "" {
+			continue
+		}
+		if err := b.deleteEntry(ctx, entry, storage.DeleteOptions{IgnoreMissing: true}); err != nil {
+			return err
+		}
+	}
+	if opts.PruneEmptyDirs {
+		b.pruneEmptyParents(parentOf(prefix))
+	}
+	return nil
+}
+
+func (b *Backend) deleteEntry(ctx context.Context, entry storage.Entry, opts storage.DeleteOptions) error {
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	nativePath, err := b.nativePath(entry.Path, false)
+	if err != nil {
+		return err
+	}
+	var removeErr error
+	if entry.Type == storage.EntryTypeDirectory {
+		removeErr = b.client.RemoveDirectory(nativePath)
+	} else {
+		removeErr = b.client.Remove(nativePath)
+	}
+	if removeErr != nil {
+		if opts.IgnoreMissing && isNotExist(removeErr) {
+			return nil
+		}
+		return b.translateError(storage.OpDeletePrefix, entry.Path, removeErr)
+	}
+	return nil
+}
+
+func (b *Backend) walkDirectory(ctx context.Context, logicalPrefix, nativePrefix string, opts storage.WalkOptions, emit func(storage.Entry) error) error {
+	entries, err := b.client.ReadDir(nativePrefix)
+	if err != nil {
+		return b.translateError(storage.OpWalk, logicalPrefix, err)
+	}
+	sort.Slice(entries, func(i, j int) bool { return entries[i].Name() < entries[j].Name() })
+	for _, info := range entries {
+		if err := ctx.Err(); err != nil {
+			return err
+		}
+		logicalPath := info.Name()
+		if logicalPrefix != "" {
+			logicalPath = logicalPrefix + "/" + info.Name()
+		}
+		if err := emit(entryFromInfo(logicalPath, info)); err != nil {
+			return err
+		}
+		if opts.Recursive && info.IsDir() {
+			if err := b.walkDirectory(ctx, logicalPath, path.Join(nativePrefix, info.Name()), opts, emit); err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+func (b *Backend) nativePath(logicalPath string, allowEmpty bool) (string, error) {
+	if logicalPath == "" {
+		if !allowEmpty {
+			return "", storage.NewError(storage.OpValidatePath, BackendName, logicalPath, storage.ErrInvalidPath, nil)
+		}
+		return b.root, nil
+	}
+	if err := storage.ValidatePath(logicalPath); err != nil {
+		return "", err
+	}
+	nativePath := path.Clean(path.Join(b.root, logicalPath))
+	if !withinRoot(b.root, nativePath) {
+		return "", storage.NewError(storage.OpValidatePath, BackendName, logicalPath, storage.ErrInvalidPath, nil)
+	}
+	return nativePath, nil
+}
+
+func (b *Backend) rejectSymlinkAncestors(ctx context.Context, logicalPath string, includeFinal bool) error {
+	if logicalPath == "" {
+		return nil
+	}
+	if err := storage.ValidatePath(logicalPath); err != nil {
+		return err
+	}
+	segments := strings.Split(logicalPath, "/")
+	limit := len(segments)
+	if !includeFinal {
+		limit--
+	}
+	current := ""
+	for index := 0; index < limit; index++ {
+		if err := ctx.Err(); err != nil {
+			return err
+		}
+		if current == "" {
+			current = segments[index]
+		} else {
+			current += "/" + segments[index]
+		}
+		nativePath, err := b.nativePath(current, false)
+		if err != nil {
+			return err
+		}
+		info, err := b.client.Lstat(nativePath)
+		if err != nil {
+			if isNotExist(err) {
+				return nil
+			}
+			return b.translateError(storage.OpStat, current, err)
+		}
+		if info.Mode()&os.ModeSymlink != 0 {
+			return storage.NewError(storage.OpStat, BackendName, current, storage.ErrUnsupported, nil)
+		}
+	}
+	return nil
+}
+
+func (b *Backend) pruneEmptyParents(logicalPath string) {
+	for logicalPath != "" {
+		nativePath, err := b.nativePath(logicalPath, false)
+		if err != nil || nativePath == b.root {
+			return
+		}
+		if err := b.client.RemoveDirectory(nativePath); err != nil {
+			return
+		}
+		logicalPath = parentOf(logicalPath)
+	}
+}
+
+func withinRoot(root, candidate string) bool {
+	if candidate == root {
+		return true
+	}
+	if root == "/" {
+		return strings.HasPrefix(candidate, "/")
+	}
+	return strings.HasPrefix(candidate, strings.TrimSuffix(root, "/")+"/")
+}
+
+func parentOf(logicalPath string) string {
+	index := strings.LastIndex(logicalPath, "/")
+	if index == -1 {
+		return ""
+	}
+	return logicalPath[:index]
+}
+
+func isNotExist(err error) bool {
+	return errors.Is(err, fs.ErrNotExist) || errors.Is(err, os.ErrNotExist) || errors.Is(err, sftp.ErrSSHFxNoSuchFile)
+}
+
+func (b *Backend) translateError(op, logicalPath string, err error) error {
+	kind := storage.ErrUnknown
+	switch {
+	case isNotExist(err):
+		kind = storage.ErrNotFound
+	case errors.Is(err, fs.ErrExist), errors.Is(err, os.ErrExist):
+		kind = storage.ErrAlreadyExist
+	case errors.Is(err, fs.ErrPermission), errors.Is(err, os.ErrPermission), errors.Is(err, sftp.ErrSSHFxPermissionDenied):
+		kind = storage.ErrPermission
+	case errors.Is(err, sftp.ErrSSHFxOpUnsupported):
+		kind = storage.ErrUnsupported
+	case errors.Is(err, sftp.ErrSSHFxNoConnection), errors.Is(err, sftp.ErrSSHFxConnectionLost):
+		kind = storage.ErrTemporary
+	}
+	return storage.NewError(op, BackendName, logicalPath, kind, err)
+}
+
+func entryFromInfo(logicalPath string, info fs.FileInfo) storage.Entry {
+	entryType := storage.EntryTypeOther
+	switch {
+	case info.Mode()&os.ModeSymlink != 0:
+		entryType = storage.EntryTypeSymlink
+	case info.Mode().IsRegular():
+		entryType = storage.EntryTypeFile
+	case info.IsDir():
+		entryType = storage.EntryTypeDirectory
+	}
+	return storage.Entry{
+		Path: logicalPath,
+		Type: entryType,
+		Size: info.Size(),
+	}
+}

internal/adapters/ssh/hostkeys.go

@@ -0,0 +1,84 @@
+package ssh
+
+import (
+	"errors"
+	"fmt"
+	"net"
+	"os"
+
+	cryptossh "golang.org/x/crypto/ssh"
+	"golang.org/x/crypto/ssh/knownhosts"
+)
+
+func hostKeyCallback(options Options) (cryptossh.HostKeyCallback, error) {
+	switch options.HostKeyPolicy {
+	case HostKeyPolicyOff:
+		return cryptossh.InsecureIgnoreHostKey(), nil
+	case HostKeyPolicyStrict:
+		if options.KnownHosts == "" {
+			return nil, fmt.Errorf("known_hosts is required for strict host key checking")
+		}
+		callback, err := knownhosts.New(options.KnownHosts)
+		if err != nil {
+			return nil, fmt.Errorf("load known_hosts %q: %w", options.KnownHosts, err)
+		}
+		return callback, nil
+	case HostKeyPolicyAcceptNew:
+		return acceptNewHostKeyCallback(options)
+	default:
+		return nil, fmt.Errorf("host_key_policy must be strict, accept-new, or off")
+	}
+}
+
+func acceptNewHostKeyCallback(options Options) (cryptossh.HostKeyCallback, error) {
+	var checker cryptossh.HostKeyCallback
+	if options.KnownHosts != "" {
+		loaded, err := knownhosts.New(options.KnownHosts)
+		if err == nil {
+			checker = loaded
+		} else if !errors.Is(err, os.ErrNotExist) {
+			return nil, fmt.Errorf("load known_hosts %q: %w", options.KnownHosts, err)
+		}
+	}
+	return func(hostname string, remote net.Addr, key cryptossh.PublicKey) error {
+		if checker != nil {
+			err := checker(hostname, remote, key)
+			if err == nil {
+				return nil
+			}
+			var keyErr *knownhosts.KeyError
+			if !errors.As(err, &keyErr) {
+				return err
+			}
+			if len(keyErr.Want) > 0 {
+				return fmt.Errorf("host key for %s has changed: %w", hostname, err)
+			}
+		}
+		if options.ReadOnlyKnownHosts {
+			return nil
+		}
+		if options.KnownHosts == "" {
+			return fmt.Errorf("host key for %s is unknown and no writable known_hosts path is available", hostname)
+		}
+		if err := appendKnownHost(options.KnownHosts, hostname, key); err != nil {
+			return err
+		}
+		loaded, err := knownhosts.New(options.KnownHosts)
+		if err == nil {
+			checker = loaded
+		}
+		return nil
+	}, nil
+}
+
+func appendKnownHost(path, host string, key cryptossh.PublicKey) error {
+	file, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o600)
+	if err != nil {
+		return fmt.Errorf("persist accepted host key to known_hosts %q: %w", path, err)
+	}
+	defer file.Close()
+	if _, err := fmt.Fprintln(file, knownhosts.Line([]string{knownhosts.Normalize(host)}, key)); err != nil {
+		return fmt.Errorf("persist accepted host key to known_hosts %q: %w", path, err)
+	}
+	return nil
+}

internal/adapters/ssh/hostkeys_test.go

@@ -0,0 +1,138 @@
+package ssh
+
+import (
+	"crypto/rand"
+	"crypto/rsa"
+	"net"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	cryptossh "golang.org/x/crypto/ssh"
+	"golang.org/x/crypto/ssh/knownhosts"
+)
+
+func TestAcceptNewHostKeyCallbackPersistsUnknownHost(t *testing.T) {
+	key := testPublicKey(t)
+	knownHosts := filepath.Join(t.TempDir(), "known_hosts")
+	callback, err := acceptNewHostKeyCallback(Options{KnownHosts: knownHosts})
+	if err != nil {
+		t.Fatalf("acceptNewHostKeyCallback() error = %v", err)
+	}
+
+	if err := callback("example.com:22", &net.TCPAddr{IP: net.ParseIP("192.0.2.10"), Port: 22}, key); err != nil {
+		t.Fatalf("callback() error = %v", err)
+	}
+	data, err := os.ReadFile(knownHosts)
+	if err != nil {
+		t.Fatalf("read known_hosts: %v", err)
+	}
+	if !strings.Contains(string(data), "example.com") {
+		t.Fatalf("known_hosts = %q, want example.com entry", data)
+	}
+	if err := callback("example.com:22", &net.TCPAddr{IP: net.ParseIP("192.0.2.10"), Port: 22}, key); err != nil {
+		t.Fatalf("second callback() error = %v", err)
+	}
+}
+
+func TestAcceptNewHostKeyCallbackReadOnlyDoesNotPersistUnknownHost(t *testing.T) {
+	key := testPublicKey(t)
+	knownHosts := filepath.Join(t.TempDir(), "known_hosts")
+	callback, err := acceptNewHostKeyCallback(Options{
+		KnownHosts:         knownHosts,
+		ReadOnlyKnownHosts: true,
+	})
+	if err != nil {
+		t.Fatalf("acceptNewHostKeyCallback() error = %v", err)
+	}
+
+	if err := callback("example.com:22", &net.TCPAddr{IP: net.ParseIP("192.0.2.10"), Port: 22}, key); err != nil {
+		t.Fatalf("callback() error = %v", err)
+	}
+	if _, err := os.Stat(knownHosts); !os.IsNotExist(err) {
+		t.Fatalf("known_hosts stat error = %v, want not exist", err)
+	}
+}
+
+func TestAcceptNewHostKeyCallbackRejectsChangedHostKey(t *testing.T) {
+	first := testPublicKey(t)
+	second := testPublicKey(t)
+	knownHosts := filepath.Join(t.TempDir(), "known_hosts")
+	if err := os.WriteFile(knownHosts, []byte(knownhosts.Line([]string{knownhosts.Normalize("example.com:22")}, first)+"\n"), 0o600); err != nil {
+		t.Fatalf("write known_hosts: %v", err)
+	}
+	callback, err := acceptNewHostKeyCallback(Options{KnownHosts: knownHosts})
+	if err != nil {
+		t.Fatalf("acceptNewHostKeyCallback() error = %v", err)
+	}
+
+	err = callback("example.com:22", &net.TCPAddr{IP: net.ParseIP("192.0.2.10"), Port: 22}, second)
+	if err == nil || !strings.Contains(err.Error(), "has changed") {
+		t.Fatalf("callback() error = %v, want changed host key", err)
+	}
+}
+
+func TestAcceptNewHostKeyCallbackReadOnlyRejectsChangedHostKey(t *testing.T) {
+	first := testPublicKey(t)
+	second := testPublicKey(t)
+	knownHosts := filepath.Join(t.TempDir(), "known_hosts")
+	if err := os.WriteFile(knownHosts, []byte(knownhosts.Line([]string{knownhosts.Normalize("example.com:22")}, first)+"\n"), 0o600); err != nil {
+		t.Fatalf("write known_hosts: %v", err)
+	}
+	callback, err := acceptNewHostKeyCallback(Options{
+		KnownHosts:         knownHosts,
+		ReadOnlyKnownHosts: true,
+	})
+	if err != nil {
+		t.Fatalf("acceptNewHostKeyCallback() error = %v", err)
+	}
+
+	err = callback("example.com:22", &net.TCPAddr{IP: net.ParseIP("192.0.2.10"), Port: 22}, second)
+	if err == nil || !strings.Contains(err.Error(), "has changed") {
+		t.Fatalf("callback() error = %v, want changed host key", err)
+	}
+}
+
+func TestAcceptNewHostKeyCallbackRequiresWritableKnownHostsForUnknownHost(t *testing.T) {
+	callback, err := acceptNewHostKeyCallback(Options{})
+	if err != nil {
+		t.Fatalf("acceptNewHostKeyCallback() error = %v", err)
+	}
+
+	err = callback("example.com:22", &net.TCPAddr{IP: net.ParseIP("192.0.2.10"), Port: 22}, testPublicKey(t))
+	if err == nil || !strings.Contains(err.Error(), "no writable known_hosts path") {
+		t.Fatalf("callback() error = %v, want no writable known_hosts path", err)
+	}
+}
+
+func TestAcceptNewHostKeyCallbackReadOnlyAllowsMissingKnownHosts(t *testing.T) {
+	callback, err := acceptNewHostKeyCallback(Options{ReadOnlyKnownHosts: true})
+	if err != nil {
+		t.Fatalf("acceptNewHostKeyCallback() error = %v", err)
+	}
+
+	if err := callback("example.com:22", &net.TCPAddr{IP: net.ParseIP("192.0.2.10"), Port: 22}, testPublicKey(t)); err != nil {
+		t.Fatalf("callback() error = %v", err)
+	}
+}
+
+func TestStrictHostKeyCallbackRequiresKnownHosts(t *testing.T) {
+	_, err := hostKeyCallback(Options{HostKeyPolicy: HostKeyPolicyStrict})
+	if err == nil || !strings.Contains(err.Error(), "known_hosts is required") {
+		t.Fatalf("hostKeyCallback() error = %v, want known_hosts required", err)
+	}
+}
+
+func testPublicKey(t *testing.T) cryptossh.PublicKey {
+	t.Helper()
+	privateKey, err := rsa.GenerateKey(rand.Reader, 2048)
+	if err != nil {
+		t.Fatalf("generate key: %v", err)
+	}
+	publicKey, err := cryptossh.NewPublicKey(&privateKey.PublicKey)
+	if err != nil {
+		t.Fatalf("new public key: %v", err)
+	}
+	return publicKey
+}

internal/adapters/ssh/integration_test.go

@@ -0,0 +1,39 @@
+package ssh
+
+import (
+	"context"
+	"os"
+	"strconv"
+	"testing"
+)
+
+func TestIntegrationSSHBackendStatRoot(t *testing.T) {
+	host := os.Getenv("DISTRIBUTOR_TEST_SSH_HOST")
+	if host == "" {
+		t.Skip("DISTRIBUTOR_TEST_SSH_HOST is not set")
+	}
+	port := 22
+	if raw := os.Getenv("DISTRIBUTOR_TEST_SSH_PORT"); raw != "" {
+		parsed, err := strconv.Atoi(raw)
+		if err != nil {
+			t.Fatalf("parse DISTRIBUTOR_TEST_SSH_PORT: %v", err)
+		}
+		port = parsed
+	}
+	backend, err := New(context.Background(), Options{
+		Host:          host,
+		User:          os.Getenv("DISTRIBUTOR_TEST_SSH_USER"),
+		Port:          port,
+		Root:          os.Getenv("DISTRIBUTOR_TEST_SSH_PATH"),
+		KeyFile:       os.Getenv("DISTRIBUTOR_TEST_SSH_KEY_FILE"),
+		KnownHosts:    os.Getenv("DISTRIBUTOR_TEST_SSH_KNOWN_HOSTS"),
+		HostKeyPolicy: HostKeyPolicyStrict,
+	})
+	if err != nil {
+		t.Fatalf("New() error = %v", err)
+	}
+	defer backend.Close()
+	if _, err := backend.Stat(context.Background(), ""); err != nil {
+		t.Fatalf("Stat(root) error = %v", err)
+	}
+}

internal/adapters/ssh/options.go

@@ -0,0 +1,78 @@
+package ssh
+
+import (
+	"fmt"
+	"os"
+	"os/user"
+	"path"
+	"path/filepath"
+	"strconv"
+)
+
+const (
+	BackendName = "ssh"
+
+	HostKeyPolicyStrict    HostKeyPolicy = "strict"
+	HostKeyPolicyAcceptNew HostKeyPolicy = "accept-new"
+	HostKeyPolicyOff       HostKeyPolicy = "off"
+)
+
+type HostKeyPolicy string
+
+type Options struct {
+	Host               string
+	User               string
+	Port               int
+	Root               string
+	KeyFile            string
+	KnownHosts         string
+	HostKeyPolicy      HostKeyPolicy
+	ReadOnlyKnownHosts bool
+}
+
+func (o Options) normalized() (Options, error) {
+	if o.Host == "" {
+		return Options{}, fmt.Errorf("host is required")
+	}
+	if o.User == "" {
+		current, err := user.Current()
+		if err != nil || current.Username == "" {
+			return Options{}, fmt.Errorf("user is required when current OS user cannot be determined")
+		}
+		o.User = current.Username
+	}
+	if o.Port == 0 {
+		o.Port = 22
+	}
+	if o.Port < 1 || o.Port > 65535 {
+		return Options{}, fmt.Errorf("port must be between 1 and 65535")
+	}
+	if o.Root == "" {
+		return Options{}, fmt.Errorf("path is required")
+	}
+	o.Root = path.Clean(o.Root)
+	if o.HostKeyPolicy == "" {
+		o.HostKeyPolicy = HostKeyPolicyAcceptNew
+	}
+	switch o.HostKeyPolicy {
+	case HostKeyPolicyStrict, HostKeyPolicyAcceptNew, HostKeyPolicyOff:
+	default:
+		return Options{}, fmt.Errorf("host_key_policy must be strict, accept-new, or off")
+	}
+	if o.KnownHosts == "" && o.HostKeyPolicy != HostKeyPolicyOff {
+		o.KnownHosts = defaultKnownHostsPath()
+	}
+	return o, nil
+}
+
+func (o Options) address() string {
+	return o.Host + ":" + strconv.Itoa(o.Port)
+}
+
+func defaultKnownHostsPath() string {
+	home, err := os.UserHomeDir()
+	if err != nil || home == "" {
+		return ""
+	}
+	return filepath.Join(home, ".ssh", "known_hosts")
+}

internal/adapters/ssh/options_test.go

@@ -0,0 +1,118 @@
+package ssh
+
+import (
+	"context"
+	"strings"
+	"testing"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+	"github.com/pkg/sftp"
+)
+
+func TestOptionsNormalizeDefaultsUserPortAndHostKeyPolicy(t *testing.T) {
+	options, err := (Options{
+		Host: "example.com",
+		Root: "/reports",
+	}).normalized()
+	if err != nil {
+		t.Fatalf("normalized() error = %v", err)
+	}
+	if options.User == "" {
+		t.Fatal("normalized user is empty")
+	}
+	if options.Port != 22 {
+		t.Fatalf("port = %d, want 22", options.Port)
+	}
+	if options.HostKeyPolicy != HostKeyPolicyAcceptNew {
+		t.Fatalf("host key policy = %q, want accept-new", options.HostKeyPolicy)
+	}
+}
+
+func TestOptionsNormalizeRejectsInvalidFields(t *testing.T) {
+	tests := map[string]Options{
+		"host": {Root: "/reports"},
+		"port": {
+			Host: "example.com",
+			Port: 70000,
+			Root: "/reports",
+		},
+		"path": {
+			Host: "example.com",
+		},
+		"host key policy": {
+			Host:          "example.com",
+			Root:          "/reports",
+			HostKeyPolicy: "prompt",
+		},
+	}
+	for name, options := range tests {
+		t.Run(name, func(t *testing.T) {
+			if _, err := options.normalized(); err == nil {
+				t.Fatal("normalized() error = nil, want error")
+			}
+		})
+	}
+}
+
+func TestNativePathEnforcesLogicalPathRules(t *testing.T) {
+	backend := &Backend{root: "/srv/reports"}
+	tests := map[string]string{
+		"bundle/report.md": "/srv/reports/bundle/report.md",
+		"":                 "/srv/reports",
+	}
+	for logicalPath, want := range tests {
+		t.Run(logicalPath, func(t *testing.T) {
+			got, err := backend.nativePath(logicalPath, true)
+			if err != nil {
+				t.Fatalf("nativePath() error = %v", err)
+			}
+			if got != want {
+				t.Fatalf("nativePath() = %q, want %q", got, want)
+			}
+		})
+	}
+
+	for _, logicalPath := range []string{"/absolute", "../escape", "a/../b", `a\b`} {
+		t.Run("reject "+logicalPath, func(t *testing.T) {
+			_, err := backend.nativePath(logicalPath, true)
+			if err == nil || !storage.IsInvalidPath(err) {
+				t.Fatalf("nativePath() error = %v, want invalid path", err)
+			}
+		})
+	}
+}
+
+func TestNewRejectsMissingAuthBeforeDial(t *testing.T) {
+	t.Setenv("SSH_AUTH_SOCK", "")
+	_, err := New(context.Background(), Options{
+		Host:          "example.com",
+		User:          "reports",
+		Root:          "/reports",
+		HostKeyPolicy: HostKeyPolicyOff,
+	})
+	if err == nil || !strings.Contains(err.Error(), "no SSH auth methods configured") {
+		t.Fatalf("New() error = %v, want missing auth", err)
+	}
+}
+
+func TestTranslateErrorMapsSFTPStatusCodes(t *testing.T) {
+	backend := &Backend{}
+	tests := []struct {
+		name string
+		err  error
+		want storage.ErrorKind
+	}{
+		{name: "not found", err: sftp.ErrSSHFxNoSuchFile, want: storage.ErrNotFound},
+		{name: "permission", err: sftp.ErrSSHFxPermissionDenied, want: storage.ErrPermission},
+		{name: "unsupported", err: sftp.ErrSSHFxOpUnsupported, want: storage.ErrUnsupported},
+		{name: "temporary", err: sftp.ErrSSHFxConnectionLost, want: storage.ErrTemporary},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			err := backend.translateError(storage.OpStat, "report.md", tt.err)
+			if !storage.IsKind(err, tt.want) {
+				t.Fatalf("translateError() = %v, want kind %s", err, tt.want)
+			}
+		})
+	}
+}

internal/app/backends.go

@@ -3,19 +3,61 @@ package app
 import (
 	"context"
 	"fmt"
+	"strconv"
 
 	"gitea.maximumdirect.net/eric/distributor/internal/adapters/local"
+	s3adapter "gitea.maximumdirect.net/eric/distributor/internal/adapters/s3"
+	sshadapter "gitea.maximumdirect.net/eric/distributor/internal/adapters/ssh"
 	"gitea.maximumdirect.net/eric/distributor/internal/config"
 	"gitea.maximumdirect.net/eric/distributor/internal/storage"
 )
 
 const storagePathKey = "path"
 
+const (
+	sshHostKey          = "host"
+	sshUserKey          = "user"
+	sshPortKey          = "port"
+	sshKeyFileKey       = "ssh_key_file"
+	sshKnownHostsKey    = "known_hosts"
+	sshHostKeyPolicyKey = "host_key_policy"
+	sshReadOnlyHostsKey = "read_only_known_hosts"
+	s3EndpointKey       = "endpoint"
+	s3BucketKey         = "bucket"
+	s3PrefixKey         = "prefix"
+	s3RegionKey         = "region"
+	s3ForcePathStyleKey = "force_path_style"
+	s3AccessKeyIDKey    = "access_key_id"
+	s3SecretAccessKey   = "secret_access_key"
+)
+
 type backendFactory struct {
-	registry *storage.Registry
+	registry           *storage.Registry
+	environment        config.Environment
+	readOnlyKnownHosts bool
+}
+
+type backendOpenSpec struct {
+	role        string
+	backend     string
+	path        string
+	host        string
+	user        string
+	port        int
+	ssh         config.SSH
+	endpoint    string
+	bucket      string
+	prefix      string
+	region      string
+	forcePath   *bool
+	credentials config.Credentials
 }
 
 func newBackendFactory() *backendFactory {
+	return newBackendFactoryWithEnvironment(config.ProcessEnvironment())
+}
+
+func newBackendFactoryWithEnvironment(environment config.Environment) *backendFactory {
 	registry := storage.NewRegistry()
 	_ = registry.Register(config.BackendLocal, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
 		if err := ctx.Err(); err != nil {
@@ -23,23 +65,149 @@ func newBackendFactory() *backendFactory {
 		}
 		return local.New(cfg[storagePathKey])
 	})
-	return &backendFactory{registry: registry}
+	_ = registry.Register(config.BackendSSH, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+		port, err := strconv.Atoi(cfg[sshPortKey])
+		if err != nil {
+			return nil, fmt.Errorf("ssh port: %w", err)
+		}
+		readOnlyKnownHosts := false
+		if raw := cfg[sshReadOnlyHostsKey]; raw != "" {
+			readOnlyKnownHosts, err = strconv.ParseBool(raw)
+			if err != nil {
+				return nil, fmt.Errorf("ssh read_only_known_hosts: %w", err)
+			}
+		}
+		return sshadapter.New(ctx, sshadapter.Options{
+			Host:               cfg[sshHostKey],
+			User:               cfg[sshUserKey],
+			Port:               port,
+			Root:               cfg[storagePathKey],
+			KeyFile:            cfg[sshKeyFileKey],
+			KnownHosts:         cfg[sshKnownHostsKey],
+			HostKeyPolicy:      sshadapter.HostKeyPolicy(cfg[sshHostKeyPolicyKey]),
+			ReadOnlyKnownHosts: readOnlyKnownHosts,
+		})
+	})
+	_ = registry.Register(config.BackendS3, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+		forcePathStyle, err := strconv.ParseBool(cfg[s3ForcePathStyleKey])
+		if err != nil {
+			return nil, fmt.Errorf("s3 force_path_style: %w", err)
+		}
+		return s3adapter.New(ctx, s3adapter.Options{
+			Endpoint:        cfg[s3EndpointKey],
+			Bucket:          cfg[s3BucketKey],
+			Prefix:          cfg[s3PrefixKey],
+			Region:          cfg[s3RegionKey],
+			ForcePathStyle:  forcePathStyle,
+			AccessKeyID:     cfg[s3AccessKeyIDKey],
+			SecretAccessKey: cfg[s3SecretAccessKey],
+		})
+	})
+	return &backendFactory{registry: registry, environment: environment}
 }
 
 func (f *backendFactory) openSource(ctx context.Context, source config.Backend) (storage.Backend, error) {
-	if source.Backend != config.BackendLocal {
-		return nil, fmt.Errorf("source backend %s is not implemented for execution", source.Backend)
-	}
-	return f.registry.Open(ctx, source.Backend, storage.OpenConfig{storagePathKey: source.Path})
+	return f.openBackend(ctx, backendOpenSpecFromSource(source))
 }
 
 func (f *backendFactory) openDestination(ctx context.Context, destination config.Destination) (storage.Backend, error) {
-	if destination.Backend != config.BackendLocal {
-		return nil, fmt.Errorf("backend %s is not implemented for execution", destination.Backend)
-	}
-	return f.registry.Open(ctx, destination.Backend, storage.OpenConfig{storagePathKey: destination.Path})
+	return f.openBackend(ctx, backendOpenSpecFromDestination(destination))
 }
 
 func (f *backendFactory) openLocalPath(ctx context.Context, path string) (storage.Backend, error) {
 	return f.registry.Open(ctx, config.BackendLocal, storage.OpenConfig{storagePathKey: path})
 }
+
+func (f *backendFactory) resolveCredentials(creds config.Credentials) (config.ResolvedCredentials, error) {
+	return f.environment.ResolveCredentials(creds)
+}
+
+func (f *backendFactory) openBackend(ctx context.Context, spec backendOpenSpec) (storage.Backend, error) {
+	if !backendExecutable(spec.backend) {
+		if spec.role == "source" {
+			return nil, fmt.Errorf("source backend %s is not implemented for execution", spec.backend)
+		}
+		return nil, fmt.Errorf("backend %s is not implemented for execution", spec.backend)
+	}
+	openConfig, err := f.openConfig(spec)
+	if err != nil {
+		return nil, err
+	}
+	return f.registry.Open(ctx, spec.backend, openConfig)
+}
+
+func backendExecutable(name string) bool {
+	return name == config.BackendLocal || name == config.BackendSSH || name == config.BackendS3
+}
+
+func (f *backendFactory) openConfig(spec backendOpenSpec) (storage.OpenConfig, error) {
+	cfg := storage.OpenConfig{storagePathKey: spec.path}
+	switch spec.backend {
+	case config.BackendSSH:
+		cfg[sshHostKey] = spec.host
+		cfg[sshUserKey] = spec.user
+		cfg[sshPortKey] = strconv.Itoa(spec.port)
+		cfg[sshKeyFileKey] = spec.ssh.KeyFile
+		cfg[sshKnownHostsKey] = spec.ssh.KnownHosts
+		cfg[sshHostKeyPolicyKey] = string(spec.ssh.HostKeyPolicy)
+		cfg[sshReadOnlyHostsKey] = strconv.FormatBool(f.readOnlyKnownHosts)
+	case config.BackendS3:
+		if err := f.addS3Config(cfg, spec); err != nil {
+			return nil, err
+		}
+	}
+	return cfg, nil
+}
+
+func (f *backendFactory) addS3Config(cfg storage.OpenConfig, spec backendOpenSpec) error {
+	cfg[s3EndpointKey] = spec.endpoint
+	cfg[s3BucketKey] = spec.bucket
+	cfg[s3PrefixKey] = spec.prefix
+	cfg[s3RegionKey] = spec.region
+	cfg[s3ForcePathStyleKey] = strconv.FormatBool(config.ForcePathStyle(spec.forcePath))
+	if spec.credentials.AccessKeyIDEnv != "" || spec.credentials.SecretAccessKeyEnv != "" {
+		resolved, err := f.resolveCredentials(spec.credentials)
+		if err != nil {
+			return err
+		}
+		cfg[s3AccessKeyIDKey] = resolved.AccessKeyID
+		cfg[s3SecretAccessKey] = resolved.SecretAccessKey
+	}
+	return nil
+}
+
+func backendOpenSpecFromSource(source config.Backend) backendOpenSpec {
+	return backendOpenSpec{
+		role:        "source",
+		backend:     source.Backend,
+		path:        source.Path,
+		host:        source.Host,
+		user:        source.User,
+		port:        source.Port,
+		ssh:         source.SSH,
+		endpoint:    source.Endpoint,
+		bucket:      source.Bucket,
+		prefix:      source.Prefix,
+		region:      source.Region,
+		forcePath:   source.ForcePath,
+		credentials: source.Creds,
+	}
+}
+
+func backendOpenSpecFromDestination(destination config.Destination) backendOpenSpec {
+	return backendOpenSpec{
+		role:        "destination",
+		backend:     destination.Backend,
+		path:        destination.Path,
+		host:        destination.Host,
+		user:        destination.User,
+		port:        destination.Port,
+		ssh:         destination.SSH,
+		endpoint:    destination.Endpoint,
+		bucket:      destination.Bucket,
+		prefix:      destination.Prefix,
+		region:      destination.Region,
+		forcePath:   destination.ForcePath,
+		credentials: destination.Creds,
+	}
+}

internal/app/backends_test.go

@@ -1,11 +1,16 @@
 package app
 
 import (
+	"bytes"
 	"context"
+	"fmt"
 	"strings"
 	"testing"
 
 	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
+	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
 )
 
 func TestBackendFactoryOpensLocalSource(t *testing.T) {
@@ -47,14 +52,125 @@ func TestBackendFactoryOpensDirectLocalPath(t *testing.T) {
 	}
 }
 
+func TestBackendFactoryOpensSSHSourceWithRegisteredOpener(t *testing.T) {
+	factory := &backendFactory{registry: storage.NewRegistry()}
+	var got storage.OpenConfig
+	if err := factory.registry.Register(config.BackendSSH, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+		got = cfg
+		return fake.New(), nil
+	}); err != nil {
+		t.Fatalf("Register() error = %v", err)
+	}
+
+	backend, err := factory.openSource(context.Background(), config.Backend{
+		Backend: config.BackendSSH,
+		Host:    "source.example.com",
+		User:    "reports",
+		Port:    22,
+		Path:    "/reports",
+		SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyAcceptNew},
+	})
+	if err != nil {
+		t.Fatalf("openSource() error = %v", err)
+	}
+	if backend == nil {
+		t.Fatal("openSource() backend = nil")
+	}
+	if got[sshHostKey] != "source.example.com" || got[storagePathKey] != "/reports" {
+		t.Fatalf("open config = %#v, want SSH source fields", got)
+	}
+}
+
+func TestBackendFactoryOpensSSHDestinationWithRegisteredOpener(t *testing.T) {
+	factory := &backendFactory{registry: storage.NewRegistry()}
+	var got storage.OpenConfig
+	if err := factory.registry.Register(config.BackendSSH, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+		got = cfg
+		return fake.New(), nil
+	}); err != nil {
+		t.Fatalf("Register() error = %v", err)
+	}
+
+	backend, err := factory.openDestination(context.Background(), config.Destination{
+		Backend: config.BackendSSH,
+		Host:    "destination.example.com",
+		User:    "deploy",
+		Port:    2222,
+		Path:    "/archive",
+		SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyStrict},
+	})
+	if err != nil {
+		t.Fatalf("openDestination() error = %v", err)
+	}
+	if backend == nil {
+		t.Fatal("openDestination() backend = nil")
+	}
+	if got[sshHostKey] != "destination.example.com" || got[sshPortKey] != "2222" || got[sshHostKeyPolicyKey] != "strict" {
+		t.Fatalf("open config = %#v, want SSH destination fields", got)
+	}
+}
+
+func TestBackendFactorySetsReadOnlyKnownHostsForDryRunSSH(t *testing.T) {
+	factory := &backendFactory{
+		registry:           storage.NewRegistry(),
+		readOnlyKnownHosts: true,
+	}
+	var got storage.OpenConfig
+	if err := factory.registry.Register(config.BackendSSH, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+		got = cfg
+		return fake.New(), nil
+	}); err != nil {
+		t.Fatalf("Register() error = %v", err)
+	}
+
+	_, err := factory.openDestination(context.Background(), config.Destination{
+		Backend: config.BackendSSH,
+		Host:    "destination.example.com",
+		User:    "deploy",
+		Port:    22,
+		Path:    "/archive",
+		SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyAcceptNew},
+	})
+	if err != nil {
+		t.Fatalf("openDestination() error = %v", err)
+	}
+	if got[sshReadOnlyHostsKey] != "true" {
+		t.Fatalf("open config %s = %q, want true", sshReadOnlyHostsKey, got[sshReadOnlyHostsKey])
+	}
+}
+
+func TestBackendFactoryUsesPersistentKnownHostsByDefault(t *testing.T) {
+	factory := &backendFactory{registry: storage.NewRegistry()}
+	var got storage.OpenConfig
+	if err := factory.registry.Register(config.BackendSSH, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+		got = cfg
+		return fake.New(), nil
+	}); err != nil {
+		t.Fatalf("Register() error = %v", err)
+	}
+
+	_, err := factory.openDestination(context.Background(), config.Destination{
+		Backend: config.BackendSSH,
+		Host:    "destination.example.com",
+		User:    "deploy",
+		Port:    22,
+		Path:    "/archive",
+		SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyAcceptNew},
+	})
+	if err != nil {
+		t.Fatalf("openDestination() error = %v", err)
+	}
+	if got[sshReadOnlyHostsKey] != "false" {
+		t.Fatalf("open config %s = %q, want false", sshReadOnlyHostsKey, got[sshReadOnlyHostsKey])
+	}
+}
+
 func TestBackendFactoryRejectsUnsupportedSource(t *testing.T) {
 	factory := newBackendFactory()
 	_, err := factory.openSource(context.Background(), config.Backend{
-		Backend: config.BackendSSH,
-		URI:     "ssh://reports@example.com:22",
-		Path:    "/reports",
+		Backend: "ftp",
 	})
-	if err == nil || !strings.Contains(err.Error(), "source backend ssh is not implemented for execution") {
+	if err == nil || !strings.Contains(err.Error(), "source backend ftp is not implemented for execution") {
 		t.Fatalf("openSource() error = %v, want not implemented", err)
 	}
 }
@@ -62,11 +178,461 @@ func TestBackendFactoryRejectsUnsupportedSource(t *testing.T) {
 func TestBackendFactoryRejectsUnsupportedDestination(t *testing.T) {
 	factory := newBackendFactory()
 	_, err := factory.openDestination(context.Background(), config.Destination{
-		Backend:  config.BackendS3,
-		Endpoint: "https://s3.example.com",
-		Bucket:   "reports",
+		Backend: "ftp",
 	})
-	if err == nil || !strings.Contains(err.Error(), "backend s3 is not implemented for execution") {
+	if err == nil || !strings.Contains(err.Error(), "backend ftp is not implemented for execution") {
 		t.Fatalf("openDestination() error = %v, want not implemented", err)
 	}
 }
+
+func TestBackendFactoryOpensS3DestinationWithRegisteredOpener(t *testing.T) {
+	factory := &backendFactory{
+		registry:    storage.NewRegistry(),
+		environment: config.NewEnvironment(nil, func(string) (string, bool) { return "", false }),
+	}
+	var got storage.OpenConfig
+	if err := factory.registry.Register(config.BackendS3, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+		got = cfg
+		return fake.New(), nil
+	}); err != nil {
+		t.Fatalf("Register() error = %v", err)
+	}
+	forcePathStyle := false
+	backend, err := factory.openDestination(context.Background(), config.Destination{
+		Backend:   config.BackendS3,
+		Endpoint:  "https://s3.example.com",
+		Bucket:    "reports",
+		Prefix:    "archive",
+		Region:    config.DefaultS3Region,
+		ForcePath: &forcePathStyle,
+	})
+	if err != nil {
+		t.Fatalf("openDestination() error = %v", err)
+	}
+	if backend == nil {
+		t.Fatal("openDestination() backend = nil")
+	}
+	assertOpenConfig(t, got, map[string]string{
+		s3EndpointKey:       "https://s3.example.com",
+		s3BucketKey:         "reports",
+		s3PrefixKey:         "archive",
+		s3RegionKey:         config.DefaultS3Region,
+		s3ForcePathStyleKey: "false",
+	})
+}
+
+func TestBackendFactoryResolvesS3CredentialsThroughSecretsAwareEnvironment(t *testing.T) {
+	factory := &backendFactory{
+		registry: storage.NewRegistry(),
+		environment: config.NewEnvironment(map[string]string{
+			"ACCESS_KEY_ID":     "secret-access",
+			"SECRET_ACCESS_KEY": "secret-secret",
+		}, func(string) (string, bool) { return "", false }),
+	}
+	var got storage.OpenConfig
+	if err := factory.registry.Register(config.BackendS3, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+		got = cfg
+		return fake.New(), nil
+	}); err != nil {
+		t.Fatalf("Register() error = %v", err)
+	}
+	forcePathStyle := true
+	_, err := factory.openSource(context.Background(), config.Backend{
+		Backend:   config.BackendS3,
+		Endpoint:  "https://s3.example.com",
+		Bucket:    "reports",
+		Region:    config.DefaultS3Region,
+		ForcePath: &forcePathStyle,
+		Creds: config.Credentials{
+			AccessKeyIDEnv:     "ACCESS_KEY_ID",
+			SecretAccessKeyEnv: "SECRET_ACCESS_KEY",
+		},
+	})
+	if err != nil {
+		t.Fatalf("openSource() error = %v", err)
+	}
+	assertOpenConfig(t, got, map[string]string{
+		s3AccessKeyIDKey:  "secret-access",
+		s3SecretAccessKey: "secret-secret",
+	})
+}
+
+func TestBackendFactoryResolvesCredentialsThroughEnvironment(t *testing.T) {
+	factory := newBackendFactoryWithEnvironment(config.NewEnvironment(map[string]string{
+		"ACCESS_KEY_ID":     "secret-access",
+		"SECRET_ACCESS_KEY": "secret-secret",
+	}, func(string) (string, bool) {
+		return "", false
+	}))
+
+	creds, err := factory.resolveCredentials(config.Credentials{
+		AccessKeyIDEnv:     "ACCESS_KEY_ID",
+		SecretAccessKeyEnv: "SECRET_ACCESS_KEY",
+	})
+	if err != nil {
+		t.Fatalf("resolveCredentials() error = %v", err)
+	}
+	if creds.AccessKeyID != "secret-access" || creds.SecretAccessKey != "secret-secret" {
+		t.Fatalf("resolved credentials = %#v", creds)
+	}
+}
+
+func TestBackendFactoryBuildsSSHSourceOpenConfig(t *testing.T) {
+	factory := &backendFactory{environment: config.NewEnvironment(nil, nil)}
+	cfg, err := factory.openConfig(backendOpenSpecFromSource(config.Backend{
+		Backend: config.BackendSSH,
+		Host:    "source.example.com",
+		User:    "reports",
+		Port:    2222,
+		Path:    "/reports",
+		SSH: config.SSH{
+			KeyFile:       "/home/reports/.ssh/id_ed25519",
+			KnownHosts:    "/home/reports/.ssh/known_hosts",
+			HostKeyPolicy: config.HostKeyPolicyStrict,
+		},
+	}))
+	if err != nil {
+		t.Fatalf("openConfig() error = %v", err)
+	}
+
+	assertOpenConfig(t, cfg, map[string]string{
+		storagePathKey:      "/reports",
+		sshHostKey:          "source.example.com",
+		sshUserKey:          "reports",
+		sshPortKey:          "2222",
+		sshKeyFileKey:       "/home/reports/.ssh/id_ed25519",
+		sshKnownHostsKey:    "/home/reports/.ssh/known_hosts",
+		sshHostKeyPolicyKey: "strict",
+	})
+}
+
+func TestBackendFactoryBuildsSSHDestinationOpenConfig(t *testing.T) {
+	factory := &backendFactory{environment: config.NewEnvironment(nil, nil)}
+	cfg, err := factory.openConfig(backendOpenSpecFromDestination(config.Destination{
+		Backend: config.BackendSSH,
+		Host:    "destination.example.com",
+		User:    "deploy",
+		Port:    22,
+		Path:    "/srv/archive",
+		SSH: config.SSH{
+			HostKeyPolicy: config.HostKeyPolicyAcceptNew,
+		},
+	}))
+	if err != nil {
+		t.Fatalf("openConfig() error = %v", err)
+	}
+
+	assertOpenConfig(t, cfg, map[string]string{
+		storagePathKey:      "/srv/archive",
+		sshHostKey:          "destination.example.com",
+		sshUserKey:          "deploy",
+		sshPortKey:          "22",
+		sshHostKeyPolicyKey: "accept-new",
+	})
+}
+
+func TestBackendFactoryBuildsEquivalentSourceAndDestinationOpenConfig(t *testing.T) {
+	forcePathStyle := false
+	tests := []struct {
+		name        string
+		source      config.Backend
+		destination config.Destination
+	}{
+		{
+			name:        "local",
+			source:      config.Backend{Backend: config.BackendLocal, Path: "/reports"},
+			destination: config.Destination{Backend: config.BackendLocal, Path: "/reports"},
+		},
+		{
+			name: "ssh",
+			source: config.Backend{
+				Backend: config.BackendSSH,
+				Host:    "reports.example.com",
+				User:    "reports",
+				Port:    2222,
+				Path:    "/reports",
+				SSH: config.SSH{
+					KeyFile:       "/home/reports/.ssh/id_ed25519",
+					KnownHosts:    "/home/reports/.ssh/known_hosts",
+					HostKeyPolicy: config.HostKeyPolicyStrict,
+				},
+			},
+			destination: config.Destination{
+				Backend: config.BackendSSH,
+				Host:    "reports.example.com",
+				User:    "reports",
+				Port:    2222,
+				Path:    "/reports",
+				SSH: config.SSH{
+					KeyFile:       "/home/reports/.ssh/id_ed25519",
+					KnownHosts:    "/home/reports/.ssh/known_hosts",
+					HostKeyPolicy: config.HostKeyPolicyStrict,
+				},
+			},
+		},
+		{
+			name: "s3",
+			source: config.Backend{
+				Backend:   config.BackendS3,
+				Endpoint:  "https://s3.example.com",
+				Bucket:    "reports",
+				Prefix:    "archive",
+				Region:    "us-west-2",
+				ForcePath: &forcePathStyle,
+			},
+			destination: config.Destination{
+				Backend:   config.BackendS3,
+				Endpoint:  "https://s3.example.com",
+				Bucket:    "reports",
+				Prefix:    "archive",
+				Region:    "us-west-2",
+				ForcePath: &forcePathStyle,
+			},
+		},
+		{
+			name: "s3 explicit credentials",
+			source: config.Backend{
+				Backend:  config.BackendS3,
+				Endpoint: "https://s3.example.com",
+				Bucket:   "reports",
+				Region:   config.DefaultS3Region,
+				Creds: config.Credentials{
+					AccessKeyIDEnv:     "ACCESS_KEY_ID",
+					SecretAccessKeyEnv: "SECRET_ACCESS_KEY",
+				},
+			},
+			destination: config.Destination{
+				Backend:  config.BackendS3,
+				Endpoint: "https://s3.example.com",
+				Bucket:   "reports",
+				Region:   config.DefaultS3Region,
+				Creds: config.Credentials{
+					AccessKeyIDEnv:     "ACCESS_KEY_ID",
+					SecretAccessKeyEnv: "SECRET_ACCESS_KEY",
+				},
+			},
+		},
+	}
+
+	factory := &backendFactory{
+		environment: config.NewEnvironment(map[string]string{
+			"ACCESS_KEY_ID":     "secret-access",
+			"SECRET_ACCESS_KEY": "secret-secret",
+		}, func(string) (string, bool) { return "", false }),
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			sourceConfig, err := factory.openConfig(backendOpenSpecFromSource(tt.source))
+			if err != nil {
+				t.Fatalf("source openConfig() error = %v", err)
+			}
+			destinationConfig, err := factory.openConfig(backendOpenSpecFromDestination(tt.destination))
+			if err != nil {
+				t.Fatalf("destination openConfig() error = %v", err)
+			}
+			if !openConfigEqual(sourceConfig, destinationConfig) {
+				t.Fatalf("source open config = %#v, destination open config = %#v, want equivalent", sourceConfig, destinationConfig)
+			}
+		})
+	}
+}
+
+func TestBackendFactoryBuildsEquivalentDryRunSSHOpenConfig(t *testing.T) {
+	factory := &backendFactory{readOnlyKnownHosts: true}
+	sourceConfig, err := factory.openConfig(backendOpenSpecFromSource(config.Backend{
+		Backend: config.BackendSSH,
+		Host:    "reports.example.com",
+		Port:    22,
+		Path:    "/reports",
+		SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyAcceptNew},
+	}))
+	if err != nil {
+		t.Fatalf("source openConfig() error = %v", err)
+	}
+	destinationConfig, err := factory.openConfig(backendOpenSpecFromDestination(config.Destination{
+		Backend: config.BackendSSH,
+		Host:    "reports.example.com",
+		Port:    22,
+		Path:    "/reports",
+		SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyAcceptNew},
+	}))
+	if err != nil {
+		t.Fatalf("destination openConfig() error = %v", err)
+	}
+	if !openConfigEqual(sourceConfig, destinationConfig) {
+		t.Fatalf("source open config = %#v, destination open config = %#v, want equivalent", sourceConfig, destinationConfig)
+	}
+	if sourceConfig[sshReadOnlyHostsKey] != "true" {
+		t.Fatalf("open config %s = %q, want true", sshReadOnlyHostsKey, sourceConfig[sshReadOnlyHostsKey])
+	}
+}
+
+func TestConfiguredSourceValidationAndRunUseEquivalentSourceOpenConfig(t *testing.T) {
+	tests := []struct {
+		name       string
+		source     config.Backend
+		sourceKey  string
+		dest       config.Destination
+		destKey    string
+		wantFields map[string]string
+	}{
+		{
+			name: "s3",
+			source: config.Backend{
+				Backend:  config.BackendS3,
+				Endpoint: "https://s3.example.com",
+				Bucket:   "source-bucket",
+				Prefix:   "source-prefix",
+				Region:   config.DefaultS3Region,
+			},
+			sourceKey: "s3:source-bucket",
+			dest: config.Destination{
+				ID:       "archive",
+				Backend:  config.BackendS3,
+				Endpoint: "https://s3.example.com",
+				Bucket:   "destination-bucket",
+				Region:   config.DefaultS3Region,
+			},
+			destKey: "s3:destination-bucket",
+			wantFields: map[string]string{
+				s3EndpointKey:       "https://s3.example.com",
+				s3BucketKey:         "source-bucket",
+				s3PrefixKey:         "source-prefix",
+				s3RegionKey:         config.DefaultS3Region,
+				s3ForcePathStyleKey: "true",
+			},
+		},
+		{
+			name: "ssh",
+			source: config.Backend{
+				Backend: config.BackendSSH,
+				Host:    "ssh.example.com",
+				User:    "reports",
+				Port:    2222,
+				Path:    "/source",
+				SSH: config.SSH{
+					KeyFile:       "/home/reports/.ssh/id_ed25519",
+					KnownHosts:    "/home/reports/.ssh/known_hosts",
+					HostKeyPolicy: config.HostKeyPolicyStrict,
+				},
+			},
+			sourceKey: "ssh:/source",
+			dest: config.Destination{
+				ID:      "archive",
+				Backend: config.BackendSSH,
+				Host:    "ssh.example.com",
+				Port:    2222,
+				Path:    "/destination",
+				SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyStrict},
+			},
+			destKey: "ssh:/destination",
+			wantFields: map[string]string{
+				storagePathKey:      "/source",
+				sshHostKey:          "ssh.example.com",
+				sshUserKey:          "reports",
+				sshPortKey:          "2222",
+				sshKeyFileKey:       "/home/reports/.ssh/id_ed25519",
+				sshKnownHostsKey:    "/home/reports/.ssh/known_hosts",
+				sshHostKeyPolicyKey: "strict",
+			},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			sourceBackend := fake.New()
+			testutil.WriteFakeSourceBundle(t, sourceBackend, "", testutil.BundleOptions{ID: "reports.source"})
+			destinationBackend := fake.New()
+			var validateSourceConfig storage.OpenConfig
+			var runSourceConfig storage.OpenConfig
+			validateProvider := recordingBackendFactoryProvider(t, map[string]storage.Backend{
+				tt.sourceKey: sourceBackend,
+				tt.destKey:   destinationBackend,
+			}, func(cfg storage.OpenConfig) {
+				validateSourceConfig = cfg
+			})
+			runProvider := recordingBackendFactoryProvider(t, map[string]storage.Backend{
+				tt.sourceKey: sourceBackend,
+				tt.destKey:   destinationBackend,
+			}, func(cfg storage.OpenConfig) {
+				runSourceConfig = cfg
+			})
+			cfg := config.Config{Pipelines: []config.Pipeline{{
+				ID:           "reports",
+				Source:       tt.source,
+				Destinations: []config.Destination{tt.dest},
+			}}}
+			config.ApplyDefaults(&cfg)
+
+			var validateOutput bytes.Buffer
+			if err := validateConfigWithBackendFactory(context.Background(), cfg, ValidateOptions{
+				PipelineID: "reports",
+				Stdout:     &validateOutput,
+			}, validateProvider); err != nil {
+				t.Fatalf("validateConfigWithBackendFactory() error = %v", err)
+			}
+			if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{}, runProvider); err != nil {
+				t.Fatalf("runConfigWithBackendFactory() error = %v", err)
+			}
+			if !openConfigEqual(validateSourceConfig, runSourceConfig) {
+				t.Fatalf("validate source config = %#v, run source config = %#v, want equivalent", validateSourceConfig, runSourceConfig)
+			}
+			assertOpenConfig(t, runSourceConfig, tt.wantFields)
+		})
+	}
+}
+
+func assertOpenConfig(t *testing.T, got map[string]string, want map[string]string) {
+	t.Helper()
+	for key, wantValue := range want {
+		if gotValue := got[key]; gotValue != wantValue {
+			t.Fatalf("open config %s = %q, want %q", key, gotValue, wantValue)
+		}
+	}
+}
+
+func openConfigEqual(left, right storage.OpenConfig) bool {
+	if len(left) != len(right) {
+		return false
+	}
+	for key, leftValue := range left {
+		if right[key] != leftValue {
+			return false
+		}
+	}
+	return true
+}
+
+func recordingBackendFactoryProvider(t *testing.T, remoteBackends map[string]storage.Backend, recordSource func(storage.OpenConfig)) backendFactoryProvider {
+	t.Helper()
+	return func(environment config.Environment) *backendFactory {
+		registry := storage.NewRegistry()
+		if err := registry.Register(config.BackendS3, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+			if cfg[s3BucketKey] == "source-bucket" {
+				recordSource(cfg)
+			}
+			key := "s3:" + cfg[s3BucketKey]
+			backend := remoteBackends[key]
+			if backend == nil {
+				return nil, fmt.Errorf("missing fake backend for %s", key)
+			}
+			return backend, nil
+		}); err != nil {
+			t.Fatalf("register s3 backend: %v", err)
+		}
+		if err := registry.Register(config.BackendSSH, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+			if cfg[storagePathKey] == "/source" {
+				recordSource(cfg)
+			}
+			key := "ssh:" + cfg[storagePathKey]
+			backend := remoteBackends[key]
+			if backend == nil {
+				return nil, fmt.Errorf("missing fake backend for %s", key)
+			}
+			return backend, nil
+		}); err != nil {
+			t.Fatalf("register ssh backend: %v", err)
+		}
+		return &backendFactory{registry: registry, environment: environment}
+	}
+}

internal/app/inspect.go

@@ -5,38 +5,130 @@ import (
 	"fmt"
 	"io"
 
-	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
 	"gitea.maximumdirect.net/eric/distributor/internal/storage"
 )
 
 type InspectOptions struct {
-	Path   string
-	Stdout io.Writer
+	Path         string
+	ConfigPath   string
+	PipelineID   string
+	BundlePath   string
+	Stdout       io.Writer
+	OutputFormat OutputFormat
 }
 
 func Inspect(ctx context.Context, options InspectOptions) error {
-	if options.Path == "" {
-		return fmt.Errorf("inspect command requires a path")
-	}
-	backend, err := newBackendFactory().openLocalPath(ctx, options.Path)
-	if err != nil {
-		return err
-	}
-	bundles, err := bundle.Discover(ctx, backend, "")
-	if err != nil {
-		return err
-	}
-	return writeInspection(options.Stdout, bundles)
+	return inspectWithBackendFactory(ctx, options, newBackendFactoryWithEnvironment)
 }
 
-func writeInspection(w io.Writer, bundles []bundle.Bundle) error {
+func inspectWithBackendFactory(ctx context.Context, options InspectOptions, provider backendFactoryProvider) error {
+	if err := ValidateOutputFormat(options.OutputFormat); err != nil {
+		return err
+	}
+	selection, err := selectSourceBundles(ctx, sourceCommandOptions{
+		CommandName: "inspect",
+		Path:        options.Path,
+		ConfigPath:  options.ConfigPath,
+		PipelineID:  options.PipelineID,
+		BundlePath:  options.BundlePath,
+	}, provider)
+	if err != nil {
+		return err
+	}
+	return writeInspectResult(options, selection)
+}
+
+func inspectConfigWithBackendFactory(ctx context.Context, cfg config.Config, options InspectOptions, provider backendFactoryProvider) error {
+	if err := ValidateOutputFormat(options.OutputFormat); err != nil {
+		return err
+	}
+	selection, err := selectSourceBundlesFromConfig(ctx, cfg, sourceCommandOptions{
+		CommandName: "inspect",
+		PipelineID:  options.PipelineID,
+		BundlePath:  options.BundlePath,
+	}, provider)
+	if err != nil {
+		return err
+	}
+	return writeInspectResult(options, selection)
+}
+
+func writeInspectResult(options InspectOptions, selection sourceSelection) error {
+	if IsJSONOutput(options.OutputFormat) {
+		return WriteJSONEnvelope(options.Stdout, "inspect", true, selection.Warnings, inspectResultFromSelection(selection), nil)
+	}
+	if err := writeWarnings(options.Stdout, selection.Warnings); err != nil {
+		return err
+	}
+	return writeInspection(options.Stdout, selection)
+}
+
+type inspectResult struct {
+	PipelineID    string                `json:"pipeline_id,omitempty"`
+	SourceBackend string                `json:"source_backend,omitempty"`
+	BundleCount   int                   `json:"bundle_count"`
+	Bundles       []inspectBundleResult `json:"bundles"`
+}
+
+type inspectBundleResult struct {
+	Path      string              `json:"path"`
+	ID        string              `json:"id"`
+	Created   string              `json:"created"`
+	Digest    string              `json:"digest"`
+	FileCount int                 `json:"file_count"`
+	TotalSize int64               `json:"total_size"`
+	Files     []inspectFileResult `json:"files"`
+}
+
+type inspectFileResult struct {
+	Path   string `json:"path"`
+	SHA256 string `json:"sha256"`
+	Size   int64  `json:"size"`
+}
+
+func inspectResultFromSelection(selection sourceSelection) inspectResult {
+	result := inspectResult{
+		PipelineID:    selection.PipelineID,
+		SourceBackend: selection.SourceBackend,
+		BundleCount:   len(selection.Bundles),
+		Bundles:       make([]inspectBundleResult, 0, len(selection.Bundles)),
+	}
+	for _, sourceBundle := range selection.Bundles {
+		bundleResult := inspectBundleResult{
+			Path:      storage.DisplayPath(sourceBundle.RootRelativePath),
+			ID:        sourceBundle.Manifest.ID,
+			Created:   sourceBundle.Manifest.Created.Format("2006-01-02T15:04:05Z07:00"),
+			Digest:    sourceBundle.Manifest.Digest,
+			FileCount: len(sourceBundle.Manifest.Files),
+			Files:     make([]inspectFileResult, 0, len(sourceBundle.Manifest.Files)),
+		}
+		for _, file := range sourceBundle.Manifest.Files {
+			bundleResult.TotalSize += file.Size
+			bundleResult.Files = append(bundleResult.Files, inspectFileResult{
+				Path:   file.Path,
+				SHA256: file.SHA256,
+				Size:   file.Size,
+			})
+		}
+		result.Bundles = append(result.Bundles, bundleResult)
+	}
+	return result
+}
+
+func writeInspection(w io.Writer, selection sourceSelection) error {
 	if w == nil {
 		return nil
 	}
-	if _, err := fmt.Fprintf(w, "Bundles: %d\n", len(bundles)); err != nil {
+	if selection.ConfigMode {
+		if _, err := fmt.Fprintf(w, "Pipeline: %s\nSource: %s\n", selection.PipelineID, selection.SourceBackend); err != nil {
+			return err
+		}
+	}
+	if _, err := fmt.Fprintf(w, "Bundles: %d\n", len(selection.Bundles)); err != nil {
 		return err
 	}
-	for _, sourceBundle := range bundles {
+	for _, sourceBundle := range selection.Bundles {
 		if _, err := fmt.Fprintf(
 			w,
 			"- path=%s id=%s created=%s digest=%s files=%d\n",

internal/app/inspect_test.go

@@ -6,6 +6,8 @@ import (
 	"path/filepath"
 	"strings"
 	"testing"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
 )
 
 func TestInspectPrintsBundleSummary(t *testing.T) {
@@ -32,6 +34,57 @@ func TestInspectPrintsBundleSummary(t *testing.T) {
 	}
 }
 
+func TestInspectConfiguredLocalSource(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "daily", testutil.BundleOptions{ID: "reports.daily"})
+	var stdout bytes.Buffer
+
+	err := Inspect(context.Background(), InspectOptions{
+		ConfigPath: testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot),
+		PipelineID: "reports",
+		Stdout:     &stdout,
+	})
+
+	if err != nil {
+		t.Fatalf("Inspect() configured source error = %v", err)
+	}
+	output := stdout.String()
+	for _, want := range []string{
+		"Pipeline: reports",
+		"Source: local",
+		"Bundles: 1",
+		"path=daily",
+		"id=reports.daily",
+	} {
+		if !strings.Contains(output, want) {
+			t.Fatalf("Inspect() output = %q, want substring %q", output, want)
+		}
+	}
+}
+
+func TestInspectConfiguredSourceJSON(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{ID: "reports.json"})
+	var stdout bytes.Buffer
+
+	err := Inspect(context.Background(), InspectOptions{
+		ConfigPath:   testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot),
+		PipelineID:   "reports",
+		Stdout:       &stdout,
+		OutputFormat: OutputFormatJSON,
+	})
+
+	if err != nil {
+		t.Fatalf("Inspect() configured JSON error = %v", err)
+	}
+	result := decodeAppResult(t, stdout.String())
+	if result["pipeline_id"] != "reports" || result["source_backend"] != "local" || result["bundle_count"] != float64(1) {
+		t.Fatalf("result = %#v, want configured inspect metadata", result)
+	}
+}
+
 func TestInspectRequiresPath(t *testing.T) {
 	err := Inspect(context.Background(), InspectOptions{})
 	if err == nil || !strings.Contains(err.Error(), "requires a path") {

internal/app/manifest.go

@@ -0,0 +1,128 @@
+package app
+
+import (
+	"context"
+	"fmt"
+	"io"
+	"path/filepath"
+	"strings"
+	"time"
+
+	producerbundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
+)
+
+type ManifestCreateOptions struct {
+	Root         string
+	ID           string
+	Created      string
+	Files        []string
+	Overwrite    bool
+	Stdout       io.Writer
+	OutputFormat OutputFormat
+}
+
+func ManifestCreate(ctx context.Context, options ManifestCreateOptions) error {
+	if err := ValidateOutputFormat(options.OutputFormat); err != nil {
+		return err
+	}
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	if options.Root == "" {
+		return fmt.Errorf("manifest create command requires a bundle path")
+	}
+	if options.ID == "" {
+		return fmt.Errorf("manifest create command requires --id")
+	}
+
+	created, err := parseOptionalCreated(options.Created)
+	if err != nil {
+		return err
+	}
+	files := normalizeManifestFiles(options.Files)
+	buildOptions := producerbundle.BuildOptions{
+		Root:    options.Root,
+		ID:      options.ID,
+		Created: created,
+		Files:   files,
+		Scan:    len(files) == 0,
+	}
+	manifest, err := producerbundle.BuildManifest(buildOptions)
+	if err != nil {
+		return err
+	}
+	if err := producerbundle.WriteManifest(options.Root, manifest, producerbundle.WriteManifestOptions{Overwrite: options.Overwrite}); err != nil {
+		return err
+	}
+	loaded, err := producerbundle.LoadManifest(options.Root)
+	if err != nil {
+		return err
+	}
+	if err := producerbundle.ValidateBundle(options.Root, loaded); err != nil {
+		return err
+	}
+
+	result := manifestCreateResultFromManifest(options.Root, loaded)
+	if IsJSONOutput(options.OutputFormat) {
+		return WriteJSONEnvelope(options.Stdout, "manifest create", true, nil, result, nil)
+	}
+	if options.Stdout != nil {
+		_, err = fmt.Fprintf(options.Stdout, "created %s\nbundle: %s\nfiles: %d\ndigest: %s\n", producerbundle.ManifestName, result.ID, result.FileCount, result.Digest)
+	}
+	return err
+}
+
+func parseOptionalCreated(value string) (time.Time, error) {
+	if value == "" {
+		return time.Time{}, nil
+	}
+	created, err := time.Parse(time.RFC3339, value)
+	if err != nil {
+		return time.Time{}, fmt.Errorf("created must be RFC3339: %w", err)
+	}
+	return created, nil
+}
+
+func normalizeManifestFiles(files []string) []string {
+	normalized := make([]string, 0, len(files))
+	for _, file := range files {
+		normalized = append(normalized, filepath.ToSlash(filepath.Clean(strings.ReplaceAll(file, "\\", string(filepath.Separator)))))
+	}
+	return normalized
+}
+
+type manifestCreateResult struct {
+	ManifestPath string                     `json:"manifest_path"`
+	Root         string                     `json:"root"`
+	ID           string                     `json:"id"`
+	Created      string                     `json:"created"`
+	Digest       string                     `json:"digest"`
+	FileCount    int                        `json:"file_count"`
+	Files        []manifestCreateFileResult `json:"files"`
+}
+
+type manifestCreateFileResult struct {
+	Path   string `json:"path"`
+	SHA256 string `json:"sha256"`
+	Size   int64  `json:"size"`
+}
+
+func manifestCreateResultFromManifest(root string, manifest producerbundle.Manifest) manifestCreateResult {
+	result := manifestCreateResult{
+		ManifestPath: filepath.ToSlash(filepath.Join(root, producerbundle.ManifestName)),
+		Root:         filepath.ToSlash(root),
+		ID:           manifest.ID,
+		Created:      manifest.Created.Format(time.RFC3339),
+		Digest:       manifest.Digest,
+		FileCount:    len(manifest.Files),
+		Files:        make([]manifestCreateFileResult, 0, len(manifest.Files)),
+	}
+	for _, file := range manifest.Files {
+		result.Files = append(result.Files, manifestCreateFileResult{
+			Path:   file.Path,
+			SHA256: file.SHA256,
+			Size:   file.Size,
+		})
+	}
+	return result
+}

internal/app/output.go

@@ -0,0 +1,76 @@
+package app
+
+import (
+	"encoding/json"
+	"fmt"
+	"io"
+)
+
+const outputSchemaVersion = 1
+
+type OutputFormat string
+
+const (
+	OutputFormatText OutputFormat = "text"
+	OutputFormatJSON OutputFormat = "json"
+)
+
+type OutputWarning struct {
+	Message string `json:"message"`
+}
+
+type OutputError struct {
+	PipelineID    string `json:"pipeline_id,omitempty"`
+	DestinationID string `json:"destination_id,omitempty"`
+	Backend       string `json:"backend,omitempty"`
+	BundlePath    string `json:"bundle_path,omitempty"`
+	Message       string `json:"message"`
+}
+
+type outputEnvelope struct {
+	SchemaVersion int             `json:"schema_version"`
+	Command       string          `json:"command"`
+	OK            bool            `json:"ok"`
+	Warnings      []OutputWarning `json:"warnings"`
+	Result        any             `json:"result"`
+	Errors        []OutputError   `json:"errors,omitempty"`
+}
+
+func NormalizeOutputFormat(format OutputFormat) OutputFormat {
+	if format == "" {
+		return OutputFormatText
+	}
+	return format
+}
+
+func ValidateOutputFormat(format OutputFormat) error {
+	switch NormalizeOutputFormat(format) {
+	case OutputFormatText, OutputFormatJSON:
+		return nil
+	default:
+		return fmt.Errorf("format must be text or json")
+	}
+}
+
+func IsJSONOutput(format OutputFormat) bool {
+	return NormalizeOutputFormat(format) == OutputFormatJSON
+}
+
+func WriteJSONEnvelope(w io.Writer, command string, ok bool, warnings []OutputWarning, result any, errors []OutputError) error {
+	if w == nil {
+		return nil
+	}
+	if warnings == nil {
+		warnings = []OutputWarning{}
+	}
+	envelope := outputEnvelope{
+		SchemaVersion: outputSchemaVersion,
+		Command:       command,
+		OK:            ok,
+		Warnings:      warnings,
+		Result:        result,
+		Errors:        errors,
+	}
+	encoder := json.NewEncoder(w)
+	return encoder.Encode(envelope)
+}

internal/app/run.go

@@ -2,10 +2,8 @@ package app
 
 import (
 	"context"
-	"errors"
 	"fmt"
 	"io"
-	"strings"
 
 	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
 	"gitea.maximumdirect.net/eric/distributor/internal/config"
@@ -15,13 +13,35 @@ import (
 )
 
 type RunOptions struct {
+	ConfigPath   string
+	DryRun       bool
+	Force        bool
+	Stdout       io.Writer
+	OutputFormat OutputFormat
+	Notifier     notify.Notifier
+}
+
+type RunPipelineOptions struct {
 	ConfigPath string
+	PipelineID string
 	DryRun     bool
-	Stdout     io.Writer
+	Force      bool
+	Notifier   notify.Notifier
+}
+
+type RunPipelineWithLocalSourceOptions struct {
+	ConfigPath string
+	PipelineID string
+	SourceRoot string
+	DryRun     bool
+	Force      bool
 	Notifier   notify.Notifier
 }
 
 func Run(ctx context.Context, options RunOptions) error {
+	if err := ValidateOutputFormat(options.OutputFormat); err != nil {
+		return err
+	}
 	if err := ctx.Err(); err != nil {
 		return err
 	}
@@ -37,237 +57,290 @@ func Run(ctx context.Context, options RunOptions) error {
 	return runConfig(ctx, cfg, options)
 }
 
+func RunPipeline(ctx context.Context, options RunPipelineOptions) (RunReport, error) {
+	if err := ctx.Err(); err != nil {
+		return RunReport{}, err
+	}
+
+	configPath := options.ConfigPath
+	if configPath == "" {
+		configPath = config.DefaultConfigPath
+	}
+	cfg, err := config.LoadFile(configPath)
+	if err != nil {
+		return RunReport{}, err
+	}
+	return runPipelineConfig(ctx, cfg, options)
+}
+
+func RunPipelineWithLocalSource(ctx context.Context, options RunPipelineWithLocalSourceOptions) (RunReport, error) {
+	if err := ctx.Err(); err != nil {
+		return RunReport{}, err
+	}
+	if options.SourceRoot == "" {
+		return RunReport{}, fmt.Errorf("source root is required")
+	}
+
+	configPath := options.ConfigPath
+	if configPath == "" {
+		configPath = config.DefaultConfigPath
+	}
+	cfg, err := config.LoadFile(configPath)
+	if err != nil {
+		return RunReport{}, err
+	}
+	return runPipelineConfigWithLocalSource(ctx, cfg, options)
+}
+
 func runConfig(ctx context.Context, cfg config.Config, options RunOptions) error {
+	return runConfigWithBackendFactory(ctx, cfg, options, newBackendFactoryWithEnvironment)
+}
+
+type backendFactoryProvider func(config.Environment) *backendFactory
+
+func runPipelineConfig(ctx context.Context, cfg config.Config, options RunPipelineOptions) (RunReport, error) {
+	return runPipelineConfigWithBackendFactory(ctx, cfg, options, newBackendFactoryWithEnvironment)
+}
+
+func runPipelineConfigWithLocalSource(ctx context.Context, cfg config.Config, options RunPipelineWithLocalSourceOptions) (RunReport, error) {
+	return runPipelineConfigWithLocalSourceAndBackendFactory(ctx, cfg, options, newBackendFactoryWithEnvironment)
+}
+
+func runPipelineConfigWithBackendFactory(ctx context.Context, cfg config.Config, options RunPipelineOptions, provider backendFactoryProvider) (RunReport, error) {
+	pipeline, ok := findPipeline(cfg, options.PipelineID)
+	if !ok {
+		return RunReport{}, PipelineNotFoundError{ID: options.PipelineID}
+	}
+	return buildRunReportWithBackendFactory(ctx, config.Config{
+		Server:    cfg.Server,
+		Secrets:   cfg.Secrets,
+		Pipelines: []config.Pipeline{pipeline},
+	}, RunOptions{
+		DryRun:   options.DryRun,
+		Force:    options.Force,
+		Notifier: options.Notifier,
+	}, provider)
+}
+
+func runPipelineConfigWithLocalSourceAndBackendFactory(ctx context.Context, cfg config.Config, options RunPipelineWithLocalSourceOptions, provider backendFactoryProvider) (RunReport, error) {
+	pipeline, ok := findPipeline(cfg, options.PipelineID)
+	if !ok {
+		return RunReport{}, PipelineNotFoundError{ID: options.PipelineID}
+	}
+	return buildRunReport(ctx, config.Config{
+		Server:    cfg.Server,
+		Secrets:   cfg.Secrets,
+		Pipelines: []config.Pipeline{pipeline},
+	}, RunOptions{
+		DryRun:   options.DryRun,
+		Force:    options.Force,
+		Notifier: options.Notifier,
+	}, provider, &localSourceRoot{
+		pipelineID: options.PipelineID,
+		root:       options.SourceRoot,
+	})
+}
+
+func runConfigWithBackendFactory(ctx context.Context, cfg config.Config, options RunOptions, provider backendFactoryProvider) error {
+	report, err := buildRunReportWithBackendFactory(ctx, cfg, options, provider)
+	if err != nil && !IsPartialResultError(err) {
+		return err
+	}
+	if outputErr := WriteRunReport(options.Stdout, options.OutputFormat, report); outputErr != nil {
+		return outputErr
+	}
+	return err
+}
+
+func buildRunReportWithBackendFactory(ctx context.Context, cfg config.Config, options RunOptions, provider backendFactoryProvider) (RunReport, error) {
+	return buildRunReport(ctx, cfg, options, provider, nil)
+}
+
+type localSourceRoot struct {
+	pipelineID string
+	root       string
+}
+
+func buildRunReport(ctx context.Context, cfg config.Config, options RunOptions, provider backendFactoryProvider, sourceRoot *localSourceRoot) (RunReport, error) {
 	notifier := options.Notifier
 	if notifier == nil {
 		notifier = notify.Noop{}
 	}
 	summary := runSummary{dryRun: options.DryRun}
-	var failures runFailures
-	backends := newBackendFactory()
-	transforms := newTransformRegistry()
-	if options.Stdout != nil {
-		if _, err := fmt.Fprintf(options.Stdout, "Configured pipelines: %d\n", len(cfg.Pipelines)); err != nil {
-			return err
-		}
+	report := RunReport{
+		DryRun:    options.DryRun,
+		Pipelines: []RunPipelineSummary{},
+		Actions:   []RunActionRecord{},
 	}
+	var failures runFailures
+	secretLoad, err := config.LoadSecretEnvironment(cfg.Secrets.Directory, nil)
+	if err != nil {
+		return report, err
+	}
+	secretWarnings := secretConflictWarnings(secretLoad.Conflicts)
+	report.PreambleWarnings = append(report.PreambleWarnings, secretWarnings...)
+	report.addWarnings(secretWarnings)
+	backends := provider(secretLoad.Environment)
+	backends.readOnlyKnownHosts = options.DryRun
+	transforms := newTransformRegistry()
 	for _, pipeline := range cfg.Pipelines {
-		sourceBackend, err := backends.openSource(ctx, pipeline.Source)
+		pipelineWarnings := sshWarnings(pipeline)
+		report.addWarnings(pipelineWarnings)
+		sourceBackend, bundles, sourceBackendName, err := openPipelineSource(ctx, backends, pipeline, sourceRoot)
 		if err != nil {
-			return fmt.Errorf("pipeline %s: %w", pipeline.ID, err)
+			return report, err
 		}
-		bundles, err := bundle.Discover(ctx, sourceBackend, "")
-		if err != nil {
-			return fmt.Errorf("pipeline %s discover source bundles: %w", pipeline.ID, err)
-		}
-		if options.Stdout != nil {
-			if _, err := fmt.Fprintf(options.Stdout, "- pipeline=%s source=%s bundles=%d destinations=%s\n", pipeline.ID, pipeline.Source.Backend, len(bundles), destinationSummary(pipeline.Destinations)); err != nil {
-				return err
-			}
-		}
-		for _, sourceBundle := range bundles {
-			for _, destination := range pipeline.Destinations {
-				destinationBackend, err := backends.openDestination(ctx, destination)
-				if err != nil {
-					failures.add(pipeline.ID, destination.ID, storage.DisplayPath(sourceBundle.RootRelativePath), err)
-					summary.recordFailure()
-					if options.Stdout != nil {
-						writeErrorLine(options.Stdout, sourceBundle.RootRelativePath, destination.ID, err)
-					}
-					continue
+		report.Pipelines = append(report.Pipelines, RunPipelineSummary{
+			ID:            pipeline.ID,
+			SourceBackend: sourceBackendName,
+			BundleCount:   len(bundles),
+			Destinations:  destinationIDs(pipeline.Destinations),
+			Warnings:      pipelineWarnings,
+		})
+		pipelineIndex := len(report.Pipelines) - 1
+		for _, destination := range pipeline.Destinations {
+			selections := selectDestinationBundles(destination, bundles)
+			if isFixedPathDestination(destination) {
+				summary.recordFixedPath()
+				if options.DryRun {
+					warning := fixedPathSelectionWarning(pipeline.ID, destination.ID, selections, len(bundles))
+					report.addWarning(warning)
+					report.Pipelines[pipelineIndex].events = append(report.Pipelines[pipelineIndex].events, warningEvent(warning))
 				}
+			}
+			if len(selections) == 0 {
+				continue
+			}
+			destinationBackend, err := backends.openDestination(ctx, destination)
+			if err != nil {
+				for _, selection := range selections {
+					failures.add(pipeline.ID, destination.ID, destination.Backend, storage.DisplayPath(selection.SourceBundle.RootRelativePath), err)
+					summary.recordFailure()
+					report.Actions = append(report.Actions, errorAction(pipeline.ID, destination.ID, destination.Backend, selection.SourceBundle.RootRelativePath, err))
+					report.Pipelines[pipelineIndex].events = append(report.Pipelines[pipelineIndex].events, actionEvent(len(report.Actions)-1))
+				}
+				continue
+			}
+			closeDestination := true
+			deferCloseDestination := func() {
+				if closeDestination {
+					closeBackend(destinationBackend)
+					closeDestination = false
+				}
+			}
+			for _, selection := range selections {
+				sourceBundle := selection.SourceBundle
 				req := publish.Request{
 					PipelineID:            pipeline.ID,
 					DestinationID:         destination.ID,
 					SourceBundle:          sourceBundle,
 					SourceBackend:         sourceBackend,
 					DestinationBackend:    destinationBackend,
-					DestinationBundlePath: sourceBundle.RootRelativePath,
+					DestinationBundlePath: selection.DestinationBundlePath,
+					PathMapping:           destination.PathMap.Mode,
 					Publish:               *destination.Publish,
 					Transform:             destination.Transform,
+					Links:                 destination.Links,
 					Transformers:          transforms,
 					Transfer:              destination.Transfer,
 					DistributorVersion:    Version,
+					Force:                 options.Force,
 				}
 				plan, err := publish.Build(ctx, req)
-				if err != nil && plan.DestinationID == "" {
-					plan = publish.Plan{DestinationID: destination.ID, BundlePath: sourceBundle.RootRelativePath}
-				}
-				if options.Stdout != nil {
-					writePlanLine(options.Stdout, plan, err)
-				}
 				if err != nil {
-					failures.add(pipeline.ID, destination.ID, storage.DisplayPath(sourceBundle.RootRelativePath), err)
+					if plan.PipelineID == "" {
+						plan.PipelineID = pipeline.ID
+					}
+					if plan.DestinationID == "" {
+						plan.DestinationID = destination.ID
+					}
+					if plan.BundleID == "" {
+						plan.BundleID = sourceBundle.Manifest.ID
+					}
+					if plan.BundlePath == "" {
+						plan.BundlePath = sourceBundle.RootRelativePath
+					}
+					if plan.DestinationBundlePath == "" {
+						plan.DestinationBundlePath = selection.DestinationBundlePath
+					}
+				}
+				if isFixedPathDestination(destination) {
+					plan.PathMapping = config.PathMappingFixed
+					if options.DryRun && isDestructiveFixedPathAction(plan.Action) {
+						warning := fixedPathReplacementWarning(plan)
+						report.addWarning(warning)
+						report.Pipelines[pipelineIndex].events = append(report.Pipelines[pipelineIndex].events, warningEvent(warning))
+					}
+				}
+				report.Actions = append(report.Actions, runActionFromPlan(destination.Backend, plan, err))
+				report.Pipelines[pipelineIndex].events = append(report.Pipelines[pipelineIndex].events, actionEvent(len(report.Actions)-1))
+				if err != nil {
+					failures.add(pipeline.ID, destination.ID, destination.Backend, storage.DisplayPath(sourceBundle.RootRelativePath), err)
 					summary.recordFailure()
 					continue
 				}
 				summary.recordPlan(plan.Action)
 				if !options.DryRun {
 					if err := publish.Execute(ctx, req, plan); err != nil {
-						failures.add(pipeline.ID, destination.ID, storage.DisplayPath(sourceBundle.RootRelativePath), err)
+						failures.add(pipeline.ID, destination.ID, destination.Backend, storage.DisplayPath(sourceBundle.RootRelativePath), err)
 						summary.recordFailure()
 						continue
 					}
 					if shouldNotify(plan.Action) {
 						if err := notifier.Notify(ctx, notifyEvent(plan)); err != nil {
-							failures.add(pipeline.ID, destination.ID, storage.DisplayPath(sourceBundle.RootRelativePath), err)
+							failures.add(pipeline.ID, destination.ID, destination.Backend, storage.DisplayPath(sourceBundle.RootRelativePath), err)
 							summary.recordFailure()
 							continue
 						}
 					}
 				}
 			}
+			deferCloseDestination()
 		}
+		closeBackend(sourceBackend)
 	}
-	if options.Stdout != nil {
-		if _, err := fmt.Fprintln(options.Stdout, summary.Line()); err != nil {
-			return err
-		}
-	}
+	report.Summary = summary.Result()
+	report.OutputErrors = failures.outputErrors()
 	if len(failures.items) > 0 {
-		return failures
+		return report, failures
 	}
-	return nil
+	return report, nil
 }
 
-func writePlanLine(w io.Writer, plan publish.Plan, planErr error) {
-	if w == nil {
-		return
-	}
-	if planErr != nil {
-		destinationID := plan.DestinationID
-		if destinationID == "" {
-			destinationID = "unknown"
+func openPipelineSource(ctx context.Context, backends *backendFactory, pipeline config.Pipeline, sourceRoot *localSourceRoot) (storage.Backend, []bundle.Bundle, string, error) {
+	if sourceRoot != nil && sourceRoot.pipelineID == pipeline.ID {
+		sourceBackend, err := backends.openLocalPath(ctx, sourceRoot.root)
+		if err != nil {
+			return nil, nil, config.BackendLocal, fmt.Errorf("pipeline %s source backend %s: %w", pipeline.ID, config.BackendLocal, err)
 		}
-		fmt.Fprintf(w, "  - bundle=%s destination=%s action=error reason=%q\n", storage.DisplayPath(plan.BundlePath), destinationID, planErr.Error())
+		sourceBundle, err := bundle.Validate(ctx, sourceBackend, "")
+		if err != nil {
+			closeBackend(sourceBackend)
+			return nil, nil, config.BackendLocal, fmt.Errorf("pipeline %s source backend %s validate source bundle: %w", pipeline.ID, config.BackendLocal, err)
+		}
+		return sourceBackend, []bundle.Bundle{sourceBundle}, config.BackendLocal, nil
+	}
+
+	sourceBackend, err := backends.openSource(ctx, pipeline.Source)
+	if err != nil {
+		return nil, nil, pipeline.Source.Backend, fmt.Errorf("pipeline %s source backend %s: %w", pipeline.ID, pipeline.Source.Backend, err)
+	}
+	bundles, err := bundle.Discover(ctx, sourceBackend, "")
+	if err != nil {
+		closeBackend(sourceBackend)
+		return nil, nil, pipeline.Source.Backend, fmt.Errorf("pipeline %s source backend %s discover source bundles: %w", pipeline.ID, pipeline.Source.Backend, err)
+	}
+	return sourceBackend, bundles, pipeline.Source.Backend, nil
+}
+
+type closeableBackend interface {
+	Close() error
+}
+
+func closeBackend(backend storage.Backend) {
+	closeable, ok := backend.(closeableBackend)
+	if !ok {
 		return
 	}
-	fmt.Fprintf(w, "  - bundle=%s destination=%s action=%s outputs=%s reason=%q\n", storage.DisplayPath(plan.BundlePath), plan.DestinationID, plan.Action, outputSummary(plan.Outputs), plan.Reason)
-}
-
-func writeErrorLine(w io.Writer, bundlePath, destinationID string, err error) {
-	if w == nil {
-		return
-	}
-	fmt.Fprintf(w, "  - bundle=%s destination=%s action=error reason=%q\n", storage.DisplayPath(bundlePath), destinationID, err.Error())
-}
-
-func outputSummary(outputs []publish.Output) string {
-	if len(outputs) == 0 {
-		return "none"
-	}
-	paths := make([]string, 0, len(outputs))
-	for _, output := range outputs {
-		paths = append(paths, output.DestinationPath)
-	}
-	return strings.Join(paths, ",")
-}
-
-func destinationSummary(destinations []config.Destination) string {
-	if len(destinations) == 0 {
-		return "none"
-	}
-	ids := make([]string, 0, len(destinations))
-	for _, destination := range destinations {
-		ids = append(ids, destination.ID)
-	}
-	return strings.Join(ids, ",")
-}
-
-func shouldNotify(action publish.Action) bool {
-	return action == publish.ActionPublishNew || action == publish.ActionReplaceOlder
-}
-
-func notifyEvent(plan publish.Plan) notify.Event {
-	outputs := make([]notify.Output, 0, len(plan.Outputs))
-	for _, output := range plan.Outputs {
-		outputs = append(outputs, notify.Output{
-			Path:       output.DestinationPath,
-			Kind:       output.Kind,
-			SourcePath: output.SourcePath,
-			Transform:  output.Transform,
-			SHA256:     output.SHA256,
-			Size:       output.Size,
-		})
-	}
-	return notify.Event{
-		PipelineID:    plan.PipelineID,
-		DestinationID: plan.DestinationID,
-		BundleID:      plan.BundleID,
-		BundlePath:    plan.BundlePath,
-		Action:        string(plan.Action),
-		Outputs:       outputs,
-	}
-}
-
-type runSummary struct {
-	dryRun       bool
-	planned      int
-	publishNew   int
-	replaceOlder int
-	skipped      int
-	failures     int
-}
-
-func (s *runSummary) recordPlan(action publish.Action) {
-	s.planned++
-	switch action {
-	case publish.ActionPublishNew:
-		s.publishNew++
-	case publish.ActionReplaceOlder:
-		s.replaceOlder++
-	case publish.ActionSkipSame, publish.ActionSkipDestinationNewer:
-		s.skipped++
-	}
-}
-
-func (s *runSummary) recordFailure() {
-	s.failures++
-}
-
-func (s runSummary) Line() string {
-	status := "ok"
-	if s.failures > 0 {
-		status = "failed"
-	}
-	return fmt.Sprintf("Final status: %s planned=%d publish_new=%d replace_older=%d skipped=%d failed=%d dry_run=%t", status, s.planned, s.publishNew, s.replaceOlder, s.skipped, s.failures, s.dryRun)
-}
-
-type runFailure struct {
-	pipelineID    string
-	destinationID string
-	bundlePath    string
-	err           error
-}
-
-type runFailures struct {
-	items []runFailure
-}
-
-func (f *runFailures) add(pipelineID, destinationID, bundlePath string, err error) {
-	f.items = append(f.items, runFailure{
-		pipelineID:    pipelineID,
-		destinationID: destinationID,
-		bundlePath:    bundlePath,
-		err:           err,
-	})
-}
-
-func (f runFailures) Error() string {
-	if len(f.items) == 0 {
-		return ""
-	}
-	parts := make([]string, 0, len(f.items))
-	for _, item := range f.items {
-		parts = append(parts, fmt.Sprintf("pipeline %s destination %s bundle %s: %v", item.pipelineID, item.destinationID, item.bundlePath, item.err))
-	}
-	return "run failed: " + strings.Join(parts, "; ")
-}
-
-func (f runFailures) Unwrap() error {
-	errs := make([]error, 0, len(f.items))
-	for _, item := range f.items {
-		errs = append(errs, item.err)
-	}
-	return errors.Join(errs...)
+	_ = closeable.Close()
 }

internal/app/run_coordinator.go

@@ -0,0 +1,131 @@
+package app
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"sync"
+	"time"
+)
+
+type PipelineRunID string
+
+type PipelineRunStatus string
+
+const (
+	PipelineRunRunning   PipelineRunStatus = "running"
+	PipelineRunSucceeded PipelineRunStatus = "succeeded"
+	PipelineRunFailed    PipelineRunStatus = "failed"
+)
+
+type PipelineRunRecord struct {
+	ID         PipelineRunID     `json:"id"`
+	PipelineID string            `json:"pipeline_id"`
+	Status     PipelineRunStatus `json:"status"`
+	StartedAt  time.Time         `json:"started_at"`
+	FinishedAt *time.Time        `json:"finished_at,omitempty"`
+	Report     RunReport         `json:"report,omitempty"`
+	Error      string            `json:"error,omitempty"`
+}
+
+type DuplicatePipelineRunError struct {
+	PipelineID string
+	RunID      PipelineRunID
+}
+
+func (err DuplicatePipelineRunError) Error() string {
+	if err.RunID == "" {
+		return fmt.Sprintf("pipeline %q already has an active run", err.PipelineID)
+	}
+	return fmt.Sprintf("pipeline %q already has active run %s", err.PipelineID, err.RunID)
+}
+
+func IsDuplicatePipelineRun(err error) bool {
+	var duplicate DuplicatePipelineRunError
+	return errors.As(err, &duplicate)
+}
+
+type PipelineRunCoordinator struct {
+	ctx    context.Context
+	run    pipelineRunFunc
+	now    func() time.Time
+	mu     sync.Mutex
+	nextID uint64
+	active map[string]PipelineRunRecord
+}
+
+type pipelineRunFunc func(context.Context, RunPipelineOptions) (RunReport, error)
+
+func NewPipelineRunCoordinator(ctx context.Context) *PipelineRunCoordinator {
+	return newPipelineRunCoordinator(ctx, RunPipeline)
+}
+
+func newPipelineRunCoordinator(ctx context.Context, run pipelineRunFunc) *PipelineRunCoordinator {
+	if ctx == nil {
+		ctx = context.Background()
+	}
+	return &PipelineRunCoordinator{
+		ctx:    ctx,
+		run:    run,
+		now:    time.Now,
+		active: map[string]PipelineRunRecord{},
+	}
+}
+
+func (coordinator *PipelineRunCoordinator) RunPipeline(ctx context.Context, options RunPipelineOptions) (PipelineRunRecord, error) {
+	if ctx == nil {
+		ctx = context.Background()
+	}
+	if err := ctx.Err(); err != nil {
+		return PipelineRunRecord{}, err
+	}
+	record, err := coordinator.admit(options.PipelineID)
+	if err != nil {
+		return PipelineRunRecord{}, err
+	}
+	defer coordinator.clear(options.PipelineID)
+
+	report, runErr := coordinator.run(coordinator.ctx, options)
+	record.Report = report
+	finishedAt := coordinator.now().UTC()
+	record.FinishedAt = &finishedAt
+	if runErr != nil {
+		record.Status = PipelineRunFailed
+		record.Error = runErr.Error()
+		return record, runErr
+	}
+	record.Status = PipelineRunSucceeded
+	return record, nil
+}
+
+func (coordinator *PipelineRunCoordinator) admit(pipelineID string) (PipelineRunRecord, error) {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	if active, ok := coordinator.active[pipelineID]; ok {
+		return PipelineRunRecord{}, DuplicatePipelineRunError{
+			PipelineID: pipelineID,
+			RunID:      active.ID,
+		}
+	}
+	coordinator.nextID++
+	record := PipelineRunRecord{
+		ID:         PipelineRunID(fmt.Sprintf("run-%016d", coordinator.nextID)),
+		PipelineID: pipelineID,
+		Status:     PipelineRunRunning,
+		StartedAt:  coordinator.now().UTC(),
+	}
+	coordinator.active[pipelineID] = record
+	return record, nil
+}
+
+func (coordinator *PipelineRunCoordinator) clear(pipelineID string) {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	delete(coordinator.active, pipelineID)
+}
+
+func (coordinator *PipelineRunCoordinator) activeCount() int {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	return len(coordinator.active)
+}

internal/app/run_coordinator_test.go

@@ -0,0 +1,223 @@
+package app
+
+import (
+	"context"
+	"errors"
+	"sync"
+	"testing"
+	"time"
+)
+
+func TestPipelineRunCoordinatorRejectsDuplicateActiveRun(t *testing.T) {
+	started := make(chan struct{})
+	release := make(chan struct{})
+	var startedOnce sync.Once
+	coordinator := newPipelineRunCoordinator(context.Background(), func(ctx context.Context, options RunPipelineOptions) (RunReport, error) {
+		startedOnce.Do(func() {
+			close(started)
+		})
+		<-release
+		return RunReport{}, nil
+	})
+	firstResult := make(chan runCoordinatorTestResult, 1)
+
+	go func() {
+		record, err := coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "reports"})
+		firstResult <- runCoordinatorTestResult{record: record, err: err}
+	}()
+	waitForSignal(t, started, "first run to start")
+
+	_, err := coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "reports"})
+	if err == nil || !IsDuplicatePipelineRun(err) {
+		t.Fatalf("RunPipeline() error = %v, want duplicate active run", err)
+	}
+	close(release)
+	result := waitForRunResult(t, firstResult)
+	if result.err != nil {
+		t.Fatalf("first RunPipeline() error = %v", result.err)
+	}
+	if result.record.Status != PipelineRunSucceeded || result.record.ID == "" || result.record.FinishedAt == nil {
+		t.Fatalf("first record = %#v, want succeeded completed record", result.record)
+	}
+	if got := coordinator.activeCount(); got != 0 {
+		t.Fatalf("active count = %d, want 0", got)
+	}
+}
+
+func TestPipelineRunCoordinatorAllowsDifferentActivePipelines(t *testing.T) {
+	started := make(chan string, 2)
+	release := make(chan struct{})
+	coordinator := newPipelineRunCoordinator(context.Background(), func(ctx context.Context, options RunPipelineOptions) (RunReport, error) {
+		started <- options.PipelineID
+		<-release
+		return RunReport{}, nil
+	})
+	firstResult := make(chan runCoordinatorTestResult, 1)
+	secondResult := make(chan runCoordinatorTestResult, 1)
+
+	go func() {
+		record, err := coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "reports-one"})
+		firstResult <- runCoordinatorTestResult{record: record, err: err}
+	}()
+	go func() {
+		record, err := coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "reports-two"})
+		secondResult <- runCoordinatorTestResult{record: record, err: err}
+	}()
+	startedPipelines := map[string]bool{
+		waitForPipelineID(t, started): true,
+		waitForPipelineID(t, started): true,
+	}
+	if !startedPipelines["reports-one"] || !startedPipelines["reports-two"] {
+		t.Fatalf("started pipelines = %#v, want both requested pipelines", startedPipelines)
+	}
+	if got := coordinator.activeCount(); got != 2 {
+		t.Fatalf("active count = %d, want 2", got)
+	}
+
+	close(release)
+	first := waitForRunResult(t, firstResult)
+	second := waitForRunResult(t, secondResult)
+	if first.err != nil || second.err != nil {
+		t.Fatalf("RunPipeline() errors = %v, %v; want nil", first.err, second.err)
+	}
+	if first.record.ID == second.record.ID {
+		t.Fatalf("run IDs matched: %q", first.record.ID)
+	}
+	if got := coordinator.activeCount(); got != 0 {
+		t.Fatalf("active count = %d, want 0", got)
+	}
+}
+
+func TestPipelineRunCoordinatorClearsActiveRunAfterSuccess(t *testing.T) {
+	coordinator := newPipelineRunCoordinator(context.Background(), func(ctx context.Context, options RunPipelineOptions) (RunReport, error) {
+		return RunReport{DryRun: options.DryRun}, nil
+	})
+
+	first, err := coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "reports", DryRun: true})
+	if err != nil {
+		t.Fatalf("first RunPipeline() error = %v", err)
+	}
+	second, err := coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "reports"})
+	if err != nil {
+		t.Fatalf("second RunPipeline() error = %v", err)
+	}
+	if first.Status != PipelineRunSucceeded || second.Status != PipelineRunSucceeded {
+		t.Fatalf("statuses = %s, %s; want succeeded", first.Status, second.Status)
+	}
+	if !first.Report.DryRun {
+		t.Fatalf("first report dry_run = false, want true")
+	}
+	if got := coordinator.activeCount(); got != 0 {
+		t.Fatalf("active count = %d, want 0", got)
+	}
+}
+
+func TestPipelineRunCoordinatorClearsActiveRunAfterFailure(t *testing.T) {
+	runError := errors.New("run failed")
+	attempt := 0
+	coordinator := newPipelineRunCoordinator(context.Background(), func(ctx context.Context, options RunPipelineOptions) (RunReport, error) {
+		attempt++
+		if attempt == 1 {
+			return RunReport{}, runError
+		}
+		return RunReport{}, nil
+	})
+
+	first, err := coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "reports"})
+	if !errors.Is(err, runError) {
+		t.Fatalf("first RunPipeline() error = %v, want run failure", err)
+	}
+	if first.Status != PipelineRunFailed || first.Error != runError.Error() || first.FinishedAt == nil {
+		t.Fatalf("first record = %#v, want failed completed record", first)
+	}
+	second, err := coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "reports"})
+	if err != nil {
+		t.Fatalf("second RunPipeline() error = %v", err)
+	}
+	if second.Status != PipelineRunSucceeded {
+		t.Fatalf("second status = %s, want succeeded", second.Status)
+	}
+	if got := coordinator.activeCount(); got != 0 {
+		t.Fatalf("active count = %d, want 0", got)
+	}
+}
+
+func TestPipelineRunCoordinatorClearsActiveRunAfterCancellation(t *testing.T) {
+	runContext, cancel := context.WithCancel(context.Background())
+	coordinator := newPipelineRunCoordinator(runContext, func(ctx context.Context, options RunPipelineOptions) (RunReport, error) {
+		return RunReport{}, ctx.Err()
+	})
+	cancel()
+
+	record, err := coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "reports"})
+	if !errors.Is(err, context.Canceled) {
+		t.Fatalf("RunPipeline() error = %v, want context canceled", err)
+	}
+	if record.Status != PipelineRunFailed || record.Error != context.Canceled.Error() {
+		t.Fatalf("record = %#v, want failed cancellation record", record)
+	}
+	_, err = coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "reports"})
+	if !errors.Is(err, context.Canceled) {
+		t.Fatalf("second RunPipeline() error = %v, want context canceled", err)
+	}
+	if IsDuplicatePipelineRun(err) {
+		t.Fatalf("second RunPipeline() error = %v, want cancellation instead of duplicate", err)
+	}
+	if got := coordinator.activeCount(); got != 0 {
+		t.Fatalf("active count = %d, want 0", got)
+	}
+}
+
+func TestPipelineRunCoordinatorUnknownPipelineDoesNotRemainActive(t *testing.T) {
+	coordinator := newPipelineRunCoordinator(context.Background(), func(ctx context.Context, options RunPipelineOptions) (RunReport, error) {
+		return RunReport{}, PipelineNotFoundError{ID: options.PipelineID}
+	})
+
+	_, err := coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "missing"})
+	if err == nil || !IsPipelineNotFound(err) {
+		t.Fatalf("RunPipeline() error = %v, want pipeline not found", err)
+	}
+	_, err = coordinator.RunPipeline(context.Background(), RunPipelineOptions{PipelineID: "missing"})
+	if err == nil || !IsPipelineNotFound(err) || IsDuplicatePipelineRun(err) {
+		t.Fatalf("second RunPipeline() error = %v, want pipeline not found without duplicate", err)
+	}
+	if got := coordinator.activeCount(); got != 0 {
+		t.Fatalf("active count = %d, want 0", got)
+	}
+}
+
+type runCoordinatorTestResult struct {
+	record PipelineRunRecord
+	err    error
+}
+
+func waitForSignal(t *testing.T, signal <-chan struct{}, name string) {
+	t.Helper()
+	select {
+	case <-signal:
+	case <-time.After(time.Second):
+		t.Fatalf("timed out waiting for %s", name)
+	}
+}
+
+func waitForPipelineID(t *testing.T, pipelineIDs <-chan string) string {
+	t.Helper()
+	select {
+	case pipelineID := <-pipelineIDs:
+		return pipelineID
+	case <-time.After(time.Second):
+		t.Fatalf("timed out waiting for pipeline start")
+		return ""
+	}
+}
+
+func waitForRunResult(t *testing.T, results <-chan runCoordinatorTestResult) runCoordinatorTestResult {
+	t.Helper()
+	select {
+	case result := <-results:
+		return result
+	case <-time.After(time.Second):
+		t.Fatalf("timed out waiting for run result")
+		return runCoordinatorTestResult{}
+	}
+}

internal/app/run_failures.go

@@ -0,0 +1,70 @@
+package app
+
+import (
+	"errors"
+	"fmt"
+	"strings"
+)
+
+type runFailure struct {
+	pipelineID    string
+	destinationID string
+	backend       string
+	bundlePath    string
+	err           error
+}
+
+type runFailures struct {
+	items []runFailure
+}
+
+func (f *runFailures) add(pipelineID, destinationID, backend, bundlePath string, err error) {
+	f.items = append(f.items, runFailure{
+		pipelineID:    pipelineID,
+		destinationID: destinationID,
+		backend:       backend,
+		bundlePath:    bundlePath,
+		err:           err,
+	})
+}
+
+func (f runFailures) Error() string {
+	if len(f.items) == 0 {
+		return ""
+	}
+	parts := make([]string, 0, len(f.items))
+	for _, item := range f.items {
+		parts = append(parts, fmt.Sprintf("pipeline %s destination %s backend %s bundle %s: %v", item.pipelineID, item.destinationID, item.backend, item.bundlePath, item.err))
+	}
+	return "run failed: " + strings.Join(parts, "; ")
+}
+
+func (f runFailures) outputErrors() []OutputError {
+	if len(f.items) == 0 {
+		return nil
+	}
+	errors := make([]OutputError, 0, len(f.items))
+	for _, item := range f.items {
+		errors = append(errors, OutputError{
+			PipelineID:    item.pipelineID,
+			DestinationID: item.destinationID,
+			Backend:       item.backend,
+			BundlePath:    item.bundlePath,
+			Message:       item.err.Error(),
+		})
+	}
+	return errors
+}
+
+func IsPartialResultError(err error) bool {
+	var failures runFailures
+	return errors.As(err, &failures)
+}
+
+func (f runFailures) Unwrap() error {
+	errs := make([]error, 0, len(f.items))
+	for _, item := range f.items {
+		errs = append(errs, item.err)
+	}
+	return errors.Join(errs...)
+}

internal/app/run_notify.go

@@ -0,0 +1,33 @@
+package app
+
+import (
+	"gitea.maximumdirect.net/eric/distributor/internal/notify"
+	"gitea.maximumdirect.net/eric/distributor/internal/publish"
+)
+
+func shouldNotify(action publish.Action) bool {
+	return action == publish.ActionPublishNew || action == publish.ActionReplaceOlder || action == publish.ActionForceReplace
+}
+
+func notifyEvent(plan publish.Plan) notify.Event {
+	outputs := make([]notify.Output, 0, len(plan.Outputs))
+	for _, output := range plan.Outputs {
+		stateOutput := output.StateOutputFile()
+		outputs = append(outputs, notify.Output{
+			Path:       stateOutput.Path,
+			Kind:       stateOutput.Kind,
+			SourcePath: stateOutput.SourcePath,
+			Transform:  stateOutput.Transform,
+			SHA256:     stateOutput.SHA256,
+			Size:       stateOutput.Size,
+		})
+	}
+	return notify.Event{
+		PipelineID:    plan.PipelineID,
+		DestinationID: plan.DestinationID,
+		BundleID:      plan.BundleID,
+		BundlePath:    plan.BundlePath,
+		Action:        string(plan.Action),
+		Outputs:       outputs,
+	}
+}

internal/app/run_output.go

@@ -0,0 +1,218 @@
+package app
+
+import (
+	"fmt"
+	"io"
+	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/publish"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+)
+
+func WriteRunReport(w io.Writer, format OutputFormat, report RunReport) error {
+	if IsJSONOutput(format) {
+		return WriteJSONEnvelope(w, "run", len(report.OutputErrors) == 0, report.Warnings, report, report.OutputErrors)
+	}
+	return writeRunReportText(w, report)
+}
+
+func writeRunReportText(w io.Writer, report RunReport) error {
+	if w == nil {
+		return nil
+	}
+	if err := writeWarnings(w, report.PreambleWarnings); err != nil {
+		return err
+	}
+	if _, err := fmt.Fprintf(w, "Configured pipelines: %d\n", len(report.Pipelines)); err != nil {
+		return err
+	}
+	for _, pipeline := range report.Pipelines {
+		if err := writeWarnings(w, pipeline.Warnings); err != nil {
+			return err
+		}
+		if _, err := fmt.Fprintf(w, "- pipeline=%s source=%s bundles=%d destinations=%s\n", pipeline.ID, pipeline.SourceBackend, pipeline.BundleCount, destinationIDSummary(pipeline.Destinations)); err != nil {
+			return err
+		}
+		for _, event := range pipeline.events {
+			if event.warning != nil {
+				if err := writeWarnings(w, []OutputWarning{*event.warning}); err != nil {
+					return err
+				}
+				continue
+			}
+			if event.actionIndex < 0 || event.actionIndex >= len(report.Actions) {
+				continue
+			}
+			writeRunActionLine(w, report.Actions[event.actionIndex])
+		}
+	}
+	_, err := fmt.Fprintln(w, report.Summary.Line())
+	return err
+}
+
+func writeRunActionLine(w io.Writer, action RunActionRecord) {
+	if action.Action == "error" {
+		destinationID := action.DestinationID
+		if destinationID == "" {
+			destinationID = "unknown"
+		}
+		fmt.Fprintf(w, "  - bundle=%s destination=%s backend=%s%s action=error reason=%q\n", action.BundlePath, destinationID, action.Backend, pathMappingRecordSummary(action), action.Reason)
+		return
+	}
+	fmt.Fprintf(w, "  - bundle=%s destination=%s backend=%s%s action=%s outputs=%s reason=%q\n", action.BundlePath, action.DestinationID, action.Backend, pathMappingRecordSummary(action), action.Action, outputRecordSummary(action.Outputs), action.Reason)
+}
+
+func pathMappingRecordSummary(action RunActionRecord) string {
+	if action.PathMapping != config.PathMappingFixed {
+		return ""
+	}
+	return fmt.Sprintf(" path_mapping=fixed target=%s", action.DestinationPath)
+}
+
+func outputRecordSummary(outputs []RunOutputRecord) string {
+	if len(outputs) == 0 {
+		return "none"
+	}
+	paths := make([]string, 0, len(outputs))
+	for _, output := range outputs {
+		paths = append(paths, output.Path)
+	}
+	return strings.Join(paths, ",")
+}
+
+func destinationIDSummary(ids []string) string {
+	if len(ids) == 0 {
+		return "none"
+	}
+	return strings.Join(ids, ",")
+}
+
+type RunReport struct {
+	DryRun           bool                 `json:"dry_run"`
+	Pipelines        []RunPipelineSummary `json:"pipelines"`
+	Actions          []RunActionRecord    `json:"actions"`
+	Summary          RunSummaryCounters   `json:"summary"`
+	Warnings         []OutputWarning      `json:"-"`
+	OutputErrors     []OutputError        `json:"-"`
+	PreambleWarnings []OutputWarning      `json:"-"`
+}
+
+func (r *RunReport) addWarning(warning OutputWarning) {
+	r.Warnings = append(r.Warnings, warning)
+}
+
+func (r *RunReport) addWarnings(warnings []OutputWarning) {
+	r.Warnings = append(r.Warnings, warnings...)
+}
+
+type RunPipelineSummary struct {
+	ID            string          `json:"id"`
+	SourceBackend string          `json:"source_backend"`
+	BundleCount   int             `json:"bundle_count"`
+	Destinations  []string        `json:"destinations"`
+	Warnings      []OutputWarning `json:"-"`
+	events        []runPipelineEvent
+}
+
+type runPipelineEvent struct {
+	warning     *OutputWarning
+	actionIndex int
+}
+
+func warningEvent(warning OutputWarning) runPipelineEvent {
+	return runPipelineEvent{warning: &warning, actionIndex: -1}
+}
+
+func actionEvent(actionIndex int) runPipelineEvent {
+	return runPipelineEvent{actionIndex: actionIndex}
+}
+
+type RunActionRecord struct {
+	PipelineID      string            `json:"pipeline_id,omitempty"`
+	DestinationID   string            `json:"destination_id"`
+	Backend         string            `json:"backend"`
+	BundleID        string            `json:"bundle_id,omitempty"`
+	BundlePath      string            `json:"bundle_path"`
+	DestinationPath string            `json:"destination_path"`
+	PathMapping     string            `json:"path_mapping,omitempty"`
+	Action          string            `json:"action"`
+	PrimaryURL      string            `json:"primary_url,omitempty"`
+	Reason          string            `json:"reason,omitempty"`
+	Outputs         []RunOutputRecord `json:"outputs"`
+}
+
+type RunOutputRecord struct {
+	Path       string `json:"path"`
+	Kind       string `json:"kind"`
+	SourcePath string `json:"source_path,omitempty"`
+	Transform  string `json:"transform,omitempty"`
+	URL        string `json:"url,omitempty"`
+	SHA256     string `json:"sha256"`
+	Size       int64  `json:"size"`
+}
+
+func runActionFromPlan(backend string, plan publish.Plan, planErr error) RunActionRecord {
+	if planErr != nil {
+		destinationID := plan.DestinationID
+		if destinationID == "" {
+			destinationID = "unknown"
+		}
+		return RunActionRecord{
+			PipelineID:      plan.PipelineID,
+			DestinationID:   destinationID,
+			Backend:         backend,
+			BundleID:        plan.BundleID,
+			BundlePath:      storage.DisplayPath(plan.BundlePath),
+			DestinationPath: storage.DisplayPath(plan.DestinationBundlePath),
+			PathMapping:     plan.PathMapping,
+			Action:          "error",
+			PrimaryURL:      plan.PrimaryURL,
+			Reason:          planErr.Error(),
+			Outputs:         []RunOutputRecord{},
+		}
+	}
+	return RunActionRecord{
+		PipelineID:      plan.PipelineID,
+		DestinationID:   plan.DestinationID,
+		Backend:         backend,
+		BundleID:        plan.BundleID,
+		BundlePath:      storage.DisplayPath(plan.BundlePath),
+		DestinationPath: storage.DisplayPath(plan.DestinationBundlePath),
+		PathMapping:     plan.PathMapping,
+		Action:          string(plan.Action),
+		PrimaryURL:      plan.PrimaryURL,
+		Reason:          plan.Reason,
+		Outputs:         runOutputsFromPlan(plan.Outputs),
+	}
+}
+
+func errorAction(pipelineID, destinationID, backend, bundlePath string, err error) RunActionRecord {
+	return RunActionRecord{
+		PipelineID:      pipelineID,
+		DestinationID:   destinationID,
+		Backend:         backend,
+		BundlePath:      storage.DisplayPath(bundlePath),
+		DestinationPath: storage.DisplayPath(bundlePath),
+		Action:          "error",
+		Reason:          err.Error(),
+		Outputs:         []RunOutputRecord{},
+	}
+}
+
+func runOutputsFromPlan(outputs []publish.Output) []RunOutputRecord {
+	results := make([]RunOutputRecord, 0, len(outputs))
+	for _, output := range outputs {
+		stateOutput := output.StateOutputFile()
+		results = append(results, RunOutputRecord{
+			Path:       stateOutput.Path,
+			Kind:       stateOutput.Kind,
+			SourcePath: stateOutput.SourcePath,
+			Transform:  stateOutput.Transform,
+			URL:        stateOutput.URL,
+			SHA256:     stateOutput.SHA256,
+			Size:       stateOutput.Size,
+		})
+	}
+	return results
+}

internal/app/run_selection.go

@@ -0,0 +1,79 @@
+package app
+
+import (
+	"fmt"
+	"sort"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/publish"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+)
+
+type destinationBundleSelection struct {
+	SourceBundle          bundle.Bundle
+	DestinationBundlePath string
+}
+
+func selectDestinationBundles(destination config.Destination, bundles []bundle.Bundle) []destinationBundleSelection {
+	if !isFixedPathDestination(destination) {
+		selections := make([]destinationBundleSelection, 0, len(bundles))
+		for _, sourceBundle := range bundles {
+			selections = append(selections, destinationBundleSelection{
+				SourceBundle:          sourceBundle,
+				DestinationBundlePath: sourceBundle.RootRelativePath,
+			})
+		}
+		return selections
+	}
+	if len(bundles) == 0 {
+		return nil
+	}
+	sourceBundle := newestBundle(bundles)
+	return []destinationBundleSelection{{
+		SourceBundle:          sourceBundle,
+		DestinationBundlePath: "",
+	}}
+}
+
+func newestBundle(bundles []bundle.Bundle) bundle.Bundle {
+	if len(bundles) == 0 {
+		return bundle.Bundle{}
+	}
+	sorted := append([]bundle.Bundle(nil), bundles...)
+	sort.Slice(sorted, func(i, j int) bool {
+		if sorted[i].Manifest.Created.Equal(sorted[j].Manifest.Created) {
+			return sorted[i].RootRelativePath < sorted[j].RootRelativePath
+		}
+		return sorted[i].Manifest.Created.After(sorted[j].Manifest.Created)
+	})
+	return sorted[0]
+}
+
+func isFixedPathDestination(destination config.Destination) bool {
+	return destination.PathMap.Mode == config.PathMappingFixed
+}
+
+func fixedPathSelectionWarning(pipelineID, destinationID string, selections []destinationBundleSelection, candidateCount int) OutputWarning {
+	selected := "none"
+	if len(selections) > 0 {
+		selected = storage.DisplayPath(selections[0].SourceBundle.RootRelativePath)
+	}
+	return OutputWarning{Message: fmt.Sprintf("pipeline=%s destination=%s path_mapping=fixed candidates=%d selected_bundle=%s destination_bundle=.", pipelineID, destinationID, candidateCount, selected)}
+}
+
+func isDestructiveFixedPathAction(action publish.Action) bool {
+	return action == publish.ActionReplaceOlder || action == publish.ActionForceReplace
+}
+
+func fixedPathReplacementWarning(plan publish.Plan) OutputWarning {
+	return OutputWarning{Message: fmt.Sprintf("pipeline=%s destination=%s path_mapping=fixed action=%s replaces destination root for selected_bundle=%s", plan.PipelineID, plan.DestinationID, plan.Action, storage.DisplayPath(plan.BundlePath))}
+}
+
+func destinationIDs(destinations []config.Destination) []string {
+	ids := make([]string, 0, len(destinations))
+	for _, destination := range destinations {
+		ids = append(ids, destination.ID)
+	}
+	return ids
+}

internal/app/run_summary.go

@@ -0,0 +1,74 @@
+package app
+
+import (
+	"fmt"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/publish"
+)
+
+type runSummary struct {
+	dryRun       bool
+	planned      int
+	publishNew   int
+	replaceOlder int
+	forceReplace int
+	skipped      int
+	failures     int
+	fixedPath    int
+}
+
+func (s *runSummary) recordPlan(action publish.Action) {
+	s.planned++
+	switch action {
+	case publish.ActionPublishNew:
+		s.publishNew++
+	case publish.ActionReplaceOlder:
+		s.replaceOlder++
+	case publish.ActionForceReplace:
+		s.forceReplace++
+	case publish.ActionSkipSame, publish.ActionSkipDestinationNewer:
+		s.skipped++
+	}
+}
+
+func (s *runSummary) recordFailure() {
+	s.failures++
+}
+
+func (s *runSummary) recordFixedPath() {
+	s.fixedPath++
+}
+
+type RunSummaryCounters struct {
+	Status       string `json:"status"`
+	Planned      int    `json:"planned"`
+	PublishNew   int    `json:"publish_new"`
+	ReplaceOlder int    `json:"replace_older"`
+	ForceReplace int    `json:"force_replace"`
+	Skipped      int    `json:"skipped"`
+	Failed       int    `json:"failed"`
+	DryRun       bool   `json:"dry_run"`
+	FixedPath    int    `json:"fixed_path"`
+}
+
+func (s RunSummaryCounters) Line() string {
+	return fmt.Sprintf("Final status: %s planned=%d publish_new=%d replace_older=%d force_replace=%d skipped=%d failed=%d dry_run=%t fixed_path=%d", s.Status, s.Planned, s.PublishNew, s.ReplaceOlder, s.ForceReplace, s.Skipped, s.Failed, s.DryRun, s.FixedPath)
+}
+
+func (s runSummary) Result() RunSummaryCounters {
+	status := "ok"
+	if s.failures > 0 {
+		status = "failed"
+	}
+	return RunSummaryCounters{
+		Status:       status,
+		Planned:      s.planned,
+		PublishNew:   s.publishNew,
+		ReplaceOlder: s.replaceOlder,
+		ForceReplace: s.forceReplace,
+		Skipped:      s.skipped,
+		Failed:       s.failures,
+		DryRun:       s.dryRun,
+		FixedPath:    s.fixedPath,
+	}
+}

internal/app/run_warnings.go

@@ -0,0 +1,47 @@
+package app
+
+import (
+	"fmt"
+	"io"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+)
+
+func secretConflictWarnings(conflicts []config.SecretConflict) []OutputWarning {
+	warnings := make([]OutputWarning, 0, len(conflicts))
+	for _, conflict := range conflicts {
+		warnings = append(warnings, OutputWarning{
+			Message: fmt.Sprintf("secret %s ignored because the real environment already has that variable", conflict.Name),
+		})
+	}
+	return warnings
+}
+
+func sshWarnings(pipeline config.Pipeline) []OutputWarning {
+	var warnings []OutputWarning
+	if pipeline.Source.Backend == config.BackendSSH && pipeline.Source.SSH.HostKeyPolicy == config.HostKeyPolicyOff {
+		warnings = append(warnings, OutputWarning{
+			Message: fmt.Sprintf("pipeline=%s source host_key_policy=off disables SSH host key checking", pipeline.ID),
+		})
+	}
+	for _, destination := range pipeline.Destinations {
+		if destination.Backend == config.BackendSSH && destination.SSH.HostKeyPolicy == config.HostKeyPolicyOff {
+			warnings = append(warnings, OutputWarning{
+				Message: fmt.Sprintf("pipeline=%s destination=%s host_key_policy=off disables SSH host key checking", pipeline.ID, destination.ID),
+			})
+		}
+	}
+	return warnings
+}
+
+func writeWarnings(w io.Writer, warnings []OutputWarning) error {
+	if w == nil {
+		return nil
+	}
+	for _, warning := range warnings {
+		if _, err := fmt.Fprintf(w, "Warning: %s\n", warning.Message); err != nil {
+			return err
+		}
+	}
+	return nil
+}

internal/app/serve.go

@@ -0,0 +1,62 @@
+package app
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"net"
+	"net/http"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+)
+
+type ServeOptions struct {
+	ConfigPath string
+}
+
+func Serve(ctx context.Context, options ServeOptions) error {
+	if ctx == nil {
+		ctx = context.Background()
+	}
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+
+	configPath := options.ConfigPath
+	if configPath == "" {
+		configPath = config.DefaultConfigPath
+	}
+	cfg, err := config.LoadFile(configPath)
+	if err != nil {
+		return err
+	}
+	secretLoad, err := config.LoadSecretEnvironment(cfg.Secrets.Directory, nil)
+	if err != nil {
+		return err
+	}
+
+	handler, err := newUploadHTTPHandler(ctx, cfg, secretLoad.Environment)
+	if err != nil {
+		return err
+	}
+	listener, err := net.Listen("tcp", cfg.Server.HTTP.Bind)
+	if err != nil {
+		return fmt.Errorf("bind HTTP server %q: %w", cfg.Server.HTTP.Bind, err)
+	}
+	defer listener.Close()
+
+	server := &http.Server{Handler: handler}
+	shutdownDone := make(chan struct{})
+	go func() {
+		defer close(shutdownDone)
+		<-ctx.Done()
+		_ = server.Shutdown(context.Background())
+	}()
+
+	err = server.Serve(listener)
+	if errors.Is(err, http.ErrServerClosed) {
+		<-shutdownDone
+		return nil
+	}
+	return err
+}

internal/app/source_select.go

@@ -0,0 +1,134 @@
+package app
+
+import (
+	"context"
+	"errors"
+	"fmt"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+)
+
+type PipelineNotFoundError struct {
+	ID string
+}
+
+func (e PipelineNotFoundError) Error() string {
+	return fmt.Sprintf("pipeline %q not found", e.ID)
+}
+
+func IsPipelineNotFound(err error) bool {
+	var notFound PipelineNotFoundError
+	return errors.As(err, &notFound)
+}
+
+type sourceCommandOptions struct {
+	CommandName string
+	Path        string
+	ConfigPath  string
+	PipelineID  string
+	BundlePath  string
+}
+
+type sourceSelection struct {
+	Bundles       []bundle.Bundle
+	PipelineID    string
+	SourceBackend string
+	ConfigMode    bool
+	Warnings      []OutputWarning
+}
+
+func selectSourceBundles(ctx context.Context, options sourceCommandOptions, provider backendFactoryProvider) (sourceSelection, error) {
+	if err := ctx.Err(); err != nil {
+		return sourceSelection{}, err
+	}
+	if options.ConfigPath != "" {
+		cfg, err := config.LoadFile(options.ConfigPath)
+		if err != nil {
+			return sourceSelection{}, err
+		}
+		return selectSourceBundlesFromConfig(ctx, cfg, options, provider)
+	}
+	if options.PipelineID != "" {
+		return sourceSelection{}, fmt.Errorf("configured source mode requires --config")
+	}
+	if options.BundlePath != "" {
+		return sourceSelection{}, fmt.Errorf("configured source mode requires --config")
+	}
+	if options.Path == "" {
+		return sourceSelection{}, fmt.Errorf("%s command requires a path", options.CommandName)
+	}
+	backend, err := newBackendFactory().openLocalPath(ctx, options.Path)
+	if err != nil {
+		return sourceSelection{}, err
+	}
+	defer closeBackend(backend)
+	bundles, err := bundle.Discover(ctx, backend, "")
+	if err != nil {
+		return sourceSelection{}, err
+	}
+	return sourceSelection{Bundles: bundles}, nil
+}
+
+func selectSourceBundlesFromConfig(ctx context.Context, cfg config.Config, options sourceCommandOptions, provider backendFactoryProvider) (sourceSelection, error) {
+	if options.Path != "" {
+		return sourceSelection{}, fmt.Errorf("configured source mode does not accept a local path")
+	}
+	if options.PipelineID == "" {
+		return sourceSelection{}, fmt.Errorf("configured source mode requires --pipeline")
+	}
+	secretLoad, err := config.LoadSecretEnvironment(cfg.Secrets.Directory, nil)
+	if err != nil {
+		return sourceSelection{}, err
+	}
+	pipeline, ok := findPipeline(cfg, options.PipelineID)
+	if !ok {
+		return sourceSelection{}, PipelineNotFoundError{ID: options.PipelineID}
+	}
+	backends := provider(secretLoad.Environment)
+	sourceBackend, err := backends.openSource(ctx, pipeline.Source)
+	if err != nil {
+		return sourceSelection{}, fmt.Errorf("pipeline %s source backend %s: %w", pipeline.ID, pipeline.Source.Backend, err)
+	}
+	defer closeBackend(sourceBackend)
+
+	var bundles []bundle.Bundle
+	if options.BundlePath != "" {
+		sourceBundle, err := bundle.Validate(ctx, sourceBackend, options.BundlePath)
+		if err != nil {
+			return sourceSelection{}, fmt.Errorf("pipeline %s source backend %s bundle %s: %w", pipeline.ID, pipeline.Source.Backend, storage.DisplayPath(options.BundlePath), err)
+		}
+		bundles = []bundle.Bundle{sourceBundle}
+	} else {
+		bundles, err = bundle.Discover(ctx, sourceBackend, "")
+		if err != nil {
+			return sourceSelection{}, fmt.Errorf("pipeline %s source backend %s discover source bundles: %w", pipeline.ID, pipeline.Source.Backend, err)
+		}
+	}
+	return sourceSelection{
+		Bundles:       bundles,
+		PipelineID:    pipeline.ID,
+		SourceBackend: pipeline.Source.Backend,
+		ConfigMode:    true,
+		Warnings:      append(secretConflictWarnings(secretLoad.Conflicts), sourceSSHWarnings(pipeline)...),
+	}, nil
+}
+
+func findPipeline(cfg config.Config, id string) (config.Pipeline, bool) {
+	for _, pipeline := range cfg.Pipelines {
+		if pipeline.ID == id {
+			return pipeline, true
+		}
+	}
+	return config.Pipeline{}, false
+}
+
+func sourceSSHWarnings(pipeline config.Pipeline) []OutputWarning {
+	if pipeline.Source.Backend != config.BackendSSH || pipeline.Source.SSH.HostKeyPolicy != config.HostKeyPolicyOff {
+		return nil
+	}
+	return []OutputWarning{{
+		Message: fmt.Sprintf("pipeline=%s source host_key_policy=off disables SSH host key checking", pipeline.ID),
+	}}
+}

internal/app/upload_coordinator.go

@@ -0,0 +1,389 @@
+package app
+
+import (
+	"context"
+	"crypto/rand"
+	"encoding/hex"
+	"errors"
+	"fmt"
+	"io"
+	"os"
+	"sync"
+	"time"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/ingest"
+)
+
+const DefaultUploadMaxFileCount = 4096
+
+type UploadRunID string
+
+type UploadStatus string
+
+const (
+	UploadStatusAccepted  UploadStatus = "accepted"
+	UploadStatusQueued    UploadStatus = "queued"
+	UploadStatusRunning   UploadStatus = "running"
+	UploadStatusSucceeded UploadStatus = "succeeded"
+	UploadStatusFailed    UploadStatus = "failed"
+	UploadStatusExpired   UploadStatus = "expired"
+)
+
+type UploadRunRecord struct {
+	ID         UploadRunID  `json:"run_id"`
+	PipelineID string       `json:"pipeline_id"`
+	Status     UploadStatus `json:"status"`
+	AcceptedAt time.Time    `json:"accepted_at"`
+	StartedAt  *time.Time   `json:"started_at,omitempty"`
+	FinishedAt *time.Time   `json:"finished_at,omitempty"`
+	Report     *RunReport   `json:"report,omitempty"`
+	Error      string       `json:"error,omitempty"`
+	StagedRoot string       `json:"-"`
+}
+
+type UploadRequest struct {
+	PipelineID   string
+	ContentType  string
+	Body         io.Reader
+	DryRun       bool
+	Force        bool
+	MaxFileCount int
+}
+
+type UploadQueueFullError struct {
+	QueueSize int
+}
+
+func (err UploadQueueFullError) Error() string {
+	return fmt.Sprintf("upload queue is full with capacity %d", err.QueueSize)
+}
+
+func IsUploadQueueFull(err error) bool {
+	var full UploadQueueFullError
+	return errors.As(err, &full)
+}
+
+type UploadCoordinator struct {
+	ctx          context.Context
+	cfg          config.Config
+	stage        uploadStageFunc
+	run          uploadRunFunc
+	now          func() time.Time
+	randomSuffix func() (string, error)
+	retention    time.Duration
+
+	mu             sync.Mutex
+	signal         chan struct{}
+	queueSize      int
+	maxConcurrency int
+	runningCount   int
+	activePipeline map[string]bool
+	pending        []*uploadJob
+	records        map[UploadRunID]UploadRunRecord
+}
+
+type uploadStageFunc func(context.Context, ingest.StageOptions) (ingest.StagedBundle, error)
+
+type uploadRunFunc func(context.Context, config.Config, RunPipelineWithLocalSourceOptions) (RunReport, error)
+
+type uploadJob struct {
+	recordID UploadRunID
+	request  UploadRequest
+	pipeline config.Pipeline
+}
+
+type uploadCoordinatorHooks struct {
+	stage        uploadStageFunc
+	run          uploadRunFunc
+	now          func() time.Time
+	randomSuffix func() (string, error)
+}
+
+func NewUploadCoordinator(ctx context.Context, cfg config.Config) *UploadCoordinator {
+	return newUploadCoordinator(ctx, cfg, uploadCoordinatorHooks{})
+}
+
+func newUploadCoordinator(ctx context.Context, cfg config.Config, hooks uploadCoordinatorHooks) *UploadCoordinator {
+	if ctx == nil {
+		ctx = context.Background()
+	}
+	config.ApplyDefaults(&cfg)
+	stage := hooks.stage
+	if stage == nil {
+		stage = ingest.StageArchive
+	}
+	run := hooks.run
+	if run == nil {
+		run = runPipelineConfigWithLocalSource
+	}
+	now := hooks.now
+	if now == nil {
+		now = time.Now
+	}
+	randomSuffix := hooks.randomSuffix
+	if randomSuffix == nil {
+		randomSuffix = randomRunIDSuffix
+	}
+	coordinator := &UploadCoordinator{
+		ctx:            ctx,
+		cfg:            cfg,
+		stage:          stage,
+		run:            run,
+		now:            now,
+		randomSuffix:   randomSuffix,
+		retention:      cfg.Server.HTTP.Retention.AsDuration(),
+		signal:         make(chan struct{}, 1),
+		queueSize:      cfg.Server.HTTP.QueueSize,
+		maxConcurrency: cfg.Server.HTTP.MaxConcurrency,
+		activePipeline: map[string]bool{},
+		records:        map[UploadRunID]UploadRunRecord{},
+	}
+	go coordinator.dispatchLoop()
+	return coordinator
+}
+
+func (coordinator *UploadCoordinator) Submit(ctx context.Context, request UploadRequest) (UploadRunRecord, error) {
+	if ctx == nil {
+		ctx = context.Background()
+	}
+	if err := ctx.Err(); err != nil {
+		return UploadRunRecord{}, err
+	}
+	if request.Body == nil {
+		return UploadRunRecord{}, fmt.Errorf("upload body is required")
+	}
+	pipeline, ok := findPipeline(coordinator.cfg, request.PipelineID)
+	if !ok {
+		return UploadRunRecord{}, PipelineNotFoundError{ID: request.PipelineID}
+	}
+	if pipeline.Source.Backend != config.BackendHTTPUpload {
+		return UploadRunRecord{}, fmt.Errorf("pipeline %s source backend %s is not configured for uploads", pipeline.ID, pipeline.Source.Backend)
+	}
+	runID, err := coordinator.newRunID(pipeline.ID)
+	if err != nil {
+		return UploadRunRecord{}, err
+	}
+
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	coordinator.expireLocked(coordinator.now().UTC())
+	if len(coordinator.pending) >= coordinator.queueSize {
+		return UploadRunRecord{}, UploadQueueFullError{QueueSize: coordinator.queueSize}
+	}
+	record := UploadRunRecord{
+		ID:         runID,
+		PipelineID: pipeline.ID,
+		Status:     UploadStatusAccepted,
+		AcceptedAt: coordinator.now().UTC(),
+	}
+	coordinator.records[runID] = record
+	coordinator.pending = append(coordinator.pending, &uploadJob{
+		recordID: runID,
+		request:  request,
+		pipeline: pipeline,
+	})
+	coordinator.notify()
+	return record, nil
+}
+
+func (coordinator *UploadCoordinator) Status(runID UploadRunID) (UploadRunRecord, bool) {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	coordinator.expireLocked(coordinator.now().UTC())
+	record, ok := coordinator.records[runID]
+	return record, ok
+}
+
+func (coordinator *UploadCoordinator) Expire() []UploadRunRecord {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	return coordinator.expireLocked(coordinator.now().UTC())
+}
+
+func (coordinator *UploadCoordinator) CanAccept() bool {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	coordinator.expireLocked(coordinator.now().UTC())
+	return len(coordinator.pending) < coordinator.queueSize
+}
+
+func (coordinator *UploadCoordinator) QueueDepth() int {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	return len(coordinator.pending)
+}
+
+func (coordinator *UploadCoordinator) RunningCount() int {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	return coordinator.runningCount
+}
+
+func (coordinator *UploadCoordinator) newRunID(pipelineID string) (UploadRunID, error) {
+	suffix, err := coordinator.randomSuffix()
+	if err != nil {
+		return "", err
+	}
+	timestamp := coordinator.now().UTC().Format("20060102T150405Z")
+	return UploadRunID(pipelineID + "." + timestamp + "." + suffix), nil
+}
+
+func (coordinator *UploadCoordinator) dispatchLoop() {
+	for {
+		select {
+		case <-coordinator.ctx.Done():
+			return
+		case <-coordinator.signal:
+			for coordinator.startNext() {
+			}
+		}
+	}
+}
+
+func (coordinator *UploadCoordinator) startNext() bool {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	if coordinator.runningCount >= coordinator.maxConcurrency {
+		coordinator.markPendingQueuedLocked()
+		return false
+	}
+	index := -1
+	for candidateIndex, job := range coordinator.pending {
+		if coordinator.activePipeline[job.pipeline.ID] {
+			record := coordinator.records[job.recordID]
+			if record.Status == UploadStatusAccepted {
+				record.Status = UploadStatusQueued
+				coordinator.records[job.recordID] = record
+			}
+			continue
+		}
+		index = candidateIndex
+		break
+	}
+	if index < 0 {
+		return false
+	}
+
+	job := coordinator.pending[index]
+	coordinator.pending = append(coordinator.pending[:index], coordinator.pending[index+1:]...)
+	now := coordinator.now().UTC()
+	record := coordinator.records[job.recordID]
+	record.Status = UploadStatusRunning
+	record.StartedAt = &now
+	coordinator.records[job.recordID] = record
+	coordinator.runningCount++
+	coordinator.activePipeline[job.pipeline.ID] = true
+
+	go coordinator.runJob(job)
+	return true
+}
+
+func (coordinator *UploadCoordinator) markPendingQueuedLocked() {
+	for _, job := range coordinator.pending {
+		record := coordinator.records[job.recordID]
+		if record.Status == UploadStatusAccepted {
+			record.Status = UploadStatusQueued
+			coordinator.records[job.recordID] = record
+		}
+	}
+}
+
+func (coordinator *UploadCoordinator) runJob(job *uploadJob) {
+	record := coordinator.currentRecord(job.recordID)
+	maxFileCount := job.request.MaxFileCount
+	if maxFileCount <= 0 {
+		maxFileCount = DefaultUploadMaxFileCount
+	}
+	staged, err := coordinator.stage(coordinator.ctx, ingest.StageOptions{
+		Body:                job.request.Body,
+		ContentType:         job.request.ContentType,
+		PipelineStagingPath: job.pipeline.Source.Upload.StagingPath,
+		RunID:               string(record.ID),
+		MaxUploadSize:       int64(*job.pipeline.Source.Upload.MaxUploadSize),
+		MaxExtractedSize:    int64(*job.pipeline.Source.Upload.MaxUploadSize),
+		MaxFileCount:        maxFileCount,
+	})
+	if err == nil {
+		coordinator.setStagedRoot(job.recordID, staged.Root)
+		var report RunReport
+		report, err = coordinator.run(coordinator.ctx, coordinator.cfg, RunPipelineWithLocalSourceOptions{
+			PipelineID: job.pipeline.ID,
+			SourceRoot: staged.Root,
+			DryRun:     job.request.DryRun,
+			Force:      job.request.Force,
+		})
+		coordinator.complete(job, &report, err)
+		return
+	}
+	coordinator.complete(job, nil, err)
+}
+
+func (coordinator *UploadCoordinator) currentRecord(runID UploadRunID) UploadRunRecord {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	return coordinator.records[runID]
+}
+
+func (coordinator *UploadCoordinator) setStagedRoot(runID UploadRunID, root string) {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	record := coordinator.records[runID]
+	record.StagedRoot = root
+	coordinator.records[runID] = record
+}
+
+func (coordinator *UploadCoordinator) complete(job *uploadJob, report *RunReport, runErr error) {
+	coordinator.mu.Lock()
+	defer coordinator.mu.Unlock()
+	record := coordinator.records[job.recordID]
+	finishedAt := coordinator.now().UTC()
+	record.FinishedAt = &finishedAt
+	record.Report = report
+	if runErr != nil {
+		record.Status = UploadStatusFailed
+		record.Error = runErr.Error()
+	} else {
+		record.Status = UploadStatusSucceeded
+	}
+	coordinator.records[job.recordID] = record
+	coordinator.runningCount--
+	delete(coordinator.activePipeline, job.pipeline.ID)
+	coordinator.notify()
+}
+
+func (coordinator *UploadCoordinator) expireLocked(now time.Time) []UploadRunRecord {
+	var expired []UploadRunRecord
+	for runID, record := range coordinator.records {
+		if record.FinishedAt == nil || record.Status == UploadStatusExpired {
+			continue
+		}
+		if now.Before(record.FinishedAt.Add(coordinator.retention)) {
+			continue
+		}
+		if record.StagedRoot != "" {
+			_ = os.RemoveAll(record.StagedRoot)
+		}
+		record.Status = UploadStatusExpired
+		record.Report = nil
+		record.Error = ""
+		expired = append(expired, record)
+		delete(coordinator.records, runID)
+	}
+	return expired
+}
+
+func (coordinator *UploadCoordinator) notify() {
+	select {
+	case coordinator.signal <- struct{}{}:
+	default:
+	}
+}
+
+func randomRunIDSuffix() (string, error) {
+	var data [4]byte
+	if _, err := rand.Read(data[:]); err != nil {
+		return "", fmt.Errorf("generate run id suffix: %w", err)
+	}
+	return hex.EncodeToString(data[:]), nil
+}

internal/app/upload_coordinator_test.go

@@ -0,0 +1,373 @@
+package app
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"os"
+	"path/filepath"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/ingest"
+)
+
+func TestUploadCoordinatorGeneratesRunIDAndAcceptedStatus(t *testing.T) {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	clock := newUploadTestClock(time.Date(2026, 6, 3, 12, 0, 0, 0, time.UTC))
+	coordinator := newUploadCoordinator(ctx, uploadCoordinatorConfig(t, uploadCoordinatorConfigOptions{
+		pipelineIDs: []string{"weather-daily"},
+	}), uploadCoordinatorHooks{
+		now:          clock.Now,
+		randomSuffix: uploadTestSuffixes("ab12cd34"),
+		stage:        successfulUploadStage,
+		run:          successfulUploadRun,
+	})
+
+	record, err := coordinator.Submit(context.Background(), UploadRequest{
+		PipelineID:  "weather-daily",
+		ContentType: ingest.ContentTypeTar,
+		Body:        strings.NewReader("archive"),
+	})
+	if err != nil {
+		t.Fatalf("Submit() error = %v", err)
+	}
+	if got, want := record.ID, UploadRunID("weather-daily.20260603T120000Z.ab12cd34"); got != want {
+		t.Fatalf("run id = %q, want %q", got, want)
+	}
+	if got, want := record.Status, UploadStatusAccepted; got != want {
+		t.Fatalf("initial status = %q, want %q", got, want)
+	}
+	waitForUploadStatus(t, coordinator, record.ID, UploadStatusSucceeded)
+}
+
+func TestUploadCoordinatorRejectsFullQueueBeforeReadingBody(t *testing.T) {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	release := make(chan struct{})
+	var reads atomic.Int64
+	coordinator := newUploadCoordinator(ctx, uploadCoordinatorConfig(t, uploadCoordinatorConfigOptions{
+		pipelineIDs:    []string{"reports"},
+		queueSize:      1,
+		maxConcurrency: 1,
+	}), uploadCoordinatorHooks{
+		randomSuffix: uploadTestSuffixes("00000001", "00000002", "00000003"),
+		stage:        successfulUploadStage,
+		run: func(ctx context.Context, cfg config.Config, options RunPipelineWithLocalSourceOptions) (RunReport, error) {
+			<-release
+			return RunReport{}, nil
+		},
+	})
+
+	first, err := coordinator.Submit(context.Background(), UploadRequest{
+		PipelineID:  "reports",
+		ContentType: ingest.ContentTypeTar,
+		Body:        strings.NewReader("first"),
+	})
+	if err != nil {
+		t.Fatalf("first Submit() error = %v", err)
+	}
+	waitForUploadStatus(t, coordinator, first.ID, UploadStatusRunning)
+
+	second, err := coordinator.Submit(context.Background(), UploadRequest{
+		PipelineID:  "reports",
+		ContentType: ingest.ContentTypeTar,
+		Body:        strings.NewReader("second"),
+	})
+	if err != nil {
+		t.Fatalf("second Submit() error = %v", err)
+	}
+	waitForUploadStatus(t, coordinator, second.ID, UploadStatusQueued)
+
+	_, err = coordinator.Submit(context.Background(), UploadRequest{
+		PipelineID:  "reports",
+		ContentType: ingest.ContentTypeTar,
+		Body: readerFunc(func(data []byte) (int, error) {
+			reads.Add(1)
+			return 0, io.EOF
+		}),
+	})
+	if err == nil || !IsUploadQueueFull(err) {
+		t.Fatalf("third Submit() error = %v, want full queue", err)
+	}
+	if got := reads.Load(); got != 0 {
+		t.Fatalf("rejected body reads = %d, want 0", got)
+	}
+	close(release)
+	waitForUploadStatus(t, coordinator, first.ID, UploadStatusSucceeded)
+	waitForUploadStatus(t, coordinator, second.ID, UploadStatusSucceeded)
+}
+
+func TestUploadCoordinatorSerializesSamePipelineUploads(t *testing.T) {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	release := make(chan struct{})
+	coordinator := newUploadCoordinator(ctx, uploadCoordinatorConfig(t, uploadCoordinatorConfigOptions{
+		pipelineIDs:    []string{"reports"},
+		queueSize:      4,
+		maxConcurrency: 2,
+	}), uploadCoordinatorHooks{
+		randomSuffix: uploadTestSuffixes("00000001", "00000002"),
+		stage:        successfulUploadStage,
+		run: func(ctx context.Context, cfg config.Config, options RunPipelineWithLocalSourceOptions) (RunReport, error) {
+			<-release
+			return RunReport{}, nil
+		},
+	})
+
+	first, err := coordinator.Submit(context.Background(), UploadRequest{PipelineID: "reports", ContentType: ingest.ContentTypeTar, Body: strings.NewReader("first")})
+	if err != nil {
+		t.Fatalf("first Submit() error = %v", err)
+	}
+	second, err := coordinator.Submit(context.Background(), UploadRequest{PipelineID: "reports", ContentType: ingest.ContentTypeTar, Body: strings.NewReader("second")})
+	if err != nil {
+		t.Fatalf("second Submit() error = %v", err)
+	}
+	waitForUploadStatus(t, coordinator, first.ID, UploadStatusRunning)
+	waitForUploadStatus(t, coordinator, second.ID, UploadStatusQueued)
+	if got := coordinator.RunningCount(); got != 1 {
+		t.Fatalf("running count = %d, want 1", got)
+	}
+	close(release)
+	waitForUploadStatus(t, coordinator, first.ID, UploadStatusSucceeded)
+	waitForUploadStatus(t, coordinator, second.ID, UploadStatusSucceeded)
+}
+
+func TestUploadCoordinatorRunsDifferentPipelinesConcurrentlyUpToLimit(t *testing.T) {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	release := make(chan struct{})
+	coordinator := newUploadCoordinator(ctx, uploadCoordinatorConfig(t, uploadCoordinatorConfigOptions{
+		pipelineIDs:    []string{"reports-one", "reports-two"},
+		queueSize:      4,
+		maxConcurrency: 2,
+	}), uploadCoordinatorHooks{
+		randomSuffix: uploadTestSuffixes("00000001", "00000002"),
+		stage:        successfulUploadStage,
+		run: func(ctx context.Context, cfg config.Config, options RunPipelineWithLocalSourceOptions) (RunReport, error) {
+			<-release
+			return RunReport{}, nil
+		},
+	})
+
+	first, err := coordinator.Submit(context.Background(), UploadRequest{PipelineID: "reports-one", ContentType: ingest.ContentTypeTar, Body: strings.NewReader("first")})
+	if err != nil {
+		t.Fatalf("first Submit() error = %v", err)
+	}
+	second, err := coordinator.Submit(context.Background(), UploadRequest{PipelineID: "reports-two", ContentType: ingest.ContentTypeTar, Body: strings.NewReader("second")})
+	if err != nil {
+		t.Fatalf("second Submit() error = %v", err)
+	}
+	waitForUploadStatus(t, coordinator, first.ID, UploadStatusRunning)
+	waitForUploadStatus(t, coordinator, second.ID, UploadStatusRunning)
+	if got := coordinator.RunningCount(); got != 2 {
+		t.Fatalf("running count = %d, want 2", got)
+	}
+	close(release)
+	waitForUploadStatus(t, coordinator, first.ID, UploadStatusSucceeded)
+	waitForUploadStatus(t, coordinator, second.ID, UploadStatusSucceeded)
+}
+
+func TestUploadCoordinatorRecordsFailureDetails(t *testing.T) {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	runErr := errors.New("publish failed")
+	coordinator := newUploadCoordinator(ctx, uploadCoordinatorConfig(t, uploadCoordinatorConfigOptions{
+		pipelineIDs: []string{"reports"},
+	}), uploadCoordinatorHooks{
+		randomSuffix: uploadTestSuffixes("00000001"),
+		stage:        successfulUploadStage,
+		run: func(ctx context.Context, cfg config.Config, options RunPipelineWithLocalSourceOptions) (RunReport, error) {
+			return RunReport{DryRun: options.DryRun}, runErr
+		},
+	})
+
+	record, err := coordinator.Submit(context.Background(), UploadRequest{
+		PipelineID:  "reports",
+		ContentType: ingest.ContentTypeTar,
+		Body:        strings.NewReader("archive"),
+		DryRun:      true,
+	})
+	if err != nil {
+		t.Fatalf("Submit() error = %v", err)
+	}
+	failed := waitForUploadStatus(t, coordinator, record.ID, UploadStatusFailed)
+	if failed.Error != runErr.Error() {
+		t.Fatalf("error = %q, want %q", failed.Error, runErr.Error())
+	}
+	if failed.Report == nil || !failed.Report.DryRun {
+		t.Fatalf("report = %#v, want retained dry-run report", failed.Report)
+	}
+}
+
+func TestUploadCoordinatorExpiresCompletedRecordsAndStagingDirectories(t *testing.T) {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	clock := newUploadTestClock(time.Date(2026, 6, 3, 12, 0, 0, 0, time.UTC))
+	coordinator := newUploadCoordinator(ctx, uploadCoordinatorConfig(t, uploadCoordinatorConfigOptions{
+		pipelineIDs: []string{"reports"},
+		retention:   time.Second,
+	}), uploadCoordinatorHooks{
+		now:          clock.Now,
+		randomSuffix: uploadTestSuffixes("00000001"),
+		stage:        successfulUploadStage,
+		run: func(ctx context.Context, cfg config.Config, options RunPipelineWithLocalSourceOptions) (RunReport, error) {
+			return RunReport{DryRun: true}, nil
+		},
+	})
+
+	record, err := coordinator.Submit(context.Background(), UploadRequest{
+		PipelineID:  "reports",
+		ContentType: ingest.ContentTypeTar,
+		Body:        strings.NewReader("archive"),
+	})
+	if err != nil {
+		t.Fatalf("Submit() error = %v", err)
+	}
+	succeeded := waitForUploadStatus(t, coordinator, record.ID, UploadStatusSucceeded)
+	if succeeded.Report == nil || !succeeded.Report.DryRun {
+		t.Fatalf("report = %#v, want retained dry-run report", succeeded.Report)
+	}
+	if _, err := os.Stat(succeeded.StagedRoot); err != nil {
+		t.Fatalf("staged root stat before expiry = %v", err)
+	}
+
+	clock.Advance(2 * time.Second)
+	expired := coordinator.Expire()
+	if got, want := len(expired), 1; got != want {
+		t.Fatalf("expired count = %d, want %d", got, want)
+	}
+	if expired[0].Status != UploadStatusExpired || expired[0].Report != nil || expired[0].Error != "" {
+		t.Fatalf("expired record = %#v, want expired without report/error", expired[0])
+	}
+	if _, ok := coordinator.Status(record.ID); ok {
+		t.Fatal("Status() ok = true after expiry, want removed status")
+	}
+	if _, err := os.Stat(succeeded.StagedRoot); !os.IsNotExist(err) {
+		t.Fatalf("staged root stat after expiry = %v, want not exist", err)
+	}
+}
+
+type readerFunc func([]byte) (int, error)
+
+func (fn readerFunc) Read(data []byte) (int, error) {
+	return fn(data)
+}
+
+func successfulUploadStage(ctx context.Context, opts ingest.StageOptions) (ingest.StagedBundle, error) {
+	root := filepath.Join(opts.PipelineStagingPath, opts.RunID)
+	if err := os.MkdirAll(root, 0o755); err != nil {
+		return ingest.StagedBundle{}, err
+	}
+	return ingest.StagedBundle{Root: root}, nil
+}
+
+func successfulUploadRun(ctx context.Context, cfg config.Config, options RunPipelineWithLocalSourceOptions) (RunReport, error) {
+	return RunReport{}, nil
+}
+
+type uploadCoordinatorConfigOptions struct {
+	pipelineIDs    []string
+	queueSize      int
+	maxConcurrency int
+	retention      time.Duration
+}
+
+func uploadCoordinatorConfig(t *testing.T, opts uploadCoordinatorConfigOptions) config.Config {
+	t.Helper()
+	queueSize := opts.queueSize
+	if queueSize == 0 {
+		queueSize = 4
+	}
+	maxConcurrency := opts.maxConcurrency
+	if maxConcurrency == 0 {
+		maxConcurrency = 1
+	}
+	retentionValue := opts.retention
+	if retentionValue == 0 {
+		retentionValue = time.Minute
+	}
+	retention := config.Duration(retentionValue)
+	maxUploadSize := config.ByteSize(1024)
+	cfg := config.Config{
+		Server: config.Server{HTTP: config.HTTPServer{
+			StagingRoot:    t.TempDir(),
+			MaxUploadSize:  &maxUploadSize,
+			QueueSize:      queueSize,
+			MaxConcurrency: maxConcurrency,
+			Retention:      &retention,
+		}},
+	}
+	for _, pipelineID := range opts.pipelineIDs {
+		cfg.Pipelines = append(cfg.Pipelines, config.Pipeline{
+			ID: pipelineID,
+			Source: config.Backend{
+				Backend: config.BackendHTTPUpload,
+				Upload:  config.HTTPUpload{TokenEnv: strings.ToUpper(strings.ReplaceAll(pipelineID, "-", "_")) + "_TOKEN"},
+			},
+			Destinations: []config.Destination{{
+				ID:      "archive",
+				Backend: config.BackendLocal,
+				Path:    t.TempDir(),
+			}},
+		})
+	}
+	return cfg
+}
+
+type uploadTestClock struct {
+	mu  sync.Mutex
+	now time.Time
+}
+
+func newUploadTestClock(now time.Time) *uploadTestClock {
+	return &uploadTestClock{now: now}
+}
+
+func (clock *uploadTestClock) Now() time.Time {
+	clock.mu.Lock()
+	defer clock.mu.Unlock()
+	return clock.now
+}
+
+func (clock *uploadTestClock) Advance(duration time.Duration) {
+	clock.mu.Lock()
+	defer clock.mu.Unlock()
+	clock.now = clock.now.Add(duration)
+}
+
+func uploadTestSuffixes(values ...string) func() (string, error) {
+	var mu sync.Mutex
+	index := 0
+	return func() (string, error) {
+		mu.Lock()
+		defer mu.Unlock()
+		if index >= len(values) {
+			return fmt.Sprintf("%08d", index+1), nil
+		}
+		value := values[index]
+		index++
+		return value, nil
+	}
+}
+
+func waitForUploadStatus(t *testing.T, coordinator *UploadCoordinator, runID UploadRunID, status UploadStatus) UploadRunRecord {
+	t.Helper()
+	deadline := time.Now().Add(time.Second)
+	for time.Now().Before(deadline) {
+		record, ok := coordinator.Status(runID)
+		if ok && record.Status == status {
+			return record
+		}
+		time.Sleep(time.Millisecond)
+	}
+	record, ok := coordinator.Status(runID)
+	t.Fatalf("timed out waiting for status %s; latest ok=%t record=%#v", status, ok, record)
+	return UploadRunRecord{}
+}

internal/app/upload_http.go

@@ -0,0 +1,209 @@
+package app
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"mime"
+	"net/http"
+	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/ingest"
+)
+
+type uploadCoordinator interface {
+	CanAccept() bool
+	Submit(context.Context, UploadRequest) (UploadRunRecord, error)
+	Status(UploadRunID) (UploadRunRecord, bool)
+}
+
+type uploadHTTPHandler struct {
+	coordinator uploadCoordinator
+	tokens      map[string]string
+	limits      map[string]int64
+}
+
+type uploadAcceptedResponse struct {
+	RunID  UploadRunID  `json:"run_id"`
+	Status UploadStatus `json:"status"`
+}
+
+type httpErrorResponse struct {
+	Error string `json:"error"`
+}
+
+func newUploadHTTPHandler(ctx context.Context, cfg config.Config, environment config.Environment) (http.Handler, error) {
+	config.ApplyDefaults(&cfg)
+	tokens, limits, err := resolveUploadTokens(cfg, environment)
+	if err != nil {
+		return nil, err
+	}
+	return uploadHTTPHandler{
+		coordinator: NewUploadCoordinator(ctx, cfg),
+		tokens:      tokens,
+		limits:      limits,
+	}, nil
+}
+
+func resolveUploadTokens(cfg config.Config, environment config.Environment) (map[string]string, map[string]int64, error) {
+	tokens := make(map[string]string)
+	limits := make(map[string]int64)
+	for _, pipeline := range cfg.Pipelines {
+		if pipeline.Source.Backend != config.BackendHTTPUpload {
+			continue
+		}
+		tokenName := pipeline.Source.Upload.TokenEnv
+		token, ok := environment.Lookup(tokenName)
+		if !ok {
+			return nil, nil, fmt.Errorf("upload token environment variable %s is not set", tokenName)
+		}
+		if token == "" {
+			return nil, nil, fmt.Errorf("upload token environment variable %s is empty", tokenName)
+		}
+		if existing, exists := tokens[token]; exists {
+			return nil, nil, fmt.Errorf("upload token environment variables for pipelines %s and %s resolve to the same value", existing, pipeline.ID)
+		}
+		tokens[token] = pipeline.ID
+		limits[pipeline.ID] = int64(*pipeline.Source.Upload.MaxUploadSize)
+	}
+	return tokens, limits, nil
+}
+
+func (handler uploadHTTPHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+	switch {
+	case r.Method == http.MethodGet && r.URL.Path == "/healthz":
+		handler.handleHealth(w)
+	case r.Method == http.MethodPost && r.URL.Path == "/upload":
+		handler.handleUpload(w, r)
+	case r.Method == http.MethodGet && strings.HasPrefix(r.URL.Path, "/runs/"):
+		handler.handleRunStatus(w, r)
+	default:
+		writeHTTPError(w, http.StatusNotFound, "not found")
+	}
+}
+
+func (handler uploadHTTPHandler) handleHealth(w http.ResponseWriter) {
+	writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
+}
+
+func (handler uploadHTTPHandler) handleUpload(w http.ResponseWriter, r *http.Request) {
+	if r.URL.Query().Has("pipeline") || r.URL.Query().Has("pipeline_id") {
+		writeHTTPError(w, http.StatusBadRequest, "pipeline id is not accepted")
+		return
+	}
+	pipelineID, ok := handler.authenticate(r.Header.Get("Authorization"))
+	if !ok {
+		writeHTTPError(w, http.StatusUnauthorized, "unauthorized")
+		return
+	}
+	contentType := r.Header.Get("Content-Type")
+	if !supportedUploadContentType(contentType) {
+		writeHTTPError(w, http.StatusUnsupportedMediaType, "unsupported content type")
+		return
+	}
+	if !handler.coordinator.CanAccept() {
+		writeHTTPError(w, http.StatusServiceUnavailable, "upload queue is full")
+		return
+	}
+	body, err := readUploadBody(r.Body, handler.limits[pipelineID])
+	if err != nil {
+		if errors.Is(err, ingest.ErrUploadTooLarge) {
+			writeHTTPError(w, http.StatusRequestEntityTooLarge, "upload exceeds maximum size")
+			return
+		}
+		writeHTTPError(w, http.StatusBadRequest, "read upload body failed")
+		return
+	}
+	record, err := handler.coordinator.Submit(r.Context(), UploadRequest{
+		PipelineID:  pipelineID,
+		ContentType: contentType,
+		Body:        bytes.NewReader(body),
+	})
+	if err != nil {
+		writeUploadSubmitError(w, err)
+		return
+	}
+	writeJSON(w, http.StatusAccepted, uploadAcceptedResponse{
+		RunID:  record.ID,
+		Status: UploadStatusAccepted,
+	})
+}
+
+func (handler uploadHTTPHandler) handleRunStatus(w http.ResponseWriter, r *http.Request) {
+	rawRunID := strings.TrimPrefix(r.URL.Path, "/runs/")
+	if rawRunID == "" || strings.Contains(rawRunID, "/") {
+		writeHTTPError(w, http.StatusNotFound, "not found")
+		return
+	}
+	record, ok := handler.coordinator.Status(UploadRunID(rawRunID))
+	if !ok {
+		writeHTTPError(w, http.StatusNotFound, "run not found")
+		return
+	}
+	writeJSON(w, http.StatusOK, record)
+}
+
+func (handler uploadHTTPHandler) authenticate(header string) (string, bool) {
+	const prefix = "Bearer "
+	if !strings.HasPrefix(header, prefix) {
+		return "", false
+	}
+	token := strings.TrimSpace(strings.TrimPrefix(header, prefix))
+	if token == "" {
+		return "", false
+	}
+	pipelineID, ok := handler.tokens[token]
+	return pipelineID, ok
+}
+
+func supportedUploadContentType(contentType string) bool {
+	mediaType, _, err := mime.ParseMediaType(contentType)
+	if err != nil {
+		mediaType = contentType
+	}
+	switch mediaType {
+	case ingest.ContentTypeTar, ingest.ContentTypeGzip, ingest.ContentTypeXGzip:
+		return true
+	default:
+		return false
+	}
+}
+
+func readUploadBody(body io.Reader, maxSize int64) ([]byte, error) {
+	limited := &io.LimitedReader{R: body, N: maxSize + 1}
+	data, err := io.ReadAll(limited)
+	if err != nil {
+		return nil, err
+	}
+	if int64(len(data)) > maxSize {
+		return nil, ingest.ErrUploadTooLarge
+	}
+	return data, nil
+}
+
+func writeUploadSubmitError(w http.ResponseWriter, err error) {
+	switch {
+	case IsUploadQueueFull(err):
+		writeHTTPError(w, http.StatusServiceUnavailable, "upload queue is full")
+	case errors.Is(err, ingest.ErrUploadTooLarge):
+		writeHTTPError(w, http.StatusRequestEntityTooLarge, "upload exceeds maximum size")
+	case errors.Is(err, ingest.ErrUnsupportedContentType):
+		writeHTTPError(w, http.StatusUnsupportedMediaType, "unsupported content type")
+	default:
+		writeHTTPError(w, http.StatusBadRequest, "upload rejected")
+	}
+}
+
+func writeHTTPError(w http.ResponseWriter, status int, message string) {
+	writeJSON(w, status, httpErrorResponse{Error: message})
+}
+
+func writeJSON(w http.ResponseWriter, status int, value any) {
+	w.Header().Set("Content-Type", "application/json")
+	w.WriteHeader(status)
+	_ = json.NewEncoder(w).Encode(value)
+}

internal/app/upload_http_integration_test.go

@@ -0,0 +1,419 @@
+package app
+
+import (
+	"archive/tar"
+	"bytes"
+	"compress/gzip"
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"io/fs"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+	"testing"
+	"time"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/ingest"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
+)
+
+func TestHTTPUploadPublishesTarAndGzipFanout(t *testing.T) {
+	tests := []struct {
+		name        string
+		compressed  bool
+		contentType string
+	}{
+		{name: "tar", contentType: ingest.ContentTypeTar},
+		{name: "gzip", compressed: true, contentType: ingest.ContentTypeGzip},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			firstDestination := t.TempDir()
+			secondDestination := t.TempDir()
+			cfg := httpUploadIntegrationConfig(t, []httpUploadPipelineSpec{{
+				id:           "reports",
+				tokenEnv:     "REPORTS_TOKEN",
+				stagingPath:  filepath.Join(t.TempDir(), "reports"),
+				destinations: []string{firstDestination, secondDestination},
+			}}, 4, 1)
+			handler, err := newUploadHTTPHandler(context.Background(), cfg, uploadHTTPTestEnvironment(map[string]string{
+				"REPORTS_TOKEN": "reports-secret",
+			}))
+			if err != nil {
+				t.Fatalf("newUploadHTTPHandler() error = %v", err)
+			}
+			server := httptest.NewServer(handler)
+			defer server.Close()
+
+			runID := submitHTTPUpload(t, server, "reports-secret", tt.contentType, bundleArchive(t, tt.compressed, testutil.BundleOptions{}))
+			record := waitForHTTPUploadStatus(t, server, runID, UploadStatusSucceeded)
+
+			if record.Report == nil {
+				t.Fatal("completed status report = nil, want run report")
+			}
+			if record.Report.Summary.Status != "ok" {
+				t.Fatalf("summary status = %q, want ok", record.Report.Summary.Status)
+			}
+			if got, want := len(record.Report.Actions), 2; got != want {
+				t.Fatalf("action count = %d, want %d", got, want)
+			}
+			assertPublishedBundle(t, firstDestination)
+			assertPublishedBundle(t, secondDestination)
+		})
+	}
+}
+
+func TestHTTPUploadInvalidArchiveFailsWithoutPublishing(t *testing.T) {
+	destination := t.TempDir()
+	cfg := httpUploadIntegrationConfig(t, []httpUploadPipelineSpec{{
+		id:           "reports",
+		tokenEnv:     "REPORTS_TOKEN",
+		stagingPath:  filepath.Join(t.TempDir(), "reports"),
+		destinations: []string{destination},
+	}}, 4, 1)
+	handler, err := newUploadHTTPHandler(context.Background(), cfg, uploadHTTPTestEnvironment(map[string]string{
+		"REPORTS_TOKEN": "reports-secret",
+	}))
+	if err != nil {
+		t.Fatalf("newUploadHTTPHandler() error = %v", err)
+	}
+	server := httptest.NewServer(handler)
+	defer server.Close()
+
+	runID := submitHTTPUpload(t, server, "reports-secret", ingest.ContentTypeTar, []byte("not a tar archive"))
+	record := waitForHTTPUploadStatus(t, server, runID, UploadStatusFailed)
+
+	if record.Error == "" {
+		t.Fatal("failed status error is empty")
+	}
+	if record.Report != nil {
+		t.Fatalf("failed staging report = %#v, want nil", record.Report)
+	}
+	assertDirectoryEmpty(t, destination)
+}
+
+func TestHTTPUploadSamePipelineRequestsSerialize(t *testing.T) {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	release := make(chan struct{})
+	started := make(chan struct{}, 1)
+	coordinator := newUploadCoordinator(ctx, httpUploadIntegrationConfig(t, []httpUploadPipelineSpec{{
+		id:           "reports",
+		tokenEnv:     "REPORTS_TOKEN",
+		stagingPath:  filepath.Join(t.TempDir(), "reports"),
+		destinations: []string{t.TempDir()},
+	}}, 4, 2), uploadCoordinatorHooks{
+		randomSuffix: uploadTestSuffixes("00000001", "00000002"),
+		stage:        successfulUploadStage,
+		run: func(ctx context.Context, cfg config.Config, options RunPipelineWithLocalSourceOptions) (RunReport, error) {
+			select {
+			case started <- struct{}{}:
+			default:
+			}
+			<-release
+			return RunReport{}, nil
+		},
+	})
+	handler := uploadHTTPHandler{
+		coordinator: coordinator,
+		tokens:      map[string]string{"reports-secret": "reports"},
+		limits:      map[string]int64{"reports": 1024},
+	}
+	server := httptest.NewServer(handler)
+	defer server.Close()
+
+	firstRunID := submitHTTPUpload(t, server, "reports-secret", ingest.ContentTypeTar, []byte("first"))
+	waitForRunStart(t, started)
+	first := waitForHTTPUploadStatus(t, server, firstRunID, UploadStatusRunning)
+	secondRunID := submitHTTPUpload(t, server, "reports-secret", ingest.ContentTypeTar, []byte("second"))
+	second := waitForHTTPUploadStatus(t, server, secondRunID, UploadStatusQueued)
+
+	if first.PipelineID != "reports" || second.PipelineID != "reports" {
+		t.Fatalf("statuses = %#v %#v, want same pipeline", first, second)
+	}
+	if got := coordinator.RunningCount(); got != 1 {
+		t.Fatalf("running count = %d, want 1", got)
+	}
+	close(release)
+	waitForHTTPUploadStatus(t, server, firstRunID, UploadStatusSucceeded)
+	waitForHTTPUploadStatus(t, server, secondRunID, UploadStatusSucceeded)
+}
+
+func TestHTTPUploadDifferentPipelinesRunConcurrently(t *testing.T) {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	release := make(chan struct{})
+	started := make(chan string, 2)
+	coordinator := newUploadCoordinator(ctx, httpUploadIntegrationConfig(t, []httpUploadPipelineSpec{
+		{
+			id:           "reports-one",
+			tokenEnv:     "REPORTS_ONE_TOKEN",
+			stagingPath:  filepath.Join(t.TempDir(), "reports-one"),
+			destinations: []string{t.TempDir()},
+		},
+		{
+			id:           "reports-two",
+			tokenEnv:     "REPORTS_TWO_TOKEN",
+			stagingPath:  filepath.Join(t.TempDir(), "reports-two"),
+			destinations: []string{t.TempDir()},
+		},
+	}, 4, 2), uploadCoordinatorHooks{
+		randomSuffix: uploadTestSuffixes("00000001", "00000002"),
+		stage:        successfulUploadStage,
+		run: func(ctx context.Context, cfg config.Config, options RunPipelineWithLocalSourceOptions) (RunReport, error) {
+			started <- options.PipelineID
+			<-release
+			return RunReport{}, nil
+		},
+	})
+	handler := uploadHTTPHandler{
+		coordinator: coordinator,
+		tokens: map[string]string{
+			"one-secret": "reports-one",
+			"two-secret": "reports-two",
+		},
+		limits: map[string]int64{
+			"reports-one": 1024,
+			"reports-two": 1024,
+		},
+	}
+	server := httptest.NewServer(handler)
+	defer server.Close()
+
+	firstRunID := submitHTTPUpload(t, server, "one-secret", ingest.ContentTypeTar, []byte("first"))
+	secondRunID := submitHTTPUpload(t, server, "two-secret", ingest.ContentTypeTar, []byte("second"))
+	waitForStartedPipelines(t, started, "reports-one", "reports-two")
+	waitForHTTPUploadStatus(t, server, firstRunID, UploadStatusRunning)
+	waitForHTTPUploadStatus(t, server, secondRunID, UploadStatusRunning)
+
+	if got := coordinator.RunningCount(); got != 2 {
+		t.Fatalf("running count = %d, want 2", got)
+	}
+	close(release)
+	waitForHTTPUploadStatus(t, server, firstRunID, UploadStatusSucceeded)
+	waitForHTTPUploadStatus(t, server, secondRunID, UploadStatusSucceeded)
+}
+
+type httpUploadPipelineSpec struct {
+	id           string
+	tokenEnv     string
+	stagingPath  string
+	destinations []string
+}
+
+func httpUploadIntegrationConfig(t *testing.T, pipelines []httpUploadPipelineSpec, queueSize, maxConcurrency int) config.Config {
+	t.Helper()
+	size := config.ByteSize(1024 * 1024)
+	retention := config.Duration(time.Minute)
+	cfg := config.Config{
+		Server: config.Server{HTTP: config.HTTPServer{
+			Bind:           config.DefaultHTTPBind,
+			StagingRoot:    t.TempDir(),
+			MaxUploadSize:  &size,
+			QueueSize:      queueSize,
+			MaxConcurrency: maxConcurrency,
+			Retention:      &retention,
+		}},
+	}
+	for _, spec := range pipelines {
+		pipeline := config.Pipeline{
+			ID: spec.id,
+			Source: config.Backend{
+				Backend: config.BackendHTTPUpload,
+				Upload: config.HTTPUpload{
+					TokenEnv:      spec.tokenEnv,
+					StagingPath:   spec.stagingPath,
+					MaxUploadSize: &size,
+				},
+			},
+		}
+		for index, destination := range spec.destinations {
+			pipeline.Destinations = append(pipeline.Destinations, config.Destination{
+				ID:      fmt.Sprintf("archive-%d", index+1),
+				Backend: config.BackendLocal,
+				Path:    destination,
+				Publish: &config.PublishPolicy{Source: true},
+			})
+		}
+		cfg.Pipelines = append(cfg.Pipelines, pipeline)
+	}
+	config.ApplyDefaults(&cfg)
+	return cfg
+}
+
+func submitHTTPUpload(t *testing.T, server *httptest.Server, token, contentType string, body []byte) UploadRunID {
+	t.Helper()
+	request, err := http.NewRequest(http.MethodPost, server.URL+"/upload", bytes.NewReader(body))
+	if err != nil {
+		t.Fatalf("NewRequest() error = %v", err)
+	}
+	request.Header.Set("Authorization", "Bearer "+token)
+	request.Header.Set("Content-Type", contentType)
+	response, err := server.Client().Do(request)
+	if err != nil {
+		t.Fatalf("POST /upload error = %v", err)
+	}
+	defer response.Body.Close()
+	if response.StatusCode != http.StatusAccepted {
+		t.Fatalf("POST /upload status = %d, want %d", response.StatusCode, http.StatusAccepted)
+	}
+	var accepted uploadAcceptedResponse
+	if err := json.NewDecoder(response.Body).Decode(&accepted); err != nil {
+		t.Fatalf("decode accepted response: %v", err)
+	}
+	if accepted.RunID == "" || accepted.Status != UploadStatusAccepted {
+		t.Fatalf("accepted response = %#v, want run id and accepted status", accepted)
+	}
+	return accepted.RunID
+}
+
+func waitForHTTPUploadStatus(t *testing.T, server *httptest.Server, runID UploadRunID, status UploadStatus) UploadRunRecord {
+	t.Helper()
+	deadline := time.Now().Add(3 * time.Second)
+	var latest UploadRunRecord
+	var latestStatus int
+	for time.Now().Before(deadline) {
+		latest, latestStatus = getHTTPUploadStatus(t, server, runID)
+		if latestStatus == http.StatusOK && latest.Status == status {
+			return latest
+		}
+		time.Sleep(time.Millisecond)
+	}
+	t.Fatalf("timed out waiting for status %s; latest HTTP status=%d record=%#v", status, latestStatus, latest)
+	return UploadRunRecord{}
+}
+
+func getHTTPUploadStatus(t *testing.T, server *httptest.Server, runID UploadRunID) (UploadRunRecord, int) {
+	t.Helper()
+	response, err := server.Client().Get(server.URL + "/runs/" + string(runID))
+	if err != nil {
+		t.Fatalf("GET /runs error = %v", err)
+	}
+	defer response.Body.Close()
+	if response.StatusCode != http.StatusOK {
+		return UploadRunRecord{}, response.StatusCode
+	}
+	var record UploadRunRecord
+	if err := json.NewDecoder(response.Body).Decode(&record); err != nil {
+		t.Fatalf("decode run status: %v", err)
+	}
+	return record, response.StatusCode
+}
+
+func bundleArchive(t *testing.T, compressed bool, opts testutil.BundleOptions) []byte {
+	t.Helper()
+	root := t.TempDir()
+	testutil.WriteSourceBundle(t, root, "", opts)
+	return tarDirectory(t, root, compressed)
+}
+
+func tarDirectory(t *testing.T, root string, compressed bool) []byte {
+	t.Helper()
+	var output bytes.Buffer
+	var writer io.WriteCloser = nopWriteCloser{writer: &output}
+	if compressed {
+		gzipWriter := gzip.NewWriter(&output)
+		writer = gzipWriter
+	}
+	tarWriter := tar.NewWriter(writer)
+	if err := filepath.WalkDir(root, func(filePath string, entry fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if entry.IsDir() {
+			return nil
+		}
+		relative, err := filepath.Rel(root, filePath)
+		if err != nil {
+			return err
+		}
+		data, err := os.ReadFile(filePath)
+		if err != nil {
+			return err
+		}
+		header := &tar.Header{
+			Name: filepath.ToSlash(relative),
+			Mode: 0o600,
+			Size: int64(len(data)),
+		}
+		if err := tarWriter.WriteHeader(header); err != nil {
+			return err
+		}
+		if _, err := tarWriter.Write(data); err != nil {
+			return err
+		}
+		return nil
+	}); err != nil {
+		t.Fatalf("walk bundle: %v", err)
+	}
+	if err := tarWriter.Close(); err != nil {
+		t.Fatalf("close tar: %v", err)
+	}
+	if err := writer.Close(); err != nil {
+		t.Fatalf("close archive: %v", err)
+	}
+	return output.Bytes()
+}
+
+type nopWriteCloser struct {
+	writer io.Writer
+}
+
+func (writer nopWriteCloser) Write(data []byte) (int, error) {
+	return writer.writer.Write(data)
+}
+
+func (writer nopWriteCloser) Close() error {
+	return nil
+}
+
+func assertPublishedBundle(t *testing.T, destinationRoot string) {
+	t.Helper()
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "summary.txt"), "Summary\n")
+	if _, err := os.Stat(filepath.Join(destinationRoot, storage.StateFileName)); err != nil {
+		t.Fatalf("destination state stat: %v", err)
+	}
+}
+
+func assertDirectoryEmpty(t *testing.T, root string) {
+	t.Helper()
+	entries, err := os.ReadDir(root)
+	if err != nil {
+		t.Fatalf("ReadDir() error = %v", err)
+	}
+	if len(entries) != 0 {
+		t.Fatalf("directory %s has %d entries, want empty", root, len(entries))
+	}
+}
+
+func waitForRunStart(t *testing.T, started <-chan struct{}) {
+	t.Helper()
+	select {
+	case <-started:
+	case <-time.After(time.Second):
+		t.Fatal("timed out waiting for run start")
+	}
+}
+
+func waitForStartedPipelines(t *testing.T, started <-chan string, want ...string) {
+	t.Helper()
+	remaining := map[string]bool{}
+	for _, pipelineID := range want {
+		remaining[pipelineID] = true
+	}
+	deadline := time.After(time.Second)
+	for len(remaining) > 0 {
+		select {
+		case pipelineID := <-started:
+			delete(remaining, pipelineID)
+		case <-deadline:
+			t.Fatalf("timed out waiting for pipelines to start; remaining=%v", remaining)
+		}
+	}
+}

internal/app/upload_http_test.go

@@ -0,0 +1,331 @@
+package app
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+)
+
+type fakeUploadCoordinator struct {
+	canAccept bool
+	submit    func(context.Context, UploadRequest) (UploadRunRecord, error)
+	status    func(UploadRunID) (UploadRunRecord, bool)
+}
+
+func (fake fakeUploadCoordinator) CanAccept() bool {
+	return fake.canAccept
+}
+
+func (fake fakeUploadCoordinator) Submit(ctx context.Context, request UploadRequest) (UploadRunRecord, error) {
+	if fake.submit == nil {
+		return UploadRunRecord{}, errors.New("unexpected submit")
+	}
+	return fake.submit(ctx, request)
+}
+
+func (fake fakeUploadCoordinator) Status(runID UploadRunID) (UploadRunRecord, bool) {
+	if fake.status == nil {
+		return UploadRunRecord{}, false
+	}
+	return fake.status(runID)
+}
+
+func TestResolveUploadTokensFailsForMissingAndDuplicateTokens(t *testing.T) {
+	cfg := uploadHTTPTestConfig()
+
+	_, _, err := resolveUploadTokens(cfg, config.NewEnvironment(nil, func(string) (string, bool) {
+		return "", false
+	}))
+	if err == nil || !strings.Contains(err.Error(), "UPLOAD_TOKEN") {
+		t.Fatalf("resolveUploadTokens() error = %v, want missing UPLOAD_TOKEN", err)
+	}
+
+	cfg.Pipelines = append(cfg.Pipelines, config.Pipeline{
+		ID: "weekly",
+		Source: config.Backend{
+			Backend: config.BackendHTTPUpload,
+			Upload:  config.HTTPUpload{TokenEnv: "OTHER_UPLOAD_TOKEN"},
+		},
+		Destinations: cfg.Pipelines[0].Destinations,
+	})
+	config.ApplyDefaults(&cfg)
+	secret := "super-secret-token"
+	_, _, err = resolveUploadTokens(cfg, uploadHTTPTestEnvironment(map[string]string{
+		"UPLOAD_TOKEN":       secret,
+		"OTHER_UPLOAD_TOKEN": secret,
+	}))
+	if err == nil {
+		t.Fatal("resolveUploadTokens() error = nil, want duplicate token error")
+	}
+	if strings.Contains(err.Error(), secret) {
+		t.Fatalf("duplicate token error exposed secret value: %q", err)
+	}
+}
+
+func TestNewUploadHTTPHandlerAcceptsDefaultedConfig(t *testing.T) {
+	cfg := uploadHTTPTestConfig()
+	cfg.Server.HTTP.Bind = ""
+	cfg.Server.HTTP.StagingRoot = ""
+	cfg.Server.HTTP.MaxUploadSize = nil
+	cfg.Server.HTTP.QueueSize = 0
+	cfg.Server.HTTP.MaxConcurrency = 0
+	cfg.Server.HTTP.Retention = nil
+	cfg.Pipelines[0].Source.Upload.StagingPath = ""
+	cfg.Pipelines[0].Source.Upload.MaxUploadSize = nil
+
+	handler, err := newUploadHTTPHandler(context.Background(), cfg, uploadHTTPTestEnvironment(map[string]string{
+		"UPLOAD_TOKEN": "secret",
+	}))
+	if err != nil {
+		t.Fatalf("newUploadHTTPHandler() error = %v", err)
+	}
+	if handler == nil {
+		t.Fatal("newUploadHTTPHandler() = nil")
+	}
+}
+
+func TestUploadHTTPHandlerAuthenticatesAndAcceptsUpload(t *testing.T) {
+	var submitted UploadRequest
+	handler := uploadHTTPHandler{
+		coordinator: fakeUploadCoordinator{
+			canAccept: true,
+			submit: func(_ context.Context, request UploadRequest) (UploadRunRecord, error) {
+				submitted = request
+				body, err := io.ReadAll(request.Body)
+				if err != nil {
+					t.Fatalf("read submitted body: %v", err)
+				}
+				if string(body) != "archive" {
+					t.Fatalf("submitted body = %q, want archive", body)
+				}
+				return UploadRunRecord{ID: "reports.20260603T120000Z.abcdef12", Status: UploadStatusAccepted}, nil
+			},
+		},
+		tokens: map[string]string{"valid-token": "reports"},
+		limits: map[string]int64{"reports": 1024},
+	}
+	recorder := httptest.NewRecorder()
+	request := httptest.NewRequest(http.MethodPost, "/upload", strings.NewReader("archive"))
+	request.Header.Set("Authorization", "Bearer valid-token")
+	request.Header.Set("Content-Type", "application/x-tar")
+
+	handler.ServeHTTP(recorder, request)
+
+	if recorder.Code != http.StatusAccepted {
+		t.Fatalf("status = %d, want %d; body = %q", recorder.Code, http.StatusAccepted, recorder.Body.String())
+	}
+	if submitted.PipelineID != "reports" {
+		t.Fatalf("submitted pipeline = %q, want reports", submitted.PipelineID)
+	}
+	var response uploadAcceptedResponse
+	if err := json.Unmarshal(recorder.Body.Bytes(), &response); err != nil {
+		t.Fatalf("decode response: %v", err)
+	}
+	if response.RunID != "reports.20260603T120000Z.abcdef12" || response.Status != UploadStatusAccepted {
+		t.Fatalf("response = %#v, want accepted run id", response)
+	}
+	if strings.Contains(recorder.Body.String(), "valid-token") {
+		t.Fatalf("response exposed token: %q", recorder.Body.String())
+	}
+}
+
+func TestUploadHTTPHandlerRejectsUnauthorizedRequests(t *testing.T) {
+	handler := uploadHTTPHandler{
+		coordinator: fakeUploadCoordinator{canAccept: true},
+		tokens:      map[string]string{"valid-token": "reports"},
+		limits:      map[string]int64{"reports": 1024},
+	}
+
+	for _, authHeader := range []string{"", "Bearer wrong-token"} {
+		recorder := httptest.NewRecorder()
+		request := httptest.NewRequest(http.MethodPost, "/upload", strings.NewReader("archive"))
+		request.Header.Set("Authorization", authHeader)
+		request.Header.Set("Content-Type", "application/x-tar")
+
+		handler.ServeHTTP(recorder, request)
+
+		if recorder.Code != http.StatusUnauthorized {
+			t.Fatalf("auth %q status = %d, want %d", authHeader, recorder.Code, http.StatusUnauthorized)
+		}
+		if strings.Contains(recorder.Body.String(), "valid-token") || strings.Contains(recorder.Body.String(), "wrong-token") {
+			t.Fatalf("unauthorized response exposed token: %q", recorder.Body.String())
+		}
+	}
+}
+
+func TestUploadHTTPHandlerRejectsUnsupportedOversizedFullQueueAndPipelineID(t *testing.T) {
+	tests := []struct {
+		name        string
+		canAccept   bool
+		url         string
+		contentType string
+		body        io.Reader
+		wantStatus  int
+	}{
+		{
+			name:        "unsupported content type",
+			canAccept:   true,
+			url:         "/upload",
+			contentType: "application/zip",
+			body:        strings.NewReader("archive"),
+			wantStatus:  http.StatusUnsupportedMediaType,
+		},
+		{
+			name:        "oversized",
+			canAccept:   true,
+			url:         "/upload",
+			contentType: "application/x-tar",
+			body:        strings.NewReader("too-large"),
+			wantStatus:  http.StatusRequestEntityTooLarge,
+		},
+		{
+			name:        "full queue",
+			canAccept:   false,
+			url:         "/upload",
+			contentType: "application/x-tar",
+			body:        &countingReader{reader: strings.NewReader("archive")},
+			wantStatus:  http.StatusServiceUnavailable,
+		},
+		{
+			name:        "submitted pipeline id",
+			canAccept:   true,
+			url:         "/upload?pipeline_id=reports",
+			contentType: "application/x-tar",
+			body:        strings.NewReader("archive"),
+			wantStatus:  http.StatusBadRequest,
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			handler := uploadHTTPHandler{
+				coordinator: fakeUploadCoordinator{
+					canAccept: tt.canAccept,
+					submit: func(context.Context, UploadRequest) (UploadRunRecord, error) {
+						t.Fatal("Submit should not be called")
+						return UploadRunRecord{}, nil
+					},
+				},
+				tokens: map[string]string{"valid-token": "reports"},
+				limits: map[string]int64{"reports": 4},
+			}
+			recorder := httptest.NewRecorder()
+			request := httptest.NewRequest(http.MethodPost, tt.url, tt.body)
+			request.Header.Set("Authorization", "Bearer valid-token")
+			request.Header.Set("Content-Type", tt.contentType)
+
+			handler.ServeHTTP(recorder, request)
+
+			if recorder.Code != tt.wantStatus {
+				t.Fatalf("status = %d, want %d; body = %q", recorder.Code, tt.wantStatus, recorder.Body.String())
+			}
+			if reader, ok := tt.body.(*countingReader); ok && reader.reads != 0 {
+				t.Fatalf("full queue read body %d time(s), want zero", reader.reads)
+			}
+		})
+	}
+}
+
+func TestUploadHTTPHandlerRunStatusAndHealth(t *testing.T) {
+	finishedAt := time.Date(2026, 6, 3, 12, 0, 0, 0, time.UTC)
+	handler := uploadHTTPHandler{
+		coordinator: fakeUploadCoordinator{
+			canAccept: true,
+			status: func(runID UploadRunID) (UploadRunRecord, bool) {
+				if runID != "reports.20260603T120000Z.abcdef12" {
+					return UploadRunRecord{}, false
+				}
+				return UploadRunRecord{
+					ID:         runID,
+					PipelineID: "reports",
+					Status:     UploadStatusSucceeded,
+					FinishedAt: &finishedAt,
+				}, true
+			},
+		},
+		tokens: map[string]string{"valid-token": "reports"},
+		limits: map[string]int64{"reports": 1024},
+	}
+
+	recorder := httptest.NewRecorder()
+	handler.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/healthz", nil))
+	if recorder.Code != http.StatusOK {
+		t.Fatalf("health status = %d, want %d", recorder.Code, http.StatusOK)
+	}
+
+	recorder = httptest.NewRecorder()
+	handler.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/runs/reports.20260603T120000Z.abcdef12", nil))
+	if recorder.Code != http.StatusOK {
+		t.Fatalf("run status = %d, want %d; body = %q", recorder.Code, http.StatusOK, recorder.Body.String())
+	}
+	var record UploadRunRecord
+	if err := json.Unmarshal(recorder.Body.Bytes(), &record); err != nil {
+		t.Fatalf("decode run status: %v", err)
+	}
+	if record.ID != "reports.20260603T120000Z.abcdef12" || record.Status != UploadStatusSucceeded {
+		t.Fatalf("record = %#v, want succeeded run status", record)
+	}
+
+	recorder = httptest.NewRecorder()
+	handler.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/runs/unknown", nil))
+	if recorder.Code != http.StatusNotFound {
+		t.Fatalf("unknown run status = %d, want %d", recorder.Code, http.StatusNotFound)
+	}
+}
+
+type countingReader struct {
+	reader io.Reader
+	reads  int
+}
+
+func (reader *countingReader) Read(data []byte) (int, error) {
+	reader.reads++
+	return reader.reader.Read(data)
+}
+
+func uploadHTTPTestConfig() config.Config {
+	size := config.ByteSize(1024)
+	retention := config.Duration(24 * time.Hour)
+	cfg := config.Config{
+		Server: config.Server{HTTP: config.HTTPServer{
+			Bind:           config.DefaultHTTPBind,
+			StagingRoot:    "/tmp/distributor-test",
+			MaxUploadSize:  &size,
+			QueueSize:      2,
+			MaxConcurrency: 1,
+			Retention:      &retention,
+		}},
+		Pipelines: []config.Pipeline{{
+			ID: "reports",
+			Source: config.Backend{
+				Backend: config.BackendHTTPUpload,
+				Upload: config.HTTPUpload{
+					TokenEnv:      "UPLOAD_TOKEN",
+					StagingPath:   "/tmp/distributor-test/reports",
+					MaxUploadSize: &size,
+				},
+			},
+			Destinations: []config.Destination{{
+				ID:      "local",
+				Backend: config.BackendLocal,
+				Path:    "/tmp/distributor-output",
+				Publish: &config.PublishPolicy{Source: true},
+			}},
+		}},
+	}
+	config.ApplyDefaults(&cfg)
+	return cfg
+}
+
+func uploadHTTPTestEnvironment(values map[string]string) config.Environment {
+	return config.NewEnvironment(values, func(string) (string, bool) {
+		return "", false
+	})
+}

internal/app/validate.go

@@ -5,28 +5,97 @@ import (
 	"fmt"
 	"io"
 
-	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
 )
 
 type ValidateOptions struct {
-	Path   string
-	Stdout io.Writer
+	Path         string
+	ConfigPath   string
+	PipelineID   string
+	BundlePath   string
+	Stdout       io.Writer
+	OutputFormat OutputFormat
 }
 
 func Validate(ctx context.Context, options ValidateOptions) error {
-	if options.Path == "" {
-		return fmt.Errorf("validate command requires a path")
+	return validateWithBackendFactory(ctx, options, newBackendFactoryWithEnvironment)
+}
+
+func validateWithBackendFactory(ctx context.Context, options ValidateOptions, provider backendFactoryProvider) error {
+	if err := ValidateOutputFormat(options.OutputFormat); err != nil {
+		return err
 	}
-	backend, err := newBackendFactory().openLocalPath(ctx, options.Path)
+	selection, err := selectSourceBundles(ctx, sourceCommandOptions{
+		CommandName: "validate",
+		Path:        options.Path,
+		ConfigPath:  options.ConfigPath,
+		PipelineID:  options.PipelineID,
+		BundlePath:  options.BundlePath,
+	}, provider)
 	if err != nil {
 		return err
 	}
-	bundles, err := bundle.Discover(ctx, backend, "")
+	return writeValidateResult(options, selection)
+}
+
+func validateConfigWithBackendFactory(ctx context.Context, cfg config.Config, options ValidateOptions, provider backendFactoryProvider) error {
+	if err := ValidateOutputFormat(options.OutputFormat); err != nil {
+		return err
+	}
+	selection, err := selectSourceBundlesFromConfig(ctx, cfg, sourceCommandOptions{
+		CommandName: "validate",
+		PipelineID:  options.PipelineID,
+		BundlePath:  options.BundlePath,
+	}, provider)
 	if err != nil {
 		return err
 	}
+	return writeValidateResult(options, selection)
+}
+
+func writeValidateResult(options ValidateOptions, selection sourceSelection) error {
+	if IsJSONOutput(options.OutputFormat) {
+		return WriteJSONEnvelope(options.Stdout, "validate", true, selection.Warnings, validateResultFromSelection(selection), nil)
+	}
+	var err error
 	if options.Stdout != nil {
-		_, err = fmt.Fprintf(options.Stdout, "Validated %d bundle(s)\n", len(bundles))
+		if err := writeWarnings(options.Stdout, selection.Warnings); err != nil {
+			return err
+		}
+		if selection.ConfigMode {
+			_, err = fmt.Fprintf(options.Stdout, "Validated %d bundle(s) for pipeline %s source %s\n", len(selection.Bundles), selection.PipelineID, selection.SourceBackend)
+		} else {
+			_, err = fmt.Fprintf(options.Stdout, "Validated %d bundle(s)\n", len(selection.Bundles))
+		}
 	}
 	return err
 }
+
+type validateResult struct {
+	PipelineID    string                 `json:"pipeline_id,omitempty"`
+	SourceBackend string                 `json:"source_backend,omitempty"`
+	BundleCount   int                    `json:"bundle_count"`
+	Bundles       []validateBundleResult `json:"bundles"`
+}
+
+type validateBundleResult struct {
+	Path string `json:"path"`
+	ID   string `json:"id"`
+}
+
+func validateResultFromSelection(selection sourceSelection) validateResult {
+	result := validateResult{
+		PipelineID:    selection.PipelineID,
+		SourceBackend: selection.SourceBackend,
+		BundleCount:   len(selection.Bundles),
+		Bundles:       make([]validateBundleResult, 0, len(selection.Bundles)),
+	}
+	for _, sourceBundle := range selection.Bundles {
+		result.Bundles = append(result.Bundles, validateBundleResult{
+			Path: storage.DisplayPath(sourceBundle.RootRelativePath),
+			ID:   sourceBundle.Manifest.ID,
+		})
+	}
+	return result
+}

internal/app/validate_test.go

@@ -3,9 +3,15 @@ package app
 import (
 	"bytes"
 	"context"
+	"encoding/json"
 	"path/filepath"
 	"strings"
 	"testing"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
+	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
 )
 
 func TestValidateLocalBundle(t *testing.T) {
@@ -31,9 +37,182 @@ func TestValidateExampleSourceBundle(t *testing.T) {
 	}
 }
 
+func TestValidateConfiguredLocalSource(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{})
+	var stdout bytes.Buffer
+
+	err := Validate(context.Background(), ValidateOptions{
+		ConfigPath: testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot),
+		PipelineID: "reports",
+		Stdout:     &stdout,
+	})
+
+	if err != nil {
+		t.Fatalf("Validate() configured source error = %v", err)
+	}
+	if got, want := stdout.String(), "Validated 1 bundle(s) for pipeline reports source local\n"; got != want {
+		t.Fatalf("stdout = %q, want %q", got, want)
+	}
+}
+
+func TestValidateConfiguredSourceBundlePath(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "daily/one", testutil.BundleOptions{ID: "reports.one"})
+	testutil.WriteSourceBundle(t, sourceRoot, "daily/two", testutil.BundleOptions{ID: "reports.two"})
+	var stdout bytes.Buffer
+
+	err := Validate(context.Background(), ValidateOptions{
+		ConfigPath:   testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot),
+		PipelineID:   "reports",
+		BundlePath:   "daily/two",
+		Stdout:       &stdout,
+		OutputFormat: OutputFormatJSON,
+	})
+
+	if err != nil {
+		t.Fatalf("Validate() configured bundle error = %v", err)
+	}
+	result := decodeAppResult(t, stdout.String())
+	if result["pipeline_id"] != "reports" || result["source_backend"] != "local" || result["bundle_count"] != float64(1) {
+		t.Fatalf("result = %#v, want configured source summary", result)
+	}
+	bundles, ok := result["bundles"].([]any)
+	if !ok || len(bundles) != 1 {
+		t.Fatalf("bundles = %#v, want one bundle", result["bundles"])
+	}
+	sourceBundle, ok := bundles[0].(map[string]any)
+	if !ok || sourceBundle["path"] != "daily/two" || sourceBundle["id"] != "reports.two" {
+		t.Fatalf("bundle = %#v, want narrowed bundle", sourceBundle)
+	}
+}
+
+func TestValidateConfiguredRemoteSourcesThroughStorageAbstraction(t *testing.T) {
+	s3Source := fake.New()
+	testutil.WriteFakeSourceBundle(t, s3Source, "", testutil.BundleOptions{ID: "reports.s3"})
+	sshSource := fake.New()
+	testutil.WriteFakeSourceBundle(t, sshSource, "daily", testutil.BundleOptions{ID: "reports.ssh"})
+	cfg := config.Config{Pipelines: []config.Pipeline{
+		{
+			ID: "s3-reports",
+			Source: config.Backend{
+				Backend:  config.BackendS3,
+				Endpoint: "http://s3.test",
+				Bucket:   "source-bucket",
+			},
+			Destinations: []config.Destination{{
+				ID:      "archive",
+				Backend: config.BackendLocal,
+				Path:    t.TempDir(),
+			}},
+		},
+		{
+			ID: "ssh-reports",
+			Source: config.Backend{
+				Backend: config.BackendSSH,
+				Host:    "ssh.test",
+				Path:    "/source",
+			},
+			Destinations: []config.Destination{{
+				ID:      "archive",
+				Backend: config.BackendLocal,
+				Path:    t.TempDir(),
+			}},
+		},
+	}}
+	config.ApplyDefaults(&cfg)
+	provider := fakeBackendFactoryProvider(t, map[string]storage.Backend{
+		"s3:source-bucket": s3Source,
+		"ssh:/source":      sshSource,
+	})
+
+	var s3Stdout bytes.Buffer
+	if err := validateConfigWithBackendFactory(context.Background(), cfg, ValidateOptions{
+		PipelineID:   "s3-reports",
+		Stdout:       &s3Stdout,
+		OutputFormat: OutputFormatJSON,
+	}, provider); err != nil {
+		t.Fatalf("validate s3 source error = %v", err)
+	}
+	s3Result := decodeAppResult(t, s3Stdout.String())
+	if s3Result["source_backend"] != "s3" || s3Result["bundle_count"] != float64(1) {
+		t.Fatalf("s3 result = %#v, want one s3 bundle", s3Result)
+	}
+
+	var sshStdout bytes.Buffer
+	if err := validateConfigWithBackendFactory(context.Background(), cfg, ValidateOptions{
+		PipelineID: "ssh-reports",
+		BundlePath: "daily",
+		Stdout:     &sshStdout,
+	}, provider); err != nil {
+		t.Fatalf("validate ssh source error = %v", err)
+	}
+	if !strings.Contains(sshStdout.String(), "pipeline ssh-reports source ssh") {
+		t.Fatalf("ssh stdout = %q, want ssh source summary", sshStdout.String())
+	}
+}
+
+func TestValidateConfiguredSourceLoadsSecretsBeforeOpeningBackend(t *testing.T) {
+	sourceRoot := filepath.Join(t.TempDir(), "missing-source")
+	destinationRoot := t.TempDir()
+	configPath := writeConfigFile(t, `
+secrets:
+  directory: `+filepath.Join(t.TempDir(), "missing-secrets")+`
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: `+sourceRoot+`
+    destinations:
+      - id: archive
+        backend: local
+        path: `+destinationRoot+`
+`)
+
+	err := Validate(context.Background(), ValidateOptions{ConfigPath: configPath, PipelineID: "reports"})
+	if err == nil {
+		t.Fatal("Validate() error = nil, want secrets directory error")
+	}
+	if !strings.Contains(err.Error(), "load secrets directory") {
+		t.Fatalf("Validate() error = %v, want secrets directory error", err)
+	}
+	if strings.Contains(err.Error(), "missing-source") {
+		t.Fatalf("Validate() error = %v, opened source before loading secrets", err)
+	}
+}
+
+func TestValidateConfiguredSourceRequiresPipeline(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{})
+
+	err := Validate(context.Background(), ValidateOptions{
+		ConfigPath: testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot),
+	})
+
+	if err == nil || !strings.Contains(err.Error(), "requires --pipeline") {
+		t.Fatalf("Validate() error = %v, want required pipeline", err)
+	}
+}
+
 func TestValidateRequiresPath(t *testing.T) {
 	err := Validate(context.Background(), ValidateOptions{})
 	if err == nil || !strings.Contains(err.Error(), "requires a path") {
 		t.Fatalf("Validate() error = %v, want required path", err)
 	}
 }
+
+func decodeAppResult(t *testing.T, output string) map[string]any {
+	t.Helper()
+	var envelope map[string]any
+	if err := json.Unmarshal([]byte(output), &envelope); err != nil {
+		t.Fatalf("decode output: %v; output = %q", err, output)
+	}
+	result, ok := envelope["result"].(map[string]any)
+	if !ok {
+		t.Fatalf("result = %#v, want object", envelope["result"])
+	}
+	return result
+}

internal/bundle/digest.go

@@ -1,49 +1,19 @@
 package bundle
 
-import (
-	"crypto/sha256"
-	"encoding/hex"
-	"fmt"
-	"regexp"
-	"strconv"
-	"strings"
-)
-
-var digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
+import publicbundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
 
 func ValidateDigest(value string) error {
-	if !digestPattern.MatchString(value) {
-		return fmt.Errorf("must be lowercase sha256:<64 hex>")
-	}
-	return nil
+	return publicbundle.ValidateDigest(value)
 }
 
 func FileDigest(data []byte) string {
-	sum := sha256.Sum256(data)
-	return "sha256:" + hex.EncodeToString(sum[:])
+	return publicbundle.FileDigest(data)
 }
 
 func BundleDigest(files []ManifestFile) string {
-	canonical := CanonicalFilePayload(files)
-	sum := sha256.Sum256([]byte(canonical))
-	return "sha256:" + hex.EncodeToString(sum[:])
+	return publicbundle.BundleDigest(files)
 }
 
 func CanonicalFilePayload(files []ManifestFile) string {
-	var builder strings.Builder
-	builder.WriteByte('[')
-	for index, file := range files {
-		if index > 0 {
-			builder.WriteByte(',')
-		}
-		builder.WriteString(`{"path":`)
-		builder.WriteString(strconv.Quote(file.Path))
-		builder.WriteString(`,"sha256":`)
-		builder.WriteString(strconv.Quote(file.SHA256))
-		builder.WriteString(`,"size":`)
-		builder.WriteString(strconv.FormatInt(file.Size, 10))
-		builder.WriteByte('}')
-	}
-	builder.WriteByte(']')
-	return builder.String()
+	return publicbundle.CanonicalFilePayload(files)
 }

internal/bundle/manifest.go

@@ -1,110 +1,32 @@
 package bundle
 
-import (
-	"bytes"
-	"encoding/json"
-	"fmt"
-	"io"
-	"time"
-)
+import publicbundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
 
-const ManifestName = "manifest.json"
+const ManifestName = publicbundle.ManifestName
 
-type Manifest struct {
-	SchemaVersion int            `json:"schema_version"`
-	ID            string         `json:"id"`
-	Digest        string         `json:"digest"`
-	Created       time.Time      `json:"created"`
-	Files         []ManifestFile `json:"files"`
-}
+const SchemaVersion = publicbundle.SchemaVersion
 
-type ManifestFile struct {
-	Path   string `json:"path"`
-	SHA256 string `json:"sha256"`
-	Size   int64  `json:"size"`
-}
+type Manifest = publicbundle.Manifest
+
+type ManifestFile = publicbundle.ManifestFile
 
 type Bundle struct {
 	RootRelativePath string
 	Manifest         Manifest
 }
 
-type rawManifest struct {
-	SchemaVersion *int              `json:"schema_version"`
-	ID            *string           `json:"id"`
-	Digest        *string           `json:"digest"`
-	Created       *string           `json:"created"`
-	Files         []rawManifestFile `json:"files"`
-}
-
-type rawManifestFile struct {
-	Path   *string `json:"path"`
-	SHA256 *string `json:"sha256"`
-	Size   *int64  `json:"size"`
-}
-
 func ParseManifest(data []byte) (Manifest, error) {
-	decoder := json.NewDecoder(bytes.NewReader(data))
-	var raw rawManifest
-	if err := decoder.Decode(&raw); err != nil {
-		return Manifest{}, fmt.Errorf("parse manifest: %w", err)
-	}
-	var extra any
-	if err := decoder.Decode(&extra); err != io.EOF {
-		return Manifest{}, fmt.Errorf("parse manifest: trailing data")
-	}
-
-	var manifest Manifest
-	if raw.SchemaVersion == nil {
-		return Manifest{}, fmt.Errorf("manifest schema_version is required")
-	}
-	manifest.SchemaVersion = *raw.SchemaVersion
-	if raw.ID == nil || *raw.ID == "" {
-		return Manifest{}, fmt.Errorf("manifest id is required")
-	}
-	manifest.ID = *raw.ID
-	if raw.Digest == nil || *raw.Digest == "" {
-		return Manifest{}, fmt.Errorf("manifest digest is required")
-	}
-	manifest.Digest = *raw.Digest
-	if raw.Created == nil || *raw.Created == "" {
-		return Manifest{}, fmt.Errorf("manifest created is required")
-	}
-	created, err := time.Parse(time.RFC3339, *raw.Created)
-	if err != nil {
-		return Manifest{}, fmt.Errorf("manifest created must be RFC3339: %w", err)
-	}
-	manifest.Created = created
-	if len(raw.Files) == 0 {
-		return Manifest{}, fmt.Errorf("manifest files is required")
-	}
-
-	for index, rawFile := range raw.Files {
-		file, err := parseManifestFile(index, rawFile)
-		if err != nil {
-			return Manifest{}, err
-		}
-		manifest.Files = append(manifest.Files, file)
-	}
-	if err := ValidateManifest(manifest); err != nil {
-		return Manifest{}, fmt.Errorf("manifest %w", err)
-	}
-	return manifest, nil
+	return publicbundle.ParseManifest(data)
 }
 
-func parseManifestFile(index int, raw rawManifestFile) (ManifestFile, error) {
-	if raw.Path == nil || *raw.Path == "" {
-		return ManifestFile{}, fmt.Errorf("manifest files[%d].path is required", index)
-	}
-	if raw.SHA256 == nil || *raw.SHA256 == "" {
-		return ManifestFile{}, fmt.Errorf("manifest files[%d].sha256 is required", index)
-	}
-	if raw.Size == nil {
-		return ManifestFile{}, fmt.Errorf("manifest files[%d].size is required", index)
-	}
-	return ManifestFile{
-		Path:   *raw.Path,
-		SHA256: *raw.SHA256,
-		Size:   *raw.Size,
-	}, nil
+func MarshalManifest(manifest Manifest) ([]byte, error) {
+	return publicbundle.MarshalManifest(manifest)
+}
+
+func ValidateManifest(manifest Manifest) error {
+	return publicbundle.ValidateManifest(manifest)
+}
+
+func ValidateSourcePath(path string) error {
+	return publicbundle.ValidateSourcePath(path)
 }

internal/bundle/validate.go

@@ -7,55 +7,6 @@ import (
 	"gitea.maximumdirect.net/eric/distributor/internal/storage"
 )
 
-func ValidateSourcePath(path string) error {
-	if err := storage.ValidatePath(path); err != nil {
-		return err
-	}
-	switch path {
-	case ManifestName, storage.StateFileName:
-		return fmt.Errorf("%q is reserved", path)
-	}
-	return nil
-}
-
-func ValidateManifest(manifest Manifest) error {
-	if manifest.SchemaVersion != 1 {
-		return fmt.Errorf("schema_version must be 1")
-	}
-	if manifest.ID == "" {
-		return fmt.Errorf("id is required")
-	}
-	if err := ValidateDigest(manifest.Digest); err != nil {
-		return fmt.Errorf("digest: %w", err)
-	}
-	if manifest.Created.IsZero() {
-		return fmt.Errorf("created is required")
-	}
-	if len(manifest.Files) == 0 {
-		return fmt.Errorf("files is required")
-	}
-	seen := make(map[string]struct{}, len(manifest.Files))
-	for index, file := range manifest.Files {
-		if err := ValidateSourcePath(file.Path); err != nil {
-			return fmt.Errorf("files[%d].path: %w", index, err)
-		}
-		if err := ValidateDigest(file.SHA256); err != nil {
-			return fmt.Errorf("files[%d].sha256: %w", index, err)
-		}
-		if file.Size < 0 {
-			return fmt.Errorf("files[%d].size must be non-negative", index)
-		}
-		if _, exists := seen[file.Path]; exists {
-			return fmt.Errorf("files[%d].path duplicates %q", index, file.Path)
-		}
-		seen[file.Path] = struct{}{}
-	}
-	if actual := BundleDigest(manifest.Files); actual != manifest.Digest {
-		return fmt.Errorf("digest mismatch: got %s want %s", actual, manifest.Digest)
-	}
-	return nil
-}
-
 func Validate(ctx context.Context, backend storage.Backend, bundleRoot string) (Bundle, error) {
 	return validateAt(ctx, backend, bundleRoot, bundleRoot)
 }

internal/cli/format.go

@@ -0,0 +1,22 @@
+package cli
+
+import (
+	"flag"
+	"fmt"
+	"io"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/app"
+)
+
+func addFormatFlag(flags *flag.FlagSet) *string {
+	return flags.String("format", string(app.OutputFormatText), "output format: text or json")
+}
+
+func parseOutputFormat(stderr io.Writer, command, raw string) (app.OutputFormat, bool) {
+	format := app.OutputFormat(raw)
+	if err := app.ValidateOutputFormat(format); err != nil {
+		fmt.Fprintf(stderr, "%s: %s --format: %s\n", app.Name, command, err)
+		return "", false
+	}
+	return app.NormalizeOutputFormat(format), true
+}

internal/cli/inspect.go

@@ -13,11 +13,18 @@ func inspectCommand(ctx context.Context, args []string, stdout, stderr io.Writer
 		printInspectHelp(stdout)
 		return exitOK
 	}
-	path, ok := parseOptionalPathArg(stderr, "inspect", args)
+	parsed, ok := parseSourceDiagnosticArgs(stderr, "inspect", args)
 	if !ok {
 		return exitUsage
 	}
-	if err := app.Inspect(ctx, app.InspectOptions{Path: path, Stdout: stdout}); err != nil {
+	if err := app.Inspect(ctx, app.InspectOptions{
+		Path:         parsed.Path,
+		ConfigPath:   parsed.ConfigPath,
+		PipelineID:   parsed.PipelineID,
+		BundlePath:   parsed.BundlePath,
+		Stdout:       stdout,
+		OutputFormat: parsed.OutputFormat,
+	}); err != nil {
 		return fail(stderr, err)
 	}
 	return exitOK
@@ -25,8 +32,16 @@ func inspectCommand(ctx context.Context, args []string, stdout, stderr io.Writer
 
 func printInspectHelp(w io.Writer) {
 	fmt.Fprint(w, `Usage:
-  distributor inspect <path>
+  distributor inspect [--format text|json] <path>
+  distributor inspect --config <path> --pipeline <id> [--bundle <path>] [--format text|json]
 
-Print a normalized summary of local source bundles.
+Options:
+  --config <path>     Path to config file for configured source inspection
+  --pipeline <id>     Pipeline id to inspect in config mode
+  --bundle <path>     Source-root-relative bundle path to inspect
+  --format text|json  Output format
+
+Print a normalized summary of local source bundles or a configured pipeline
+source.
 `)
 }

internal/cli/manifest.go

@@ -0,0 +1,139 @@
+package cli
+
+import (
+	"context"
+	"flag"
+	"fmt"
+	"io"
+	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/app"
+)
+
+func manifestCommand(ctx context.Context, args []string, stdout, stderr io.Writer) int {
+	if len(args) == 0 || args[0] == "-h" || args[0] == "--help" || args[0] == "help" {
+		printManifestHelp(stdout)
+		return exitOK
+	}
+	switch args[0] {
+	case "create":
+		return manifestCreateCommand(ctx, args[1:], stdout, stderr)
+	default:
+		fmt.Fprintf(stderr, "%s: manifest unknown command %q\n\n", app.Name, args[0])
+		printManifestHelp(stderr)
+		return exitUsage
+	}
+}
+
+func manifestCreateCommand(ctx context.Context, args []string, stdout, stderr io.Writer) int {
+	if hasHelp(args) {
+		printManifestCreateHelp(stdout)
+		return exitOK
+	}
+	flags := flag.NewFlagSet("manifest create", flag.ContinueOnError)
+	flags.SetOutput(stderr)
+	id := flags.String("id", "", "source bundle id")
+	created := flags.String("created", "", "source created timestamp")
+	overwrite := flags.Bool("overwrite", false, "replace an existing manifest.json")
+	formatFlag := addFormatFlag(flags)
+	var files repeatedFlag
+	flags.Var(&files, "file", "bundle-relative file to include")
+	flagArgs, positionalArgs, ok := splitManifestCreateArgs(stderr, args)
+	if !ok {
+		return exitUsage
+	}
+	if err := flags.Parse(flagArgs); err != nil {
+		return exitUsage
+	}
+	if len(positionalArgs) != 1 {
+		fmt.Fprintf(stderr, "%s: manifest create requires exactly one bundle path\n", app.Name)
+		return exitUsage
+	}
+	format, ok := parseOutputFormat(stderr, "manifest create", *formatFlag)
+	if !ok {
+		return exitUsage
+	}
+	err := app.ManifestCreate(ctx, app.ManifestCreateOptions{
+		Root:         positionalArgs[0],
+		ID:           *id,
+		Created:      *created,
+		Files:        []string(files),
+		Overwrite:    *overwrite,
+		Stdout:       stdout,
+		OutputFormat: format,
+	})
+	if err != nil {
+		return fail(stderr, err)
+	}
+	return exitOK
+}
+
+func splitManifestCreateArgs(stderr io.Writer, args []string) ([]string, []string, bool) {
+	var flagArgs []string
+	var positionalArgs []string
+	for index := 0; index < len(args); index++ {
+		arg := args[index]
+		switch arg {
+		case "--overwrite":
+			flagArgs = append(flagArgs, arg)
+		case "--id", "--created", "--file", "--format":
+			if index+1 >= len(args) {
+				fmt.Fprintf(stderr, "%s: manifest create %s requires a value\n", app.Name, arg)
+				return nil, nil, false
+			}
+			flagArgs = append(flagArgs, arg, args[index+1])
+			index++
+		default:
+			if strings.HasPrefix(arg, "--id=") ||
+				strings.HasPrefix(arg, "--created=") ||
+				strings.HasPrefix(arg, "--file=") ||
+				strings.HasPrefix(arg, "--format=") {
+				flagArgs = append(flagArgs, arg)
+				continue
+			}
+			if strings.HasPrefix(arg, "-") {
+				flagArgs = append(flagArgs, arg)
+				continue
+			}
+			positionalArgs = append(positionalArgs, arg)
+		}
+	}
+	return flagArgs, positionalArgs, true
+}
+
+type repeatedFlag []string
+
+func (f *repeatedFlag) String() string {
+	return fmt.Sprint([]string(*f))
+}
+
+func (f *repeatedFlag) Set(value string) error {
+	*f = append(*f, value)
+	return nil
+}
+
+func printManifestHelp(w io.Writer) {
+	fmt.Fprint(w, `Usage:
+  distributor manifest <command> [options]
+
+Commands:
+  create    Create a source bundle manifest
+
+Use "distributor manifest <command> --help" for command-specific help.
+`)
+}
+
+func printManifestCreateHelp(w io.Writer) {
+	fmt.Fprint(w, `Usage:
+  distributor manifest create <bundle-path> --id <bundle-id> [options]
+
+Options:
+  --id <bundle-id>     Source bundle id
+  --file <path>        Bundle-relative file to include; repeatable
+  --created <time>     RFC3339 source created timestamp
+  --overwrite          Replace an existing manifest.json
+  --format text|json   Output format
+
+Create manifest.json for a local source bundle directory.
+`)
+}

internal/cli/root.go

@@ -29,10 +29,14 @@ func Execute(ctx context.Context, args []string, stdout, stderr io.Writer) int {
 		return versionCommand(ctx, args[1:], stdout, stderr)
 	case "run":
 		return runCommand(ctx, args[1:], stdout, stderr)
+	case "serve":
+		return serveCommand(ctx, args[1:], stdout, stderr)
 	case "validate":
 		return validateCommand(ctx, args[1:], stdout, stderr)
 	case "inspect":
 		return inspectCommand(ctx, args[1:], stdout, stderr)
+	case "manifest":
+		return manifestCommand(ctx, args[1:], stdout, stderr)
 	default:
 		fmt.Fprintf(stderr, "%s: unknown command %q\n\n", app.Name, args[0])
 		printRootHelp(stderr)
@@ -49,8 +53,10 @@ Usage:
 Commands:
   version    Print version information
   run        Run configured distribution pipelines
+  serve      Run the HTTP upload server
   validate   Validate a source bundle or bundle tree
   inspect    Inspect bundles or distributor state
+  manifest   Create source bundle manifests
 
 Use "%s <command> --help" for command-specific help.
 `, app.Name, app.Name, app.Name)

internal/cli/root_test.go

@@ -3,15 +3,43 @@ package cli
 import (
 	"bytes"
 	"context"
+	"encoding/json"
+	"fmt"
+	"io"
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
 
+	"gitea.maximumdirect.net/eric/distributor/internal/app"
 	"gitea.maximumdirect.net/eric/distributor/internal/storage"
 	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
+	producerbundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
 )
 
+func decodeEnvelope(t *testing.T, stdout *bytes.Buffer) map[string]any {
+	t.Helper()
+	decoder := json.NewDecoder(strings.NewReader(stdout.String()))
+	var envelope map[string]any
+	if err := decoder.Decode(&envelope); err != nil {
+		t.Fatalf("decode JSON envelope: %v; stdout = %q", err, stdout.String())
+	}
+	var extra any
+	if err := decoder.Decode(&extra); err != io.EOF {
+		t.Fatalf("stdout contains more than one JSON document: %q", stdout.String())
+	}
+	return envelope
+}
+
+func envelopeResult(t *testing.T, envelope map[string]any) map[string]any {
+	t.Helper()
+	result, ok := envelope["result"].(map[string]any)
+	if !ok {
+		t.Fatalf("result = %#v, want object", envelope["result"])
+	}
+	return result
+}
+
 func TestExecuteRootHelp(t *testing.T) {
 	var stdout, stderr bytes.Buffer
 
@@ -44,6 +72,71 @@ func TestExecuteVersion(t *testing.T) {
 	}
 }
 
+func TestExecuteVersionJSON(t *testing.T) {
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"version", "--format", "json"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	envelope := decodeEnvelope(t, &stdout)
+	if envelope["command"] != "version" || envelope["ok"] != true {
+		t.Fatalf("envelope = %#v, want version ok", envelope)
+	}
+	result := envelopeResult(t, envelope)
+	if result["application"] != "distributor" || result["version"] != "dev" {
+		t.Fatalf("result = %#v, want application/version", result)
+	}
+	if stderr.Len() != 0 {
+		t.Fatalf("stderr = %q, want empty", stderr.String())
+	}
+}
+
+func TestExecuteServeParsesConfig(t *testing.T) {
+	originalServeApp := serveApp
+	defer func() {
+		serveApp = originalServeApp
+	}()
+	var gotOptions app.ServeOptions
+	serveApp = func(_ context.Context, options app.ServeOptions) error {
+		gotOptions = options
+		return nil
+	}
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"serve", "--config", "config.yml"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	if gotOptions.ConfigPath != "config.yml" {
+		t.Fatalf("ConfigPath = %q, want config.yml", gotOptions.ConfigPath)
+	}
+	if stdout.Len() != 0 {
+		t.Fatalf("stdout = %q, want empty", stdout.String())
+	}
+	if stderr.Len() != 0 {
+		t.Fatalf("stderr = %q, want empty", stderr.String())
+	}
+}
+
+func TestExecuteRejectsInvalidFormat(t *testing.T) {
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"version", "--format", "xml"}, &stdout, &stderr)
+
+	if code != exitUsage {
+		t.Fatalf("exit code = %d, want %d", code, exitUsage)
+	}
+	if stdout.Len() != 0 {
+		t.Fatalf("stdout = %q, want empty", stdout.String())
+	}
+	if !strings.Contains(stderr.String(), "format must be text or json") {
+		t.Fatalf("stderr = %q, want invalid format error", stderr.String())
+	}
+}
+
 func TestExecuteValidate(t *testing.T) {
 	var stdout, stderr bytes.Buffer
 
@@ -57,6 +150,63 @@ func TestExecuteValidate(t *testing.T) {
 	}
 }
 
+func TestExecuteValidateJSON(t *testing.T) {
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"validate", "--format", "json", filepath.Join("..", "bundle", "testdata", "valid_bundle")}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	envelope := decodeEnvelope(t, &stdout)
+	if envelope["command"] != "validate" || envelope["ok"] != true {
+		t.Fatalf("envelope = %#v, want validate ok", envelope)
+	}
+	result := envelopeResult(t, envelope)
+	if result["bundle_count"] != float64(1) {
+		t.Fatalf("result = %#v, want one bundle", result)
+	}
+}
+
+func TestExecuteValidateConfiguredSource(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{})
+	configPath := testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot)
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"validate", "--config", configPath, "--pipeline", "reports"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	if got, want := stdout.String(), "Validated 1 bundle(s) for pipeline reports source local\n"; got != want {
+		t.Fatalf("stdout = %q, want %q", got, want)
+	}
+}
+
+func TestExecuteValidateConfiguredSourceJSON(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "daily", testutil.BundleOptions{ID: "reports.daily"})
+	configPath := testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot)
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"validate", "--config", configPath, "--pipeline", "reports", "--bundle", "daily", "--format", "json"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	envelope := decodeEnvelope(t, &stdout)
+	if envelope["command"] != "validate" || envelope["ok"] != true {
+		t.Fatalf("envelope = %#v, want validate ok", envelope)
+	}
+	result := envelopeResult(t, envelope)
+	if result["pipeline_id"] != "reports" || result["source_backend"] != "local" || result["bundle_count"] != float64(1) {
+		t.Fatalf("result = %#v, want configured source metadata", result)
+	}
+}
+
 func TestExecuteValidateArgs(t *testing.T) {
 	validPath := filepath.Join("..", "bundle", "testdata", "valid_bundle")
 	tests := []struct {
@@ -84,6 +234,36 @@ func TestExecuteValidateArgs(t *testing.T) {
 			wantCode:   exitUsage,
 			wantStderr: "accepts at most one path",
 		},
+		{
+			name:       "path plus config",
+			args:       []string{"validate", "--config", "config.yml", "--pipeline", "reports", validPath},
+			wantCode:   exitUsage,
+			wantStderr: "does not accept a local path",
+		},
+		{
+			name:       "pipeline without config",
+			args:       []string{"validate", "--pipeline", "reports"},
+			wantCode:   exitUsage,
+			wantStderr: "requires --config",
+		},
+		{
+			name:       "bundle without config",
+			args:       []string{"validate", "--bundle", "daily"},
+			wantCode:   exitUsage,
+			wantStderr: "requires --config",
+		},
+		{
+			name:       "config without pipeline",
+			args:       []string{"validate", "--config", "config.yml"},
+			wantCode:   exitUsage,
+			wantStderr: "requires --pipeline",
+		},
+		{
+			name:       "invalid format",
+			args:       []string{"validate", "--format", "xml", validPath},
+			wantCode:   exitUsage,
+			wantStderr: "format must be text or json",
+		},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
@@ -115,6 +295,56 @@ func TestExecuteInspect(t *testing.T) {
 	}
 }
 
+func TestExecuteInspectJSON(t *testing.T) {
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"inspect", "--format", "json", filepath.Join("..", "bundle", "testdata", "valid_bundle")}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	envelope := decodeEnvelope(t, &stdout)
+	if envelope["command"] != "inspect" || envelope["ok"] != true {
+		t.Fatalf("envelope = %#v, want inspect ok", envelope)
+	}
+	result := envelopeResult(t, envelope)
+	bundles, ok := result["bundles"].([]any)
+	if !ok || len(bundles) != 1 {
+		t.Fatalf("bundles = %#v, want one bundle", result["bundles"])
+	}
+	bundle, ok := bundles[0].(map[string]any)
+	if !ok {
+		t.Fatalf("bundle = %#v, want object", bundles[0])
+	}
+	if bundle["id"] != "weather.daily.brentwood.2026-05-30" || bundle["file_count"] != float64(2) || bundle["total_size"] != float64(24) {
+		t.Fatalf("bundle = %#v, want normalized metadata", bundle)
+	}
+}
+
+func TestExecuteInspectConfiguredSource(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "daily", testutil.BundleOptions{ID: "reports.daily"})
+	configPath := testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot)
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"inspect", "--config", configPath, "--pipeline", "reports"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	for _, want := range []string{
+		"Pipeline: reports",
+		"Source: local",
+		"path=daily",
+		"id=reports.daily",
+	} {
+		if !strings.Contains(stdout.String(), want) {
+			t.Fatalf("stdout = %q, want substring %q", stdout.String(), want)
+		}
+	}
+}
+
 func TestExecuteInspectArgs(t *testing.T) {
 	validPath := filepath.Join("..", "bundle", "testdata", "valid_bundle")
 	tests := []struct {
@@ -142,6 +372,36 @@ func TestExecuteInspectArgs(t *testing.T) {
 			wantCode:   exitUsage,
 			wantStderr: "accepts at most one path",
 		},
+		{
+			name:       "path plus config",
+			args:       []string{"inspect", "--config", "config.yml", "--pipeline", "reports", validPath},
+			wantCode:   exitUsage,
+			wantStderr: "does not accept a local path",
+		},
+		{
+			name:       "pipeline without config",
+			args:       []string{"inspect", "--pipeline", "reports"},
+			wantCode:   exitUsage,
+			wantStderr: "requires --config",
+		},
+		{
+			name:       "bundle without config",
+			args:       []string{"inspect", "--bundle", "daily"},
+			wantCode:   exitUsage,
+			wantStderr: "requires --config",
+		},
+		{
+			name:       "config without pipeline",
+			args:       []string{"inspect", "--config", "config.yml"},
+			wantCode:   exitUsage,
+			wantStderr: "requires --pipeline",
+		},
+		{
+			name:       "invalid format",
+			args:       []string{"inspect", "--format", "xml", validPath},
+			wantCode:   exitUsage,
+			wantStderr: "format must be text or json",
+		},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
@@ -160,6 +420,206 @@ func TestExecuteInspectArgs(t *testing.T) {
 	}
 }
 
+func TestExecuteManifestCreateExplicitFiles(t *testing.T) {
+	root := t.TempDir()
+	writeCLIFile(t, root, "b.txt", "bravo")
+	writeCLIFile(t, root, "nested/a.txt", "alpha")
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{
+		"manifest", "create", root,
+		"--id", "reports.explicit",
+		"--created", "2026-06-01T11:00:00Z",
+		"--file", "b.txt",
+		"--file", "nested/a.txt",
+	}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	for _, want := range []string{
+		"created manifest.json",
+		"bundle: reports.explicit",
+		"files: 2",
+		"digest: sha256:",
+	} {
+		if !strings.Contains(stdout.String(), want) {
+			t.Fatalf("stdout = %q, want substring %q", stdout.String(), want)
+		}
+	}
+	manifest, err := producerbundle.LoadManifest(root)
+	if err != nil {
+		t.Fatalf("LoadManifest() error = %v", err)
+	}
+	if got, want := manifestPaths(manifest), []string{"b.txt", "nested/a.txt"}; !equalStrings(got, want) {
+		t.Fatalf("manifest paths = %v, want %v", got, want)
+	}
+	if err := producerbundle.ValidateBundle(root, manifest); err != nil {
+		t.Fatalf("ValidateBundle() error = %v", err)
+	}
+	var validateStdout, validateStderr bytes.Buffer
+	validateCode := Execute(context.Background(), []string{"validate", root}, &validateStdout, &validateStderr)
+	if validateCode != exitOK {
+		t.Fatalf("validate exit code = %d, want %d; stderr = %q", validateCode, exitOK, validateStderr.String())
+	}
+}
+
+func TestExecuteManifestCreateScansBundle(t *testing.T) {
+	root := t.TempDir()
+	writeCLIFile(t, root, "z.txt", "zulu")
+	writeCLIFile(t, root, ".env", "dotfile")
+	writeCLIFile(t, root, "nested/report.md", "# Report\n")
+	writeCLIFile(t, root, storage.StateFileName, "destination state")
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"manifest", "create", root, "--id", "reports.scan"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	manifest, err := producerbundle.LoadManifest(root)
+	if err != nil {
+		t.Fatalf("LoadManifest() error = %v", err)
+	}
+	if got, want := manifestPaths(manifest), []string{".env", "nested/report.md", "z.txt"}; !equalStrings(got, want) {
+		t.Fatalf("manifest paths = %v, want %v", got, want)
+	}
+}
+
+func TestExecuteManifestCreateJSON(t *testing.T) {
+	root := t.TempDir()
+	writeCLIFile(t, root, "report.md", "# Report\n")
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"manifest", "create", root, "--id", "reports.json", "--file", "report.md", "--format", "json"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	envelope := decodeEnvelope(t, &stdout)
+	if envelope["command"] != "manifest create" || envelope["ok"] != true {
+		t.Fatalf("envelope = %#v, want manifest create ok", envelope)
+	}
+	result := envelopeResult(t, envelope)
+	if result["id"] != "reports.json" || result["file_count"] != float64(1) {
+		t.Fatalf("result = %#v, want manifest summary", result)
+	}
+	if stderr.Len() != 0 {
+		t.Fatalf("stderr = %q, want empty", stderr.String())
+	}
+}
+
+func TestExecuteManifestCreateOverwrite(t *testing.T) {
+	root := t.TempDir()
+	writeCLIFile(t, root, "report.md", "old\n")
+	var stdout, stderr bytes.Buffer
+	code := Execute(context.Background(), []string{"manifest", "create", root, "--id", "reports.old", "--file", "report.md"}, &stdout, &stderr)
+	if code != exitOK {
+		t.Fatalf("initial exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	writeCLIFile(t, root, "report.md", "new\n")
+
+	stdout.Reset()
+	stderr.Reset()
+	code = Execute(context.Background(), []string{"manifest", "create", root, "--id", "reports.new", "--file", "report.md"}, &stdout, &stderr)
+	if code != exitError {
+		t.Fatalf("overwrite exit code = %d, want %d", code, exitError)
+	}
+	if !strings.Contains(stderr.String(), "write manifest") {
+		t.Fatalf("stderr = %q, want write manifest error", stderr.String())
+	}
+	manifest, err := producerbundle.LoadManifest(root)
+	if err != nil {
+		t.Fatalf("LoadManifest() error = %v", err)
+	}
+	if manifest.ID != "reports.old" {
+		t.Fatalf("manifest id = %q, want reports.old", manifest.ID)
+	}
+
+	stdout.Reset()
+	stderr.Reset()
+	code = Execute(context.Background(), []string{"manifest", "create", root, "--id", "reports.new", "--file", "report.md", "--overwrite"}, &stdout, &stderr)
+	if code != exitOK {
+		t.Fatalf("overwrite exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	manifest, err = producerbundle.LoadManifest(root)
+	if err != nil {
+		t.Fatalf("LoadManifest() error = %v", err)
+	}
+	if manifest.ID != "reports.new" {
+		t.Fatalf("manifest id = %q, want reports.new", manifest.ID)
+	}
+}
+
+func TestExecuteManifestCreateRejectsSymlink(t *testing.T) {
+	root := t.TempDir()
+	writeCLIFile(t, root, "target.md", "# Report\n")
+	if err := os.Symlink("target.md", filepath.Join(root, "link.md")); err != nil {
+		t.Skipf("symlink unavailable: %v", err)
+	}
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"manifest", "create", root, "--id", "reports.link", "--file", "link.md"}, &stdout, &stderr)
+
+	if code != exitError {
+		t.Fatalf("exit code = %d, want %d", code, exitError)
+	}
+	if !strings.Contains(stderr.String(), "regular file") {
+		t.Fatalf("stderr = %q, want regular file error", stderr.String())
+	}
+}
+
+func TestExecuteManifestCreateArgs(t *testing.T) {
+	root := t.TempDir()
+	writeCLIFile(t, root, "report.md", "# Report\n")
+	tests := []struct {
+		name       string
+		args       []string
+		wantCode   int
+		wantStderr string
+	}{
+		{
+			name:       "missing path",
+			args:       []string{"manifest", "create", "--id", "reports.missing"},
+			wantCode:   exitUsage,
+			wantStderr: "requires exactly one bundle path",
+		},
+		{
+			name:       "missing id",
+			args:       []string{"manifest", "create", root},
+			wantCode:   exitError,
+			wantStderr: "requires --id",
+		},
+		{
+			name:       "bad created",
+			args:       []string{"manifest", "create", root, "--id", "reports.bad", "--created", "June 1"},
+			wantCode:   exitError,
+			wantStderr: "created must be RFC3339",
+		},
+		{
+			name:       "bad format",
+			args:       []string{"manifest", "create", root, "--id", "reports.bad", "--format", "xml"},
+			wantCode:   exitUsage,
+			wantStderr: "format must be text or json",
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			var stdout, stderr bytes.Buffer
+			code := Execute(context.Background(), tt.args, &stdout, &stderr)
+			if code != tt.wantCode {
+				t.Fatalf("exit code = %d, want %d; stderr = %q", code, tt.wantCode, stderr.String())
+			}
+			if stdout.Len() != 0 {
+				t.Fatalf("stdout = %q, want empty", stdout.String())
+			}
+			if !strings.Contains(stderr.String(), tt.wantStderr) {
+				t.Fatalf("stderr = %q, want substring %q", stderr.String(), tt.wantStderr)
+			}
+		})
+	}
+}
+
 func TestExecuteRunDryRun(t *testing.T) {
 	sourceRoot := t.TempDir()
 	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{})
@@ -172,14 +632,323 @@ func TestExecuteRunDryRun(t *testing.T) {
 	if code != exitOK {
 		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
 	}
-	if !strings.Contains(stdout.String(), "action=publish_new") {
-		t.Fatalf("stdout = %q, want config summary", stdout.String())
+	wantStdout := "Configured pipelines: 1\n" +
+		"- pipeline=reports source=local bundles=1 destinations=archive\n" +
+		"  - bundle=. destination=archive backend=local action=publish_new outputs=report.md,summary.txt reason=\"destination state is absent\"\n" +
+		"Final status: ok planned=1 publish_new=1 replace_older=0 force_replace=0 skipped=0 failed=0 dry_run=true fixed_path=0\n"
+	if got := stdout.String(); got != wantStdout {
+		t.Fatalf("stdout = %q, want %q", got, wantStdout)
 	}
 	if stderr.Len() != 0 {
 		t.Fatalf("stderr = %q, want empty", stderr.String())
 	}
 }
 
+func TestExecuteRunJSONDryRun(t *testing.T) {
+	sourceRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{})
+	configPath := testutil.WriteMinimalLocalConfig(t, sourceRoot, t.TempDir())
+
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"run", "--config", configPath, "--dry-run", "--format", "json"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	envelope := decodeEnvelope(t, &stdout)
+	if envelope["command"] != "run" || envelope["ok"] != true {
+		t.Fatalf("envelope = %#v, want run ok", envelope)
+	}
+	result := envelopeResult(t, envelope)
+	if result["dry_run"] != true {
+		t.Fatalf("result = %#v, want dry_run true", result)
+	}
+	pipelines, ok := result["pipelines"].([]any)
+	if !ok || len(pipelines) != 1 {
+		t.Fatalf("pipelines = %#v, want one pipeline", result["pipelines"])
+	}
+	pipeline, ok := pipelines[0].(map[string]any)
+	if !ok || pipeline["id"] != "reports" || pipeline["source_backend"] != "local" || pipeline["bundle_count"] != float64(1) {
+		t.Fatalf("pipeline = %#v, want reports/local summary", pipelines[0])
+	}
+	actions, ok := result["actions"].([]any)
+	if !ok || len(actions) != 1 {
+		t.Fatalf("actions = %#v, want one action", result["actions"])
+	}
+	action, ok := actions[0].(map[string]any)
+	if !ok || action["action"] != "publish_new" {
+		t.Fatalf("action = %#v, want publish_new", actions[0])
+	}
+	outputs, ok := action["outputs"].([]any)
+	if !ok || len(outputs) != 2 {
+		t.Fatalf("outputs = %#v, want source outputs", action["outputs"])
+	}
+	summary, ok := result["summary"].(map[string]any)
+	if !ok || summary["status"] != "ok" || summary["planned"] != float64(1) || summary["publish_new"] != float64(1) || summary["dry_run"] != true {
+		t.Fatalf("summary = %#v, want ok dry-run publish counters", result["summary"])
+	}
+	if stderr.Len() != 0 {
+		t.Fatalf("stderr = %q, want empty", stderr.String())
+	}
+}
+
+func TestExecuteRunJSONDryRunReportsFixedPathMapping(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{})
+	configPath := filepath.Join(t.TempDir(), "config.yml")
+	if err := os.WriteFile(configPath, []byte(`
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: `+sourceRoot+`
+    destinations:
+      - id: latest
+        backend: local
+        path: `+destinationRoot+`
+        path_mapping:
+          mode: fixed
+`), 0o600); err != nil {
+		t.Fatalf("write config: %v", err)
+	}
+
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"run", "--config", configPath, "--dry-run", "--format", "json"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	envelope := decodeEnvelope(t, &stdout)
+	warnings, ok := envelope["warnings"].([]any)
+	if !ok || len(warnings) != 1 {
+		t.Fatalf("warnings = %#v, want one fixed-path warning", envelope["warnings"])
+	}
+	warning, ok := warnings[0].(map[string]any)
+	if !ok || !strings.Contains(fmt.Sprint(warning["message"]), "path_mapping=fixed candidates=1 selected_bundle=.") {
+		t.Fatalf("warning = %#v, want fixed-path selection warning", warnings[0])
+	}
+	result := envelopeResult(t, envelope)
+	actions, ok := result["actions"].([]any)
+	if !ok || len(actions) != 1 {
+		t.Fatalf("actions = %#v, want one action", result["actions"])
+	}
+	action, ok := actions[0].(map[string]any)
+	if !ok || action["path_mapping"] != "fixed" || action["destination_path"] != "." || action["action"] != "publish_new" {
+		t.Fatalf("action = %#v, want fixed publish_new at root", actions[0])
+	}
+	summary, ok := result["summary"].(map[string]any)
+	if !ok || summary["fixed_path"] != float64(1) {
+		t.Fatalf("summary = %#v, want fixed_path 1", result["summary"])
+	}
+	if stderr.Len() != 0 {
+		t.Fatalf("stderr = %q, want empty", stderr.String())
+	}
+}
+
+func TestExecuteRunJSONDryRunReportsLinks(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{})
+	configPath := filepath.Join(t.TempDir(), "config.yml")
+	if err := os.WriteFile(configPath, []byte(`
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: `+sourceRoot+`
+    destinations:
+      - id: web
+        backend: local
+        path: `+destinationRoot+`
+        links:
+          base_url: https://reports.example.com/archive
+          primary: source
+`), 0o600); err != nil {
+		t.Fatalf("write config: %v", err)
+	}
+
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"run", "--config", configPath, "--dry-run", "--format", "json"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	envelope := decodeEnvelope(t, &stdout)
+	result := envelopeResult(t, envelope)
+	actions, ok := result["actions"].([]any)
+	if !ok || len(actions) != 1 {
+		t.Fatalf("actions = %#v, want one action", result["actions"])
+	}
+	action, ok := actions[0].(map[string]any)
+	if !ok || action["primary_url"] != "https://reports.example.com/archive/report.md" {
+		t.Fatalf("action = %#v, want primary URL", actions[0])
+	}
+	outputs, ok := action["outputs"].([]any)
+	if !ok || len(outputs) != 2 {
+		t.Fatalf("outputs = %#v, want two outputs", action["outputs"])
+	}
+	output, ok := outputs[0].(map[string]any)
+	if !ok || output["url"] != "https://reports.example.com/archive/report.md" {
+		t.Fatalf("output = %#v, want output URL", outputs[0])
+	}
+	if stderr.Len() != 0 {
+		t.Fatalf("stderr = %q, want empty", stderr.String())
+	}
+}
+
+func TestExecuteRunJSONWarningsAreStructured(t *testing.T) {
+	name := "DISTRIBUTOR_TEST_CLI_JSON_SECRET"
+	t.Setenv(name, "process-value")
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	secretsRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{})
+	if err := os.WriteFile(filepath.Join(secretsRoot, name), []byte("secret-value\n"), 0o600); err != nil {
+		t.Fatalf("write secret: %v", err)
+	}
+	configPath := filepath.Join(t.TempDir(), "config.yml")
+	if err := os.WriteFile(configPath, []byte(`
+secrets:
+  directory: `+secretsRoot+`
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: `+sourceRoot+`
+    destinations:
+      - id: archive
+        backend: local
+        path: `+destinationRoot+`
+`), 0o600); err != nil {
+		t.Fatalf("write config: %v", err)
+	}
+
+	var stdout, stderr bytes.Buffer
+	code := Execute(context.Background(), []string{"run", "--config", configPath, "--dry-run", "--format", "json"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	envelope := decodeEnvelope(t, &stdout)
+	warnings, ok := envelope["warnings"].([]any)
+	if !ok || len(warnings) != 1 {
+		t.Fatalf("warnings = %#v, want one warning", envelope["warnings"])
+	}
+	warning, ok := warnings[0].(map[string]any)
+	if !ok || !strings.Contains(fmt.Sprint(warning["message"]), name) {
+		t.Fatalf("warning = %#v, want secret name", warnings[0])
+	}
+	if strings.Contains(stdout.String(), "Warning:") || strings.Contains(stdout.String(), "process-value") || strings.Contains(stdout.String(), "secret-value") {
+		t.Fatalf("stdout exposed text warning or secret values: %q", stdout.String())
+	}
+	if stderr.Len() != 0 {
+		t.Fatalf("stderr = %q, want empty", stderr.String())
+	}
+}
+
+func TestExecuteRunJSONFatalSetupErrorWritesNoJSON(t *testing.T) {
+	var stdout, stderr bytes.Buffer
+
+	code := Execute(context.Background(), []string{"run", "--config", filepath.Join(t.TempDir(), "missing.yml"), "--format", "json"}, &stdout, &stderr)
+
+	if code != exitError {
+		t.Fatalf("exit code = %d, want %d", code, exitError)
+	}
+	if stdout.Len() != 0 {
+		t.Fatalf("stdout = %q, want empty", stdout.String())
+	}
+	if !strings.Contains(stderr.String(), "no such file or directory") {
+		t.Fatalf("stderr = %q, want setup error", stderr.String())
+	}
+}
+
+func TestExecuteRunJSONPartialFailure(t *testing.T) {
+	sourceRoot := t.TempDir()
+	firstDestination := t.TempDir()
+	secondDestination := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{})
+	if err := os.WriteFile(filepath.Join(firstDestination, "unmanaged.txt"), []byte("data"), 0o600); err != nil {
+		t.Fatalf("write unmanaged file: %v", err)
+	}
+	configPath := filepath.Join(t.TempDir(), "config.yml")
+	if err := os.WriteFile(configPath, []byte(`
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: `+sourceRoot+`
+    destinations:
+      - id: archive-one
+        backend: local
+        path: `+firstDestination+`
+      - id: archive-two
+        backend: local
+        path: `+secondDestination+`
+`), 0o600); err != nil {
+		t.Fatalf("write config: %v", err)
+	}
+
+	var stdout, stderr bytes.Buffer
+	code := Execute(context.Background(), []string{"run", "--config", configPath, "--format", "json"}, &stdout, &stderr)
+
+	if code != exitError {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitError, stderr.String())
+	}
+	if stderr.Len() != 0 {
+		t.Fatalf("stderr = %q, want empty for partial JSON result", stderr.String())
+	}
+	envelope := decodeEnvelope(t, &stdout)
+	if envelope["command"] != "run" || envelope["ok"] != false {
+		t.Fatalf("envelope = %#v, want failed run envelope", envelope)
+	}
+	errors, ok := envelope["errors"].([]any)
+	if !ok || len(errors) != 1 {
+		t.Fatalf("errors = %#v, want one error", envelope["errors"])
+	}
+	result := envelopeResult(t, envelope)
+	summary, ok := result["summary"].(map[string]any)
+	if !ok || summary["status"] != "failed" || summary["failed"] != float64(1) {
+		t.Fatalf("summary = %#v, want failed summary", result["summary"])
+	}
+	actions, ok := result["actions"].([]any)
+	if !ok || len(actions) != 2 {
+		t.Fatalf("actions = %#v, want two actions", result["actions"])
+	}
+	if _, err := os.Stat(filepath.Join(secondDestination, storage.StateFileName)); err != nil {
+		t.Fatalf("second destination state stat error = %v", err)
+	}
+}
+
+func TestExecuteRunForceDryRunReportsWithoutWriting(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	testutil.WriteSourceBundle(t, sourceRoot, "", testutil.BundleOptions{})
+	if err := os.WriteFile(filepath.Join(destinationRoot, "unmanaged.txt"), []byte("old"), 0o600); err != nil {
+		t.Fatalf("write unmanaged file: %v", err)
+	}
+	configPath := testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot)
+
+	var stdout, stderr bytes.Buffer
+	code := Execute(context.Background(), []string{"run", "--config", configPath, "--force", "--dry-run"}, &stdout, &stderr)
+
+	if code != exitOK {
+		t.Fatalf("exit code = %d, want %d; stderr = %q", code, exitOK, stderr.String())
+	}
+	if !strings.Contains(stdout.String(), "action=force_replace") {
+		t.Fatalf("stdout = %q, want force_replace", stdout.String())
+	}
+	if _, err := os.Stat(filepath.Join(destinationRoot, "unmanaged.txt")); err != nil {
+		t.Fatalf("unmanaged file stat error = %v", err)
+	}
+	if _, err := os.Stat(filepath.Join(destinationRoot, storage.StateFileName)); !os.IsNotExist(err) {
+		t.Fatalf("state stat error = %v, want not exist", err)
+	}
+}
+
 func TestExecuteRunRejectsExtraPositionalArgs(t *testing.T) {
 	var stdout, stderr bytes.Buffer
 
@@ -223,3 +992,34 @@ func TestUnknownCommandIsUsageError(t *testing.T) {
 		t.Fatalf("stderr = %q, want unknown command error", stderr.String())
 	}
 }
+
+func manifestPaths(manifest producerbundle.Manifest) []string {
+	paths := make([]string, 0, len(manifest.Files))
+	for _, file := range manifest.Files {
+		paths = append(paths, file.Path)
+	}
+	return paths
+}
+
+func equalStrings(a, b []string) bool {
+	if len(a) != len(b) {
+		return false
+	}
+	for index := range a {
+		if a[index] != b[index] {
+			return false
+		}
+	}
+	return true
+}
+
+func writeCLIFile(t *testing.T, root, relative, body string) {
+	t.Helper()
+	path := filepath.Join(root, filepath.FromSlash(relative))
+	if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+		t.Fatalf("mkdir %s: %v", filepath.Dir(path), err)
+	}
+	if err := os.WriteFile(path, []byte(body), 0o600); err != nil {
+		t.Fatalf("write %s: %v", path, err)
+	}
+}

internal/cli/run.go

@@ -19,18 +19,29 @@ func runCommand(ctx context.Context, args []string, stdout, stderr io.Writer) in
 	flags.SetOutput(stderr)
 	configPath := flags.String("config", "", "path to config file")
 	dryRun := flags.Bool("dry-run", false, "load and validate config without publishing")
+	force := flags.Bool("force", false, "allow explicit destructive replacement for supported conflicts")
+	formatFlag := addFormatFlag(flags)
 	if err := flags.Parse(args); err != nil {
 		return exitUsage
 	}
 	if rejectPositionalArgs(stderr, "run", flags.Args()) {
 		return exitUsage
 	}
+	format, ok := parseOutputFormat(stderr, "run", *formatFlag)
+	if !ok {
+		return exitUsage
+	}
 
 	if err := app.Run(ctx, app.RunOptions{
-		ConfigPath: *configPath,
-		DryRun:     *dryRun,
-		Stdout:     stdout,
+		ConfigPath:   *configPath,
+		DryRun:       *dryRun,
+		Force:        *force,
+		Stdout:       stdout,
+		OutputFormat: format,
 	}); err != nil {
+		if app.IsJSONOutput(format) && app.IsPartialResultError(err) {
+			return exitError
+		}
 		return fail(stderr, err)
 	}
 	return exitOK
@@ -38,13 +49,16 @@ func runCommand(ctx context.Context, args []string, stdout, stderr io.Writer) in
 
 func printRunHelp(w io.Writer) {
 	fmt.Fprint(w, `Usage:
-  distributor run --config <path> --dry-run
+  distributor run --config <path> [--dry-run] [--force] [--format text|json]
 
 Options:
   --config <path>  Path to config file
   --dry-run        Load and validate config without publishing
+  --force          Allow explicit destructive replacement for supported conflicts
+  --format text|json
+                   Output format
 
-Run discovers local source bundles, plans each configured destination, publishes
+Run discovers configured source bundles, plans each destination, publishes
 selected outputs unless --dry-run is set, and prints a final status summary.
 `)
 }

internal/cli/serve.go

@@ -0,0 +1,46 @@
+package cli
+
+import (
+	"context"
+	"flag"
+	"fmt"
+	"io"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/app"
+)
+
+var serveApp = app.Serve
+
+func serveCommand(ctx context.Context, args []string, stdout, stderr io.Writer) int {
+	if hasHelp(args) {
+		printServeHelp(stdout)
+		return exitOK
+	}
+
+	flags := flag.NewFlagSet("serve", flag.ContinueOnError)
+	flags.SetOutput(stderr)
+	configPath := flags.String("config", "", "path to config file")
+	if err := flags.Parse(args); err != nil {
+		return exitUsage
+	}
+	if rejectPositionalArgs(stderr, "serve", flags.Args()) {
+		return exitUsage
+	}
+
+	if err := serveApp(ctx, app.ServeOptions{ConfigPath: *configPath}); err != nil {
+		return fail(stderr, err)
+	}
+	return exitOK
+}
+
+func printServeHelp(w io.Writer) {
+	fmt.Fprint(w, `Usage:
+  distributor serve --config <path>
+
+Options:
+  --config <path>  Path to config file
+
+Serve loads configured HTTP upload sources, resolves upload tokens through the
+configured secret environment, and starts the HTTP upload API.
+`)
+}

internal/cli/source_mode.go

@@ -0,0 +1,67 @@
+package cli
+
+import (
+	"flag"
+	"fmt"
+	"io"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/app"
+)
+
+type sourceDiagnosticArgs struct {
+	Path         string
+	ConfigPath   string
+	PipelineID   string
+	BundlePath   string
+	OutputFormat app.OutputFormat
+}
+
+func parseSourceDiagnosticArgs(stderr io.Writer, command string, args []string) (sourceDiagnosticArgs, bool) {
+	flags := flag.NewFlagSet(command, flag.ContinueOnError)
+	flags.SetOutput(stderr)
+	configPath := flags.String("config", "", "path to config file")
+	pipelineID := flags.String("pipeline", "", "pipeline id")
+	bundlePath := flags.String("bundle", "", "source-root-relative bundle path")
+	formatFlag := addFormatFlag(flags)
+	if err := flags.Parse(args); err != nil {
+		return sourceDiagnosticArgs{}, false
+	}
+	format, ok := parseOutputFormat(stderr, command, *formatFlag)
+	if !ok {
+		return sourceDiagnosticArgs{}, false
+	}
+	path, ok := parseOptionalPathArg(stderr, command, flags.Args())
+	if !ok {
+		return sourceDiagnosticArgs{}, false
+	}
+	if !validateInspectModeOK(stderr, command, path, *configPath, *pipelineID, *bundlePath) {
+		return sourceDiagnosticArgs{}, false
+	}
+	return sourceDiagnosticArgs{
+		Path:         path,
+		ConfigPath:   *configPath,
+		PipelineID:   *pipelineID,
+		BundlePath:   *bundlePath,
+		OutputFormat: format,
+	}, true
+}
+
+func validateInspectModeOK(stderr io.Writer, command, path, configPath, pipelineID, bundlePath string) bool {
+	configMode := configPath != "" || pipelineID != "" || bundlePath != ""
+	if !configMode {
+		return true
+	}
+	if path != "" {
+		fmt.Fprintf(stderr, "distributor: %s does not accept a local path with --config, --pipeline, or --bundle\n", command)
+		return false
+	}
+	if configPath == "" {
+		fmt.Fprintf(stderr, "distributor: %s requires --config when --pipeline or --bundle is set\n", command)
+		return false
+	}
+	if pipelineID == "" {
+		fmt.Fprintf(stderr, "distributor: %s requires --pipeline in config mode\n", command)
+		return false
+	}
+	return true
+}

internal/cli/validate.go

@@ -13,11 +13,18 @@ func validateCommand(ctx context.Context, args []string, stdout, stderr io.Write
 		printValidateHelp(stdout)
 		return exitOK
 	}
-	path, ok := parseOptionalPathArg(stderr, "validate", args)
+	parsed, ok := parseSourceDiagnosticArgs(stderr, "validate", args)
 	if !ok {
 		return exitUsage
 	}
-	if err := app.Validate(ctx, app.ValidateOptions{Path: path, Stdout: stdout}); err != nil {
+	if err := app.Validate(ctx, app.ValidateOptions{
+		Path:         parsed.Path,
+		ConfigPath:   parsed.ConfigPath,
+		PipelineID:   parsed.PipelineID,
+		BundlePath:   parsed.BundlePath,
+		Stdout:       stdout,
+		OutputFormat: parsed.OutputFormat,
+	}); err != nil {
 		return fail(stderr, err)
 	}
 	return exitOK
@@ -25,8 +32,16 @@ func validateCommand(ctx context.Context, args []string, stdout, stderr io.Write
 
 func printValidateHelp(w io.Writer) {
 	fmt.Fprint(w, `Usage:
-  distributor validate <path>
+  distributor validate [--format text|json] <path>
+  distributor validate --config <path> --pipeline <id> [--bundle <path>] [--format text|json]
 
-Validate a local source bundle directory or a tree containing source bundles.
+Options:
+  --config <path>     Path to config file for configured source validation
+  --pipeline <id>     Pipeline id to validate in config mode
+  --bundle <path>     Source-root-relative bundle path to validate
+  --format text|json  Output format
+
+Validate a local source bundle directory, a local source bundle tree, or a
+configured pipeline source.
 `)
 }

internal/cli/version.go

@@ -2,6 +2,7 @@ package cli
 
 import (
 	"context"
+	"flag"
 	"fmt"
 	"io"
 
@@ -13,16 +14,44 @@ func versionCommand(_ context.Context, args []string, stdout, stderr io.Writer)
 		printVersionHelp(stdout)
 		return exitOK
 	}
-	if rejectExtraArgs(stderr, "version", args) {
+	flags := flag.NewFlagSet("version", flag.ContinueOnError)
+	flags.SetOutput(stderr)
+	formatFlag := addFormatFlag(flags)
+	if err := flags.Parse(args); err != nil {
 		return exitUsage
 	}
+	if rejectExtraArgs(stderr, "version", flags.Args()) {
+		return exitUsage
+	}
+	format, ok := parseOutputFormat(stderr, "version", *formatFlag)
+	if !ok {
+		return exitUsage
+	}
+	if app.IsJSONOutput(format) {
+		err := app.WriteJSONEnvelope(stdout, "version", true, nil, versionResult{
+			Application: app.Name,
+			Version:     app.Version,
+		}, nil)
+		if err != nil {
+			return fail(stderr, err)
+		}
+		return exitOK
+	}
 	fmt.Fprintln(stdout, app.VersionString())
 	return exitOK
 }
 
+type versionResult struct {
+	Application string `json:"application"`
+	Version     string `json:"version"`
+}
+
 func printVersionHelp(w io.Writer) {
 	fmt.Fprint(w, `Usage:
-  distributor version
+  distributor version [--format text|json]
+
+Options:
+  --format text|json  Output format
 
 Print version information.
 `)

internal/config/config.go

@@ -1,9 +1,28 @@
 package config
 
 type Config struct {
+	Server    Server     `yaml:"server"`
+	Secrets   Secrets    `yaml:"secrets"`
 	Pipelines []Pipeline `yaml:"pipelines"`
 }
 
+type Server struct {
+	HTTP HTTPServer `yaml:"http"`
+}
+
+type HTTPServer struct {
+	Bind           string    `yaml:"bind"`
+	StagingRoot    string    `yaml:"staging_root"`
+	MaxUploadSize  *ByteSize `yaml:"max_upload_size"`
+	QueueSize      int       `yaml:"queue_size"`
+	MaxConcurrency int       `yaml:"max_concurrency"`
+	Retention      *Duration `yaml:"retention"`
+}
+
+type Secrets struct {
+	Directory string `yaml:"directory"`
+}
+
 type Pipeline struct {
 	ID           string           `yaml:"id"`
 	Source       Backend          `yaml:"source"`
@@ -14,29 +33,50 @@ type Pipeline struct {
 type Destination struct {
 	ID        string         `yaml:"id"`
 	Backend   string         `yaml:"backend"`
+	Host      string         `yaml:"host"`
+	User      string         `yaml:"user"`
+	Port      int            `yaml:"port"`
 	Path      string         `yaml:"path"`
-	URI       string         `yaml:"uri"`
 	Endpoint  string         `yaml:"endpoint"`
 	Bucket    string         `yaml:"bucket"`
 	Prefix    string         `yaml:"prefix"`
 	Region    string         `yaml:"region"`
-	ForcePath bool           `yaml:"force_path_style"`
+	ForcePath *bool          `yaml:"force_path_style"`
 	Creds     Credentials    `yaml:"credentials"`
+	SSH       SSH            `yaml:",inline"`
 	Publish   *PublishPolicy `yaml:"publish"`
 	Transform Transform      `yaml:"transform"`
+	PathMap   PathMapping    `yaml:"path_mapping"`
+	Links     *Links         `yaml:"links"`
 	Transfer  TransferPolicy `yaml:"transfer"`
 }
 
 type Backend struct {
 	Backend   string      `yaml:"backend"`
+	Host      string      `yaml:"host"`
+	User      string      `yaml:"user"`
+	Port      int         `yaml:"port"`
 	Path      string      `yaml:"path"`
-	URI       string      `yaml:"uri"`
 	Endpoint  string      `yaml:"endpoint"`
 	Bucket    string      `yaml:"bucket"`
 	Prefix    string      `yaml:"prefix"`
 	Region    string      `yaml:"region"`
-	ForcePath bool        `yaml:"force_path_style"`
+	ForcePath *bool       `yaml:"force_path_style"`
 	Creds     Credentials `yaml:"credentials"`
+	SSH       SSH         `yaml:",inline"`
+	Upload    HTTPUpload  `yaml:",inline"`
+}
+
+type HTTPUpload struct {
+	TokenEnv      string    `yaml:"token_env"`
+	StagingPath   string    `yaml:"staging_path"`
+	MaxUploadSize *ByteSize `yaml:"max_upload_size"`
+}
+
+type SSH struct {
+	KeyFile       string        `yaml:"ssh_key_file"`
+	KnownHosts    string        `yaml:"known_hosts"`
+	HostKeyPolicy HostKeyPolicy `yaml:"host_key_policy"`
 }
 
 type Credentials struct {
@@ -60,6 +100,16 @@ type Transform struct {
 type MarkdownToHTML struct {
 	Enabled bool   `yaml:"enabled"`
 	Mode    string `yaml:"mode"`
+	Input   string `yaml:"input"`
+}
+
+type PathMapping struct {
+	Mode string `yaml:"mode"`
+}
+
+type Links struct {
+	BaseURL string `yaml:"base_url"`
+	Primary string `yaml:"primary"`
 }
 
 type TransferPolicy struct {

internal/config/defaults.go

@@ -1,11 +1,19 @@
 package config
 
+import (
+	"path/filepath"
+	"time"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/transform"
+)
+
 const DefaultConfigPath = "/usr/local/etc/distributor/config.yml"
 
 const (
-	BackendLocal = "local"
-	BackendSSH   = "ssh"
-	BackendS3    = "s3"
+	BackendLocal      = "local"
+	BackendSSH        = "ssh"
+	BackendS3         = "s3"
+	BackendHTTPUpload = "http_upload"
 )
 
 const (
@@ -19,20 +27,58 @@ const (
 )
 
 const (
-	TransformModeSidecar = "sidecar"
+	TransformModeSidecar = transform.MarkdownModeSidecar
+	TransformModeIndex   = transform.MarkdownModeIndex
+)
+
+const (
+	PathMappingPreserveRelative = "preserve_relative"
+	PathMappingFixed            = "fixed"
+)
+
+const (
+	LinkPrimaryAuto   = "auto"
+	LinkPrimaryHTML   = "html"
+	LinkPrimarySource = "source"
+)
+
+const DefaultS3Region = "us-east-1"
+
+const (
+	DefaultHTTPBind           = "127.0.0.1:8080"
+	DefaultHTTPStagingRoot    = "/var/spool/distributor"
+	DefaultHTTPMaxUploadSize  = ByteSize(20 * 1024 * 1024)
+	DefaultHTTPQueueSize      = 16
+	DefaultHTTPMaxConcurrency = 1
+	DefaultHTTPRetention      = Duration(24 * time.Hour)
 )
 
 func ApplyDefaults(cfg *Config) {
+	applyHTTPServerDefaults(&cfg.Server.HTTP)
 	for pipelineIndex := range cfg.Pipelines {
 		pipeline := &cfg.Pipelines[pipelineIndex]
+		applyBackendDefaults(&pipeline.Source)
+		if pipeline.Source.Backend == BackendHTTPUpload {
+			applyHTTPUploadDefaults(&pipeline.Source.Upload, pipeline.ID, cfg.Server.HTTP)
+		}
 		if pipeline.Validation.OnDigestMismatch == "" {
 			pipeline.Validation.OnDigestMismatch = ValidationActionFail
 		}
 		for destinationIndex := range pipeline.Destinations {
 			destination := &pipeline.Destinations[destinationIndex]
+			applyDestinationDefaults(destination)
 			if destination.Publish == nil {
 				destination.Publish = &PublishPolicy{Source: true}
 			}
+			if destination.Transform.MarkdownToHTML != nil && destination.Transform.MarkdownToHTML.Mode == "" {
+				destination.Transform.MarkdownToHTML.Mode = TransformModeSidecar
+			}
+			if destination.PathMap.Mode == "" {
+				destination.PathMap.Mode = PathMappingPreserveRelative
+			}
+			if destination.Links != nil && destination.Links.Primary == "" {
+				destination.Links.Primary = LinkPrimaryAuto
+			}
 			if destination.Transfer.OnDestinationSame == "" {
 				destination.Transfer.OnDestinationSame = TransferActionSkip
 			}
@@ -48,3 +94,80 @@ func ApplyDefaults(cfg *Config) {
 		}
 	}
 }
+
+func applyHTTPServerDefaults(server *HTTPServer) {
+	if server.Bind == "" {
+		server.Bind = DefaultHTTPBind
+	}
+	if server.StagingRoot == "" {
+		server.StagingRoot = DefaultHTTPStagingRoot
+	}
+	if server.MaxUploadSize == nil {
+		server.MaxUploadSize = byteSize(DefaultHTTPMaxUploadSize)
+	}
+	if server.QueueSize == 0 {
+		server.QueueSize = DefaultHTTPQueueSize
+	}
+	if server.MaxConcurrency == 0 {
+		server.MaxConcurrency = DefaultHTTPMaxConcurrency
+	}
+	if server.Retention == nil {
+		server.Retention = duration(DefaultHTTPRetention)
+	}
+}
+
+func applyHTTPUploadDefaults(upload *HTTPUpload, pipelineID string, server HTTPServer) {
+	if upload.StagingPath == "" && pipelineID != "" {
+		upload.StagingPath = filepath.Join(server.StagingRoot, pipelineID)
+	}
+	if upload.MaxUploadSize == nil && server.MaxUploadSize != nil {
+		upload.MaxUploadSize = byteSize(*server.MaxUploadSize)
+	}
+}
+
+func byteSize(value ByteSize) *ByteSize {
+	return &value
+}
+
+func duration(value Duration) *Duration {
+	return &value
+}
+
+func applyBackendDefaults(backend *Backend) {
+	if backend.Backend == BackendSSH {
+		if backend.Port == 0 {
+			backend.Port = 22
+		}
+		if backend.SSH.HostKeyPolicy == "" {
+			backend.SSH.HostKeyPolicy = HostKeyPolicyAcceptNew
+		}
+	}
+	if backend.Backend == BackendS3 {
+		applyS3Defaults(&backend.Region, &backend.Prefix, &backend.ForcePath)
+	}
+}
+
+func applyDestinationDefaults(destination *Destination) {
+	if destination.Backend == BackendSSH {
+		if destination.Port == 0 {
+			destination.Port = 22
+		}
+		if destination.SSH.HostKeyPolicy == "" {
+			destination.SSH.HostKeyPolicy = HostKeyPolicyAcceptNew
+		}
+	}
+	if destination.Backend == BackendS3 {
+		applyS3Defaults(&destination.Region, &destination.Prefix, &destination.ForcePath)
+	}
+}
+
+func applyS3Defaults(region, prefix *string, forcePath **bool) {
+	if *region == "" {
+		*region = DefaultS3Region
+	}
+	*prefix = NormalizeS3Prefix(*prefix)
+	if *forcePath == nil {
+		defaultForcePath := true
+		*forcePath = &defaultForcePath
+	}
+}

internal/config/load_test.go

@@ -33,6 +33,29 @@ pipelines:
 	if got, want := destination.Transfer.OnDestinationOlder, TransferActionReplace; got != want {
 		t.Fatalf("transfer default = %q, want %q", got, want)
 	}
+	if cfg.Secrets.Directory != "" {
+		t.Fatalf("secrets.directory = %q, want empty", cfg.Secrets.Directory)
+	}
+}
+
+func TestLoadFileValidSecretsDirectoryConfig(t *testing.T) {
+	cfg := loadConfig(t, `
+secrets:
+  directory: /run/secrets/distributor
+pipelines:
+  - id: local-copy
+    source:
+      backend: local
+      path: /var/spool/reports
+    destinations:
+      - id: archive
+        backend: local
+        path: /srv/archive
+`)
+
+	if got, want := cfg.Secrets.Directory, "/run/secrets/distributor"; got != want {
+		t.Fatalf("secrets.directory = %q, want %q", got, want)
+	}
 }
 
 func TestLoadFileValidFanOutConfig(t *testing.T) {
@@ -53,7 +76,9 @@ pipelines:
           html: false
       - id: static-site
         backend: ssh
-        uri: ssh://deploy@example.com:22
+        host: example.com
+        user: deploy
+        port: 22
         path: /srv/www/reports
         publish:
           source: false
@@ -69,6 +94,237 @@ pipelines:
 	}
 }
 
+func TestLoadFileValidMarkdownIndexConfig(t *testing.T) {
+	cfg := loadConfig(t, `
+pipelines:
+  - id: static-site
+    source:
+      backend: local
+      path: /var/spool/reports
+    destinations:
+      - id: web
+        backend: local
+        path: /srv/www/reports
+        publish:
+          source: false
+          html: true
+        transform:
+          markdown_to_html:
+            enabled: true
+            mode: index
+            input: report.md
+`)
+
+	markdown := cfg.Pipelines[0].Destinations[0].Transform.MarkdownToHTML
+	if markdown == nil || markdown.Mode != TransformModeIndex || markdown.Input != "report.md" {
+		t.Fatalf("markdown config = %#v, want index input", markdown)
+	}
+}
+
+func TestLoadFileDefaultsMarkdownModeToSidecar(t *testing.T) {
+	cfg := loadConfig(t, `
+pipelines:
+  - id: static-site
+    source:
+      backend: local
+      path: /var/spool/reports
+    destinations:
+      - id: web
+        backend: local
+        path: /srv/www/reports
+        publish:
+          source: false
+          html: true
+        transform:
+          markdown_to_html:
+            enabled: true
+`)
+
+	markdown := cfg.Pipelines[0].Destinations[0].Transform.MarkdownToHTML
+	if markdown == nil || markdown.Mode != TransformModeSidecar {
+		t.Fatalf("markdown mode = %#v, want sidecar default", markdown)
+	}
+}
+
+func TestLoadFileDefaultsPathMappingToPreserveRelative(t *testing.T) {
+	cfg := loadConfig(t, `
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: /source
+    destinations:
+      - id: archive
+        backend: local
+        path: /destination
+`)
+
+	if got, want := cfg.Pipelines[0].Destinations[0].PathMap.Mode, PathMappingPreserveRelative; got != want {
+		t.Fatalf("path mapping mode = %q, want %q", got, want)
+	}
+}
+
+func TestLoadFileAcceptsFixedPathMapping(t *testing.T) {
+	cfg := loadConfig(t, `
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: /source
+    destinations:
+      - id: latest
+        backend: local
+        path: /destination/latest
+        path_mapping:
+          mode: fixed
+`)
+
+	if got, want := cfg.Pipelines[0].Destinations[0].PathMap.Mode, PathMappingFixed; got != want {
+		t.Fatalf("path mapping mode = %q, want %q", got, want)
+	}
+}
+
+func TestLoadFileDefaultsLinksPrimaryToAuto(t *testing.T) {
+	cfg := loadConfig(t, `
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: /source
+    destinations:
+      - id: web
+        backend: local
+        path: /destination
+        links:
+          base_url: https://reports.example.com/archive
+`)
+
+	links := cfg.Pipelines[0].Destinations[0].Links
+	if links == nil {
+		t.Fatal("links = nil, want config")
+	}
+	if links.BaseURL != "https://reports.example.com/archive" || links.Primary != LinkPrimaryAuto {
+		t.Fatalf("links = %#v, want base URL with auto primary", links)
+	}
+}
+
+func TestLoadFileDefaultsHTTPServerConfig(t *testing.T) {
+	cfg := loadConfig(t, `
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: /source
+    destinations:
+      - id: archive
+        backend: local
+        path: /archive
+`)
+
+	server := cfg.Server.HTTP
+	if got, want := server.Bind, DefaultHTTPBind; got != want {
+		t.Fatalf("server.http.bind = %q, want %q", got, want)
+	}
+	if got, want := server.StagingRoot, DefaultHTTPStagingRoot; got != want {
+		t.Fatalf("server.http.staging_root = %q, want %q", got, want)
+	}
+	if got, want := *server.MaxUploadSize, DefaultHTTPMaxUploadSize; got != want {
+		t.Fatalf("server.http.max_upload_size = %s, want %s", got, want)
+	}
+	if got, want := server.QueueSize, DefaultHTTPQueueSize; got != want {
+		t.Fatalf("server.http.queue_size = %d, want %d", got, want)
+	}
+	if got, want := server.MaxConcurrency, DefaultHTTPMaxConcurrency; got != want {
+		t.Fatalf("server.http.max_concurrency = %d, want %d", got, want)
+	}
+	if got, want := *server.Retention, DefaultHTTPRetention; got != want {
+		t.Fatalf("server.http.retention = %s, want %s", got, want)
+	}
+}
+
+func TestLoadFileAcceptsHTTPUploadSourceConfig(t *testing.T) {
+	cfg := loadConfig(t, `
+server:
+  http:
+    bind: 127.0.0.1:9090
+    staging_root: /srv/distributor/staging
+    max_upload_size: 64MB
+    queue_size: 32
+    max_concurrency: 2
+    retention: 48h
+pipelines:
+  - id: weather-daily
+    source:
+      backend: http_upload
+      token_env: WEATHER_DAILY_UPLOAD_TOKEN
+      staging_path: /srv/distributor/staging/weather-daily
+      max_upload_size: 32MB
+    destinations:
+      - id: archive
+        backend: local
+        path: /archive
+`)
+
+	server := cfg.Server.HTTP
+	if got, want := server.Bind, "127.0.0.1:9090"; got != want {
+		t.Fatalf("server.http.bind = %q, want %q", got, want)
+	}
+	if got, want := server.StagingRoot, "/srv/distributor/staging"; got != want {
+		t.Fatalf("server.http.staging_root = %q, want %q", got, want)
+	}
+	if got, want := *server.MaxUploadSize, ByteSize(64*1024*1024); got != want {
+		t.Fatalf("server.http.max_upload_size = %s, want %s", got, want)
+	}
+	if got, want := server.QueueSize, 32; got != want {
+		t.Fatalf("server.http.queue_size = %d, want %d", got, want)
+	}
+	if got, want := server.MaxConcurrency, 2; got != want {
+		t.Fatalf("server.http.max_concurrency = %d, want %d", got, want)
+	}
+	if got, want := server.Retention.String(), "48h0m0s"; got != want {
+		t.Fatalf("server.http.retention = %s, want %s", got, want)
+	}
+
+	source := cfg.Pipelines[0].Source
+	if got, want := source.Backend, BackendHTTPUpload; got != want {
+		t.Fatalf("source.backend = %q, want %q", got, want)
+	}
+	if got, want := source.Upload.TokenEnv, "WEATHER_DAILY_UPLOAD_TOKEN"; got != want {
+		t.Fatalf("source.token_env = %q, want %q", got, want)
+	}
+	if got, want := source.Upload.StagingPath, "/srv/distributor/staging/weather-daily"; got != want {
+		t.Fatalf("source.staging_path = %q, want %q", got, want)
+	}
+	if got, want := *source.Upload.MaxUploadSize, ByteSize(32*1024*1024); got != want {
+		t.Fatalf("source.max_upload_size = %s, want %s", got, want)
+	}
+}
+
+func TestLoadFileDefaultsHTTPUploadSourceConfig(t *testing.T) {
+	cfg := loadConfig(t, `
+server:
+  http:
+    max_upload_size: 12MB
+pipelines:
+  - id: weather-daily
+    source:
+      backend: http_upload
+      token_env: WEATHER_DAILY_UPLOAD_TOKEN
+    destinations:
+      - id: archive
+        backend: local
+        path: /archive
+`)
+
+	source := cfg.Pipelines[0].Source
+	if got, want := source.Upload.StagingPath, "/var/spool/distributor/weather-daily"; got != want {
+		t.Fatalf("source.staging_path = %q, want %q", got, want)
+	}
+	if got, want := *source.Upload.MaxUploadSize, ByteSize(12*1024*1024); got != want {
+		t.Fatalf("source.max_upload_size = %s, want %s", got, want)
+	}
+}
+
 func TestLoadFileValidBackendConfigs(t *testing.T) {
 	tests := map[string]string{
 		"local": `
@@ -87,13 +343,19 @@ pipelines:
   - id: ssh-backend
     source:
       backend: ssh
-      uri: ssh://reports@example.com:22
+      host: source.example.com
+      user: reports
       path: /source
     destinations:
       - id: ssh-destination
         backend: ssh
-        uri: ssh://deploy@example.com:22
+        host: destination.example.com
+        user: deploy
+        port: 2222
         path: /destination
+        ssh_key_file: /home/deploy/.ssh/id_ed25519
+        known_hosts: /home/deploy/.ssh/known_hosts
+        host_key_policy: strict
 `,
 		"s3": `
 pipelines:
@@ -124,6 +386,61 @@ pipelines:
 	}
 }
 
+func TestLoadFileDefaultsS3Config(t *testing.T) {
+	cfg := loadConfig(t, `
+pipelines:
+  - id: s3-defaults
+    source:
+      backend: s3
+      endpoint: http://127.0.0.1:9000
+      bucket: source
+      prefix: /incoming/reports/
+    destinations:
+      - id: archive
+        backend: s3
+        endpoint: http://127.0.0.1:9000
+        bucket: destination
+`)
+
+	source := cfg.Pipelines[0].Source
+	if got, want := source.Region, DefaultS3Region; got != want {
+		t.Fatalf("source region = %q, want %q", got, want)
+	}
+	if got, want := source.Prefix, "incoming/reports"; got != want {
+		t.Fatalf("source prefix = %q, want %q", got, want)
+	}
+	if !ForcePathStyle(source.ForcePath) {
+		t.Fatal("source force_path_style = false, want true")
+	}
+	destination := cfg.Pipelines[0].Destinations[0]
+	if got, want := destination.Region, DefaultS3Region; got != want {
+		t.Fatalf("destination region = %q, want %q", got, want)
+	}
+	if !ForcePathStyle(destination.ForcePath) {
+		t.Fatal("destination force_path_style = false, want true")
+	}
+}
+
+func TestLoadFilePreservesExplicitS3ForcePathStyleFalse(t *testing.T) {
+	cfg := loadConfig(t, `
+pipelines:
+  - id: s3-force-path
+    source:
+      backend: s3
+      endpoint: https://s3.example.com
+      bucket: source
+      force_path_style: false
+    destinations:
+      - id: archive
+        backend: local
+        path: /archive
+`)
+
+	if ForcePathStyle(cfg.Pipelines[0].Source.ForcePath) {
+		t.Fatal("force_path_style = true, want explicit false")
+	}
+}
+
 func TestLoadFileRejectsDuplicatePipelineIDs(t *testing.T) {
 	assertLoadError(t, `
 pipelines:
@@ -171,7 +488,7 @@ func TestLoadFileRejectsMissingRequiredFields(t *testing.T) {
 		"destinations":    `pipelines: [{id: reports, source: {backend: local, path: /source}}]`,
 		"destination id":  `pipelines: [{id: reports, source: {backend: local, path: /source}, destinations: [{backend: local, path: /archive}]}]`,
 		"local path":      `pipelines: [{id: reports, source: {backend: local}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
-		"ssh uri":         `pipelines: [{id: reports, source: {backend: ssh, path: /source}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
+		"ssh host":        `pipelines: [{id: reports, source: {backend: ssh, path: /source}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
 		"s3 bucket":       `pipelines: [{id: reports, source: {backend: s3, endpoint: "https://s3.example.com"}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
 		"publish outputs": `pipelines: [{id: reports, source: {backend: local, path: /source}, destinations: [{id: archive, backend: local, path: /archive, publish: {source: false, html: false}}]}]`,
 	}
@@ -183,6 +500,121 @@ func TestLoadFileRejectsMissingRequiredFields(t *testing.T) {
 	}
 }
 
+func TestLoadFileRejectsInvalidS3Config(t *testing.T) {
+	tests := map[string]string{
+		"prefix traversal": `pipelines: [{id: reports, source: {backend: s3, endpoint: "https://s3.example.com", bucket: source, prefix: "../reports"}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
+		"prefix backslash": `pipelines: [{id: reports, source: {backend: s3, endpoint: "https://s3.example.com", bucket: source, prefix: 'a\b'}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
+		"partial creds":    `pipelines: [{id: reports, source: {backend: s3, endpoint: "https://s3.example.com", bucket: source, credentials: {access_key_id_env: ACCESS_KEY_ID}}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
+	}
+	for name, body := range tests {
+		t.Run(name, func(t *testing.T) {
+			assertLoadError(t, body, "")
+		})
+	}
+}
+
+func TestLoadFileRejectsInvalidHTTPUploadConfig(t *testing.T) {
+	tests := map[string]string{
+		"server size":             `server: {http: {max_upload_size: 20XB}}`,
+		"source size":             `pipelines: [{id: reports, source: {backend: http_upload, token_env: UPLOAD_TOKEN, max_upload_size: 20XB}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
+		"zero source size":        `pipelines: [{id: reports, source: {backend: http_upload, token_env: UPLOAD_TOKEN, max_upload_size: 0B}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
+		"server duration":         `server: {http: {retention: forever}}`,
+		"zero server duration":    `server: {http: {retention: 0s}}`,
+		"missing token env":       `pipelines: [{id: reports, source: {backend: http_upload}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
+		"destination http upload": `pipelines: [{id: reports, source: {backend: local, path: /source}, destinations: [{id: ingest, backend: http_upload}]}]`,
+		"literal token":           `pipelines: [{id: reports, source: {backend: http_upload, token: secret, token_env: UPLOAD_TOKEN}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
+		"unknown server field":    `server: {http: {surprise: true}}`,
+		"unknown source field":    `pipelines: [{id: reports, source: {backend: http_upload, token_env: UPLOAD_TOKEN, surprise: true}, destinations: [{id: archive, backend: local, path: /archive}]}]`,
+	}
+	for name, body := range tests {
+		t.Run(name, func(t *testing.T) {
+			assertLoadError(t, body, "")
+		})
+	}
+}
+
+func TestLoadFileDefaultsSSHConfig(t *testing.T) {
+	cfg := loadConfig(t, `
+pipelines:
+  - id: ssh-defaults
+    source:
+      backend: ssh
+      host: source.example.com
+      path: /source
+    destinations:
+      - id: archive
+        backend: ssh
+        host: destination.example.com
+        path: /archive
+        host_key_policy: false
+`)
+
+	source := cfg.Pipelines[0].Source
+	if source.Port != 22 {
+		t.Fatalf("source port = %d, want 22", source.Port)
+	}
+	if source.SSH.HostKeyPolicy != HostKeyPolicyAcceptNew {
+		t.Fatalf("source host key policy = %q, want accept-new", source.SSH.HostKeyPolicy)
+	}
+	destination := cfg.Pipelines[0].Destinations[0]
+	if destination.Port != 22 {
+		t.Fatalf("destination port = %d, want 22", destination.Port)
+	}
+	if destination.SSH.HostKeyPolicy != HostKeyPolicyOff {
+		t.Fatalf("destination host key policy = %q, want off", destination.SSH.HostKeyPolicy)
+	}
+}
+
+func TestLoadFileNormalizesSSHHostKeyPolicies(t *testing.T) {
+	tests := map[string]HostKeyPolicy{
+		`true`:         HostKeyPolicyStrict,
+		`"true"`:       HostKeyPolicyStrict,
+		`strict`:       HostKeyPolicyStrict,
+		`accept-new`:   HostKeyPolicyAcceptNew,
+		`false`:        HostKeyPolicyOff,
+		`"false"`:      HostKeyPolicyOff,
+		`off`:          HostKeyPolicyOff,
+		`"STRICT"`:     HostKeyPolicyStrict,
+		`"ACCEPT-NEW"`: HostKeyPolicyAcceptNew,
+		`"OFF"`:        HostKeyPolicyOff,
+	}
+	for value, want := range tests {
+		t.Run(value, func(t *testing.T) {
+			cfg := loadConfig(t, `
+pipelines:
+  - id: ssh-policy
+    source:
+      backend: ssh
+      host: source.example.com
+      path: /source
+      host_key_policy: `+value+`
+    destinations:
+      - id: archive
+        backend: local
+        path: /archive
+`)
+			if got := cfg.Pipelines[0].Source.SSH.HostKeyPolicy; got != want {
+				t.Fatalf("host key policy = %q, want %q", got, want)
+			}
+		})
+	}
+}
+
+func TestLoadFileRejectsLegacySSHURIFieldAsUnknown(t *testing.T) {
+	assertLoadError(t, `
+pipelines:
+  - id: reports
+    source:
+      backend: ssh
+      uri: ssh://reports@example.com:22
+      path: /source
+    destinations:
+      - id: archive
+        backend: local
+        path: /archive
+`, "field uri not found")
+}
+
 func TestLoadFileRejectsUnsupportedBackend(t *testing.T) {
 	assertLoadError(t, `
 pipelines:
@@ -261,12 +693,34 @@ pipelines:
 `, "field surprise not found")
 }
 
+func TestLoadFileRejectsUnknownSecretsFields(t *testing.T) {
+	assertLoadError(t, `
+secrets:
+  directory: /run/secrets/distributor
+  surprise: true
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: /source
+    destinations:
+      - id: archive
+        backend: local
+        path: /archive
+`, "field surprise not found")
+}
+
 func TestExampleConfigsLoad(t *testing.T) {
 	for _, path := range []string{
 		"../../examples/local-to-local.yml",
 		"../../examples/local-publish.yml",
 		"../../examples/local-html.yml",
+		"../../examples/local-index.yml",
 		"../../examples/fan-out.yml",
+		"../../examples/archive-and-latest.yml",
+		"../../examples/http-upload-local.yml",
+		"../../examples/ssh-destination.yml",
+		"../../examples/s3-destination.yml",
 	} {
 		t.Run(path, func(t *testing.T) {
 			if _, err := LoadFile(path); err != nil {

internal/config/quantity.go

@@ -0,0 +1,117 @@
+package config
+
+import (
+	"fmt"
+	"strconv"
+	"strings"
+	"time"
+
+	"gopkg.in/yaml.v3"
+)
+
+type ByteSize int64
+
+type Duration time.Duration
+
+func (size *ByteSize) UnmarshalYAML(value *yaml.Node) error {
+	if value.Kind != yaml.ScalarNode || value.Tag != "!!str" {
+		return fmt.Errorf("size must be a string with B, KB, MB, or GB suffix")
+	}
+
+	var raw string
+	if err := value.Decode(&raw); err != nil {
+		return err
+	}
+	parsed, err := ParseByteSize(raw)
+	if err != nil {
+		return err
+	}
+	*size = parsed
+	return nil
+}
+
+func (size ByteSize) String() string {
+	value := int64(size)
+	if value == 0 {
+		return "0B"
+	}
+	units := []struct {
+		suffix     string
+		multiplier int64
+	}{
+		{suffix: "GB", multiplier: 1024 * 1024 * 1024},
+		{suffix: "MB", multiplier: 1024 * 1024},
+		{suffix: "KB", multiplier: 1024},
+		{suffix: "B", multiplier: 1},
+	}
+	for _, unit := range units {
+		if value%unit.multiplier == 0 {
+			return strconv.FormatInt(value/unit.multiplier, 10) + unit.suffix
+		}
+	}
+	return strconv.FormatInt(value, 10) + "B"
+}
+
+func ParseByteSize(raw string) (ByteSize, error) {
+	value := strings.TrimSpace(raw)
+	if value == "" {
+		return 0, fmt.Errorf("size is required")
+	}
+
+	units := []struct {
+		suffix     string
+		multiplier int64
+	}{
+		{suffix: "GB", multiplier: 1024 * 1024 * 1024},
+		{suffix: "MB", multiplier: 1024 * 1024},
+		{suffix: "KB", multiplier: 1024},
+		{suffix: "B", multiplier: 1},
+	}
+	for _, unit := range units {
+		number, ok := strings.CutSuffix(value, unit.suffix)
+		if !ok {
+			continue
+		}
+		if strings.TrimSpace(number) != number || number == "" {
+			return 0, fmt.Errorf("size must be an integer followed by B, KB, MB, or GB")
+		}
+		parsed, err := strconv.ParseInt(number, 10, 64)
+		if err != nil {
+			return 0, fmt.Errorf("size must be an integer followed by B, KB, MB, or GB")
+		}
+		if parsed < 0 {
+			return 0, fmt.Errorf("size must be non-negative")
+		}
+		const maxInt64 = int64(1<<63 - 1)
+		if parsed > 0 && parsed > maxInt64/unit.multiplier {
+			return 0, fmt.Errorf("size is too large")
+		}
+		return ByteSize(parsed * unit.multiplier), nil
+	}
+	return 0, fmt.Errorf("size must use B, KB, MB, or GB suffix")
+}
+
+func (duration *Duration) UnmarshalYAML(value *yaml.Node) error {
+	if value.Kind != yaml.ScalarNode || value.Tag != "!!str" {
+		return fmt.Errorf("duration must be a string duration")
+	}
+
+	var raw string
+	if err := value.Decode(&raw); err != nil {
+		return err
+	}
+	parsed, err := time.ParseDuration(raw)
+	if err != nil {
+		return fmt.Errorf("duration must be a valid duration: %w", err)
+	}
+	*duration = Duration(parsed)
+	return nil
+}
+
+func (duration Duration) String() string {
+	return time.Duration(duration).String()
+}
+
+func (duration Duration) AsDuration() time.Duration {
+	return time.Duration(duration)
+}

internal/config/s3.go

@@ -0,0 +1,19 @@
+package config
+
+import (
+	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+)
+
+func NormalizeS3Prefix(prefix string) string {
+	return strings.Trim(prefix, "/")
+}
+
+func ValidateS3Prefix(prefix string) error {
+	return storage.ValidatePrefix(prefix)
+}
+
+func ForcePathStyle(value *bool) bool {
+	return value == nil || *value
+}

internal/config/secrets.go

@@ -0,0 +1,154 @@
+package config
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"regexp"
+	"sort"
+	"strings"
+)
+
+var secretNamePattern = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
+
+type EnvLookup func(string) (string, bool)
+
+type Environment struct {
+	values map[string]string
+	lookup EnvLookup
+}
+
+type SecretConflict struct {
+	Name string
+}
+
+type SecretLoadResult struct {
+	Environment Environment
+	Conflicts   []SecretConflict
+}
+
+type ResolvedCredentials struct {
+	AccessKeyID     string
+	SecretAccessKey string
+}
+
+func ProcessEnvironment() Environment {
+	return NewEnvironment(nil, os.LookupEnv)
+}
+
+func NewEnvironment(values map[string]string, lookup EnvLookup) Environment {
+	copied := make(map[string]string, len(values))
+	for key, value := range values {
+		copied[key] = value
+	}
+	if lookup == nil {
+		lookup = os.LookupEnv
+	}
+	return Environment{values: copied, lookup: lookup}
+}
+
+func (e Environment) Lookup(name string) (string, bool) {
+	if e.lookup != nil {
+		if value, ok := e.lookup(name); ok {
+			return value, true
+		}
+	}
+	value, ok := e.values[name]
+	return value, ok
+}
+
+func (e Environment) ResolveCredentials(creds Credentials) (ResolvedCredentials, error) {
+	var resolved ResolvedCredentials
+	var err error
+	if creds.AccessKeyIDEnv != "" {
+		resolved.AccessKeyID, err = e.required(creds.AccessKeyIDEnv)
+		if err != nil {
+			return ResolvedCredentials{}, err
+		}
+	}
+	if creds.SecretAccessKeyEnv != "" {
+		resolved.SecretAccessKey, err = e.required(creds.SecretAccessKeyEnv)
+		if err != nil {
+			return ResolvedCredentials{}, err
+		}
+	}
+	return resolved, nil
+}
+
+func (e Environment) required(name string) (string, error) {
+	value, ok := e.Lookup(name)
+	if !ok {
+		return "", fmt.Errorf("credential environment variable %s is not set", name)
+	}
+	if value == "" {
+		return "", fmt.Errorf("credential environment variable %s is empty", name)
+	}
+	return value, nil
+}
+
+func LoadSecretEnvironment(directory string, lookup EnvLookup) (SecretLoadResult, error) {
+	if directory == "" {
+		return SecretLoadResult{Environment: NewEnvironment(nil, lookup)}, nil
+	}
+	values, err := loadSecretValues(directory)
+	if err != nil {
+		return SecretLoadResult{}, err
+	}
+	var conflicts []SecretConflict
+	for name, value := range values {
+		if processValue, ok := lookupValue(lookup, name); ok && processValue != value {
+			conflicts = append(conflicts, SecretConflict{Name: name})
+		}
+	}
+	sort.Slice(conflicts, func(i, j int) bool {
+		return conflicts[i].Name < conflicts[j].Name
+	})
+	return SecretLoadResult{
+		Environment: NewEnvironment(values, lookup),
+		Conflicts:   conflicts,
+	}, nil
+}
+
+func loadSecretValues(directory string) (map[string]string, error) {
+	entries, err := os.ReadDir(directory)
+	if err != nil {
+		return nil, fmt.Errorf("load secrets directory %q: %w", directory, err)
+	}
+	values := make(map[string]string)
+	for _, entry := range entries {
+		path := filepath.Join(directory, entry.Name())
+		info, err := os.Stat(path)
+		if err != nil {
+			return nil, fmt.Errorf("inspect secret file %q: %w", entry.Name(), err)
+		}
+		if info.IsDir() || !info.Mode().IsRegular() {
+			continue
+		}
+		if !secretNamePattern.MatchString(entry.Name()) {
+			return nil, fmt.Errorf("secret filename %q is invalid", entry.Name())
+		}
+		data, err := os.ReadFile(path)
+		if err != nil {
+			return nil, fmt.Errorf("read secret file %q: %w", entry.Name(), err)
+		}
+		values[entry.Name()] = trimOneTrailingLineEnding(string(data))
+	}
+	return values, nil
+}
+
+func trimOneTrailingLineEnding(value string) string {
+	if strings.HasSuffix(value, "\r\n") {
+		return strings.TrimSuffix(value, "\r\n")
+	}
+	if strings.HasSuffix(value, "\n") {
+		return strings.TrimSuffix(value, "\n")
+	}
+	return value
+}
+
+func lookupValue(lookup EnvLookup, name string) (string, bool) {
+	if lookup == nil {
+		lookup = os.LookupEnv
+	}
+	return lookup(name)
+}

internal/config/secrets_test.go

@@ -0,0 +1,248 @@
+package config
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+func TestLoadSecretEnvironmentLoadsValidFiles(t *testing.T) {
+	directory := t.TempDir()
+	writeSecret(t, directory, "API_KEY", "value\n")
+	writeSecret(t, directory, "CRLF", "value\r\n")
+	writeSecret(t, directory, "MULTILINE", "value\n\n")
+	writeSecret(t, directory, "SPACES", "  value  \n")
+	writeSecret(t, directory, "CARRIAGE", "value\r")
+
+	result, err := LoadSecretEnvironment(directory, emptyLookup)
+	if err != nil {
+		t.Fatalf("LoadSecretEnvironment() error = %v", err)
+	}
+	assertEnvValue(t, result.Environment, "API_KEY", "value")
+	assertEnvValue(t, result.Environment, "CRLF", "value")
+	assertEnvValue(t, result.Environment, "MULTILINE", "value\n")
+	assertEnvValue(t, result.Environment, "SPACES", "  value  ")
+	assertEnvValue(t, result.Environment, "CARRIAGE", "value\r")
+}
+
+func TestLoadSecretEnvironmentRejectsInvalidFilenames(t *testing.T) {
+	directory := t.TempDir()
+	secretValue := "do-not-print"
+	writeSecret(t, directory, "1INVALID", secretValue)
+
+	_, err := LoadSecretEnvironment(directory, emptyLookup)
+	if err == nil {
+		t.Fatal("LoadSecretEnvironment() error = nil, want error")
+	}
+	if !strings.Contains(err.Error(), "secret filename") {
+		t.Fatalf("LoadSecretEnvironment() error = %q, want filename error", err)
+	}
+	if strings.Contains(err.Error(), secretValue) {
+		t.Fatalf("LoadSecretEnvironment() error exposed secret value: %q", err)
+	}
+}
+
+func TestLoadSecretEnvironmentIgnoresDirectoriesAndFollowsSymlinks(t *testing.T) {
+	directory := t.TempDir()
+	if err := os.Mkdir(filepath.Join(directory, "IGNORED_DIR"), 0o700); err != nil {
+		t.Fatalf("mkdir ignored dir: %v", err)
+	}
+	targetFile := filepath.Join(t.TempDir(), "target")
+	if err := os.WriteFile(targetFile, []byte("linked\n"), 0o600); err != nil {
+		t.Fatalf("write target file: %v", err)
+	}
+	if err := os.Symlink(targetFile, filepath.Join(directory, "LINKED_SECRET")); err != nil {
+		t.Fatalf("symlink file: %v", err)
+	}
+	targetDir := t.TempDir()
+	if err := os.Symlink(targetDir, filepath.Join(directory, "LINKED_DIR")); err != nil {
+		t.Fatalf("symlink dir: %v", err)
+	}
+
+	result, err := LoadSecretEnvironment(directory, emptyLookup)
+	if err != nil {
+		t.Fatalf("LoadSecretEnvironment() error = %v", err)
+	}
+	assertEnvValue(t, result.Environment, "LINKED_SECRET", "linked")
+	if _, ok := result.Environment.Lookup("IGNORED_DIR"); ok {
+		t.Fatal("directory appeared in environment")
+	}
+	if _, ok := result.Environment.Lookup("LINKED_DIR"); ok {
+		t.Fatal("directory symlink appeared in environment")
+	}
+}
+
+func TestLoadSecretEnvironmentMissingDirectoryFails(t *testing.T) {
+	missing := filepath.Join(t.TempDir(), "missing")
+	_, err := LoadSecretEnvironment(missing, emptyLookup)
+	if err == nil {
+		t.Fatal("LoadSecretEnvironment() error = nil, want error")
+	}
+	if !strings.Contains(err.Error(), "load secrets directory") || !strings.Contains(err.Error(), missing) {
+		t.Fatalf("LoadSecretEnvironment() error = %q, want directory context", err)
+	}
+}
+
+func TestEnvironmentPrefersProcessValuesAndReportsDifferingConflicts(t *testing.T) {
+	directory := t.TempDir()
+	secretValue := "secret-value"
+	processValue := "process-value"
+	writeSecret(t, directory, "TOKEN", secretValue)
+
+	result, err := LoadSecretEnvironment(directory, mapLookup(map[string]string{"TOKEN": processValue}))
+	if err != nil {
+		t.Fatalf("LoadSecretEnvironment() error = %v", err)
+	}
+	assertEnvValue(t, result.Environment, "TOKEN", processValue)
+	if got, want := len(result.Conflicts), 1; got != want {
+		t.Fatalf("conflict count = %d, want %d", got, want)
+	}
+	if result.Conflicts[0].Name != "TOKEN" {
+		t.Fatalf("conflict name = %q, want TOKEN", result.Conflicts[0].Name)
+	}
+	if strings.Contains(result.Conflicts[0].Name, secretValue) || strings.Contains(result.Conflicts[0].Name, processValue) {
+		t.Fatalf("conflict exposed secret values: %#v", result.Conflicts[0])
+	}
+}
+
+func TestEnvironmentDoesNotWarnWhenProcessValueMatchesSecret(t *testing.T) {
+	directory := t.TempDir()
+	writeSecret(t, directory, "TOKEN", "same-value")
+
+	result, err := LoadSecretEnvironment(directory, mapLookup(map[string]string{"TOKEN": "same-value"}))
+	if err != nil {
+		t.Fatalf("LoadSecretEnvironment() error = %v", err)
+	}
+	if len(result.Conflicts) != 0 {
+		t.Fatalf("conflicts = %#v, want none", result.Conflicts)
+	}
+}
+
+func TestEnvironmentDoesNotMutateProcessEnvironment(t *testing.T) {
+	name := "DISTRIBUTOR_TEST_SECRET_ONLY"
+	t.Setenv(name, "")
+	if err := os.Unsetenv(name); err != nil {
+		t.Fatalf("unset env: %v", err)
+	}
+	directory := t.TempDir()
+	writeSecret(t, directory, name, "secret")
+
+	if _, err := LoadSecretEnvironment(directory, nil); err != nil {
+		t.Fatalf("LoadSecretEnvironment() error = %v", err)
+	}
+	if _, ok := os.LookupEnv(name); ok {
+		t.Fatalf("%s was added to process environment", name)
+	}
+}
+
+func TestResolveCredentialsUsesSecretsAwareEnvironment(t *testing.T) {
+	directory := t.TempDir()
+	writeSecret(t, directory, "ACCESS_KEY_ID", "secret-access")
+	writeSecret(t, directory, "SECRET_ACCESS_KEY", "secret-secret")
+	result, err := LoadSecretEnvironment(directory, emptyLookup)
+	if err != nil {
+		t.Fatalf("LoadSecretEnvironment() error = %v", err)
+	}
+
+	creds, err := result.Environment.ResolveCredentials(Credentials{
+		AccessKeyIDEnv:     "ACCESS_KEY_ID",
+		SecretAccessKeyEnv: "SECRET_ACCESS_KEY",
+	})
+	if err != nil {
+		t.Fatalf("ResolveCredentials() error = %v", err)
+	}
+	if creds.AccessKeyID != "secret-access" || creds.SecretAccessKey != "secret-secret" {
+		t.Fatalf("resolved credentials = %#v", creds)
+	}
+}
+
+func TestResolveCredentialsPrefersProcessEnvironment(t *testing.T) {
+	directory := t.TempDir()
+	writeSecret(t, directory, "ACCESS_KEY_ID", "secret-access")
+	result, err := LoadSecretEnvironment(directory, mapLookup(map[string]string{"ACCESS_KEY_ID": "process-access"}))
+	if err != nil {
+		t.Fatalf("LoadSecretEnvironment() error = %v", err)
+	}
+
+	creds, err := result.Environment.ResolveCredentials(Credentials{AccessKeyIDEnv: "ACCESS_KEY_ID"})
+	if err != nil {
+		t.Fatalf("ResolveCredentials() error = %v", err)
+	}
+	if creds.AccessKeyID != "process-access" {
+		t.Fatalf("access key = %q, want process-access", creds.AccessKeyID)
+	}
+}
+
+func TestResolveCredentialsDoesNotFeedProcessEnvironment(t *testing.T) {
+	name := "DISTRIBUTOR_TEST_SDK_CHAIN_VALUE"
+	t.Setenv(name, "")
+	if err := os.Unsetenv(name); err != nil {
+		t.Fatalf("unset env: %v", err)
+	}
+	directory := t.TempDir()
+	writeSecret(t, directory, name, "secret")
+
+	result, err := LoadSecretEnvironment(directory, nil)
+	if err != nil {
+		t.Fatalf("LoadSecretEnvironment() error = %v", err)
+	}
+	assertEnvValue(t, result.Environment, name, "secret")
+	if _, ok := os.LookupEnv(name); ok {
+		t.Fatalf("%s is visible to process environment", name)
+	}
+}
+
+func TestResolveCredentialsMissingReferenceFailsWithoutSecretValue(t *testing.T) {
+	env := NewEnvironment(map[string]string{"PRESENT": "do-not-print"}, emptyLookup)
+	_, err := env.ResolveCredentials(Credentials{AccessKeyIDEnv: "MISSING"})
+	if err == nil {
+		t.Fatal("ResolveCredentials() error = nil, want error")
+	}
+	if !strings.Contains(err.Error(), "MISSING") {
+		t.Fatalf("ResolveCredentials() error = %q, want missing variable name", err)
+	}
+	if strings.Contains(err.Error(), "do-not-print") {
+		t.Fatalf("ResolveCredentials() error exposed secret value: %q", err)
+	}
+}
+
+func TestResolveCredentialsRejectsEmptyReferencedValue(t *testing.T) {
+	env := NewEnvironment(map[string]string{"EMPTY": ""}, emptyLookup)
+	_, err := env.ResolveCredentials(Credentials{AccessKeyIDEnv: "EMPTY"})
+	if err == nil {
+		t.Fatal("ResolveCredentials() error = nil, want error")
+	}
+	if !strings.Contains(err.Error(), "EMPTY") || !strings.Contains(err.Error(), "empty") {
+		t.Fatalf("ResolveCredentials() error = %q, want empty variable context", err)
+	}
+}
+
+func writeSecret(t *testing.T, directory, name, value string) {
+	t.Helper()
+	if err := os.WriteFile(filepath.Join(directory, name), []byte(value), 0o600); err != nil {
+		t.Fatalf("write secret %s: %v", name, err)
+	}
+}
+
+func assertEnvValue(t *testing.T, env Environment, name, want string) {
+	t.Helper()
+	got, ok := env.Lookup(name)
+	if !ok {
+		t.Fatalf("Lookup(%q) ok = false", name)
+	}
+	if got != want {
+		t.Fatalf("Lookup(%q) = %q, want %q", name, got, want)
+	}
+}
+
+func emptyLookup(string) (string, bool) {
+	return "", false
+}
+
+func mapLookup(values map[string]string) EnvLookup {
+	return func(name string) (string, bool) {
+		value, ok := values[name]
+		return value, ok
+	}
+}

internal/config/ssh.go

@@ -0,0 +1,64 @@
+package config
+
+import (
+	"fmt"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+)
+
+type HostKeyPolicy string
+
+const (
+	HostKeyPolicyStrict    HostKeyPolicy = "strict"
+	HostKeyPolicyAcceptNew HostKeyPolicy = "accept-new"
+	HostKeyPolicyOff       HostKeyPolicy = "off"
+)
+
+func (p *HostKeyPolicy) UnmarshalYAML(value *yaml.Node) error {
+	switch value.Kind {
+	case yaml.ScalarNode:
+	default:
+		return fmt.Errorf("host_key_policy must be a boolean or string")
+	}
+
+	switch value.Tag {
+	case "!!bool":
+		var enabled bool
+		if err := value.Decode(&enabled); err != nil {
+			return err
+		}
+		if enabled {
+			*p = HostKeyPolicyStrict
+		} else {
+			*p = HostKeyPolicyOff
+		}
+		return nil
+	case "!!str":
+		var raw string
+		if err := value.Decode(&raw); err != nil {
+			return err
+		}
+		normalized, ok := NormalizeHostKeyPolicy(raw)
+		if !ok {
+			return fmt.Errorf("host_key_policy must be strict, true, accept-new, off, or false")
+		}
+		*p = normalized
+		return nil
+	default:
+		return fmt.Errorf("host_key_policy must be a boolean or string")
+	}
+}
+
+func NormalizeHostKeyPolicy(value string) (HostKeyPolicy, bool) {
+	switch strings.ToLower(value) {
+	case "", string(HostKeyPolicyAcceptNew):
+		return HostKeyPolicyAcceptNew, true
+	case string(HostKeyPolicyStrict), "true":
+		return HostKeyPolicyStrict, true
+	case string(HostKeyPolicyOff), "false":
+		return HostKeyPolicyOff, true
+	default:
+		return "", false
+	}
+}

internal/config/validate.go

@@ -4,6 +4,8 @@ import (
 	"fmt"
 	"regexp"
 	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/link"
 )
 
 var idPattern = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]*$`)
@@ -20,6 +22,8 @@ func (e ValidationErrors) Error() string {
 func Validate(cfg Config) error {
 	var errs ValidationErrors
 
+	errs = validateHTTPServer(errs, "server.http", cfg.Server.HTTP)
+
 	if len(cfg.Pipelines) == 0 {
 		errs = append(errs, "pipelines is required")
 	}
@@ -37,7 +41,7 @@ func Validate(cfg Config) error {
 			pipelineIDs[pipeline.ID] = struct{}{}
 		}
 
-		errs = validateBackend(errs, pipelineContext+".source", pipeline.Source.Backend, pipeline.Source.Path, pipeline.Source.URI, pipeline.Source.Endpoint, pipeline.Source.Bucket)
+		errs = validateSourceBackend(errs, pipelineContext+".source", pipeline.Source)
 		errs = validateValidationPolicy(errs, pipelineContext+".validation", pipeline.Validation)
 		if len(pipeline.Destinations) == 0 {
 			errs = append(errs, pipelineContext+".destinations is required")
@@ -56,8 +60,10 @@ func Validate(cfg Config) error {
 				destinationIDs[destination.ID] = struct{}{}
 			}
 
-			errs = validateBackend(errs, destinationContext, destination.Backend, destination.Path, destination.URI, destination.Endpoint, destination.Bucket)
+			errs = validateDestinationBackend(errs, destinationContext, destination)
 			errs = validatePublishTransformPolicy(errs, destinationContext, destination.Publish, destination.Transform)
+			errs = validatePathMapping(errs, destinationContext+".path_mapping", destination.PathMap)
+			errs = validateLinks(errs, destinationContext+".links", destination.Links)
 			errs = validateTransferPolicy(errs, destinationContext+".transfer", destination.Transfer)
 		}
 	}
@@ -68,7 +74,57 @@ func Validate(cfg Config) error {
 	return nil
 }
 
-func validateBackend(errs ValidationErrors, context, backend, path, uri, endpoint, bucket string) ValidationErrors {
+func validateHTTPServer(errs ValidationErrors, context string, server HTTPServer) ValidationErrors {
+	if server.Bind == "" {
+		errs = append(errs, context+".bind is required")
+	}
+	if server.StagingRoot == "" {
+		errs = append(errs, context+".staging_root is required")
+	}
+	if server.MaxUploadSize == nil || *server.MaxUploadSize <= 0 {
+		errs = append(errs, context+".max_upload_size must be greater than zero")
+	}
+	if server.QueueSize <= 0 {
+		errs = append(errs, context+".queue_size must be greater than zero")
+	}
+	if server.MaxConcurrency <= 0 {
+		errs = append(errs, context+".max_concurrency must be greater than zero")
+	}
+	if server.Retention == nil || *server.Retention <= 0 {
+		errs = append(errs, context+".retention must be greater than zero")
+	}
+	return errs
+}
+
+func validateSourceBackend(errs ValidationErrors, context string, backend Backend) ValidationErrors {
+	if backend.Backend == BackendHTTPUpload {
+		return validateHTTPUploadSource(errs, context, backend.Upload)
+	}
+	return validateBackend(errs, context, backend.Backend, backend.Host, backend.Port, backend.Path, backend.Endpoint, backend.Bucket, backend.Prefix, backend.SSH.HostKeyPolicy, backend.Creds)
+}
+
+func validateDestinationBackend(errs ValidationErrors, context string, destination Destination) ValidationErrors {
+	if destination.Backend == BackendHTTPUpload {
+		errs = append(errs, context+".backend "+BackendHTTPUpload+" is only supported for sources")
+		return errs
+	}
+	return validateBackend(errs, context, destination.Backend, destination.Host, destination.Port, destination.Path, destination.Endpoint, destination.Bucket, destination.Prefix, destination.SSH.HostKeyPolicy, destination.Creds)
+}
+
+func validateHTTPUploadSource(errs ValidationErrors, context string, upload HTTPUpload) ValidationErrors {
+	if upload.TokenEnv == "" {
+		errs = append(errs, context+".token_env is required for http_upload backend")
+	}
+	if upload.StagingPath == "" {
+		errs = append(errs, context+".staging_path is required for http_upload backend")
+	}
+	if upload.MaxUploadSize == nil || *upload.MaxUploadSize <= 0 {
+		errs = append(errs, context+".max_upload_size must be greater than zero")
+	}
+	return errs
+}
+
+func validateBackend(errs ValidationErrors, context, backend, host string, port int, path, endpoint, bucket, prefix string, hostKeyPolicy HostKeyPolicy, creds Credentials) ValidationErrors {
 	switch backend {
 	case "":
 		errs = append(errs, context+".backend is required")
@@ -77,12 +133,23 @@ func validateBackend(errs ValidationErrors, context, backend, path, uri, endpoin
 			errs = append(errs, context+".path is required for local backend")
 		}
 	case BackendSSH:
-		if uri == "" {
-			errs = append(errs, context+".uri is required for ssh backend")
+		if host == "" {
+			errs = append(errs, context+".host is required for ssh backend")
 		}
 		if path == "" {
 			errs = append(errs, context+".path is required for ssh backend")
 		}
+		if port < 0 || port > 65535 {
+			errs = append(errs, context+".port must be between 1 and 65535")
+		}
+		if port == 0 {
+			errs = append(errs, context+".port is required for ssh backend after defaults are applied")
+		}
+		if hostKeyPolicy != "" {
+			if _, ok := NormalizeHostKeyPolicy(string(hostKeyPolicy)); !ok {
+				errs = append(errs, context+".host_key_policy must be strict, true, accept-new, off, or false")
+			}
+		}
 	case BackendS3:
 		if endpoint == "" {
 			errs = append(errs, context+".endpoint is required for s3 backend")
@@ -90,6 +157,12 @@ func validateBackend(errs ValidationErrors, context, backend, path, uri, endpoin
 		if bucket == "" {
 			errs = append(errs, context+".bucket is required for s3 backend")
 		}
+		if err := ValidateS3Prefix(prefix); err != nil {
+			errs = append(errs, context+".prefix must be a clean relative slash-separated path")
+		}
+		if (creds.AccessKeyIDEnv == "") != (creds.SecretAccessKeyEnv == "") {
+			errs = append(errs, context+".credentials.access_key_id_env and credentials.secret_access_key_env must be configured together")
+		}
 	default:
 		errs = append(errs, context+".backend "+backend+" is unsupported")
 	}
@@ -124,21 +197,52 @@ func ValidatePublishTransformPolicy(publish PublishPolicy, transform Transform)
 	if transform.MarkdownToHTML == nil {
 		return nil
 	}
+	mode := transform.MarkdownToHTML.Mode
+	if mode == "" {
+		mode = TransformModeSidecar
+	}
+	if mode != TransformModeSidecar && mode != TransformModeIndex {
+		return fmt.Errorf("transform.markdown_to_html.mode must be %s or %s", TransformModeSidecar, TransformModeIndex)
+	}
+	if transform.MarkdownToHTML.Input != "" && !transform.MarkdownToHTML.Enabled {
+		return fmt.Errorf("transform.markdown_to_html.input requires transform.markdown_to_html.enabled to be true")
+	}
+	if transform.MarkdownToHTML.Input != "" && mode != TransformModeIndex {
+		return fmt.Errorf("transform.markdown_to_html.input is only valid when mode is %s", TransformModeIndex)
+	}
+	if transform.MarkdownToHTML.Enabled && !publish.HTML {
+		return fmt.Errorf("transform.markdown_to_html.enabled requires publish.html to be true")
+	}
 	if publish.HTML && !transform.MarkdownToHTML.Enabled {
 		return fmt.Errorf("transform.markdown_to_html.enabled must be true when publish.html is true")
 	}
-	if publish.HTML && transform.MarkdownToHTML.Mode != TransformModeSidecar {
-		return fmt.Errorf("transform.markdown_to_html.mode must be %s", TransformModeSidecar)
-	}
-	if transform.MarkdownToHTML.Enabled && transform.MarkdownToHTML.Mode != TransformModeSidecar {
-		return fmt.Errorf("transform.markdown_to_html.mode must be %s", TransformModeSidecar)
-	}
-	if !transform.MarkdownToHTML.Enabled && transform.MarkdownToHTML.Mode != "" && transform.MarkdownToHTML.Mode != TransformModeSidecar {
-		return fmt.Errorf("transform.markdown_to_html.mode must be %s", TransformModeSidecar)
-	}
 	return nil
 }
 
+func validatePathMapping(errs ValidationErrors, context string, mapping PathMapping) ValidationErrors {
+	if mapping.Mode != PathMappingPreserveRelative && mapping.Mode != PathMappingFixed {
+		errs = append(errs, context+".mode must be "+PathMappingPreserveRelative+" or "+PathMappingFixed)
+	}
+	return errs
+}
+
+func validateLinks(errs ValidationErrors, context string, links *Links) ValidationErrors {
+	if links == nil {
+		return errs
+	}
+	if links.BaseURL == "" {
+		errs = append(errs, context+".base_url is required")
+	} else if err := link.ValidateHTTPURL(links.BaseURL); err != nil {
+		errs = append(errs, context+".base_url "+err.Error())
+	}
+	switch links.Primary {
+	case LinkPrimaryAuto, LinkPrimaryHTML, LinkPrimarySource:
+	default:
+		errs = append(errs, context+".primary must be "+LinkPrimaryAuto+", "+LinkPrimaryHTML+", or "+LinkPrimarySource)
+	}
+	return errs
+}
+
 func validateTransferPolicy(errs ValidationErrors, context string, policy TransferPolicy) ValidationErrors {
 	if policy.OnDestinationSame != TransferActionSkip && policy.OnDestinationSame != TransferActionFail {
 		errs = append(errs, context+".on_destination_same must be skip or fail")
@@ -146,11 +250,11 @@ func validateTransferPolicy(errs ValidationErrors, context string, policy Transf
 	if policy.OnDestinationOlder != TransferActionReplace && policy.OnDestinationOlder != TransferActionFail {
 		errs = append(errs, context+".on_destination_older must be replace or fail")
 	}
-	if policy.OnDestinationNewer != TransferActionSkip && policy.OnDestinationNewer != TransferActionFail {
-		errs = append(errs, context+".on_destination_newer must be skip or fail")
+	if policy.OnDestinationNewer != TransferActionSkip && policy.OnDestinationNewer != TransferActionFail && policy.OnDestinationNewer != TransferActionReplace {
+		errs = append(errs, context+".on_destination_newer must be skip, replace, or fail")
 	}
-	if policy.OnConflict != TransferActionFail {
-		errs = append(errs, context+".on_conflict must be fail")
+	if policy.OnConflict != TransferActionFail && policy.OnConflict != TransferActionReplace {
+		errs = append(errs, context+".on_conflict must be fail or replace")
 	}
 	return errs
 }

internal/config/validate_test.go

@@ -1,6 +1,9 @@
 package config
 
-import "testing"
+import (
+	"strings"
+	"testing"
+)
 
 func TestValidatePublishTransformPolicy(t *testing.T) {
 	tests := publishTransformPolicyCases()
@@ -47,6 +50,126 @@ func TestValidateChecksPublishTransformPolicy(t *testing.T) {
 	}
 }
 
+func TestValidateAcceptsForceReplacementTransferActions(t *testing.T) {
+	cfg := Config{Pipelines: []Pipeline{{
+		ID: "reports",
+		Source: Backend{
+			Backend: BackendLocal,
+			Path:    "/source",
+		},
+		Destinations: []Destination{{
+			ID:      "archive",
+			Backend: BackendLocal,
+			Path:    "/destination",
+			Transfer: TransferPolicy{
+				OnDestinationNewer: TransferActionReplace,
+				OnConflict:         TransferActionReplace,
+			},
+		}},
+	}}}
+	ApplyDefaults(&cfg)
+	if err := Validate(cfg); err != nil {
+		t.Fatalf("Validate() error = %v", err)
+	}
+}
+
+func TestValidatePathMapping(t *testing.T) {
+	tests := []struct {
+		name    string
+		mode    string
+		wantErr bool
+	}{
+		{name: "preserve relative", mode: PathMappingPreserveRelative},
+		{name: "fixed", mode: PathMappingFixed},
+		{name: "invalid", mode: "archive", wantErr: true},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			cfg := Config{Pipelines: []Pipeline{{
+				ID:     "reports",
+				Source: Backend{Backend: BackendLocal, Path: "/source"},
+				Destinations: []Destination{{
+					ID:      "archive",
+					Backend: BackendLocal,
+					Path:    "/destination",
+					PathMap: PathMapping{Mode: tt.mode},
+				}},
+			}}}
+			ApplyDefaults(&cfg)
+			err := Validate(cfg)
+			if tt.wantErr && err == nil {
+				t.Fatal("Validate() error = nil, want error")
+			}
+			if !tt.wantErr && err != nil {
+				t.Fatalf("Validate() error = %v", err)
+			}
+		})
+	}
+}
+
+func TestValidateLinks(t *testing.T) {
+	tests := []struct {
+		name    string
+		links   *Links
+		wantErr bool
+	}{
+		{name: "absent links"},
+		{name: "http", links: &Links{BaseURL: "http://reports.example.com/archive", Primary: LinkPrimaryAuto}},
+		{name: "https", links: &Links{BaseURL: "https://reports.example.com/archive/", Primary: LinkPrimaryHTML}},
+		{name: "source primary", links: &Links{BaseURL: "https://reports.example.com", Primary: LinkPrimarySource}},
+		{name: "missing base", links: &Links{Primary: LinkPrimaryAuto}, wantErr: true},
+		{name: "ftp scheme", links: &Links{BaseURL: "ftp://reports.example.com", Primary: LinkPrimaryAuto}, wantErr: true},
+		{name: "missing host", links: &Links{BaseURL: "https:///archive", Primary: LinkPrimaryAuto}, wantErr: true},
+		{name: "query", links: &Links{BaseURL: "https://reports.example.com/archive?preview=1", Primary: LinkPrimaryAuto}, wantErr: true},
+		{name: "fragment", links: &Links{BaseURL: "https://reports.example.com/archive#top", Primary: LinkPrimaryAuto}, wantErr: true},
+		{name: "invalid primary", links: &Links{BaseURL: "https://reports.example.com", Primary: "document"}, wantErr: true},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			cfg := Config{Pipelines: []Pipeline{{
+				ID:     "reports",
+				Source: Backend{Backend: BackendLocal, Path: "/source"},
+				Destinations: []Destination{{
+					ID:      "web",
+					Backend: BackendLocal,
+					Path:    "/destination",
+					Links:   tt.links,
+				}},
+			}}}
+			ApplyDefaults(&cfg)
+			err := Validate(cfg)
+			if tt.wantErr && err == nil {
+				t.Fatal("Validate() error = nil, want error")
+			}
+			if !tt.wantErr && err != nil {
+				t.Fatalf("Validate() error = %v", err)
+			}
+		})
+	}
+}
+
+func TestValidateLinksReportsFieldContext(t *testing.T) {
+	cfg := Config{Pipelines: []Pipeline{{
+		ID:     "reports",
+		Source: Backend{Backend: BackendLocal, Path: "/source"},
+		Destinations: []Destination{{
+			ID:      "web",
+			Backend: BackendLocal,
+			Path:    "/destination",
+			Links:   &Links{BaseURL: "https://reports.example.com/archive?preview=1", Primary: LinkPrimaryAuto},
+		}},
+	}}}
+	ApplyDefaults(&cfg)
+	err := Validate(cfg)
+	if err == nil {
+		t.Fatal("Validate() error = nil, want error")
+	}
+	want := "pipelines[0].destinations[0].links.base_url must not include a query string"
+	if !strings.Contains(err.Error(), want) {
+		t.Fatalf("Validate() error = %q, want %q", err, want)
+	}
+}
+
 type publishTransformPolicyCase struct {
 	name      string
 	publish   PublishPolicy
@@ -68,6 +191,30 @@ func publishTransformPolicyCases() []publishTransformPolicyCase {
 				Mode:    TransformModeSidecar,
 			}},
 		},
+		{
+			name:    "html only default mode allowed",
+			publish: PublishPolicy{HTML: true},
+			transform: Transform{MarkdownToHTML: &MarkdownToHTML{
+				Enabled: true,
+			}},
+		},
+		{
+			name:    "html only index allowed",
+			publish: PublishPolicy{HTML: true},
+			transform: Transform{MarkdownToHTML: &MarkdownToHTML{
+				Enabled: true,
+				Mode:    TransformModeIndex,
+			}},
+		},
+		{
+			name:    "html only index input allowed",
+			publish: PublishPolicy{HTML: true},
+			transform: Transform{MarkdownToHTML: &MarkdownToHTML{
+				Enabled: true,
+				Mode:    TransformModeIndex,
+				Input:   "report.md",
+			}},
+		},
 		{
 			name:    "source and html sidecar allowed",
 			publish: PublishPolicy{Source: true, HTML: true},
@@ -104,6 +251,15 @@ func publishTransformPolicyCases() []publishTransformPolicyCase {
 			}},
 			wantErr: true,
 		},
+		{
+			name:    "source only enabled transform rejected",
+			publish: PublishPolicy{Source: true},
+			transform: Transform{MarkdownToHTML: &MarkdownToHTML{
+				Enabled: true,
+				Mode:    TransformModeSidecar,
+			}},
+			wantErr: true,
+		},
 		{
 			name:    "enabled markdown wrong mode rejected",
 			publish: PublishPolicy{Source: true},
@@ -113,6 +269,16 @@ func publishTransformPolicyCases() []publishTransformPolicyCase {
 			}},
 			wantErr: true,
 		},
+		{
+			name:    "sidecar input rejected",
+			publish: PublishPolicy{HTML: true},
+			transform: Transform{MarkdownToHTML: &MarkdownToHTML{
+				Enabled: true,
+				Mode:    TransformModeSidecar,
+				Input:   "report.md",
+			}},
+			wantErr: true,
+		},
 		{
 			name:    "disabled markdown empty mode allowed",
 			publish: PublishPolicy{Source: true},
@@ -128,6 +294,24 @@ func publishTransformPolicyCases() []publishTransformPolicyCase {
 				Mode:    TransformModeSidecar,
 			}},
 		},
+		{
+			name:    "disabled markdown index mode allowed",
+			publish: PublishPolicy{Source: true},
+			transform: Transform{MarkdownToHTML: &MarkdownToHTML{
+				Enabled: false,
+				Mode:    TransformModeIndex,
+			}},
+		},
+		{
+			name:    "disabled markdown input rejected",
+			publish: PublishPolicy{Source: true},
+			transform: Transform{MarkdownToHTML: &MarkdownToHTML{
+				Enabled: false,
+				Mode:    TransformModeIndex,
+				Input:   "report.md",
+			}},
+			wantErr: true,
+		},
 		{
 			name:    "disabled markdown wrong mode rejected",
 			publish: PublishPolicy{Source: true},

internal/ingest/archive.go

@@ -0,0 +1,346 @@
+package ingest
+
+import (
+	"archive/tar"
+	"compress/gzip"
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"mime"
+	"os"
+	"path"
+	"path/filepath"
+	"strings"
+
+	sourcebundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
+)
+
+const (
+	ContentTypeTar   = "application/x-tar"
+	ContentTypeGzip  = "application/gzip"
+	ContentTypeXGzip = "application/x-gzip"
+)
+
+var (
+	ErrUnsupportedContentType = errors.New("unsupported archive content type")
+	ErrUploadTooLarge         = errors.New("upload exceeds maximum size")
+	ErrExtractedTooLarge      = errors.New("extracted bundle exceeds maximum size")
+	ErrTooManyFiles           = errors.New("extracted bundle has too many files")
+	ErrUnsafeArchivePath      = errors.New("unsafe archive path")
+)
+
+type StageOptions struct {
+	Body                io.Reader
+	ContentType         string
+	PipelineStagingPath string
+	RunID               string
+	MaxUploadSize       int64
+	MaxExtractedSize    int64
+	MaxFileCount        int
+}
+
+type StagedBundle struct {
+	Root     string
+	Manifest sourcebundle.Manifest
+}
+
+func StageArchive(ctx context.Context, opts StageOptions) (StagedBundle, error) {
+	if ctx == nil {
+		ctx = context.Background()
+	}
+	if err := validateStageOptions(opts); err != nil {
+		return StagedBundle{}, err
+	}
+	format, err := archiveFormat(opts.ContentType)
+	if err != nil {
+		return StagedBundle{}, err
+	}
+
+	if err := os.MkdirAll(opts.PipelineStagingPath, 0o755); err != nil {
+		return StagedBundle{}, fmt.Errorf("create pipeline staging path: %w", err)
+	}
+	tempDir, err := os.MkdirTemp(opts.PipelineStagingPath, "."+opts.RunID+"-")
+	if err != nil {
+		return StagedBundle{}, fmt.Errorf("create staging temp dir: %w", err)
+	}
+	cleanupTemp := true
+	defer func() {
+		if cleanupTemp {
+			_ = os.RemoveAll(tempDir)
+		}
+	}()
+
+	archivePath := filepath.Join(tempDir, "upload.archive")
+	if err := writeLimited(ctx, archivePath, opts.Body, opts.MaxUploadSize); err != nil {
+		return StagedBundle{}, err
+	}
+
+	extractRoot := filepath.Join(tempDir, "bundle")
+	if err := os.Mkdir(extractRoot, 0o755); err != nil {
+		return StagedBundle{}, fmt.Errorf("create extraction root: %w", err)
+	}
+	if err := extractArchive(ctx, archivePath, extractRoot, format, opts.MaxExtractedSize, opts.MaxFileCount); err != nil {
+		return StagedBundle{}, err
+	}
+
+	manifest, err := sourcebundle.LoadManifest(extractRoot)
+	if err != nil {
+		return StagedBundle{}, err
+	}
+	if err := sourcebundle.ValidateBundle(extractRoot, manifest); err != nil {
+		return StagedBundle{}, err
+	}
+
+	finalRoot := filepath.Join(opts.PipelineStagingPath, opts.RunID)
+	if err := os.Rename(extractRoot, finalRoot); err != nil {
+		return StagedBundle{}, fmt.Errorf("commit staged bundle: %w", err)
+	}
+	cleanupTemp = false
+	if err := os.RemoveAll(tempDir); err != nil {
+		return StagedBundle{}, fmt.Errorf("remove staging temp dir: %w", err)
+	}
+	return StagedBundle{
+		Root:     finalRoot,
+		Manifest: manifest,
+	}, nil
+}
+
+func validateStageOptions(opts StageOptions) error {
+	if opts.Body == nil {
+		return fmt.Errorf("body is required")
+	}
+	if opts.PipelineStagingPath == "" {
+		return fmt.Errorf("pipeline staging path is required")
+	}
+	if err := validateRunID(opts.RunID); err != nil {
+		return err
+	}
+	if opts.MaxUploadSize <= 0 {
+		return fmt.Errorf("max upload size must be greater than zero")
+	}
+	if opts.MaxExtractedSize <= 0 {
+		return fmt.Errorf("max extracted size must be greater than zero")
+	}
+	if opts.MaxFileCount <= 0 {
+		return fmt.Errorf("max file count must be greater than zero")
+	}
+	return nil
+}
+
+func validateRunID(value string) error {
+	if value == "" {
+		return fmt.Errorf("run id is required")
+	}
+	if value == "." || value == ".." || strings.ContainsAny(value, `/\`) {
+		return fmt.Errorf("run id must be a single filesystem path segment")
+	}
+	return nil
+}
+
+type archiveKind int
+
+const (
+	archiveKindTar archiveKind = iota + 1
+	archiveKindGzip
+)
+
+func archiveFormat(contentType string) (archiveKind, error) {
+	mediaType, _, err := mime.ParseMediaType(contentType)
+	if err != nil {
+		mediaType = contentType
+	}
+	switch mediaType {
+	case ContentTypeTar:
+		return archiveKindTar, nil
+	case ContentTypeGzip, ContentTypeXGzip:
+		return archiveKindGzip, nil
+	default:
+		return 0, ErrUnsupportedContentType
+	}
+}
+
+func writeLimited(ctx context.Context, destination string, body io.Reader, maxSize int64) error {
+	file, err := os.OpenFile(destination, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
+	if err != nil {
+		return fmt.Errorf("create upload archive: %w", err)
+	}
+	defer file.Close()
+
+	limited := &limitedReader{ctx: ctx, reader: body, limit: maxSize}
+	if _, err := io.Copy(file, limited); err != nil {
+		return err
+	}
+	if err := file.Close(); err != nil {
+		return fmt.Errorf("write upload archive: %w", err)
+	}
+	return nil
+}
+
+type limitedReader struct {
+	ctx    context.Context
+	reader io.Reader
+	limit  int64
+	read   int64
+}
+
+func (r *limitedReader) Read(data []byte) (int, error) {
+	if err := r.ctx.Err(); err != nil {
+		return 0, err
+	}
+	if r.read == r.limit {
+		var probe [1]byte
+		n, err := r.reader.Read(probe[:])
+		if n > 0 {
+			return 0, ErrUploadTooLarge
+		}
+		return 0, err
+	}
+	remaining := r.limit - r.read
+	if int64(len(data)) > remaining+1 {
+		data = data[:remaining+1]
+	}
+	n, err := r.reader.Read(data)
+	if r.read+int64(n) > r.limit {
+		allowed := int(r.limit - r.read)
+		r.read = r.limit
+		return allowed, ErrUploadTooLarge
+	}
+	r.read += int64(n)
+	return n, err
+}
+
+func extractArchive(ctx context.Context, archivePath, destination string, format archiveKind, maxExtractedSize int64, maxFileCount int) error {
+	file, err := os.Open(archivePath)
+	if err != nil {
+		return fmt.Errorf("open upload archive: %w", err)
+	}
+	defer file.Close()
+
+	var reader io.Reader = file
+	var gzipReader *gzip.Reader
+	if format == archiveKindGzip {
+		gzipReader, err = gzip.NewReader(file)
+		if err != nil {
+			return fmt.Errorf("open gzip archive: %w", err)
+		}
+		defer gzipReader.Close()
+		reader = gzipReader
+	}
+
+	extractor := archiveExtractor{
+		ctx:              ctx,
+		destination:      destination,
+		maxExtractedSize: maxExtractedSize,
+		maxFileCount:     maxFileCount,
+	}
+	if err := extractor.extract(tar.NewReader(reader)); err != nil {
+		return err
+	}
+	if extractor.rootManifestCount != 1 {
+		return fmt.Errorf("archive must contain exactly one root-level manifest.json")
+	}
+	return nil
+}
+
+type archiveExtractor struct {
+	ctx               context.Context
+	destination       string
+	maxExtractedSize  int64
+	maxFileCount      int
+	extractedSize     int64
+	fileCount         int
+	rootManifestCount int
+	seenFiles         map[string]struct{}
+}
+
+func (e *archiveExtractor) extract(reader *tar.Reader) error {
+	e.seenFiles = make(map[string]struct{})
+	for {
+		if err := e.ctx.Err(); err != nil {
+			return err
+		}
+		header, err := reader.Next()
+		if errors.Is(err, io.EOF) {
+			return nil
+		}
+		if err != nil {
+			return fmt.Errorf("read tar archive: %w", err)
+		}
+		name, err := cleanArchivePath(header.Name)
+		if err != nil {
+			return err
+		}
+		if path.Base(name) == sourcebundle.ManifestName {
+			if name != sourcebundle.ManifestName {
+				return fmt.Errorf("nested manifest %q is not allowed", name)
+			}
+			e.rootManifestCount++
+		}
+
+		switch header.Typeflag {
+		case tar.TypeDir:
+			if err := os.MkdirAll(filepath.Join(e.destination, filepath.FromSlash(name)), 0o755); err != nil {
+				return fmt.Errorf("create archive directory %q: %w", name, err)
+			}
+		case tar.TypeReg, tar.TypeRegA:
+			if err := e.extractFile(reader, name, header.Size); err != nil {
+				return err
+			}
+		default:
+			return fmt.Errorf("archive entry %q has unsupported type %c", name, header.Typeflag)
+		}
+	}
+}
+
+func (e *archiveExtractor) extractFile(reader *tar.Reader, name string, size int64) error {
+	if size < 0 {
+		return fmt.Errorf("archive entry %q has invalid size", name)
+	}
+	e.fileCount++
+	if e.fileCount > e.maxFileCount {
+		return ErrTooManyFiles
+	}
+	e.extractedSize += size
+	if e.extractedSize > e.maxExtractedSize {
+		return ErrExtractedTooLarge
+	}
+	if _, exists := e.seenFiles[name]; exists {
+		return fmt.Errorf("archive entry %q is duplicated", name)
+	}
+	e.seenFiles[name] = struct{}{}
+
+	fullPath := filepath.Join(e.destination, filepath.FromSlash(name))
+	if err := os.MkdirAll(filepath.Dir(fullPath), 0o755); err != nil {
+		return fmt.Errorf("create archive parent for %q: %w", name, err)
+	}
+	file, err := os.OpenFile(fullPath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644)
+	if err != nil {
+		return fmt.Errorf("create archive file %q: %w", name, err)
+	}
+	defer file.Close()
+
+	if _, err := io.CopyN(file, reader, size); err != nil {
+		return fmt.Errorf("extract archive file %q: %w", name, err)
+	}
+	if err := file.Close(); err != nil {
+		return fmt.Errorf("extract archive file %q: %w", name, err)
+	}
+	return nil
+}
+
+func cleanArchivePath(value string) (string, error) {
+	if value == "" || strings.Contains(value, `\`) || strings.HasPrefix(value, "/") {
+		return "", ErrUnsafeArchivePath
+	}
+	cleaned := path.Clean(value)
+	if cleaned != value {
+		return "", ErrUnsafeArchivePath
+	}
+	for _, segment := range strings.Split(value, "/") {
+		if segment == "" || segment == "." || segment == ".." {
+			return "", ErrUnsafeArchivePath
+		}
+	}
+	return value, nil
+}

internal/ingest/archive_test.go

@@ -0,0 +1,382 @@
+package ingest
+
+import (
+	"archive/tar"
+	"bytes"
+	"compress/gzip"
+	"context"
+	"errors"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+
+	sourcebundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
+)
+
+func TestStageArchiveAcceptsTar(t *testing.T) {
+	archive := validArchive(t, false)
+	staged := stageArchive(t, archive, ContentTypeTar)
+
+	if got, want := staged.Manifest.ID, "reports.ingest"; got != want {
+		t.Fatalf("manifest id = %q, want %q", got, want)
+	}
+	if got := readFile(t, staged.Root, "report.md"); got != "# Report\n" {
+		t.Fatalf("report = %q", got)
+	}
+}
+
+func TestStageArchiveAcceptsGzipTar(t *testing.T) {
+	archive := validArchive(t, true)
+	staged := stageArchive(t, archive, ContentTypeGzip+"; charset=binary")
+
+	if got, want := staged.Manifest.ID, "reports.ingest"; got != want {
+		t.Fatalf("manifest id = %q, want %q", got, want)
+	}
+	if got := readFile(t, staged.Root, "summary.txt"); got != "Summary\n" {
+		t.Fatalf("summary = %q", got)
+	}
+}
+
+func TestStageArchiveRejectsUnsupportedContentType(t *testing.T) {
+	err := stageArchiveError(t, validArchive(t, false), "application/zip", nil)
+	if !errors.Is(err, ErrUnsupportedContentType) {
+		t.Fatalf("StageArchive() error = %v, want ErrUnsupportedContentType", err)
+	}
+}
+
+func TestStageArchiveEnforcesMaxUploadSize(t *testing.T) {
+	archive := validArchive(t, false)
+	err := stageArchiveError(t, archive, ContentTypeTar, func(opts *StageOptions) {
+		opts.MaxUploadSize = int64(len(archive) - 1)
+	})
+	if !errors.Is(err, ErrUploadTooLarge) {
+		t.Fatalf("StageArchive() error = %v, want ErrUploadTooLarge", err)
+	}
+}
+
+func TestStageArchiveEnforcesExtractionLimits(t *testing.T) {
+	archive := validArchive(t, false)
+	tests := map[string]struct {
+		mutate  func(*StageOptions)
+		wantErr error
+	}{
+		"size": {
+			mutate: func(opts *StageOptions) {
+				opts.MaxExtractedSize = 1
+			},
+			wantErr: ErrExtractedTooLarge,
+		},
+		"files": {
+			mutate: func(opts *StageOptions) {
+				opts.MaxFileCount = 1
+			},
+			wantErr: ErrTooManyFiles,
+		},
+	}
+	for name, tt := range tests {
+		t.Run(name, func(t *testing.T) {
+			err := stageArchiveError(t, archive, ContentTypeTar, tt.mutate)
+			if !errors.Is(err, tt.wantErr) {
+				t.Fatalf("StageArchive() error = %v, want %v", err, tt.wantErr)
+			}
+		})
+	}
+}
+
+func TestStageArchiveRejectsUnsafeEntries(t *testing.T) {
+	tests := map[string][]tarEntry{
+		"absolute path": {
+			fileEntry("/report.md", "report"),
+		},
+		"path traversal": {
+			fileEntry("../report.md", "report"),
+		},
+		"backslash path": {
+			fileEntry(`nested\report.md`, "report"),
+		},
+		"symlink": {
+			{name: "link.md", typeflag: tar.TypeSymlink, linkname: "report.md"},
+		},
+		"hardlink": {
+			{name: "link.md", typeflag: tar.TypeLink, linkname: "report.md"},
+		},
+		"device": {
+			{name: "device", typeflag: tar.TypeChar},
+		},
+	}
+	for name, entries := range tests {
+		t.Run(name, func(t *testing.T) {
+			err := stageArchiveError(t, makeArchive(t, false, entries...), ContentTypeTar, nil)
+			if err == nil {
+				t.Fatal("StageArchive() error = nil, want error")
+			}
+		})
+	}
+}
+
+func TestStageArchiveRejectsBundleValidationFailures(t *testing.T) {
+	tests := map[string][]tarEntry{
+		"missing manifest": {
+			fileEntry("report.md", "report"),
+		},
+		"nested manifest": {
+			fileEntry("manifest.json", manifestJSON(t, manifestFor("reports.nested", fileSpec{path: "report.md", body: "report"}))),
+			fileEntry("nested/manifest.json", "{}"),
+			fileEntry("report.md", "report"),
+		},
+		"missing listed file": {
+			fileEntry("manifest.json", manifestJSON(t, manifestFor("reports.missing", fileSpec{path: "missing.md", body: "missing"}))),
+		},
+		"digest mismatch": {
+			fileEntry("manifest.json", manifestJSON(t, manifestFor("reports.digest", fileSpec{path: "report.md", body: "expected"}))),
+			fileEntry("report.md", "actual"),
+		},
+		"non regular listed file": {
+			fileEntry("manifest.json", manifestJSON(t, manifestFor("reports.directory", fileSpec{path: "report.md", body: "report"}))),
+			{name: "report.md", typeflag: tar.TypeDir},
+		},
+	}
+	for name, entries := range tests {
+		t.Run(name, func(t *testing.T) {
+			err := stageArchiveError(t, makeArchive(t, false, entries...), ContentTypeTar, nil)
+			if err == nil {
+				t.Fatal("StageArchive() error = nil, want error")
+			}
+		})
+	}
+}
+
+func TestStageArchiveCleansUpFailedExtraction(t *testing.T) {
+	stagingPath := filepath.Join(t.TempDir(), "staging")
+	archive := makeArchive(t, false, fileEntry("../report.md", "report"))
+	_, err := StageArchive(context.Background(), StageOptions{
+		Body:                bytes.NewReader(archive),
+		ContentType:         ContentTypeTar,
+		PipelineStagingPath: stagingPath,
+		RunID:               "reports.20260603T120000Z.abcd",
+		MaxUploadSize:       int64(len(archive)),
+		MaxExtractedSize:    1024 * 1024,
+		MaxFileCount:        10,
+	})
+	if err == nil {
+		t.Fatal("StageArchive() error = nil, want error")
+	}
+	entries, err := os.ReadDir(stagingPath)
+	if err != nil {
+		t.Fatalf("ReadDir() error = %v", err)
+	}
+	if len(entries) != 0 {
+		t.Fatalf("staging entries = %d, want cleanup", len(entries))
+	}
+}
+
+func stageArchive(t *testing.T, archive []byte, contentType string) StagedBundle {
+	t.Helper()
+	staged, err := StageArchive(context.Background(), defaultStageOptions(t, archive, contentType))
+	if err != nil {
+		t.Fatalf("StageArchive() error = %v", err)
+	}
+	return staged
+}
+
+func stageArchiveError(t *testing.T, archive []byte, contentType string, mutate func(*StageOptions)) error {
+	t.Helper()
+	opts := defaultStageOptions(t, archive, contentType)
+	if mutate != nil {
+		mutate(&opts)
+	}
+	_, err := StageArchive(context.Background(), opts)
+	if err == nil {
+		t.Fatal("StageArchive() error = nil, want error")
+	}
+	return err
+}
+
+func defaultStageOptions(t *testing.T, archive []byte, contentType string) StageOptions {
+	t.Helper()
+	return StageOptions{
+		Body:                bytes.NewReader(archive),
+		ContentType:         contentType,
+		PipelineStagingPath: filepath.Join(t.TempDir(), "staging"),
+		RunID:               "reports.20260603T120000Z.abcd",
+		MaxUploadSize:       int64(len(archive)),
+		MaxExtractedSize:    1024 * 1024,
+		MaxFileCount:        10,
+	}
+}
+
+func validArchive(t *testing.T, compressed bool) []byte {
+	t.Helper()
+	root := filepath.Join(t.TempDir(), "bundle")
+	sourceRoot := t.TempDir()
+	writeFile(t, sourceRoot, "report.md", "# Report\n")
+	writeFile(t, sourceRoot, "summary.txt", "Summary\n")
+	_, err := sourcebundle.WriteBundle(sourcebundle.WriteBundleOptions{
+		Root:    root,
+		ID:      "reports.ingest",
+		Created: time.Date(2026, 6, 3, 12, 0, 0, 0, time.UTC),
+		Files: []sourcebundle.BundleFile{
+			{SourcePath: filepath.Join(sourceRoot, "report.md"), Path: "report.md"},
+			{SourcePath: filepath.Join(sourceRoot, "summary.txt"), Path: "summary.txt"},
+		},
+	})
+	if err != nil {
+		t.Fatalf("WriteBundle() error = %v", err)
+	}
+
+	var entries []tarEntry
+	if err := filepath.WalkDir(root, func(filePath string, entry fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if entry.IsDir() {
+			return nil
+		}
+		relative, err := filepath.Rel(root, filePath)
+		if err != nil {
+			return err
+		}
+		data, err := os.ReadFile(filePath)
+		if err != nil {
+			return err
+		}
+		entries = append(entries, fileEntry(filepath.ToSlash(relative), string(data)))
+		return nil
+	}); err != nil {
+		t.Fatalf("walk bundle: %v", err)
+	}
+	return makeArchive(t, compressed, entries...)
+}
+
+type tarEntry struct {
+	name     string
+	typeflag byte
+	body     []byte
+	linkname string
+}
+
+func fileEntry(name, body string) tarEntry {
+	return tarEntry{name: name, typeflag: tar.TypeReg, body: []byte(body)}
+}
+
+func makeArchive(t *testing.T, compressed bool, entries ...tarEntry) []byte {
+	t.Helper()
+	var output bytes.Buffer
+	var writer *tar.Writer
+	var gzipWriter *gzip.Writer
+	if compressed {
+		gzipWriter = gzip.NewWriter(&output)
+		writer = tar.NewWriter(gzipWriter)
+	} else {
+		writer = tar.NewWriter(&output)
+	}
+	for _, entry := range entries {
+		header := &tar.Header{
+			Name:     entry.name,
+			Typeflag: entry.typeflag,
+			Size:     int64(len(entry.body)),
+			Mode:     0o644,
+			Linkname: entry.linkname,
+		}
+		if entry.typeflag == tar.TypeDir {
+			header.Size = 0
+			header.Mode = 0o755
+		}
+		if err := writer.WriteHeader(header); err != nil {
+			t.Fatalf("WriteHeader(%q) error = %v", entry.name, err)
+		}
+		if len(entry.body) > 0 {
+			if _, err := writer.Write(entry.body); err != nil {
+				t.Fatalf("Write(%q) error = %v", entry.name, err)
+			}
+		}
+	}
+	if err := writer.Close(); err != nil {
+		t.Fatalf("close tar writer: %v", err)
+	}
+	if gzipWriter != nil {
+		if err := gzipWriter.Close(); err != nil {
+			t.Fatalf("close gzip writer: %v", err)
+		}
+	}
+	return output.Bytes()
+}
+
+type fileSpec struct {
+	path string
+	body string
+}
+
+func manifestFor(id string, files ...fileSpec) sourcebundle.Manifest {
+	manifest := sourcebundle.Manifest{
+		SchemaVersion: sourcebundle.SchemaVersion,
+		ID:            id,
+		Created:       time.Date(2026, 6, 3, 12, 0, 0, 0, time.UTC),
+	}
+	for _, file := range files {
+		manifest.Files = append(manifest.Files, sourcebundle.ManifestFile{
+			Path:   file.path,
+			SHA256: sourcebundle.FileDigest([]byte(file.body)),
+			Size:   int64(len(file.body)),
+		})
+	}
+	manifest.Digest = sourcebundle.BundleDigest(manifest.Files)
+	return manifest
+}
+
+func manifestJSON(t *testing.T, manifest sourcebundle.Manifest) string {
+	t.Helper()
+	data, err := sourcebundle.MarshalManifest(manifest)
+	if err != nil {
+		t.Fatalf("MarshalManifest() error = %v", err)
+	}
+	return string(data)
+}
+
+func writeFile(t *testing.T, root, relative, body string) {
+	t.Helper()
+	fullPath := filepath.Join(root, filepath.FromSlash(relative))
+	if err := os.MkdirAll(filepath.Dir(fullPath), 0o755); err != nil {
+		t.Fatalf("MkdirAll() error = %v", err)
+	}
+	if err := os.WriteFile(fullPath, []byte(body), 0o644); err != nil {
+		t.Fatalf("WriteFile() error = %v", err)
+	}
+}
+
+func readFile(t *testing.T, root, relative string) string {
+	t.Helper()
+	data, err := os.ReadFile(filepath.Join(root, filepath.FromSlash(relative)))
+	if err != nil {
+		t.Fatalf("ReadFile() error = %v", err)
+	}
+	return string(data)
+}
+
+func TestCleanArchivePath(t *testing.T) {
+	tests := map[string]bool{
+		"manifest.json":       true,
+		"nested/report.md":    true,
+		"":                    false,
+		"/absolute.md":        false,
+		"../escape.md":        false,
+		"nested/../report.md": false,
+		`nested\report.md`:    false,
+		"./report.md":         false,
+		"nested//report.md":   false,
+	}
+	for value, wantOK := range tests {
+		t.Run(strings.ReplaceAll(value, "/", "_"), func(t *testing.T) {
+			_, err := cleanArchivePath(value)
+			if wantOK && err != nil {
+				t.Fatalf("cleanArchivePath(%q) error = %v", value, err)
+			}
+			if !wantOK && err == nil {
+				t.Fatalf("cleanArchivePath(%q) error = nil, want error", value)
+			}
+		})
+	}
+}

internal/link/url.go

@@ -0,0 +1,26 @@
+package link
+
+import (
+	"fmt"
+	"net/url"
+)
+
+func ValidateHTTPURL(value string) error {
+	parsed, err := url.Parse(value)
+	if err != nil {
+		return fmt.Errorf("must be a valid URL")
+	}
+	if parsed.Scheme != "http" && parsed.Scheme != "https" {
+		return fmt.Errorf("must use http or https")
+	}
+	if parsed.Host == "" {
+		return fmt.Errorf("must include a host")
+	}
+	if parsed.RawQuery != "" {
+		return fmt.Errorf("must not include a query string")
+	}
+	if parsed.Fragment != "" {
+		return fmt.Errorf("must not include a fragment")
+	}
+	return nil
+}

internal/link/url_test.go

@@ -0,0 +1,39 @@
+package link
+
+import (
+	"strings"
+	"testing"
+)
+
+func TestValidateHTTPURL(t *testing.T) {
+	tests := []struct {
+		name    string
+		value   string
+		wantErr string
+	}{
+		{name: "http", value: "http://reports.example.com/archive"},
+		{name: "https", value: "https://reports.example.com/archive"},
+		{name: "missing host", value: "https:///archive", wantErr: "must include a host"},
+		{name: "unsupported scheme", value: "ftp://reports.example.com/archive", wantErr: "must use http or https"},
+		{name: "query string", value: "https://reports.example.com/archive?preview=1", wantErr: "must not include a query string"},
+		{name: "fragment", value: "https://reports.example.com/archive#top", wantErr: "must not include a fragment"},
+		{name: "parse failure", value: "http://[::1", wantErr: "must be a valid URL"},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			err := ValidateHTTPURL(tt.value)
+			if tt.wantErr == "" {
+				if err != nil {
+					t.Fatalf("ValidateHTTPURL() error = %v", err)
+				}
+				return
+			}
+			if err == nil {
+				t.Fatal("ValidateHTTPURL() error = nil, want error")
+			}
+			if !strings.Contains(err.Error(), tt.wantErr) {
+				t.Fatalf("ValidateHTTPURL() error = %q, want %q", err, tt.wantErr)
+			}
+		})
+	}
+}

internal/publish/execute.go

@@ -14,7 +14,7 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
 	switch plan.Action {
 	case ActionSkipSame, ActionSkipDestinationNewer:
 		return nil
-	case ActionPublishNew, ActionReplaceOlder:
+	case ActionPublishNew, ActionReplaceOlder, ActionForceReplace:
 	default:
 		return fmt.Errorf("cannot execute action %s: %s", plan.Action, plan.Reason)
 	}
@@ -23,7 +23,15 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
 		if plan.ExistingState == nil {
 			return fmt.Errorf("replace requires existing destination state")
 		}
-		if err := req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, existingManagedOutputPaths(*plan.ExistingState), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true}); err != nil {
+		if err := req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, stateOutputManagedPaths(plan.ExistingState.Outputs), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true}); err != nil {
+			return err
+		}
+		if err := ensureDestinationEmpty(ctx, req.DestinationBackend, req.DestinationBundlePath); err != nil {
+			return err
+		}
+	}
+	if plan.Action == ActionForceReplace {
+		if err := req.DestinationBackend.DeletePrefix(ctx, req.DestinationBundlePath, storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true}); err != nil {
 			return err
 		}
 		if err := ensureDestinationEmpty(ctx, req.DestinationBackend, req.DestinationBundlePath); err != nil {
@@ -33,7 +41,7 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
 
 	writtenOutputs := make([]Output, 0, len(plan.Outputs))
 	cleanup := func() {
-		_ = req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, managedOutputPaths(writtenOutputs), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true})
+		_ = req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, ManagedOutputPaths(writtenOutputs), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true})
 	}
 	for _, output := range plan.Outputs {
 		destinationPath, err := storage.Join(req.DestinationBundlePath, output.DestinationPath)
@@ -68,7 +76,10 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
 		DestinationID:      req.DestinationID,
 		PublishedAt:        time.Now().UTC(),
 		Source:             state.SourceState{Manifest: req.SourceBundle.Manifest},
-		Outputs:            stateOutputs(plan.Outputs),
+		Outputs:            StateOutputFiles(plan.Outputs),
+	}
+	if plan.PrimaryURL != "" {
+		destinationState.Links = &state.LinkState{PrimaryURL: plan.PrimaryURL}
 	}
 	if err := state.Validate(destinationState); err != nil {
 		cleanup()
@@ -91,11 +102,3 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
 	}
 	return nil
 }
-
-func existingManagedOutputPaths(destinationState state.DistributorState) []string {
-	paths := make([]string, 0, len(destinationState.Outputs))
-	for _, output := range destinationState.Outputs {
-		paths = append(paths, output.Path)
-	}
-	return paths
-}

internal/publish/force_test.go

@@ -0,0 +1,193 @@
+package publish
+
+import (
+	"context"
+	"strings"
+	"testing"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
+	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
+)
+
+func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
+	tests := []struct {
+		name        string
+		prepare     func(t *testing.T, backend *fake.Backend, source bundle.Manifest)
+		transfer    config.TransferPolicy
+		wantReason  string
+		forceAction bool
+	}{
+		{
+			name: "unmanaged content",
+			prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
+				t.Helper()
+				testutil.WriteFakeFile(t, backend, "bundle/old.txt", "old")
+			},
+			transfer:    defaultTransfer(),
+			wantReason:  "fail_unmanaged",
+			forceAction: true,
+		},
+		{
+			name: "different source id",
+			prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
+				t.Helper()
+				conflict := source
+				conflict.ID = "other.source"
+				testutil.WriteFakeDestinationState(t, backend, "bundle", conflict, testutil.DestinationStateOptions{})
+			},
+			transfer:    conflictReplaceTransfer(),
+			wantReason:  "requires --force",
+			forceAction: true,
+		},
+		{
+			name: "same created digest conflict",
+			prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
+				t.Helper()
+				conflict := testutil.ValidManifest(testutil.BundleOptions{Files: []testutil.SourceFile{{Path: "report.md", Data: "# Different\n"}}})
+				testutil.WriteFakeDestinationState(t, backend, "bundle", conflict, testutil.DestinationStateOptions{})
+			},
+			transfer:    conflictReplaceTransfer(),
+			wantReason:  "requires --force",
+			forceAction: true,
+		},
+		{
+			name: "pipeline mismatch",
+			prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
+				t.Helper()
+				testutil.WriteFakeDestinationState(t, backend, "bundle", source, testutil.DestinationStateOptions{PipelineID: "other-pipeline"})
+			},
+			transfer:    conflictReplaceTransfer(),
+			wantReason:  "requires --force",
+			forceAction: true,
+		},
+		{
+			name: "destination mismatch",
+			prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
+				t.Helper()
+				testutil.WriteFakeDestinationState(t, backend, "bundle", source, testutil.DestinationStateOptions{DestinationID: "other-destination"})
+			},
+			transfer:    conflictReplaceTransfer(),
+			wantReason:  "requires --force",
+			forceAction: true,
+		},
+		{
+			name: "newer destination",
+			prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
+				t.Helper()
+				newer := source
+				newer.Created = newer.Created.AddDate(0, 0, 1)
+				testutil.WriteFakeDestinationState(t, backend, "bundle", newer, testutil.DestinationStateOptions{})
+			},
+			transfer:    newerReplaceTransfer(),
+			wantReason:  "requires --force",
+			forceAction: true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			sourceBackend := fake.New()
+			sourceBundle := testutil.WriteFakeSourceBundle(t, sourceBackend, "bundle", testutil.BundleOptions{})
+			destinationBackend := fake.New()
+			tt.prepare(t, destinationBackend, sourceBundle.Manifest)
+
+			req := forceRequest(sourceBackend, destinationBackend, sourceBundle, tt.transfer)
+			_, err := Build(context.Background(), req)
+			if err == nil || !strings.Contains(err.Error(), tt.wantReason) {
+				t.Fatalf("Build() error = %v, want %q", err, tt.wantReason)
+			}
+
+			req.Force = true
+			plan, err := Build(context.Background(), req)
+			if tt.forceAction {
+				if err != nil {
+					t.Fatalf("Build() with force error = %v", err)
+				}
+				if plan.Action != ActionForceReplace || !plan.Force {
+					t.Fatalf("forced plan action = %s force=%t", plan.Action, plan.Force)
+				}
+			}
+		})
+	}
+}
+
+func TestBuildRequiresConflictPolicyForStateConflicts(t *testing.T) {
+	sourceBackend := fake.New()
+	sourceBundle := testutil.WriteFakeSourceBundle(t, sourceBackend, "bundle", testutil.BundleOptions{})
+	destinationBackend := fake.New()
+	conflict := sourceBundle.Manifest
+	conflict.ID = "other.source"
+	testutil.WriteFakeDestinationState(t, destinationBackend, "bundle", conflict, testutil.DestinationStateOptions{})
+
+	req := forceRequest(sourceBackend, destinationBackend, sourceBundle, defaultTransfer())
+	req.Force = true
+	_, err := Build(context.Background(), req)
+	if err == nil || !strings.Contains(err.Error(), "destination source id differs") {
+		t.Fatalf("Build() error = %v, want conservative conflict", err)
+	}
+}
+
+func TestExecuteForcedReplacementDeletesOnlyBundlePath(t *testing.T) {
+	sourceBackend := fake.New()
+	sourceBundle := testutil.WriteFakeSourceBundle(t, sourceBackend, "bundle", testutil.BundleOptions{})
+	destinationBackend := fake.New()
+	testutil.WriteFakeFile(t, destinationBackend, "bundle/old.txt", "old")
+	testutil.WriteFakeFile(t, destinationBackend, "bundle/nested/old.txt", "old")
+	testutil.WriteFakeFile(t, destinationBackend, "bundle-sibling/keep.txt", "keep")
+	testutil.WriteFakeFile(t, destinationBackend, "outside.txt", "outside")
+
+	req := forceRequest(sourceBackend, destinationBackend, sourceBundle, defaultTransfer())
+	req.Force = true
+	plan, err := Build(context.Background(), req)
+	if err != nil {
+		t.Fatalf("Build() error = %v", err)
+	}
+	if plan.Action != ActionForceReplace {
+		t.Fatalf("plan action = %s, want force_replace", plan.Action)
+	}
+	if err := Execute(context.Background(), req, plan); err != nil {
+		t.Fatalf("Execute() error = %v", err)
+	}
+	testutil.AssertFakeFile(t, destinationBackend, "bundle/report.md", "# Report\nSunny.\n")
+	testutil.AssertFakeMissing(t, destinationBackend, "bundle/old.txt")
+	testutil.AssertFakeMissing(t, destinationBackend, "bundle/nested/old.txt")
+	testutil.AssertFakeFile(t, destinationBackend, "bundle-sibling/keep.txt", "keep")
+	testutil.AssertFakeFile(t, destinationBackend, "outside.txt", "outside")
+}
+
+func forceRequest(sourceBackend, destinationBackend *fake.Backend, sourceBundle bundle.Bundle, transfer config.TransferPolicy) Request {
+	return Request{
+		PipelineID:            "reports",
+		DestinationID:         "archive",
+		SourceBundle:          sourceBundle,
+		SourceBackend:         sourceBackend,
+		DestinationBackend:    destinationBackend,
+		DestinationBundlePath: sourceBundle.RootRelativePath,
+		Publish:               config.PublishPolicy{Source: true},
+		Transfer:              transfer,
+		DistributorVersion:    "test",
+	}
+}
+
+func defaultTransfer() config.TransferPolicy {
+	return config.TransferPolicy{
+		OnDestinationSame:  config.TransferActionSkip,
+		OnDestinationOlder: config.TransferActionReplace,
+		OnDestinationNewer: config.TransferActionSkip,
+		OnConflict:         config.TransferActionFail,
+	}
+}
+
+func conflictReplaceTransfer() config.TransferPolicy {
+	transfer := defaultTransfer()
+	transfer.OnConflict = config.TransferActionReplace
+	return transfer
+}
+
+func newerReplaceTransfer() config.TransferPolicy {
+	transfer := defaultTransfer()
+	transfer.OnDestinationNewer = config.TransferActionReplace
+	return transfer
+}

internal/publish/links.go

@@ -0,0 +1,120 @@
+package publish
+
+import (
+	"fmt"
+	"net/url"
+	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/link"
+	"gitea.maximumdirect.net/eric/distributor/internal/state"
+)
+
+func PlanLinks(req Request, outputs []Output) ([]Output, string, error) {
+	if req.Links == nil {
+		return outputs, "", nil
+	}
+	if err := link.ValidateHTTPURL(req.Links.BaseURL); err != nil {
+		return nil, "", fmt.Errorf("link base URL: %w", err)
+	}
+	linked := make([]Output, 0, len(outputs))
+	for _, output := range outputs {
+		outputURL, err := OutputURL(req.Links.BaseURL, req.DestinationBundlePath, output.DestinationPath)
+		if err != nil {
+			return nil, "", err
+		}
+		output.URL = outputURL
+		linked = append(linked, output)
+	}
+	return linked, primaryURL(linked, req.Links.Primary), nil
+}
+
+func OutputURL(baseURL, destinationBundlePath, outputPath string) (string, error) {
+	parsed, err := url.Parse(baseURL)
+	if err != nil {
+		return "", fmt.Errorf("link base URL: %w", err)
+	}
+	segments := pathSegments(parsed.Path)
+	segments = append(segments, pathSegments(destinationBundlePath)...)
+	outputSegments := pathSegments(outputPath)
+	trailingSlash := false
+	if len(outputSegments) > 0 && outputSegments[len(outputSegments)-1] == "index.html" {
+		outputSegments = outputSegments[:len(outputSegments)-1]
+		trailingSlash = true
+	}
+	segments = append(segments, outputSegments...)
+	parsed.Path = urlPath(segments, trailingSlash)
+	parsed.RawPath = ""
+	return parsed.String(), nil
+}
+
+func pathSegments(value string) []string {
+	trimmed := strings.Trim(value, "/")
+	if trimmed == "" {
+		return nil
+	}
+	return strings.Split(trimmed, "/")
+}
+
+func urlPath(segments []string, trailingSlash bool) string {
+	if len(segments) == 0 {
+		return "/"
+	}
+	path := "/" + strings.Join(segments, "/")
+	if trailingSlash && !strings.HasSuffix(path, "/") {
+		path += "/"
+	}
+	return path
+}
+
+func primaryURL(outputs []Output, policy string) string {
+	switch policy {
+	case config.LinkPrimaryHTML:
+		return firstGeneratedHTMLURL(outputs)
+	case config.LinkPrimarySource:
+		return firstSourceURL(outputs)
+	default:
+		if url := firstIndexURL(outputs); url != "" {
+			return url
+		}
+		if url := firstGeneratedHTMLURL(outputs); url != "" {
+			return url
+		}
+		return firstSourceURL(outputs)
+	}
+}
+
+func firstIndexURL(outputs []Output) string {
+	for _, output := range outputs {
+		if isIndexOutput(output.DestinationPath) {
+			return output.URL
+		}
+	}
+	return ""
+}
+
+func firstGeneratedHTMLURL(outputs []Output) string {
+	for _, output := range outputs {
+		if output.Kind == state.OutputKindGenerated && isHTMLOutput(output.DestinationPath) {
+			return output.URL
+		}
+	}
+	return ""
+}
+
+func firstSourceURL(outputs []Output) string {
+	for _, output := range outputs {
+		if output.Kind == state.OutputKindSource {
+			return output.URL
+		}
+	}
+	return ""
+}
+
+func isIndexOutput(path string) bool {
+	return path == "index.html" || strings.HasSuffix(path, "/index.html")
+}
+
+func isHTMLOutput(path string) bool {
+	return strings.HasSuffix(path, ".html")
+}

internal/publish/links_test.go

@@ -0,0 +1,169 @@
+package publish
+
+import (
+	"strings"
+	"testing"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/state"
+)
+
+func TestOutputURLUsesURLPathSemantics(t *testing.T) {
+	tests := []struct {
+		name                  string
+		baseURL               string
+		destinationBundlePath string
+		outputPath            string
+		want                  string
+	}{
+		{
+			name:                  "nested non index",
+			baseURL:               "https://reports.example.com/archive",
+			destinationBundlePath: "daily/brentwood",
+			outputPath:            "report.html",
+			want:                  "https://reports.example.com/archive/daily/brentwood/report.html",
+		},
+		{
+			name:                  "nested index",
+			baseURL:               "https://reports.example.com/archive",
+			destinationBundlePath: "daily/brentwood",
+			outputPath:            "index.html",
+			want:                  "https://reports.example.com/archive/daily/brentwood/",
+		},
+		{
+			name:                  "fixed index",
+			baseURL:               "https://reports.example.com/latest",
+			destinationBundlePath: "",
+			outputPath:            "index.html",
+			want:                  "https://reports.example.com/latest/",
+		},
+		{
+			name:                  "escaped segments",
+			baseURL:               "https://reports.example.com/archive root",
+			destinationBundlePath: "daily reports",
+			outputPath:            "morning report.html",
+			want:                  "https://reports.example.com/archive%20root/daily%20reports/morning%20report.html",
+		},
+		{
+			name:                  "nested output index",
+			baseURL:               "https://reports.example.com",
+			destinationBundlePath: "daily",
+			outputPath:            "site/index.html",
+			want:                  "https://reports.example.com/daily/site/",
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			got, err := OutputURL(tt.baseURL, tt.destinationBundlePath, tt.outputPath)
+			if err != nil {
+				t.Fatalf("OutputURL() error = %v", err)
+			}
+			if got != tt.want {
+				t.Fatalf("OutputURL() = %q, want %q", got, tt.want)
+			}
+		})
+	}
+}
+
+func TestPlanLinksSelectsPrimaryURL(t *testing.T) {
+	outputs := []Output{
+		{
+			DestinationPath: "report.md",
+			Kind:            state.OutputKindSource,
+		},
+		{
+			DestinationPath: "report.html",
+			Kind:            state.OutputKindGenerated,
+		},
+		{
+			DestinationPath: "index.html",
+			Kind:            state.OutputKindGenerated,
+		},
+	}
+	tests := []struct {
+		name    string
+		primary string
+		want    string
+	}{
+		{name: "auto prefers index", primary: config.LinkPrimaryAuto, want: "https://reports.example.com/daily/"},
+		{name: "html uses first generated html", primary: config.LinkPrimaryHTML, want: "https://reports.example.com/daily/report.html"},
+		{name: "source uses first source", primary: config.LinkPrimarySource, want: "https://reports.example.com/daily/report.md"},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			linked, primaryURL, err := PlanLinks(Request{
+				DestinationBundlePath: "daily",
+				Links: &config.Links{
+					BaseURL: "https://reports.example.com",
+					Primary: tt.primary,
+				},
+			}, outputs)
+			if err != nil {
+				t.Fatalf("PlanLinks() error = %v", err)
+			}
+			if primaryURL != tt.want {
+				t.Fatalf("primary URL = %q, want %q", primaryURL, tt.want)
+			}
+			for index, output := range linked {
+				if output.URL == "" {
+					t.Fatalf("linked output %d has empty URL", index)
+				}
+			}
+		})
+	}
+}
+
+func TestPlanLinksReturnsNoPrimaryWhenPolicyHasNoMatch(t *testing.T) {
+	linked, primaryURL, err := PlanLinks(Request{
+		DestinationBundlePath: "daily",
+		Links: &config.Links{
+			BaseURL: "https://reports.example.com",
+			Primary: config.LinkPrimaryHTML,
+		},
+	}, []Output{{
+		DestinationPath: "report.md",
+		Kind:            state.OutputKindSource,
+	}})
+	if err != nil {
+		t.Fatalf("PlanLinks() error = %v", err)
+	}
+	if primaryURL != "" {
+		t.Fatalf("primary URL = %q, want empty", primaryURL)
+	}
+	if linked[0].URL != "https://reports.example.com/daily/report.md" {
+		t.Fatalf("linked URL = %q", linked[0].URL)
+	}
+}
+
+func TestPlanLinksLeavesOutputsUnchangedWithoutConfig(t *testing.T) {
+	outputs := []Output{{DestinationPath: "report.md", Kind: state.OutputKindSource}}
+	linked, primaryURL, err := PlanLinks(Request{}, outputs)
+	if err != nil {
+		t.Fatalf("PlanLinks() error = %v", err)
+	}
+	if primaryURL != "" {
+		t.Fatalf("primary URL = %q, want empty", primaryURL)
+	}
+	if linked[0].URL != "" {
+		t.Fatalf("output URL = %q, want empty", linked[0].URL)
+	}
+}
+
+func TestPlanLinksValidatesBaseURLBeforePlanning(t *testing.T) {
+	_, _, err := PlanLinks(Request{
+		DestinationBundlePath: "daily",
+		Links: &config.Links{
+			BaseURL: "https://reports.example.com/archive?preview=1",
+			Primary: config.LinkPrimaryAuto,
+		},
+	}, []Output{{
+		DestinationPath: "report.md",
+		Kind:            state.OutputKindSource,
+	}})
+	if err == nil {
+		t.Fatal("PlanLinks() error = nil, want error")
+	}
+	if !strings.Contains(err.Error(), "link base URL: must not include a query string") {
+		t.Fatalf("PlanLinks() error = %q, want link base URL context", err)
+	}
+}

internal/publish/output.go

@@ -26,6 +26,10 @@ func PlanOutputs(ctx context.Context, req Request) ([]Output, error) {
 		generatedOutputs, err := transformer.Generate(ctx, transform.Request{
 			SourceBundle:  req.SourceBundle,
 			SourceBackend: req.SourceBackend,
+			Markdown: transform.MarkdownOptions{
+				Mode:  req.Transform.MarkdownToHTML.Mode,
+				Input: req.Transform.MarkdownToHTML.Input,
+			},
 		})
 		if err != nil {
 			return nil, err
@@ -93,25 +97,42 @@ func rejectOutputCollisions(outputs []Output) error {
 	return nil
 }
 
-func stateOutputs(outputs []Output) []state.OutputFile {
+func (o Output) StateOutputFile() state.OutputFile {
+	return state.OutputFile{
+		Path:       o.DestinationPath,
+		Kind:       o.Kind,
+		SourcePath: o.SourcePath,
+		Transform:  o.Transform,
+		URL:        o.URL,
+		SHA256:     o.SHA256,
+		Size:       o.Size,
+	}
+}
+
+func (o Output) ManagedPath() string {
+	return o.DestinationPath
+}
+
+func StateOutputFiles(outputs []Output) []state.OutputFile {
 	files := make([]state.OutputFile, 0, len(outputs))
 	for _, output := range outputs {
-		files = append(files, state.OutputFile{
-			Path:       output.DestinationPath,
-			Kind:       output.Kind,
-			SourcePath: output.SourcePath,
-			Transform:  output.Transform,
-			SHA256:     output.SHA256,
-			Size:       output.Size,
-		})
+		files = append(files, output.StateOutputFile())
 	}
 	return files
 }
 
-func managedOutputPaths(outputs []Output) []string {
+func ManagedOutputPaths(outputs []Output) []string {
 	paths := make([]string, 0, len(outputs))
 	for _, output := range outputs {
-		paths = append(paths, output.DestinationPath)
+		paths = append(paths, output.ManagedPath())
+	}
+	return paths
+}
+
+func stateOutputManagedPaths(outputs []state.OutputFile) []string {
+	paths := make([]string, 0, len(outputs))
+	for _, output := range outputs {
+		paths = append(paths, output.Path)
 	}
 	return paths
 }

internal/publish/output_test.go

@@ -6,11 +6,58 @@ import (
 
 	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
 	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/state"
 	"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
 	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
 	"gitea.maximumdirect.net/eric/distributor/internal/transform"
 )
 
+func TestOutputStateProjection(t *testing.T) {
+	sourceOutput := Output{
+		SourcePath:      "report.md",
+		DestinationPath: "report.md",
+		Kind:            state.OutputKindSource,
+		URL:             "https://reports.example.com/report.md",
+		SHA256:          "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+		Size:            123,
+	}
+	sourceState := sourceOutput.StateOutputFile()
+	if sourceState.Path != "report.md" || sourceState.Kind != state.OutputKindSource || sourceState.SourcePath != "report.md" || sourceState.URL != sourceOutput.URL || sourceState.SHA256 != sourceOutput.SHA256 || sourceState.Size != sourceOutput.Size {
+		t.Fatalf("source state output = %#v", sourceState)
+	}
+
+	generatedOutput := Output{
+		SourcePath:      "report.md",
+		DestinationPath: "report.html",
+		Kind:            state.OutputKindGenerated,
+		Transform:       transform.MarkdownToHTML,
+		URL:             "https://reports.example.com/report.html",
+		SHA256:          "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+		Size:            456,
+	}
+	generatedState := generatedOutput.StateOutputFile()
+	if generatedState.Path != "report.html" || generatedState.Kind != state.OutputKindGenerated || generatedState.SourcePath != "report.md" || generatedState.Transform != transform.MarkdownToHTML || generatedState.URL != generatedOutput.URL || generatedState.SHA256 != generatedOutput.SHA256 || generatedState.Size != generatedOutput.Size {
+		t.Fatalf("generated state output = %#v", generatedState)
+	}
+}
+
+func TestOutputSliceProjections(t *testing.T) {
+	outputs := []Output{
+		{SourcePath: "report.md", DestinationPath: "report.md", Kind: state.OutputKindSource, SHA256: "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", Size: 1},
+		{SourcePath: "report.md", DestinationPath: "report.html", Kind: state.OutputKindGenerated, Transform: transform.MarkdownToHTML, URL: "https://reports.example.com/report.html", SHA256: "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", Size: 2},
+	}
+
+	stateOutputs := StateOutputFiles(outputs)
+	if len(stateOutputs) != 2 || stateOutputs[1].Path != "report.html" || stateOutputs[1].Transform != transform.MarkdownToHTML || stateOutputs[1].URL != outputs[1].URL {
+		t.Fatalf("state outputs = %#v", stateOutputs)
+	}
+
+	paths := ManagedOutputPaths(outputs)
+	if len(paths) != 2 || paths[0] != "report.md" || paths[1] != "report.html" {
+		t.Fatalf("managed paths = %#v", paths)
+	}
+}
+
 func TestPlanOutputsRejectsCollision(t *testing.T) {
 	sourceBackend := fake.New()
 	sourceBundle := testutil.WriteFakeSourceBundle(t, sourceBackend, "", testutil.BundleOptions{
@@ -103,6 +150,35 @@ func TestPlanOutputsUsesRegisteredTransformer(t *testing.T) {
 	}
 }
 
+func TestPlanOutputsPassesMarkdownOptions(t *testing.T) {
+	data := []byte("<p>Generated</p>\n")
+	transformer := &recordingTransformer{outputs: []transform.Output{{
+		Path:       "index.html",
+		SourcePath: "report.md",
+		Transform:  transform.MarkdownToHTML,
+		Data:       data,
+		SHA256:     bundle.FileDigest(data),
+		Size:       int64(len(data)),
+	}}}
+
+	_, err := PlanOutputs(context.Background(), Request{
+		Publish: config.PublishPolicy{HTML: true},
+		Transform: config.Transform{MarkdownToHTML: &config.MarkdownToHTML{
+			Enabled: true,
+			Mode:    config.TransformModeIndex,
+			Input:   "report.md",
+		}},
+		Transformers: testResolver{transform.MarkdownToHTML: transformer},
+	})
+
+	if err != nil {
+		t.Fatalf("PlanOutputs() error = %v", err)
+	}
+	if transformer.request.Markdown.Mode != config.TransformModeIndex || transformer.request.Markdown.Input != "report.md" {
+		t.Fatalf("markdown options = %#v, want index/report.md", transformer.request.Markdown)
+	}
+}
+
 func TestBuildRejectsHTMLWithoutTransform(t *testing.T) {
 	sourceBackend := fake.New()
 	destinationBackend := fake.New()
@@ -148,6 +224,14 @@ func TestValidateRequestChecksPublishTransformPolicy(t *testing.T) {
 				Mode:    config.TransformModeSidecar,
 			}},
 		},
+		{
+			name:    "html only index allowed",
+			publish: config.PublishPolicy{HTML: true},
+			transform: config.Transform{MarkdownToHTML: &config.MarkdownToHTML{
+				Enabled: true,
+				Mode:    config.TransformModeIndex,
+			}},
+		},
 		{
 			name:    "source and html sidecar allowed",
 			publish: config.PublishPolicy{Source: true, HTML: true},
@@ -184,6 +268,15 @@ func TestValidateRequestChecksPublishTransformPolicy(t *testing.T) {
 			}},
 			wantErr: true,
 		},
+		{
+			name:    "source only enabled transform rejected",
+			publish: config.PublishPolicy{Source: true},
+			transform: config.Transform{MarkdownToHTML: &config.MarkdownToHTML{
+				Enabled: true,
+				Mode:    config.TransformModeSidecar,
+			}},
+			wantErr: true,
+		},
 		{
 			name:    "enabled markdown wrong mode rejected",
 			publish: config.PublishPolicy{Source: true},
@@ -253,3 +346,14 @@ type testTransformer struct {
 func (t testTransformer) Generate(context.Context, transform.Request) ([]transform.Output, error) {
 	return t.outputs, t.err
 }
+
+type recordingTransformer struct {
+	outputs []transform.Output
+	request transform.Request
+	err     error
+}
+
+func (t *recordingTransformer) Generate(_ context.Context, req transform.Request) ([]transform.Output, error) {
+	t.request = req
+	return t.outputs, t.err
+}

internal/publish/plan.go

@@ -20,6 +20,7 @@ const (
 	ActionSkipDestinationNewer Action = "skip_destination_newer"
 	ActionFailConflict         Action = "fail_conflict"
 	ActionFailUnmanaged        Action = "fail_unmanaged"
+	ActionForceReplace         Action = "force_replace"
 )
 
 type Request struct {
@@ -29,11 +30,14 @@ type Request struct {
 	SourceBackend         storage.Backend
 	DestinationBackend    storage.Backend
 	DestinationBundlePath string
+	PathMapping           string
 	Publish               config.PublishPolicy
 	Transform             config.Transform
+	Links                 *config.Links
 	Transformers          TransformerResolver
 	Transfer              config.TransferPolicy
 	DistributorVersion    string
+	Force                 bool
 }
 
 type TransformerResolver interface {
@@ -46,8 +50,11 @@ type Plan struct {
 	BundleID              string
 	BundlePath            string
 	DestinationBundlePath string
+	PathMapping           string
 	Action                Action
 	Reason                string
+	Force                 bool
+	PrimaryURL            string
 	Outputs               []Output
 	ExistingState         *state.DistributorState
 }
@@ -57,6 +64,7 @@ type Output struct {
 	DestinationPath string
 	Kind            string
 	Transform       string
+	URL             string
 	Data            []byte
 	SHA256          string
 	Size            int64
@@ -70,20 +78,27 @@ func Build(ctx context.Context, req Request) (Plan, error) {
 	if err != nil {
 		return Plan{}, err
 	}
+	outputs, primaryURL, err := PlanLinks(req, outputs)
+	if err != nil {
+		return Plan{}, err
+	}
 	status, err := inspectDestination(ctx, req.DestinationBackend, req.DestinationBundlePath)
 	if err != nil {
 		return Plan{}, err
 	}
-	comparison := state.Compare(req.SourceBundle.Manifest, req.PipelineID, req.DestinationID, status)
-	action, reason := actionForComparison(comparison, req.Transfer)
+	comparison := compareDestination(req, status)
+	action, reason := actionForComparison(comparison, req.Transfer, req.Force)
 	plan := Plan{
 		PipelineID:            req.PipelineID,
 		DestinationID:         req.DestinationID,
 		BundleID:              req.SourceBundle.Manifest.ID,
 		BundlePath:            req.SourceBundle.RootRelativePath,
 		DestinationBundlePath: req.DestinationBundlePath,
+		PathMapping:           req.PathMapping,
 		Action:                action,
 		Reason:                reason,
+		Force:                 action == ActionForceReplace,
+		PrimaryURL:            primaryURL,
 		Outputs:               outputs,
 		ExistingState:         status.State,
 	}
@@ -112,13 +127,39 @@ func validateRequest(req Request) error {
 	return nil
 }
 
-func actionForComparison(comparison state.Comparison, transfer config.TransferPolicy) (Action, string) {
+func compareDestination(req Request, status state.DestinationStatus) state.Comparison {
+	comparison := state.Compare(req.SourceBundle.Manifest, req.PipelineID, req.DestinationID, status)
+	if req.PathMapping != config.PathMappingFixed || comparison.Outcome != state.OutcomeDifferentSourceConflict || status.State == nil {
+		return comparison
+	}
+	destinationManifest := status.State.Source.Manifest
+	if destinationManifest.Created.Before(req.SourceBundle.Manifest.Created) {
+		return state.Comparison{Outcome: state.OutcomeDestinationOlder, Reason: "fixed destination source is older than selected source"}
+	}
+	if destinationManifest.Created.After(req.SourceBundle.Manifest.Created) {
+		return state.Comparison{Outcome: state.OutcomeDestinationNewer, Reason: "fixed destination source is newer than selected source"}
+	}
+	return comparison
+}
+
+func actionForComparison(comparison state.Comparison, transfer config.TransferPolicy, force bool) (Action, string) {
 	switch comparison.Outcome {
 	case state.OutcomeDestinationAbsent:
 		return ActionPublishNew, comparison.Reason
 	case state.OutcomeDestinationUnmanaged:
+		if force {
+			return ActionForceReplace, "forced replacement of unmanaged destination content"
+		}
 		return ActionFailUnmanaged, comparison.Reason
-	case state.OutcomeInvalidState, state.OutcomeIdentityMismatch, state.OutcomeSameCreatedConflict, state.OutcomeDifferentSourceConflict:
+	case state.OutcomeInvalidState:
+		return ActionFailConflict, comparison.Reason
+	case state.OutcomeIdentityMismatch, state.OutcomeSameCreatedConflict, state.OutcomeDifferentSourceConflict:
+		if transfer.OnConflict == config.TransferActionReplace {
+			if force {
+				return ActionForceReplace, "forced replacement of conflicting destination state: " + comparison.Reason
+			}
+			return ActionFailConflict, "destination conflict replacement requires --force"
+		}
 		return ActionFailConflict, comparison.Reason
 	case state.OutcomeSameSource:
 		if transfer.OnDestinationSame == config.TransferActionFail {
@@ -131,6 +172,12 @@ func actionForComparison(comparison state.Comparison, transfer config.TransferPo
 		}
 		return ActionReplaceOlder, comparison.Reason
 	case state.OutcomeDestinationNewer:
+		if transfer.OnDestinationNewer == config.TransferActionReplace {
+			if force {
+				return ActionForceReplace, "forced replacement of newer destination state"
+			}
+			return ActionFailConflict, "destination is newer and replacement requires --force"
+		}
 		if transfer.OnDestinationNewer == config.TransferActionFail {
 			return ActionFailConflict, "destination is newer and transfer policy requires failure"
 		}