Removed the completed code audit and implementation plan

docs/internal/app.md

@@ -26,9 +26,22 @@ The runner:
 
 Destination failures are collected while later destinations continue to run. Source open and source discovery failures stop the run because there are no valid bundles to fan out.
 
+## Run implementation
+
+`run.go` contains the public `Run` entrypoint and the main configuration orchestration path. Package-local run helpers are grouped by responsibility:
+
+- `run_selection.go`: destination bundle selection, path mapping decisions, and fixed-path warnings;
+- `run_warnings.go`: secret and SSH warning data;
+- `run_output.go`: text plan lines, JSON action records, and output projections;
+- `run_summary.go`: summary counters and JSON summary records;
+- `run_failures.go`: destination failure aggregation and partial-result detection;
+- `run_notify.go`: notification event projection and action filtering.
+
+These helpers remain in `internal/app` because command output, warning collection, destination failure aggregation, notifier handoff, and backend construction are app-owned orchestration concerns.
+
 ## Backend and transform wiring
 
-The app-level backend factory registers local, SSH, and S3 backends for execution. S3 explicit credential references are resolved through the config environment resolver before adapter construction.
+The app-level backend factory registers local, SSH, and S3 backends for execution. Source and destination backend config is converted through a shared app-local open spec before adapter construction. S3 explicit credential references are resolved through the config environment resolver.
 
 The app-level transform registry registers Markdown-to-HTML using `internal/transform/markdown`. Lower-level publish code receives a resolver and does not import concrete transform implementations.
 

docs/internal/bundle.md

@@ -20,6 +20,8 @@ The source manifest requires:
 
 Each file requires `path`, `sha256`, and `size`. Digests must use lowercase `sha256:<64 hex>` format. `created` must parse as RFC3339.
 
+`pkg/bundle.ValidateDigest` is the canonical digest format validator for producer-facing and internal code. `internal/bundle.ValidateDigest` delegates to that public validator so source manifests and destination state use the same digest grammar.
+
 ## Validation
 
 `pkg/bundle.ValidateManifest` owns normalized source manifest semantics: schema version, id, digest format, timestamp presence, file list presence, source path safety, duplicate file paths, reserved paths, file digest format, non-negative file sizes, and the top-level bundle digest.

docs/internal/link.md

@@ -0,0 +1,27 @@
+# Link URL Policy
+
+## Purpose
+
+`internal/link` defines shared validation for configured and persisted HTTP link URLs.
+
+## Inputs and outputs
+
+Input is a URL string. Output is either nil for an accepted URL or a concise validation error that callers wrap with field context.
+
+## Validation behavior
+
+Accepted URLs must parse successfully, use `http` or `https`, include a host, and omit query strings and fragments.
+
+## Boundaries
+
+This package validates URL shape only. It does not construct destination output URLs, choose primary URLs, infer public URLs from backend configuration, or read configuration files.
+
+## Tests
+
+Before changing link URL policy, inspect tests under `internal/link` and callers in `internal/config`, `internal/state`, and `internal/publish`.
+
+## Invariants
+
+- Configured `links.base_url`, persisted `links.primary_url`, persisted output `url`, and publish link planning use the same URL policy.
+- Callers own field-specific error context.
+- URL path construction remains in `internal/publish`.

docs/internal/publish.md

@@ -26,6 +26,8 @@ The package publishes source files and Markdown-to-HTML outputs. Markdown sideca
 
 The package uses `internal/state` for destination comparison, `internal/storage` for IO, and the shared `internal/config` publish/transform policy helper for request validation. It resolves transforms through a narrow resolver supplied by the caller; concrete transform registration is owned by the app layer. It does not parse CLI flags, load config files, or choose which source bundles a destination receives.
 
+The package owns projection from planned publish outputs to destination state output records and managed destination output paths. App JSON results and notification events keep their own schemas, but may use the publish output projection to avoid field-mapping drift.
+
 The app layer computes the destination bundle path before planning. `preserve_relative` destinations pass the source-root-relative bundle path. `fixed` destinations pass an empty destination bundle path, which means the destination backend root, and pass only the newest selected source bundle for that destination.
 
 When link config is present, publish planning builds per-output URLs from `links.base_url`, the destination bundle path, and each output path. `index.html` outputs use directory-style URLs. The primary URL is selected from planned outputs according to the destination primary policy.

docs/internal/storage.md

@@ -26,6 +26,19 @@ Storage errors use typed categories such as not found, already exists, invalid p
 
 Backends may wrap implementation-specific errors, but callers should receive storage errors where practical. Traversal can stop cleanly with `ErrStopWalk`.
 
+## Traversal helpers
+
+Backends own their traversal mechanics. The local adapter owns filesystem walking, the SSH adapter owns SFTP directory walking, and the S3 adapter owns object listing and pagination.
+
+`internal/storage` owns the shared callback emission rules used by backends:
+
+- context cancellation is checked before callback emission;
+- `WalkOptions.Limit` bounds the number of emitted entries;
+- `ErrStopWalk` stops traversal without becoming a caller-visible error;
+- callback errors are wrapped as storage walk errors.
+
+`storage.HasAny(ctx, backend, prefix)` provides the shared destination-content check. It calls `Walk` with non-recursive, limit-one traversal and stops after the first emitted entry.
+
 ## Deletion
 
 `DeleteManagedBundle` may delete listed managed outputs plus `.distributor.json`.

docs/operations.md

@@ -101,7 +101,7 @@ Do not edit `.distributor.json` by hand during normal operation. If it is missin
 
 ## Go Producer Bundles
 
-Go producer applications can import `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to create complete local source bundles with the same path, digest, timestamp, and validation rules used by `distributor`.
+Go producer applications can import `gitea.maximumdirect.net/eric/distributor/pkg/bundle` to create complete local source bundles with the same path, digest, timestamp, and validation rules used by `distributor`. The package also exposes digest helpers, including `ValidateDigest`, for producer code that needs to validate lowercase `sha256:<64 hex>` strings before writing manifests.
 
 Minimal producer-side bundle creation:
 

docs/policy/architecture.md

@@ -202,6 +202,7 @@ Use this current layout unless the project has a documented reason to differ:
 - `internal/config`: configuration structs, defaults, loading, precedence, and validation.
 - `internal/bundle`: storage-backed source bundle discovery and validation over the public manifest contract.
 - `internal/state`: `.distributor.json` parsing, validation, comparison, and output metadata.
+- `internal/link`: shared HTTP URL validation for configured and persisted link metadata.
 - `internal/storage`: backend interfaces, shared path/resource types, backend registry, and storage errors.
 - `internal/adapters/local`: local filesystem backend.
 - `internal/adapters/ssh`: SSH/SFTP backend.

docs/roadmap/audit.md

@@ -1,512 +0,0 @@
-# Code Quality And Deduplication Audit
-
-## Executive summary
-
-Overall code quality is strong for the current project stage. The repository generally follows the documented architecture: CLI parsing stays in `internal/cli`, orchestration stays in `internal/app`, storage behavior is behind `internal/storage`, adapters are thin enough for their protocols, and source manifest semantics are mostly centralized through `pkg/bundle`.
-
-The top three cleanup targets are:
-
-1. Backend open-config construction in `internal/app/backends.go`.
-2. HTTP URL validation shared by config and destination state.
-3. Configured-source command scaffolding for `validate` and `inspect`.
-
-The codebase appears ready for a limited cleanup pass. I did not find a major architectural risk or evidence that a broad rewrite is warranted. The best follow-up work is a sequence of small, behavior-preserving refactors with focused tests.
-
-This report belongs in `docs/roadmap/audit.md` because it describes future cleanup opportunities rather than implemented behavior.
-
-## Repository map reviewed
-
-Reviewed documentation:
-
-- `README.md`
-- `docs/policy/architecture.md`
-- `docs/policy/development.md`
-- `docs/policy/documentation.md`
-- `docs/config.md`
-- `docs/cli.md`
-- `docs/operations.md`
-- `docs/troubleshooting.md`
-- `docs/internal/app.md`
-- `docs/internal/bundle.md`
-- `docs/internal/config.md`
-- `docs/internal/publish.md`
-- `docs/internal/state.md`
-- `docs/internal/storage.md`
-- `docs/internal/transform.md`
-- `docs/integrations/markdown.md`
-- `docs/roadmap/implementation.md`
-
-Reviewed implementation areas:
-
-- `cmd/distributor`
-- `pkg/bundle`
-- `internal/app`
-- `internal/cli`
-- `internal/config`
-- `internal/bundle`
-- `internal/state`
-- `internal/storage`
-- `internal/storage/fake`
-- `internal/adapters/local`
-- `internal/adapters/ssh`
-- `internal/adapters/s3`
-- `internal/publish`
-- `internal/transform`
-- `internal/transform/markdown`
-- `internal/notify`
-- `internal/testutil`
-- `examples`
-
-Major execution paths reviewed:
-
-- `run` config load, source discovery, destination selection, publish planning, dry-run, execution, summary, JSON output, and notification handoff.
-- `validate` and `inspect` in local-path and configured-source modes.
-- `manifest create`.
-- Local, SSH/SFTP, S3, and fake storage behavior.
-- Source manifest validation, destination state comparison, Markdown-to-HTML output planning, link generation, and replacement safety.
-
-No important repository areas were deliberately skipped. There is no `internal/stage`, `internal/modules`, `internal/validators`, `internal/artifacts`, `internal/manifest`, `internal/schema`, or `internal/report` tree in this repository, so those requested audit categories were mapped to the actual bundle, state, config, publish, transform, storage, and CLI packages.
-
-## High-confidence deduplication opportunities
-
-### 1. Centralize backend open-config construction
-
-Affected files/packages:
-
-- `internal/app/backends.go`
-- `internal/config/config.go`
-- `internal/config/defaults.go`
-- `internal/adapters/ssh/options.go`
-- `internal/adapters/s3/options.go`
-- `internal/app/backends_test.go`
-
-Duplicated or near-duplicated behavior:
-
-`internal/app/backends.go` has parallel source and destination conversion paths. `sourceOpenConfig` and `destinationOpenConfig` both copy `path`, SSH host/user/port/key/known-hosts/host-key-policy/read-only fields into `storage.OpenConfig` with nearly the same code (`internal/app/backends.go:123`, `internal/app/backends.go:136`, `internal/app/backends.go:166`, `internal/app/backends.go:180`). S3 fields are centralized only partially through `addS3Config` (`internal/app/backends.go:149`).
-
-Why it matters:
-
-Source and destination backend shapes are intentionally similar. Any new executable backend field, credential policy, SSH option, S3 option, or dry-run mutation flag must be added to two mapping paths. That creates a realistic drift risk between `run` and configured-source `validate`/`inspect`, because all three paths share source opening but destinations have a separate config struct.
-
-Recommended refactor:
-
-Add a small app-local backend spec adapter, for example:
-
-- `backendOpenSpec` with backend name, path, SSH fields, S3 fields, credentials, and role label.
-- `openConfigForBackend(spec, readOnlyKnownHosts, environment)`.
-- thin constructors such as `sourceBackendSpec(config.Backend)` and `destinationBackendSpec(config.Destination)`.
-
-Keep this in `internal/app`; do not move concrete adapter construction into `internal/config`. Config should continue to own YAML shape, defaults, and validation only.
-
-Suggested tests:
-
-- Update `internal/app/backends_test.go` to assert one shared source/destination conversion table for local, SSH, S3, explicit S3 credentials, and dry-run known-host behavior.
-- Add a regression test that `validate` configured-source mode and `run` source opening produce the same SSH/S3 `storage.OpenConfig` for equivalent source config.
-- Keep adapter-specific tests unchanged.
-
-Risk level:
-
-Low to medium. The refactor touches runtime backend opening, but the behavior can be covered with existing backend factory tests and fake backends.
-
-### 2. Share HTTP URL validation rules
-
-Affected files/packages:
-
-- `internal/config/validate.go`
-- `internal/state/validate.go`
-- `internal/publish/links.go`
-- `internal/config/validate_test.go`
-- `internal/state/distributor_test.go`
-- `internal/publish/links_test.go`
-
-Duplicated or near-duplicated behavior:
-
-`validateLinkBaseURL` validates `http`/`https`, host presence, no query string, and no fragment for `links.base_url` (`internal/config/validate.go:201`). `validateStateURL` repeats the same scheme, host, query, and fragment checks for `links.primary_url` and output URLs (`internal/state/validate.go:86`). `publish.OutputURL` parses and mutates a URL independently while assuming config already enforced the base URL shape (`internal/publish/links.go:28`).
-
-Why it matters:
-
-These values are related parts of the same public URL contract. If URL policy changes, for example allowing fragments in one place or requiring canonical path handling, config validation and persisted state validation could drift. State validation also returns a raw parse error while config wraps parse failures as `must be a valid URL`, so user-facing and operator-facing diagnostics can diverge.
-
-Recommended refactor:
-
-Create a small internal URL policy helper in the narrowest suitable package. Two reasonable homes are:
-
-- `internal/state`, if state remains the canonical persisted URL contract and config calls a state helper would be acceptable; or
-- a new small package such as `internal/link` if avoiding a config-to-state dependency is preferred.
-
-The helper should validate absolute HTTP(S) URLs without query or fragment, and optionally expose a base-url-specific wrapper for config error wording. `publish.OutputURL` can continue to build URLs, but should use the same parser/validator for base URLs before mutation.
-
-Suggested tests:
-
-- One table-driven URL policy test for valid `http`, valid `https`, missing host, unsupported scheme, query, fragment, and parse failure.
-- Keep config and state tests focused on field context and wiring.
-- Keep `publish/links_test.go` for path semantics and `index.html` directory-style URLs.
-
-Risk level:
-
-Low. This is a small policy centralization with existing tests in all affected packages.
-
-### 3. Share configured-source CLI scaffolding for `validate` and `inspect`
-
-Affected files/packages:
-
-- `internal/cli/validate.go`
-- `internal/cli/inspect.go`
-- `internal/cli/source_mode.go`
-- `internal/app/source_select.go`
-- `internal/cli/root_test.go`
-
-Duplicated or near-duplicated behavior:
-
-`validateCommand` and `inspectCommand` define the same `--config`, `--pipeline`, `--bundle`, and `--format` flags; parse output format the same way; parse at most one optional path; call the same `validateInspectModeOK`; and then construct nearly identical app options (`internal/cli/validate.go:17`, `internal/cli/inspect.go:17`). The app layer already centralizes the actual source selection in `selectSourceBundles` and `selectSourceBundlesFromConfig` (`internal/app/source_select.go:28`, `internal/app/source_select.go:60`).
-
-Why it matters:
-
-These commands intentionally support the same two modes. Future changes to configured source mode, such as a new source-only diagnostic flag or error wording, would need coordinated edits in both command files and CLI tests.
-
-Recommended refactor:
-
-Keep command entrypoints explicit, but introduce a small `parseSourceCommandFlags(command, args, stderr)` helper returning `{path, configPath, pipelineID, bundlePath, outputFormat}`. `validateCommand` and `inspectCommand` would then differ only in help text and the app function they call.
-
-Do not introduce a CLI framework or a generic command engine.
-
-Suggested tests:
-
-- Keep command-specific smoke tests.
-- Add or preserve one table covering local path, too many paths, config without pipeline, pipeline without config, bundle without config, path plus config, invalid format, and JSON format for both commands.
-
-Risk level:
-
-Low. This is confined to CLI parsing and has strong existing tests.
-
-### 4. Centralize publish output metadata projection
-
-Affected files/packages:
-
-- `internal/publish/output.go`
-- `internal/publish/execute.go`
-- `internal/app/run.go`
-- `internal/notify/notify.go`
-- `internal/state/distributor.go`
-
-Duplicated or near-duplicated behavior:
-
-The same publish output metadata is projected into several destination structs:
-
-- `stateOutputs` maps `publish.Output` to `state.OutputFile` (`internal/publish/output.go:100`).
-- `managedOutputPaths` and `existingManagedOutputPaths` extract output paths (`internal/publish/output.go:116`, `internal/publish/execute.go:106`).
-- `runOutputsFromPlan` maps `publish.Output` to JSON output records (`internal/app/run.go:542`).
-- `notifyEvent` maps `publish.Output` to notification output records (`internal/app/run.go:434`).
-
-Why it matters:
-
-Output metadata is a central contract: path, kind, source path, transform, URL, digest, and size appear in destination state, JSON output, and notifications. Adding or renaming an output metadata field would require edits in several projection functions. A missed update could make state, JSON, and notification output disagree.
-
-Recommended refactor:
-
-Do not force all external output shapes into a single struct, because JSON/state/notification schemas are not identical. Instead, add small package-local helpers with clear names, for example:
-
-- `publish.Output.StateFile() state.OutputFile`
-- `publish.Output.ManagedPath() string`
-- possibly `publish.Output.PublicMetadata()` if app and notify projections grow.
-
-Keep app-owned JSON result structs in `internal/app`, but reduce repeated field-by-field copying where the destination schema is identical enough.
-
-Suggested tests:
-
-- Add focused publish tests for output-to-state conversion, including URL and generated transform fields.
-- Add an app JSON regression test that generated outputs include `path`, `kind`, `source_path`, `transform`, `url`, `sha256`, and `size`.
-- Add a notification mapping test for generated output metadata.
-
-Risk level:
-
-Low. The refactor is mechanical but should be covered because it touches user-visible JSON and destination state.
-
-## Medium-confidence opportunities
-
-### 1. Extract shared storage walk emission behavior carefully
-
-Affected files/packages:
-
-- `internal/adapters/local/backend.go`
-- `internal/adapters/ssh/backend.go`
-- `internal/adapters/s3/backend.go`
-- `internal/storage/backend.go`
-- `internal/storage/fake/backend.go`
-
-Duplicated or near-duplicated behavior:
-
-Local, SSH, and S3 backends each implement an inner `emit` function that checks context, applies `WalkOptions.Limit`, increments a visit counter, handles `storage.ErrStopWalk`, and wraps callback errors as storage walk errors (`internal/adapters/local/backend.go:164`, `internal/adapters/ssh/backend.go:212`, `internal/adapters/s3/backend.go:186`). Each backend also implements `HasAny` as `Walk(... Limit: 1)` (`internal/adapters/local/backend.go:221`, `internal/adapters/ssh/backend.go:247`, `internal/adapters/s3/backend.go:225`).
-
-Why it matters:
-
-Walk semantics are part of the storage contract. A change to limit behavior or `ErrStopWalk` handling would need repeated edits across adapters. Fake backend semantics already differ slightly because it handles `ErrStopWalk` in its own loop.
-
-Recommended refactor:
-
-Consider a tiny storage helper that wraps emission state, such as `storage.NewWalkEmitter(ctx, backendName, opts, fn)`, plus a default `storage.HasAny(ctx, backend, prefix)` helper. Keep traversal mechanics in adapters. Avoid extracting directory walking, SFTP behavior, S3 pagination, or deletion logic.
-
-Suggested tests:
-
-- Shared storage contract tests for `WalkOptions.Limit`, callback error wrapping, stopping with `ErrStopWalk`, and `HasAny` for empty prefix, missing prefix, file prefix, and directory/prefix content.
-- Run adapter tests after any change.
-
-Risk level:
-
-Medium. The duplication is real, but adapters have protocol-specific walk behavior. Keep the helper narrow.
-
-### 2. Unify digest format validation without expanding public API accidentally
-
-Affected files/packages:
-
-- `pkg/bundle/digest.go`
-- `internal/bundle/digest.go`
-- `internal/state/validate.go`
-
-Duplicated or near-duplicated behavior:
-
-`pkg/bundle` has a private `validateDigest` using a `sha256:<64 lowercase hex>` regex (`pkg/bundle/digest.go:12`). `internal/bundle` repeats the regex to expose `ValidateDigest` for state validation (`internal/bundle/digest.go:10`, `internal/state/validate.go:77`).
-
-Why it matters:
-
-Digest format is a source manifest and destination state invariant. If the digest grammar ever changes, the public package and internal state validation must be updated together.
-
-Recommended refactor:
-
-Either expose `pkg/bundle.ValidateDigest` as part of the producer-facing API, or keep the internal duplicate if the project does not want that API commitment. If exposed, document it in `pkg/bundle` tests and use it from `internal/bundle`.
-
-Suggested tests:
-
-- Move existing digest format tests to cover the public function if exposed.
-- Add state validation coverage that proves state uses the same digest validator.
-
-Risk level:
-
-Medium. The code change is small, but adding a public API has compatibility implications.
-
-### 3. Reduce local test fixture builders after behavior cleanup
-
-Affected files/packages:
-
-- `internal/app/run_test.go`
-- `internal/testutil/fixtures.go`
-- `internal/cli/root_test.go`
-- `internal/publish/force_test.go`
-
-Duplicated or near-duplicated behavior:
-
-`internal/testutil` already provides source bundle, fake source bundle, minimal config, fan-out config, destination state, and state-reading helpers. `internal/app/run_test.go` still maintains a local `testBundleOptions`, local source-bundle adapter, several YAML config builders, file assertions, fake-backend assertions, and state helpers (`internal/app/run_test.go:1237`, `internal/app/run_test.go:1275`, `internal/app/run_test.go:1318`, `internal/app/run_test.go:1349`, `internal/app/run_test.go:1410`, `internal/app/run_test.go:1437`). CLI and publish tests also have small local file/fake helpers.
-
-Why it matters:
-
-The large app test file is valuable coverage, but local fixture helpers make it easier for config examples and test setup patterns to drift from shared fixtures. This is mostly maintenance friction rather than production risk.
-
-Recommended refactor:
-
-After production cleanup is complete, move only generally reusable helpers into `internal/testutil`: config builder variants for publish policy, path mapping, links, and Markdown transform; file assertion helpers; fake backend assertion helpers. Keep highly specific scenario builders local.
-
-Suggested tests:
-
-- This is a test-only cleanup. Run `go test ./internal/app ./internal/cli ./internal/publish`.
-- Avoid rewriting test cases and helpers in the same commit if failures would become hard to diagnose.
-
-Risk level:
-
-Low to medium. Test-only, but high-volume edits can obscure behavior changes if combined with production refactors.
-
-### 4. Split run output formatting from run orchestration
-
-Affected files/packages:
-
-- `internal/app/run.go`
-- `docs/internal/app.md`
-- `internal/app/run_test.go`
-- `internal/cli/root_test.go`
-
-Duplicated or near-duplicated behavior:
-
-`internal/app/run.go` contains orchestration, destination selection, fixed-path warnings, text plan lines, JSON result types, summary counters, failure aggregation, notification mapping, secret warnings, and SSH warnings in one file. There is some duplication between text and JSON summary/status projection (`internal/app/run.go:591`, `internal/app/run.go:611`) and between plan error line formatting and JSON error action creation (`internal/app/run.go:264`, `internal/app/run.go:494`, `internal/app/run.go:529`).
-
-Why it matters:
-
-This is the largest production file and it carries multiple responsibilities. The current boundaries are not wrong, but future run output changes may require touching orchestration code, making regressions more likely.
-
-Recommended refactor:
-
-Split `internal/app/run.go` into package-local files by responsibility, for example `run_output.go`, `run_summary.go`, `run_warnings.go`, and `run_selection.go`. Keep the package and public behavior unchanged. This is file organization and small helper extraction, not a new reporting subsystem.
-
-Suggested tests:
-
-- Preserve existing app and CLI output tests.
-- Add focused tests for summary text/JSON consistency if the summary helpers are extracted.
-
-Risk level:
-
-Medium. Mostly mechanical, but output text and JSON are public behavior.
-
-## Boundary and responsibility concerns
-
-The primary boundary concern is `internal/app/backends.go`. It is acceptable for app-level wiring to import concrete adapters, but app also owns string-keyed conversion from config structs to adapter option maps. This is currently the right package, but the duplicated source/destination mapping should be narrowed through a shared app-local conversion helper.
-
-`internal/app/run.go` also mixes orchestration with output presentation and notification projection. This is not a boundary violation because app owns top-level command output, but it is a maintenance concern. Keep formatting in `internal/app`; do not move command output into `internal/publish`, because publish should remain focused on destination planning and execution.
-
-`internal/config` correctly owns config defaults and validation. It should not open backends or resolve storage paths. `ValidatePublishTransformPolicy` being shared with publish planning is appropriate because it prevents config and runtime policy drift.
-
-`internal/publish` correctly depends on `internal/storage`, `internal/state`, and transform interfaces rather than concrete adapters. Link generation in publish is destination publication behavior and fits the package, but URL validation should share policy with config/state.
-
-The storage adapters contain some similar code, but most of it is protocol-specific implementation. Do not move filesystem, SFTP, or S3 semantics into core packages.
-
-## Path, key, and naming construction review
-
-Path and naming construction is mostly centralized enough:
-
-- Source manifest filename is centralized through `pkg/bundle.ManifestName` and re-exported by `internal/bundle`.
-- Destination state filename and path helpers are centralized through `storage.StateFileName`, `storage.StatePath`, and `storage.ManagedBundleTargets` (`internal/storage/path.go:9`, `internal/storage/path.go:38`, `internal/storage/path.go:52`).
-- Runtime storage paths use `storage.Join`, `storage.ValidatePath`, `storage.ValidatePrefix`, and `storage.DisplayPath`.
-- S3 prefix normalization is centralized in `internal/config/s3.go`.
-- Public URL path construction is in `internal/publish/links.go`.
-
-Exact cleanup areas:
-
-- `pkg/bundle` has a private `.distributor.json` reserved-name constant while `internal/storage` has `StateFileName` (`pkg/bundle/path.go:9`, `internal/storage/path.go:9`). This is probably intentional because `pkg/bundle` cannot import `internal/storage`, but the reserved name should be documented near the public bundle contract if it is not already clear.
-- `storage.ValidatePath` and `pkg/bundle.ValidateSourcePath` intentionally share similar clean relative slash path rules, but source paths also reject reserved filenames (`pkg/bundle/path.go:11`, `internal/storage/path.go:11`). Do not merge them unless the semantic difference remains explicit.
-- S3 object-key construction should remain in the S3 adapter. Existing `objectKey`, `listPrefix`, and `logicalPathFromKey` helpers are appropriately local to S3 behavior.
-- URL construction is centralized in publish, but URL validation is duplicated and should be cleaned up as described above.
-
-## Resolution and catalog review
-
-There are no stage, module, schema, prompt, profile, artifact catalog, or validator catalogs in this repository.
-
-Implemented resolution paths are consistent:
-
-- Backend names are config constants and runtime adapter construction is registered in `internal/app`.
-- Transform names are centralized in `internal/transform`, and app registers Markdown-to-HTML once through `newTransformRegistry`.
-- Configured-source `validate` and `inspect` share `selectSourceBundles` and `selectSourceBundlesFromConfig`, so pipeline lookup, secret loading, source backend opening, bundle narrowing, and warnings are consistent.
-- Destination bundle resolution for `preserve_relative` and `fixed` mappings is centralized in app run selection.
-- Link primary selection is centralized in publish.
-
-Recommended centralization:
-
-- Keep transform registration in app. Do not create a plugin or catalog layer.
-- Consider extracting destination selection helpers from `run.go` into a package-local file, but not a new package.
-- Keep configured-source selection shared between validate and inspect; focus cleanup on duplicated CLI flag parsing.
-
-## Config and command-loading review
-
-Config loading is consistent:
-
-- `run` uses the supplied config path or `config.DefaultConfigPath`.
-- Configured-source `validate` and `inspect` require an explicit `--config`, matching the documented CLI.
-- `config.LoadFile` rejects unknown YAML fields, applies defaults, then validates.
-- `run`, configured-source `validate`, and configured-source `inspect` load `secrets.directory` before opening backends.
-- S3 explicit credentials are resolved through the config environment resolver, not direct `os.Getenv`.
-
-Intentional differences:
-
-- Local-path `validate` and `inspect` do not load config.
-- `run` opens destinations; configured-source diagnostics open only the selected source.
-- `run --dry-run` marks SSH known-host handling read-only; configured-source diagnostics do not have a dry-run flag.
-
-Likely accidental duplication:
-
-- `validate` and `inspect` CLI flag setup is repeated even though the mode semantics are identical.
-- Backend source/destination open-config conversion is repeated and should be centralized within app-level wiring.
-
-## State, manifest, or progress handling review
-
-Destination state handling is coherent:
-
-- `.distributor.json` is the sentinel and state record.
-- Publish planning inspects destination state through `storage.StatePath`, parses with `state.Parse`, and compares with `state.Compare`.
-- Replacement safety uses managed outputs from existing state for normal replacement and bounded prefix deletion for forced replacement.
-- Dry-run builds plans and output records without writing outputs or state.
-
-Manifest handling is coherent:
-
-- `pkg/bundle` owns public source manifest parsing, digesting, local writing, and local validation.
-- `internal/bundle` reuses the public manifest model for storage-backed discovery and validation.
-- State validation uses the same embedded manifest validation path.
-
-Potential cleanup:
-
-- Digest format validation is duplicated between `pkg/bundle` and `internal/bundle`. Decide whether to expose a public digest validator or keep the duplicate as a conscious public API boundary.
-- Output metadata projection into state, JSON output, and notifications should be made less repetitive before new output metadata is added.
-
-Progress handling:
-
-- There is no separate run checkpoint or progress store. This is consistent with current docs and architecture.
-
-## Refactors to avoid
-
-Avoid these refactors for the next cleanup pass:
-
-- A generic workflow engine for `run`, `validate`, `inspect`, and `manifest create`.
-- A CLI framework or broad command abstraction.
-- A broad plugin architecture for backends or transforms.
-- Moving backend construction into `internal/config`.
-- Moving publish planning policy into adapters.
-- Consolidating local, SSH, and S3 deletion or traversal implementations beyond tiny shared walk helpers.
-- Merging `pkg/bundle.ValidateSourcePath` and `storage.ValidatePath` without preserving their different semantics and error contracts.
-- Rewriting destination state schema or source manifest schema as part of deduplication.
-- Replacing explicit app output structs with a generic reporting layer.
-- Large test rewrites in the same commit as production refactors.
-
-## Recommended implementation sequence
-
-1. Centralize backend open-config construction in `internal/app/backends.go`.
-2. Add a shared internal URL policy helper and use it from config, state, and publish link planning.
-3. Extract configured-source CLI flag parsing for `validate` and `inspect`.
-4. Add small publish output projection helpers for state outputs, managed output paths, and notification/JSON metadata where appropriate.
-5. Split `internal/app/run.go` into package-local files for run selection, warnings, summary, and output formatting.
-6. Add a narrow storage walk-emitter or `HasAny` helper only if adapter tests can prove identical semantics.
-7. Decide whether `pkg/bundle` should expose digest validation; if yes, wire `internal/bundle.ValidateDigest` through it.
-8. Move broadly reusable test fixture helpers from `internal/app/run_test.go` into `internal/testutil`.
-9. Run a dead-code and legacy-helper sweep after the above changes, keeping public behavior unchanged.
-
-Each item should be a separate commit or prompt unless the implementation is very small.
-
-## Test strategy
-
-Tests to run for the full cleanup sequence:
-
-- `go test ./internal/app ./internal/cli ./internal/config`
-- `go test ./internal/publish ./internal/state`
-- `go test ./internal/storage ./internal/storage/fake ./internal/adapters/local ./internal/adapters/ssh ./internal/adapters/s3`
-- `go test ./internal/bundle ./pkg/bundle`
-- `go test ./...` after cross-package or documentation changes
-
-Tests to add before or during refactors:
-
-- Backend open-config equivalence tests for source and destination SSH/S3 mapping.
-- Configured-source `validate`/`inspect` shared CLI parsing table.
-- Shared URL policy table, plus config/state field-context tests.
-- Publish output projection tests for generated outputs with URLs.
-- Run summary text/JSON consistency tests if summary logic is extracted.
-- Storage walk contract tests if a shared walk helper is introduced.
-
-Validation performed during this audit:
-
-- `go list ./...` was run with workspace-safe Go cache paths. It required network access to fetch missing modules and completed successfully after approval.
-- No full test suite was run because this was a report-only task and no code behavior was changed.
-
-## Appendix: findings not worth acting on
-
-### Adapter-specific path/key conversion
-
-Local `nativePath`, SSH `nativePath`, and S3 `objectKey` all convert logical storage paths into backend-native addresses. They look similar in purpose, but their semantics differ: local must guard filesystem roots and symlink ancestors, SSH must use remote slash paths and SFTP behavior, and S3 must handle bucket/prefix object keys. Keep them local to adapters.
-
-### `storage.ValidatePath` and `pkg/bundle.ValidateSourcePath`
-
-Both enforce clean relative slash paths, but source bundle paths also reject reserved source/destination state filenames and use public producer-facing error wording. Keep them separate unless a future helper can preserve the different contracts.
-
-### Explicit command files
-
-`run`, `validate`, `inspect`, `version`, and `manifest` each having a command file is clear and appropriate. Only the shared configured-source flag parsing for `validate` and `inspect` should be extracted.
-
-### Text output and JSON output are intentionally separate shapes
-
-Text output is human-oriented and JSON output is a stable machine-readable envelope. Do not collapse them into one generic renderer. Small helpers for shared summary data are enough.
-
-### Test-local helpers for very specific scenarios
-
-Some local test helpers make individual scenarios easier to read. Move only broadly reusable builders and assertions into `internal/testutil`; leave one-off scenario setup local.

docs/roadmap/cleanup.md

@@ -1,408 +0,0 @@
-# Cleanup Implementation Roadmap
-
-This roadmap turns the audit findings in `docs/roadmap/audit.md` into staged
-implementation work for LLM coding agents.
-
-Each stage is intended to be implemented in order. Treat each stage as a
-separate prompt or commit unless the change is trivially small. Before starting
-any stage, read:
-
-- `docs/policy/architecture.md`
-- `docs/policy/development.md`
-- `docs/policy/documentation.md`
-
-## Global Rules
-
-- Preserve public CLI behavior, config semantics, source manifest schema,
-  destination state schema, implemented backend behavior, and documented output
-  contracts unless a stage explicitly says otherwise.
-- Do not introduce external dependencies.
-- Do not create a generic workflow engine, CLI framework, plugin system, broad
-  adapter abstraction, or generic reporting layer.
-- Keep concrete backend construction in `internal/app`.
-- Keep backend-specific filesystem, SSH/SFTP, and S3 details inside adapters.
-- Keep user-facing documentation unchanged unless public behavior changes.
-  These stages are intended to be behavior-preserving.
-- After each stage, run the tests listed for that stage. If Go cache
-  permissions fail in a restricted environment, use:
-
-```sh
-GOCACHE=/private/tmp/distributor-gocache GOMODCACHE=/private/tmp/distributor-gomodcache go test <packages>
-```
-
-## Stage 1: Backend Open-Config Unification
-
-Goal: make source and destination backend opening use one app-local conversion
-path while preserving current runtime behavior.
-
-Implement:
-
-- In `internal/app`, introduce a package-local backend-open spec that can
-  represent both `config.Backend` sources and `config.Destination`
-  destinations.
-- The spec must include backend name, path, SSH fields, S3 fields, credentials,
-  and a role label for error wording where needed.
-- Add two thin constructors:
-  - one from `config.Backend`;
-  - one from `config.Destination`.
-- Replace separate source/destination SSH and S3 `storage.OpenConfig`
-  construction with one helper that:
-  - sets the local path key;
-  - copies SSH host, user, port, key file, known-hosts path, host-key policy,
-    and read-only known-hosts flag;
-  - copies S3 endpoint, bucket, prefix, region, force-path-style setting, and
-    resolved explicit credentials;
-  - resolves explicit S3 credentials only through `config.Environment`.
-- Keep adapter registration and concrete adapter imports in `internal/app`.
-- Do not move backend opening, credential resolution, or runtime support checks
-  into `internal/config`.
-
-Tests:
-
-- Update `internal/app/backends_test.go` to assert equivalent source and
-  destination conversion for local, SSH, S3, S3 explicit credentials, and
-  dry-run known-host behavior.
-- Add or preserve a regression test proving configured-source validation and
-  run source opening produce equivalent SSH/S3 open config for the same source
-  settings.
-- Run:
-
-```sh
-go test ./internal/app
-```
-
-Acceptance criteria:
-
-- `run`, configured-source `validate`, and configured-source `inspect` still
-  open local, SSH, and S3 sources as before.
-- `run --dry-run` still makes SSH known-host writes read-only.
-- S3 explicit credentials still prefer resolved environment/secrets-directory
-  values and never call `os.Getenv` directly from backend wiring.
-
-## Stage 2: Shared Link URL Policy
-
-Goal: centralize HTTP URL validation for configured links and persisted link
-state.
-
-Implement:
-
-- Add a small `internal/link` package.
-- In that package, define the canonical link URL rule:
-  - URL must parse successfully;
-  - scheme must be `http` or `https`;
-  - host must be present;
-  - query string must be absent;
-  - fragment must be absent.
-- Use the helper from:
-  - config validation for `links.base_url`;
-  - state validation for `links.primary_url`;
-  - state validation for output `url`;
-  - publish link planning before building output URLs.
-- Preserve existing field-context error messages from config and state callers
-  where tests assert them. The helper may return concise common errors, while
-  callers add context such as `links.base_url` or `state outputs[i].url`.
-- Keep URL path construction and primary URL selection in `internal/publish`.
-
-Tests:
-
-- Add table-driven tests for `internal/link` covering valid `http`, valid
-  `https`, missing host, unsupported scheme, query string, fragment, and parse
-  failure.
-- Keep config tests focused on config field context and accepted primary-link
-  policies.
-- Keep state tests focused on persisted state validation context.
-- Keep publish tests focused on URL path semantics and `index.html`
-  directory-style URLs.
-- Run:
-
-```sh
-go test ./internal/config ./internal/state ./internal/publish
-```
-
-Acceptance criteria:
-
-- Config, state, and publish all enforce the same link URL policy.
-- Existing public URL generation behavior is unchanged.
-- No config package dependency on `internal/state` is introduced.
-
-## Stage 3: Configured-Source CLI Parsing
-
-Goal: remove duplicate `validate` and `inspect` configured-source flag parsing
-without obscuring command behavior.
-
-Implement:
-
-- In `internal/cli`, add a small helper that parses the shared source-diagnostic
-  command flags for a command name:
-  - `--config`;
-  - `--pipeline`;
-  - `--bundle`;
-  - `--format`;
-  - at most one optional local path.
-- The helper should return a simple parsed value containing path, config path,
-  pipeline id, bundle path, and normalized output format.
-- Keep `validateCommand` and `inspectCommand` as explicit entrypoints.
-- Keep command help text separate and command-specific.
-- Keep `validateInspectModeOK` behavior or fold it into the shared helper with
-  the same error semantics.
-- Do not introduce a CLI framework, generic command engine, or new root command
-  dispatch model.
-
-Tests:
-
-- Add or preserve CLI tests for both commands covering:
-  - local path mode;
-  - configured source mode;
-  - configured source JSON output;
-  - too many local paths;
-  - local path combined with config flags;
-  - `--pipeline` without `--config`;
-  - `--bundle` without `--config`;
-  - `--config` without `--pipeline`;
-  - invalid `--format`.
-- Run:
-
-```sh
-go test ./internal/cli ./internal/app
-```
-
-Acceptance criteria:
-
-- `validate` and `inspect` still support the exact documented modes.
-- Help output remains command-specific.
-- User-facing error wording remains compatible with current tests.
-
-## Stage 4: Publish Output Projection Helpers
-
-Goal: reduce drift when publish output metadata is projected into state, run
-JSON output, managed cleanup paths, and notifications.
-
-Implement:
-
-- Add package-local helper methods or functions in `internal/publish` for:
-  - converting a `publish.Output` to `state.OutputFile`;
-  - extracting a managed destination output path;
-  - converting output slices to state output slices;
-  - converting output slices to managed path slices.
-- Replace existing duplicate state-output and managed-path projections with the
-  new helpers.
-- Keep app-owned JSON result structs in `internal/app`.
-- Keep notify-owned event structs in `internal/notify`.
-- In `internal/app`, use publish helpers only where they reduce duplicate field
-  mapping without changing JSON or notification schemas.
-- Do not collapse state, JSON, and notification schemas into one shared public
-  type.
-
-Tests:
-
-- Add publish tests for output-to-state conversion, including source output,
-  generated output, transform name, URL, digest, and size.
-- Add or preserve app JSON regression coverage proving generated output records
-  include path, kind, source path, transform, URL, digest, and size.
-- Add or preserve notification mapping coverage for generated output metadata.
-- Run:
-
-```sh
-go test ./internal/publish ./internal/app
-```
-
-Acceptance criteria:
-
-- Destination state output metadata is unchanged.
-- Run JSON output metadata is unchanged.
-- Notification event metadata is unchanged.
-- Managed cleanup still deletes only recorded output paths plus state through
-  existing storage safety helpers.
-
-## Stage 5: Run Orchestration File Split
-
-Goal: make `internal/app/run.go` easier to maintain by moving package-local
-helpers into focused files without changing behavior.
-
-Implement:
-
-- Split current run-related helpers into package-local files such as:
-  - `run_selection.go` for destination bundle selection and fixed-path helpers;
-  - `run_warnings.go` for secret and SSH warning helpers;
-  - `run_output.go` for text and JSON action projection helpers;
-  - `run_summary.go` for summary counters and summary result projection;
-  - `run_failures.go` for run failure aggregation and partial-result detection.
-- Keep `Run`, `runConfig`, and `runConfigWithBackendFactory` as the primary
-  orchestration functions.
-- Do not change exported APIs, text output, JSON output, warning wording,
-  summary counters, failure aggregation, fixed-path selection, notification
-  behavior, or dry-run behavior.
-- Do not create a separate reporting package.
-
-Tests:
-
-- Add focused tests for summary text and JSON projection if extracting summary
-  code exposes a natural package-local test seam.
-- Preserve existing app and CLI output tests.
-- Run:
-
-```sh
-go test ./internal/app ./internal/cli
-```
-
-Acceptance criteria:
-
-- `run.go` is smaller and orchestration-focused.
-- All run output remains byte-for-byte compatible where tests currently assert
-  exact output.
-- JSON partial-failure behavior remains unchanged.
-
-## Stage 6: Storage Walk Helper
-
-Goal: centralize the narrow shared parts of storage walk callback handling
-without moving backend traversal semantics out of adapters.
-
-Implement:
-
-- Add an `internal/storage` helper for walk emission that owns only:
-  - context checks before callback emission;
-  - `WalkOptions.Limit` counting;
-  - `ErrStopWalk` handling;
-  - callback error wrapping as storage walk errors.
-- Use this helper in local, SSH, and S3 adapters only where the semantics match.
-- Add `storage.HasAny(ctx, backend, prefix)` as a shared helper implemented via
-  `Walk` with non-recursive limit-one traversal.
-- Update adapter `HasAny` implementations to call the shared helper if doing so
-  preserves current behavior.
-- Do not centralize:
-  - local filesystem walking;
-  - SSH/SFTP directory walking;
-  - S3 pagination or object listing;
-  - deletion behavior;
-  - native path or object key construction;
-  - symlink handling.
-
-Tests:
-
-- Add storage-level tests for the walk emitter or helper behavior:
-  - limit stops after the expected number of emissions;
-  - `ErrStopWalk` stops without error;
-  - callback errors are wrapped as storage walk errors;
-  - context cancellation is honored.
-- Add or preserve `HasAny` behavior tests for empty backends, missing prefixes,
-  file prefixes, directory prefixes, and object-prefix cases where applicable.
-- Run:
-
-```sh
-go test ./internal/storage ./internal/storage/fake ./internal/adapters/local ./internal/adapters/ssh ./internal/adapters/s3
-```
-
-Acceptance criteria:
-
-- Adapter traversal order and recursive/non-recursive behavior are unchanged.
-- Backend-specific path, key, symlink, deletion, and pagination logic remains
-  adapter-local.
-- `HasAny` behavior is consistent with existing tests.
-
-## Stage 7: Digest Validation Centralization
-
-Goal: make `pkg/bundle` the canonical home for digest format validation.
-
-Implement:
-
-- Expose `pkg/bundle.ValidateDigest(value string) error`.
-- Use the same lowercase `sha256:<64 hex>` rule currently used by manifest
-  validation.
-- Update `pkg/bundle.ValidateManifest` to call the public validator.
-- Update `internal/bundle.ValidateDigest` to delegate to
-  `pkg/bundle.ValidateDigest`.
-- Keep digest formatting and bundle digest generation unchanged.
-- Treat this as a public producer-facing API addition. Do not remove or rename
-  existing public APIs.
-
-Tests:
-
-- Add public package tests for `pkg/bundle.ValidateDigest` covering valid
-  digest, uppercase hex rejection, missing prefix, wrong algorithm, short hex,
-  long hex, and non-hex characters.
-- Add or preserve state validation coverage proving destination state output
-  SHA-256 validation uses the same rule.
-- Run:
-
-```sh
-go test ./pkg/bundle ./internal/bundle ./internal/state
-```
-
-Acceptance criteria:
-
-- There is one canonical digest validation rule.
-- Producer-facing code can validate digest strings directly.
-- Existing manifest and state validation behavior is unchanged.
-
-## Stage 8: Test Fixture Cleanup
-
-Goal: move broadly reusable test helpers into `internal/testutil` after
-production refactors are complete.
-
-Implement:
-
-- Move only reusable helpers into `internal/testutil`, such as:
-  - local config builders for publish policy, path mapping, links, and Markdown
-    transform combinations;
-  - file assertion helpers;
-  - fake backend file assertion helpers;
-  - state read/write helpers already duplicated in package tests.
-- Keep scenario-specific setup local to the package test that uses it.
-- Do not import `internal/testutil` from production code.
-- Do not combine this stage with production refactors.
-- Prefer small helper names that describe behavior over generic fixture-builder
-  abstractions.
-
-Tests:
-
-- Run:
-
-```sh
-go test ./internal/app ./internal/cli ./internal/publish
-```
-
-Acceptance criteria:
-
-- Test behavior and coverage are unchanged.
-- Test helpers reduce repeated setup without hiding scenario-specific behavior.
-- No production package imports `internal/testutil`.
-
-## Stage 9: Dead-Code And Legacy Sweep
-
-Goal: remove obsolete helpers left behind by the previous stages.
-
-Implement:
-
-- Search for unused or now-duplicative helpers after stages 1 through 8.
-- Remove only clearly obsolete code.
-- Keep this limited to dead code, duplicate helpers, and stale comments created
-  by earlier cleanup.
-- Do not perform unrelated refactors.
-- Do not change user-facing docs unless a previous stage intentionally changed
-  public behavior.
-
-Tests:
-
-- Run:
-
-```sh
-go test ./...
-```
-
-Acceptance criteria:
-
-- Full test suite passes.
-- No stale helper names remain from the old implementation paths.
-- No public behavior, schemas, or documented workflows change.
-
-## Final Cleanup Acceptance
-
-After all stages are complete:
-
-- Run `go test ./...`.
-- Re-read `docs/roadmap/audit.md` and confirm every high-confidence and
-  medium-confidence opportunity has been addressed or intentionally deferred in
-  a short note.
-- Confirm no refactor-to-avoid item from the audit was introduced.
-- Confirm `git status --short` contains only intentional changes for the final
-  stage.

internal/adapters/local/backend.go

@@ -161,31 +161,10 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
 		}
 		return b.translateError(storage.OpWalk, prefix, err)
 	}
-	visited := 0
-	emit := func(entry storage.Entry) error {
-		if err := ctx.Err(); err != nil {
-			return err
-		}
-		if opts.Limit > 0 && visited >= opts.Limit {
-			return storage.ErrStopWalk
-		}
-		visited++
-		if err := fn(entry); err != nil {
-			if errors.Is(err, storage.ErrStopWalk) {
-				return storage.ErrStopWalk
-			}
-			return storage.NewError(storage.OpWalk, backendName, entry.Path, storage.ErrUnknown, err)
-		}
-		return nil
-	}
+	emitter := storage.NewWalkEmitter(ctx, backendName, opts, fn)
 
 	if !info.IsDir() {
-		if err := emit(entryFromInfo(prefix, info)); errors.Is(err, storage.ErrStopWalk) {
-			return nil
-		} else if err != nil {
-			return err
-		}
-		return nil
+		return storage.FinishWalk(emitter.Emit(entryFromInfo(prefix, info)))
 	}
 
 	walkErr := filepath.WalkDir(nativePrefix, func(nativePath string, dirEntry fs.DirEntry, err error) error {
@@ -210,24 +189,13 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
 		if err != nil {
 			return b.translateError(storage.OpWalk, logicalPath, err)
 		}
-		return emit(entryFromInfo(logicalPath, info))
+		return emitter.Emit(entryFromInfo(logicalPath, info))
 	})
-	if errors.Is(walkErr, storage.ErrStopWalk) {
-		return nil
-	}
-	return walkErr
+	return storage.FinishWalk(walkErr)
 }
 
 func (b *Backend) HasAny(ctx context.Context, prefix string) (bool, error) {
-	found := false
-	err := b.Walk(ctx, prefix, storage.WalkOptions{Recursive: false, Limit: 1}, func(storage.Entry) error {
-		found = true
-		return storage.ErrStopWalk
-	})
-	if err != nil {
-		return false, err
-	}
-	return found, nil
+	return storage.HasAny(ctx, b, prefix)
 }
 
 func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {

internal/adapters/s3/backend.go

@@ -183,55 +183,25 @@ func (b *Backend) Walk(ctx context.Context, logicalPrefix string, opts storage.W
 	if err := storage.ValidatePrefix(logicalPrefix); err != nil {
 		return err
 	}
-	visited := 0
-	emit := func(entry storage.Entry) error {
-		if err := ctx.Err(); err != nil {
-			return err
-		}
-		if opts.Limit > 0 && visited >= opts.Limit {
-			return storage.ErrStopWalk
-		}
-		visited++
-		if err := fn(entry); err != nil {
-			if errors.Is(err, storage.ErrStopWalk) {
-				return storage.ErrStopWalk
-			}
-			return storage.NewError(storage.OpWalk, BackendName, entry.Path, storage.ErrUnknown, err)
-		}
-		return nil
-	}
+	emitter := storage.NewWalkEmitter(ctx, BackendName, opts, fn)
 	if logicalPrefix != "" {
 		entry, err := b.Stat(ctx, logicalPrefix)
 		if err == nil {
-			if err := emit(entry); errors.Is(err, storage.ErrStopWalk) {
-				return nil
-			} else if err != nil {
-				return err
+			if err := emitter.Emit(entry); err != nil {
+				return storage.FinishWalk(err)
 			}
-			if opts.Limit > 0 && visited >= opts.Limit {
+			if emitter.LimitReached() {
 				return nil
 			}
 		} else if !storage.IsNotFound(err) {
 			return err
 		}
 	}
-	err := b.walkObjects(ctx, logicalPrefix, opts, emit)
-	if errors.Is(err, storage.ErrStopWalk) {
-		return nil
-	}
-	return err
+	return storage.FinishWalk(b.walkObjects(ctx, logicalPrefix, opts, emitter.Emit))
 }
 
 func (b *Backend) HasAny(ctx context.Context, logicalPrefix string) (bool, error) {
-	found := false
-	err := b.Walk(ctx, logicalPrefix, storage.WalkOptions{Recursive: false, Limit: 1}, func(storage.Entry) error {
-		found = true
-		return storage.ErrStopWalk
-	})
-	if err != nil {
-		return false, err
-	}
-	return found, nil
+	return storage.HasAny(ctx, b, logicalPrefix)
 }
 
 func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {

internal/adapters/ssh/backend.go

@@ -209,51 +209,17 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
 		return b.translateError(storage.OpWalk, prefix, err)
 	}
 
-	visited := 0
-	emit := func(entry storage.Entry) error {
-		if err := ctx.Err(); err != nil {
-			return err
-		}
-		if opts.Limit > 0 && visited >= opts.Limit {
-			return storage.ErrStopWalk
-		}
-		visited++
-		if err := fn(entry); err != nil {
-			if errors.Is(err, storage.ErrStopWalk) {
-				return storage.ErrStopWalk
-			}
-			return storage.NewError(storage.OpWalk, BackendName, entry.Path, storage.ErrUnknown, err)
-		}
-		return nil
-	}
+	emitter := storage.NewWalkEmitter(ctx, BackendName, opts, fn)
 
 	if !info.IsDir() {
-		if err := emit(entryFromInfo(prefix, info)); errors.Is(err, storage.ErrStopWalk) {
-			return nil
-		} else if err != nil {
-			return err
-		}
-		return nil
+		return storage.FinishWalk(emitter.Emit(entryFromInfo(prefix, info)))
 	}
 
-	if err := b.walkDirectory(ctx, prefix, nativePrefix, opts, emit); errors.Is(err, storage.ErrStopWalk) {
-		return nil
-	} else if err != nil {
-		return err
-	}
-	return nil
+	return storage.FinishWalk(b.walkDirectory(ctx, prefix, nativePrefix, opts, emitter.Emit))
 }
 
 func (b *Backend) HasAny(ctx context.Context, prefix string) (bool, error) {
-	found := false
-	err := b.Walk(ctx, prefix, storage.WalkOptions{Recursive: false, Limit: 1}, func(storage.Entry) error {
-		found = true
-		return storage.ErrStopWalk
-	})
-	if err != nil {
-		return false, err
-	}
-	return found, nil
+	return storage.HasAny(ctx, b, prefix)
 }
 
 func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {

internal/app/backends.go

@@ -37,6 +37,22 @@ type backendFactory struct {
 	readOnlyKnownHosts bool
 }
 
+type backendOpenSpec struct {
+	role        string
+	backend     string
+	path        string
+	host        string
+	user        string
+	port        int
+	ssh         config.SSH
+	endpoint    string
+	bucket      string
+	prefix      string
+	region      string
+	forcePath   *bool
+	credentials config.Credentials
+}
+
 func newBackendFactory() *backendFactory {
 	return newBackendFactoryWithEnvironment(config.ProcessEnvironment())
 }
@@ -91,25 +107,11 @@ func newBackendFactoryWithEnvironment(environment config.Environment) *backendFa
 }
 
 func (f *backendFactory) openSource(ctx context.Context, source config.Backend) (storage.Backend, error) {
-	if source.Backend != config.BackendLocal && source.Backend != config.BackendSSH && source.Backend != config.BackendS3 {
-		return nil, fmt.Errorf("source backend %s is not implemented for execution", source.Backend)
-	}
-	openConfig, err := f.sourceOpenConfig(source)
-	if err != nil {
-		return nil, err
-	}
-	return f.registry.Open(ctx, source.Backend, openConfig)
+	return f.openBackend(ctx, backendOpenSpecFromSource(source))
 }
 
 func (f *backendFactory) openDestination(ctx context.Context, destination config.Destination) (storage.Backend, error) {
-	if destination.Backend != config.BackendLocal && destination.Backend != config.BackendSSH && destination.Backend != config.BackendS3 {
-		return nil, fmt.Errorf("backend %s is not implemented for execution", destination.Backend)
-	}
-	openConfig, err := f.destinationOpenConfig(destination)
-	if err != nil {
-		return nil, err
-	}
-	return f.registry.Open(ctx, destination.Backend, openConfig)
+	return f.openBackend(ctx, backendOpenSpecFromDestination(destination))
 }
 
 func (f *backendFactory) openLocalPath(ctx context.Context, path string) (storage.Backend, error) {
@@ -120,40 +122,51 @@ func (f *backendFactory) resolveCredentials(creds config.Credentials) (config.Re
 	return f.environment.ResolveCredentials(creds)
 }
 
-func (f *backendFactory) sourceOpenConfig(source config.Backend) (storage.OpenConfig, error) {
-	cfg := sourceOpenConfig(source)
-	if source.Backend == config.BackendS3 {
-		if err := f.addS3Config(cfg, source.Endpoint, source.Bucket, source.Prefix, source.Region, source.ForcePath, source.Creds); err != nil {
+func (f *backendFactory) openBackend(ctx context.Context, spec backendOpenSpec) (storage.Backend, error) {
+	if !backendExecutable(spec.backend) {
+		if spec.role == "source" {
+			return nil, fmt.Errorf("source backend %s is not implemented for execution", spec.backend)
+		}
+		return nil, fmt.Errorf("backend %s is not implemented for execution", spec.backend)
+	}
+	openConfig, err := f.openConfig(spec)
+	if err != nil {
+		return nil, err
+	}
+	return f.registry.Open(ctx, spec.backend, openConfig)
+}
+
+func backendExecutable(name string) bool {
+	return name == config.BackendLocal || name == config.BackendSSH || name == config.BackendS3
+}
+
+func (f *backendFactory) openConfig(spec backendOpenSpec) (storage.OpenConfig, error) {
+	cfg := storage.OpenConfig{storagePathKey: spec.path}
+	switch spec.backend {
+	case config.BackendSSH:
+		cfg[sshHostKey] = spec.host
+		cfg[sshUserKey] = spec.user
+		cfg[sshPortKey] = strconv.Itoa(spec.port)
+		cfg[sshKeyFileKey] = spec.ssh.KeyFile
+		cfg[sshKnownHostsKey] = spec.ssh.KnownHosts
+		cfg[sshHostKeyPolicyKey] = string(spec.ssh.HostKeyPolicy)
+		cfg[sshReadOnlyHostsKey] = strconv.FormatBool(f.readOnlyKnownHosts)
+	case config.BackendS3:
+		if err := f.addS3Config(cfg, spec); err != nil {
 			return nil, err
 		}
 	}
-	if source.Backend == config.BackendSSH {
-		cfg[sshReadOnlyHostsKey] = strconv.FormatBool(f.readOnlyKnownHosts)
-	}
 	return cfg, nil
 }
 
-func (f *backendFactory) destinationOpenConfig(destination config.Destination) (storage.OpenConfig, error) {
-	cfg := destinationOpenConfig(destination)
-	if destination.Backend == config.BackendS3 {
-		if err := f.addS3Config(cfg, destination.Endpoint, destination.Bucket, destination.Prefix, destination.Region, destination.ForcePath, destination.Creds); err != nil {
-			return nil, err
-		}
-	}
-	if destination.Backend == config.BackendSSH {
-		cfg[sshReadOnlyHostsKey] = strconv.FormatBool(f.readOnlyKnownHosts)
-	}
-	return cfg, nil
-}
-
-func (f *backendFactory) addS3Config(cfg storage.OpenConfig, endpoint, bucket, prefix, region string, forcePath *bool, creds config.Credentials) error {
-	cfg[s3EndpointKey] = endpoint
-	cfg[s3BucketKey] = bucket
-	cfg[s3PrefixKey] = prefix
-	cfg[s3RegionKey] = region
-	cfg[s3ForcePathStyleKey] = strconv.FormatBool(config.ForcePathStyle(forcePath))
-	if creds.AccessKeyIDEnv != "" || creds.SecretAccessKeyEnv != "" {
-		resolved, err := f.resolveCredentials(creds)
+func (f *backendFactory) addS3Config(cfg storage.OpenConfig, spec backendOpenSpec) error {
+	cfg[s3EndpointKey] = spec.endpoint
+	cfg[s3BucketKey] = spec.bucket
+	cfg[s3PrefixKey] = spec.prefix
+	cfg[s3RegionKey] = spec.region
+	cfg[s3ForcePathStyleKey] = strconv.FormatBool(config.ForcePathStyle(spec.forcePath))
+	if spec.credentials.AccessKeyIDEnv != "" || spec.credentials.SecretAccessKeyEnv != "" {
+		resolved, err := f.resolveCredentials(spec.credentials)
 		if err != nil {
 			return err
 		}
@@ -163,30 +176,38 @@ func (f *backendFactory) addS3Config(cfg storage.OpenConfig, endpoint, bucket, p
 	return nil
 }
 
-func sourceOpenConfig(source config.Backend) storage.OpenConfig {
-	cfg := storage.OpenConfig{storagePathKey: source.Path}
-	if source.Backend == config.BackendSSH {
-		cfg[sshHostKey] = source.Host
-		cfg[sshUserKey] = source.User
-		cfg[sshPortKey] = strconv.Itoa(source.Port)
-		cfg[sshKeyFileKey] = source.SSH.KeyFile
-		cfg[sshKnownHostsKey] = source.SSH.KnownHosts
-		cfg[sshHostKeyPolicyKey] = string(source.SSH.HostKeyPolicy)
-		cfg[sshReadOnlyHostsKey] = "false"
+func backendOpenSpecFromSource(source config.Backend) backendOpenSpec {
+	return backendOpenSpec{
+		role:        "source",
+		backend:     source.Backend,
+		path:        source.Path,
+		host:        source.Host,
+		user:        source.User,
+		port:        source.Port,
+		ssh:         source.SSH,
+		endpoint:    source.Endpoint,
+		bucket:      source.Bucket,
+		prefix:      source.Prefix,
+		region:      source.Region,
+		forcePath:   source.ForcePath,
+		credentials: source.Creds,
 	}
-	return cfg
 }
 
-func destinationOpenConfig(destination config.Destination) storage.OpenConfig {
-	cfg := storage.OpenConfig{storagePathKey: destination.Path}
-	if destination.Backend == config.BackendSSH {
-		cfg[sshHostKey] = destination.Host
-		cfg[sshUserKey] = destination.User
-		cfg[sshPortKey] = strconv.Itoa(destination.Port)
-		cfg[sshKeyFileKey] = destination.SSH.KeyFile
-		cfg[sshKnownHostsKey] = destination.SSH.KnownHosts
-		cfg[sshHostKeyPolicyKey] = string(destination.SSH.HostKeyPolicy)
-		cfg[sshReadOnlyHostsKey] = "false"
+func backendOpenSpecFromDestination(destination config.Destination) backendOpenSpec {
+	return backendOpenSpec{
+		role:        "destination",
+		backend:     destination.Backend,
+		path:        destination.Path,
+		host:        destination.Host,
+		user:        destination.User,
+		port:        destination.Port,
+		ssh:         destination.SSH,
+		endpoint:    destination.Endpoint,
+		bucket:      destination.Bucket,
+		prefix:      destination.Prefix,
+		region:      destination.Region,
+		forcePath:   destination.ForcePath,
+		credentials: destination.Creds,
 	}
-	return cfg
 }

internal/app/backends_test.go

@@ -1,13 +1,16 @@
 package app
 
 import (
+	"bytes"
 	"context"
+	"fmt"
 	"strings"
 	"testing"
 
 	"gitea.maximumdirect.net/eric/distributor/internal/config"
 	"gitea.maximumdirect.net/eric/distributor/internal/storage"
 	"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
+	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
 )
 
 func TestBackendFactoryOpensLocalSource(t *testing.T) {
@@ -275,7 +278,8 @@ func TestBackendFactoryResolvesCredentialsThroughEnvironment(t *testing.T) {
 }
 
 func TestBackendFactoryBuildsSSHSourceOpenConfig(t *testing.T) {
-	cfg := sourceOpenConfig(config.Backend{
+	factory := &backendFactory{environment: config.NewEnvironment(nil, nil)}
+	cfg, err := factory.openConfig(backendOpenSpecFromSource(config.Backend{
 		Backend: config.BackendSSH,
 		Host:    "source.example.com",
 		User:    "reports",
@@ -286,7 +290,10 @@ func TestBackendFactoryBuildsSSHSourceOpenConfig(t *testing.T) {
 			KnownHosts:    "/home/reports/.ssh/known_hosts",
 			HostKeyPolicy: config.HostKeyPolicyStrict,
 		},
-	})
+	}))
+	if err != nil {
+		t.Fatalf("openConfig() error = %v", err)
+	}
 
 	assertOpenConfig(t, cfg, map[string]string{
 		storagePathKey:      "/reports",
@@ -300,7 +307,8 @@ func TestBackendFactoryBuildsSSHSourceOpenConfig(t *testing.T) {
 }
 
 func TestBackendFactoryBuildsSSHDestinationOpenConfig(t *testing.T) {
-	cfg := destinationOpenConfig(config.Destination{
+	factory := &backendFactory{environment: config.NewEnvironment(nil, nil)}
+	cfg, err := factory.openConfig(backendOpenSpecFromDestination(config.Destination{
 		Backend: config.BackendSSH,
 		Host:    "destination.example.com",
 		User:    "deploy",
@@ -309,7 +317,10 @@ func TestBackendFactoryBuildsSSHDestinationOpenConfig(t *testing.T) {
 		SSH: config.SSH{
 			HostKeyPolicy: config.HostKeyPolicyAcceptNew,
 		},
-	})
+	}))
+	if err != nil {
+		t.Fatalf("openConfig() error = %v", err)
+	}
 
 	assertOpenConfig(t, cfg, map[string]string{
 		storagePathKey:      "/srv/archive",
@@ -320,6 +331,257 @@ func TestBackendFactoryBuildsSSHDestinationOpenConfig(t *testing.T) {
 	})
 }
 
+func TestBackendFactoryBuildsEquivalentSourceAndDestinationOpenConfig(t *testing.T) {
+	forcePathStyle := false
+	tests := []struct {
+		name        string
+		source      config.Backend
+		destination config.Destination
+	}{
+		{
+			name:        "local",
+			source:      config.Backend{Backend: config.BackendLocal, Path: "/reports"},
+			destination: config.Destination{Backend: config.BackendLocal, Path: "/reports"},
+		},
+		{
+			name: "ssh",
+			source: config.Backend{
+				Backend: config.BackendSSH,
+				Host:    "reports.example.com",
+				User:    "reports",
+				Port:    2222,
+				Path:    "/reports",
+				SSH: config.SSH{
+					KeyFile:       "/home/reports/.ssh/id_ed25519",
+					KnownHosts:    "/home/reports/.ssh/known_hosts",
+					HostKeyPolicy: config.HostKeyPolicyStrict,
+				},
+			},
+			destination: config.Destination{
+				Backend: config.BackendSSH,
+				Host:    "reports.example.com",
+				User:    "reports",
+				Port:    2222,
+				Path:    "/reports",
+				SSH: config.SSH{
+					KeyFile:       "/home/reports/.ssh/id_ed25519",
+					KnownHosts:    "/home/reports/.ssh/known_hosts",
+					HostKeyPolicy: config.HostKeyPolicyStrict,
+				},
+			},
+		},
+		{
+			name: "s3",
+			source: config.Backend{
+				Backend:   config.BackendS3,
+				Endpoint:  "https://s3.example.com",
+				Bucket:    "reports",
+				Prefix:    "archive",
+				Region:    "us-west-2",
+				ForcePath: &forcePathStyle,
+			},
+			destination: config.Destination{
+				Backend:   config.BackendS3,
+				Endpoint:  "https://s3.example.com",
+				Bucket:    "reports",
+				Prefix:    "archive",
+				Region:    "us-west-2",
+				ForcePath: &forcePathStyle,
+			},
+		},
+		{
+			name: "s3 explicit credentials",
+			source: config.Backend{
+				Backend:  config.BackendS3,
+				Endpoint: "https://s3.example.com",
+				Bucket:   "reports",
+				Region:   config.DefaultS3Region,
+				Creds: config.Credentials{
+					AccessKeyIDEnv:     "ACCESS_KEY_ID",
+					SecretAccessKeyEnv: "SECRET_ACCESS_KEY",
+				},
+			},
+			destination: config.Destination{
+				Backend:  config.BackendS3,
+				Endpoint: "https://s3.example.com",
+				Bucket:   "reports",
+				Region:   config.DefaultS3Region,
+				Creds: config.Credentials{
+					AccessKeyIDEnv:     "ACCESS_KEY_ID",
+					SecretAccessKeyEnv: "SECRET_ACCESS_KEY",
+				},
+			},
+		},
+	}
+
+	factory := &backendFactory{
+		environment: config.NewEnvironment(map[string]string{
+			"ACCESS_KEY_ID":     "secret-access",
+			"SECRET_ACCESS_KEY": "secret-secret",
+		}, func(string) (string, bool) { return "", false }),
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			sourceConfig, err := factory.openConfig(backendOpenSpecFromSource(tt.source))
+			if err != nil {
+				t.Fatalf("source openConfig() error = %v", err)
+			}
+			destinationConfig, err := factory.openConfig(backendOpenSpecFromDestination(tt.destination))
+			if err != nil {
+				t.Fatalf("destination openConfig() error = %v", err)
+			}
+			if !openConfigEqual(sourceConfig, destinationConfig) {
+				t.Fatalf("source open config = %#v, destination open config = %#v, want equivalent", sourceConfig, destinationConfig)
+			}
+		})
+	}
+}
+
+func TestBackendFactoryBuildsEquivalentDryRunSSHOpenConfig(t *testing.T) {
+	factory := &backendFactory{readOnlyKnownHosts: true}
+	sourceConfig, err := factory.openConfig(backendOpenSpecFromSource(config.Backend{
+		Backend: config.BackendSSH,
+		Host:    "reports.example.com",
+		Port:    22,
+		Path:    "/reports",
+		SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyAcceptNew},
+	}))
+	if err != nil {
+		t.Fatalf("source openConfig() error = %v", err)
+	}
+	destinationConfig, err := factory.openConfig(backendOpenSpecFromDestination(config.Destination{
+		Backend: config.BackendSSH,
+		Host:    "reports.example.com",
+		Port:    22,
+		Path:    "/reports",
+		SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyAcceptNew},
+	}))
+	if err != nil {
+		t.Fatalf("destination openConfig() error = %v", err)
+	}
+	if !openConfigEqual(sourceConfig, destinationConfig) {
+		t.Fatalf("source open config = %#v, destination open config = %#v, want equivalent", sourceConfig, destinationConfig)
+	}
+	if sourceConfig[sshReadOnlyHostsKey] != "true" {
+		t.Fatalf("open config %s = %q, want true", sshReadOnlyHostsKey, sourceConfig[sshReadOnlyHostsKey])
+	}
+}
+
+func TestConfiguredSourceValidationAndRunUseEquivalentSourceOpenConfig(t *testing.T) {
+	tests := []struct {
+		name       string
+		source     config.Backend
+		sourceKey  string
+		dest       config.Destination
+		destKey    string
+		wantFields map[string]string
+	}{
+		{
+			name: "s3",
+			source: config.Backend{
+				Backend:  config.BackendS3,
+				Endpoint: "https://s3.example.com",
+				Bucket:   "source-bucket",
+				Prefix:   "source-prefix",
+				Region:   config.DefaultS3Region,
+			},
+			sourceKey: "s3:source-bucket",
+			dest: config.Destination{
+				ID:       "archive",
+				Backend:  config.BackendS3,
+				Endpoint: "https://s3.example.com",
+				Bucket:   "destination-bucket",
+				Region:   config.DefaultS3Region,
+			},
+			destKey: "s3:destination-bucket",
+			wantFields: map[string]string{
+				s3EndpointKey:       "https://s3.example.com",
+				s3BucketKey:         "source-bucket",
+				s3PrefixKey:         "source-prefix",
+				s3RegionKey:         config.DefaultS3Region,
+				s3ForcePathStyleKey: "true",
+			},
+		},
+		{
+			name: "ssh",
+			source: config.Backend{
+				Backend: config.BackendSSH,
+				Host:    "ssh.example.com",
+				User:    "reports",
+				Port:    2222,
+				Path:    "/source",
+				SSH: config.SSH{
+					KeyFile:       "/home/reports/.ssh/id_ed25519",
+					KnownHosts:    "/home/reports/.ssh/known_hosts",
+					HostKeyPolicy: config.HostKeyPolicyStrict,
+				},
+			},
+			sourceKey: "ssh:/source",
+			dest: config.Destination{
+				ID:      "archive",
+				Backend: config.BackendSSH,
+				Host:    "ssh.example.com",
+				Port:    2222,
+				Path:    "/destination",
+				SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyStrict},
+			},
+			destKey: "ssh:/destination",
+			wantFields: map[string]string{
+				storagePathKey:      "/source",
+				sshHostKey:          "ssh.example.com",
+				sshUserKey:          "reports",
+				sshPortKey:          "2222",
+				sshKeyFileKey:       "/home/reports/.ssh/id_ed25519",
+				sshKnownHostsKey:    "/home/reports/.ssh/known_hosts",
+				sshHostKeyPolicyKey: "strict",
+			},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			sourceBackend := fake.New()
+			testutil.WriteFakeSourceBundle(t, sourceBackend, "", testutil.BundleOptions{ID: "reports.source"})
+			destinationBackend := fake.New()
+			var validateSourceConfig storage.OpenConfig
+			var runSourceConfig storage.OpenConfig
+			validateProvider := recordingBackendFactoryProvider(t, map[string]storage.Backend{
+				tt.sourceKey: sourceBackend,
+				tt.destKey:   destinationBackend,
+			}, func(cfg storage.OpenConfig) {
+				validateSourceConfig = cfg
+			})
+			runProvider := recordingBackendFactoryProvider(t, map[string]storage.Backend{
+				tt.sourceKey: sourceBackend,
+				tt.destKey:   destinationBackend,
+			}, func(cfg storage.OpenConfig) {
+				runSourceConfig = cfg
+			})
+			cfg := config.Config{Pipelines: []config.Pipeline{{
+				ID:           "reports",
+				Source:       tt.source,
+				Destinations: []config.Destination{tt.dest},
+			}}}
+			config.ApplyDefaults(&cfg)
+
+			var validateOutput bytes.Buffer
+			if err := validateConfigWithBackendFactory(context.Background(), cfg, ValidateOptions{
+				PipelineID: "reports",
+				Stdout:     &validateOutput,
+			}, validateProvider); err != nil {
+				t.Fatalf("validateConfigWithBackendFactory() error = %v", err)
+			}
+			if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{}, runProvider); err != nil {
+				t.Fatalf("runConfigWithBackendFactory() error = %v", err)
+			}
+			if !openConfigEqual(validateSourceConfig, runSourceConfig) {
+				t.Fatalf("validate source config = %#v, run source config = %#v, want equivalent", validateSourceConfig, runSourceConfig)
+			}
+			assertOpenConfig(t, runSourceConfig, tt.wantFields)
+		})
+	}
+}
+
 func assertOpenConfig(t *testing.T, got map[string]string, want map[string]string) {
 	t.Helper()
 	for key, wantValue := range want {
@@ -328,3 +590,49 @@ func assertOpenConfig(t *testing.T, got map[string]string, want map[string]strin
 		}
 	}
 }
+
+func openConfigEqual(left, right storage.OpenConfig) bool {
+	if len(left) != len(right) {
+		return false
+	}
+	for key, leftValue := range left {
+		if right[key] != leftValue {
+			return false
+		}
+	}
+	return true
+}
+
+func recordingBackendFactoryProvider(t *testing.T, remoteBackends map[string]storage.Backend, recordSource func(storage.OpenConfig)) backendFactoryProvider {
+	t.Helper()
+	return func(environment config.Environment) *backendFactory {
+		registry := storage.NewRegistry()
+		if err := registry.Register(config.BackendS3, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+			if cfg[s3BucketKey] == "source-bucket" {
+				recordSource(cfg)
+			}
+			key := "s3:" + cfg[s3BucketKey]
+			backend := remoteBackends[key]
+			if backend == nil {
+				return nil, fmt.Errorf("missing fake backend for %s", key)
+			}
+			return backend, nil
+		}); err != nil {
+			t.Fatalf("register s3 backend: %v", err)
+		}
+		if err := registry.Register(config.BackendSSH, func(ctx context.Context, cfg storage.OpenConfig) (storage.Backend, error) {
+			if cfg[storagePathKey] == "/source" {
+				recordSource(cfg)
+			}
+			key := "ssh:" + cfg[storagePathKey]
+			backend := remoteBackends[key]
+			if backend == nil {
+				return nil, fmt.Errorf("missing fake backend for %s", key)
+			}
+			return backend, nil
+		}); err != nil {
+			t.Fatalf("register ssh backend: %v", err)
+		}
+		return &backendFactory{registry: registry, environment: environment}
+	}
+}

internal/app/run.go

@@ -2,11 +2,8 @@ package app
 
 import (
 	"context"
-	"errors"
 	"fmt"
 	"io"
-	"sort"
-	"strings"
 
 	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
 	"gitea.maximumdirect.net/eric/distributor/internal/config"
@@ -260,431 +257,3 @@ func closeBackend(backend storage.Backend) {
 	}
 	_ = closeable.Close()
 }
-
-func writePlanLine(w io.Writer, backend string, plan publish.Plan, planErr error) {
-	if w == nil {
-		return
-	}
-	if planErr != nil {
-		destinationID := plan.DestinationID
-		if destinationID == "" {
-			destinationID = "unknown"
-		}
-		fmt.Fprintf(w, "  - bundle=%s destination=%s backend=%s%s action=error reason=%q\n", storage.DisplayPath(plan.BundlePath), destinationID, backend, pathMappingSummary(plan), planErr.Error())
-		return
-	}
-	fmt.Fprintf(w, "  - bundle=%s destination=%s backend=%s%s action=%s outputs=%s reason=%q\n", storage.DisplayPath(plan.BundlePath), plan.DestinationID, backend, pathMappingSummary(plan), plan.Action, outputSummary(plan.Outputs), plan.Reason)
-}
-
-func pathMappingSummary(plan publish.Plan) string {
-	if plan.PathMapping != config.PathMappingFixed {
-		return ""
-	}
-	return fmt.Sprintf(" path_mapping=fixed target=%s", storage.DisplayPath(plan.DestinationBundlePath))
-}
-
-func writeErrorLine(w io.Writer, bundlePath, destinationID, backend string, err error) {
-	if w == nil {
-		return
-	}
-	fmt.Fprintf(w, "  - bundle=%s destination=%s backend=%s action=error reason=%q\n", storage.DisplayPath(bundlePath), destinationID, backend, err.Error())
-}
-
-func outputSummary(outputs []publish.Output) string {
-	if len(outputs) == 0 {
-		return "none"
-	}
-	paths := make([]string, 0, len(outputs))
-	for _, output := range outputs {
-		paths = append(paths, output.DestinationPath)
-	}
-	return strings.Join(paths, ",")
-}
-
-type destinationBundleSelection struct {
-	SourceBundle          bundle.Bundle
-	DestinationBundlePath string
-}
-
-func selectDestinationBundles(destination config.Destination, bundles []bundle.Bundle) []destinationBundleSelection {
-	if !isFixedPathDestination(destination) {
-		selections := make([]destinationBundleSelection, 0, len(bundles))
-		for _, sourceBundle := range bundles {
-			selections = append(selections, destinationBundleSelection{
-				SourceBundle:          sourceBundle,
-				DestinationBundlePath: sourceBundle.RootRelativePath,
-			})
-		}
-		return selections
-	}
-	if len(bundles) == 0 {
-		return nil
-	}
-	sourceBundle := newestBundle(bundles)
-	return []destinationBundleSelection{{
-		SourceBundle:          sourceBundle,
-		DestinationBundlePath: "",
-	}}
-}
-
-func newestBundle(bundles []bundle.Bundle) bundle.Bundle {
-	if len(bundles) == 0 {
-		return bundle.Bundle{}
-	}
-	sorted := append([]bundle.Bundle(nil), bundles...)
-	sort.Slice(sorted, func(i, j int) bool {
-		if sorted[i].Manifest.Created.Equal(sorted[j].Manifest.Created) {
-			return sorted[i].RootRelativePath < sorted[j].RootRelativePath
-		}
-		return sorted[i].Manifest.Created.After(sorted[j].Manifest.Created)
-	})
-	return sorted[0]
-}
-
-func isFixedPathDestination(destination config.Destination) bool {
-	return destination.PathMap.Mode == config.PathMappingFixed
-}
-
-func fixedPathSelectionWarning(pipelineID, destinationID string, selections []destinationBundleSelection, candidateCount int) OutputWarning {
-	selected := "none"
-	if len(selections) > 0 {
-		selected = storage.DisplayPath(selections[0].SourceBundle.RootRelativePath)
-	}
-	return OutputWarning{Message: fmt.Sprintf("pipeline=%s destination=%s path_mapping=fixed candidates=%d selected_bundle=%s destination_bundle=.", pipelineID, destinationID, candidateCount, selected)}
-}
-
-func isDestructiveFixedPathAction(action publish.Action) bool {
-	return action == publish.ActionReplaceOlder || action == publish.ActionForceReplace
-}
-
-func fixedPathReplacementWarning(plan publish.Plan) OutputWarning {
-	return OutputWarning{Message: fmt.Sprintf("pipeline=%s destination=%s path_mapping=fixed action=%s replaces destination root for selected_bundle=%s", plan.PipelineID, plan.DestinationID, plan.Action, storage.DisplayPath(plan.BundlePath))}
-}
-
-func destinationIDs(destinations []config.Destination) []string {
-	ids := make([]string, 0, len(destinations))
-	for _, destination := range destinations {
-		ids = append(ids, destination.ID)
-	}
-	return ids
-}
-
-func destinationSummary(destinations []config.Destination) string {
-	if len(destinations) == 0 {
-		return "none"
-	}
-	ids := make([]string, 0, len(destinations))
-	for _, destination := range destinations {
-		ids = append(ids, destination.ID)
-	}
-	return strings.Join(ids, ",")
-}
-
-func writeSecretConflictWarnings(w io.Writer, conflicts []config.SecretConflict) error {
-	return writeWarnings(w, secretConflictWarnings(conflicts))
-}
-
-func secretConflictWarnings(conflicts []config.SecretConflict) []OutputWarning {
-	warnings := make([]OutputWarning, 0, len(conflicts))
-	for _, conflict := range conflicts {
-		warnings = append(warnings, OutputWarning{
-			Message: fmt.Sprintf("secret %s ignored because the real environment already has that variable", conflict.Name),
-		})
-	}
-	return warnings
-}
-
-func writeSSHWarnings(w io.Writer, pipeline config.Pipeline) error {
-	return writeWarnings(w, sshWarnings(pipeline))
-}
-
-func sshWarnings(pipeline config.Pipeline) []OutputWarning {
-	var warnings []OutputWarning
-	if pipeline.Source.Backend == config.BackendSSH && pipeline.Source.SSH.HostKeyPolicy == config.HostKeyPolicyOff {
-		warnings = append(warnings, OutputWarning{
-			Message: fmt.Sprintf("pipeline=%s source host_key_policy=off disables SSH host key checking", pipeline.ID),
-		})
-	}
-	for _, destination := range pipeline.Destinations {
-		if destination.Backend == config.BackendSSH && destination.SSH.HostKeyPolicy == config.HostKeyPolicyOff {
-			warnings = append(warnings, OutputWarning{
-				Message: fmt.Sprintf("pipeline=%s destination=%s host_key_policy=off disables SSH host key checking", pipeline.ID, destination.ID),
-			})
-		}
-	}
-	return warnings
-}
-
-func writeWarnings(w io.Writer, warnings []OutputWarning) error {
-	if w == nil {
-		return nil
-	}
-	for _, warning := range warnings {
-		if _, err := fmt.Fprintf(w, "Warning: %s\n", warning.Message); err != nil {
-			return err
-		}
-	}
-	return nil
-}
-
-func shouldNotify(action publish.Action) bool {
-	return action == publish.ActionPublishNew || action == publish.ActionReplaceOlder || action == publish.ActionForceReplace
-}
-
-func notifyEvent(plan publish.Plan) notify.Event {
-	outputs := make([]notify.Output, 0, len(plan.Outputs))
-	for _, output := range plan.Outputs {
-		outputs = append(outputs, notify.Output{
-			Path:       output.DestinationPath,
-			Kind:       output.Kind,
-			SourcePath: output.SourcePath,
-			Transform:  output.Transform,
-			SHA256:     output.SHA256,
-			Size:       output.Size,
-		})
-	}
-	return notify.Event{
-		PipelineID:    plan.PipelineID,
-		DestinationID: plan.DestinationID,
-		BundleID:      plan.BundleID,
-		BundlePath:    plan.BundlePath,
-		Action:        string(plan.Action),
-		Outputs:       outputs,
-	}
-}
-
-type runResult struct {
-	DryRun    bool                `json:"dry_run"`
-	Pipelines []runPipelineResult `json:"pipelines"`
-	Actions   []runActionResult   `json:"actions"`
-	Summary   runSummaryResult    `json:"summary"`
-}
-
-type runPipelineResult struct {
-	ID            string   `json:"id"`
-	SourceBackend string   `json:"source_backend"`
-	BundleCount   int      `json:"bundle_count"`
-	Destinations  []string `json:"destinations"`
-}
-
-type runActionResult struct {
-	PipelineID      string            `json:"pipeline_id,omitempty"`
-	DestinationID   string            `json:"destination_id"`
-	Backend         string            `json:"backend"`
-	BundleID        string            `json:"bundle_id,omitempty"`
-	BundlePath      string            `json:"bundle_path"`
-	DestinationPath string            `json:"destination_path"`
-	PathMapping     string            `json:"path_mapping,omitempty"`
-	Action          string            `json:"action"`
-	PrimaryURL      string            `json:"primary_url,omitempty"`
-	Reason          string            `json:"reason,omitempty"`
-	Outputs         []runOutputResult `json:"outputs"`
-}
-
-type runOutputResult struct {
-	Path       string `json:"path"`
-	Kind       string `json:"kind"`
-	SourcePath string `json:"source_path,omitempty"`
-	Transform  string `json:"transform,omitempty"`
-	URL        string `json:"url,omitempty"`
-	SHA256     string `json:"sha256"`
-	Size       int64  `json:"size"`
-}
-
-func runActionFromPlan(backend string, plan publish.Plan, planErr error) runActionResult {
-	if planErr != nil {
-		destinationID := plan.DestinationID
-		if destinationID == "" {
-			destinationID = "unknown"
-		}
-		return runActionResult{
-			PipelineID:      plan.PipelineID,
-			DestinationID:   destinationID,
-			Backend:         backend,
-			BundleID:        plan.BundleID,
-			BundlePath:      storage.DisplayPath(plan.BundlePath),
-			DestinationPath: storage.DisplayPath(plan.DestinationBundlePath),
-			PathMapping:     plan.PathMapping,
-			Action:          "error",
-			PrimaryURL:      plan.PrimaryURL,
-			Reason:          planErr.Error(),
-			Outputs:         []runOutputResult{},
-		}
-	}
-	return runActionResult{
-		PipelineID:      plan.PipelineID,
-		DestinationID:   plan.DestinationID,
-		Backend:         backend,
-		BundleID:        plan.BundleID,
-		BundlePath:      storage.DisplayPath(plan.BundlePath),
-		DestinationPath: storage.DisplayPath(plan.DestinationBundlePath),
-		PathMapping:     plan.PathMapping,
-		Action:          string(plan.Action),
-		PrimaryURL:      plan.PrimaryURL,
-		Reason:          plan.Reason,
-		Outputs:         runOutputsFromPlan(plan.Outputs),
-	}
-}
-
-func errorAction(pipelineID, destinationID, backend, bundlePath string, err error) runActionResult {
-	return runActionResult{
-		PipelineID:      pipelineID,
-		DestinationID:   destinationID,
-		Backend:         backend,
-		BundlePath:      storage.DisplayPath(bundlePath),
-		DestinationPath: storage.DisplayPath(bundlePath),
-		Action:          "error",
-		Reason:          err.Error(),
-		Outputs:         []runOutputResult{},
-	}
-}
-
-func runOutputsFromPlan(outputs []publish.Output) []runOutputResult {
-	results := make([]runOutputResult, 0, len(outputs))
-	for _, output := range outputs {
-		results = append(results, runOutputResult{
-			Path:       output.DestinationPath,
-			Kind:       output.Kind,
-			SourcePath: output.SourcePath,
-			Transform:  output.Transform,
-			URL:        output.URL,
-			SHA256:     output.SHA256,
-			Size:       output.Size,
-		})
-	}
-	return results
-}
-
-type runSummary struct {
-	dryRun       bool
-	planned      int
-	publishNew   int
-	replaceOlder int
-	forceReplace int
-	skipped      int
-	failures     int
-	fixedPath    int
-}
-
-func (s *runSummary) recordPlan(action publish.Action) {
-	s.planned++
-	switch action {
-	case publish.ActionPublishNew:
-		s.publishNew++
-	case publish.ActionReplaceOlder:
-		s.replaceOlder++
-	case publish.ActionForceReplace:
-		s.forceReplace++
-	case publish.ActionSkipSame, publish.ActionSkipDestinationNewer:
-		s.skipped++
-	}
-}
-
-func (s *runSummary) recordFailure() {
-	s.failures++
-}
-
-func (s *runSummary) recordFixedPath() {
-	s.fixedPath++
-}
-
-func (s runSummary) Line() string {
-	status := "ok"
-	if s.failures > 0 {
-		status = "failed"
-	}
-	return fmt.Sprintf("Final status: %s planned=%d publish_new=%d replace_older=%d force_replace=%d skipped=%d failed=%d dry_run=%t fixed_path=%d", status, s.planned, s.publishNew, s.replaceOlder, s.forceReplace, s.skipped, s.failures, s.dryRun, s.fixedPath)
-}
-
-type runSummaryResult struct {
-	Status       string `json:"status"`
-	Planned      int    `json:"planned"`
-	PublishNew   int    `json:"publish_new"`
-	ReplaceOlder int    `json:"replace_older"`
-	ForceReplace int    `json:"force_replace"`
-	Skipped      int    `json:"skipped"`
-	Failed       int    `json:"failed"`
-	DryRun       bool   `json:"dry_run"`
-	FixedPath    int    `json:"fixed_path"`
-}
-
-func (s runSummary) Result() runSummaryResult {
-	status := "ok"
-	if s.failures > 0 {
-		status = "failed"
-	}
-	return runSummaryResult{
-		Status:       status,
-		Planned:      s.planned,
-		PublishNew:   s.publishNew,
-		ReplaceOlder: s.replaceOlder,
-		ForceReplace: s.forceReplace,
-		Skipped:      s.skipped,
-		Failed:       s.failures,
-		DryRun:       s.dryRun,
-		FixedPath:    s.fixedPath,
-	}
-}
-
-type runFailure struct {
-	pipelineID    string
-	destinationID string
-	backend       string
-	bundlePath    string
-	err           error
-}
-
-type runFailures struct {
-	items []runFailure
-}
-
-func (f *runFailures) add(pipelineID, destinationID, backend, bundlePath string, err error) {
-	f.items = append(f.items, runFailure{
-		pipelineID:    pipelineID,
-		destinationID: destinationID,
-		backend:       backend,
-		bundlePath:    bundlePath,
-		err:           err,
-	})
-}
-
-func (f runFailures) Error() string {
-	if len(f.items) == 0 {
-		return ""
-	}
-	parts := make([]string, 0, len(f.items))
-	for _, item := range f.items {
-		parts = append(parts, fmt.Sprintf("pipeline %s destination %s backend %s bundle %s: %v", item.pipelineID, item.destinationID, item.backend, item.bundlePath, item.err))
-	}
-	return "run failed: " + strings.Join(parts, "; ")
-}
-
-func (f runFailures) outputErrors() []OutputError {
-	if len(f.items) == 0 {
-		return nil
-	}
-	errors := make([]OutputError, 0, len(f.items))
-	for _, item := range f.items {
-		errors = append(errors, OutputError{
-			PipelineID:    item.pipelineID,
-			DestinationID: item.destinationID,
-			Backend:       item.backend,
-			BundlePath:    item.bundlePath,
-			Message:       item.err.Error(),
-		})
-	}
-	return errors
-}
-
-func IsPartialResultError(err error) bool {
-	var failures runFailures
-	return errors.As(err, &failures)
-}
-
-func (f runFailures) Unwrap() error {
-	errs := make([]error, 0, len(f.items))
-	for _, item := range f.items {
-		errs = append(errs, item.err)
-	}
-	return errors.Join(errs...)
-}

internal/app/run_failures.go

@@ -0,0 +1,70 @@
+package app
+
+import (
+	"errors"
+	"fmt"
+	"strings"
+)
+
+type runFailure struct {
+	pipelineID    string
+	destinationID string
+	backend       string
+	bundlePath    string
+	err           error
+}
+
+type runFailures struct {
+	items []runFailure
+}
+
+func (f *runFailures) add(pipelineID, destinationID, backend, bundlePath string, err error) {
+	f.items = append(f.items, runFailure{
+		pipelineID:    pipelineID,
+		destinationID: destinationID,
+		backend:       backend,
+		bundlePath:    bundlePath,
+		err:           err,
+	})
+}
+
+func (f runFailures) Error() string {
+	if len(f.items) == 0 {
+		return ""
+	}
+	parts := make([]string, 0, len(f.items))
+	for _, item := range f.items {
+		parts = append(parts, fmt.Sprintf("pipeline %s destination %s backend %s bundle %s: %v", item.pipelineID, item.destinationID, item.backend, item.bundlePath, item.err))
+	}
+	return "run failed: " + strings.Join(parts, "; ")
+}
+
+func (f runFailures) outputErrors() []OutputError {
+	if len(f.items) == 0 {
+		return nil
+	}
+	errors := make([]OutputError, 0, len(f.items))
+	for _, item := range f.items {
+		errors = append(errors, OutputError{
+			PipelineID:    item.pipelineID,
+			DestinationID: item.destinationID,
+			Backend:       item.backend,
+			BundlePath:    item.bundlePath,
+			Message:       item.err.Error(),
+		})
+	}
+	return errors
+}
+
+func IsPartialResultError(err error) bool {
+	var failures runFailures
+	return errors.As(err, &failures)
+}
+
+func (f runFailures) Unwrap() error {
+	errs := make([]error, 0, len(f.items))
+	for _, item := range f.items {
+		errs = append(errs, item.err)
+	}
+	return errors.Join(errs...)
+}

internal/app/run_notify.go

@@ -0,0 +1,33 @@
+package app
+
+import (
+	"gitea.maximumdirect.net/eric/distributor/internal/notify"
+	"gitea.maximumdirect.net/eric/distributor/internal/publish"
+)
+
+func shouldNotify(action publish.Action) bool {
+	return action == publish.ActionPublishNew || action == publish.ActionReplaceOlder || action == publish.ActionForceReplace
+}
+
+func notifyEvent(plan publish.Plan) notify.Event {
+	outputs := make([]notify.Output, 0, len(plan.Outputs))
+	for _, output := range plan.Outputs {
+		stateOutput := output.StateOutputFile()
+		outputs = append(outputs, notify.Output{
+			Path:       stateOutput.Path,
+			Kind:       stateOutput.Kind,
+			SourcePath: stateOutput.SourcePath,
+			Transform:  stateOutput.Transform,
+			SHA256:     stateOutput.SHA256,
+			Size:       stateOutput.Size,
+		})
+	}
+	return notify.Event{
+		PipelineID:    plan.PipelineID,
+		DestinationID: plan.DestinationID,
+		BundleID:      plan.BundleID,
+		BundlePath:    plan.BundlePath,
+		Action:        string(plan.Action),
+		Outputs:       outputs,
+	}
+}

internal/app/run_output.go

@@ -0,0 +1,154 @@
+package app
+
+import (
+	"fmt"
+	"io"
+	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/publish"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+)
+
+func writePlanLine(w io.Writer, backend string, plan publish.Plan, planErr error) {
+	if w == nil {
+		return
+	}
+	if planErr != nil {
+		destinationID := plan.DestinationID
+		if destinationID == "" {
+			destinationID = "unknown"
+		}
+		fmt.Fprintf(w, "  - bundle=%s destination=%s backend=%s%s action=error reason=%q\n", storage.DisplayPath(plan.BundlePath), destinationID, backend, pathMappingSummary(plan), planErr.Error())
+		return
+	}
+	fmt.Fprintf(w, "  - bundle=%s destination=%s backend=%s%s action=%s outputs=%s reason=%q\n", storage.DisplayPath(plan.BundlePath), plan.DestinationID, backend, pathMappingSummary(plan), plan.Action, outputSummary(plan.Outputs), plan.Reason)
+}
+
+func pathMappingSummary(plan publish.Plan) string {
+	if plan.PathMapping != config.PathMappingFixed {
+		return ""
+	}
+	return fmt.Sprintf(" path_mapping=fixed target=%s", storage.DisplayPath(plan.DestinationBundlePath))
+}
+
+func writeErrorLine(w io.Writer, bundlePath, destinationID, backend string, err error) {
+	if w == nil {
+		return
+	}
+	fmt.Fprintf(w, "  - bundle=%s destination=%s backend=%s action=error reason=%q\n", storage.DisplayPath(bundlePath), destinationID, backend, err.Error())
+}
+
+func outputSummary(outputs []publish.Output) string {
+	if len(outputs) == 0 {
+		return "none"
+	}
+	paths := make([]string, 0, len(outputs))
+	for _, output := range outputs {
+		paths = append(paths, output.DestinationPath)
+	}
+	return strings.Join(paths, ",")
+}
+
+type runResult struct {
+	DryRun    bool                `json:"dry_run"`
+	Pipelines []runPipelineResult `json:"pipelines"`
+	Actions   []runActionResult   `json:"actions"`
+	Summary   runSummaryResult    `json:"summary"`
+}
+
+type runPipelineResult struct {
+	ID            string   `json:"id"`
+	SourceBackend string   `json:"source_backend"`
+	BundleCount   int      `json:"bundle_count"`
+	Destinations  []string `json:"destinations"`
+}
+
+type runActionResult struct {
+	PipelineID      string            `json:"pipeline_id,omitempty"`
+	DestinationID   string            `json:"destination_id"`
+	Backend         string            `json:"backend"`
+	BundleID        string            `json:"bundle_id,omitempty"`
+	BundlePath      string            `json:"bundle_path"`
+	DestinationPath string            `json:"destination_path"`
+	PathMapping     string            `json:"path_mapping,omitempty"`
+	Action          string            `json:"action"`
+	PrimaryURL      string            `json:"primary_url,omitempty"`
+	Reason          string            `json:"reason,omitempty"`
+	Outputs         []runOutputResult `json:"outputs"`
+}
+
+type runOutputResult struct {
+	Path       string `json:"path"`
+	Kind       string `json:"kind"`
+	SourcePath string `json:"source_path,omitempty"`
+	Transform  string `json:"transform,omitempty"`
+	URL        string `json:"url,omitempty"`
+	SHA256     string `json:"sha256"`
+	Size       int64  `json:"size"`
+}
+
+func runActionFromPlan(backend string, plan publish.Plan, planErr error) runActionResult {
+	if planErr != nil {
+		destinationID := plan.DestinationID
+		if destinationID == "" {
+			destinationID = "unknown"
+		}
+		return runActionResult{
+			PipelineID:      plan.PipelineID,
+			DestinationID:   destinationID,
+			Backend:         backend,
+			BundleID:        plan.BundleID,
+			BundlePath:      storage.DisplayPath(plan.BundlePath),
+			DestinationPath: storage.DisplayPath(plan.DestinationBundlePath),
+			PathMapping:     plan.PathMapping,
+			Action:          "error",
+			PrimaryURL:      plan.PrimaryURL,
+			Reason:          planErr.Error(),
+			Outputs:         []runOutputResult{},
+		}
+	}
+	return runActionResult{
+		PipelineID:      plan.PipelineID,
+		DestinationID:   plan.DestinationID,
+		Backend:         backend,
+		BundleID:        plan.BundleID,
+		BundlePath:      storage.DisplayPath(plan.BundlePath),
+		DestinationPath: storage.DisplayPath(plan.DestinationBundlePath),
+		PathMapping:     plan.PathMapping,
+		Action:          string(plan.Action),
+		PrimaryURL:      plan.PrimaryURL,
+		Reason:          plan.Reason,
+		Outputs:         runOutputsFromPlan(plan.Outputs),
+	}
+}
+
+func errorAction(pipelineID, destinationID, backend, bundlePath string, err error) runActionResult {
+	return runActionResult{
+		PipelineID:      pipelineID,
+		DestinationID:   destinationID,
+		Backend:         backend,
+		BundlePath:      storage.DisplayPath(bundlePath),
+		DestinationPath: storage.DisplayPath(bundlePath),
+		Action:          "error",
+		Reason:          err.Error(),
+		Outputs:         []runOutputResult{},
+	}
+}
+
+func runOutputsFromPlan(outputs []publish.Output) []runOutputResult {
+	results := make([]runOutputResult, 0, len(outputs))
+	for _, output := range outputs {
+		stateOutput := output.StateOutputFile()
+		results = append(results, runOutputResult{
+			Path:       stateOutput.Path,
+			Kind:       stateOutput.Kind,
+			SourcePath: stateOutput.SourcePath,
+			Transform:  stateOutput.Transform,
+			URL:        stateOutput.URL,
+			SHA256:     stateOutput.SHA256,
+			Size:       stateOutput.Size,
+		})
+	}
+	return results
+}

internal/app/run_selection.go

@@ -0,0 +1,91 @@
+package app
+
+import (
+	"fmt"
+	"sort"
+	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/publish"
+	"gitea.maximumdirect.net/eric/distributor/internal/storage"
+)
+
+type destinationBundleSelection struct {
+	SourceBundle          bundle.Bundle
+	DestinationBundlePath string
+}
+
+func selectDestinationBundles(destination config.Destination, bundles []bundle.Bundle) []destinationBundleSelection {
+	if !isFixedPathDestination(destination) {
+		selections := make([]destinationBundleSelection, 0, len(bundles))
+		for _, sourceBundle := range bundles {
+			selections = append(selections, destinationBundleSelection{
+				SourceBundle:          sourceBundle,
+				DestinationBundlePath: sourceBundle.RootRelativePath,
+			})
+		}
+		return selections
+	}
+	if len(bundles) == 0 {
+		return nil
+	}
+	sourceBundle := newestBundle(bundles)
+	return []destinationBundleSelection{{
+		SourceBundle:          sourceBundle,
+		DestinationBundlePath: "",
+	}}
+}
+
+func newestBundle(bundles []bundle.Bundle) bundle.Bundle {
+	if len(bundles) == 0 {
+		return bundle.Bundle{}
+	}
+	sorted := append([]bundle.Bundle(nil), bundles...)
+	sort.Slice(sorted, func(i, j int) bool {
+		if sorted[i].Manifest.Created.Equal(sorted[j].Manifest.Created) {
+			return sorted[i].RootRelativePath < sorted[j].RootRelativePath
+		}
+		return sorted[i].Manifest.Created.After(sorted[j].Manifest.Created)
+	})
+	return sorted[0]
+}
+
+func isFixedPathDestination(destination config.Destination) bool {
+	return destination.PathMap.Mode == config.PathMappingFixed
+}
+
+func fixedPathSelectionWarning(pipelineID, destinationID string, selections []destinationBundleSelection, candidateCount int) OutputWarning {
+	selected := "none"
+	if len(selections) > 0 {
+		selected = storage.DisplayPath(selections[0].SourceBundle.RootRelativePath)
+	}
+	return OutputWarning{Message: fmt.Sprintf("pipeline=%s destination=%s path_mapping=fixed candidates=%d selected_bundle=%s destination_bundle=.", pipelineID, destinationID, candidateCount, selected)}
+}
+
+func isDestructiveFixedPathAction(action publish.Action) bool {
+	return action == publish.ActionReplaceOlder || action == publish.ActionForceReplace
+}
+
+func fixedPathReplacementWarning(plan publish.Plan) OutputWarning {
+	return OutputWarning{Message: fmt.Sprintf("pipeline=%s destination=%s path_mapping=fixed action=%s replaces destination root for selected_bundle=%s", plan.PipelineID, plan.DestinationID, plan.Action, storage.DisplayPath(plan.BundlePath))}
+}
+
+func destinationIDs(destinations []config.Destination) []string {
+	ids := make([]string, 0, len(destinations))
+	for _, destination := range destinations {
+		ids = append(ids, destination.ID)
+	}
+	return ids
+}
+
+func destinationSummary(destinations []config.Destination) string {
+	if len(destinations) == 0 {
+		return "none"
+	}
+	ids := make([]string, 0, len(destinations))
+	for _, destination := range destinations {
+		ids = append(ids, destination.ID)
+	}
+	return strings.Join(ids, ",")
+}

internal/app/run_summary.go

@@ -0,0 +1,78 @@
+package app
+
+import (
+	"fmt"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/publish"
+)
+
+type runSummary struct {
+	dryRun       bool
+	planned      int
+	publishNew   int
+	replaceOlder int
+	forceReplace int
+	skipped      int
+	failures     int
+	fixedPath    int
+}
+
+func (s *runSummary) recordPlan(action publish.Action) {
+	s.planned++
+	switch action {
+	case publish.ActionPublishNew:
+		s.publishNew++
+	case publish.ActionReplaceOlder:
+		s.replaceOlder++
+	case publish.ActionForceReplace:
+		s.forceReplace++
+	case publish.ActionSkipSame, publish.ActionSkipDestinationNewer:
+		s.skipped++
+	}
+}
+
+func (s *runSummary) recordFailure() {
+	s.failures++
+}
+
+func (s *runSummary) recordFixedPath() {
+	s.fixedPath++
+}
+
+func (s runSummary) Line() string {
+	status := "ok"
+	if s.failures > 0 {
+		status = "failed"
+	}
+	return fmt.Sprintf("Final status: %s planned=%d publish_new=%d replace_older=%d force_replace=%d skipped=%d failed=%d dry_run=%t fixed_path=%d", status, s.planned, s.publishNew, s.replaceOlder, s.forceReplace, s.skipped, s.failures, s.dryRun, s.fixedPath)
+}
+
+type runSummaryResult struct {
+	Status       string `json:"status"`
+	Planned      int    `json:"planned"`
+	PublishNew   int    `json:"publish_new"`
+	ReplaceOlder int    `json:"replace_older"`
+	ForceReplace int    `json:"force_replace"`
+	Skipped      int    `json:"skipped"`
+	Failed       int    `json:"failed"`
+	DryRun       bool   `json:"dry_run"`
+	FixedPath    int    `json:"fixed_path"`
+}
+
+func (s runSummary) Result() runSummaryResult {
+	status := "ok"
+	if s.failures > 0 {
+		status = "failed"
+	}
+	return runSummaryResult{
+		Status:       status,
+		Planned:      s.planned,
+		PublishNew:   s.publishNew,
+		ReplaceOlder: s.replaceOlder,
+		ForceReplace: s.forceReplace,
+		Skipped:      s.skipped,
+		Failed:       s.failures,
+		DryRun:       s.dryRun,
+		FixedPath:    s.fixedPath,
+	}
+}

internal/app/run_test.go

@@ -159,9 +159,9 @@ pipelines:
 	}
 }
 
-func TestWriteSSHWarningsReportsInsecureHostKeyPolicy(t *testing.T) {
+func TestSSHWarningsReportInsecureHostKeyPolicy(t *testing.T) {
 	var stdout bytes.Buffer
-	err := writeSSHWarnings(&stdout, config.Pipeline{
+	err := writeWarnings(&stdout, sshWarnings(config.Pipeline{
 		ID: "reports",
 		Source: config.Backend{
 			Backend: config.BackendSSH,
@@ -172,9 +172,9 @@ func TestWriteSSHWarningsReportsInsecureHostKeyPolicy(t *testing.T) {
 			Backend: config.BackendSSH,
 			SSH:     config.SSH{HostKeyPolicy: config.HostKeyPolicyOff},
 		}},
-	})
+	}))
 	if err != nil {
-		t.Fatalf("writeSSHWarnings() error = %v", err)
+		t.Fatalf("writeWarnings() error = %v", err)
 	}
 	output := stdout.String()
 	for _, want := range []string{
@@ -200,8 +200,8 @@ func TestRunPublishesNewLocalBundle(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
-	assertFile(t, filepath.Join(destinationRoot, "summary.txt"), "Summary\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "summary.txt"), "Summary\n")
 	if _, err := os.Stat(filepath.Join(destinationRoot, "manifest.json")); !os.IsNotExist(err) {
 		t.Fatalf("destination manifest stat error = %v, want not exist", err)
 	}
@@ -225,11 +225,11 @@ func TestRunExplicitPreserveRelativePathMappingMatchesDefault(t *testing.T) {
 	destinationRoot := t.TempDir()
 	writeSourceBundle(t, sourceRoot, "daily/report", testBundleOptions{})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingPreserveRelative)})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingPreserveRelative)})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFile(t, filepath.Join(destinationRoot, "daily", "report", "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "daily", "report", "report.md"), "# Report\nSunny.\n")
 	if _, err := os.Stat(filepath.Join(destinationRoot, "report.md")); !os.IsNotExist(err) {
 		t.Fatalf("root report.md stat error = %v, want not exist", err)
 	}
@@ -241,7 +241,7 @@ func TestRunRecordsLinksForNestedBundlePath(t *testing.T) {
 	writeSourceBundle(t, sourceRoot, "daily/brentwood", testBundleOptions{})
 
 	err := Run(context.Background(), RunOptions{
-		ConfigPath: writeLocalConfigWithLinks(t, sourceRoot, destinationRoot, config.PathMappingPreserveRelative, "https://reports.example.com/archive", config.LinkPrimaryAuto, true, false, ""),
+		ConfigPath: testutil.WriteLocalConfigWithLinks(t, sourceRoot, destinationRoot, config.PathMappingPreserveRelative, "https://reports.example.com/archive", config.LinkPrimaryAuto, true, false, ""),
 	})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
@@ -266,7 +266,7 @@ func TestRunRecordsLinksForFixedIndexDestination(t *testing.T) {
 	writeSourceBundle(t, sourceRoot, "newer", testBundleOptions{ID: "reports.newer", Created: testutil.DefaultCreated.Add(time.Hour)})
 
 	err := Run(context.Background(), RunOptions{
-		ConfigPath: writeLocalConfigWithLinks(t, sourceRoot, destinationRoot, config.PathMappingFixed, "https://reports.example.com/latest", config.LinkPrimaryAuto, false, true, config.TransformModeIndex),
+		ConfigPath: testutil.WriteLocalConfigWithLinks(t, sourceRoot, destinationRoot, config.PathMappingFixed, "https://reports.example.com/latest", config.LinkPrimaryAuto, false, true, config.TransformModeIndex),
 	})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
@@ -303,11 +303,11 @@ func TestRunFixedPathPublishesNewestBundleAtDestinationRoot(t *testing.T) {
 		},
 	})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nNew.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nNew.\n")
 	if _, err := os.Stat(filepath.Join(destinationRoot, "new", "report.md")); !os.IsNotExist(err) {
 		t.Fatalf("nested new report stat error = %v, want not exist", err)
 	}
@@ -337,7 +337,7 @@ func TestRunFixedPathTieBreaksByBundlePath(t *testing.T) {
 		},
 	})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
@@ -355,7 +355,7 @@ func TestRunFixedPathDryRunReportsSelection(t *testing.T) {
 	var stdout bytes.Buffer
 
 	err := Run(context.Background(), RunOptions{
-		ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
+		ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
 		DryRun:     true,
 		Stdout:     &stdout,
 	})
@@ -391,7 +391,7 @@ func TestRunFixedPathDryRunWarnsForReplacement(t *testing.T) {
 			{Path: "summary.txt", Data: "Old summary\n"},
 		},
 	})
-	configPath := writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)
+	configPath := testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)
 	if err := Run(context.Background(), RunOptions{ConfigPath: configPath}); err != nil {
 		t.Fatalf("first Run() error = %v", err)
 	}
@@ -422,7 +422,7 @@ func TestRunFixedPathDryRunWarnsForReplacement(t *testing.T) {
 			t.Fatalf("stdout = %q, want substring %q", output, want)
 		}
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nOld.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nOld.\n")
 }
 
 func TestRunFixedPathReplacesOlderManagedState(t *testing.T) {
@@ -436,11 +436,11 @@ func TestRunFixedPathReplacesOlderManagedState(t *testing.T) {
 			{Path: "summary.txt", Data: "Old summary\n"},
 		},
 	})
-	configPath := writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)
+	configPath := testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)
 	if err := Run(context.Background(), RunOptions{ConfigPath: configPath}); err != nil {
 		t.Fatalf("first Run() error = %v", err)
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nOld.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nOld.\n")
 
 	writeSourceBundle(t, sourceRoot, "new", testBundleOptions{
 		ID:      "reports.new",
@@ -454,7 +454,7 @@ func TestRunFixedPathReplacesOlderManagedState(t *testing.T) {
 	if err := Run(context.Background(), RunOptions{ConfigPath: configPath}); err != nil {
 		t.Fatalf("second Run() error = %v", err)
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nNew.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nNew.\n")
 	destinationState := readStateFile(t, filepath.Join(destinationRoot, storage.StateFileName))
 	if destinationState.Source.Manifest.ID != "reports.new" {
 		t.Fatalf("state source id = %q, want reports.new", destinationState.Source.Manifest.ID)
@@ -479,7 +479,7 @@ func TestRunFixedPathSkipsWhenDestinationStateIsNewer(t *testing.T) {
 
 	var stdout bytes.Buffer
 	err := Run(context.Background(), RunOptions{
-		ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
+		ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
 		Stdout:     &stdout,
 	})
 	if err != nil {
@@ -488,7 +488,7 @@ func TestRunFixedPathSkipsWhenDestinationStateIsNewer(t *testing.T) {
 	if !strings.Contains(stdout.String(), "action=skip_destination_newer") {
 		t.Fatalf("stdout = %q, want skip_destination_newer", stdout.String())
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nExisting.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nExisting.\n")
 }
 
 func TestRunFixedPathFailsUnmanagedWithoutForce(t *testing.T) {
@@ -499,7 +499,7 @@ func TestRunFixedPathFailsUnmanagedWithoutForce(t *testing.T) {
 		t.Fatalf("write unmanaged file: %v", err)
 	}
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed)})
 	if err == nil || !strings.Contains(err.Error(), "fail_unmanaged") {
 		t.Fatalf("Run() error = %v, want unmanaged failure", err)
 	}
@@ -521,14 +521,14 @@ func TestRunFixedPathForceReplacementStaysWithinDestinationRoot(t *testing.T) {
 	writeSourceBundle(t, sourceRoot, "bundle", testBundleOptions{})
 
 	err := Run(context.Background(), RunOptions{
-		ConfigPath: writeLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
+		ConfigPath: testutil.WriteLocalConfigWithPathMapping(t, sourceRoot, destinationRoot, config.PathMappingFixed),
 		Force:      true,
 	})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
-	assertFile(t, filepath.Join(parent, "keep.txt"), "keep")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(parent, "keep.txt"), "keep")
 	if _, err := os.Stat(filepath.Join(destinationRoot, "unmanaged.txt")); !os.IsNotExist(err) {
 		t.Fatalf("unmanaged stat error = %v, want removed", err)
 	}
@@ -583,12 +583,12 @@ func TestRunFixedPathRemoteBackendsUseBackendRoots(t *testing.T) {
 	if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{}, provider); err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFakeFile(t, s3Destination, "report.md", "# Report\nNew.\n")
-	assertFakeFile(t, s3Destination, "summary.txt", "New summary\n")
-	assertFakeMissing(t, s3Destination, "new/report.md")
-	assertFakeFile(t, sshDestination, "report.md", "# Report\nNew.\n")
-	assertFakeFile(t, sshDestination, "summary.txt", "New summary\n")
-	assertFakeMissing(t, sshDestination, "new/report.md")
+	testutil.AssertFakeFile(t, s3Destination, "report.md", "# Report\nNew.\n")
+	testutil.AssertFakeFile(t, s3Destination, "summary.txt", "New summary\n")
+	testutil.AssertFakeMissing(t, s3Destination, "new/report.md")
+	testutil.AssertFakeFile(t, sshDestination, "report.md", "# Report\nNew.\n")
+	testutil.AssertFakeFile(t, sshDestination, "summary.txt", "New summary\n")
+	testutil.AssertFakeMissing(t, sshDestination, "new/report.md")
 }
 
 func TestRunNotifiesAfterPublication(t *testing.T) {
@@ -622,6 +622,32 @@ func TestRunNotifiesAfterPublication(t *testing.T) {
 	}
 }
 
+func TestRunNotifiesGeneratedOutputMetadata(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
+	notifier := &recordingNotifier{}
+
+	err := Run(context.Background(), RunOptions{
+		ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, false, true),
+		Notifier:   notifier,
+	})
+	if err != nil {
+		t.Fatalf("Run() error = %v", err)
+	}
+	if got, want := len(notifier.events), 1; got != want {
+		t.Fatalf("notification count = %d, want %d", got, want)
+	}
+	outputs := notifier.events[0].Outputs
+	if got, want := len(outputs), 1; got != want {
+		t.Fatalf("notification output count = %d, want %d", got, want)
+	}
+	output := outputs[0]
+	if output.Path != "report.html" || output.Kind != state.OutputKindGenerated || output.SourcePath != "report.md" || output.Transform != "markdown_to_html" || output.SHA256 == "" || output.Size <= 0 {
+		t.Fatalf("notification output = %#v", output)
+	}
+}
+
 func TestRunNotifiesAfterReplacement(t *testing.T) {
 	sourceRoot := t.TempDir()
 	destinationRoot := t.TempDir()
@@ -649,6 +675,49 @@ func TestRunNotifiesAfterReplacement(t *testing.T) {
 	}
 }
 
+func TestRunJSONIncludesGeneratedOutputMetadata(t *testing.T) {
+	sourceRoot := t.TempDir()
+	destinationRoot := t.TempDir()
+	writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
+
+	var stdout bytes.Buffer
+	err := Run(context.Background(), RunOptions{
+		ConfigPath:   testutil.WriteLocalConfigWithLinks(t, sourceRoot, destinationRoot, config.PathMappingFixed, "https://reports.example.com/latest", config.LinkPrimaryAuto, false, true, config.TransformModeIndex),
+		DryRun:       true,
+		Stdout:       &stdout,
+		OutputFormat: OutputFormatJSON,
+	})
+	if err != nil {
+		t.Fatalf("Run() error = %v", err)
+	}
+	result := decodeAppResult(t, stdout.String())
+	actions, ok := result["actions"].([]any)
+	if !ok || len(actions) != 1 {
+		t.Fatalf("actions = %#v, want one action", result["actions"])
+	}
+	action, ok := actions[0].(map[string]any)
+	if !ok {
+		t.Fatalf("action = %#v, want object", actions[0])
+	}
+	if action["primary_url"] != "https://reports.example.com/latest/" {
+		t.Fatalf("action primary_url = %#v", action["primary_url"])
+	}
+	outputs, ok := action["outputs"].([]any)
+	if !ok || len(outputs) != 1 {
+		t.Fatalf("outputs = %#v, want one output", action["outputs"])
+	}
+	output, ok := outputs[0].(map[string]any)
+	if !ok {
+		t.Fatalf("output = %#v, want object", outputs[0])
+	}
+	if output["path"] != "index.html" || output["kind"] != state.OutputKindGenerated || output["source_path"] != "report.md" || output["transform"] != "markdown_to_html" || output["url"] != "https://reports.example.com/latest/" {
+		t.Fatalf("output = %#v, want generated index metadata", output)
+	}
+	if output["sha256"] == "" || output["size"] == nil {
+		t.Fatalf("output = %#v, want digest and size", output)
+	}
+}
+
 func TestRunDoesNotNotifyForSkippedDestination(t *testing.T) {
 	sourceRoot := t.TempDir()
 	destinationRoot := t.TempDir()
@@ -720,7 +789,7 @@ func TestRunContinuesAfterDestinationFailure(t *testing.T) {
 			t.Fatalf("stdout = %q, want substring %q", output, want)
 		}
 	}
-	assertFile(t, filepath.Join(secondDestination, "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(secondDestination, "report.md"), "# Report\nSunny.\n")
 }
 
 func TestRunPublishesHTMLOnly(t *testing.T) {
@@ -728,11 +797,11 @@ func TestRunPublishesHTMLOnly(t *testing.T) {
 	destinationRoot := t.TempDir()
 	writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPolicy(t, sourceRoot, destinationRoot, false, true)})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, false, true)})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFileContains(t, filepath.Join(destinationRoot, "report.html"), "<h1>Report</h1>")
+	testutil.AssertFileContains(t, filepath.Join(destinationRoot, "report.html"), "<h1>Report</h1>")
 	if _, err := os.Stat(filepath.Join(destinationRoot, "report.md")); !os.IsNotExist(err) {
 		t.Fatalf("report.md stat error = %v, want not exist", err)
 	}
@@ -753,11 +822,11 @@ func TestRunPublishesHTMLIndexWithExplicitInput(t *testing.T) {
 		ExtraFiles: []testFile{{Path: "notes.md", Data: "# Notes\nHidden.\n"}},
 	})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "report.md")})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "report.md")})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<h1>Report</h1>")
+	testutil.AssertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<h1>Report</h1>")
 	if _, err := os.Stat(filepath.Join(destinationRoot, "report.html")); !os.IsNotExist(err) {
 		t.Fatalf("report.html stat error = %v, want not exist", err)
 	}
@@ -776,11 +845,11 @@ func TestRunPublishesHTMLIndexWithSingleMarkdownFallback(t *testing.T) {
 	destinationRoot := t.TempDir()
 	writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<h1>Report</h1>")
+	testutil.AssertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<h1>Report</h1>")
 }
 
 func TestRunFailsIndexModeWithAmbiguousMarkdownInput(t *testing.T) {
@@ -790,7 +859,7 @@ func TestRunFailsIndexModeWithAmbiguousMarkdownInput(t *testing.T) {
 		ExtraFiles: []testFile{{Path: "notes.md", Data: "# Notes\n"}},
 	})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")})
 	if err == nil || !strings.Contains(err.Error(), "multiple markdown source files") {
 		t.Fatalf("Run() error = %v, want ambiguous input error", err)
 	}
@@ -804,13 +873,13 @@ func TestRunPublishesSourceAndHTML(t *testing.T) {
 	destinationRoot := t.TempDir()
 	writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPolicy(t, sourceRoot, destinationRoot, true, true)})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, true, true)})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
-	assertFileContains(t, filepath.Join(destinationRoot, "report.html"), "<p>Sunny.</p>")
-	assertFile(t, filepath.Join(destinationRoot, "summary.txt"), "Summary\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFileContains(t, filepath.Join(destinationRoot, "report.html"), "<p>Sunny.</p>")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "summary.txt"), "Summary\n")
 	destinationState := readStateFile(t, filepath.Join(destinationRoot, storage.StateFileName))
 	if got, want := len(destinationState.Outputs), 3; got != want {
 		t.Fatalf("state output count = %d, want %d", got, want)
@@ -827,7 +896,7 @@ func TestRunDoesNotMutateSourceBundle(t *testing.T) {
 		t.Fatalf("read source before: %v", err)
 	}
 
-	err = Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPolicy(t, sourceRoot, destinationRoot, true, true)})
+	err = Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, true, true)})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
@@ -845,7 +914,7 @@ func TestRunFailsOnOutputPathCollision(t *testing.T) {
 	destinationRoot := t.TempDir()
 	writeSourceBundle(t, sourceRoot, "", testBundleOptions{ExtraFiles: []testFile{{Path: "report.html", Data: "<p>source html</p>\n"}}})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithPolicy(t, sourceRoot, destinationRoot, true, true)})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, true, true)})
 	if err == nil || !strings.Contains(err.Error(), "destination output path collision") {
 		t.Fatalf("Run() error = %v, want collision", err)
 	}
@@ -861,7 +930,7 @@ func TestRunFailsOnIndexOutputPathCollision(t *testing.T) {
 		ExtraFiles: []testFile{{Path: "index.html", Data: "<p>source index</p>\n"}},
 	})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, true, true, config.TransformModeIndex, "report.md")})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, true, true, config.TransformModeIndex, "report.md")})
 	if err == nil || !strings.Contains(err.Error(), "destination output path collision") {
 		t.Fatalf("Run() error = %v, want collision", err)
 	}
@@ -877,7 +946,7 @@ func TestRunDryRunReportsGeneratedOutputs(t *testing.T) {
 
 	var stdout bytes.Buffer
 	err := Run(context.Background(), RunOptions{
-		ConfigPath: writeLocalConfigWithPolicy(t, sourceRoot, destinationRoot, false, true),
+		ConfigPath: testutil.WriteLocalConfigWithPublishPolicy(t, sourceRoot, destinationRoot, false, true),
 		DryRun:     true,
 		Stdout:     &stdout,
 	})
@@ -896,7 +965,7 @@ func TestRunDryRunReportsIndexOutputWithoutWriting(t *testing.T) {
 
 	var stdout bytes.Buffer
 	err := Run(context.Background(), RunOptions{
-		ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, ""),
+		ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, ""),
 		DryRun:     true,
 		Stdout:     &stdout,
 	})
@@ -916,11 +985,11 @@ func TestRunSourceOnlyDoesNotWriteIndexOutput(t *testing.T) {
 	destinationRoot := t.TempDir()
 	writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, true, false, config.TransformModeIndex, "")})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, true, false, config.TransformModeIndex, "")})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
 	if _, err := os.Stat(filepath.Join(destinationRoot, "index.html")); !os.IsNotExist(err) {
 		t.Fatalf("index.html stat error = %v, want not exist", err)
 	}
@@ -936,11 +1005,11 @@ func TestRunReplacesHTMLIndexOutput(t *testing.T) {
 			{Path: "summary.txt", Data: "Summary\n"},
 		},
 	})
-	configPath := writeLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")
+	configPath := testutil.WriteLocalConfigWithMarkdownTransform(t, sourceRoot, destinationRoot, false, true, config.TransformModeIndex, "")
 	if err := Run(context.Background(), RunOptions{ConfigPath: configPath}); err != nil {
 		t.Fatalf("first Run() error = %v", err)
 	}
-	assertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<p>Old.</p>")
+	testutil.AssertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<p>Old.</p>")
 
 	writeSourceBundle(t, sourceRoot, "", testBundleOptions{
 		Created: testutil.DefaultCreated.Add(time.Hour),
@@ -953,7 +1022,7 @@ func TestRunReplacesHTMLIndexOutput(t *testing.T) {
 	if err := Run(context.Background(), RunOptions{ConfigPath: configPath}); err != nil {
 		t.Fatalf("second Run() error = %v", err)
 	}
-	assertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<p>New.</p>")
+	testutil.AssertFileContains(t, filepath.Join(destinationRoot, "index.html"), "<p>New.</p>")
 }
 
 func TestRunSkipsWhenDestinationStateMatches(t *testing.T) {
@@ -994,7 +1063,7 @@ func TestRunReplacesOlderDestination(t *testing.T) {
 	if !strings.Contains(stdout.String(), "action=replace_older") {
 		t.Fatalf("stdout = %q, want replace_older", stdout.String())
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
 }
 
 func TestRunSkipsNewerDestination(t *testing.T) {
@@ -1016,7 +1085,7 @@ func TestRunSkipsNewerDestination(t *testing.T) {
 	if !strings.Contains(stdout.String(), "action=skip_destination_newer") {
 		t.Fatalf("stdout = %q, want skip_destination_newer", stdout.String())
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "newer\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "newer\n")
 }
 
 func TestRunFailsOnConflict(t *testing.T) {
@@ -1069,7 +1138,7 @@ func TestRunForceReplacesUnmanagedDestination(t *testing.T) {
 	if _, err := os.Stat(filepath.Join(destinationRoot, "unmanaged.txt")); !os.IsNotExist(err) {
 		t.Fatalf("unmanaged file stat error = %v, want not exist", err)
 	}
-	assertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(destinationRoot, "report.md"), "# Report\nSunny.\n")
 }
 
 func TestRunFansOutToLocalDestinations(t *testing.T) {
@@ -1082,8 +1151,8 @@ func TestRunFansOutToLocalDestinations(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFile(t, filepath.Join(firstDestination, "daily", "report.md"), "# Report\nSunny.\n")
-	assertFile(t, filepath.Join(secondDestination, "daily", "summary.txt"), "Summary\n")
+	testutil.AssertFile(t, filepath.Join(firstDestination, "daily", "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(secondDestination, "daily", "summary.txt"), "Summary\n")
 }
 
 func TestRunFansOutWithDifferentPublishPolicies(t *testing.T) {
@@ -1092,16 +1161,16 @@ func TestRunFansOutWithDifferentPublishPolicies(t *testing.T) {
 	htmlDestination := t.TempDir()
 	writeSourceBundle(t, sourceRoot, "", testBundleOptions{})
 
-	err := Run(context.Background(), RunOptions{ConfigPath: writeMixedPolicyFanoutConfig(t, sourceRoot, archiveDestination, htmlDestination)})
+	err := Run(context.Background(), RunOptions{ConfigPath: testutil.WriteMixedPolicyFanoutLocalConfig(t, sourceRoot, archiveDestination, htmlDestination)})
 	if err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFile(t, filepath.Join(archiveDestination, "report.md"), "# Report\nSunny.\n")
-	assertFile(t, filepath.Join(archiveDestination, "summary.txt"), "Summary\n")
+	testutil.AssertFile(t, filepath.Join(archiveDestination, "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(archiveDestination, "summary.txt"), "Summary\n")
 	if _, err := os.Stat(filepath.Join(archiveDestination, "report.html")); !os.IsNotExist(err) {
 		t.Fatalf("archive report.html stat error = %v, want not exist", err)
 	}
-	assertFileContains(t, filepath.Join(htmlDestination, "report.html"), "<h1>Report</h1>")
+	testutil.AssertFileContains(t, filepath.Join(htmlDestination, "report.html"), "<h1>Report</h1>")
 	if _, err := os.Stat(filepath.Join(htmlDestination, "report.md")); !os.IsNotExist(err) {
 		t.Fatalf("html report.md stat error = %v, want not exist", err)
 	}
@@ -1159,10 +1228,10 @@ func TestRunExercisesRemoteBackendShapesThroughCommonPath(t *testing.T) {
 	if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{Stdout: &publishOutput}, provider); err != nil {
 		t.Fatalf("publish error = %v", err)
 	}
-	assertFakeFile(t, s3Destination, "report.md", "# Report\nSunny.\n")
-	assertFakeFile(t, sshDestination, "summary.txt", "Summary\n")
-	assertFile(t, filepath.Join(s3ToLocalDestination, "report.md"), "# Report\nSunny.\n")
-	assertFile(t, filepath.Join(sshToLocalDestination, "summary.txt"), "Summary\n")
+	testutil.AssertFakeFile(t, s3Destination, "report.md", "# Report\nSunny.\n")
+	testutil.AssertFakeFile(t, sshDestination, "summary.txt", "Summary\n")
+	testutil.AssertFile(t, filepath.Join(s3ToLocalDestination, "report.md"), "# Report\nSunny.\n")
+	testutil.AssertFile(t, filepath.Join(sshToLocalDestination, "summary.txt"), "Summary\n")
 
 	var repeatOutput bytes.Buffer
 	if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{Stdout: &repeatOutput}, provider); err != nil {
@@ -1178,10 +1247,10 @@ func TestRunForceReplacementStaysWithinRemoteBundlePaths(t *testing.T) {
 	writeSourceBundle(t, localSourceRoot, "bundle", testBundleOptions{})
 	s3Destination := fake.New()
 	sshDestination := fake.New()
-	mustWriteFake(t, s3Destination, "bundle/old.txt", "old")
-	mustWriteFake(t, s3Destination, "bundle-sibling/keep.txt", "keep")
-	mustWriteFake(t, sshDestination, "bundle/old.txt", "old")
-	mustWriteFake(t, sshDestination, "bundle-sibling/keep.txt", "keep")
+	testutil.WriteFakeFile(t, s3Destination, "bundle/old.txt", "old")
+	testutil.WriteFakeFile(t, s3Destination, "bundle-sibling/keep.txt", "keep")
+	testutil.WriteFakeFile(t, sshDestination, "bundle/old.txt", "old")
+	testutil.WriteFakeFile(t, sshDestination, "bundle-sibling/keep.txt", "keep")
 	cfg := config.Config{Pipelines: []config.Pipeline{{
 		ID:     "reports",
 		Source: config.Backend{Backend: config.BackendLocal, Path: localSourceRoot},
@@ -1209,12 +1278,12 @@ func TestRunForceReplacementStaysWithinRemoteBundlePaths(t *testing.T) {
 	if err := runConfigWithBackendFactory(context.Background(), cfg, RunOptions{Force: true}, provider); err != nil {
 		t.Fatalf("Run() error = %v", err)
 	}
-	assertFakeFile(t, s3Destination, "bundle/report.md", "# Report\nSunny.\n")
-	assertFakeMissing(t, s3Destination, "bundle/old.txt")
-	assertFakeFile(t, s3Destination, "bundle-sibling/keep.txt", "keep")
-	assertFakeFile(t, sshDestination, "bundle/report.md", "# Report\nSunny.\n")
-	assertFakeMissing(t, sshDestination, "bundle/old.txt")
-	assertFakeFile(t, sshDestination, "bundle-sibling/keep.txt", "keep")
+	testutil.AssertFakeFile(t, s3Destination, "bundle/report.md", "# Report\nSunny.\n")
+	testutil.AssertFakeMissing(t, s3Destination, "bundle/old.txt")
+	testutil.AssertFakeFile(t, s3Destination, "bundle-sibling/keep.txt", "keep")
+	testutil.AssertFakeFile(t, sshDestination, "bundle/report.md", "# Report\nSunny.\n")
+	testutil.AssertFakeMissing(t, sshDestination, "bundle/old.txt")
+	testutil.AssertFakeFile(t, sshDestination, "bundle-sibling/keep.txt", "keep")
 }
 
 func TestRunDryRunDoesNotWrite(t *testing.T) {
@@ -1272,141 +1341,11 @@ func writeLocalConfig(t *testing.T, sourceRoot, destinationRoot string) string {
 	return testutil.WriteMinimalLocalConfig(t, sourceRoot, destinationRoot)
 }
 
-func writeLocalConfigWithPolicy(t *testing.T, sourceRoot, destinationRoot string, publishSource, publishHTML bool) string {
-	t.Helper()
-	transformConfig := ""
-	if publishHTML {
-		transformConfig = `
-        transform:
-          markdown_to_html:
-            enabled: true
-            mode: sidecar`
-	}
-	return writeConfigFile(t, `
-pipelines:
-  - id: reports
-    source:
-      backend: local
-      path: `+sourceRoot+`
-    destinations:
-      - id: archive
-        backend: local
-        path: `+destinationRoot+`
-        publish:
-          source: `+fmt.Sprintf("%t", publishSource)+`
-          html: `+fmt.Sprintf("%t", publishHTML)+transformConfig+`
-`)
-}
-
-func writeLocalConfigWithPathMapping(t *testing.T, sourceRoot, destinationRoot, mode string) string {
-	t.Helper()
-	return writeConfigFile(t, `
-pipelines:
-  - id: reports
-    source:
-      backend: local
-      path: `+sourceRoot+`
-    destinations:
-      - id: archive
-        backend: local
-        path: `+destinationRoot+`
-        path_mapping:
-          mode: `+mode+`
-`)
-}
-
-func writeLocalConfigWithLinks(t *testing.T, sourceRoot, destinationRoot, pathMapping, baseURL, primary string, publishSource, publishHTML bool, transformMode string) string {
-	t.Helper()
-	transformConfig := ""
-	if publishHTML {
-		transformConfig = `
-        transform:
-          markdown_to_html:
-            enabled: true
-            mode: ` + transformMode
-	}
-	return writeConfigFile(t, `
-pipelines:
-  - id: reports
-    source:
-      backend: local
-      path: `+sourceRoot+`
-    destinations:
-      - id: archive
-        backend: local
-        path: `+destinationRoot+`
-        path_mapping:
-          mode: `+pathMapping+`
-        links:
-          base_url: `+baseURL+`
-          primary: `+primary+`
-        publish:
-          source: `+fmt.Sprintf("%t", publishSource)+`
-          html: `+fmt.Sprintf("%t", publishHTML)+transformConfig+`
-`)
-}
-
-func writeLocalConfigWithMarkdownTransform(t *testing.T, sourceRoot, destinationRoot string, publishSource, publishHTML bool, mode, input string) string {
-	t.Helper()
-	enabled := publishHTML
-	inputConfig := ""
-	if input != "" {
-		inputConfig = `
-            input: ` + input
-	}
-	return writeConfigFile(t, `
-pipelines:
-  - id: reports
-    source:
-      backend: local
-      path: `+sourceRoot+`
-    destinations:
-      - id: archive
-        backend: local
-        path: `+destinationRoot+`
-        publish:
-          source: `+fmt.Sprintf("%t", publishSource)+`
-          html: `+fmt.Sprintf("%t", publishHTML)+`
-        transform:
-          markdown_to_html:
-            enabled: `+fmt.Sprintf("%t", enabled)+`
-            mode: `+mode+inputConfig+`
-`)
-}
-
 func writeFanoutConfig(t *testing.T, sourceRoot, firstDestination, secondDestination string) string {
 	t.Helper()
 	return testutil.WriteFanoutLocalConfig(t, sourceRoot, firstDestination, secondDestination)
 }
 
-func writeMixedPolicyFanoutConfig(t *testing.T, sourceRoot, archiveDestination, htmlDestination string) string {
-	t.Helper()
-	return writeConfigFile(t, `
-pipelines:
-  - id: reports
-    source:
-      backend: local
-      path: `+sourceRoot+`
-    destinations:
-      - id: archive
-        backend: local
-        path: `+archiveDestination+`
-        publish:
-          source: true
-          html: false
-      - id: html
-        backend: local
-        path: `+htmlDestination+`
-        publish:
-          source: false
-          html: true
-        transform:
-          markdown_to_html:
-            enabled: true
-            mode: sidecar
-`)
-}
-
 func writeConfigFile(t *testing.T, body string) string {
 	t.Helper()
 	path := filepath.Join(t.TempDir(), "config.yml")
@@ -1434,53 +1373,6 @@ func outputsByPath(outputs []state.OutputFile) map[string]state.OutputFile {
 	return byPath
 }
 
-func assertFile(t *testing.T, path, want string) {
-	t.Helper()
-	data, err := os.ReadFile(path)
-	if err != nil {
-		t.Fatalf("read file %s: %v", path, err)
-	}
-	if got := string(data); got != want {
-		t.Fatalf("%s = %q, want %q", path, got, want)
-	}
-}
-
-func assertFileContains(t *testing.T, path, want string) {
-	t.Helper()
-	data, err := os.ReadFile(path)
-	if err != nil {
-		t.Fatalf("read file %s: %v", path, err)
-	}
-	if !strings.Contains(string(data), want) {
-		t.Fatalf("%s = %q, want substring %q", path, data, want)
-	}
-}
-
-func assertFakeFile(t *testing.T, backend *fake.Backend, path, want string) {
-	t.Helper()
-	data, err := backend.ReadFile(context.Background(), path)
-	if err != nil {
-		t.Fatalf("read fake file %s: %v", path, err)
-	}
-	if got := string(data); got != want {
-		t.Fatalf("%s = %q, want %q", path, got, want)
-	}
-}
-
-func assertFakeMissing(t *testing.T, backend *fake.Backend, path string) {
-	t.Helper()
-	if _, err := backend.Stat(context.Background(), path); !storage.IsNotFound(err) {
-		t.Fatalf("fake file %s stat error = %v, want not found", path, err)
-	}
-}
-
-func mustWriteFake(t *testing.T, backend *fake.Backend, path, data string) {
-	t.Helper()
-	if _, err := backend.WriteFile(context.Background(), path, []byte(data), storage.WriteOptions{}); err != nil {
-		t.Fatalf("write fake file %s: %v", path, err)
-	}
-}
-
 func crossBackendConfig(localSourceRoot, s3ToLocalDestination, sshToLocalDestination string) config.Config {
 	cfg := config.Config{
 		Pipelines: []config.Pipeline{

internal/app/run_warnings.go

@@ -0,0 +1,47 @@
+package app
+
+import (
+	"fmt"
+	"io"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/config"
+)
+
+func secretConflictWarnings(conflicts []config.SecretConflict) []OutputWarning {
+	warnings := make([]OutputWarning, 0, len(conflicts))
+	for _, conflict := range conflicts {
+		warnings = append(warnings, OutputWarning{
+			Message: fmt.Sprintf("secret %s ignored because the real environment already has that variable", conflict.Name),
+		})
+	}
+	return warnings
+}
+
+func sshWarnings(pipeline config.Pipeline) []OutputWarning {
+	var warnings []OutputWarning
+	if pipeline.Source.Backend == config.BackendSSH && pipeline.Source.SSH.HostKeyPolicy == config.HostKeyPolicyOff {
+		warnings = append(warnings, OutputWarning{
+			Message: fmt.Sprintf("pipeline=%s source host_key_policy=off disables SSH host key checking", pipeline.ID),
+		})
+	}
+	for _, destination := range pipeline.Destinations {
+		if destination.Backend == config.BackendSSH && destination.SSH.HostKeyPolicy == config.HostKeyPolicyOff {
+			warnings = append(warnings, OutputWarning{
+				Message: fmt.Sprintf("pipeline=%s destination=%s host_key_policy=off disables SSH host key checking", pipeline.ID, destination.ID),
+			})
+		}
+	}
+	return warnings
+}
+
+func writeWarnings(w io.Writer, warnings []OutputWarning) error {
+	if w == nil {
+		return nil
+	}
+	for _, warning := range warnings {
+		if _, err := fmt.Fprintf(w, "Warning: %s\n", warning.Message); err != nil {
+			return err
+		}
+	}
+	return nil
+}

internal/bundle/digest.go

@@ -1,19 +1,9 @@
 package bundle
 
-import (
-	"fmt"
-	"regexp"
-
-	publicbundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
-)
-
-var digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
+import publicbundle "gitea.maximumdirect.net/eric/distributor/pkg/bundle"
 
 func ValidateDigest(value string) error {
-	if !digestPattern.MatchString(value) {
-		return fmt.Errorf("must be lowercase sha256:<64 hex>")
-	}
-	return nil
+	return publicbundle.ValidateDigest(value)
 }
 
 func FileDigest(data []byte) string {

internal/cli/inspect.go

@@ -2,7 +2,6 @@ package cli
 
 import (
 	"context"
-	"flag"
 	"fmt"
 	"io"
 
@@ -14,33 +13,17 @@ func inspectCommand(ctx context.Context, args []string, stdout, stderr io.Writer
 		printInspectHelp(stdout)
 		return exitOK
 	}
-	flags := flag.NewFlagSet("inspect", flag.ContinueOnError)
-	flags.SetOutput(stderr)
-	configPath := flags.String("config", "", "path to config file")
-	pipelineID := flags.String("pipeline", "", "pipeline id")
-	bundlePath := flags.String("bundle", "", "source-root-relative bundle path")
-	formatFlag := addFormatFlag(flags)
-	if err := flags.Parse(args); err != nil {
-		return exitUsage
-	}
-	format, ok := parseOutputFormat(stderr, "inspect", *formatFlag)
+	parsed, ok := parseSourceDiagnosticArgs(stderr, "inspect", args)
 	if !ok {
 		return exitUsage
 	}
-	path, ok := parseOptionalPathArg(stderr, "inspect", flags.Args())
-	if !ok {
-		return exitUsage
-	}
-	if !validateInspectModeOK(stderr, "inspect", path, *configPath, *pipelineID, *bundlePath) {
-		return exitUsage
-	}
 	if err := app.Inspect(ctx, app.InspectOptions{
-		Path:         path,
-		ConfigPath:   *configPath,
-		PipelineID:   *pipelineID,
-		BundlePath:   *bundlePath,
+		Path:         parsed.Path,
+		ConfigPath:   parsed.ConfigPath,
+		PipelineID:   parsed.PipelineID,
+		BundlePath:   parsed.BundlePath,
 		Stdout:       stdout,
-		OutputFormat: format,
+		OutputFormat: parsed.OutputFormat,
 	}); err != nil {
 		return fail(stderr, err)
 	}

internal/cli/root_test.go

@@ -217,12 +217,24 @@ func TestExecuteValidateArgs(t *testing.T) {
 			wantCode:   exitUsage,
 			wantStderr: "requires --config",
 		},
+		{
+			name:       "bundle without config",
+			args:       []string{"validate", "--bundle", "daily"},
+			wantCode:   exitUsage,
+			wantStderr: "requires --config",
+		},
 		{
 			name:       "config without pipeline",
 			args:       []string{"validate", "--config", "config.yml"},
 			wantCode:   exitUsage,
 			wantStderr: "requires --pipeline",
 		},
+		{
+			name:       "invalid format",
+			args:       []string{"validate", "--format", "xml", validPath},
+			wantCode:   exitUsage,
+			wantStderr: "format must be text or json",
+		},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
@@ -343,12 +355,24 @@ func TestExecuteInspectArgs(t *testing.T) {
 			wantCode:   exitUsage,
 			wantStderr: "requires --config",
 		},
+		{
+			name:       "bundle without config",
+			args:       []string{"inspect", "--bundle", "daily"},
+			wantCode:   exitUsage,
+			wantStderr: "requires --config",
+		},
 		{
 			name:       "config without pipeline",
 			args:       []string{"inspect", "--config", "config.yml"},
 			wantCode:   exitUsage,
 			wantStderr: "requires --pipeline",
 		},
+		{
+			name:       "invalid format",
+			args:       []string{"inspect", "--format", "xml", validPath},
+			wantCode:   exitUsage,
+			wantStderr: "format must be text or json",
+		},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {

internal/cli/source_mode.go

@@ -1,10 +1,51 @@
 package cli
 
 import (
+	"flag"
 	"fmt"
 	"io"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/app"
 )
 
+type sourceDiagnosticArgs struct {
+	Path         string
+	ConfigPath   string
+	PipelineID   string
+	BundlePath   string
+	OutputFormat app.OutputFormat
+}
+
+func parseSourceDiagnosticArgs(stderr io.Writer, command string, args []string) (sourceDiagnosticArgs, bool) {
+	flags := flag.NewFlagSet(command, flag.ContinueOnError)
+	flags.SetOutput(stderr)
+	configPath := flags.String("config", "", "path to config file")
+	pipelineID := flags.String("pipeline", "", "pipeline id")
+	bundlePath := flags.String("bundle", "", "source-root-relative bundle path")
+	formatFlag := addFormatFlag(flags)
+	if err := flags.Parse(args); err != nil {
+		return sourceDiagnosticArgs{}, false
+	}
+	format, ok := parseOutputFormat(stderr, command, *formatFlag)
+	if !ok {
+		return sourceDiagnosticArgs{}, false
+	}
+	path, ok := parseOptionalPathArg(stderr, command, flags.Args())
+	if !ok {
+		return sourceDiagnosticArgs{}, false
+	}
+	if !validateInspectModeOK(stderr, command, path, *configPath, *pipelineID, *bundlePath) {
+		return sourceDiagnosticArgs{}, false
+	}
+	return sourceDiagnosticArgs{
+		Path:         path,
+		ConfigPath:   *configPath,
+		PipelineID:   *pipelineID,
+		BundlePath:   *bundlePath,
+		OutputFormat: format,
+	}, true
+}
+
 func validateInspectModeOK(stderr io.Writer, command, path, configPath, pipelineID, bundlePath string) bool {
 	configMode := configPath != "" || pipelineID != "" || bundlePath != ""
 	if !configMode {

internal/cli/validate.go

@@ -2,7 +2,6 @@ package cli
 
 import (
 	"context"
-	"flag"
 	"fmt"
 	"io"
 
@@ -14,33 +13,17 @@ func validateCommand(ctx context.Context, args []string, stdout, stderr io.Write
 		printValidateHelp(stdout)
 		return exitOK
 	}
-	flags := flag.NewFlagSet("validate", flag.ContinueOnError)
-	flags.SetOutput(stderr)
-	configPath := flags.String("config", "", "path to config file")
-	pipelineID := flags.String("pipeline", "", "pipeline id")
-	bundlePath := flags.String("bundle", "", "source-root-relative bundle path")
-	formatFlag := addFormatFlag(flags)
-	if err := flags.Parse(args); err != nil {
-		return exitUsage
-	}
-	format, ok := parseOutputFormat(stderr, "validate", *formatFlag)
+	parsed, ok := parseSourceDiagnosticArgs(stderr, "validate", args)
 	if !ok {
 		return exitUsage
 	}
-	path, ok := parseOptionalPathArg(stderr, "validate", flags.Args())
-	if !ok {
-		return exitUsage
-	}
-	if !validateInspectModeOK(stderr, "validate", path, *configPath, *pipelineID, *bundlePath) {
-		return exitUsage
-	}
 	if err := app.Validate(ctx, app.ValidateOptions{
-		Path:         path,
-		ConfigPath:   *configPath,
-		PipelineID:   *pipelineID,
-		BundlePath:   *bundlePath,
+		Path:         parsed.Path,
+		ConfigPath:   parsed.ConfigPath,
+		PipelineID:   parsed.PipelineID,
+		BundlePath:   parsed.BundlePath,
 		Stdout:       stdout,
-		OutputFormat: format,
+		OutputFormat: parsed.OutputFormat,
 	}); err != nil {
 		return fail(stderr, err)
 	}

internal/config/validate.go

@@ -2,9 +2,10 @@ package config
 
 import (
 	"fmt"
-	"net/url"
 	"regexp"
 	"strings"
+
+	"gitea.maximumdirect.net/eric/distributor/internal/link"
 )
 
 var idPattern = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]*$`)
@@ -187,7 +188,7 @@ func validateLinks(errs ValidationErrors, context string, links *Links) Validati
 	}
 	if links.BaseURL == "" {
 		errs = append(errs, context+".base_url is required")
-	} else if err := validateLinkBaseURL(links.BaseURL); err != nil {
+	} else if err := link.ValidateHTTPURL(links.BaseURL); err != nil {
 		errs = append(errs, context+".base_url "+err.Error())
 	}
 	switch links.Primary {
@@ -198,26 +199,6 @@ func validateLinks(errs ValidationErrors, context string, links *Links) Validati
 	return errs
 }
 
-func validateLinkBaseURL(value string) error {
-	parsed, err := url.Parse(value)
-	if err != nil {
-		return fmt.Errorf("must be a valid URL")
-	}
-	if parsed.Scheme != "http" && parsed.Scheme != "https" {
-		return fmt.Errorf("must use http or https")
-	}
-	if parsed.Host == "" {
-		return fmt.Errorf("must include a host")
-	}
-	if parsed.RawQuery != "" {
-		return fmt.Errorf("must not include a query string")
-	}
-	if parsed.Fragment != "" {
-		return fmt.Errorf("must not include a fragment")
-	}
-	return nil
-}
-
 func validateTransferPolicy(errs ValidationErrors, context string, policy TransferPolicy) ValidationErrors {
 	if policy.OnDestinationSame != TransferActionSkip && policy.OnDestinationSame != TransferActionFail {
 		errs = append(errs, context+".on_destination_same must be skip or fail")

internal/config/validate_test.go

@@ -1,6 +1,9 @@
 package config
 
-import "testing"
+import (
+	"strings"
+	"testing"
+)
 
 func TestValidatePublishTransformPolicy(t *testing.T) {
 	tests := publishTransformPolicyCases()
@@ -145,6 +148,28 @@ func TestValidateLinks(t *testing.T) {
 	}
 }
 
+func TestValidateLinksReportsFieldContext(t *testing.T) {
+	cfg := Config{Pipelines: []Pipeline{{
+		ID:     "reports",
+		Source: Backend{Backend: BackendLocal, Path: "/source"},
+		Destinations: []Destination{{
+			ID:      "web",
+			Backend: BackendLocal,
+			Path:    "/destination",
+			Links:   &Links{BaseURL: "https://reports.example.com/archive?preview=1", Primary: LinkPrimaryAuto},
+		}},
+	}}}
+	ApplyDefaults(&cfg)
+	err := Validate(cfg)
+	if err == nil {
+		t.Fatal("Validate() error = nil, want error")
+	}
+	want := "pipelines[0].destinations[0].links.base_url must not include a query string"
+	if !strings.Contains(err.Error(), want) {
+		t.Fatalf("Validate() error = %q, want %q", err, want)
+	}
+}
+
 type publishTransformPolicyCase struct {
 	name      string
 	publish   PublishPolicy

internal/link/url.go

@@ -0,0 +1,26 @@
+package link
+
+import (
+	"fmt"
+	"net/url"
+)
+
+func ValidateHTTPURL(value string) error {
+	parsed, err := url.Parse(value)
+	if err != nil {
+		return fmt.Errorf("must be a valid URL")
+	}
+	if parsed.Scheme != "http" && parsed.Scheme != "https" {
+		return fmt.Errorf("must use http or https")
+	}
+	if parsed.Host == "" {
+		return fmt.Errorf("must include a host")
+	}
+	if parsed.RawQuery != "" {
+		return fmt.Errorf("must not include a query string")
+	}
+	if parsed.Fragment != "" {
+		return fmt.Errorf("must not include a fragment")
+	}
+	return nil
+}

internal/link/url_test.go

@@ -0,0 +1,39 @@
+package link
+
+import (
+	"strings"
+	"testing"
+)
+
+func TestValidateHTTPURL(t *testing.T) {
+	tests := []struct {
+		name    string
+		value   string
+		wantErr string
+	}{
+		{name: "http", value: "http://reports.example.com/archive"},
+		{name: "https", value: "https://reports.example.com/archive"},
+		{name: "missing host", value: "https:///archive", wantErr: "must include a host"},
+		{name: "unsupported scheme", value: "ftp://reports.example.com/archive", wantErr: "must use http or https"},
+		{name: "query string", value: "https://reports.example.com/archive?preview=1", wantErr: "must not include a query string"},
+		{name: "fragment", value: "https://reports.example.com/archive#top", wantErr: "must not include a fragment"},
+		{name: "parse failure", value: "http://[::1", wantErr: "must be a valid URL"},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			err := ValidateHTTPURL(tt.value)
+			if tt.wantErr == "" {
+				if err != nil {
+					t.Fatalf("ValidateHTTPURL() error = %v", err)
+				}
+				return
+			}
+			if err == nil {
+				t.Fatal("ValidateHTTPURL() error = nil, want error")
+			}
+			if !strings.Contains(err.Error(), tt.wantErr) {
+				t.Fatalf("ValidateHTTPURL() error = %q, want %q", err, tt.wantErr)
+			}
+		})
+	}
+}

internal/publish/execute.go

@@ -23,7 +23,7 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
 		if plan.ExistingState == nil {
 			return fmt.Errorf("replace requires existing destination state")
 		}
-		if err := req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, existingManagedOutputPaths(*plan.ExistingState), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true}); err != nil {
+		if err := req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, stateOutputManagedPaths(plan.ExistingState.Outputs), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true}); err != nil {
 			return err
 		}
 		if err := ensureDestinationEmpty(ctx, req.DestinationBackend, req.DestinationBundlePath); err != nil {
@@ -41,7 +41,7 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
 
 	writtenOutputs := make([]Output, 0, len(plan.Outputs))
 	cleanup := func() {
-		_ = req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, managedOutputPaths(writtenOutputs), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true})
+		_ = req.DestinationBackend.DeleteManagedBundle(ctx, req.DestinationBundlePath, ManagedOutputPaths(writtenOutputs), storage.DeleteOptions{IgnoreMissing: true, PruneEmptyDirs: true})
 	}
 	for _, output := range plan.Outputs {
 		destinationPath, err := storage.Join(req.DestinationBundlePath, output.DestinationPath)
@@ -76,7 +76,7 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
 		DestinationID:      req.DestinationID,
 		PublishedAt:        time.Now().UTC(),
 		Source:             state.SourceState{Manifest: req.SourceBundle.Manifest},
-		Outputs:            stateOutputs(plan.Outputs),
+		Outputs:            StateOutputFiles(plan.Outputs),
 	}
 	if plan.PrimaryURL != "" {
 		destinationState.Links = &state.LinkState{PrimaryURL: plan.PrimaryURL}
@@ -102,11 +102,3 @@ func Execute(ctx context.Context, req Request, plan Plan) error {
 	}
 	return nil
 }
-
-func existingManagedOutputPaths(destinationState state.DistributorState) []string {
-	paths := make([]string, 0, len(destinationState.Outputs))
-	for _, output := range destinationState.Outputs {
-		paths = append(paths, output.Path)
-	}
-	return paths
-}

internal/publish/force_test.go

@@ -2,13 +2,11 @@ package publish
 
 import (
 	"context"
-	"encoding/json"
 	"strings"
 	"testing"
 
 	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
 	"gitea.maximumdirect.net/eric/distributor/internal/config"
-	"gitea.maximumdirect.net/eric/distributor/internal/storage"
 	"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
 	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
 )
@@ -25,7 +23,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
 			name: "unmanaged content",
 			prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
 				t.Helper()
-				writeFakeFile(t, backend, "bundle/old.txt", "old")
+				testutil.WriteFakeFile(t, backend, "bundle/old.txt", "old")
 			},
 			transfer:    defaultTransfer(),
 			wantReason:  "fail_unmanaged",
@@ -37,7 +35,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
 				t.Helper()
 				conflict := source
 				conflict.ID = "other.source"
-				writeFakeDestinationState(t, backend, "bundle", conflict, testutil.DestinationStateOptions{})
+				testutil.WriteFakeDestinationState(t, backend, "bundle", conflict, testutil.DestinationStateOptions{})
 			},
 			transfer:    conflictReplaceTransfer(),
 			wantReason:  "requires --force",
@@ -48,7 +46,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
 			prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
 				t.Helper()
 				conflict := testutil.ValidManifest(testutil.BundleOptions{Files: []testutil.SourceFile{{Path: "report.md", Data: "# Different\n"}}})
-				writeFakeDestinationState(t, backend, "bundle", conflict, testutil.DestinationStateOptions{})
+				testutil.WriteFakeDestinationState(t, backend, "bundle", conflict, testutil.DestinationStateOptions{})
 			},
 			transfer:    conflictReplaceTransfer(),
 			wantReason:  "requires --force",
@@ -58,7 +56,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
 			name: "pipeline mismatch",
 			prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
 				t.Helper()
-				writeFakeDestinationState(t, backend, "bundle", source, testutil.DestinationStateOptions{PipelineID: "other-pipeline"})
+				testutil.WriteFakeDestinationState(t, backend, "bundle", source, testutil.DestinationStateOptions{PipelineID: "other-pipeline"})
 			},
 			transfer:    conflictReplaceTransfer(),
 			wantReason:  "requires --force",
@@ -68,7 +66,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
 			name: "destination mismatch",
 			prepare: func(t *testing.T, backend *fake.Backend, source bundle.Manifest) {
 				t.Helper()
-				writeFakeDestinationState(t, backend, "bundle", source, testutil.DestinationStateOptions{DestinationID: "other-destination"})
+				testutil.WriteFakeDestinationState(t, backend, "bundle", source, testutil.DestinationStateOptions{DestinationID: "other-destination"})
 			},
 			transfer:    conflictReplaceTransfer(),
 			wantReason:  "requires --force",
@@ -80,7 +78,7 @@ func TestBuildPlansForcedReplacementOnlyWhenExplicit(t *testing.T) {
 				t.Helper()
 				newer := source
 				newer.Created = newer.Created.AddDate(0, 0, 1)
-				writeFakeDestinationState(t, backend, "bundle", newer, testutil.DestinationStateOptions{})
+				testutil.WriteFakeDestinationState(t, backend, "bundle", newer, testutil.DestinationStateOptions{})
 			},
 			transfer:    newerReplaceTransfer(),
 			wantReason:  "requires --force",
@@ -121,7 +119,7 @@ func TestBuildRequiresConflictPolicyForStateConflicts(t *testing.T) {
 	destinationBackend := fake.New()
 	conflict := sourceBundle.Manifest
 	conflict.ID = "other.source"
-	writeFakeDestinationState(t, destinationBackend, "bundle", conflict, testutil.DestinationStateOptions{})
+	testutil.WriteFakeDestinationState(t, destinationBackend, "bundle", conflict, testutil.DestinationStateOptions{})
 
 	req := forceRequest(sourceBackend, destinationBackend, sourceBundle, defaultTransfer())
 	req.Force = true
@@ -135,10 +133,10 @@ func TestExecuteForcedReplacementDeletesOnlyBundlePath(t *testing.T) {
 	sourceBackend := fake.New()
 	sourceBundle := testutil.WriteFakeSourceBundle(t, sourceBackend, "bundle", testutil.BundleOptions{})
 	destinationBackend := fake.New()
-	writeFakeFile(t, destinationBackend, "bundle/old.txt", "old")
-	writeFakeFile(t, destinationBackend, "bundle/nested/old.txt", "old")
-	writeFakeFile(t, destinationBackend, "bundle-sibling/keep.txt", "keep")
-	writeFakeFile(t, destinationBackend, "outside.txt", "outside")
+	testutil.WriteFakeFile(t, destinationBackend, "bundle/old.txt", "old")
+	testutil.WriteFakeFile(t, destinationBackend, "bundle/nested/old.txt", "old")
+	testutil.WriteFakeFile(t, destinationBackend, "bundle-sibling/keep.txt", "keep")
+	testutil.WriteFakeFile(t, destinationBackend, "outside.txt", "outside")
 
 	req := forceRequest(sourceBackend, destinationBackend, sourceBundle, defaultTransfer())
 	req.Force = true
@@ -152,11 +150,11 @@ func TestExecuteForcedReplacementDeletesOnlyBundlePath(t *testing.T) {
 	if err := Execute(context.Background(), req, plan); err != nil {
 		t.Fatalf("Execute() error = %v", err)
 	}
-	assertFakeFile(t, destinationBackend, "bundle/report.md", "# Report\nSunny.\n")
-	assertFakeMissing(t, destinationBackend, "bundle/old.txt")
-	assertFakeMissing(t, destinationBackend, "bundle/nested/old.txt")
-	assertFakeFile(t, destinationBackend, "bundle-sibling/keep.txt", "keep")
-	assertFakeFile(t, destinationBackend, "outside.txt", "outside")
+	testutil.AssertFakeFile(t, destinationBackend, "bundle/report.md", "# Report\nSunny.\n")
+	testutil.AssertFakeMissing(t, destinationBackend, "bundle/old.txt")
+	testutil.AssertFakeMissing(t, destinationBackend, "bundle/nested/old.txt")
+	testutil.AssertFakeFile(t, destinationBackend, "bundle-sibling/keep.txt", "keep")
+	testutil.AssertFakeFile(t, destinationBackend, "outside.txt", "outside")
 }
 
 func forceRequest(sourceBackend, destinationBackend *fake.Backend, sourceBundle bundle.Bundle, transfer config.TransferPolicy) Request {
@@ -193,49 +191,3 @@ func newerReplaceTransfer() config.TransferPolicy {
 	transfer.OnDestinationNewer = config.TransferActionReplace
 	return transfer
 }
-
-func writeFakeDestinationState(t *testing.T, backend *fake.Backend, relative string, manifest bundle.Manifest, opts testutil.DestinationStateOptions) {
-	t.Helper()
-	destinationState := testutil.DestinationState(manifest, opts)
-	data, err := json.MarshalIndent(destinationState, "", "  ")
-	if err != nil {
-		t.Fatalf("marshal destination state: %v", err)
-	}
-	statePath, err := storage.StatePath(relative)
-	if err != nil {
-		t.Fatalf("state path: %v", err)
-	}
-	writeFakeFile(t, backend, statePath, string(append(data, '\n')))
-	for _, output := range destinationState.Outputs {
-		path, err := storage.Join(relative, output.Path)
-		if err != nil {
-			t.Fatalf("join output path: %v", err)
-		}
-		writeFakeFile(t, backend, path, "old")
-	}
-}
-
-func writeFakeFile(t *testing.T, backend *fake.Backend, path, data string) {
-	t.Helper()
-	if _, err := backend.WriteFile(context.Background(), path, []byte(data), storage.WriteOptions{}); err != nil {
-		t.Fatalf("write fake file %s: %v", path, err)
-	}
-}
-
-func assertFakeFile(t *testing.T, backend *fake.Backend, path, want string) {
-	t.Helper()
-	data, err := backend.ReadFile(context.Background(), path)
-	if err != nil {
-		t.Fatalf("read fake file %s: %v", path, err)
-	}
-	if got := string(data); got != want {
-		t.Fatalf("fake file %s = %q, want %q", path, got, want)
-	}
-}
-
-func assertFakeMissing(t *testing.T, backend *fake.Backend, path string) {
-	t.Helper()
-	if _, err := backend.Stat(context.Background(), path); !storage.IsNotFound(err) {
-		t.Fatalf("fake file %s stat error = %v, want not found", path, err)
-	}
-}

internal/publish/links.go

@@ -6,6 +6,7 @@ import (
 	"strings"
 
 	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/link"
 	"gitea.maximumdirect.net/eric/distributor/internal/state"
 )
 
@@ -13,6 +14,9 @@ func PlanLinks(req Request, outputs []Output) ([]Output, string, error) {
 	if req.Links == nil {
 		return outputs, "", nil
 	}
+	if err := link.ValidateHTTPURL(req.Links.BaseURL); err != nil {
+		return nil, "", fmt.Errorf("link base URL: %w", err)
+	}
 	linked := make([]Output, 0, len(outputs))
 	for _, output := range outputs {
 		outputURL, err := OutputURL(req.Links.BaseURL, req.DestinationBundlePath, output.DestinationPath)

internal/publish/links_test.go

@@ -1,6 +1,7 @@
 package publish
 
 import (
+	"strings"
 	"testing"
 
 	"gitea.maximumdirect.net/eric/distributor/internal/config"
@@ -147,3 +148,22 @@ func TestPlanLinksLeavesOutputsUnchangedWithoutConfig(t *testing.T) {
 		t.Fatalf("output URL = %q, want empty", linked[0].URL)
 	}
 }
+
+func TestPlanLinksValidatesBaseURLBeforePlanning(t *testing.T) {
+	_, _, err := PlanLinks(Request{
+		DestinationBundlePath: "daily",
+		Links: &config.Links{
+			BaseURL: "https://reports.example.com/archive?preview=1",
+			Primary: config.LinkPrimaryAuto,
+		},
+	}, []Output{{
+		DestinationPath: "report.md",
+		Kind:            state.OutputKindSource,
+	}})
+	if err == nil {
+		t.Fatal("PlanLinks() error = nil, want error")
+	}
+	if !strings.Contains(err.Error(), "link base URL: must not include a query string") {
+		t.Fatalf("PlanLinks() error = %q, want link base URL context", err)
+	}
+}

internal/publish/output.go

@@ -97,26 +97,42 @@ func rejectOutputCollisions(outputs []Output) error {
 	return nil
 }
 
-func stateOutputs(outputs []Output) []state.OutputFile {
+func (o Output) StateOutputFile() state.OutputFile {
+	return state.OutputFile{
+		Path:       o.DestinationPath,
+		Kind:       o.Kind,
+		SourcePath: o.SourcePath,
+		Transform:  o.Transform,
+		URL:        o.URL,
+		SHA256:     o.SHA256,
+		Size:       o.Size,
+	}
+}
+
+func (o Output) ManagedPath() string {
+	return o.DestinationPath
+}
+
+func StateOutputFiles(outputs []Output) []state.OutputFile {
 	files := make([]state.OutputFile, 0, len(outputs))
 	for _, output := range outputs {
-		files = append(files, state.OutputFile{
-			Path:       output.DestinationPath,
-			Kind:       output.Kind,
-			SourcePath: output.SourcePath,
-			Transform:  output.Transform,
-			URL:        output.URL,
-			SHA256:     output.SHA256,
-			Size:       output.Size,
-		})
+		files = append(files, output.StateOutputFile())
 	}
 	return files
 }
 
-func managedOutputPaths(outputs []Output) []string {
+func ManagedOutputPaths(outputs []Output) []string {
 	paths := make([]string, 0, len(outputs))
 	for _, output := range outputs {
-		paths = append(paths, output.DestinationPath)
+		paths = append(paths, output.ManagedPath())
+	}
+	return paths
+}
+
+func stateOutputManagedPaths(outputs []state.OutputFile) []string {
+	paths := make([]string, 0, len(outputs))
+	for _, output := range outputs {
+		paths = append(paths, output.Path)
 	}
 	return paths
 }

internal/publish/output_test.go

@@ -6,11 +6,58 @@ import (
 
 	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
 	"gitea.maximumdirect.net/eric/distributor/internal/config"
+	"gitea.maximumdirect.net/eric/distributor/internal/state"
 	"gitea.maximumdirect.net/eric/distributor/internal/storage/fake"
 	"gitea.maximumdirect.net/eric/distributor/internal/testutil"
 	"gitea.maximumdirect.net/eric/distributor/internal/transform"
 )
 
+func TestOutputStateProjection(t *testing.T) {
+	sourceOutput := Output{
+		SourcePath:      "report.md",
+		DestinationPath: "report.md",
+		Kind:            state.OutputKindSource,
+		URL:             "https://reports.example.com/report.md",
+		SHA256:          "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+		Size:            123,
+	}
+	sourceState := sourceOutput.StateOutputFile()
+	if sourceState.Path != "report.md" || sourceState.Kind != state.OutputKindSource || sourceState.SourcePath != "report.md" || sourceState.URL != sourceOutput.URL || sourceState.SHA256 != sourceOutput.SHA256 || sourceState.Size != sourceOutput.Size {
+		t.Fatalf("source state output = %#v", sourceState)
+	}
+
+	generatedOutput := Output{
+		SourcePath:      "report.md",
+		DestinationPath: "report.html",
+		Kind:            state.OutputKindGenerated,
+		Transform:       transform.MarkdownToHTML,
+		URL:             "https://reports.example.com/report.html",
+		SHA256:          "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+		Size:            456,
+	}
+	generatedState := generatedOutput.StateOutputFile()
+	if generatedState.Path != "report.html" || generatedState.Kind != state.OutputKindGenerated || generatedState.SourcePath != "report.md" || generatedState.Transform != transform.MarkdownToHTML || generatedState.URL != generatedOutput.URL || generatedState.SHA256 != generatedOutput.SHA256 || generatedState.Size != generatedOutput.Size {
+		t.Fatalf("generated state output = %#v", generatedState)
+	}
+}
+
+func TestOutputSliceProjections(t *testing.T) {
+	outputs := []Output{
+		{SourcePath: "report.md", DestinationPath: "report.md", Kind: state.OutputKindSource, SHA256: "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", Size: 1},
+		{SourcePath: "report.md", DestinationPath: "report.html", Kind: state.OutputKindGenerated, Transform: transform.MarkdownToHTML, URL: "https://reports.example.com/report.html", SHA256: "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", Size: 2},
+	}
+
+	stateOutputs := StateOutputFiles(outputs)
+	if len(stateOutputs) != 2 || stateOutputs[1].Path != "report.html" || stateOutputs[1].Transform != transform.MarkdownToHTML || stateOutputs[1].URL != outputs[1].URL {
+		t.Fatalf("state outputs = %#v", stateOutputs)
+	}
+
+	paths := ManagedOutputPaths(outputs)
+	if len(paths) != 2 || paths[0] != "report.md" || paths[1] != "report.html" {
+		t.Fatalf("managed paths = %#v", paths)
+	}
+}
+
 func TestPlanOutputsRejectsCollision(t *testing.T) {
 	sourceBackend := fake.New()
 	sourceBundle := testutil.WriteFakeSourceBundle(t, sourceBackend, "", testutil.BundleOptions{

internal/state/distributor_test.go

@@ -179,6 +179,23 @@ func TestParseRejectsInvalidOutputMetadata(t *testing.T) {
 	}
 }
 
+func TestValidateReportsURLFieldContext(t *testing.T) {
+	source := validManifest(t)
+	state := *withState(t, source, func(s *DistributorState) {
+		s.Links = &LinkState{PrimaryURL: "https://reports.example.com/archive#top"}
+	})
+	err := Validate(state)
+	assertStateErrorContains(t, err, "state links.primary_url")
+	assertStateErrorContains(t, err, "must not include a fragment")
+
+	state = *withState(t, source, func(s *DistributorState) {
+		s.Outputs[0].URL = "https://reports.example.com/archive?preview=1"
+	})
+	err = Validate(state)
+	assertStateErrorContains(t, err, "state outputs[0].url")
+	assertStateErrorContains(t, err, "must not include a query string")
+}
+
 func TestParseRejectsMalformedPublishedTimestamp(t *testing.T) {
 	body := strings.Replace(validStateJSON(t), `"published_at": "2026-05-30T11:12:00Z"`, `"published_at": "May 30"`, 1)
 	_, err := Parse([]byte(body))

internal/state/validate.go

@@ -2,9 +2,9 @@ package state
 
 import (
 	"fmt"
-	"net/url"
 
 	"gitea.maximumdirect.net/eric/distributor/internal/bundle"
+	"gitea.maximumdirect.net/eric/distributor/internal/link"
 	"gitea.maximumdirect.net/eric/distributor/internal/storage"
 )
 
@@ -30,7 +30,7 @@ func Validate(s DistributorState) error {
 		return fmt.Errorf("state source.manifest: %w", err)
 	}
 	if s.Links != nil && s.Links.PrimaryURL != "" {
-		if err := validateStateURL(s.Links.PrimaryURL); err != nil {
+		if err := link.ValidateHTTPURL(s.Links.PrimaryURL); err != nil {
 			return fmt.Errorf("state links.primary_url: %w", err)
 		}
 	}
@@ -70,7 +70,7 @@ func validateOutput(index int, output OutputFile) error {
 		return fmt.Errorf("state outputs[%d].transform is required for generated output", index)
 	}
 	if output.URL != "" {
-		if err := validateStateURL(output.URL); err != nil {
+		if err := link.ValidateHTTPURL(output.URL); err != nil {
 			return fmt.Errorf("state outputs[%d].url: %w", index, err)
 		}
 	}
@@ -82,23 +82,3 @@ func validateOutput(index int, output OutputFile) error {
 	}
 	return nil
 }
-
-func validateStateURL(value string) error {
-	parsed, err := url.Parse(value)
-	if err != nil {
-		return err
-	}
-	if parsed.Scheme != "http" && parsed.Scheme != "https" {
-		return fmt.Errorf("must use http or https")
-	}
-	if parsed.Host == "" {
-		return fmt.Errorf("must include a host")
-	}
-	if parsed.RawQuery != "" {
-		return fmt.Errorf("must not include a query string")
-	}
-	if parsed.Fragment != "" {
-		return fmt.Errorf("must not include a fragment")
-	}
-	return nil
-}

internal/storage/fake/backend.go

@@ -3,7 +3,6 @@ package fake
 import (
 	"bytes"
 	"context"
-	"errors"
 	"io"
 	"sort"
 	"strings"
@@ -133,14 +132,14 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
 	if err := storage.ValidatePrefix(prefix); err != nil {
 		return err
 	}
+	emitter := storage.NewWalkEmitter(ctx, backendName, opts, fn)
 	if entry, err := b.Stat(ctx, prefix); err == nil && entry.Type != storage.EntryTypeDirectory {
-		return emit(ctx, entry, opts, fn)
+		return storage.FinishWalk(emitter.Emit(entry))
 	} else if err != nil && !storage.IsNotFound(err) {
 		return err
 	}
 
 	entries := b.entries()
-	visited := 0
 	for _, entry := range entries {
 		if entry.Path == "" || !entryBelow(prefix, entry.Path) {
 			continue
@@ -148,33 +147,15 @@ func (b *Backend) Walk(ctx context.Context, prefix string, opts storage.WalkOpti
 		if !opts.Recursive && !isImmediateChild(prefix, entry.Path) {
 			continue
 		}
-		if opts.Limit > 0 && visited >= opts.Limit {
-			return nil
-		}
-		visited++
-		if err := ctx.Err(); err != nil {
-			return err
-		}
-		if err := fn(entry); err != nil {
-			if errors.Is(err, storage.ErrStopWalk) {
-				return nil
-			}
-			return storage.NewError(storage.OpWalk, backendName, entry.Path, storage.ErrUnknown, err)
+		if err := emitter.Emit(entry); err != nil {
+			return storage.FinishWalk(err)
 		}
 	}
 	return nil
 }
 
 func (b *Backend) HasAny(ctx context.Context, prefix string) (bool, error) {
-	found := false
-	err := b.Walk(ctx, prefix, storage.WalkOptions{Recursive: false, Limit: 1}, func(storage.Entry) error {
-		found = true
-		return storage.ErrStopWalk
-	})
-	if err != nil {
-		return false, err
-	}
-	return found, nil
+	return storage.HasAny(ctx, b, prefix)
 }
 
 func (b *Backend) DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts storage.DeleteOptions) error {
@@ -302,19 +283,6 @@ func (b *Backend) entries() []storage.Entry {
 	return entries
 }
 
-func emit(ctx context.Context, entry storage.Entry, opts storage.WalkOptions, fn storage.WalkFunc) error {
-	if opts.Limit > 0 && opts.Limit < 1 {
-		return nil
-	}
-	if err := ctx.Err(); err != nil {
-		return err
-	}
-	if err := fn(entry); err != nil && !errors.Is(err, storage.ErrStopWalk) {
-		return err
-	}
-	return nil
-}
-
 func entryBelow(prefix, path string) bool {
 	if prefix == "" {
 		return path != ""

internal/storage/walk.go

@@ -0,0 +1,63 @@
+package storage
+
+import (
+	"context"
+	"errors"
+)
+
+type WalkEmitter struct {
+	ctx     context.Context
+	backend string
+	opts    WalkOptions
+	fn      WalkFunc
+	count   int
+}
+
+func NewWalkEmitter(ctx context.Context, backend string, opts WalkOptions, fn WalkFunc) *WalkEmitter {
+	return &WalkEmitter{
+		ctx:     ctx,
+		backend: backend,
+		opts:    opts,
+		fn:      fn,
+	}
+}
+
+func (e *WalkEmitter) Emit(entry Entry) error {
+	if err := e.ctx.Err(); err != nil {
+		return err
+	}
+	if e.opts.Limit > 0 && e.count >= e.opts.Limit {
+		return ErrStopWalk
+	}
+	e.count++
+	if err := e.fn(entry); err != nil {
+		if errors.Is(err, ErrStopWalk) {
+			return ErrStopWalk
+		}
+		return NewError(OpWalk, e.backend, entry.Path, ErrUnknown, err)
+	}
+	return nil
+}
+
+func (e *WalkEmitter) LimitReached() bool {
+	return e.opts.Limit > 0 && e.count >= e.opts.Limit
+}
+
+func FinishWalk(err error) error {
+	if errors.Is(err, ErrStopWalk) {
+		return nil
+	}
+	return err
+}
+
+func HasAny(ctx context.Context, backend Backend, prefix string) (bool, error) {
+	found := false
+	err := backend.Walk(ctx, prefix, WalkOptions{Recursive: false, Limit: 1}, func(Entry) error {
+		found = true
+		return ErrStopWalk
+	})
+	if err != nil {
+		return false, err
+	}
+	return found, nil
+}

internal/storage/walk_test.go

@@ -0,0 +1,107 @@
+package storage
+
+import (
+	"context"
+	"errors"
+	"testing"
+)
+
+func TestWalkEmitterHonorsLimit(t *testing.T) {
+	emitter := NewWalkEmitter(context.Background(), "test", WalkOptions{Limit: 2}, func(Entry) error {
+		return nil
+	})
+
+	if err := emitter.Emit(Entry{Path: "one"}); err != nil {
+		t.Fatalf("first Emit() error = %v", err)
+	}
+	if err := emitter.Emit(Entry{Path: "two"}); err != nil {
+		t.Fatalf("second Emit() error = %v", err)
+	}
+	if err := emitter.Emit(Entry{Path: "three"}); !errors.Is(err, ErrStopWalk) {
+		t.Fatalf("third Emit() error = %v, want ErrStopWalk", err)
+	}
+	if !emitter.LimitReached() {
+		t.Fatal("LimitReached() = false, want true")
+	}
+}
+
+func TestWalkEmitterStopsWithoutError(t *testing.T) {
+	emitter := NewWalkEmitter(context.Background(), "test", WalkOptions{}, func(Entry) error {
+		return ErrStopWalk
+	})
+
+	err := emitter.Emit(Entry{Path: "one"})
+	if !errors.Is(err, ErrStopWalk) {
+		t.Fatalf("Emit() error = %v, want ErrStopWalk", err)
+	}
+	if err := FinishWalk(err); err != nil {
+		t.Fatalf("FinishWalk() error = %v, want nil", err)
+	}
+}
+
+func TestWalkEmitterWrapsCallbackErrors(t *testing.T) {
+	callbackErr := errors.New("callback failed")
+	emitter := NewWalkEmitter(context.Background(), "test", WalkOptions{}, func(Entry) error {
+		return callbackErr
+	})
+
+	err := emitter.Emit(Entry{Path: "one"})
+	if !errors.Is(err, callbackErr) {
+		t.Fatalf("Emit() error = %v, want callback error", err)
+	}
+	var storageErr *Error
+	if !errors.As(err, &storageErr) {
+		t.Fatalf("Emit() error type = %T, want *Error", err)
+	}
+	if storageErr.Op != OpWalk || storageErr.Backend != "test" || storageErr.Path != "one" || storageErr.Kind != ErrUnknown {
+		t.Fatalf("wrapped error = %#v", storageErr)
+	}
+}
+
+func TestWalkEmitterHonorsContextCancellation(t *testing.T) {
+	ctx, cancel := context.WithCancel(context.Background())
+	cancel()
+	called := false
+	emitter := NewWalkEmitter(ctx, "test", WalkOptions{}, func(Entry) error {
+		called = true
+		return nil
+	})
+
+	err := emitter.Emit(Entry{Path: "one"})
+	if !errors.Is(err, context.Canceled) {
+		t.Fatalf("Emit() error = %v, want context.Canceled", err)
+	}
+	if called {
+		t.Fatal("callback was called after context cancellation")
+	}
+}
+
+func TestHasAnyUsesNonRecursiveLimitOneWalk(t *testing.T) {
+	backend := &recordingBackend{}
+
+	found, err := HasAny(context.Background(), backend, "bundle")
+	if err != nil {
+		t.Fatalf("HasAny() error = %v", err)
+	}
+	if !found {
+		t.Fatal("HasAny() = false, want true")
+	}
+	if backend.prefix != "bundle" {
+		t.Fatalf("walk prefix = %q, want bundle", backend.prefix)
+	}
+	if backend.opts != (WalkOptions{Recursive: false, Limit: 1}) {
+		t.Fatalf("walk options = %#v, want non-recursive limit one", backend.opts)
+	}
+}
+
+type recordingBackend struct {
+	Backend
+	prefix string
+	opts   WalkOptions
+}
+
+func (b *recordingBackend) Walk(_ context.Context, prefix string, opts WalkOptions, fn WalkFunc) error {
+	b.prefix = prefix
+	b.opts = opts
+	return FinishWalk(fn(Entry{Path: "bundle/file.txt", Type: EntryTypeFile}))
+}

internal/testutil/fixtures.go

@@ -3,6 +3,7 @@ package testutil
 import (
 	"context"
 	"encoding/json"
+	"fmt"
 	"os"
 	"path/filepath"
 	"strings"
@@ -106,6 +107,53 @@ func WriteFakeSourceBundle(t testing.TB, backend *fake.Backend, relative string,
 	return bundle.Bundle{RootRelativePath: relative, Manifest: manifest}
 }
 
+func WriteFakeFile(t testing.TB, backend *fake.Backend, path, data string) {
+	t.Helper()
+	if _, err := backend.WriteFile(context.Background(), path, []byte(data), storage.WriteOptions{}); err != nil {
+		t.Fatalf("write fake file %s: %v", path, err)
+	}
+}
+
+func AssertFakeFile(t testing.TB, backend *fake.Backend, path, want string) {
+	t.Helper()
+	data, err := backend.ReadFile(context.Background(), path)
+	if err != nil {
+		t.Fatalf("read fake file %s: %v", path, err)
+	}
+	if got := string(data); got != want {
+		t.Fatalf("fake file %s = %q, want %q", path, got, want)
+	}
+}
+
+func AssertFakeMissing(t testing.TB, backend *fake.Backend, path string) {
+	t.Helper()
+	if _, err := backend.Stat(context.Background(), path); !storage.IsNotFound(err) {
+		t.Fatalf("fake file %s stat error = %v, want not found", path, err)
+	}
+}
+
+func WriteFakeDestinationState(t testing.TB, backend *fake.Backend, relative string, manifest bundle.Manifest, opts DestinationStateOptions) state.DistributorState {
+	t.Helper()
+	destinationState := DestinationState(manifest, opts)
+	data, err := json.MarshalIndent(destinationState, "", "  ")
+	if err != nil {
+		t.Fatalf("marshal destination state: %v", err)
+	}
+	statePath, err := storage.StatePath(relative)
+	if err != nil {
+		t.Fatalf("state path: %v", err)
+	}
+	WriteFakeFile(t, backend, statePath, string(append(data, '\n')))
+	for _, output := range destinationState.Outputs {
+		path, err := storage.Join(relative, output.Path)
+		if err != nil {
+			t.Fatalf("join output path: %v", err)
+		}
+		WriteFakeFile(t, backend, path, "old")
+	}
+	return destinationState
+}
+
 func WriteMinimalLocalConfig(t testing.TB, sourceRoot, destinationRoot string) string {
 	t.Helper()
 	return writeConfigFile(t, `
@@ -139,6 +187,136 @@ pipelines:
 `)
 }
 
+func WriteLocalConfigWithPublishPolicy(t testing.TB, sourceRoot, destinationRoot string, publishSource, publishHTML bool) string {
+	t.Helper()
+	transformConfig := ""
+	if publishHTML {
+		transformConfig = `
+        transform:
+          markdown_to_html:
+            enabled: true
+            mode: sidecar`
+	}
+	return writeConfigFile(t, `
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: `+sourceRoot+`
+    destinations:
+      - id: archive
+        backend: local
+        path: `+destinationRoot+`
+        publish:
+          source: `+fmt.Sprintf("%t", publishSource)+`
+          html: `+fmt.Sprintf("%t", publishHTML)+transformConfig+`
+`)
+}
+
+func WriteLocalConfigWithPathMapping(t testing.TB, sourceRoot, destinationRoot, mode string) string {
+	t.Helper()
+	return writeConfigFile(t, `
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: `+sourceRoot+`
+    destinations:
+      - id: archive
+        backend: local
+        path: `+destinationRoot+`
+        path_mapping:
+          mode: `+mode+`
+`)
+}
+
+func WriteLocalConfigWithLinks(t testing.TB, sourceRoot, destinationRoot, pathMapping, baseURL, primary string, publishSource, publishHTML bool, transformMode string) string {
+	t.Helper()
+	transformConfig := ""
+	if publishHTML {
+		transformConfig = `
+        transform:
+          markdown_to_html:
+            enabled: true
+            mode: ` + transformMode
+	}
+	return writeConfigFile(t, `
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: `+sourceRoot+`
+    destinations:
+      - id: archive
+        backend: local
+        path: `+destinationRoot+`
+        path_mapping:
+          mode: `+pathMapping+`
+        links:
+          base_url: `+baseURL+`
+          primary: `+primary+`
+        publish:
+          source: `+fmt.Sprintf("%t", publishSource)+`
+          html: `+fmt.Sprintf("%t", publishHTML)+transformConfig+`
+`)
+}
+
+func WriteLocalConfigWithMarkdownTransform(t testing.TB, sourceRoot, destinationRoot string, publishSource, publishHTML bool, mode, input string) string {
+	t.Helper()
+	enabled := publishHTML
+	inputConfig := ""
+	if input != "" {
+		inputConfig = `
+            input: ` + input
+	}
+	return writeConfigFile(t, `
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: `+sourceRoot+`
+    destinations:
+      - id: archive
+        backend: local
+        path: `+destinationRoot+`
+        publish:
+          source: `+fmt.Sprintf("%t", publishSource)+`
+          html: `+fmt.Sprintf("%t", publishHTML)+`
+        transform:
+          markdown_to_html:
+            enabled: `+fmt.Sprintf("%t", enabled)+`
+            mode: `+mode+inputConfig+`
+`)
+}
+
+func WriteMixedPolicyFanoutLocalConfig(t testing.TB, sourceRoot, archiveDestination, htmlDestination string) string {
+	t.Helper()
+	return writeConfigFile(t, `
+pipelines:
+  - id: reports
+    source:
+      backend: local
+      path: `+sourceRoot+`
+    destinations:
+      - id: archive
+        backend: local
+        path: `+archiveDestination+`
+        publish:
+          source: true
+          html: false
+      - id: html
+        backend: local
+        path: `+htmlDestination+`
+        publish:
+          source: false
+          html: true
+        transform:
+          markdown_to_html:
+            enabled: true
+            mode: sidecar
+`)
+}
+
 func WriteDestinationState(t testing.TB, root, relative string, manifest bundle.Manifest, opts DestinationStateOptions) state.DistributorState {
 	t.Helper()
 	bundleRoot := filepath.Join(root, filepath.FromSlash(relative))
@@ -175,6 +353,28 @@ func ReadDestinationState(t testing.TB, path string) state.DistributorState {
 	return destinationState
 }
 
+func AssertFile(t testing.TB, path, want string) {
+	t.Helper()
+	data, err := os.ReadFile(path)
+	if err != nil {
+		t.Fatalf("read file %s: %v", path, err)
+	}
+	if got := string(data); got != want {
+		t.Fatalf("%s = %q, want %q", path, got, want)
+	}
+}
+
+func AssertFileContains(t testing.TB, path, want string) {
+	t.Helper()
+	data, err := os.ReadFile(path)
+	if err != nil {
+		t.Fatalf("read file %s: %v", path, err)
+	}
+	if !strings.Contains(string(data), want) {
+		t.Fatalf("%s = %q, want substring %q", path, data, want)
+	}
+}
+
 func sourceFiles(opts BundleOptions) []SourceFile {
 	files := opts.Files
 	if files == nil {

pkg/bundle/digest.go

@@ -11,7 +11,8 @@ import (
 
 var digestPattern = regexp.MustCompile(`^sha256:[0-9a-f]{64}$`)
 
-func validateDigest(value string) error {
+// ValidateDigest reports whether value uses the lowercase sha256:<64 hex> form.
+func ValidateDigest(value string) error {
 	if !digestPattern.MatchString(value) {
 		return fmt.Errorf("must be lowercase sha256:<64 hex>")
 	}

pkg/bundle/digest_test.go

@@ -0,0 +1,58 @@
+package bundle
+
+import "testing"
+
+func TestValidateDigest(t *testing.T) {
+	tests := []struct {
+		name    string
+		value   string
+		wantErr bool
+	}{
+		{
+			name:  "valid",
+			value: "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
+		},
+		{
+			name:    "uppercase hex",
+			value:   "sha256:0123456789ABCDEF0123456789abcdef0123456789abcdef0123456789abcdef",
+			wantErr: true,
+		},
+		{
+			name:    "missing prefix",
+			value:   "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
+			wantErr: true,
+		},
+		{
+			name:    "wrong algorithm",
+			value:   "sha512:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
+			wantErr: true,
+		},
+		{
+			name:    "short hex",
+			value:   "sha256:0123456789abcdef",
+			wantErr: true,
+		},
+		{
+			name:    "long hex",
+			value:   "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0",
+			wantErr: true,
+		},
+		{
+			name:    "non hex",
+			value:   "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdeg",
+			wantErr: true,
+		},
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			err := ValidateDigest(test.value)
+			if test.wantErr && err == nil {
+				t.Fatal("ValidateDigest() error = nil, want error")
+			}
+			if !test.wantErr && err != nil {
+				t.Fatalf("ValidateDigest() error = %v", err)
+			}
+		})
+	}
+}

pkg/bundle/validate.go

@@ -13,7 +13,7 @@ func ValidateManifest(manifest Manifest) error {
 	if manifest.ID == "" {
 		return fmt.Errorf("id is required")
 	}
-	if err := validateDigest(manifest.Digest); err != nil {
+	if err := ValidateDigest(manifest.Digest); err != nil {
 		return fmt.Errorf("digest: %w", err)
 	}
 	if manifest.Created.IsZero() {
@@ -27,7 +27,7 @@ func ValidateManifest(manifest Manifest) error {
 		if err := ValidateSourcePath(file.Path); err != nil {
 			return fmt.Errorf("files[%d].path: %w", index, err)
 		}
-		if err := validateDigest(file.SHA256); err != nil {
+		if err := ValidateDigest(file.SHA256); err != nil {
 			return fmt.Errorf("files[%d].sha256: %w", index, err)
 		}
 		if file.Size < 0 {