Update woodpecker to use golang:1.26

.woodpecker/release.yml

@@ -0,0 +1,50 @@
+when:
+  - event: tag
+
+steps:
+  - name: build-release-assets
+    image: golang:1.26
+    commands:
+      - |
+        set -eu
+
+        version="$CI_COMMIT_TAG"
+        dist="dist"
+        pkg="gitea.maximumdirect.net/eric/distributor/cmd/distributor"
+
+        rm -rf "$dist"
+        mkdir -p "$dist"
+
+        build_binary() {
+          goos="$1"
+          goarch="$2"
+          suffix="$3"
+          output="$dist/distributor-$version-$goos-$goarch$suffix"
+
+          CGO_ENABLED=0 GOOS="$goos" GOARCH="$goarch" \
+            go build -trimpath -ldflags "-s -w -X gitea.maximumdirect.net/eric/distributor/internal/buildinfo.Version=$version" \
+            -o "$output" "$pkg"
+        }
+
+        build_binary linux amd64 ""
+        build_binary linux arm64 ""
+        build_binary darwin amd64 ""
+        build_binary darwin arm64 ""
+        build_binary windows amd64 ".exe"
+        build_binary windows arm64 ".exe"
+
+  - name: publish-release
+    image: woodpeckerci/plugin-release
+    depends_on:
+      - build-release-assets
+    settings:
+      api_key:
+        from_secret: GITEA_RELEASE_TOKEN
+      files:
+        - dist/distributor-*
+      checksum: sha256
+      checksum-file: SHA256SUMS
+      checksum-flatten: true
+      file-exists: skip
+      overwrite: false
+      prerelease: false

README.md

@@ -1,6 +1,8 @@
 # distributor
 
-`distributor` validates and publishes manifested report bundles.
+`distributor` validates manifested report bundles and publishes selected source or generated artifacts to configured destinations.
+
+It is currently a local-first CLI: source bundles are read from local storage, destinations are local directories, and Markdown files can be rendered to HTML sidecars.
 
 Run the local example pipeline:
 
@@ -8,4 +10,4 @@ Run the local example pipeline:
 go run ./cmd/distributor run --config examples/local-publish.yml
 ```
 
-See [docs/cli.md](docs/cli.md), [docs/config.md](docs/config.md), and [docs/operations.md](docs/operations.md) for the implemented CLI, configuration, and operating notes. Current design and implementation planning lives under `docs/roadmap/`.
+See [docs/cli.md](docs/cli.md), [docs/config.md](docs/config.md), [docs/operations.md](docs/operations.md), and [docs/troubleshooting.md](docs/troubleshooting.md) for the implemented CLI, configuration, operating notes, and common failure modes. Planning material lives under `docs/roadmap/`.

docs/cli.md

@@ -6,44 +6,41 @@
 go run ./cmd/distributor run --config examples/local-publish.yml
 ```
 
-This validates and publishes the example source bundle to `workspace/published/source-bundle`.
+This discovers the example source bundle and publishes source files to `workspace/published/source-bundle`.
 
 ## Command overview
 
 ```sh
-distributor --help
+distributor [--help]
 distributor version
-distributor run
-distributor validate
-distributor inspect
+distributor run [--config <path>] [--dry-run]
+distributor validate <path>
+distributor inspect <path>
 ```
 
-`version` prints the application name and version. The default development version is `dev`; release builds may replace it at build time.
+- `version`: prints the application name and version. Development builds print `distributor dev`.
+- `run`: loads a YAML config, discovers local source bundles, plans each configured destination, writes selected outputs unless `--dry-run` is set, and prints a final status summary.
+- `validate`: validates a local source bundle directory or a local tree containing source bundles.
+- `inspect`: validates local source bundles and prints normalized bundle metadata.
 
-`run --config <path>` executes configured local-to-local pipelines that publish source files, generated HTML files, or both.
-
-`run --config <path> --dry-run` discovers source bundles, inspects destination state, and prints planned actions plus a final status summary without writing files.
-
-`validate <path>` validates a local source bundle directory or a local tree containing source bundles.
-
-`inspect <path>` validates discovered local source bundles and prints a concise normalized summary.
-
-Remote backends are not implemented yet.
+`validate` and `inspect` accept local paths only. `run` currently executes local backends only. SSH and S3 config can be parsed and validated, but configured SSH or S3 execution fails with a clear unsupported-execution error.
 
 ## Flag reference
 
-The root command supports:
+Root command:
 
-- `--help`, `-h`: print root help.
+- `--help`, `-h`, or `help`: print root help.
 
-Each subcommand supports:
+All subcommands:
 
 - `--help`, `-h`: print command-specific help.
 
-`run` supports:
+`run` flags:
 
-- `--config <path>`: config file to load.
-- `--dry-run`: validate config, print planned actions and final status, and do not publish.
+- `--config <path>`: config file to load. If omitted, `run` uses `/usr/local/etc/distributor/config.yml`.
+- `--dry-run`: load config, discover bundles, inspect destination state, print planned actions and final status, and do not write files.
+
+`run` does not accept positional arguments. `validate` and `inspect` accept at most one path; omitting the path returns a required-path error.
 
 ## Common workflows
 
@@ -65,7 +62,7 @@ Preview local publication without writing:
 go run ./cmd/distributor run --config examples/local-publish.yml --dry-run
 ```
 
-Publish the local example:
+Publish the local source example:
 
 ```sh
 go run ./cmd/distributor run --config examples/local-publish.yml
@@ -76,3 +73,27 @@ Publish the local HTML example:
 ```sh
 go run ./cmd/distributor run --config examples/local-html.yml
 ```
+
+Preview local fan-out publication:
+
+```sh
+go run ./cmd/distributor run --config examples/fan-out.yml --dry-run
+```
+
+## Output
+
+`run` prints the number of configured pipelines, one line per pipeline, one line per planned destination action, and a final status line. Actions include:
+
+- `publish_new`: destination has no managed state and is empty.
+- `replace_older`: destination state is older than the source manifest.
+- `skip_same`: destination state already matches the source manifest.
+- `skip_destination_newer`: destination state is newer than the source manifest.
+- `error`: planning or execution failed for that destination.
+
+The command exits non-zero if any destination fails. Independent later destinations are still attempted.
+
+## Diagnostics
+
+Use `validate` before publication when a producer has written a new bundle. Use `inspect` to confirm normalized ids, timestamps, digests, file paths, and file sizes.
+
+For symptom-oriented recovery steps, see [troubleshooting](troubleshooting.md). For destination state and retry behavior, see [operations](operations.md). For config fields and defaults, see [configuration](config.md).

docs/config.md

@@ -1,18 +1,18 @@
 # Distributor Configuration
 
-## Config file location
+## Config File Location
 
-`distributor run --config <path>` loads the YAML config at the path provided by `--config`.
+`distributor run --config <path>` loads the YAML config at the provided path.
 
-If `--config` is omitted during run, the built-in default path is:
+If `--config` is omitted, `run` uses:
 
 ```text
 /usr/local/etc/distributor/config.yml
 ```
 
-The current implementation supports local-to-local publication of source files, generated HTML files, or both. Remote backends are not implemented yet.
+Config parsing rejects unknown YAML fields. The current executable backend support is local only. SSH and S3 config fields are accepted by config validation, but runtime execution for those backends is unavailable.
 
-## Minimal config
+## Minimal Local Config
 
 ```yaml
 pipelines:
@@ -26,9 +26,9 @@ pipelines:
         path: /srv/reports/archive
 ```
 
-This uses the default publish policy of source files only and the default transfer policy.
+This publishes source files only. It uses the default validation and transfer policies.
 
-## Production-oriented config
+## Production-Oriented Local Config
 
 ```yaml
 pipelines:
@@ -52,6 +52,22 @@ pipelines:
           on_conflict: fail
 ```
 
+## HTML Publication
+
+To publish generated HTML from Markdown files:
+
+```yaml
+publish:
+  source: false
+  html: true
+transform:
+  markdown_to_html:
+    enabled: true
+    mode: sidecar
+```
+
+Sidecar generation writes `report.html` for `report.md`. It does not mutate the source bundle.
+
 ## Reference
 
 Top level:
@@ -60,37 +76,78 @@ Top level:
 
 Pipeline:
 
-- `id`: required unique identifier.
+- `id`: required unique slug-like identifier.
 - `source`: required backend config.
+- `validation.on_digest_mismatch`: optional; defaults to `fail`; only `fail` is supported.
 - `destinations`: required non-empty destination list.
-- `validation.on_digest_mismatch`: optional, defaults to `fail`; only `fail` is supported.
 
-Backends:
+Source backend:
 
-- `local`: requires `path`.
-- `ssh`: requires `uri` and `path`.
-- `s3`: requires `endpoint` and `bucket`; supports optional `prefix`, `region`, `force_path_style`, and `credentials`.
+- `backend`: required.
+- `path`: required for `local` and `ssh`.
+- `uri`: required for `ssh`.
+- `endpoint`: required for `s3`.
+- `bucket`: required for `s3`.
+- `prefix`: optional for `s3`.
+- `region`: optional for `s3`.
+- `force_path_style`: optional for `s3`.
+- `credentials.access_key_id_env`: optional S3 credential environment variable name.
+- `credentials.secret_access_key_env`: optional S3 credential environment variable name.
 
-Destination policy:
+Destination:
+
+- `id`: required unique slug-like identifier within the pipeline.
+- Backend fields: same accepted shape as source backends, with destination fields at the destination level.
+- `publish`: optional; defaults to source-only publication.
+- `transform`: required only for generated HTML publication.
+- `transfer`: optional; defaults described below.
+
+Accepted backend names:
+
+- `local`: executable; requires `path`.
+- `ssh`: config validation only; execution is unavailable.
+- `s3`: config validation only; execution is unavailable.
+
+Publish policy:
 
 - `publish.source`: publish source artifacts.
 - `publish.html`: publish generated HTML artifacts from Markdown source files.
-- `transfer.on_destination_same`: `skip` or `fail`, defaults to `skip`.
-- `transfer.on_destination_older`: `replace` or `fail`, defaults to `replace`.
-- `transfer.on_destination_newer`: `skip` or `fail`, defaults to `skip`.
-- `transfer.on_conflict`: only `fail`, defaults to `fail`.
 
-When `publish.html` is true, `transform.markdown_to_html.enabled: true` and `transform.markdown_to_html.mode: sidecar` are required.
+At least one output type must be enabled. When `publish.html` is true, `transform.markdown_to_html.enabled` must be `true` and `transform.markdown_to_html.mode` must be `sidecar`.
 
-Markdown-to-HTML sidecar generation writes `report.html` for `report.md` and does not mutate the source bundle.
+Transfer policy:
+
+- `transfer.on_destination_same`: `skip` or `fail`; defaults to `skip`.
+- `transfer.on_destination_older`: `replace` or `fail`; defaults to `replace`.
+- `transfer.on_destination_newer`: `skip` or `fail`; defaults to `skip`.
+- `transfer.on_conflict`: only `fail`; defaults to `fail`.
+
+## Defaults
+
+Defaults are applied after YAML decoding and before validation:
+
+- `validation.on_digest_mismatch: fail`
+- `publish.source: true`
+- `publish.html: false`
+- `transfer.on_destination_same: skip`
+- `transfer.on_destination_older: replace`
+- `transfer.on_destination_newer: skip`
+- `transfer.on_conflict: fail`
 
 ## Secrets
 
-Do not put literal secrets in config files. S3 credentials may refer to environment variable names with:
+Do not put literal secrets in config files. S3 credentials may name environment variables:
 
 - `credentials.access_key_id_env`
 - `credentials.secret_access_key_env`
 
+S3 execution is unavailable; these fields are accepted so config shape can be validated.
+
 ## Examples
 
-Maintained examples live under [examples/](../examples/).
+Maintained examples live under [examples](../examples/):
+
+- `local-to-local.yml`: minimal local config.
+- `local-publish.yml`: runnable local source publication.
+- `local-html.yml`: runnable local HTML publication.
+- `fan-out.yml`: runnable local fan-out publication to source and HTML destinations.

docs/integrations/markdown.md

@@ -0,0 +1,55 @@
+# Markdown Integration
+
+## Purpose
+
+Markdown-to-HTML is the only implemented external file-format integration. This note documents the renderer behavior that is externally visible in generated destination artifacts.
+
+## Dependency
+
+Rendering uses `github.com/yuin/goldmark`. The exact dependency version is pinned in `go.mod`; review that file before changing renderer behavior or diagnosing version-specific output changes.
+
+## Renderer behavior
+
+`internal/transform/markdown.New` constructs the renderer with `goldmark.New()` and no project-specific extensions or renderer options.
+
+For each source bundle file ending in `.md`, the transform reads the Markdown source and generates an HTML sidecar in the same logical directory. The output path replaces the `.md` suffix with `.html`, so `report.md` produces `report.html`. Non-Markdown source files produce no Markdown outputs.
+
+Raw HTML embedded in Markdown is not passed through by the current renderer behavior. Tests allow Goldmark's disabled-or-escaped raw HTML output forms and reject literal script tags in generated HTML.
+
+## Wrapper
+
+Rendered Markdown body HTML is wrapped in a fixed document shell:
+
+- `<!doctype html>`
+- `<html lang="en">`
+- UTF-8 `<meta charset>`
+- empty `<title>`
+- `<body>` containing the rendered Markdown body
+
+The wrapper is deterministic and does not read configuration, templates, CSS, or source manifest metadata.
+
+## Output metadata
+
+Generated outputs record:
+
+- destination path;
+- source path;
+- transform id `markdown_to_html`;
+- SHA-256 digest of the wrapped HTML bytes;
+- byte size of the wrapped HTML bytes.
+
+## Boundaries
+
+Markdown rendering does not mutate source bundles, publish files, write `.distributor.json`, select outputs, or choose transfer actions. Publish planning decides whether generated HTML is selected for a destination.
+
+Only sidecar output mode is supported for current behavior.
+
+## Tests
+
+Before changing Markdown renderer behavior, inspect and run:
+
+```bash
+go test ./internal/transform/markdown
+```
+
+The tests cover sidecar naming, ignored non-Markdown files, raw HTML handling, deterministic output, digest metadata, and size metadata.

docs/internal/app.md

@@ -0,0 +1,64 @@
+# Application Orchestration
+
+## Purpose
+
+`internal/app` owns top-level use cases for `run`, `validate`, and `inspect`. It wires configuration, storage backends, transforms, publish planning, execution, summaries, and notification handoff.
+
+## Inputs and outputs
+
+`Run` accepts a context, optional config path, dry-run flag, stdout writer, and optional notifier. It loads YAML config, discovers source bundles for each configured pipeline, plans each destination independently, optionally executes publish plans, writes summary output when stdout is supplied, and returns an aggregated error if any destination fails.
+
+`Validate` and `Inspect` accept a local path. `Validate` discovers and validates bundles. `Inspect` writes bundle metadata and manifest file entries to stdout when provided.
+
+## Run flow
+
+The runner:
+
+1. loads config from the supplied path or `config.DefaultConfigPath`;
+2. opens the configured source backend;
+3. discovers validated bundles from the source root;
+4. opens each destination backend independently;
+5. builds a publish plan for each bundle and destination;
+6. prints plan lines and records summary counters;
+7. executes publish or replacement plans unless dry-run is enabled;
+8. invokes the notifier after successful publish or replacement actions.
+
+Destination failures are collected while later destinations continue to run. Source open and source discovery failures stop the run because there are no valid bundles to fan out.
+
+## Backend and transform wiring
+
+The app-level backend factory registers only the local backend for execution. Config validation accepts other backend shapes, but `Run` can execute only local sources and local destinations.
+
+The app-level transform registry registers Markdown-to-HTML using `internal/transform/markdown`. Lower-level publish code receives a resolver and does not import concrete transform implementations.
+
+## Dry-run behavior
+
+Dry-run still loads config, opens backends, discovers bundles, inspects destinations, resolves transforms, and builds publish plans. It does not write destination outputs, write `.distributor.json`, delete managed outputs, or notify.
+
+## Failure behavior
+
+`Run` returns immediately for config loading errors, context cancellation before work starts, source open errors, and source discovery errors. Per-destination backend, planning, execution, and notification errors are aggregated into one run error after remaining destinations have been attempted.
+
+Stdout write errors are returned immediately because the caller's requested output stream can no longer be trusted.
+
+## Boundaries
+
+`internal/app` coordinates packages but does not own manifest validation rules, destination state comparison, storage path rules, output planning, transform rendering, or backend-specific filesystem behavior.
+
+`Validate` and `Inspect` are local path commands. Remote execution wiring is outside current behavior.
+
+## Tests
+
+Before changing app orchestration, inspect tests under:
+
+- `internal/app`
+- `internal/cli`
+- `internal/publish`
+
+## Invariants
+
+- One source fans out to each destination independently.
+- Destination failures do not prevent later destinations from being planned.
+- Dry-run must not mutate destination storage or invoke notifications.
+- Concrete backend and transform registration stays at the app layer.
+- The default notifier is `notify.Noop`.

docs/internal/bundle.md

@@ -32,10 +32,22 @@ The bundle digest is SHA-256 of a deterministic JSON array of file records in ma
 
 Discovery walks a storage backend beneath a source root, finds `manifest.json` files, sorts bundle paths lexically, and rejects nested manifests.
 
+## Failure behavior
+
+Manifest parsing and validation fail before destination planning. Storage-backed validation fails when listed files are missing, are not regular files, have unexpected sizes, have unexpected SHA-256 digests, or when a source bundle includes unsafe or reserved paths.
+
 ## Boundaries
 
-Bundle code uses `internal/storage` and does not import local, SSH, or S3 adapters. CLI local path support is wired in `internal/app`.
+Bundle code uses `internal/storage` and does not import concrete adapters. CLI local path support is wired in `internal/app`.
 
 ## Tests
 
 Before changing bundle behavior, inspect tests under `internal/bundle`.
+
+## Invariants
+
+- `manifest.json` is the only source bundle contract.
+- Source file paths must stay relative to the bundle root.
+- The top-level bundle digest is derived from manifest file records in order.
+- Discovery order is lexical and deterministic.
+- Nested manifests are rejected.

docs/internal/config.md

@@ -0,0 +1,61 @@
+# Configuration Internals
+
+## Purpose
+
+`internal/config` defines YAML-backed configuration structs, defaulting, and validation for distributor pipelines.
+
+## Inputs and outputs
+
+Input is a YAML file containing `pipelines`. Output is a `Config` value with defaults applied and validation completed. Load failures include the config path and whether the failure occurred during file loading, YAML parsing, or validation.
+
+## Loading flow
+
+`LoadFile` opens the requested path, decodes YAML with known-field checking enabled, applies defaults, and validates the result. The app uses `DefaultConfigPath` when the CLI does not supply a config path.
+
+Known-field checking rejects misspelled or unknown YAML keys before defaults and validation run.
+
+## Defaults
+
+Defaults are applied in `ApplyDefaults`:
+
+- pipeline validation defaults `on_digest_mismatch` to `fail`;
+- destination publish policy defaults to source output only;
+- `transfer.on_destination_same` defaults to `skip`;
+- `transfer.on_destination_older` defaults to `replace`;
+- `transfer.on_destination_newer` defaults to `skip`;
+- `transfer.on_conflict` defaults to `fail`.
+
+## Validation responsibilities
+
+Validation requires at least one pipeline, slug-like unique pipeline ids, one source per pipeline, at least one destination, slug-like unique destination ids within each pipeline, backend-specific required fields, valid validation policy, valid publish and transform combinations, and valid transfer actions.
+
+`ValidatePublishTransformPolicy` is shared with publish planning so destination policy combinations are checked consistently. Publishing HTML requires an enabled Markdown-to-HTML transform in `sidecar` mode. A publish policy must select source output, HTML output, or both.
+
+## Executable support boundary
+
+Config validation accepts `local`, `ssh`, and `s3` backend shapes so config files can be validated as schemas. Runtime execution currently opens only local backends through `internal/app`.
+
+The user-facing configuration reference is `docs/config.md`; this file documents package behavior for maintainers.
+
+## Failure behavior
+
+Load errors wrap the underlying file, YAML, or validation error with context. Validation collects all detected field errors into one error value instead of stopping at the first invalid field.
+
+Unsupported backend names fail validation. Accepted backend names without runtime execution support fail later during app backend opening.
+
+## Tests
+
+Before changing config behavior, inspect:
+
+- `internal/config/load_test.go`
+- `internal/config/validate_test.go`
+- example-loading coverage in `internal/config`
+- user-facing examples under `examples/`
+
+## Invariants
+
+- Defaults are applied before validation.
+- Unknown YAML fields are rejected.
+- `docs/config.md` remains the canonical user-facing config reference.
+- Runtime backend execution support is not inferred from config validation support.
+- New user-visible config behavior must be covered by tests and docs in the same change.

docs/internal/notify.md

@@ -4,10 +4,32 @@
 
 `internal/notify` defines the internal notification interface used by the application runner.
 
+## Inputs and outputs
+
+Input is a notification event containing pipeline id, destination id, bundle id, bundle path, action, and output metadata. The interface returns an error so app orchestration can treat notification failures as destination failures.
+
 ## Current behavior
 
 The implemented notifier is a no-op. It is invoked only after a successful publish or replacement. Dry-run, skipped destinations, and failed destinations do not invoke it.
 
+## Failure behavior
+
+`notify.Noop` always succeeds unless the context is already canceled. If a configured notifier returns an error, `internal/app` records that destination as failed and continues with remaining destinations.
+
 ## Boundaries
 
-No external notification adapters are implemented. Notification configuration is not part of the current user-facing config schema.
+External notification adapters and user-facing notification configuration are outside current behavior.
+
+## Tests
+
+Before changing notification behavior, inspect:
+
+- `internal/notify`
+- `internal/app/run_test.go`
+
+## Invariants
+
+- Notifications are emitted only after successful publish or replacement execution.
+- Dry-run never notifies.
+- Skipped and failed destinations never notify.
+- The default app notifier is `notify.Noop`.

docs/internal/publish.md

@@ -12,11 +12,17 @@ Output is a plan with an action, reason, and selected source or generated output
 
 ## Actions
 
-Supported actions are publish new, replace older destination, skip same source, skip newer destination, fail conflict, and fail unmanaged destination.
+Supported actions are `publish_new`, `replace_older`, `skip_same`, `skip_destination_newer`, `fail_conflict`, and `fail_unmanaged`.
+
+## Failure behavior
+
+Planning fails when request fields are incomplete, publish and transform policies are invalid, selected outputs collide, HTML output is requested without Markdown inputs, destination state is invalid, destination content is unmanaged, or transfer policy maps the comparison outcome to failure.
+
+Execution fails if a write, delete, state serialization, or context check fails. Outputs written during a failed publish attempt are cleaned up through managed deletion where possible.
 
 ## Boundaries
 
-The current implementation publishes source files and Markdown-to-HTML sidecar outputs. Remote backend execution is not implemented.
+The current implementation publishes source files and Markdown-to-HTML sidecar outputs. Backend behavior is supplied through `internal/storage`; app runtime currently supplies local backends.
 
 The package uses `internal/state` for destination comparison, `internal/storage` for IO, and the shared `internal/config` publish/transform policy helper for request validation. It resolves transforms through a narrow resolver supplied by the caller; concrete transform registration is owned by the app layer. It does not parse CLI flags or load config files.
 
@@ -27,3 +33,11 @@ Replacement deletes only outputs recorded in existing destination state plus `.d
 ## Tests
 
 Before changing publish behavior, inspect tests under `internal/publish` and local run tests under `internal/app`.
+
+## Invariants
+
+- Publish planning is deterministic for the same source, destination state, policies, and transform outputs.
+- Replacement deletes only managed paths recorded in existing state plus `.distributor.json`.
+- Publish execution writes destination state after selected outputs are written.
+- Transform implementations are resolved through an interface supplied by the caller.
+- Unmanaged destination content is never overwritten.

docs/internal/state.md

@@ -31,6 +31,10 @@ Each output records `path`, `kind`, `source_path`, `sha256`, and `size`. Support
 
 Comparison outcomes cover absent destination state, unmanaged destination content, invalid state, pipeline or destination mismatch, same source manifest, older destination source, newer destination source, same-created digest conflict, and different source id conflict.
 
+## Failure behavior
+
+Invalid JSON, invalid state schema, invalid embedded source manifests, unsafe output paths, unsupported output kinds, missing generated-output transform names, and mismatched pipeline or destination ids produce comparison outcomes that publish planning can turn into fail actions.
+
 ## Boundaries
 
 This package does not publish files, delete files, inspect storage backends, or choose transfer policy actions. Publish planning consumes these comparison outcomes later.
@@ -38,3 +42,11 @@ This package does not publish files, delete files, inspect storage backends, or
 ## Tests
 
 Before changing destination state behavior, inspect tests under `internal/state`.
+
+## Invariants
+
+- `.distributor.json` is the destination sentinel and state record.
+- Embedded source manifests use the same validation rules as source bundles.
+- Generated outputs always record a transform id.
+- Comparison returns outcomes and reasons; it does not mutate storage.
+- `distributor_version` is diagnostic metadata, not a comparison key.

docs/internal/storage.md

@@ -24,10 +24,18 @@ Logical file paths must be non-empty, relative, clean, slash-separated, and must
 
 Storage errors use typed categories such as not found, already exists, invalid path, conflict, permission, temporary, unsupported, and unknown. Callers should use helper predicates rather than matching error strings.
 
+Backends may wrap implementation-specific errors, but callers should receive storage errors where practical. Traversal can stop cleanly with `ErrStopWalk`.
+
 ## Deletion
 
 Backends expose guarded managed deletion only. `DeleteManagedBundle` may delete listed managed outputs plus `.distributor.json`; it does not provide broad recursive deletion.
 
+## Local and fake backends
+
+The local adapter maps logical paths to a configured filesystem root and keeps adapter-specific path handling behind the storage interface.
+
+The fake backend is an in-memory implementation for package tests. It is not registered for runtime use.
+
 ## Tests
 
 Before changing storage behavior, inspect tests under:
@@ -35,3 +43,11 @@ Before changing storage behavior, inspect tests under:
 - `internal/storage`
 - `internal/storage/fake`
 - `internal/adapters/local`
+
+## Invariants
+
+- Core packages depend on `internal/storage`, not concrete adapters.
+- Logical paths are slash-separated and confined to the backend root.
+- `storage.List` uses backend traversal and returns deterministic entries.
+- Managed deletion is limited to recorded outputs plus `.distributor.json`.
+- Runtime backend registration is owned by `internal/app`.

docs/internal/transform.md

@@ -8,12 +8,22 @@
 
 Inputs are a validated source bundle and source backend. Outputs include destination path, source path, transform id, generated bytes, SHA-256, and size.
 
+## Registry
+
+`internal/transform` defines the transform interface and registry. The app layer registers the Markdown implementation; publish planning receives only a resolver.
+
 ## Markdown behavior
 
 Markdown files ending in `.md` generate `.html` files in the same logical directory. Non-Markdown files do not generate outputs. Raw HTML embedded in Markdown is not passed through by the renderer.
 
 Generated HTML is deterministic for the same source content and transform configuration.
 
+See `docs/integrations/markdown.md` for the Goldmark integration contract.
+
+## Failure behavior
+
+Transform resolution fails when a requested transform id is not registered. Markdown rendering fails when the source file cannot be read or rendered. Publish planning fails when HTML output is requested and the selected transform produces no outputs for a bundle.
+
 ## Boundaries
 
 Transforms do not publish files, mutate source bundles, or write destination state. Publish planning selects and writes transform outputs.
@@ -22,4 +32,15 @@ The app layer owns default transform registration. The transform package does no
 
 ## Tests
 
-Before changing transform behavior, inspect tests under `internal/transform`.
+Before changing transform behavior, inspect tests under:
+
+- `internal/transform`
+- `internal/transform/markdown`
+
+## Invariants
+
+- Source bundle files are never mutated by transforms.
+- Generated outputs record destination path, source path, transform id, SHA-256, and size.
+- Markdown sidecar naming changes only the `.md` extension to `.html`.
+- Non-Markdown source files do not generate Markdown outputs.
+- Transform registration stays outside publish planning.

docs/operations.md

@@ -1,6 +1,12 @@
 # Distributor Operations
 
-## Normal workflow
+## Normal Workflow
+
+Validate a source bundle:
+
+```sh
+go run ./cmd/distributor validate examples/source-bundle
+```
 
 Preview a local publication:
 
@@ -20,30 +26,60 @@ Run the local HTML publication:
 go run ./cmd/distributor run --config examples/local-html.yml
 ```
 
-## Filesystem layout
+Preview local fan-out publication:
 
-Source bundles are discovered beneath the configured local source root. Destination bundle paths preserve the source bundle path relative to that source root.
+```sh
+go run ./cmd/distributor run --config examples/fan-out.yml --dry-run
+```
 
-The maintained example writes under `workspace/`, which is ignored by Git.
+## Filesystem Layout
 
-## Destination state
+Source bundles are discovered beneath the configured local source root. Each bundle is a directory containing `manifest.json`.
 
-Each published destination bundle contains `.distributor.json`. This state file records the source manifest, copied source outputs, and generated outputs. It is the authoritative marker that a destination path is managed by `distributor`.
+Destination bundle paths preserve the source bundle path relative to the source root. A source bundle at the source root publishes to the destination root. A source bundle under `daily/` publishes under `daily/` at each destination.
+
+The maintained examples write under `workspace/`, which is ignored by Git.
+
+## Destination State
+
+Each published destination bundle contains `.distributor.json`. This file is the managed sentinel and destination state record. It stores:
+
+- pipeline and destination identity;
+- publication timestamp;
+- source manifest used for publication;
+- copied source output metadata;
+- generated output metadata.
 
 `manifest.json` from the source bundle is not copied as destination state.
 
-## Retry behavior
+Do not edit `.distributor.json` by hand during normal operation. If it is missing or invalid while destination files remain, `distributor` treats the destination as unmanaged or conflicted.
 
-If a destination already has matching `.distributor.json`, publication skips it as already published.
+## Dry Runs
 
-If destination state is older than the source manifest, publication replaces only managed outputs recorded in `.distributor.json` plus the state file.
+`--dry-run` loads and validates config, discovers source bundles, inspects destination state, plans outputs, and prints summary lines. It does not write output files or destination state.
 
-If a write fails during local publication, `distributor` removes outputs written during that failed attempt where possible so a retry does not see an unmanaged destination.
+Dry-run output is useful before publishing to confirm actions such as `publish_new`, `replace_older`, `skip_same`, and `skip_destination_newer`.
 
-If one destination fails, later destinations in the same fan-out are still planned and run where they are independent. The command exits non-zero after printing the final status when any destination fails.
+## Retry and Replacement Behavior
 
-After a successful publish or replacement, the internal notifier hook runs as a no-op. Skipped destinations do not invoke it.
+If a destination has matching `.distributor.json`, publication skips it as already published.
+
+If destination state is older than the source manifest and transfer policy allows replacement, publication deletes only managed outputs recorded in `.distributor.json` plus the state file, then writes the new outputs and state.
+
+If destination state is newer than the source manifest, the default behavior is to skip. If destination state has the same source id and created timestamp but a different digest, publication fails as a conflict.
+
+If a destination path has files but no valid `.distributor.json`, publication fails as unmanaged content. There is no force overwrite option.
+
+## Failure Handling
+
+If one destination fails in a fan-out run, independent later destinations are still planned and executed. The command exits non-zero after printing the final status if any destination failed.
+
+If a write fails during local publication, `distributor` attempts to remove outputs written during that failed attempt so a retry does not see those partial outputs as unmanaged destination content.
+
+After a successful publish or replacement, the internal notifier hook runs. The current default notifier is a no-op. Skipped destinations do not invoke it.
 
 ## Caveats
 
-Only local-to-local publication is implemented. SSH, S3, external notification adapters, and force overwrite behavior are not implemented.
+Only local-to-local execution is available. SSH execution, S3 execution, external notification adapters, and force overwrite behavior are unavailable.
+
+For symptom-oriented fixes, see [troubleshooting](troubleshooting.md). For config details, see [configuration](config.md). For command syntax, see [CLI](cli.md).

docs/policy/architecture.md

@@ -8,7 +8,7 @@ This document defines the development principles for `distributor`. It is inward
 
 Producer applications generate manifested bundles. `distributor` discovers those bundles, validates them, optionally derives publication artifacts such as HTML, and publishes selected source and generated artifacts to one or more configured destinations.
 
-`distributor` does not generate domain reports, interpret domain-specific report content, run producer pipelines, edit reports, or act as a CMS. Weather reports, D&D recaps, calendar summaries, email digests, and future report types should all enter `distributor` through the same bundle contract.
+`distributor` does not generate domain reports, interpret domain-specific report content, run producer pipelines, edit reports, or act as a CMS. Weather reports, D&D recaps, calendar summaries, email digests, and additional report types should all enter `distributor` through the same bundle contract.
 
 ## Project Shape
 
@@ -16,7 +16,7 @@ Default to a small, explicit, dependency-light Go application. Keep the design m
 
 Business logic should live outside CLI, transport, and external-adapter packages. The core application should reason in terms of pipelines, bundles, destination state, transforms, and publish plans—not S3 SDK calls, SFTP sessions, shell commands, or filesystem details.
 
-The core workflow is:
+The current core workflow is:
 
 1. load configured pipelines;
 2. open the source backend;
@@ -28,13 +28,13 @@ The core workflow is:
 8. optionally transform Markdown to HTML for that destination;
 9. publish selected source and generated artifacts;
 10. write `.distributor.json` as the destination sentinel/state file;
-11. run the notification stage, which is a no-op in the MVP.
+11. run the notification hook, which is a no-op in the MVP.
 
 ## Pipeline Model
 
 A pipeline has exactly one source and one or more destinations.
 
-The source is discovered and validated once. Each destination has independent backend configuration, publication policy, transform policy, replacement behavior, state, and future notification behavior.
+The source is discovered and validated once. Each destination has independent backend configuration, publication policy, transform policy, replacement behavior, state, and notification behavior.
 
 The pipeline model is fan-out by design:
 
@@ -45,7 +45,7 @@ source bundle
   -> destination C: source files + HTML
 ```
 
-Destination-specific behavior must not leak back into the source bundle contract. A producer should not need to know whether a bundle will be published to local storage, SSH/SFTP, S3, a static site, email, RSS, or a future notification channel.
+Destination-specific behavior must not leak back into the source bundle contract. A producer should not need to know whether a bundle will be published to local storage, another storage backend, a static site, email, RSS, or another notification channel.
 
 ## Source Bundle Contract
 
@@ -166,13 +166,13 @@ For example, one destination may publish source files only as a long-term archiv
 
 ## Backend Abstraction
 
-Sources and destinations use the same storage abstraction. Local filesystem, SSH/SFTP, and S3-compatible object storage are peer backends. Any backend may appear as a source or a destination unless a specific limitation is documented.
+Sources and destinations use the same storage abstraction. Current runtime execution uses the local filesystem backend. Additional storage backends should be peer implementations behind the same interface, and any backend-specific execution limitation must be documented.
 
-Application logic must interact with storage through internal backend interfaces. Backend-specific behavior belongs in adapter packages. Pipeline, bundle, state, publish, and transform packages must not import S3, SSH/SFTP, or filesystem adapter implementation details.
+Application logic must interact with storage through internal backend interfaces. Backend-specific behavior belongs in adapter packages. Pipeline, bundle, state, publish, and transform packages must not import service-specific or filesystem adapter implementation details.
 
 Adapters should be thin. Backend adapters should implement storage operations and translate backend-specific errors, but should not make bundle comparison, transform, routing, or replacement decisions.
 
-SSH support should prefer a native SFTP implementation over shelling out to `ssh`, `scp`, or `rsync`, unless a later design document records a reason to differ.
+Remote file-transfer support should prefer native protocol implementations over shelling out, unless a later design document records a reason to differ.
 
 ## Dependency Policy
 
@@ -184,7 +184,7 @@ Avoid dependencies for small conveniences. Do not let external dependency types
 
 ## Package Layout
 
-Use this layout unless the project has a documented reason to differ:
+Use this current layout unless the project has a documented reason to differ:
 
 - `cmd/distributor`: application entrypoint only.
 - `internal/app`: application orchestration and top-level use cases.
@@ -194,14 +194,14 @@ Use this layout unless the project has a documented reason to differ:
 - `internal/state`: `.distributor.json` parsing, validation, comparison, and output metadata.
 - `internal/storage`: backend interfaces, shared path/resource types, backend registry, and storage errors.
 - `internal/adapters/local`: local filesystem backend.
-- `internal/adapters/ssh`: SSH/SFTP backend.
-- `internal/adapters/s3`: S3-compatible object storage backend.
 - `internal/transform`: transform interfaces, registry, planning, and shared transform models.
 - `internal/transform/markdown`: Markdown-to-HTML implementation.
 - `internal/publish`: destination planning, reconciliation, safety checks, and publish execution.
 - `internal/notify`: notification interface and MVP no-op notifier.
 - `internal/logging`: logging setup and shared logging helpers.
 
+New storage adapters should live under `internal/adapters/<name>` and stay thin.
+
 Package-private implementation constants may live near the package that owns them, preferably in `constants.go` when useful.
 
 ## Configuration
@@ -244,9 +244,9 @@ Each major stage should have an explicit input/output contract:
 - publish execution;
 - notification.
 
-If users can select backends, transforms, notifiers, or future renderers, selection should go through a registry or equivalent mechanism rather than scattered conditionals.
+If users can select backends, transforms, notifiers, or renderers, selection should go through a registry or equivalent mechanism rather than scattered conditionals.
 
-The orchestrator should be able to plan, dry-run, and execute configured pipelines. Dry-run behavior should be first-class because the application may delete, overwrite, or publish files to remote storage.
+The orchestrator should be able to plan, dry-run, and execute configured pipelines. Dry-run behavior should be first-class because the application may delete, overwrite, or publish files.
 
 ## Embedded Assets
 
@@ -264,7 +264,7 @@ Skip and no-op decisions should be logged at an appropriate level so operators c
 
 ## Context, Timeouts, and Cancellation
 
-Long-running operations should accept `context.Context`. Storage operations, SSH/SFTP sessions, S3 requests, transforms, and multi-stage workflows should respect cancellation and timeouts.
+Long-running operations should accept `context.Context`. Storage operations, service requests, transforms, and multi-step workflows should respect cancellation and timeouts.
 
 ## State, Files, and Safety
 
@@ -282,7 +282,7 @@ Where practical, publish operations should use staging paths or temporary object
 
 Core logic should be testable without real external services. Use fakes, fixtures, or local test doubles for adapters where practical.
 
-Config examples should be load-tested. Important CLI workflows should have parser or command tests. Stage/module contracts should have focused tests that do not require running the full application unless end-to-end coverage is intentional.
+Config examples should be load-tested. Important CLI workflows should have parser or command tests. Component contracts should have focused tests that do not require running the full application unless end-to-end coverage is intentional.
 
 Important tests include:
 
@@ -297,13 +297,13 @@ Important tests include:
 - transform output planning and metadata recording;
 - dry-run output;
 - local backend behavior with temporary directories;
-- fake backend behavior for S3 and SSH/SFTP-facing core logic.
+- fake backend behavior for storage-facing core logic.
 
 ## Documentation
 
 Documentation should follow the project documentation policy. Keep user docs focused on implemented behavior. Put future, planned, or aspirational work only under `docs/roadmap/`.
 
-When changing architecture, config, CLI behavior, adapters, manifest/state contracts, transform behavior, publish behavior, or stage/module contracts, update the relevant docs and examples in the same change.
+When changing architecture, config, CLI behavior, adapters, manifest/state contracts, transform behavior, publish behavior, or component contracts, update the relevant docs and examples in the same change.
 
 The source manifest and destination `.distributor.json` schemas should have canonical documentation once implemented. Example configs should be valid and load-tested where practical.
 
@@ -321,4 +321,4 @@ The source manifest and destination `.distributor.json` schemas should have cano
 - a backup system;
 - a notification platform.
 
-It may later support notification adapters, RSS/feed generation, richer HTML templates, or additional transforms, but those features must preserve the core bundle-distribution boundary.
+Additional notification, feed, template, or transform behavior must preserve the core bundle-distribution boundary.

docs/policy/development.md

@@ -1 +1,184 @@
-# Not yet implemented
+# Development Policy
+
+This document defines the day-to-day development workflow for `distributor`.
+Use it with `docs/policy/architecture.md` and `docs/policy/documentation.md`.
+
+## Repository Layout
+
+- `cmd/distributor`: executable entrypoint only.
+- `internal/app`: top-level use cases for `run`, `validate`, and `inspect`.
+- `internal/cli`: standard-library command parsing, flags, help text, and command wiring.
+- `internal/config`: YAML configuration structs, loading, defaults, and validation.
+- `internal/bundle`: source bundle discovery, manifest parsing, digest calculation, and validation.
+- `internal/state`: destination `.distributor.json` parsing, validation, and comparison.
+- `internal/storage`: backend interface, registry, logical path rules, typed errors, and shared storage helpers.
+- `internal/adapters/local`: local filesystem backend.
+- `internal/storage/fake`: in-memory backend for tests.
+- `internal/publish`: destination inspection, output planning, reconciliation, execution, and managed cleanup.
+- `internal/transform`: transform interface and registry.
+- `internal/transform/markdown`: Markdown-to-HTML sidecar transform.
+- `internal/notify`: notification interface and current no-op notifier.
+- `internal/testutil`: shared test fixtures. Production code must not import this package.
+- `docs`: current user, operator, policy, internal, and roadmap documentation.
+- `examples`: copyable example configs and source bundles.
+
+Do not create new top-level package families such as `pkg`, `internal/stage`,
+`internal/modules`, or service-specific adapter directories unless the
+architecture policy or a current roadmap explicitly calls for them.
+
+## Common Commands
+
+Run the full test suite:
+
+```bash
+go test ./...
+```
+
+Run targeted packages while developing:
+
+```bash
+go test ./internal/config
+go test ./internal/cli ./internal/app
+go test ./internal/publish ./internal/state
+go test ./internal/transform/markdown
+```
+
+Run the CLI against an example config:
+
+```bash
+go run ./cmd/distributor run --config examples/local-publish.yml --dry-run
+```
+
+Validate or inspect a local source bundle:
+
+```bash
+go run ./cmd/distributor validate examples/source-bundle
+go run ./cmd/distributor inspect examples/source-bundle
+```
+
+If Go cache permissions fail in a restricted environment, use workspace-safe
+temporary caches:
+
+```bash
+GOCACHE=/private/tmp/distributor-gocache GOMODCACHE=/private/tmp/distributor-gomodcache go test ./...
+```
+
+## Coding Conventions
+
+- Keep the application small, explicit, and dependency-light.
+- Prefer package-local helpers over broad abstractions until behavior is shared by multiple packages.
+- Keep CLI parsing in `internal/cli`; business decisions belong in `internal/app`, `internal/bundle`, `internal/publish`, `internal/state`, and related core packages.
+- Keep adapter packages thin. Backend-specific filesystem or service behavior belongs in adapters; bundle, state, transform, and publish policy belongs outside adapters.
+- Preserve public CLI behavior, config semantics, manifest schema, destination state schema, and local MVP behavior unless the current task explicitly changes them.
+- Use `storage.DisplayPath`, `storage.StateFileName`, `storage.StatePath`, and `storage.ManagedBundleTargets` instead of duplicating those conventions.
+- Use `bundle.ValidateManifest` for normalized source manifest semantics, including embedded source manifests in destination state.
+- Use `config.ValidatePublishTransformPolicy` for publish and transform policy combinations.
+- Do not import concrete transform implementations from `internal/publish`; app-level wiring owns transform registration.
+- Do not import `internal/testutil` from production code.
+
+## Dependency Policy
+
+The project currently depends on:
+
+- `gopkg.in/yaml.v3` for YAML configuration loading.
+- `github.com/yuin/goldmark` for Markdown rendering.
+
+Add external dependencies only when they materially improve correctness,
+security, interoperability, or implementation complexity. Avoid dependencies
+for small conveniences. Do not let dependency-specific types leak across
+internal package boundaries unless that dependency is the explicit package
+contract.
+
+## Configuration Changes
+
+When adding or changing configuration:
+
+1. Update `internal/config/config.go` structs and YAML tags.
+2. Add defaults in `internal/config/defaults.go` only for built-in defaults.
+3. Add validation in `internal/config/validate.go` with clear field context.
+4. Update config load and validation tests.
+5. Update `docs/config.md` in the same change if current user-visible config behavior changes.
+6. Update examples only with configs that are valid and executable for implemented behavior.
+
+Config validation may accept fields for backends that are not executable yet,
+but user-facing docs and examples must clearly state execution support. At the
+time of this policy, only the local backend is executable.
+
+## CLI Changes
+
+The CLI is hand-written with the Go standard library. Do not introduce a CLI
+framework without a documented reason.
+
+When adding or changing commands or flags:
+
+1. Keep parsing and help text in `internal/cli`.
+2. Keep command work in `internal/app` or a lower-level package.
+3. Add or update CLI tests in `internal/cli`.
+4. Update `docs/cli.md` if syntax, flags, output expectations, or workflows change.
+
+`validate` and `inspect` are local path commands. `run` loads configured
+pipelines and currently executes local backends only.
+
+## Storage Backends
+
+Storage behavior is defined by `internal/storage.Backend` and shared path rules
+in `internal/storage`.
+
+When adding a backend:
+
+1. Implement the storage interface in an adapter package.
+2. Translate backend-specific errors into storage errors where practical.
+3. Keep bundle comparison, transform, routing, and replacement policy out of the adapter.
+4. Register runtime construction through app-level backend factory wiring.
+5. Add focused adapter tests and app-level wiring tests.
+6. Update user docs, operations docs, examples, and internal docs only for behavior that is actually implemented.
+
+Do not document SSH/SFTP or S3 execution as available until corresponding
+adapter packages and app wiring exist.
+
+## Transforms
+
+Transforms use `internal/transform` interfaces and registry wiring.
+
+When adding or changing a transform:
+
+1. Keep the transform implementation in its own package under `internal/transform`.
+2. Register default runtime transforms from `internal/app`.
+3. Keep `internal/publish` dependent only on the transform interface or resolver.
+4. Record deterministic output metadata: path, source path, transform name, digest, and size.
+5. Add transform tests and app or publish tests for wiring and policy behavior.
+6. Update `docs/internal/transform.md` and any relevant integration docs for implemented behavior.
+
+## Tests
+
+Test close to the behavior being changed:
+
+- Use package tests for parsing, validation, comparison, planning, and adapter behavior.
+- Use `internal/app` and `internal/cli` tests for user-facing workflows.
+- Use `internal/testutil` for shared valid fixtures only; keep edge cases near the package under test.
+- Run `go test ./...` after cross-package changes or documentation/example changes tied to tests.
+
+## Examples
+
+Examples under `examples/` must be valid, maintained, and free of secrets.
+They should be copyable for implemented behavior. Do not leave examples that
+look runnable but require unsupported backend execution.
+
+When changing examples:
+
+1. Keep paths relative to the repository where practical.
+2. Prefer local examples until remote backend support exists.
+3. Run `go test ./internal/config` because config tests load examples.
+4. Update README, CLI, or config docs if links or recommended workflows change.
+
+## Documentation
+
+Follow `docs/policy/documentation.md`.
+
+- Document implemented behavior outside `docs/roadmap/`.
+- Keep future, planned, or aspirational behavior under `docs/roadmap/`.
+- Keep `docs/config.md` canonical for user-facing config reference.
+- Keep `docs/cli.md` canonical for command syntax and workflows.
+- Keep `docs/operations.md` canonical for operational and recovery behavior.
+- Keep `docs/internal/` focused on implemented package contracts.
+- Update docs in the same change as behavior when public behavior, config, CLI, examples, or internal contracts change.

docs/roadmap/audit.md

@@ -1,255 +0,0 @@
-# Code Quality and Deduplication Audit
-
-## 1. Executive summary
-
-The Stage 1-8 implementation is generally clean, modular, and close to the roadmap boundaries. The codebase is ready for a limited cleanup pass before remote backend work. I did not find evidence of a major architectural problem or a need for broad redesign.
-
-Top three refactoring targets:
-
-1. Runtime backend and transform resolution bypasses the registries already present in `internal/storage` and `internal/transform`.
-2. Source manifest validation semantics are split between `internal/bundle` and `internal/state`, and publish/transform policy is checked in both `internal/config` and `internal/publish`.
-3. Managed deletion target construction is duplicated in the local and fake storage backends and will otherwise be repeated in SSH/SFTP and S3 adapters.
-
-The recommended cleanup is a series of small, behavior-preserving commits. The public CLI, config schema, manifest schema, and destination state schema should remain stable.
-
-## 2. Repository map reviewed
-
-Reviewed implementation areas:
-
-- `cmd/distributor`: process entrypoint.
-- `internal/cli`: root command dispatch, `run`, `validate`, `inspect`, `version`, and CLI tests.
-- `internal/app`: run orchestration, validation, inspection, notification integration, and integration-style tests.
-- `internal/config`: config structs, defaults, YAML loading, validation, examples tests.
-- `internal/bundle`: manifest parsing, digest logic, discovery, validation, fixtures, and tests.
-- `internal/state`: `.distributor.json` parsing, validation, comparison, marshaling, and tests.
-- `internal/storage`: storage interface, path helpers, typed errors, registry, fake backend, and tests.
-- `internal/adapters/local`: local storage backend and tests.
-- `internal/publish`: plan, output selection, destination inspection, safety, execution, and tests.
-- `internal/transform` and `internal/transform/markdown`: transform interface, registry, Markdown-to-HTML implementation, and tests.
-- `internal/notify`: notifier interface and no-op implementation.
-- `internal/logging`: placeholder logging setup.
-- `examples`, `docs/config.md`, `docs/cli.md`, `docs/operations.md`, and `docs/internal/*`.
-
-Requested areas that do not currently exist as separate packages: `internal/stage`, `internal/modules`, `internal/validators`, `internal/artifacts`, `internal/manifest`, `internal/schema`, `internal/report`, and `pkg`.
-
-Major execution paths reviewed:
-
-- `distributor validate <path>`: CLI to app to local backend to bundle discovery and validation.
-- `distributor inspect <path>`: CLI to app to local backend to bundle discovery and inspection output.
-- `distributor run --config <path> [--dry-run]`: CLI to app to config loading, local source discovery, publish planning, local execution, destination state writing, and no-op notification.
-
-## 3. High-confidence deduplication opportunities
-
-### Runtime adapter and transform resolution bypass existing registries
-
-- Affected files/packages: `internal/app/run.go`, `internal/app/validate.go`, `internal/app/inspect.go`, `internal/publish/plan.go`, `internal/storage/registry.go`, `internal/transform/registry.go`.
-- Duplicated or near-duplicated behavior: app code directly checks `config.BackendLocal` and calls `local.New` in multiple paths. Publish code directly imports `internal/transform/markdown` and constructs the Markdown transformer through `markdownTransformer`, while `internal/transform.Registry` is unused.
-- Why it matters: Stage 9 and Stage 10 will add SSH/SFTP and S3 backends. If app orchestration continues to branch on backend names directly, every command path and run mode becomes a possible drift point. Direct Markdown construction also makes future transform registration less useful.
-- Recommended refactor: introduce an app-level backend factory that converts `config.Backend` and `config.Destination` into `storage.OpenConfig` and opens through a registry. Register only local for now. Move Markdown transformer selection behind a transform registry or a small transform resolver owned by app/publish boundaries.
-- Suggested tests: keep existing local run/validate/inspect tests; add an app factory unit test for local source and destination config resolution; add a publish/app test proving HTML generation uses the registered Markdown transform.
-- Risk level: medium. The behavior should remain unchanged, but this touches orchestration wiring used by all commands.
-
-### Source manifest validation rules are duplicated between bundle and state
-
-- Affected files/packages: `internal/bundle/manifest.go`, `internal/bundle/validate.go`, `internal/state/distributor.go`, `internal/state/validate.go`, `internal/state/compare.go`.
-- Duplicated or near-duplicated behavior: source manifest schema version, id, digest format, timestamp presence, file path safety, duplicate file path checks, file size rules, and bundle digest validation are enforced in both bundle parsing/validation and destination state validation.
-- Why it matters: `.distributor.json` embeds the normalized source manifest model. If the source manifest contract changes, fixes will likely need to be made in multiple packages, and state validation can drift from source validation.
-- Recommended refactor: expose a single `bundle` helper for validating an in-memory normalized manifest, including duplicate paths and canonical bundle digest. Let `ParseManifest` use parsing-specific checks and then call that helper. Let `state.Validate` call the same helper for embedded source manifests.
-- Suggested tests: preserve current `bundle` and `state` validation tests; add one cross-package regression fixture proving a manifest accepted by `bundle` is accepted when embedded in state, and a digest/path violation is rejected through the shared helper.
-- Risk level: medium. The refactor is behavior-preserving but affects core contract validation.
-
-### Managed deletion target construction is repeated in storage backends
-
-- Affected files/packages: `internal/adapters/local/backend.go`, `internal/storage/fake/backend.go`, `internal/storage/path.go`, `internal/publish/execute.go`.
-- Duplicated or near-duplicated behavior: both local and fake backends build the managed deletion target list by joining every output path under `bundlePath`, appending `storage.StatePath(bundlePath)`, and then applying backend-specific deletion.
-- Why it matters: SSH/SFTP and S3 adapters will need the same target derivation. Duplicating it in each adapter increases the chance of inconsistent state-file handling, invalid path behavior, or root deletion safeguards.
-- Recommended refactor: add a storage helper such as `ManagedBundleTargets(bundlePath string, managedOutputPaths []string) ([]string, error)` that validates and returns exact logical targets. Keep actual deletion backend-specific.
-- Suggested tests: add storage helper tests for root bundle path, nested bundle path, invalid output path, and inclusion of `.distributor.json`; keep local and fake managed deletion tests focused on backend deletion behavior.
-- Risk level: low. Target construction is small and already deterministic.
-
-### Publish/HTML policy validation is split between config and publish
-
-- Affected files/packages: `internal/config/validate.go`, `internal/publish/plan.go`, `internal/publish/output.go`, `internal/config/load_test.go`, `internal/publish/output_test.go`.
-- Duplicated or near-duplicated behavior: `internal/config` validates that `publish.html` requires `transform.markdown_to_html.enabled: true` and sidecar mode. `internal/publish` independently validates the same effective policy in `validateRequest`.
-- Why it matters: config-loaded runs are protected, but tests and programmatic callers can see different error text or future behavior if config and publish validation evolve separately.
-- Recommended refactor: centralize the effective publish/transform policy check in `internal/config` or a small policy helper that both config validation and publish request validation call.
-- Suggested tests: keep the existing config rejection test and publish rejection test, but assert both route through the same allowed combinations table.
-- Risk level: low.
-
-### Repeated bundle, config, and state test fixtures obscure behavior changes
-
-- Affected files/packages: `internal/app/run_test.go`, `internal/cli/root_test.go`, `internal/publish/output_test.go`, `internal/publish/execute_test.go`, `internal/transform/markdown/markdown_test.go`, `internal/state/distributor_test.go`, `internal/bundle/testdata`.
-- Duplicated or near-duplicated behavior: several packages recreate the same valid bundle shape, timestamps, file contents, digest calculation, local YAML snippets, and destination state data. Some tests use filesystem fixtures, some use fake storage, and some inline JSON/YAML.
-- Why it matters: changing the bundle contract, adding output metadata, or changing default config will require edits across many test files. The duplication also makes it harder to tell which tests are exercising unique behavior.
-- Recommended refactor: add an internal test helper package or package-local shared fixtures for valid manifests, source bundles, state files, and minimal configs. Keep package-specific edge cases local.
-- Suggested tests: no behavior tests are needed for helpers themselves beyond using them; add helper-backed tests incrementally while preserving current assertions.
-- Risk level: low.
-
-## 4. Medium-confidence opportunities
-
-### CLI command scaffolding is lightly duplicated
-
-- Affected files/packages: `internal/cli/run.go`, `internal/cli/validate.go`, `internal/cli/inspect.go`, `internal/cli/root.go`.
-- Duplicated or near-duplicated behavior: every command checks help, validates positional arguments, prints command-specific usage, calls app functions, and maps errors to exit codes. `validate` and `inspect` have nearly identical optional-path parsing.
-- Why it matters: adding `--pipeline`, output modes, or future command preflight behavior could create command drift.
-- Recommended refactor: add small CLI helpers for help detection, optional single path parsing, no-positional-argument rejection, and command usage printing. Do not introduce a large command framework unless the CLI grows substantially.
-- Suggested tests: keep existing CLI root tests; add table tests for validate/inspect path arity and run positional-argument rejection.
-- Risk level: low.
-
-### Output metadata projection appears in multiple layers
-
-- Affected files/packages: `internal/publish/output.go`, `internal/app/run.go`, `internal/notify/notify.go`, `internal/state/validate.go`.
-- Duplicated or near-duplicated behavior: publish outputs are converted to state output files in `stateOutputs`, and separately converted to notification outputs in `notifyEvent`.
-- Why it matters: when output metadata grows, fields can be added to state and omitted from notification, or vice versa, without a compiler-visible central projection.
-- Recommended refactor: keep destination state conversion in publish, but consider a small helper for notification output projection if output metadata changes in the next feature stage.
-- Suggested tests: extend notifier tests when output fields change.
-- Risk level: low.
-
-### Empty-path display helpers are repeated
-
-- Affected files/packages: `internal/bundle/validate.go`, `internal/app/inspect.go`, `internal/publish/plan.go`, `internal/publish/safety.go`.
-- Duplicated or near-duplicated behavior: `displayRoot`, `displayBundlePath`, and `displayPath` all render an empty logical path as `"."`.
-- Why it matters: this is user-facing output and error text. Minor drift can make tests brittle and logs inconsistent.
-- Recommended refactor: add a small formatting helper in the package that owns logical paths, or keep command/report formatting in app if avoiding cross-package formatting dependencies.
-- Suggested tests: update affected output/error tests to use behavior assertions rather than exact helper names.
-- Risk level: low.
-
-### Backend config structs repeat fields by shape
-
-- Affected files/packages: `internal/config/config.go`, `internal/config/validate.go`.
-- Duplicated or near-duplicated behavior: source `Backend` and destination `Destination` share backend fields such as `backend`, `path`, `uri`, `endpoint`, `bucket`, `prefix`, `region`, `force_path_style`, and credentials.
-- Why it matters: new backend fields need to be added in two places and passed through validation manually.
-- Recommended refactor: consider embedding a shared backend config struct in `Destination` only when adding remote backend implementations. Do not refactor now if it would complicate YAML decoding or docs.
-- Suggested tests: config load tests should cover any newly added shared fields for source and destination.
-- Risk level: low.
-
-## 5. Boundary and responsibility concerns
-
-- `internal/app` currently imports `internal/adapters/local` directly and branches on backend names. This is acceptable for the local MVP, but the documented architecture says app should construct backends through registries and avoid adapter-specific logic. The app layer is the right place for backend factory wiring; backend implementation details should remain in adapter packages.
-- `internal/publish` imports `internal/transform/markdown` directly. Publish should plan and execute outputs, but transform implementation selection belongs in transform/app wiring. A registry-backed resolver would better match the current package layout.
-- `internal/state` owns destination state, but it partially revalidates source manifest semantics. State should validate state-specific fields and delegate embedded source manifest contract checks to `internal/bundle`.
-- `internal/storage` owns logical path and state path helpers, but the state filename constant is private while callers and tests still hard-code `.distributor.json`. Expose the constant or provide a clearer canonical helper to reduce cross-package string reuse.
-
-## 6. Path, key, and naming construction review
-
-Local path safety is centralized well in `internal/storage/path.go` and the local adapter. Bundle-relative path composition generally uses `storage.Join`, and destination state paths use `storage.StatePath`.
-
-Cleanup targets:
-
-- `.distributor.json` is canonical in `storage.StatePath`, but the literal is also hard-coded in `bundle.ValidateSourcePath`, adapter tests, app tests, CLI tests, and docs. Exporting a canonical state filename would reduce drift.
-- Managed deletion target construction is repeated in local and fake storage backends. A storage helper should derive exact managed targets once.
-- Empty logical path display as `"."` is repeated across app, bundle, and publish.
-- Markdown sidecar naming is currently local to `internal/transform/markdown` through `strings.TrimSuffix(file.Path, ".md") + ".html"`. That is acceptable while Markdown sidecar is the only transform, but future path remapping or additional transforms should introduce a transform output naming helper rather than spreading suffix logic.
-
-## 7. Resolution and catalog review
-
-Backend resolution is not centralized enough for the next roadmap stage. The storage registry exists and is tested, but runtime code does not use it. App orchestration currently resolves only local backends manually.
-
-Transform resolution is also not centralized enough. The transform registry exists, but publish planning directly constructs the Markdown transformer.
-
-There are no separate artifact, schema, prompt, profile, module, validator, stage, source catalog, or report catalog packages in the current repository. That absence is appropriate for this application; no new catalog layer should be introduced unless a later feature creates multiple named implementations with shared resolution semantics.
-
-## 8. Config and command-loading review
-
-Config loading is mostly centralized: `app.Run` applies the default config path, calls `config.LoadFile`, and `config.LoadFile` handles YAML strict decoding, defaults, and validation. I did not find multiple independent config-loading paths for `run`.
-
-Intentional differences:
-
-- `validate` and `inspect` take direct local paths and do not load config. That matches current CLI behavior.
-- `run` supports `--config` and `--dry-run`; `validate` and `inspect` do not.
-
-Likely cleanup:
-
-- Keep default config path resolution close to config/app rather than in CLI.
-- Add shared CLI parsing helpers only for repeated command preflight, not for business policy.
-- Centralize publish/transform policy validation so config-loaded and programmatic publish requests cannot drift.
-
-## 9. State, manifest, or progress handling review
-
-Manifest handling is deterministic and well-covered: parsing is strict, digest validation is canonical, discovery sorts bundle paths, and source validation rejects unsafe paths and symlinks through storage metadata.
-
-Destination state handling is generally strong: `.distributor.json` is the success marker, comparison is centralized in `internal/state`, and publish execution writes state only after outputs are written.
-
-Cleanup targets:
-
-- Destination state validation should delegate embedded manifest validation to `internal/bundle`.
-- State output metadata construction should remain centralized in publish; if notification output metadata grows, add a projection helper to avoid field drift.
-- Run summary and status output currently live in `internal/app/run.go`. That is acceptable for MVP. Only extract a report/formatting package if additional output formats or commands start sharing the same summaries.
-
-I did not find run checkpoint, resume, retry, force, or progress-file logic in the implemented MVP.
-
-## 10. Refactors to avoid
-
-- Do not introduce a generic workflow engine for the pipeline. The current sequential orchestration is readable and matches MVP requirements.
-- Do not add a broad plugin architecture. Registries for storage and transforms are enough for the next stage.
-- Do not redesign the CLI around a larger framework solely to remove small parsing duplication.
-- Do not merge local and fake backends into one implementation. Their shared contract should be tested, but their storage behavior is intentionally different.
-- Do not generalize every test fixture immediately. Preserve package-local edge-case setup where it makes the behavior clearer.
-- Do not create catalog packages for artifacts, schemas, prompts, profiles, modules, stages, or reports unless the product adds multiple named implementations in those domains.
-- Do not rewrite manifest or state schemas as part of cleanup. Any schema change should be its own explicit compatibility task.
-
-## 11. Recommended implementation sequence
-
-1. Centralize storage names and managed target construction.
-   - Export or otherwise canonicalize the destination state filename.
-   - Add a storage helper for managed bundle deletion targets.
-   - Update local and fake backends to use it.
-
-2. Centralize source manifest model validation.
-   - Add a `bundle` helper for validating normalized manifests.
-   - Reuse it from manifest parsing and destination state validation.
-
-3. Centralize publish/transform policy validation.
-   - Move the allowed publish/transform combinations into one helper.
-   - Reuse it from config validation and publish request validation.
-
-4. Introduce runtime backend factory wiring.
-   - Register local storage through the existing storage registry.
-   - Update app run/validate/inspect paths to use the factory where appropriate.
-   - Keep unsupported remote backends returning the same user-facing behavior.
-
-5. Introduce transform resolver wiring.
-   - Register Markdown-to-HTML through the existing transform registry.
-   - Remove direct Markdown construction from publish.
-
-6. Add focused CLI preflight helpers.
-   - Share optional-path parsing and no-extra-argument handling.
-   - Keep command bodies explicit.
-
-7. Consolidate high-value test fixtures.
-   - Add helpers for valid source bundles, minimal configs, destination state, and fake backend source data.
-   - Migrate tests opportunistically while preserving package-specific assertions.
-
-8. Revisit output/report formatting only if additional output formats are added.
-
-9. Do a small dead-code sweep after the above.
-   - Remove `ErrNotImplemented`, `Pipeline`, or placeholder logging only if they are still unused and not needed by planned next work.
-
-## 12. Test strategy
-
-Tests to add before or during cleanup:
-
-- `internal/storage`: tests for managed target helper behavior, including root bundle path, nested bundle path, invalid output path, and `.distributor.json` inclusion.
-- `internal/bundle` and `internal/state`: shared manifest validation regression tests for embedded destination manifests.
-- `internal/config` and `internal/publish`: table-driven tests for allowed and rejected publish/transform policy combinations.
-- `internal/app`: backend factory tests that preserve current unsupported-backend errors and local backend behavior.
-- `internal/transform`: registry-backed Markdown resolution test if publish/app wiring moves to the registry.
-- `internal/cli`: table tests for command help, optional path handling, and rejected extra args.
-
-Tests that can accompany refactors:
-
-- Local and fake backend managed deletion tests after target helper extraction.
-- Existing app run tests after backend factory introduction.
-- Existing Markdown and publish output tests after transform resolver introduction.
-- CLI root tests after preflight helper extraction.
-
-The full suite should be run after any cleanup implementation. For this audit report itself, no full test run is required.
-
-## 13. Appendix: findings not worth acting on
-
-- The local and fake backend tests intentionally cover many of the same storage contract behaviors. Keep this parity. Only extract small test helpers if setup starts obscuring assertions.
-- `config.Backend` and `config.Destination` duplicate backend-shaped fields, but this is not urgent. A premature struct embedding refactor could make YAML behavior and docs less obvious.
-- `runSummary` and text output formatting are local to app orchestration. They are not worth extracting until more commands or output formats need the same formatting.
-- `internal/logging.Configure` is currently a placeholder. Do not build a logging abstraction until real logging requirements appear.
-- `internal/app/pipeline.go` currently contains only a minimal `Pipeline` type. Treat it as harmless unless it remains unused after the next implementation pass.
-- The small `displayRoot`/`displayPath` helpers are duplicated, but this is a low-risk cleanup after higher-value policy and storage centralization.

docs/roadmap/cleanup.md

@@ -1,537 +0,0 @@
-# Cleanup Implementation Roadmap
-
-This roadmap converts the findings in `docs/roadmap/audit.md` into staged cleanup work. It is written for LLM coding agents that will implement one stage at a time.
-
-The cleanup goal is to reduce drift before remote backend work without changing public CLI behavior, config semantics, manifest/state schemas, or local MVP behavior.
-
-## Global Cleanup Rules
-
-Every implementation stage must:
-
-1. read `AGENTS.md`, `docs/policy/architecture.md`, `docs/policy/documentation.md`, and this roadmap before editing;
-2. implement only the current stage;
-3. preserve current public CLI behavior and config behavior unless the stage explicitly says otherwise;
-4. keep cleanup behavior-preserving and avoid broad rewrites;
-5. add or update focused tests for the changed behavior;
-6. run relevant package tests, and run `go test ./...` when the stage touches cross-package behavior;
-7. update implemented internal docs only when an internal contract actually changes;
-8. leave user-facing docs unchanged unless public behavior changes;
-9. avoid implementing future remote backend features as part of cleanup.
-
-If Go cache or module cache permissions fail, use workspace-safe temporary caches:
-
-```bash
-GOCACHE=/private/tmp/distributor-gocache GOMODCACHE=/private/tmp/distributor-gomodcache go test ./...
-```
-
-## Stage 1: Centralize Storage State Names and Managed Delete Targets
-
-### Goal
-
-Make state-file naming and managed deletion target construction canonical before SSH/SFTP and S3 adapters are added.
-
-### Implementation
-
-In `internal/storage`:
-
-- Export the destination state filename as:
-
-```go
-const StateFileName = ".distributor.json"
-```
-
-- Update `StatePath` to use `StateFileName`.
-- Add:
-
-```go
-func ManagedBundleTargets(bundlePath string, managedOutputPaths []string) ([]string, error)
-```
-
-Required behavior:
-
-- `bundlePath` is validated as a prefix, so `""` is valid.
-- each managed output path is validated as a file path below `bundlePath`;
-- returned targets contain each managed output target followed by the `.distributor.json` target;
-- root bundle path returns output paths unchanged plus `.distributor.json`;
-- nested bundle path returns `bundlePath/output` plus `bundlePath/.distributor.json`;
-- invalid output paths fail before any backend deletion occurs;
-- duplicate output paths do not need to be de-duplicated in this stage.
-
-Update local and fake backends to call `storage.ManagedBundleTargets` inside `DeleteManagedBundle`. Keep actual deletion, missing-file handling, directory pruning, and backend-specific error translation in the backend implementations.
-
-Replace code/test references to literal `.distributor.json` with `storage.StateFileName` where the code already imports or reasonably can import `internal/storage`. Do not contort docs or unrelated tests only to remove literals from prose.
-
-### Tests
-
-Add or update tests for:
-
-- `StatePath("") == ".distributor.json"`;
-- `StatePath("bundle") == "bundle/.distributor.json"`;
-- managed targets for root and nested bundle paths;
-- invalid managed output path rejection;
-- local and fake managed deletion still delete only listed outputs plus state;
-- local and fake managed deletion still preserve unlisted files.
-
-Run:
-
-```bash
-go test ./internal/storage ./internal/storage/fake ./internal/adapters/local ./internal/bundle ./internal/publish
-```
-
-### Completion Criteria
-
-- Managed target derivation lives in one storage helper.
-- Local and fake backend behavior is unchanged.
-- No broad recursive delete behavior is introduced.
-
-## Stage 2: Centralize Normalized Source Manifest Validation
-
-### Goal
-
-Make `internal/bundle` the single owner of source manifest semantics, including embedded source manifests in destination state.
-
-### Implementation
-
-In `internal/bundle`, add a model-level validation helper:
-
-```go
-func ValidateManifest(manifest Manifest) error
-```
-
-Required behavior:
-
-- validate `SchemaVersion == 1`;
-- validate non-empty `ID`;
-- validate top-level `Digest` format;
-- validate non-zero `Created`;
-- require at least one file;
-- validate every file path with `ValidateSourcePath`;
-- validate every file digest format;
-- reject negative file sizes;
-- reject duplicate logical file paths;
-- recompute `BundleDigest(manifest.Files)` and require it to match `manifest.Digest`.
-
-Keep `ParseManifest` responsible for raw JSON parsing, missing-field detection, RFC3339 timestamp parsing, and trailing-data detection. After building the normalized `Manifest`, call `ValidateManifest` for model-level validation. Preserve current error substrings where practical, especially for existing tests that assert user-facing diagnostics.
-
-In `internal/state`, replace `validateEmbeddedManifest` logic with delegation to `bundle.ValidateManifest`, wrapping the error as `state source.manifest: ...` where current callers expect state context.
-
-Do not change the manifest JSON schema, destination state schema, digest algorithm, timestamp normalization policy, or source path policy.
-
-### Tests
-
-Add or update tests for:
-
-- `bundle.ValidateManifest` accepts the existing valid fixture manifest;
-- `bundle.ValidateManifest` rejects bad schema version, empty id, bad digest, zero created time, empty files, unsafe paths, duplicate paths, negative size, and bundle digest mismatch;
-- `state.Validate` rejects the same embedded manifest violations through the shared helper;
-- existing manifest parser and destination state parser tests continue to pass.
-
-Run:
-
-```bash
-go test ./internal/bundle ./internal/state ./internal/publish ./internal/app
-```
-
-### Completion Criteria
-
-- Source manifest contract semantics are implemented once in `internal/bundle`.
-- Destination state validation delegates embedded source manifest semantics to `internal/bundle`.
-- Existing local MVP behavior is unchanged.
-
-## Stage 3: Centralize Publish and Transform Policy Validation
-
-### Goal
-
-Prevent drift between config validation and publish request validation for allowed source/html/transform combinations.
-
-### Implementation
-
-Keep ownership in `internal/config`, because the policy is expressed in config types and used by config validation.
-
-Add a helper such as:
-
-```go
-func ValidatePublishTransformPolicy(publish PublishPolicy, transform Transform) error
-```
-
-Required behavior:
-
-- fail when both `publish.Source` and `publish.HTML` are false;
-- when `publish.HTML` is true, require `transform.MarkdownToHTML != nil`;
-- when `publish.HTML` is true, require `transform.MarkdownToHTML.Enabled == true`;
-- when `publish.HTML` is true, require `transform.MarkdownToHTML.Mode == TransformModeSidecar`;
-- when Markdown-to-HTML is configured and enabled, reject any mode other than `TransformModeSidecar`;
-- when Markdown-to-HTML is configured but disabled, allow empty mode or `TransformModeSidecar` and reject other modes;
-- return concise errors that can be wrapped with config field context.
-
-Update `internal/config.Validate` to use this helper while preserving contextual error messages such as `pipelines[0].destinations[0].transform...`.
-
-Update `internal/publish.validateRequest` to use the same helper for programmatic requests. The publish-layer error may be less field-path-specific, but it must remain actionable.
-
-Do not change defaulting behavior in `ApplyDefaults`.
-
-### Tests
-
-Add table tests covering:
-
-- source-only publish allowed;
-- html-only publish with enabled sidecar transform allowed;
-- source-plus-html with enabled sidecar transform allowed;
-- no outputs rejected;
-- html without transform rejected;
-- html with disabled transform rejected;
-- html with wrong mode rejected;
-- enabled Markdown transform with wrong mode rejected.
-
-Apply the table at both config validation and publish request validation levels.
-
-Run:
-
-```bash
-go test ./internal/config ./internal/publish ./internal/app
-```
-
-### Completion Criteria
-
-- Effective publish/transform policy is checked through one helper.
-- Config-loaded and programmatic publish paths cannot drift on this policy.
-
-## Stage 4: Introduce App Backend Factory Wiring
-
-### Goal
-
-Move runtime backend construction toward the storage registry before remote backends are implemented.
-
-### Implementation
-
-Create an app-level backend factory, preferably in `internal/app/backends.go`.
-
-Required shape:
-
-- The factory owns a `*storage.Registry`.
-- The default factory registers only the local backend for now.
-- Local backend registration maps storage open config key `path` to `local.New(path)`.
-- Source and destination config conversion stays in `internal/app`; adapter packages must not import config types.
-- Unsupported SSH/SFTP and S3 execution must continue to fail clearly as not implemented for execution.
-
-Suggested API:
-
-```go
-type backendFactory struct {
-    registry *storage.Registry
-}
-
-func newBackendFactory() *backendFactory
-func (f *backendFactory) openSource(ctx context.Context, source config.Backend) (storage.Backend, error)
-func (f *backendFactory) openDestination(ctx context.Context, destination config.Destination) (storage.Backend, error)
-func (f *backendFactory) openLocalPath(ctx context.Context, path string) (storage.Backend, error)
-```
-
-Use the factory from:
-
-- `app.Run` for pipeline sources and destinations;
-- `app.Validate` for direct local path validation;
-- `app.Inspect` for direct local path inspection.
-
-Keep `validate` and `inspect` direct-path commands local-only in this stage. Do not add config-driven validation or remote validation.
-
-Keep public error behavior stable enough that current tests continue to assert meaningful substrings. It is acceptable to update exact error text if the new text is clearer and tests assert stable behavior rather than brittle phrasing.
-
-### Tests
-
-Add tests for:
-
-- factory opens a local source backend;
-- factory opens a local destination backend;
-- factory opens a direct local path;
-- factory rejects unsupported source backend with a clear execution-not-implemented error;
-- factory rejects unsupported destination backend with a clear execution-not-implemented error.
-
-Run:
-
-```bash
-go test ./internal/app ./internal/cli
-```
-
-Then run:
-
-```bash
-go test ./...
-```
-
-### Completion Criteria
-
-- `internal/app` no longer directly constructs local backends in multiple command paths.
-- Backend construction goes through one app-level factory and storage registry.
-- No SSH/SFTP or S3 backend implementation is added.
-
-## Stage 5: Introduce Transform Resolver Wiring
-
-### Goal
-
-Remove direct Markdown transform construction from `internal/publish` and make transform selection explicit and testable.
-
-### Implementation
-
-Do not make `internal/transform` import `internal/transform/markdown`; that would create the wrong dependency direction. The app layer should own default transform registration.
-
-In `internal/publish`, define a narrow resolver interface:
-
-```go
-type TransformerResolver interface {
-    Get(name string) (transform.Transformer, bool)
-}
-```
-
-Add a resolver field to `publish.Request`, for example:
-
-```go
-Transformers TransformerResolver
-```
-
-Update output planning so HTML generation:
-
-- looks up `transform.MarkdownToHTML` through the resolver;
-- fails clearly if the resolver is nil or the Markdown transformer is not registered;
-- uses the resolved transformer to generate outputs.
-
-Create app-level transform registry wiring, preferably in `internal/app/transforms.go`:
-
-- create a `transform.Registry`;
-- register `transform.MarkdownToHTML` with `markdown.New()`;
-- pass the registry into every publish request created by `app.Run`.
-
-Update publish tests to use either:
-
-- a tiny fake resolver and fake transformer for publish package tests; or
-- a local registry assembled in the test.
-
-Use app tests to prove the real Markdown transformer remains wired for end-to-end local HTML publication.
-
-### Tests
-
-Add or update tests for:
-
-- publish planning fails when HTML is requested and no transformer resolver is supplied;
-- publish planning fails when Markdown transformer is missing from the resolver;
-- publish planning uses a registered transformer;
-- app local HTML publication still produces `report.html`;
-- existing Markdown transformer tests remain focused on Markdown rendering.
-
-Run:
-
-```bash
-go test ./internal/publish ./internal/transform ./internal/transform/markdown ./internal/app
-```
-
-Then run:
-
-```bash
-go test ./...
-```
-
-### Completion Criteria
-
-- `internal/publish` no longer imports `internal/transform/markdown`.
-- App wiring registers the MVP Markdown transform explicitly.
-- Transform behavior and public CLI behavior are unchanged.
-
-## Stage 6: Add Focused CLI Preflight Helpers
-
-### Goal
-
-Reduce small CLI parsing drift without hiding command behavior behind a large framework.
-
-### Implementation
-
-Keep the hand-written standard-library CLI. Do not introduce a new CLI dependency.
-
-Add small helpers in `internal/cli`, such as:
-
-```go
-func parseOptionalPathArg(stderr io.Writer, command string, args []string) (string, bool)
-func rejectPositionalArgs(stderr io.Writer, command string, args []string) bool
-```
-
-Use them to simplify:
-
-- `validateCommand`;
-- `inspectCommand`;
-- run positional argument rejection after flag parsing.
-
-Keep each command's help text local to that command. Keep `hasHelp`, `fail`, and root dispatch behavior simple and explicit.
-
-Do not add aliases, output modes, config loading for validate/inspect, or new flags.
-
-### Tests
-
-Add table tests for:
-
-- `validate` with zero args returns app-level required-path error;
-- `validate` with one arg succeeds for a valid bundle;
-- `validate` with two args returns usage;
-- `inspect` with zero args returns app-level required-path error;
-- `inspect` with one arg succeeds for a valid bundle;
-- `inspect` with two args returns usage;
-- `run` rejects extra positional args after flags.
-
-Run:
-
-```bash
-go test ./internal/cli ./internal/app
-```
-
-### Completion Criteria
-
-- CLI command bodies are still readable.
-- Common preflight parsing behavior is centralized where it is actually shared.
-- Public CLI behavior remains unchanged.
-
-## Stage 7: Add Test Fixture Helper Foundation
-
-### Goal
-
-Create shared test helpers for high-value fixtures without forcing every test to use them immediately.
-
-### Implementation
-
-Create `internal/testutil` for test support used by multiple internal packages.
-
-This package may contain regular Go files even though it is intended only for tests. Production code must not import `internal/testutil`.
-
-Initial helper coverage:
-
-- valid source bundle data:
-  - default id `weather.daily.brentwood.2026-05-30`;
-  - default created time `2026-05-30T11:10:00Z`;
-  - default files `report.md` with `# Report\nSunny.\n` and `summary.txt` with `Summary\n`;
-- filesystem source bundle writer;
-- fake-backend source bundle writer;
-- minimal local config writer;
-- fan-out local config writer;
-- destination state writer;
-- destination state reader.
-
-Helpers should return normal project types such as `bundle.Manifest`, `bundle.Bundle`, and `state.DistributorState`.
-
-Do not move edge-case test logic into `testutil`. Tests for invalid manifests, collisions, symlinks, failures, and backend-specific behavior should remain close to the package being tested.
-
-### Tests
-
-Do not add tests for `testutil` itself unless helpers contain nontrivial logic not covered by consuming tests.
-
-Migrate only one or two low-risk test files in this stage to prove the helpers work. Good candidates:
-
-- `internal/publish/execute_test.go`;
-- `internal/transform/markdown/markdown_test.go`.
-
-Run:
-
-```bash
-go test ./internal/testutil ./internal/publish ./internal/transform/markdown
-```
-
-Then run:
-
-```bash
-go test ./...
-```
-
-### Completion Criteria
-
-- A shared fixture foundation exists.
-- At least two packages use it successfully.
-- The migration is incremental and does not obscure package-specific assertions.
-
-## Stage 8: Migrate High-Value Duplicate Test Fixtures
-
-### Goal
-
-Reduce the largest remaining test fixture duplication after the helper foundation is proven.
-
-### Implementation
-
-Migrate duplicated valid bundle/config/state setup in:
-
-- `internal/app/run_test.go`;
-- `internal/cli/root_test.go`;
-- `internal/publish/output_test.go`;
-- `internal/state/distributor_test.go`, where helper use improves clarity.
-
-Keep tests local when custom setup makes the behavior clearer than a shared helper. Do not chase 100 percent fixture reuse.
-
-Preserve all existing behavioral assertions.
-
-### Tests
-
-Run:
-
-```bash
-go test ./internal/app ./internal/cli ./internal/publish ./internal/state
-```
-
-Then run:
-
-```bash
-go test ./...
-```
-
-### Completion Criteria
-
-- The largest repeated valid bundle/config/state setup is centralized.
-- Edge-case tests remain readable.
-- No production code imports `internal/testutil`.
-
-## Stage 9: Final Dead-Code and Low-Value Cleanup Sweep
-
-### Goal
-
-Remove or defer remaining low-value cleanup items after the higher-impact centralization work is complete.
-
-### Implementation
-
-Review and decide on these items:
-
-- remove `app.ErrNotImplemented` if it is still unused;
-- remove or expand `internal/app/pipeline.go` if the placeholder `Pipeline` type is still unused;
-- keep `internal/logging.Configure` if it is still a planned extension point, otherwise remove it only if no code or docs reference it;
-- decide whether empty-path display helpers should remain local or move to a single helper;
-- leave `config.Backend` and `config.Destination` field duplication alone unless remote backend implementation work is starting immediately.
-
-Do not add:
-
-- generic workflow engine;
-- plugin architecture;
-- broad CLI framework;
-- schema rewrites;
-- remote backend behavior;
-- output/report formatting package unless it is now clearly shared by multiple commands.
-
-### Tests
-
-Run the full suite:
-
-```bash
-go test ./...
-```
-
-If removals affect docs or internal docs, update only implemented-behavior docs.
-
-### Completion Criteria
-
-- Obvious dead code is removed or explicitly left in place for a documented reason.
-- Remaining duplication is either low-value or intentionally deferred.
-- The codebase is ready to resume roadmap work on remote backends.
-
-## Deferred Cleanup
-
-Do not implement these as part of the cleanup roadmap unless a later roadmap explicitly promotes them:
-
-- embedding a shared backend config struct into `config.Destination`;
-- generic output/report formatting package;
-- broader test fixture migration beyond the high-value repeated fixtures;
-- remote backend implementations;
-- force overwrite behavior;
-- generic pipeline/workflow engine;
-- broad plugin system.
-

docs/roadmap/config.md

@@ -1,580 +0,0 @@
-# Distributor Configuration Roadmap
-
-This roadmap defines the planned `config.yml` schema for the `distributor` MVP. The goal is to support one-to-many publication pipelines where each pipeline has one source and one or more destinations. Each destination independently controls backend configuration, publication outputs, transform behavior, and replacement policy.
-
-## Configuration Goals
-
-The MVP configuration should be:
-
-- explicit enough to avoid hidden publication behavior;
-- compact enough for routine self-hosted use;
-- backend-agnostic at the pipeline layer;
-- capable of local, SSH/SFTP, and S3-compatible source and destination backends;
-- ready for future notification adapters without exposing a fake notification feature in the MVP.
-
-## Top-Level Shape
-
-```yaml
-pipelines:
-  - id: weather-daily
-    source:
-      backend: local
-      path: /var/spool/distributor/weather
-
-    validation:
-      on_digest_mismatch: fail
-
-    destinations:
-      - id: markdown-archive
-        backend: s3
-        endpoint: https://s3.example.com
-        bucket: reports
-        prefix: weather/archive
-        region: us-east-1
-        force_path_style: true
-        credentials:
-          access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
-          secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
-
-        publish:
-          source: true
-          html: false
-
-        transfer:
-          on_destination_same: skip
-          on_destination_older: replace
-          on_destination_newer: skip
-          on_conflict: fail
-
-      - id: static-site
-        backend: ssh
-        uri: ssh://deploy@example.com:22
-        path: /srv/www/weather
-
-        publish:
-          source: false
-          html: true
-
-        transform:
-          markdown_to_html:
-            enabled: true
-            mode: sidecar
-
-        transfer:
-          on_destination_same: skip
-          on_destination_older: replace
-          on_destination_newer: skip
-          on_conflict: fail
-```
-
-## Pipeline Fields
-
-Each pipeline must include:
-
-- `id`: Required stable pipeline identifier.
-- `source`: Required source backend configuration.
-- `destinations`: Required non-empty list of destination configurations.
-
-Optional pipeline-level fields:
-
-- `validation`: Source validation behavior.
-- Future: `notifications` or `notify`, when notification adapters are implemented.
-
-A pipeline has exactly one source and one or more destinations.
-
-## Pipeline ID Rules
-
-`pipelines[].id` should:
-
-- be required;
-- be unique across the config file;
-- be stable over time;
-- use a simple slug-like format, such as `weather-daily` or `dnd-session-recaps`.
-
-Recommended validation:
-
-```text
-^[a-zA-Z0-9][a-zA-Z0-9._-]*$
-```
-
-## Source Configuration
-
-`source` defines the source root where bundles are discovered.
-
-The source backend may be:
-
-- `local`;
-- `ssh`;
-- `s3`.
-
-The source is scanned for `manifest.json` files beneath the configured root.
-
-### Local source
-
-```yaml
-source:
-  backend: local
-  path: /var/spool/distributor/weather
-```
-
-Required fields:
-
-- `backend: local`
-- `path`
-
-### SSH source
-
-```yaml
-source:
-  backend: ssh
-  uri: ssh://reports@example.com:22
-  path: /var/spool/distributor/weather
-```
-
-Required fields:
-
-- `backend: ssh`
-- `uri`
-- `path`
-
-Recommended authentication behavior:
-
-- use SSH agent by default;
-- use local known_hosts validation by default;
-- support optional key file configuration later if needed;
-- do not require passwords in YAML.
-
-Optional future fields:
-
-```yaml
-known_hosts: /home/user/.ssh/known_hosts
-key_file: /home/user/.ssh/id_ed25519
-```
-
-### S3 source
-
-```yaml
-source:
-  backend: s3
-  endpoint: https://s3.example.com
-  bucket: reports
-  prefix: incoming/weather
-  region: us-east-1
-  force_path_style: true
-  credentials:
-    access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
-    secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
-```
-
-Required fields:
-
-- `backend: s3`
-- `endpoint`
-- `bucket`
-
-Optional fields:
-
-- `prefix`
-- `region`
-- `force_path_style`
-- `credentials`
-
-Credential configuration should prefer environment variables or standard SDK behavior over literal secrets in config.
-
-## Destination Configuration
-
-Each destination defines one publication target for a pipeline.
-
-Required destination fields:
-
-- `id`
-- `backend`
-- backend-specific location fields;
-- `publish`
-
-Optional destination fields:
-
-- `transform`
-- `transfer`
-
-Each destination is independently planned and published. A destination may receive source files, generated HTML, or both.
-
-## Destination ID Rules
-
-`destinations[].id` should:
-
-- be required;
-- be unique within the containing pipeline;
-- be stable over time;
-- use a slug-like format.
-
-Recommended examples:
-
-- `markdown-archive`
-- `static-site`
-- `full-mirror`
-
-## Local Destination
-
-```yaml
-destinations:
-  - id: local-static
-    backend: local
-    path: /srv/www/reports
-    publish:
-      source: false
-      html: true
-```
-
-Required fields:
-
-- `backend: local`
-- `path`
-- `publish`
-
-## SSH Destination
-
-```yaml
-destinations:
-  - id: static-site
-    backend: ssh
-    uri: ssh://deploy@example.com:22
-    path: /srv/www/weather
-    publish:
-      source: false
-      html: true
-```
-
-Required fields:
-
-- `backend: ssh`
-- `uri`
-- `path`
-- `publish`
-
-The MVP should use a native SFTP implementation rather than shelling out to `ssh`, `scp`, or `rsync`.
-
-## S3 Destination
-
-```yaml
-destinations:
-  - id: markdown-archive
-    backend: s3
-    endpoint: https://s3.example.com
-    bucket: reports
-    prefix: weather/archive
-    region: us-east-1
-    force_path_style: true
-    credentials:
-      access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
-      secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
-    publish:
-      source: true
-      html: false
-```
-
-Required fields:
-
-- `backend: s3`
-- `endpoint`
-- `bucket`
-- `publish`
-
-Optional fields:
-
-- `prefix`
-- `region`
-- `force_path_style`
-- `credentials`
-
-## Backend Configuration Normalization
-
-The config loader should normalize backend configuration into internal source and destination backend specs. Pipeline logic should not branch on backend-specific fields.
-
-Validation should catch:
-
-- missing backend names;
-- unsupported backend names;
-- missing backend-specific required fields;
-- duplicate pipeline IDs;
-- duplicate destination IDs within a pipeline;
-- empty destination lists;
-- invalid policy values.
-
-## Publication Policy
-
-`publish` controls which categories of files are written to a destination.
-
-```yaml
-publish:
-  source: true
-  html: false
-```
-
-Fields:
-
-- `source`: Publish source artifacts listed in `manifest.json`.
-- `html`: Publish HTML files generated from Markdown source artifacts.
-
-At least one of `source` or `html` must be true.
-
-Recommended defaults:
-
-```yaml
-publish:
-  source: true
-  html: false
-```
-
-No implicit HTML transformation should occur. When `publish.html` is true, `transform.markdown_to_html.enabled: true` and `mode: sidecar` are required for the MVP.
-
-## Transform Configuration
-
-For MVP, the only supported transform is Markdown to HTML.
-
-```yaml
-transform:
-  markdown_to_html:
-    enabled: true
-    mode: sidecar
-```
-
-Fields:
-
-- `enabled`: Whether Markdown-to-HTML transform is enabled.
-- `mode`: Output mode. MVP value: `sidecar`.
-
-MVP `sidecar` behavior:
-
-- each listed Markdown source file generates an HTML file with the same base path and `.html` extension;
-- `report.md` generates `report.html`;
-- generated files are destination publication artifacts;
-- source bundles are not mutated.
-
-MVP defaulting:
-
-- If `publish.html` is false, transform may be omitted.
-- If `publish.html` is true and `transform.markdown_to_html` is omitted or disabled, config validation must fail.
-
-## Validation Policy
-
-Pipeline-level validation is intentionally narrow in the MVP.
-
-```yaml
-validation:
-  on_digest_mismatch: fail
-```
-
-Supported value:
-
-- `fail`
-
-Default:
-
-```yaml
-on_digest_mismatch: fail
-```
-
-Validation should happen before any destination writes. Warning-only digest mismatch handling is deferred and must be rejected if configured.
-
-## Transfer Policy
-
-Destination-level transfer policy controls behavior after inspecting `.distributor.json` at the destination bundle path.
-
-```yaml
-transfer:
-  on_destination_same: skip
-  on_destination_older: replace
-  on_destination_newer: skip
-  on_conflict: fail
-```
-
-Supported fields:
-
-- `on_destination_same`
-- `on_destination_older`
-- `on_destination_newer`
-- `on_conflict`
-
-MVP supported values are intentionally limited by field:
-
-- `on_destination_same`: `skip` or `fail`
-- `on_destination_older`: `replace` or `fail`
-- `on_destination_newer`: `skip` or `fail`
-- `on_conflict`: `fail`
-
-Recommended MVP defaults:
-
-```yaml
-transfer:
-  on_destination_same: skip
-  on_destination_older: replace
-  on_destination_newer: skip
-  on_conflict: fail
-```
-
-Safety rule:
-
-- `replace` must never perform broad deletion against a destination root.
-- `replace` may only operate within a resolved destination bundle path and should delete only files recorded in existing `.distributor.json.outputs` plus `.distributor.json` where practical.
-- Unmanaged non-empty destination paths fail in the MVP. Force or unmanaged overwrite configuration is deferred.
-- Broader replacement values, including replacing newer destinations or conflicts, are deferred to a later explicit force-overwrite stage.
-
-## Path Mapping
-
-MVP path mapping is fixed:
-
-```text
-destination bundle path = destination root + source relative bundle path
-```
-
-Example:
-
-```text
-source root: /var/spool/reports
-source bundle: /var/spool/reports/weather/daily/brentwood/2026-05-30
-relative bundle path: weather/daily/brentwood/2026-05-30
-
-destination root: /srv/www/reports
-destination bundle path: /srv/www/reports/weather/daily/brentwood/2026-05-30
-```
-
-Future config may support explicit path mapping, but MVP should not.
-
-## Dry Run Configuration and CLI Behavior
-
-Dry-run should be a CLI flag rather than a persistent config setting.
-
-```bash
-distributor run --config config.yml --dry-run
-```
-
-Dry-run should report:
-
-- pipeline ID;
-- source backend;
-- destination ID;
-- destination backend;
-- discovered bundle ID;
-- relative bundle path;
-- planned action;
-- reason;
-- transform outputs that would be generated;
-- files that would be written or deleted.
-
-## Example: Weather Pipeline
-
-```yaml
-pipelines:
-  - id: weather-daily
-    source:
-      backend: local
-      path: /var/spool/distributor/weather
-
-    validation:
-      on_digest_mismatch: fail
-
-    destinations:
-      - id: markdown-archive
-        backend: s3
-        endpoint: https://s3.example.com
-        bucket: reports
-        prefix: weather/archive
-        region: us-east-1
-        force_path_style: true
-        credentials:
-          access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
-          secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
-        publish:
-          source: true
-          html: false
-
-      - id: static-site
-        backend: ssh
-        uri: ssh://deploy@web.example.com:22
-        path: /srv/www/weather
-        publish:
-          source: false
-          html: true
-        transform:
-          markdown_to_html:
-            enabled: true
-            mode: sidecar
-```
-
-## Example: D&D Recap Pipeline
-
-```yaml
-pipelines:
-  - id: dnd-session-recaps
-    source:
-      backend: local
-      path: /var/spool/distributor/dnd/session-recaps
-
-    destinations:
-      - id: private-markdown-archive
-        backend: s3
-        endpoint: https://s3.example.com
-        bucket: reports
-        prefix: dnd/session-recaps
-        region: us-east-1
-        force_path_style: true
-        credentials:
-          access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
-          secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
-        publish:
-          source: true
-          html: false
-
-      - id: private-html-site
-        backend: local
-        path: /srv/www/private/dnd/session-recaps
-        publish:
-          source: false
-          html: true
-        transform:
-          markdown_to_html:
-            enabled: true
-            mode: sidecar
-```
-
-## Future Notification Configuration
-
-Notification should not be exposed as a functional MVP feature unless an adapter exists.
-
-The internal pipeline may include a no-op notification stage. Future config may look like:
-
-```yaml
-notifications:
-  - id: weather-email
-    backend: email
-    after_destinations:
-      - static-site
-    subject: "Weather report published"
-```
-
-Future notification policy should require:
-
-- notification after successful relevant publication;
-- idempotency by source manifest id and digest;
-- no duplicate notification unless explicitly forced.
-
-## Implementation Stages
-
-1. Define config structs for pipelines, sources, destinations, validation, publish, transform, and transfer policies.
-2. Implement config loading and strict validation.
-3. Implement backend-specific config validation for local, SSH, and S3.
-4. Implement defaulting for validation and transfer policies.
-5. Require explicit transform configuration when `publish.html` is true.
-6. Add example config fixtures for local-to-local, local-to-SSH, local-to-S3, and fan-out scenarios.
-7. Connect config to backend registry and publish planner.
-8. Add `--pipeline` filtering for targeted runs.
-9. Add `--dry-run` output that reflects the resolved config and planned actions.
-
-## Deferred Configuration
-
-The following configuration ideas are intentionally outside the MVP:
-
-- warning-only digest mismatch handling;
-- unmanaged destination overwrite flags until the explicit force-overwrite roadmap stage;
-- force replacement of destinations with different source ids until the explicit force-overwrite roadmap stage.

docs/roadmap/contracts.md

@@ -1,398 +0,0 @@
-# Distributor Contracts Roadmap
-
-This roadmap defines the contracts that `distributor` should implement before or alongside the MVP. The goal is to make bundle validation, destination state, digest verification, and safe replacement deterministic and testable before backend-specific publication behavior is layered on top.
-
-## Purpose
-
-`distributor` publishes manifested report bundles produced by other applications. Producer applications own domain-specific report generation. `distributor` owns validation, optional transformation, destination publication, and destination state.
-
-The MVP contract has two durable files:
-
-- `manifest.json`: source-owned bundle manifest produced by the upstream application.
-- `.distributor.json`: destination-owned publication state written by `distributor`.
-
-`manifest.json` is not copied to the destination as destination state. Instead, `.distributor.json` records the normalized source manifest, generated output metadata, and distributor-owned publication metadata.
-
-All manifest and state timestamps should be serialized as RFC3339. Internal comparison should use parsed timestamp values, and distributor-written timestamps should be normalized to RFC3339 UTC.
-
-## Terminology
-
-- **Source root**: Configured root path for a pipeline source.
-- **Bundle root**: Directory beneath the source root that contains `manifest.json`.
-- **Relative bundle path**: Bundle root path relative to the source root.
-- **Destination root**: Configured root path or prefix for a destination.
-- **Destination bundle path**: Destination root plus the relative bundle path, unless a future mapping option overrides that behavior.
-- **Source artifact**: File listed in the source `manifest.json`.
-- **Generated artifact**: File created by `distributor`, such as an HTML file derived from Markdown.
-- **Destination state**: `.distributor.json` at the destination bundle path.
-
-## Source Bundle Contract
-
-A source bundle is a directory containing a `manifest.json` file. For MVP, the manifest schema is intentionally minimal.
-
-### Required `manifest.json` fields
-
-```json
-{
-  "schema_version": 1,
-  "id": "weather.daily.brentwood.2026-05-30",
-  "digest": "sha256:0000000000000000000000000000000000000000000000000000000000000000",
-  "created": "2026-05-30T11:10:00Z",
-  "files": [
-    {
-      "path": "report.md",
-      "sha256": "sha256:1111111111111111111111111111111111111111111111111111111111111111",
-      "size": 12345
-    }
-  ]
-}
-```
-
-Required top-level fields:
-
-- `schema_version`: Source manifest schema version. MVP value: `1`.
-- `id`: Stable bundle identifier. Required, non-empty string.
-- `digest`: Bundle digest. Required, lowercase `sha256:<64 hex>` string.
-- `created`: Bundle creation timestamp. Required RFC3339 timestamp. UTC is preferred; explicit offsets are allowed.
-- `files`: Non-empty array of file objects.
-
-Required file fields:
-
-- `path`: Relative path from bundle root to source artifact.
-- `sha256`: Per-file digest as lowercase `sha256:<64 hex>`.
-- `size`: File size in bytes.
-
-No other source manifest fields are required for the MVP. Additional fields may be ignored unless later documented. Destination state records the normalized source manifest model, not raw unknown manifest fields.
-
-## Source Path Safety Rules
-
-For every `files[].path`:
-
-- Path must be relative.
-- Path must not be empty.
-- Path must not contain `..` segments.
-- Path must not resolve outside the bundle root.
-- Path must use slash-separated logical paths in the manifest.
-- Absolute paths are invalid.
-- Symlinks should be rejected for MVP unless a later policy deliberately supports them.
-- `manifest.json` itself should not be listed as a source artifact.
-- `.distributor.json` should not be listed as a source artifact.
-- Duplicate logical file paths are invalid after path normalization.
-
-The implementation should validate paths before reading file contents.
-
-## Digest Contract
-
-The MVP validates both per-file digests and the bundle digest.
-
-### Per-file digest
-
-For each file listed in `files`, compute:
-
-```text
-sha256(file bytes)
-```
-
-The computed digest must match `files[].sha256`.
-
-The actual file size must match `files[].size`.
-
-### Bundle digest
-
-The bundle digest is computed from the listed file records in the listed order. The canonical algorithm is:
-
-1. For each file listed in `files`, in order:
-   - validate the file path;
-   - compute the file SHA256;
-   - determine the file size.
-2. Construct a canonical JSON array containing only:
-   - `path`;
-   - `sha256`;
-   - `size`.
-3. Preserve the source manifest's file order.
-4. Encode the array deterministically with:
-   - object fields in exactly this order: `path`, `sha256`, `size`;
-   - no extra spaces;
-   - no trailing newline;
-   - lowercase `sha256:<64 hex>` digest strings.
-5. Compute `sha256(canonical JSON bytes)`.
-6. Compare the result to top-level `digest`.
-
-Conceptual canonical payload:
-
-```json
-[
-  {"path":"report.md","sha256":"sha256:...","size":12345},
-  {"path":"summary.txt","sha256":"sha256:...","size":234}
-]
-```
-
-This avoids ambiguous concatenation of file bytes while keeping the manifest small.
-
-Implementation fixtures should include at least one reference manifest and canonical payload with known per-file and bundle digests.
-
-## Digest Mismatch Behavior
-
-Digest validation is always fatal in the MVP. A digest mismatch must fail the affected bundle, pipeline, and run before any destination writes occur.
-
-Warning-only digest behavior is intentionally deferred until a later roadmap accepts transitional ingestion semantics.
-
-## Bundle Discovery Contract
-
-A source path may contain one bundle or a tree of bundles. Discovery should scan beneath the configured source root for `manifest.json` files.
-
-For each discovered manifest:
-
-- Bundle root is the directory containing `manifest.json`.
-- Relative bundle path is computed relative to source root.
-- Destination bundle path is destination root plus relative bundle path, unless a future mapping option overrides it.
-
-If nested manifests are found, the MVP should fail with a clear error unless a later policy defines nested-bundle semantics.
-
-## Destination State Contract
-
-Destinations are managed by `.distributor.json`, not by copying `manifest.json`.
-
-A destination bundle path is considered distributor-managed only when it contains a valid `.distributor.json` written by `distributor`. Force or unmanaged-overwrite behavior is not part of the MVP.
-
-### Required `.distributor.json` shape
-
-```json
-{
-  "schema_version": 1,
-  "distributor_version": "0.1.0",
-  "pipeline_id": "weather-daily",
-  "destination_id": "static-site",
-  "published_at": "2026-05-30T11:12:00Z",
-  "source": {
-    "manifest": {
-      "schema_version": 1,
-      "id": "weather.daily.brentwood.2026-05-30",
-      "digest": "sha256:0000000000000000000000000000000000000000000000000000000000000000",
-      "created": "2026-05-30T11:10:00Z",
-      "files": [
-        {
-          "path": "report.md",
-          "sha256": "sha256:1111111111111111111111111111111111111111111111111111111111111111",
-          "size": 12345
-        }
-      ]
-    }
-  },
-  "outputs": [
-    {
-      "path": "report.html",
-      "kind": "generated",
-      "source_path": "report.md",
-      "transform": "markdown_to_html",
-      "sha256": "sha256:3333333333333333333333333333333333333333333333333333333333333333",
-      "size": 23456
-    }
-  ]
-}
-```
-
-Required fields:
-
-- `schema_version`: Destination state schema version. MVP value: `1`.
-- `distributor_version`: Optional diagnostic distributor version. It must not affect source comparison.
-- `pipeline_id`: Pipeline that produced the destination publication.
-- `destination_id`: Destination within the pipeline.
-- `published_at`: RFC3339 timestamp.
-- `source.manifest`: Normalized source manifest model used for this publication.
-- `outputs`: Array of files written by `distributor` for this destination.
-
-Required output fields:
-
-- `path`: Destination-relative output path within the destination bundle path.
-- `kind`: `source` or `generated`.
-- `source_path`: Source artifact path that produced this output. For copied source files, this should equal `path` unless renamed by a future feature.
-- `transform`: Transform identifier for generated files. Empty or omitted may be allowed for copied source files.
-- `sha256`: Output file digest as lowercase `sha256:<64 hex>`.
-- `size`: Output file size in bytes.
-
-## Destination Comparison Rules
-
-Destination comparison uses `.distributor.json`, not destination `manifest.json`.
-
-For a source bundle and destination bundle path:
-
-### No `.distributor.json`
-
-If no `.distributor.json` exists and the destination path is empty:
-
-- Publish normally.
-
-If no `.distributor.json` exists and the destination path is non-empty:
-
-- Fail as unmanaged content.
-
-Local destination paths are empty when the destination bundle directory does not exist or exists with no entries. S3-compatible destination prefixes are empty when no objects exist below the destination bundle prefix, ignoring objects outside that exact prefix.
-
-### Pipeline or destination mismatch
-
-If `.distributor.json` exists but its `pipeline_id` or `destination_id` differs from the current pipeline or destination config:
-
-- Fail as a conflict.
-
-### Same source manifest
-
-If `.distributor.json` exists and `source.manifest` exactly matches the current normalized source manifest:
-
-- Skip as already published.
-
-### Same source id, destination older
-
-If `.distributor.json` exists, `source.manifest.id` matches the source manifest `id`, and destination `source.manifest.created` is older than the source `created`:
-
-- Replace destination contents, subject to replacement safety rules.
-
-### Same source id, destination newer
-
-If `.distributor.json` exists, `source.manifest.id` matches the source manifest `id`, and destination `source.manifest.created` is newer than the source `created`:
-
-- Skip and log that destination is newer than source.
-
-### Same source id and created, different digest
-
-If `.distributor.json` exists, `source.manifest.id` and `created` match but `digest` differs:
-
-- Fail as a conflict.
-
-### Different source id
-
-If `.distributor.json` exists and `source.manifest.id` differs from the source manifest `id`:
-
-- Fail as a conflict.
-
-## Replacement Safety Rules
-
-Replacement is destructive and must be narrow.
-
-`distributor` must never perform broad deletion against a configured destination root.
-
-Replacement may occur only at a resolved destination bundle path when:
-
-- a valid `.distributor.json` exists at that destination bundle path; and
-- the state identifies the path as distributor-managed; and
-- the replacement decision follows the destination comparison rules.
-
-For MVP, replacement should delete only known managed outputs where practical:
-
-- files listed in existing `.distributor.json.outputs`;
-- existing `.distributor.json`;
-- empty directories created by those files, where applicable for filesystem-like backends.
-
-For S3, replacement should delete only objects under the destination bundle prefix that are listed in `.distributor.json.outputs` plus `.distributor.json`, unless a later managed-prefix deletion policy is explicitly implemented.
-
-Before any write, planned destination output paths must be checked for collisions. For example, if `report.md` is copied as a source artifact and Markdown transformation would also generate `report.html`, but `report.html` is already a source artifact or another generated output, planning must fail before writing.
-
-## Transform Output Contract
-
-Source files are canonical. Generated files are derived publication artifacts.
-
-For MVP, the only supported transform is Markdown to HTML.
-
-Recommended MVP behavior:
-
-- Transform is configured per destination.
-- Markdown source files are files listed in `manifest.json` with `.md` extension.
-- Generated HTML files are sidecars by default.
-- `report.md` generates `report.html`.
-- Source bundles are never mutated.
-- Generated outputs are recorded in `.distributor.json.outputs`.
-- Raw HTML embedded in Markdown is escaped or disabled by default for the MVP to keep generated output deterministic and conservative.
-
-Future versions may add templates, `index.html`, CSS assets, email-safe HTML, and per-file transform selection.
-
-## Destination Output Contract
-
-Each destination chooses which categories of files it receives.
-
-MVP categories:
-
-- `source`: copied source artifacts listed in `manifest.json`.
-- `html`: generated HTML derived from Markdown source artifacts.
-
-Examples:
-
-- Markdown archive: `source: true`, `html: false`.
-- Static HTML site: `source: false`, `html: true`.
-- Full mirror: `source: true`, `html: true`.
-
-Every file written to the destination must be represented in `.distributor.json.outputs`.
-
-## Atomicity and Partial Failure
-
-The MVP should prefer staging and promotion where backend semantics permit it.
-
-Minimum behavior:
-
-- Validate source before writing destination files.
-- Do not write `.distributor.json` until all configured destination outputs are successfully written.
-- If publication fails before `.distributor.json` is written, the destination must not be treated as successfully published on a later run.
-- Local publication must use staging or equivalent cleanup behavior so a failed write does not leave a confusing unmanaged destination bundle path.
-- Later cleanup may remove orphaned files, but MVP correctness should rely on `.distributor.json` as the success marker.
-
-## Example Source Bundle
-
-```text
-weather/daily/brentwood/2026-05-30/
-  manifest.json
-  report.md
-  summary.txt
-```
-
-Example manifest:
-
-```json
-{
-  "schema_version": 1,
-  "id": "weather.daily.brentwood.2026-05-30",
-  "digest": "sha256:0000000000000000000000000000000000000000000000000000000000000000",
-  "created": "2026-05-30T11:10:00Z",
-  "files": [
-    {
-      "path": "report.md",
-      "sha256": "sha256:1111111111111111111111111111111111111111111111111111111111111111",
-      "size": 12345
-    },
-    {
-      "path": "summary.txt",
-      "sha256": "sha256:2222222222222222222222222222222222222222222222222222222222222222",
-      "size": 234
-    }
-  ]
-}
-```
-
-## Example Destination Bundle: HTML Only
-
-```text
-weather/daily/brentwood/2026-05-30/
-  report.html
-  .distributor.json
-```
-
-## Example Destination Bundle: Source Archive
-
-```text
-weather/daily/brentwood/2026-05-30/
-  report.md
-  summary.txt
-  .distributor.json
-```
-
-## Implementation Stages
-
-1. Define Go structs for source manifest and destination state.
-2. Implement source path validation.
-3. Implement per-file SHA256 and size validation.
-4. Implement canonical bundle digest validation.
-5. Implement source bundle discovery beneath a source root.
-6. Implement `.distributor.json` parsing and validation.
-7. Implement destination comparison rules.
-8. Implement replacement safety checks.
-9. Add fixture bundles for valid, invalid, duplicate path, older, newer, same, and conflict scenarios.
-10. Add reference canonical digest fixtures.
-11. Use the contract layer from the publish pipeline and backend adapters.

docs/roadmap/packages.md

@@ -1,526 +0,0 @@
-# Package Layout Roadmap
-
-This roadmap defines the proposed package layout, boundaries, and implementation responsibilities for the `distributor` MVP.
-
-`distributor` is expected to be a domain-agnostic bundle publisher. Producer applications emit source bundles containing `manifest.json`; `distributor` validates those bundles and publishes selected source and generated artifacts to one or more configured destinations.
-
-This document is roadmap material. It describes the intended package design before implementation and should move into `docs/internal/` only after corresponding behavior exists.
-
-## Accepted Package Layout
-
-```text
-cmd/distributor/
-  main.go
-
-internal/app/
-  app.go
-  run.go
-  pipeline.go
-
-internal/cli/
-  root.go
-  run.go
-  validate.go
-  inspect.go
-
-internal/config/
-  config.go
-  defaults.go
-  load.go
-  validate.go
-
-internal/bundle/
-  manifest.go
-  digest.go
-  validate.go
-  discover.go
-
-internal/state/
-  distributor.go
-  compare.go
-  validate.go
-
-internal/storage/
-  backend.go
-  registry.go
-  path.go
-  errors.go
-
-internal/storage/fake/
-  backend.go
-
-internal/adapters/local/
-  backend.go
-
-internal/adapters/ssh/
-  backend.go
-  config.go
-
-internal/adapters/s3/
-  backend.go
-  config.go
-
-internal/transform/
-  transform.go
-  registry.go
-  plan.go
-
-internal/transform/markdown/
-  markdown.go
-  template.go
-
-internal/publish/
-  plan.go
-  execute.go
-  output.go
-  reconcile.go
-  safety.go
-
-internal/notify/
-  notify.go
-  noop.go
-
-internal/logging/
-  logging.go
-```
-
-## Package Responsibilities
-
-### `cmd/distributor`
-
-Application entrypoint only.
-
-Responsibilities:
-
-- call CLI execution;
-- translate process exit status;
-- avoid business logic.
-
-Non-responsibilities:
-
-- config loading;
-- backend construction;
-- bundle validation;
-- publish decisions.
-
-### `internal/cli`
-
-CLI command definitions, flags, argument parsing, and command wiring.
-
-Expected MVP commands:
-
-- `distributor run` — run configured pipelines.
-- `distributor run --dry-run` — plan without modifying destinations.
-- `distributor run --pipeline <id>` — run one configured pipeline.
-- `distributor validate <path>` — validate a source bundle or source tree where feasible.
-- `distributor inspect <path>` — inspect a bundle or destination state where feasible.
-
-Boundaries:
-
-- CLI should call `internal/app` use cases.
-- CLI should not parse manifests directly except through application APIs.
-- CLI should not import backend adapter implementation details unless only for registration side effects.
-
-### `internal/app`
-
-Application orchestration and top-level use cases.
-
-Responsibilities:
-
-- load and validate configuration;
-- construct configured pipelines;
-- build source and destination backends through registries;
-- orchestrate discovery, validation, planning, publishing, and notification;
-- coordinate dry-run output;
-- run destination fan-out deterministically and sequentially;
-- aggregate destination outcomes into run-level failure behavior.
-
-Core orchestration shape:
-
-```text
-for each selected pipeline:
-  open source backend
-  discover source bundles
-  for each source bundle:
-    validate source manifest and digest
-    for each destination:
-      inspect .distributor.json
-      build publish plan
-      transform as required by that destination
-      execute publish plan unless dry-run
-      run noop notifier after actual publication or replacement
-```
-
-Boundaries:
-
-- `internal/app` composes packages but should not contain backend-specific logic.
-- Publish decisions should live in `internal/publish`, not inline in orchestration.
-- Destination state comparison should live in `internal/state` or `internal/publish`, not CLI code.
-
-### `internal/config`
-
-Configuration structs, defaults, loading, precedence, and validation.
-
-Responsibilities:
-
-- load `/usr/local/etc/distributor/config.yml` by default;
-- support `--config` override;
-- apply defaults;
-- validate required fields;
-- validate pipeline ids and destination ids;
-- validate backend-specific config shapes;
-- validate transform and publish policy combinations.
-
-MVP config model:
-
-```yaml
-pipelines:
-  - id: weather-daily
-    source:
-      backend: local
-      path: /var/spool/distributor/weather
-    validation:
-      on_digest_mismatch: fail
-    destinations:
-      - id: markdown-archive
-        backend: s3
-        endpoint: https://s3.example.com
-        bucket: reports
-        prefix: weather/archive
-        region: us-east-1
-        force_path_style: true
-        publish:
-          source: true
-          html: false
-        transfer:
-          on_destination_same: skip
-          on_destination_older: replace
-          on_destination_newer: skip
-          on_conflict: fail
-      - id: static-site
-        backend: ssh
-        uri: ssh://deploy@example.com:22
-        path: /srv/www/weather
-        publish:
-          source: false
-          html: true
-        transform:
-          markdown_to_html:
-            enabled: true
-            mode: sidecar
-```
-
-Configuration principles:
-
-- one source per pipeline;
-- one or more destinations per pipeline;
-- transforms are destination-specific;
-- publish policy is destination-specific;
-- secrets should use environment variables, secret files, SSH agent, or standard credential mechanisms rather than raw YAML values.
-
-### `internal/bundle`
-
-Source bundle contract and validation.
-
-Responsibilities:
-
-- parse source `manifest.json`;
-- represent source manifests and files;
-- discover bundle roots beneath a configured source root;
-- validate required manifest fields;
-- validate RFC3339 `created` values;
-- validate relative paths;
-- validate file existence, size, per-file SHA-256, and bundle digest;
-- expose normalized source bundle models to other packages.
-
-Core types:
-
-```go
-type Manifest struct {
-    SchemaVersion int
-    ID      string
-    Digest  string
-    Created time.Time
-    Files   []ManifestFile
-}
-
-type ManifestFile struct {
-    Path   string
-    SHA256 string
-    Size   int64
-}
-
-type Bundle struct {
-    RootRelativePath string
-    Manifest         Manifest
-}
-```
-
-Boundaries:
-
-- `internal/bundle` does not know about `.distributor.json`.
-- `internal/bundle` does not know about destinations, transforms, or notification.
-- `internal/bundle` may use the storage abstraction to read source files, but it should not import backend adapter packages.
-
-### `internal/state`
-
-Destination state contract for `.distributor.json`.
-
-Responsibilities:
-
-- parse `.distributor.json`;
-- validate destination state;
-- represent copied source outputs and generated outputs;
-- embed the source manifest used for publication;
-- compare destination state against a current source manifest;
-- classify destination state as same, older, newer, conflict, absent, invalid, or unmanaged.
-
-Core types:
-
-```go
-type DistributorState struct {
-    SchemaVersion int
-    DistributorVersion string
-    PipelineID    string
-    DestinationID string
-    PublishedAt   time.Time
-    Source        SourceState
-    Outputs       []OutputFile
-}
-
-type SourceState struct {
-    Manifest bundle.Manifest
-}
-
-type OutputFile struct {
-    Path       string
-    Kind       string // source | generated
-    SourcePath string
-    Transform  string
-    SHA256     string
-    Size       int64
-}
-```
-
-Comparison rules:
-
-- same source manifest: skip;
-- same source id, older destination source `created`: replace;
-- same source id, newer destination source `created`: skip;
-- same source id, same `created`, different digest: conflict;
-- different source id: conflict;
-- pipeline id or destination id mismatch: conflict;
-- absent state: publish only if safe;
-- unmanaged non-empty path: fail.
-
-Boundaries:
-
-- `internal/state` owns destination state semantics, not publish execution.
-- `internal/state` should not know about S3, SSH/SFTP, local filesystem details, or Markdown rendering.
-
-### `internal/storage`
-
-Backend abstraction and shared storage types.
-
-Responsibilities:
-
-- define storage backend interfaces;
-- define object/file metadata types;
-- define path/prefix helpers;
-- define common storage errors;
-- provide backend registry mechanisms;
-- provide a fake backend for core package tests.
-
-The detailed storage contract is defined in `docs/roadmap/storage.md`. Core application code should use that storage interface for backend-rooted logical paths, byte and stream IO, metadata, traversal, typed errors, emptiness checks, and managed deletion.
-
-Destructive APIs should remain narrow. Prefer managed deletion of files recorded in `.distributor.json` instead of broad recursive deletion.
-
-Boundaries:
-
-- `internal/storage` should not contain backend implementation details.
-- Adapter dependencies must not leak through storage interfaces.
-- The fake backend exists for tests and should not become an application runtime backend.
-
-### `internal/adapters/local`
-
-Local filesystem backend.
-
-Responsibilities:
-
-- implement `storage.Backend` for local paths;
-- clean and constrain paths;
-- perform safe reads/writes/listing/deletion;
-- use atomic writes where practical;
-- reject unsafe path traversal;
-- handle symlink policy explicitly.
-
-Testing expectations:
-
-- use temporary directories;
-- verify path traversal rejection;
-- verify write and delete safety.
-
-### `internal/adapters/ssh`
-
-SSH/SFTP backend.
-
-Responsibilities:
-
-- implement `storage.Backend` over SSH/SFTP;
-- support `uri` and `path` config;
-- prefer native SFTP implementation;
-- use SSH agent, key files, known hosts, or documented auth mechanisms;
-- avoid raw passwords in config unless explicitly designed and documented later;
-- translate SSH/SFTP errors into storage-level errors.
-
-Testing expectations:
-
-- core app tests should use fake backends;
-- adapter tests may use local test servers or targeted integration tests if practical;
-- do not require a real production SSH host for normal unit tests.
-
-### `internal/adapters/s3`
-
-S3-compatible object storage backend.
-
-Responsibilities:
-
-- implement `storage.Backend` over S3-compatible object storage;
-- support endpoint, bucket, prefix, region, and force-path-style configuration;
-- support standard credential mechanisms or explicit environment-variable references;
-- treat S3 as an object tree, not a filesystem;
-- set reasonable content types where practical;
-- guard against prefix/root deletion mistakes.
-
-Testing expectations:
-
-- core app tests should use fake backends;
-- adapter behavior may be tested through mocks, local S3-compatible services, or narrow integration tests;
-- config examples should avoid real secrets.
-
-### `internal/transform`
-
-Transform interfaces, registry, and transform planning.
-
-Responsibilities:
-
-- define transform interfaces;
-- register available transforms;
-- represent transform requests and outputs;
-- keep transform execution independent of destination backend details.
-
-Boundaries:
-
-- transforms operate on source bundle content and destination transform config;
-- transforms do not publish outputs;
-- transforms do not mutate source bundles;
-- transforms should return generated output metadata for `.distributor.json`.
-
-### `internal/transform/markdown`
-
-Markdown-to-HTML implementation.
-
-Responsibilities:
-
-- render listed Markdown files to HTML;
-- support MVP sidecar behavior, such as `report.md` -> `report.html`;
-- record generated output path, source path, transform name, SHA-256, and size;
-- optionally use embedded templates if needed.
-
-MVP scope:
-
-- Markdown to HTML only;
-- no PDF generation;
-- no email-specific HTML;
-- no complex theming unless required for basic output correctness.
-
-### `internal/publish`
-
-Destination planning, reconciliation, safety checks, and publish execution.
-
-Responsibilities:
-
-- inspect destination state;
-- plan destination action;
-- enforce destination conflict rules;
-- enforce destructive-operation safety rules;
-- detect output path collisions before writing;
-- combine source files and transform outputs according to destination publish policy;
-- write destination outputs;
-- write `.distributor.json`;
-- use staging or equivalent cleanup behavior where practical;
-- support dry-run planning;
-- report skipped, replaced, failed, and published actions.
-
-Action model:
-
-```text
-publish
-replace
-skip_same
-skip_destination_newer
-fail_conflict
-fail_unmanaged
-```
-
-Boundaries:
-
-- publish logic should not parse CLI flags;
-- publish logic should not know adapter implementation details;
-- publish logic should use `internal/state` for destination state semantics;
-- publish logic should use `internal/storage` interfaces for IO.
-
-### `internal/notify`
-
-Notification stage abstraction.
-
-MVP responsibilities:
-
-- define notifier interface;
-- implement no-op notifier;
-- preserve future extension point for email, ntfy, Gotify, RSS update hooks, or other notification channels.
-
-Future notification rules:
-
-- notify only after successful publication to the relevant destination or destinations;
-- notification must be idempotent with respect to source id, digest, pipeline id, and destination id where applicable;
-- notification should not run for skipped or failed publications unless explicitly configured.
-
-### `internal/logging`
-
-Logging setup and helpers.
-
-Responsibilities:
-
-- centralize structured logging setup;
-- ensure logs omit secrets;
-- provide consistent fields for pipeline id, bundle id, destination id, backend, path, action, and reason.
-
-## Deferred Ideas
-
-The following are intentionally out of MVP unless separately accepted in a later roadmap:
-
-- email, ntfy, Gotify, or other real notification adapters;
-- RSS/Atom feed generation;
-- PDF generation;
-- web UI;
-- full-text search;
-- dynamic plugin loading;
-- arbitrary transform chains;
-- workflow DAGs;
-- producer execution;
-- complex templating/theming;
-- bidirectional sync;
-- backup semantics.
-
-## Key Invariants
-
-- Producer apps own source bundle creation.
-- `distributor` owns destination publication state.
-- Source `manifest.json` is not copied as destination state.
-- Destination `.distributor.json` is the managed sentinel.
-- One pipeline has one source and one or more destinations.
-- Transform and publish policy are destination-specific.
-- Source files are canonical; HTML is derived.
-- Destructive replacement is allowed only inside managed destination bundle paths.
-- Core logic must be testable without real S3, SSH, or remote services.

docs/roadmap/storage.md

@@ -1,265 +0,0 @@
-# Storage Interface Roadmap
-
-This roadmap defines the planned `internal/storage` contract for the `distributor` MVP. The goal is to give bundle validation, destination state inspection, publish planning, and backend adapters one consistent IO boundary without leaking local filesystem, SSH/SFTP, or S3-specific behavior into core packages.
-
-## Purpose and Invariants
-
-The storage layer is responsible for safe, backend-rooted access to files, objects, prefixes, and destination bundle paths.
-
-Core invariants:
-
-- Backends are opened at configured roots.
-- Core packages operate on backend-rooted logical paths, not absolute filesystem paths or raw object keys.
-- Backend adapters translate native storage behavior into common storage entries and typed errors.
-- Destructive operations remain narrow and managed.
-- Staging or atomic write behavior belongs behind the storage interface where practical.
-- The fake backend exists for tests only and must not be registered as a runtime backend.
-
-## Logical Path Model
-
-Storage paths are slash-separated logical paths relative to an already configured backend root.
-
-File paths:
-
-- must be non-empty;
-- must be relative;
-- must be clean;
-- must not contain `.` or `..` segments;
-- must not start with `/`;
-- must not contain backslashes;
-- must not resolve outside the backend root.
-
-Prefix paths use the same slash-separated model. A prefix may be empty to represent the backend root for traversal and destination emptiness checks.
-
-Prefix matching must preserve logical path boundaries. A prefix of `foo` matches `foo` and entries below `foo/`; it must not match a sibling path such as `foobar`. Backends that map logical paths to object keys must apply the same normalized boundary rule after combining configured backend prefixes with caller-provided logical prefixes.
-
-Backends own conversion from logical paths to native paths or object keys. Core packages should not construct local filesystem paths, SFTP paths, or S3 object keys directly.
-
-## Core Interface Shape
-
-The MVP should use a hybrid byte and stream interface:
-
-```go
-type Backend interface {
-    ReadFile(ctx context.Context, path string) ([]byte, error)
-    OpenReader(ctx context.Context, path string) (io.ReadCloser, error)
-    WriteFile(ctx context.Context, path string, data []byte, opts WriteOptions) (Entry, error)
-    WriteFrom(ctx context.Context, path string, r io.Reader, opts WriteOptions) (Entry, error)
-    Stat(ctx context.Context, path string) (Entry, error)
-    Walk(ctx context.Context, prefix string, opts WalkOptions, fn WalkFunc) error
-    HasAny(ctx context.Context, prefix string) (bool, error)
-    DeleteManagedBundle(ctx context.Context, bundlePath string, managedOutputPaths []string, opts DeleteOptions) error
-}
-```
-
-Byte helpers are expected to cover manifests, destination state, small source artifacts, and generated outputs. Stream methods are included from the start for backend flexibility and larger future artifacts.
-
-Write operations should create required parent directories or prefixes as needed.
-
-Concrete option and callback types should use this shape:
-
-```go
-type WalkOptions struct {
-    Recursive bool
-    Limit     int
-}
-
-type WalkFunc func(Entry) error
-
-var ErrStopWalk = errors.New("stop walk")
-
-type WriteOptions struct {
-    ContentType  string
-    Overwrite    bool
-    PreferAtomic bool
-    Size         int64
-    SizeKnown    bool
-}
-
-type DeleteOptions struct {
-    IgnoreMissing  bool
-    PruneEmptyDirs bool
-}
-```
-
-`WalkOptions.Limit == 0` means no explicit limit. `SizeKnown` applies primarily to `WriteFrom`; byte writes can infer size from the provided data.
-
-## Entries and Metadata
-
-Storage metadata should be represented by an `Entry` model with at least:
-
-- backend-relative logical path;
-- entry type;
-- size, where available.
-
-Entry types:
-
-- `file`: filesystem file or object-storage object;
-- `directory`: filesystem directory or logical prefix;
-- `symlink`: local filesystem symlink;
-- `other`: unknown or unsupported native entry type.
-
-`Stat` returns metadata for one exact logical path. It may report a real filesystem directory, symlink, file, or exact object. It must not synthesize S3-like directory metadata solely because objects exist below a prefix; callers that need prefix existence or destination emptiness must use `HasAny` or `Walk`.
-
-`Walk` traverses entries below a prefix and calls a callback for each entry. `WalkOptions` should include:
-
-- whether traversal is recursive;
-- an optional entry limit for callers that only need to know whether content exists.
-
-If a callback returns `ErrStopWalk`, traversal stops successfully and `Walk` returns nil. Any other callback error stops traversal and is returned with storage context where practical. If `WalkOptions.Limit` is greater than zero, reaching the limit stops traversal successfully.
-
-Backends may stream or paginate traversal internally. S3-compatible adapters should not need to load a whole prefix into memory to satisfy traversal.
-
-Raw traversal is not required to be lexically sorted. A helper that materializes walk results for bundle discovery, tests, or CLI output should sort entries lexically by logical path before returning them.
-
-`HasAny` reports whether at least one entry exists below a prefix. It should stop as soon as content is found.
-
-Source validation must reject symlink entries reported by local `Stat` or `Walk`.
-
-## Read Behavior
-
-`ReadFile` reads the whole object into memory and is appropriate for MVP manifest, state, and ordinary artifact handling.
-
-`OpenReader` returns a stream for callers that need to copy or hash content without requiring a second storage-specific API. Callers must close the returned reader.
-
-Both read methods must:
-
-- validate logical paths before backend access;
-- reject directories, prefixes, symlinks, and unsupported entries;
-- return typed not-found and invalid-path errors where applicable.
-
-## Write Behavior
-
-`WriteOptions` should include:
-
-- content type, when the destination backend can use it;
-- overwrite permission;
-- atomic or staged write preference;
-- optional known size for stream writes.
-
-Backends own staging and atomic behavior where practical:
-
-- Local backend writes to a temporary file in the destination directory and renames or promotes into place.
-- SSH/SFTP backend should use a temporary remote file and rename where available.
-- S3-compatible backend treats a successful object PUT as publish-on-success and applies content type metadata.
-
-Remote adapters may buffer or spool `WriteFrom` input when needed to satisfy backend requirements such as content length, multipart upload, or retry behavior. Callers that know the stream size should set `SizeKnown` and `Size`.
-
-If overwrite is false and the target exists, writes should fail with an already-exists error.
-
-`WriteFile` and `WriteFrom` should return the written `Entry`, including final path and size where available.
-
-`DeleteOptions` should include:
-
-- whether missing managed output paths are ignored;
-- whether empty parent directories may be pruned for filesystem-like backends.
-
-## Managed Deletion
-
-The storage interface should expose a guarded managed deletion operation rather than raw recursive delete.
-
-`DeleteManagedBundle(ctx, bundlePath, managedOutputPaths, opts)` may delete only:
-
-- files or objects listed in valid `.distributor.json.outputs`;
-- `.distributor.json` at the destination bundle path;
-- empty directories created by those files, for filesystem-like backends.
-
-`managedOutputPaths` are relative to the destination bundle path. The backend validates each path and resolves it under `bundlePath`.
-
-If `bundlePath == ""`, deletion may remove explicit managed files at the destination root, but must never delete the root itself.
-
-Prefix or recursive deletion is out of MVP scope. A future force-overwrite stage may add broader behavior, but it must remain explicit and separately documented.
-
-## Destination Emptiness
-
-Destination emptiness should use `HasAny(prefix)` and typed not-found behavior. Callers that only need emptiness must not materialize a full recursive traversal.
-
-Rules:
-
-- A local destination bundle path is empty when the directory does not exist or exists with no entries.
-- An S3-compatible prefix is empty when no objects exist below that exact destination bundle prefix.
-- Entries outside the exact destination bundle path or prefix do not affect emptiness.
-
-## Error Model
-
-Storage should expose typed error categories with wrapping context. Callers should use helper predicates rather than string matching.
-
-Required categories:
-
-- not found;
-- already exists;
-- not empty;
-- invalid path;
-- conflict;
-- permission;
-- temporary;
-- unsupported;
-- unknown.
-
-Adapters should translate backend-native errors into these categories while preserving useful operation, backend, path, and cause context.
-
-## Adapter Expectations
-
-### Local
-
-The local backend should:
-
-- constrain all operations beneath the configured root;
-- reject traversal and absolute logical paths;
-- report symlinks through metadata;
-- reject symlink reads for source artifacts;
-- use staged writes where practical;
-- perform managed deletion only for explicit managed files and `.distributor.json`;
-- clean up empty directories created by managed outputs where safe.
-
-### Fake
-
-The fake backend should:
-
-- be in-memory and deterministic;
-- implement the same logical path validation rules;
-- support `Stat`, `Walk`, `HasAny`, byte reads and writes, stream reads and writes, managed deletion, and destination emptiness helper behavior;
-- support configured symlink entries for validation tests;
-- be used only by tests.
-
-### SSH/SFTP
-
-The SSH/SFTP backend should:
-
-- use native SFTP operations;
-- enforce the same logical path rules as local storage;
-- use temporary file plus rename for staged writes where available;
-- translate remote errors into storage error categories;
-- avoid exposing SSH or SFTP dependency types through `internal/storage`.
-
-### S3-Compatible
-
-The S3-compatible backend should:
-
-- treat prefixes as object trees, not real directories;
-- normalize configured prefix plus logical path into object keys;
-- use object PUT as publish-on-success;
-- set content type from `WriteOptions`;
-- implement traversal and emptiness by exact prefix;
-- use backend pagination for traversal where available;
-- allow `HasAny` to stop after the first matching object;
-- constrain managed deletion to listed output objects and `.distributor.json`.
-
-## Tests and Fixtures
-
-Storage implementation stages should test:
-
-- path validation rejects absolute paths, traversal, empty file paths, backslashes, and dot segments;
-- `Walk` visits backend-rooted logical paths under a prefix and supports recursive traversal;
-- a materializing helper sorts walk results lexically for deterministic tests and CLI output;
-- `HasAny` returns quickly for non-empty prefixes without requiring full traversal;
-- `ReadFile` and `OpenReader` return equivalent bytes;
-- `WriteFile` and `WriteFrom` honor overwrite and content-type options;
-- local staged writes do not leave final files on failure where testable;
-- managed deletion deletes only state-listed files and `.distributor.json`;
-- managed deletion never deletes destination root or unlisted files;
-- destination emptiness helper handles missing, empty, and non-empty local paths;
-- S3-compatible traversal can use pagination without loading a whole prefix into memory;
-- symlink entries are reported and rejected by source validation;
-- typed errors are usable through helper predicates;
-- fake backend behavior matches local backend semantics relevant to core tests.

docs/troubleshooting.md

@@ -0,0 +1,135 @@
+# Distributor Troubleshooting
+
+## `load config ... no such file or directory`
+
+Likely cause: `run` could not find the config path. If `--config` is omitted, the default path is `/usr/local/etc/distributor/config.yml`.
+
+Diagnostic:
+
+```sh
+ls -l <config-path>
+```
+
+Safe fix: pass an existing config path with `--config`, or install a config at the default path. See [configuration](config.md).
+
+## `parse config ... field not found`
+
+Likely cause: the YAML contains an unknown field. Config loading rejects unknown keys.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --dry-run
+```
+
+Safe fix: compare the file to the reference in [configuration](config.md) and remove or rename unsupported fields.
+
+## `validate config ... backend ... is unsupported`
+
+Likely cause: a source or destination uses a backend name other than `local`, `ssh`, or `s3`.
+
+Diagnostic:
+
+```sh
+rg -n "backend:" <config-path>
+```
+
+Safe fix: use `backend: local` for executable workflows. SSH and S3 config shapes are accepted only for validation; runtime execution is unavailable.
+
+## `backend ssh is not implemented for execution` or `backend s3 is not implemented for execution`
+
+Likely cause: the config validates but `run` tried to execute a remote backend.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --dry-run
+```
+
+Safe fix: use local destinations for current executable workflows, or keep remote backend configs under roadmap material unless those adapters are added. See [configuration](config.md).
+
+## `validate command requires a path` or `inspect command requires a path`
+
+Likely cause: `validate` or `inspect` was run without a path.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor validate --help
+go run ./cmd/distributor inspect --help
+```
+
+Safe fix: pass a local source bundle directory or a local tree containing source bundles.
+
+## `no bundles found under "."`
+
+Likely cause: the selected source root does not contain a `manifest.json` source bundle.
+
+Diagnostic:
+
+```sh
+find <source-root> -name manifest.json -print
+```
+
+Safe fix: point the command or config at the directory containing the source bundle, or write a valid `manifest.json` and listed files. See [CLI](cli.md).
+
+## `sha256 mismatch`, `size mismatch`, or `digest mismatch`
+
+Likely cause: a listed source file changed after `manifest.json` was created, or the manifest digest does not match its file list.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor validate <source-root>
+```
+
+Safe fix: regenerate the producer bundle and manifest together. Do not edit destination state to work around source digest failures.
+
+## `destination has content but no distributor state`
+
+Likely cause: the destination path is not empty and has no `.distributor.json` state file, so `distributor` will not claim it as managed.
+
+Diagnostic:
+
+```sh
+find <destination-path> -maxdepth 2 -print
+```
+
+Safe fix: choose an empty destination path or move existing files aside after confirming they are not needed. There is no force overwrite option.
+
+## `fail_conflict`
+
+Likely cause: existing `.distributor.json` belongs to a different pipeline, a different destination, a different source id, or a same-created source with a different digest.
+
+Diagnostic:
+
+```sh
+cat <destination-path>/.distributor.json
+go run ./cmd/distributor inspect <source-root>
+```
+
+Safe fix: verify you are publishing the intended source to the intended destination. Use a separate destination path for unrelated content.
+
+## `destination output path collision`
+
+Likely cause: configured publication would write two outputs to the same destination path, such as publishing a source `report.html` while also generating `report.html` from `report.md`.
+
+Diagnostic:
+
+```sh
+go run ./cmd/distributor run --config <config-path> --dry-run
+```
+
+Safe fix: adjust the source bundle contents or publish policy so source and generated outputs do not collide.
+
+## A run failed after writing some files
+
+Likely cause: a write failed partway through publication. Local execution attempts to clean up outputs written during the failed attempt.
+
+Diagnostic:
+
+```sh
+find <destination-path> -maxdepth 2 -print
+```
+
+Safe fix: inspect the destination before retrying. If only unrelated unmanaged files remain, move them aside or choose a clean destination. Re-run with `--dry-run` before publishing again. See [operations](operations.md).

examples/fan-out.yml

@@ -1,28 +1,18 @@
 pipelines:
-  - id: reports
+  - id: example-fan-out
     source:
       backend: local
-      path: /var/spool/distributor/reports
-    validation:
-      on_digest_mismatch: fail
+      path: examples/source-bundle
     destinations:
-      - id: markdown-archive
-        backend: s3
-        endpoint: https://s3.example.com
-        bucket: reports
-        prefix: archive
-        region: us-east-1
-        force_path_style: true
-        credentials:
-          access_key_id_env: DISTRIBUTOR_S3_ACCESS_KEY_ID
-          secret_access_key_env: DISTRIBUTOR_S3_SECRET_ACCESS_KEY
+      - id: local-source-archive
+        backend: local
+        path: workspace/published/fan-out/source
         publish:
           source: true
           html: false
-      - id: static-site
-        backend: ssh
-        uri: ssh://deploy@example.com:22
-        path: /srv/www/reports
+      - id: local-html-site
+        backend: local
+        path: workspace/published/fan-out/html
         publish:
           source: false
           html: true