distributor/docs/roadmap/latest_paths.md

# Roadmap: Latest Path Destinations

## Purpose

Support stable "latest" publication paths as normal fan-out destinations.

A latest destination republishes the current winning source bundle to a fixed
destination path such as `/weather/latest/`, while archive destinations preserve
source-relative bundle paths such as `/weather/daily/brentwood/2026-06-01/`.

This feature should be implemented before link generation so URL construction
can use final destination path semantics. It is operationally destructive and
benefits from index-mode HTML, although it must not require index mode or link
generation.

## Current Implementation Grounding

Current run behavior computes the destination bundle path as the source bundle
path relative to the source root. Publication then writes selected outputs and
`.distributor.json` below that destination bundle path.

Backend roots differ by backend:

- local and SSH/SFTP use configured `path` as the backend root;
- S3 uses configured bucket plus optional `prefix` as the backend root.

Destination comparison uses `.distributor.json` at the destination bundle path.
Replacement and skip decisions are based on the normalized source manifest
recorded in destination state. Unmanaged non-empty destinations fail unless
explicit force behavior is requested.

## Goals

- Add destination-local path mapping.
- Preserve source-relative destination paths as the default.
- Add fixed path mapping for latest-style destinations.
- For fixed destinations, select only the newest discovered source bundle for
  publication.
- Make fixed mapping work for local, SSH/SFTP, and S3 backends.
- Continue using normal destination state comparison and replacement rules.
- Keep link generation and index-mode HTML complementary, not required.

## Non-Goals

- Do not add symlink-based latest behavior.
- Do not add feed or index generation.
- Do not change source manifest schema.
- Do not make producers responsible for latest publication.
- Do not add domain-specific latest selection rules.

## Configuration Shape

Add destination-level path mapping:

```yaml
path_mapping:
  mode: preserve_relative
```

Modes:

- `preserve_relative`: existing/default behavior.
- `fixed`: publish selected outputs directly at the destination backend root.

Example fixed destination:

```yaml
destinations:
  - id: latest-html
    backend: ssh
    host: web.example.com
    user: deploy
    path: /srv/www/weather/latest
    path_mapping:
      mode: fixed
    publish:
      source: false
      html: true
```

For S3, the fixed destination root is the configured bucket plus `prefix`.

## Path Mapping Semantics

`preserve_relative`:

```text
destination bundle path = source-root-relative bundle path
```

`fixed`:

```text
destination bundle path = ""
```

That empty logical destination bundle path means the destination backend root.
Storage writes still use normal backend-rooted logical paths for outputs and
state.

## Newest Bundle Selection

`preserve_relative` destinations should continue publishing every selected
source bundle independently.

`fixed` destinations should publish only one bundle per run: the newest
discovered source bundle selected for that destination. Newest selection should
be deterministic:

1. choose the bundle with the greatest source manifest `created` timestamp;
2. if multiple bundles have the same greatest `created` timestamp, use the
   source-root-relative bundle path ascending as a tie-breaker.

The selected bundle then uses the existing destination comparison, skip,
replacement, cleanup, and force rules. Older discovered bundles should not be
planned or written to that fixed destination in the same run.

Dry-run output should identify the number of fixed-destination candidates and
the selected source bundle so operators can see which bundle would become
latest.

## Safety Rules

Fixed path mapping is more destructive than archive-style publication because
newer bundles replace prior contents at the same destination path.

Required behavior:

- dry-run output must identify fixed path mapping and planned replacement;
- dry-run output must include an extra fixed-path warning or summary count when
  destructive replacement is possible;
- replacement should delete only managed outputs recorded in valid destination
  state when possible;
- unmanaged non-empty fixed destinations must fail unless force is explicitly
  requested;
- `--force` may be used when fixed mapping targets the backend root, but force
  replacement must remain bounded to the configured destination backend root and
  must never delete above it;
- fixed mapping to the backend root requires especially clear dry-run reporting.

## Relationship To Other Roadmaps

HTML index mode is useful for fixed web destinations because `index.html`
supports stable directory-style URLs, but sidecar HTML and source-only outputs
should still work.

Link generation should use fixed path semantics so latest URLs are based on the
fixed destination root, not the original source-relative archive path.

Producer manifest creation and remote validation are independent of destination
path mapping.

## Implementation Stages

1. Add destination-level `path_mapping` config validation with
   `preserve_relative` as the default and `fixed` as the new mode.
2. Refactor run planning so destination bundle path mapping is destination-local
   and can be evaluated before writes.
3. Add fixed-destination newest selection so each fixed destination receives
   only the newest discovered source bundle.
4. Integrate fixed mapping with destination comparison, managed deletion, force
   replacement, and dry-run reporting.
5. Update local, SSH, and S3 tests to verify fixed root behavior under each
   backend root model.
6. Update current-behavior documentation after implementation.

## Testing Expectations

Suggested coverage:

- omitted `path_mapping` preserves existing behavior;
- `preserve_relative` explicitly matches existing behavior;
- `fixed` publishes outputs and `.distributor.json` at the backend root;
- fixed destinations select only the newest discovered bundle;
- fixed destination tie-break behavior is deterministic;
- older discovered bundles are not planned or written to fixed destinations;
- newer source replaces older managed fixed destination state;
- older source skips newer fixed destination state;
- unmanaged non-empty fixed destination fails without force;
- dry-run reports fixed path mapping, candidate count, selected bundle, and
  destructive replacement warnings clearly;
- `--force` remains bounded when fixed mapping targets the backend root;
- S3 fixed mapping respects bucket plus prefix as backend root.

## Documentation Updates After Implementation

- Update `docs/config.md` with `path_mapping`.
- Update `docs/operations.md` with archive-plus-latest fan-out examples.
- Update `docs/cli.md` if dry-run output gains fixed-path indicators.
- Add an environment-safe example only after behavior is implemented.

Keep this roadmap under `docs/roadmap/` until implemented.

## Decisions

- `--force` is allowed when fixed mapping targets the backend root, but dry-run
  and run output must make the fixed-root replacement explicit and force remains
  bounded to the configured backend root.
- Fixed destinations publish only the newest discovered bundle, rather than
  processing every discovered bundle and letting later writes replace earlier
  writes.
- Dry-run includes an extra warning or summary count for fixed-path destructive
  replacements.

## Future Work

- Add richer source selection policies if deployments need something other than
  newest-by-`created` for fixed destinations.
- Consider stricter ambiguity handling for equal latest timestamps if real
  producer workflows make timestamp ties common.
- Add higher-level status or approval workflows for fixed-root replacements if
  dry-run output is not enough operational protection.