Consolidate future work into docs/roadmap/future.md and remove completed roadmap docs
All checks were successful
ci/woodpecker/tag/release Pipeline was successful

This commit is contained in:
2026-06-09 08:42:27 -05:00
parent c804fd604a
commit e3b92a3b5d
7 changed files with 23 additions and 237 deletions

View File

@@ -1,25 +0,0 @@
# API Roadmap
This document records API work that is not part of the current implementation. Current HTTP upload behavior is documented in `docs/integrations/http-upload.md`, current configuration behavior is documented in `docs/config.md`, and current producer package usage is documented under `docs/consumers/`.
## Deferred Upload API Work
- Durable upload status storage.
- Durable idempotency records across server restarts.
- Run listing, retry, and cancellation endpoints.
- Long-polling or wait-for-completion workflows.
- Multipart, resumable, or streaming upload protocols.
- Additional archive content negotiation beyond tar and gzip-compressed tar.
- URL-token authentication for constrained clients.
- Upload token lifecycle tooling.
- Mutual TLS or other in-app identity mechanisms.
- In-app TLS termination.
- In-app public exposure policy.
- In-app upload rate limiting.
## Boundaries
- Producers do not choose destination ids, destination paths, transforms, links, publish policy, transfer policy, or storage backends through upload requests.
- Source manifests remain free of routing, destination, transform, and credential data.
- `http_upload` remains source-only unless a future design changes that contract.
- Public access policy, TLS termination, and rate limiting belong in deployment infrastructure unless a future design changes that boundary.

View File

@@ -33,6 +33,21 @@ internal docs.
- Resumable upload support. - Resumable upload support.
- Streaming upload protocols. - Streaming upload protocols.
## Destination State Repair
- Unmanaged-file adoption workflow for destination state repair.
- Digest-audit mode for managed destination outputs.
- Explicit invalid-state repair workflow.
- Whole-config state repair command.
## Destination Retention And Pruning
- One-off retention overrides for `prune`.
- Automatic post-publish pruning.
- Path/date parsing retention policies.
- Group-level pruning by source publication.
- Removal of empty state files.
## Destination Backends ## Destination Backends
- GitHub Gist destination backend support. - GitHub Gist destination backend support.
@@ -68,6 +83,7 @@ internal docs.
- Mutual TLS or other in-app identity mechanisms. - Mutual TLS or other in-app identity mechanisms.
- In-app TLS. - In-app TLS.
- Public exposure defaults. - Public exposure defaults.
- In-app upload rate limiting.
- Browser UI. - Browser UI.
## Boundaries ## Boundaries
@@ -76,5 +92,12 @@ internal docs.
contract. contract.
- Current upload status, queue, and idempotency state are memory-only. - Current upload status, queue, and idempotency state are memory-only.
- Producers submit complete tar or gzip-compressed tar source bundles today. - Producers submit complete tar or gzip-compressed tar source bundles today.
- Producers do not choose destination ids, destination paths, transforms, links,
publish policy, transfer policy, storage backends, reconciliation mode, state
mode, or retention policy through upload requests.
- Source manifests remain free of routing, destination, transform, credential,
reconciliation, state, and retention data.
- Public access policy, TLS termination, and rate limiting belong outside - Public access policy, TLS termination, and rate limiting belong outside
`distributor` unless a future implementation changes that boundary. `distributor` unless a future implementation changes that boundary.
- `distributor` is not a broad storage synchronization tool unless a future
implementation explicitly changes that non-goal.

View File

@@ -1,67 +0,0 @@
# Reconciliation, Shared-Root, Reconcile-State, And Prune Status
This roadmap file tracks the status and remaining boundaries for the feature
group that includes destination reconciliation, shared-root destination state,
state repair, and managed-output pruning.
Current behavior is documented in:
- `docs/config.md`
- `docs/cli.md`
- `docs/operations.md`
- `docs/troubleshooting.md`
- `docs/integrations/destination-state.md`
- `docs/internal/app.md`
- `docs/internal/publish.md`
- `docs/internal/state.md`
- `docs/internal/storage.md`
## Current Scope
Current scope includes:
- destination `reconciliation.mode: replace|merge`;
- destination `state.mode: single_owner|shared_root`;
- schema version `2` single-owner destination state;
- schema version `3` shared-root destination state;
- owner-scoped shared-root publishing;
- `reconcile-state` for removing missing managed-output records from valid
destination state;
- destination `retention.prune` policy with `older_than` and `keep_latest`;
- `prune --dry-run` and `prune --apply` for configured managed-output pruning.
## Current Boundaries
- Producer manifests and upload requests do not carry destination routing,
reconciliation, state mode, retention, transform, link, or backend policy.
- Reconcile-state repairs state records only. It does not adopt unmanaged
files, validate output digests, repair invalid JSON, or reconcile an entire
config at once.
- Pruning is explicit operator action. It does not run automatically after
publish and does not accept one-off retention overrides.
- Pruning deletes only managed output paths recorded in destination state. It
does not delete unmanaged content or `.distributor.json`.
- Shared-root publishing preserves unrelated owners and rejects implicit
takeover of another owner path.
## Deferred Work
Deferred or future work belongs in the other roadmap files under
`docs/roadmap/`. Current deferred items include:
- unmanaged-file adoption workflow;
- digest-audit mode for state repair;
- whole-config state repair;
- automatic post-publish pruning;
- path/date parsing retention policies;
- producer-facing retention, reconciliation, or destination routing APIs;
- broad storage synchronization behavior.
## Verification
Before changing this feature group, run the focused tests for affected
packages and finish with:
```sh
go test ./...
```

View File

@@ -1,35 +0,0 @@
# Multi-Pipeline Shared Root Roadmap
Current shared-root destination state behavior is documented in:
- `docs/config.md`
- `docs/operations.md`
- `docs/troubleshooting.md`
- `docs/integrations/destination-state.md`
- `docs/internal/publish.md`
- `docs/internal/state.md`
## Current Scope
Current scope includes `state.mode: shared_root` for destinations that
intentionally share one destination root across multiple pipeline/destination
owners.
Shared-root state:
- records owner identity by both `pipeline_id` and `destination_id`;
- records each managed output with its owner and compact source identity;
- preserves unrelated owners during owner-scoped publish, state repair, and
pruning;
- rejects implicit takeover of another owner path;
- treats unmanaged storage content as unmanaged unless explicit forced
replacement is selected where supported.
## Deferred Work
No additional shared-root publishing work is planned here. Related deferred work
is tracked in:
- `docs/roadmap/future.md`
- `docs/roadmap/reconcile-state.md`
- `docs/roadmap/prune.md`

View File

@@ -1,45 +0,0 @@
# Prune Roadmap
Current managed-output pruning behavior is documented in:
- `docs/config.md`
- `docs/cli.md`
- `docs/operations.md`
- `docs/troubleshooting.md`
- `docs/integrations/destination-state.md`
- `docs/internal/app.md`
- `docs/internal/publish.md`
- `docs/internal/state.md`
## Current Scope
Current scope includes:
- destination `retention.prune.enabled`;
- `retention.prune.older_than`;
- `retention.prune.keep_latest`;
- `distributor prune --dry-run`;
- `distributor prune --apply`;
- text and JSON reports;
- owner-scoped pruning for shared-root state.
Pruning uses managed output `updated_at` timestamps from destination state.
When both `keep_latest` and `older_than` are configured, the newest
`keep_latest` outputs are preserved before age-based pruning is applied to the
remaining managed outputs.
Apply mode deletes only planned managed output paths, preserves unmanaged
content, preserves `.distributor.json`, and rewrites state only for confirmed
deletions.
## Deferred Work
Deferred pruning work:
- one-off retention overrides;
- automatic post-publish pruning;
- path/date parsing retention policies;
- group-level pruning by source publication;
- removal of empty state files.
Pruning remains an operator command rather than a general storage cleanup tool.

View File

@@ -1,37 +0,0 @@
# Reconcile-State Roadmap
Current destination state repair behavior is documented in:
- `docs/cli.md`
- `docs/operations.md`
- `docs/troubleshooting.md`
- `docs/integrations/destination-state.md`
- `docs/internal/app.md`
- `docs/internal/state.md`
- `docs/internal/storage.md`
## Current Scope
Current scope includes:
- `distributor reconcile-state --config <path> --pipeline <id>
--destination <id>`;
- read-only inspection with `--dry-run`;
- default apply behavior when `--dry-run` is omitted;
- shared-root owner-scoped repair;
- shared-root all-owner repair with `--all-owners`;
- text and JSON reports.
Reconcile-state removes records for missing managed outputs from valid
destination state. It reports unmanaged entries but does not delete destination
files, adopt unmanaged entries, validate output digests, or repair invalid
state JSON.
## Deferred Work
Deferred state repair work:
- unmanaged-file adoption workflow;
- digest-audit mode;
- explicit invalid-state repair workflow;
- whole-config repair command.

View File

@@ -1,28 +0,0 @@
# Reconciliation Roadmap
Current destination reconciliation behavior is documented in:
- `docs/config.md`
- `docs/operations.md`
- `docs/integrations/destination-state.md`
- `docs/internal/publish.md`
- `docs/internal/state.md`
## Current Scope
Current scope includes destination `reconciliation.mode` values:
- `replace`: the managed output set is reconciled to the newly planned outputs.
- `merge`: omitted managed outputs are retained, and planned paths may overwrite
only paths already recorded as managed.
`replace` is the default. Merge mode does not adopt unmanaged content and does
not let producers choose destination reconciliation policy.
## Deferred Work
No additional reconciliation-specific work is planned here. Related deferred
work is tracked in:
- `docs/roadmap/future.md`
- `docs/roadmap/prune.md`