Consolidate future work into docs/roadmap/future.md and remove completed roadmap docs
All checks were successful
ci/woodpecker/tag/release Pipeline was successful
All checks were successful
ci/woodpecker/tag/release Pipeline was successful
This commit is contained in:
@@ -1,25 +0,0 @@
|
|||||||
# API Roadmap
|
|
||||||
|
|
||||||
This document records API work that is not part of the current implementation. Current HTTP upload behavior is documented in `docs/integrations/http-upload.md`, current configuration behavior is documented in `docs/config.md`, and current producer package usage is documented under `docs/consumers/`.
|
|
||||||
|
|
||||||
## Deferred Upload API Work
|
|
||||||
|
|
||||||
- Durable upload status storage.
|
|
||||||
- Durable idempotency records across server restarts.
|
|
||||||
- Run listing, retry, and cancellation endpoints.
|
|
||||||
- Long-polling or wait-for-completion workflows.
|
|
||||||
- Multipart, resumable, or streaming upload protocols.
|
|
||||||
- Additional archive content negotiation beyond tar and gzip-compressed tar.
|
|
||||||
- URL-token authentication for constrained clients.
|
|
||||||
- Upload token lifecycle tooling.
|
|
||||||
- Mutual TLS or other in-app identity mechanisms.
|
|
||||||
- In-app TLS termination.
|
|
||||||
- In-app public exposure policy.
|
|
||||||
- In-app upload rate limiting.
|
|
||||||
|
|
||||||
## Boundaries
|
|
||||||
|
|
||||||
- Producers do not choose destination ids, destination paths, transforms, links, publish policy, transfer policy, or storage backends through upload requests.
|
|
||||||
- Source manifests remain free of routing, destination, transform, and credential data.
|
|
||||||
- `http_upload` remains source-only unless a future design changes that contract.
|
|
||||||
- Public access policy, TLS termination, and rate limiting belong in deployment infrastructure unless a future design changes that boundary.
|
|
||||||
@@ -33,6 +33,21 @@ internal docs.
|
|||||||
- Resumable upload support.
|
- Resumable upload support.
|
||||||
- Streaming upload protocols.
|
- Streaming upload protocols.
|
||||||
|
|
||||||
|
## Destination State Repair
|
||||||
|
|
||||||
|
- Unmanaged-file adoption workflow for destination state repair.
|
||||||
|
- Digest-audit mode for managed destination outputs.
|
||||||
|
- Explicit invalid-state repair workflow.
|
||||||
|
- Whole-config state repair command.
|
||||||
|
|
||||||
|
## Destination Retention And Pruning
|
||||||
|
|
||||||
|
- One-off retention overrides for `prune`.
|
||||||
|
- Automatic post-publish pruning.
|
||||||
|
- Path/date parsing retention policies.
|
||||||
|
- Group-level pruning by source publication.
|
||||||
|
- Removal of empty state files.
|
||||||
|
|
||||||
## Destination Backends
|
## Destination Backends
|
||||||
|
|
||||||
- GitHub Gist destination backend support.
|
- GitHub Gist destination backend support.
|
||||||
@@ -68,6 +83,7 @@ internal docs.
|
|||||||
- Mutual TLS or other in-app identity mechanisms.
|
- Mutual TLS or other in-app identity mechanisms.
|
||||||
- In-app TLS.
|
- In-app TLS.
|
||||||
- Public exposure defaults.
|
- Public exposure defaults.
|
||||||
|
- In-app upload rate limiting.
|
||||||
- Browser UI.
|
- Browser UI.
|
||||||
|
|
||||||
## Boundaries
|
## Boundaries
|
||||||
@@ -76,5 +92,12 @@ internal docs.
|
|||||||
contract.
|
contract.
|
||||||
- Current upload status, queue, and idempotency state are memory-only.
|
- Current upload status, queue, and idempotency state are memory-only.
|
||||||
- Producers submit complete tar or gzip-compressed tar source bundles today.
|
- Producers submit complete tar or gzip-compressed tar source bundles today.
|
||||||
|
- Producers do not choose destination ids, destination paths, transforms, links,
|
||||||
|
publish policy, transfer policy, storage backends, reconciliation mode, state
|
||||||
|
mode, or retention policy through upload requests.
|
||||||
|
- Source manifests remain free of routing, destination, transform, credential,
|
||||||
|
reconciliation, state, and retention data.
|
||||||
- Public access policy, TLS termination, and rate limiting belong outside
|
- Public access policy, TLS termination, and rate limiting belong outside
|
||||||
`distributor` unless a future implementation changes that boundary.
|
`distributor` unless a future implementation changes that boundary.
|
||||||
|
- `distributor` is not a broad storage synchronization tool unless a future
|
||||||
|
implementation explicitly changes that non-goal.
|
||||||
|
|||||||
@@ -1,67 +0,0 @@
|
|||||||
# Reconciliation, Shared-Root, Reconcile-State, And Prune Status
|
|
||||||
|
|
||||||
This roadmap file tracks the status and remaining boundaries for the feature
|
|
||||||
group that includes destination reconciliation, shared-root destination state,
|
|
||||||
state repair, and managed-output pruning.
|
|
||||||
|
|
||||||
Current behavior is documented in:
|
|
||||||
|
|
||||||
- `docs/config.md`
|
|
||||||
- `docs/cli.md`
|
|
||||||
- `docs/operations.md`
|
|
||||||
- `docs/troubleshooting.md`
|
|
||||||
- `docs/integrations/destination-state.md`
|
|
||||||
- `docs/internal/app.md`
|
|
||||||
- `docs/internal/publish.md`
|
|
||||||
- `docs/internal/state.md`
|
|
||||||
- `docs/internal/storage.md`
|
|
||||||
|
|
||||||
## Current Scope
|
|
||||||
|
|
||||||
Current scope includes:
|
|
||||||
|
|
||||||
- destination `reconciliation.mode: replace|merge`;
|
|
||||||
- destination `state.mode: single_owner|shared_root`;
|
|
||||||
- schema version `2` single-owner destination state;
|
|
||||||
- schema version `3` shared-root destination state;
|
|
||||||
- owner-scoped shared-root publishing;
|
|
||||||
- `reconcile-state` for removing missing managed-output records from valid
|
|
||||||
destination state;
|
|
||||||
- destination `retention.prune` policy with `older_than` and `keep_latest`;
|
|
||||||
- `prune --dry-run` and `prune --apply` for configured managed-output pruning.
|
|
||||||
|
|
||||||
## Current Boundaries
|
|
||||||
|
|
||||||
- Producer manifests and upload requests do not carry destination routing,
|
|
||||||
reconciliation, state mode, retention, transform, link, or backend policy.
|
|
||||||
- Reconcile-state repairs state records only. It does not adopt unmanaged
|
|
||||||
files, validate output digests, repair invalid JSON, or reconcile an entire
|
|
||||||
config at once.
|
|
||||||
- Pruning is explicit operator action. It does not run automatically after
|
|
||||||
publish and does not accept one-off retention overrides.
|
|
||||||
- Pruning deletes only managed output paths recorded in destination state. It
|
|
||||||
does not delete unmanaged content or `.distributor.json`.
|
|
||||||
- Shared-root publishing preserves unrelated owners and rejects implicit
|
|
||||||
takeover of another owner path.
|
|
||||||
|
|
||||||
## Deferred Work
|
|
||||||
|
|
||||||
Deferred or future work belongs in the other roadmap files under
|
|
||||||
`docs/roadmap/`. Current deferred items include:
|
|
||||||
|
|
||||||
- unmanaged-file adoption workflow;
|
|
||||||
- digest-audit mode for state repair;
|
|
||||||
- whole-config state repair;
|
|
||||||
- automatic post-publish pruning;
|
|
||||||
- path/date parsing retention policies;
|
|
||||||
- producer-facing retention, reconciliation, or destination routing APIs;
|
|
||||||
- broad storage synchronization behavior.
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
Before changing this feature group, run the focused tests for affected
|
|
||||||
packages and finish with:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test ./...
|
|
||||||
```
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
# Multi-Pipeline Shared Root Roadmap
|
|
||||||
|
|
||||||
Current shared-root destination state behavior is documented in:
|
|
||||||
|
|
||||||
- `docs/config.md`
|
|
||||||
- `docs/operations.md`
|
|
||||||
- `docs/troubleshooting.md`
|
|
||||||
- `docs/integrations/destination-state.md`
|
|
||||||
- `docs/internal/publish.md`
|
|
||||||
- `docs/internal/state.md`
|
|
||||||
|
|
||||||
## Current Scope
|
|
||||||
|
|
||||||
Current scope includes `state.mode: shared_root` for destinations that
|
|
||||||
intentionally share one destination root across multiple pipeline/destination
|
|
||||||
owners.
|
|
||||||
|
|
||||||
Shared-root state:
|
|
||||||
|
|
||||||
- records owner identity by both `pipeline_id` and `destination_id`;
|
|
||||||
- records each managed output with its owner and compact source identity;
|
|
||||||
- preserves unrelated owners during owner-scoped publish, state repair, and
|
|
||||||
pruning;
|
|
||||||
- rejects implicit takeover of another owner path;
|
|
||||||
- treats unmanaged storage content as unmanaged unless explicit forced
|
|
||||||
replacement is selected where supported.
|
|
||||||
|
|
||||||
## Deferred Work
|
|
||||||
|
|
||||||
No additional shared-root publishing work is planned here. Related deferred work
|
|
||||||
is tracked in:
|
|
||||||
|
|
||||||
- `docs/roadmap/future.md`
|
|
||||||
- `docs/roadmap/reconcile-state.md`
|
|
||||||
- `docs/roadmap/prune.md`
|
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
# Prune Roadmap
|
|
||||||
|
|
||||||
Current managed-output pruning behavior is documented in:
|
|
||||||
|
|
||||||
- `docs/config.md`
|
|
||||||
- `docs/cli.md`
|
|
||||||
- `docs/operations.md`
|
|
||||||
- `docs/troubleshooting.md`
|
|
||||||
- `docs/integrations/destination-state.md`
|
|
||||||
- `docs/internal/app.md`
|
|
||||||
- `docs/internal/publish.md`
|
|
||||||
- `docs/internal/state.md`
|
|
||||||
|
|
||||||
## Current Scope
|
|
||||||
|
|
||||||
Current scope includes:
|
|
||||||
|
|
||||||
- destination `retention.prune.enabled`;
|
|
||||||
- `retention.prune.older_than`;
|
|
||||||
- `retention.prune.keep_latest`;
|
|
||||||
- `distributor prune --dry-run`;
|
|
||||||
- `distributor prune --apply`;
|
|
||||||
- text and JSON reports;
|
|
||||||
- owner-scoped pruning for shared-root state.
|
|
||||||
|
|
||||||
Pruning uses managed output `updated_at` timestamps from destination state.
|
|
||||||
When both `keep_latest` and `older_than` are configured, the newest
|
|
||||||
`keep_latest` outputs are preserved before age-based pruning is applied to the
|
|
||||||
remaining managed outputs.
|
|
||||||
|
|
||||||
Apply mode deletes only planned managed output paths, preserves unmanaged
|
|
||||||
content, preserves `.distributor.json`, and rewrites state only for confirmed
|
|
||||||
deletions.
|
|
||||||
|
|
||||||
## Deferred Work
|
|
||||||
|
|
||||||
Deferred pruning work:
|
|
||||||
|
|
||||||
- one-off retention overrides;
|
|
||||||
- automatic post-publish pruning;
|
|
||||||
- path/date parsing retention policies;
|
|
||||||
- group-level pruning by source publication;
|
|
||||||
- removal of empty state files.
|
|
||||||
|
|
||||||
Pruning remains an operator command rather than a general storage cleanup tool.
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
# Reconcile-State Roadmap
|
|
||||||
|
|
||||||
Current destination state repair behavior is documented in:
|
|
||||||
|
|
||||||
- `docs/cli.md`
|
|
||||||
- `docs/operations.md`
|
|
||||||
- `docs/troubleshooting.md`
|
|
||||||
- `docs/integrations/destination-state.md`
|
|
||||||
- `docs/internal/app.md`
|
|
||||||
- `docs/internal/state.md`
|
|
||||||
- `docs/internal/storage.md`
|
|
||||||
|
|
||||||
## Current Scope
|
|
||||||
|
|
||||||
Current scope includes:
|
|
||||||
|
|
||||||
- `distributor reconcile-state --config <path> --pipeline <id>
|
|
||||||
--destination <id>`;
|
|
||||||
- read-only inspection with `--dry-run`;
|
|
||||||
- default apply behavior when `--dry-run` is omitted;
|
|
||||||
- shared-root owner-scoped repair;
|
|
||||||
- shared-root all-owner repair with `--all-owners`;
|
|
||||||
- text and JSON reports.
|
|
||||||
|
|
||||||
Reconcile-state removes records for missing managed outputs from valid
|
|
||||||
destination state. It reports unmanaged entries but does not delete destination
|
|
||||||
files, adopt unmanaged entries, validate output digests, or repair invalid
|
|
||||||
state JSON.
|
|
||||||
|
|
||||||
## Deferred Work
|
|
||||||
|
|
||||||
Deferred state repair work:
|
|
||||||
|
|
||||||
- unmanaged-file adoption workflow;
|
|
||||||
- digest-audit mode;
|
|
||||||
- explicit invalid-state repair workflow;
|
|
||||||
- whole-config repair command.
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
# Reconciliation Roadmap
|
|
||||||
|
|
||||||
Current destination reconciliation behavior is documented in:
|
|
||||||
|
|
||||||
- `docs/config.md`
|
|
||||||
- `docs/operations.md`
|
|
||||||
- `docs/integrations/destination-state.md`
|
|
||||||
- `docs/internal/publish.md`
|
|
||||||
- `docs/internal/state.md`
|
|
||||||
|
|
||||||
## Current Scope
|
|
||||||
|
|
||||||
Current scope includes destination `reconciliation.mode` values:
|
|
||||||
|
|
||||||
- `replace`: the managed output set is reconciled to the newly planned outputs.
|
|
||||||
- `merge`: omitted managed outputs are retained, and planned paths may overwrite
|
|
||||||
only paths already recorded as managed.
|
|
||||||
|
|
||||||
`replace` is the default. Merge mode does not adopt unmanaged content and does
|
|
||||||
not let producers choose destination reconciliation policy.
|
|
||||||
|
|
||||||
## Deferred Work
|
|
||||||
|
|
||||||
No additional reconciliation-specific work is planned here. Related deferred
|
|
||||||
work is tracked in:
|
|
||||||
|
|
||||||
- `docs/roadmap/future.md`
|
|
||||||
- `docs/roadmap/prune.md`
|
|
||||||
Reference in New Issue
Block a user