eric/distributor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Expose managed output pruning

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-08 19:38:49 +00:00
parent 6daddad543
commit ea562c1c3a
12 changed files with 481 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/internal/app.md
View File

@@ -16,7 +16,7 @@ Outputs include `RunReport`, `ReconcileStateReport`, `PrunePlanReport`, `PruneRe
`internal/app` wires packages together but does not own manifest validation rules, destination state comparison, storage path rules, publish safety policy, transform rendering, config schema validation, or backend protocol behavior.
User-facing command parsing stays in `internal/cli`, including `reconcile-state` flag validation and help text. User-facing config reference stays in `docs/config.md`. External contracts live under `docs/integrations/`.
User-facing command parsing stays in `internal/cli`, including `reconcile-state` and `prune` flag validation and help text. User-facing config reference stays in `docs/config.md`. External contracts live under `docs/integrations/`.
## Config Fields Used
@@ -38,7 +38,7 @@ Reconcile-state workflows load one configured pipeline/destination selector, ope
Prune planning consumes a parsed destination state document and a validated retention prune policy, then returns owner-scoped managed output records that would be pruned or preserved. Planning uses output `updated_at` timestamps, applies `keep_latest` before `older_than` when both are configured, and does not open storage, delete files, or rewrite state.
Prune execution loads one configured pipeline/destination selector, opens that destination root, parses the root `.distributor.json`, and builds a plan from the destination retention policy. Dry-run returns the same planned and preserved managed output records without deleting files or rewriting state. Apply mode deletes only planned managed output paths, never unmanaged files or `.distributor.json`, then removes confirmed deleted records from state and updates the state timestamp. If a delete fails after earlier deletes succeeded, it rewrites state only for the confirmed deletions and preserves records for the failed and unattempted outputs so a retry remains accurate.
Prune execution loads one configured pipeline/destination selector, opens that destination root, parses the root `.distributor.json`, and builds a plan from the destination retention policy. Dry-run returns the same planned and preserved managed output records without deleting files or rewriting state. Apply mode deletes only planned managed output paths, never unmanaged files or `.distributor.json`, then removes confirmed deleted records from state and updates the state timestamp. If a delete fails after earlier deletes succeeded, it rewrites state only for the confirmed deletions and preserves records for the failed and unattempted outputs so a retry remains accurate. Text output reports `changed`, `would_change`, or `unchanged`; JSON output uses the shared app envelope.
HTTP uploads stage and validate archives before enqueueing a pipeline run with a local staged source root. Go producers can use the public `pkg/upload` package to create client-side gzip tar uploads for this server contract; `internal/app` remains the server-side orchestration boundary and does not import that producer package.
@@ -65,6 +65,7 @@ HTTP upload startup fails if upload tokens are missing, empty, or duplicated. Up
- `internal/app/*_test.go`
- `internal/app/prune_test.go`
- `internal/cli/reconcile_state_test.go`
- `internal/cli/prune_test.go`
- `internal/cli/root_test.go`
- `internal/config/*_test.go`
- `internal/ingest/*_test.go`