Add post-archive cleanup policies

This commit is contained in:
2026-05-16 23:09:39 +00:00
parent 6fbefb9867
commit d5a9ad38f8
11 changed files with 639 additions and 5 deletions

View File

@@ -20,11 +20,13 @@ Implemented:
- archive uploads configured promoted outputs to session-level keys.
- archive uploads `current/manifest.json`.
- archive uploads `current/run_id.txt` last as the effective commit marker.
- optional post-archive local cleanup:
- `pipeline.spool.delete_audio_after_archive: true` removes only the run-scoped spool audio directory
- `pipeline.workspace.cleanup_after_archive: true` removes only the run-scoped local workdir
- tests use fake storage and do not require live S3.
Future work:
- spool audio cleanup/deletion behavior
- `notify` stage behavior
- stale detection
- optional future source-audio upload mode
@@ -89,6 +91,7 @@ Archive writes:
Writing `current/run_id.txt` last makes it the effective commit marker for published session state.
If any required run upload, promotion upload, or current-manifest upload fails, archive returns failure and does not write `current/run_id.txt`.
Cleanup runs only after this commit-marker write has succeeded.
## Audio Upload Policy
@@ -99,6 +102,7 @@ Original audio is expected at the session-level audio prefix and is not duplicat
- `archive.enabled: false` skips archive cleanly.
- `archive.upload_run: false` skips run upload cleanly.
- both skip cases also skip post-archive local cleanup.
## Metadata