Add post-archive cleanup policies
This commit is contained in:
@@ -20,11 +20,13 @@ Implemented:
|
||||
- archive uploads configured promoted outputs to session-level keys.
|
||||
- archive uploads `current/manifest.json`.
|
||||
- archive uploads `current/run_id.txt` last as the effective commit marker.
|
||||
- optional post-archive local cleanup:
|
||||
- `pipeline.spool.delete_audio_after_archive: true` removes only the run-scoped spool audio directory
|
||||
- `pipeline.workspace.cleanup_after_archive: true` removes only the run-scoped local workdir
|
||||
- tests use fake storage and do not require live S3.
|
||||
|
||||
Future work:
|
||||
|
||||
- spool audio cleanup/deletion behavior
|
||||
- `notify` stage behavior
|
||||
- stale detection
|
||||
- optional future source-audio upload mode
|
||||
@@ -89,6 +91,7 @@ Archive writes:
|
||||
Writing `current/run_id.txt` last makes it the effective commit marker for published session state.
|
||||
|
||||
If any required run upload, promotion upload, or current-manifest upload fails, archive returns failure and does not write `current/run_id.txt`.
|
||||
Cleanup runs only after this commit-marker write has succeeded.
|
||||
|
||||
## Audio Upload Policy
|
||||
|
||||
@@ -99,6 +102,7 @@ Original audio is expected at the session-level audio prefix and is not duplicat
|
||||
|
||||
- `archive.enabled: false` skips archive cleanly.
|
||||
- `archive.upload_run: false` skips run upload cleanly.
|
||||
- both skip cases also skip post-archive local cleanup.
|
||||
|
||||
## Metadata
|
||||
|
||||
|
||||
Reference in New Issue
Block a user