Upload successful run records to storage
This commit is contained in:
73
docs/archive-storage.md
Normal file
73
docs/archive-storage.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# Archive Storage
|
||||
|
||||
This document describes implemented archive-stage storage behavior.
|
||||
|
||||
## Scope
|
||||
|
||||
Implemented:
|
||||
|
||||
- archive uploads successful run records to remote object storage through the storage backend abstraction.
|
||||
- upload destination is the run prefix:
|
||||
- `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/runs/{run_id}/`
|
||||
- tests use fake storage and do not require live S3.
|
||||
|
||||
Not implemented:
|
||||
|
||||
- promotion uploads to top-level `transcripts/` or `artifacts/`
|
||||
- `current/manifest.json` write
|
||||
- `current/run_id.txt` write
|
||||
- audio re-upload from local workdir
|
||||
- failed/incomplete run uploads
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Archive verifies the following stages succeeded before upload:
|
||||
|
||||
- `prepare`
|
||||
- `transcribe`
|
||||
- `merge`
|
||||
- `polish`
|
||||
- `normalize`
|
||||
- `trim`
|
||||
- `analyze`
|
||||
|
||||
If any prerequisite is missing or not succeeded, archive fails and does not upload.
|
||||
|
||||
## Uploaded Run Record
|
||||
|
||||
Archive uploads existing files from the run workdir when present:
|
||||
|
||||
- `inputs/`
|
||||
- `transcripts/`
|
||||
- `artifacts/`
|
||||
- `reports/` (optional)
|
||||
- `config/`
|
||||
- `logs/`
|
||||
- `manifest.json`
|
||||
|
||||
Archive preserves relative file paths under `runs/{run_id}/`.
|
||||
|
||||
Example:
|
||||
|
||||
- local `transcripts/raw/speaker.json`
|
||||
- remote `runs/{run_id}/transcripts/raw/speaker.json`
|
||||
|
||||
## Audio Upload Policy
|
||||
|
||||
Archive does not upload local `audio/` by default.
|
||||
Original audio is expected at the session-level audio prefix and is not duplicated under `runs/{run_id}/`.
|
||||
|
||||
## Config Controls
|
||||
|
||||
- `archive.enabled: false` skips archive cleanly.
|
||||
- `archive.upload_run: false` skips run upload cleanly.
|
||||
|
||||
## Metadata
|
||||
|
||||
Archive stage metadata includes non-secret upload context:
|
||||
|
||||
- `s3_bucket`
|
||||
- `s3_run_prefix`
|
||||
- `files_uploaded`
|
||||
- `uploaded_paths`
|
||||
- `audio_upload_skipped`
|
||||
Reference in New Issue
Block a user