Upload successful run records to storage
This commit is contained in:
@@ -40,8 +40,7 @@ Implemented:
|
||||
|
||||
Still placeholder/future:
|
||||
|
||||
- `archive` stage behavior
|
||||
- archive-stage S3 upload/promotion behavior
|
||||
- archive-stage promotion behavior
|
||||
- `notify` stage behavior
|
||||
- additional Scriptorium artifact types beyond `session_recap`
|
||||
- artifact-to-artifact workflows beyond the initial single-artifact implementation
|
||||
@@ -168,7 +167,8 @@ Remote object-store backend scope:
|
||||
- callers pass full bucket-relative object keys
|
||||
- backend methods do not prepend `root_prefix` or infer campaign/session/run paths
|
||||
- `prepare` now uses object-store `List` and `Download` for S3 audio input
|
||||
- archive upload/promotion behavior using `Upload`/`Exists` remains future work
|
||||
- `archive` now uses object-store `Upload` for successful run-record upload under the run prefix
|
||||
- archive promotion behavior using top-level writes and current pointers remains future work
|
||||
|
||||
Prepare S3 audio behavior (implemented):
|
||||
|
||||
@@ -183,6 +183,25 @@ Prepare S3 audio behavior (implemented):
|
||||
- record S3 provenance in manifest input records (bucket/key/metadata/local paths/checksum)
|
||||
- no AWS SDK types are used in stage code; storage implementation details stay in storage adapter packages
|
||||
|
||||
Archive run upload behavior (implemented):
|
||||
|
||||
- `archive` verifies prerequisite stage success before upload:
|
||||
- `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `analyze`
|
||||
- only successful/completed runs are uploaded
|
||||
- uploaded run record destination is:
|
||||
- `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/runs/{run_id}/`
|
||||
- uploaded existing local paths include:
|
||||
- `inputs/`, `transcripts/`, `artifacts/`, optional `reports/`, `config/`, `logs/`, and `manifest.json`
|
||||
- local `audio/` is intentionally excluded from upload by default
|
||||
- file upload order is deterministic (sorted relative paths)
|
||||
- `archive.enabled: false` and `archive.upload_run: false` skip upload cleanly
|
||||
- stage metadata records non-secret upload context:
|
||||
- `s3_bucket`, `s3_run_prefix`, `files_uploaded`, `uploaded_paths`, `audio_upload_skipped`
|
||||
- archive does not yet:
|
||||
- write top-level promoted outputs
|
||||
- write `current/manifest.json`
|
||||
- write `current/run_id.txt`
|
||||
|
||||
`pipeline.scriptorium` is optional. Existing pipelines without Scriptorium continue to work.
|
||||
|
||||
`pipeline.trim` is optional. Existing pipelines without trim config continue to work.
|
||||
|
||||
Reference in New Issue
Block a user