Promote current session artifacts to storage

This commit is contained in:
2026-05-16 15:01:01 +00:00
parent d40c91acde
commit f7f8f1a949
7 changed files with 555 additions and 104 deletions

View File

@@ -40,7 +40,6 @@ Implemented:
Still placeholder/future:
- archive-stage promotion behavior
- `notify` stage behavior
- additional Scriptorium artifact types beyond `session_recap`
- artifact-to-artifact workflows beyond the initial single-artifact implementation
@@ -168,7 +167,7 @@ Remote object-store backend scope:
- backend methods do not prepend `root_prefix` or infer campaign/session/run paths
- `prepare` now uses object-store `List` and `Download` for S3 audio input
- `archive` now uses object-store `Upload` for successful run-record upload under the run prefix
- archive promotion behavior using top-level writes and current pointers remains future work
- `archive` now uses object-store `Upload` for promoted outputs and current pointers
Prepare S3 audio behavior (implemented):
@@ -183,7 +182,7 @@ Prepare S3 audio behavior (implemented):
- record S3 provenance in manifest input records (bucket/key/metadata/local paths/checksum)
- no AWS SDK types are used in stage code; storage implementation details stay in storage adapter packages
Archive run upload behavior (implemented):
Archive publishing behavior (implemented):
- `archive` verifies prerequisite stage success before upload:
- `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `analyze`
@@ -196,11 +195,22 @@ Archive run upload behavior (implemented):
- file upload order is deterministic (sorted relative paths)
- `archive.enabled: false` and `archive.upload_run: false` skip upload cleanly
- stage metadata records non-secret upload context:
- `s3_bucket`, `s3_run_prefix`, `files_uploaded`, `uploaded_paths`, `audio_upload_skipped`
- archive does not yet:
- write top-level promoted outputs
- write `current/manifest.json`
- write `current/run_id.txt`
- run upload details, promoted output details, current manifest key, current pointer key
- no secrets, transcript contents, prompt contents, or environment dumps
- promotion rules:
- `from` resolves from local workdir
- `to` resolves under session-level S3 root
- missing required source fails archive
- missing optional source is skipped and recorded
- default promoted outputs:
- `transcripts/trimmed.json`
- `artifacts/session_recap.md`
- current pointers:
- `current/manifest.json` uploaded after run upload and promotions
- `current/run_id.txt` uploaded last with `{run_id}\n`
- `current/run_id.txt` is the effective commit marker
- if promotion or current-manifest upload fails, archive returns failure and does not write `current/run_id.txt`
- failed/incomplete runs remain local and are not uploaded
`pipeline.scriptorium` is optional. Existing pipelines without Scriptorium continue to work.