Add remote storage backend

This commit is contained in:
2026-05-16 14:22:04 +00:00
parent 0454296c81
commit 1e6db89dd4
16 changed files with 1016 additions and 11 deletions

View File

@@ -32,11 +32,15 @@ Implemented:
- S3 session/run/current key builders
- campaign/session/run local work/spool path helpers
- manifest run/path identity fields (`campaign`, `run_id`, local and S3 prefixes)
- remote storage backend layer:
- narrow object-store interface (`List`, `Download`, `Upload`, `Exists`)
- fake storage backend for deterministic tests (no network dependency)
- S3-compatible backend using AWS SDK v2
- config-based object-store construction helper
Still placeholder/future:
- `archive` stage behavior
- real S3 storage backend (list/download/upload/exists)
- prepare-stage S3 audio download behavior
- archive-stage S3 upload/promotion behavior
- `notify` stage behavior
@@ -157,6 +161,15 @@ Cross-config validation scope:
- `pipeline.storage.s3.bucket` is required only when an S3-dependent feature is explicitly configured (for current foundations, that includes `session.inputs.audio_s3`, and archive upload intent when using `storage.backend: s3`)
- no AWS credentials are stored in Narratio config; credential resolution remains an external runtime concern
Remote object-store backend scope:
- remote storage APIs are isolated to `internal/adapters/storage`
- AWS SDK types remain contained within the S3 backend implementation package
- S3 key/session path semantics remain outside the backend, with this invariant:
- callers pass full bucket-relative object keys
- backend methods do not prepend `root_prefix` or infer campaign/session/run paths
- the backend layer is available for future prepare/archive usage, but no stage currently invokes `List`/`Download`/`Upload`/`Exists`
`pipeline.scriptorium` is optional. Existing pipelines without Scriptorium continue to work.
`pipeline.trim` is optional. Existing pipelines without trim config continue to work.