Add remote storage backend
This commit is contained in:
@@ -32,11 +32,15 @@ Implemented:
|
||||
- S3 session/run/current key builders
|
||||
- campaign/session/run local work/spool path helpers
|
||||
- manifest run/path identity fields (`campaign`, `run_id`, local and S3 prefixes)
|
||||
- remote storage backend layer:
|
||||
- narrow object-store interface (`List`, `Download`, `Upload`, `Exists`)
|
||||
- fake storage backend for deterministic tests (no network dependency)
|
||||
- S3-compatible backend using AWS SDK v2
|
||||
- config-based object-store construction helper
|
||||
|
||||
Still placeholder/future:
|
||||
|
||||
- `archive` stage behavior
|
||||
- real S3 storage backend (list/download/upload/exists)
|
||||
- prepare-stage S3 audio download behavior
|
||||
- archive-stage S3 upload/promotion behavior
|
||||
- `notify` stage behavior
|
||||
@@ -157,6 +161,15 @@ Cross-config validation scope:
|
||||
- `pipeline.storage.s3.bucket` is required only when an S3-dependent feature is explicitly configured (for current foundations, that includes `session.inputs.audio_s3`, and archive upload intent when using `storage.backend: s3`)
|
||||
- no AWS credentials are stored in Narratio config; credential resolution remains an external runtime concern
|
||||
|
||||
Remote object-store backend scope:
|
||||
|
||||
- remote storage APIs are isolated to `internal/adapters/storage`
|
||||
- AWS SDK types remain contained within the S3 backend implementation package
|
||||
- S3 key/session path semantics remain outside the backend, with this invariant:
|
||||
- callers pass full bucket-relative object keys
|
||||
- backend methods do not prepend `root_prefix` or infer campaign/session/run paths
|
||||
- the backend layer is available for future prepare/archive usage, but no stage currently invokes `List`/`Download`/`Upload`/`Exists`
|
||||
|
||||
`pipeline.scriptorium` is optional. Existing pipelines without Scriptorium continue to work.
|
||||
|
||||
`pipeline.trim` is optional. Existing pipelines without trim config continue to work.
|
||||
|
||||
Reference in New Issue
Block a user