59 lines
1.6 KiB
Markdown
59 lines
1.6 KiB
Markdown
# Storage Backends
|
|
|
|
This document describes the currently implemented remote object storage backend layer used by Narratio, and its intended role in later prepare/archive work.
|
|
|
|
## Implemented
|
|
|
|
Remote object store abstraction:
|
|
|
|
- `List(ctx, prefix)`
|
|
- `Download(ctx, key, localPath)`
|
|
- `Upload(ctx, localPath, key, opts)`
|
|
- `Exists(ctx, key)`
|
|
|
|
Object metadata model includes:
|
|
|
|
- key
|
|
- size
|
|
- ETag (provider metadata only)
|
|
- last modified time when available
|
|
|
|
Backends:
|
|
|
|
- fake storage backend for deterministic tests
|
|
- S3-compatible backend implemented with AWS SDK for Go v2
|
|
|
|
Construction:
|
|
|
|
- config-based constructor builds S3 backend from `pipeline.storage.s3` values:
|
|
- bucket
|
|
- region
|
|
- endpoint
|
|
- force_path_style
|
|
|
|
## Key Invariant
|
|
|
|
- callers pass full bucket-relative object keys
|
|
- storage backends do not prepend `root_prefix`
|
|
- storage backends do not infer campaign/session/run paths
|
|
|
|
S3 session/run key builders remain separate and continue to live outside backend implementations.
|
|
|
|
## Security Boundary
|
|
|
|
- do not store AWS credentials in Narratio config
|
|
- AWS credentials are resolved through standard AWS SDK credential chains
|
|
- AWS SDK-specific types remain isolated to the storage adapter package
|
|
|
|
## Testing
|
|
|
|
- fake storage tests cover list/download/upload/exists and error paths
|
|
- S3 backend tests use injected fake S3 API clients
|
|
- tests do not require live S3 services, AWS credentials, or network access
|
|
|
|
## Not Implemented Yet
|
|
|
|
- prepare-stage S3 object listing or downloads
|
|
- archive-stage S3 uploads or promotion writes
|
|
- writing `current/manifest.json` or `current/run_id.txt` to S3
|