Storage Backends

This document describes the currently implemented remote object storage backend layer used by Narratio, and its intended role in later prepare/archive work.

Implemented

Remote object store abstraction:

List(ctx, prefix)
Download(ctx, key, localPath)
Upload(ctx, localPath, key, opts)
Exists(ctx, key)

Object metadata model includes:

key
size
ETag (provider metadata only)
last modified time when available

Backends:

fake storage backend for deterministic tests
S3-compatible backend implemented with AWS SDK for Go v2

Construction:

config-based constructor builds S3 backend from pipeline.storage.s3 values:
- bucket
- region
- endpoint
- force_path_style

Key Invariant

callers pass full bucket-relative object keys
storage backends do not prepend root_prefix
storage backends do not infer campaign/session/run paths

S3 session/run key builders remain separate and continue to live outside backend implementations.

Security Boundary

do not store AWS credentials in Narratio config
AWS credentials are resolved through standard AWS SDK credential chains
AWS SDK-specific types remain isolated to the storage adapter package

Testing

fake storage tests cover list/download/upload/exists and error paths
S3 backend tests use injected fake S3 API clients
tests do not require live S3 services, AWS credentials, or network access

Not Implemented Yet

prepare-stage S3 object listing or downloads
archive-stage S3 uploads or promotion writes
writing current/manifest.json or current/run_id.txt to S3

1.6 KiB Raw Blame History