narratio/docs/internal/storage.md

Internal: Storage

Purpose

Document remote object-store contracts and S3 implementation behavior.

Primary Contract

storage.ObjectStore interface:

• List(ctx, prefix)
• Download(ctx, key, localPath)
• Upload(ctx, localPath, key, opts)
• Exists(ctx, key)

Key invariant:

• callers pass full bucket-relative keys;
• storage implementations do not infer campaign/session/run prefixes.

Configuration

NewObjectStoreFromConfig currently supports S3-backed stores from pipeline.storage.* config.

S3 constructor behavior:

• requires configured bucket;
• uses region/endpoint/path-style options when set;
• resolves credentials from configured env var names (with defaults).

S3 Backend Behavior

• normalizes object keys.
• List paginates and returns normalized ObjectInfo.
• Download writes local files with parent directory creation.
• Upload streams local file and returns remote metadata.
• Exists maps not-found responses to false.

Legacy Compatibility Interface

storage.Backend (with ArchiveRequest) remains as compatibility surface with NoopBackend; it is not used by current stage execution.

Invariants

• storage layer is stateless regarding manifest/stage progression.
• publish ordering semantics are owned by stage/app code, not storage adapters.