Internal: Storage

Purpose

Explain the object-store interface and S3 implementation used by Narratio. Remote key layout and lifecycle belong in Operations, while operator-selected storage fields and credential mechanisms belong in Configuration.

Primary Contract

storage.ObjectStore interface:

List(ctx, prefix)
Read(ctx, key) returns an object body and the generation observed with it
Download(ctx, key, localPath)
Upload(ctx, localPath, key, opts)
UploadConditional(ctx, source, key, opts, condition)
Exists(ctx, key)

Key invariant:

callers pass full bucket-relative keys;
storage implementations do not infer campaign/session/run prefixes.

ReadObjectBounded is the shared mechanism for small control objects. It opens one object version, returns the metadata observed with that body, rejects an oversized known size before transfer, and still performs a context-aware limit-plus-one read. It closes the body on every exit. Callers own the policy limit and add the control-object category to errors; this helper is not used for large artifact payloads.

Composition

NewObjectStoreFromConfig constructs the S3-backed implementation from resolved configuration. The application loads configured filesystem secrets before calling it. The storage adapter consumes already-resolved values; it does not own discovery, defaults, or configuration validation.

S3 Backend Behavior

normalizes object keys.
List paginates and returns normalized ObjectInfo.
A truncated S3 listing must supply a new, non-empty continuation token; otherwise listing fails with bucket and prefix context instead of looping.
Download writes local files with parent directory creation.
Upload streams local file and returns remote metadata.
Read binds a returned body to its S3 ETag. UploadConditional maps an ETag match or absence precondition directly to the provider request and reports a failed precondition without performing a local check-then-write replacement.
Exists maps not-found responses to false.

Invariants

storage layer is stateless regarding manifest/stage progression.
bounded reads never retain more than the caller's limit plus one byte and do not replace owner-specific size policy.
publish ordering semantics are owned by stage/app code, not storage adapters.

Implementation And Tests

Contract and S3 adapter: internal/adapters/storage
Composition: internal/app/object_store.go
Tests: internal/adapters/storage/*_test.go, internal/app/object_store_test.go

2.5 KiB Raw Blame History