narratio/docs/archive-storage.md

Archive Storage

This document describes implemented archive-stage publish behavior.

S3 Paths

Session root:

{root_prefix}/campaigns/{campaign}/sessions/{session_id}/

Run prefix:

{root_prefix}/campaigns/{campaign}/sessions/{session_id}/runs/{run_id}/

Scope

Implemented:

• archive uploads successful run records to remote object storage through the storage backend abstraction.
• archive uploads configured promoted outputs to session-level keys.
• archive uploads current/manifest.json.
• archive uploads current/run_id.txt last as the effective commit marker.
• tests use fake storage and do not require live S3.

Future work:

• spool audio cleanup/deletion behavior
• notify stage behavior
• stale detection
• optional future source-audio upload mode
• additional artifact generation beyond current implemented set

Prerequisites

Archive verifies these stages succeeded before upload:

• prepare
• transcribe
• merge
• polish
• normalize
• trim
• analyze

If any prerequisite is missing or not succeeded, archive fails and does not upload. Failed or incomplete runs remain local only.

Run Upload

Archive uploads existing files from the run workdir when present:

• inputs/
• transcripts/
• artifacts/
• reports/ (optional)
• config/
• logs/
• manifest.json

Relative paths are preserved under runs/{run_id}/.

Promotion Rules

Archive applies archive.promote_artifacts in config order.

Rule behavior:

• from: local workdir-relative source path
• to: session-root-relative destination key
• required: true: missing source fails archive
• required: false: missing source is skipped and recorded

Default promoted outputs:

• transcripts/trimmed.json
• artifacts/session_recap.md

Current Pointers

Archive writes:

1. current/manifest.json (after run upload + promotions)
2. current/run_id.txt last

current/run_id.txt contains exactly:

• {run_id} plus trailing newline

Writing current/run_id.txt last makes it the effective commit marker for published session state.

If any required run upload, promotion upload, or current-manifest upload fails, archive returns failure and does not write current/run_id.txt.

Audio Upload Policy

Archive does not upload local audio/ by default. Original audio is expected at the session-level audio prefix and is not duplicated under runs/{run_id}/.

Config Controls

• archive.enabled: false skips archive cleanly.
• archive.upload_run: false skips run upload cleanly.

Metadata

Archive stage metadata includes non-secret upload context (for example):

• s3_bucket
• s3_run_prefix
• run upload counts/paths
• promoted upload counts/paths
• skipped optional promotions
• current_manifest_key
• current_run_id_key
• current_pointer_written
• audio_upload_skipped