narratio/docs/internal/stage-archive.md

Stage: archive

Purpose

Publish run records and promoted session artifacts to object storage, then atomically advance the remote current pointer.

Inputs and Outputs

Inputs:

• session manifest and prerequisite stage records
• run root contents under runs/{run_id}/
• promotion sources from session root (archive.promote_artifacts)

Outputs:

• uploaded run files under {session_prefix}/runs/{run_id}/...
• uploaded promoted artifacts under {session_prefix}/...
• {session_prefix}/current/manifest.json
• {session_prefix}/current/run_id.txt written last

Boundaries

Owns:

• Archive enable/disable gate behavior
• Prerequisite stage success enforcement
• Run file collection and upload (excluding audio/)
• Promotion rule resolution and upload
• Commit pointer publish order

Does not own:

• Stage execution before archive
• Post-archive local cleanup policy execution (handled by app cleanup logic)

Config Fields Used

• pipeline.archive.enabled
• pipeline.archive.upload_run
• pipeline.archive.promote_artifacts
• pipeline.storage.s3.bucket
• pipeline.storage.s3.root_prefix
• pipeline.workspace.root
• session.campaign
• session.session_id

External Adapters Used

• Object storage backend (env.ObjectStore) for upload/list primitives.

State and Manifest Behavior

• Requires prepare, transcribe, merge, polish, normalize, trim, and analyze status succeeded.
• Resolves bucket/prefix from manifest identity first, then config fallback.
• Writes metadata including:
  • upload counts/paths
  • current_manifest_key
  • current_run_id_key
  • current_pointer_written
• On skipped archive path, returns metadata with skipped=true and pointer not written.

Skip and Resume Behavior

• Stage may self-skip (metadata skip) when archive disabled or run upload disabled.
• Runner-level skip also applies for previously succeeded stage unless forced.

Failure Behavior

• Fails on missing prerequisite success, missing object store when required, missing run root, missing required promotion source, upload failures, or pointer write failures.
• Pointer semantics are fail-safe: current/run_id.txt is not written if prior required uploads fail.

Tests to Inspect Before Changing

• internal/stage/archive_test.go
• internal/app/post_archive_cleanup_test.go

Architectural Invariants

• Run upload excludes audio/ subtree.
• current/manifest.json uploads before current/run_id.txt.
• current/run_id.txt is the remote publish commit marker.