narratio/docs/internal/stage-archive.md

Stage: archive

Purpose

Publish durable run/session state to object storage, then atomically advance remote current state.

Inputs and Outputs

Inputs:

• session manifest and prerequisite stage records
• run root contents under runs/{run_id}/
• promotion rules with artifact source IDs and archive dest paths (archive.promote_artifacts)
• effective source-based promotion locks from static config and remote session lock store
• session-level previous/** cache files when present

Outputs:

• uploaded run files under {session_prefix}/runs/{run_id}/...
• uploaded promoted artifacts under {session_prefix}/...
• uploaded session previous-cache files under {session_prefix}/previous/... when present
• {session_prefix}/current/manifest.json
• {session_prefix}/current/run_id.txt written last

Boundaries

Owns:

• Archive enable/disable gate behavior
• Prerequisite stage success enforcement
• Run file collection and upload (excluding audio/)
• Promotion rule resolution and upload
• Promotion lock enforcement
• Session previous-cache file collection/upload
• Commit pointer publish order

Does not own:

• Stage execution before archive
• Post-archive local cleanup policy execution (handled by app cleanup logic)

Config Fields Used

• pipeline.archive.enabled
• pipeline.archive.upload_run
• pipeline.archive.promote_artifacts
• pipeline.archive.locks
• {session_prefix}/locks.yml loaded by app orchestration before archive execution
• pipeline.storage.s3.bucket
• pipeline.storage.s3.root_prefix
• pipeline.workspace.root
• session.campaign
• session.session_id

External Adapters Used

• Object storage backend (env.ObjectStore) for upload/list primitives.

State and Manifest Behavior

• Requires prepare, transcribe, merge, polish, normalize, trim, and analyze status succeeded.
• Resolves bucket/prefix from manifest identity first, then config fallback.
• Uploads session previous/** files as durable session state when the local previous/ directory exists.
• Skips top-level promotion uploads for effective locked sources; run-local uploads still publish.
• When selected configured artifact keys are supplied, skips promotion rules for unselected narratio.artifact.<key> sources; built-in transcript and bounds promotions still publish.
• Effective locks are the union of pipeline.archive.locks and remote {session_prefix}/locks.yml; static pipeline locks win on duplicate sources.
• Writes metadata including:
  • upload counts/paths
  • previous_files_uploaded and previous_uploaded_paths
  • skipped_unselected_promotions
  • locked_promotion_count and locked_promotions
  • current_manifest_key
  • current_run_id_key
  • current_pointer_written
• On skipped archive path, returns metadata with skipped=true and pointer not written.

Skip and Resume Behavior

• Stage may self-skip (metadata skip) when archive disabled or run upload disabled.
• Runner-level skip also applies for previously succeeded stage unless forced.

Failure Behavior

• Fails on missing prerequisite success, missing object store when required, missing run root, missing unlocked required promotion source, upload failures, or pointer write failures.
• Locked required promotions are intentional skips and do not fail archive.
• Pointer semantics are fail-safe: current/run_id.txt is not written if prior required uploads fail.

Tests to Inspect Before Changing

• internal/stage/archive_test.go
• internal/app/post_archive_cleanup_test.go

Architectural Invariants

• Run upload excludes audio/ subtree.
• Session previous/** is archiveable durable input/provenance state, not run-local output.
• Ordinary --force does not override archive locks.
• Malformed or unreadable remote lock store fails archive-capable execution before promotion.
• current/manifest.json uploads before current/run_id.txt.
• current/run_id.txt is the remote publish commit marker.