# Stage: publish ## Purpose Publish durable run/session state to object storage, then atomically advance remote current state. ## Inputs and Outputs Inputs: - session manifest and prerequisite stage records - run root contents under `runs/{run_id}/` - publish output rules with artifact `source` IDs and publish `dest` paths (`pipeline.publish.outputs`) - effective source-based publish locks from static config and remote session lock store - session-level `previous/**` cache files when present Outputs: - uploaded run files under `{session_prefix}/runs/{run_id}/...` - uploaded published outputs under `{session_prefix}/...` - uploaded session previous-cache files under `{session_prefix}/previous/...` when present - `{session_prefix}/current/manifest.json` - `{session_prefix}/current/run_id.txt` written last ## Boundaries Owns: - publish enable/disable gate behavior - prerequisite stage success enforcement - run file collection and upload (excluding `audio/`) - publish output rule resolution and upload - publish lock enforcement - session previous-cache file collection/upload - commit pointer publish order Does not own: - stage execution before publish - post-publish local cleanup policy execution (handled by app cleanup logic) ## Config Fields Used - `pipeline.publish.enabled` - `pipeline.publish.upload_run` - `pipeline.publish.outputs` - `pipeline.publish.locks` - `{session_prefix}/locks.yml` loaded by app orchestration before publish execution - `pipeline.storage.s3.bucket` - `pipeline.storage.s3.root_prefix` - `pipeline.workspace.root` - `session.campaign` - `session.session_id` ## External Adapters Used - Object storage backend (`env.ObjectStore`) for upload/list primitives. ## State and Manifest Behavior - Requires `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, and `analyze` status `succeeded`. - Resolves bucket/prefix from manifest identity first, then config fallback. - Uploads session `previous/**` files as durable session state when the local `previous/` directory exists. - Skips top-level published output uploads for effective locked sources; run-local materialized outputs remain unchanged. - When selected configured artifact keys are supplied, skips publish rules for unselected `narratio.artifact.` sources; built-in transcript and bounds outputs still publish. - Effective locks are the union of `pipeline.publish.locks` and remote `{session_prefix}/locks.yml`; static pipeline locks win on duplicate sources. - Writes metadata including: - upload counts/paths - `previous_files_uploaded` and `previous_uploaded_paths` - `published_files_uploaded` and `published_paths` - `skipped_optional_outputs` - `skipped_unselected_outputs` - `locked_output_count` and `locked_outputs` - `current_manifest_key` - `current_run_id_key` - `current_pointer_written` - On skipped publish path, returns metadata with `skipped=true` and pointer not written. ## Skip and Resume Behavior - Stage may self-skip (metadata skip) when publish disabled or run upload disabled. - Runner-level skip also applies for previously succeeded stage unless forced. ## Failure Behavior - Fails on missing prerequisite success, missing object store when required, missing run root, missing unlocked required output source, upload failures, or pointer write failures. - Locked required outputs are intentional skips and do not fail publish. - Pointer semantics are fail-safe: `current/run_id.txt` is not written if prior required uploads fail. ## Tests to Inspect Before Changing - `internal/stage/archive_test.go` - `internal/app/post_archive_cleanup_test.go` ## Architectural Invariants - Run upload excludes `audio/` subtree. - Session `previous/**` is publishable durable input/provenance state, not run-local output. - Ordinary `--force` does not override publish locks. - Malformed or unreadable remote lock store fails publish-capable execution before output uploads. - `current/manifest.json` uploads before `current/run_id.txt`. - `current/run_id.txt` is the remote publish commit marker.