narratio/docs/operations.md

Operations

This guide describes the implemented operator lifecycle for Narratio.

For field-level configuration, see docs/config.md. For full command/flag reference, see docs/cli.md.

Normal workflow (S3-first path)

1. Create or upload session.yml, or pass a local session.yml explicitly.
2. Upload session .flac files to object storage under the configured session audio prefix.
3. Run Narratio:

narratio run --session-id 2026-04-04

4. Read success output:

• narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>
• use manifest=<path> with status for inspection.

Notes:

• default config/campaign/session discovery checks system config locations unless --config, --campaign, and --session are passed.
• when local session.yml discovery misses, --session-id loads remote session.yml from {root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml.
• S3 audio mode requires session.inputs.audio_s3.prefix and valid object-store access.

Initialize a remote session skeleton:

narratio session init --config /etc/narratio/pipeline.yml --campaign /etc/narratio/campaign.yml --session-id 2026-04-04 --remote

Remote init writes {root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml. It fails if the object already exists unless --force is passed.

Validate before running:

narratio session validate --session-id 2026-04-04

Restore workflow

Use restore when local durable session state is missing or stale and archive current state is authoritative.

Dry-run (no local writes):

narratio restore --session-id 2026-04-04 --dry-run

Execution:

narratio restore --session-id 2026-04-04

Post-restore analyze rerun pattern:

narratio run-stage --session-id 2026-04-04 --force analyze

Restore source-of-truth:

• remote commit marker: current/run_id.txt
• remote current manifest: current/manifest.json

Restore default scope:

• includes manifest.json, transcripts/**, artifacts/**, previous/**
• includes audio/** only with --include-audio
• excludes runs/**, logs/**, reports/**, config/**, inputs/**, and current/** (except remote current/manifest.json as source)

Local filesystem layout and state artifacts

Session root:

• {workspace.root}/work/{campaign}/{session_id}/

Primary state:

• manifest.json: session-level stage state.
• runs/{run_id}/manifest.json: invocation-level state.
• .lock: session lock while a modifying command is active.
• inputs/campaign.yml, inputs/session.yml, and inputs/pipeline.resolved.yml: materialized config inputs for the run.

Canonical session directories:

• inputs/
• audio/
• transcripts/
• artifacts/
• previous/
• reports/
• logs/
• config/
• current/
• runs/

Run-local stage directories:

• runs/{run_id}/{stage}/ with stage-local outputs/, logs/, reports/, config/, scratch/.

Behavior:

• directory creation is idempotent.
• stage outputs are generally generated run-local first, then promoted to canonical paths on success.
• restore installs downloaded files to canonical session paths and does not recreate historical run sandboxes.

Analyze artifact execution lifecycle

Analyze executes configured artifacts from pipeline.scriptorium.artifacts.

Execution model:

• executable set = enabled artifacts, filtered by --artifacts when provided.
• artifact-to-artifact dependencies are declared via depends_on.
• selected artifacts run in deterministic dependency order.
• after each successful artifact run, output is promoted to configured canonical output_path.

Configured artifact source reuse:

• a non-executable configured artifact can satisfy inputs if its configured output file already exists and is valid.
• reused configured artifact provenance is filesystem.disabled_artifact_output.

--artifacts behavior:

• accepted on run, resume, and run-stage analyze.
• filters analyze execution only; does not force stage rerun.

Canonical previous-session input behavior:

• canonical sources use narratio.previous_session.artifact.<artifact_key>.
• these inputs are hydrated by prepare, not analyze.
• if analyze fails due to missing canonical previous cache, rerun:
  • narratio run-stage --session-id <id> --force prepare

Remote archive layout and publish contract

When archive is enabled and run upload is enabled, archive publishes under:

• session prefix: {root_prefix}/campaigns/{campaign}/sessions/{session_id}/
• run prefix: {session_prefix}/runs/{run_id}/

Archive uploads:

• run record files from run root (excluding audio/).
• promoted files from explicit archive.promote_artifacts rules.
• mutable session locks from helper commands live at {session_prefix}/locks.yml.

Publish order:

1. upload current/manifest.json
2. upload current/run_id.txt last

current/run_id.txt is the remote commit marker.

Archive promotion is explicit and source-based:

• Narratio does not auto-promote all generated analyze artifacts.
• each rule resolves source through the artifact resolver/catalog model, then uploads to dest.
• missing required promotion sources fail archive stage.
• missing optional promotion sources are skipped.
• invalid resolved artifacts fail archive stage.
• archive.locks skips top-level promotion overwrites for locked sources while run-local uploads still publish.
• remote locks from {session_prefix}/locks.yml are merged with static archive.locks; static locks win on duplicate sources.
• locked required promotions are treated as intentional successful skips and are recorded in archive metadata.

Lock helper behavior:

• narratio locks --session-id <id> lists effective static and remote locks.
• narratio locks add --session-id <id> --reason <text> <source> writes a remote lock.
• narratio locks add --session-id <id> --force --reason <text> <source> updates an existing remote lock reason.
• narratio locks remove --session-id <id> <source> removes only a remote lock.
• locks remove cannot remove static pipeline locks.
• remote lock writes check whether the lock store exists, but are not compare-and-swap atomic.

Resume, retry, restore, and safe rerun behavior

Default skip:

• run and run-stage skip already-succeeded stages unless --force is set.

Resume:

• resume starts at first non-succeeded stage.
• resume --force runs full stage order.

Restore conflict policy:

• restore classifies local differences as conflicts.
• without --force, restore fails when conflicts exist.
• with --force, conflicting local files are overwritten by remote archive files.

Forced reruns:

• force-rerunning an upstream succeeded stage marks downstream succeeded stages as stale.
• ordinary --force does not override archive locks.

Safe rerun pattern:

1. rerun the changed stage with --force.
2. run resume to rebuild downstream stages.

Cleanup behavior

Cleanup is considered only when archive stage executed and succeeded.

Cleanup toggles:

• pipeline.spool.delete_audio_after_archive=true deletes run-scoped spool audio.
• pipeline.workspace.cleanup_after_archive=true deletes run-scoped local run directory.

The S3 audio cache under pipeline.cache.root is durable input cache state, not workspace or spool state. Cleanup does not delete it.

Cleanup eligibility gates:

• archive enabled
• archive run upload enabled
• run record upload completed
• current pointer write completed (current/run_id.txt written)

No cleanup for failed/incomplete/unarchived/archive-skipped runs.

Failure and recovery playbooks

After run failure, Narratio keeps:

• session manifest
• run manifest
• run-local artifacts/logs/config/reports

Failed or incomplete runs remain local-only.

After restore failure:

• already-installed restore files remain in place.
• restore does not roll back prior successful installs.
• existing local manifest is preserved if restored manifest validation/install fails.

Recommended recovery:

1. inspect state:

narratio status --session-id 2026-04-04

This reports local manifest state, committed remote current state, expected remote transcript/artifact availability, and archive locks.

2. for one manifest file, run:

narratio status --manifest <manifest-path>

3. for restore-specific checks, run:

narratio restore --session-id 2026-04-04 --dry-run

4. fix root cause (config/input/credentials/storage/service availability).
5. continue with resume, or targeted run-stage --force followed by resume.

Restore report

Non-dry-run restore writes a durable report at:

• reports/restore-latest.json

Report content includes:

• identity (campaign, session_id, run_id)
• mode flags (dry_run, force, include_audio)
• plan counts and execution counts
• per-action status

Dry-run does not write restore report files.

Operational caveats

• status with no config/session flags still requires explicit --manifest.
• status --session-id <id> uses normal config/session loading, including remote session fallback.
• status --session-id <id> includes the same promoted remote output availability view as artifacts list --remote when storage is configured.
• local and S3 audio input modes are mutually exclusive.
• archive publish requires upstream stages through analyze to be succeeded.
• required promotion rules can fail when selected analyze artifacts did not generate a required file path.
• restore requires configured remote object storage and committed remote current state.