Files
narratio/docs/troubleshooting.md

5.0 KiB

Troubleshooting

Purpose

Canonical operator troubleshooting guide for recurring Narratio failures.

Config discovery failure

Symptom:

  • command fails because pipeline.yml, campaign.yml, or session.yml was not found.

Likely cause:

  • missing files in discovery paths.
  • missing/incorrect campaign selection.
  • local file exists but was not passed explicitly.

Diagnostics:

ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
ls -l /usr/local/etc/narratio/session.yml /etc/narratio/session.yml

Safe fix:

  • pass explicit --config, --campaign or --campaign-file, and --session.

Templated session file rejected

Symptom:

  • load fails because session.yml must be concrete.

Likely cause:

  • template placeholders ({{ ... }}) still present in loaded session config.

Diagnostics:

narratio session plan 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml

Safe fix:

  • generate concrete session YAML via narratio session init.

Strict decode or validation failure

Symptom:

  • unknown field or invalid value error during config load.

Likely cause:

  • typo, stale field name, or invalid value.

Diagnostics:

narratio session plan 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml

Safe fix:

--artifacts selection failure

Symptom:

  • command fails on unknown/invalid selected artifact key.

Likely cause:

  • artifact key not defined in pipeline.scriptorium.artifacts.
  • empty token in --artifacts input.

Safe fix:

  • use only configured artifact keys.

run-stage --artifacts unsupported stage

Symptom:

  • run-stage rejects --artifacts for the selected stage.

Likely cause:

  • --artifacts used with a stage other than analyze or publish.

Safe fix:

  • use --artifacts only with run-stage analyze ... or run-stage publish ....

Previous-session input unavailable

Symptom:

  • analyze fails on required previous-session artifact input.

Likely cause:

  • previous/** cache not hydrated for this session.

Diagnostics:

narratio session status 2026-04-04

Safe fix:

narratio run-stage prepare 2026-04-04 --force

Or rehydrate from remote current state:

narratio session restore 2026-04-04

Session lock conflict (.lock)

Symptom:

  • command fails with lock conflict.

Likely cause:

  • another process is running for the same session.
  • stale lock file from interrupted command.

Diagnostics:

ls -l {workspace.root}/work/{campaign}/{session_id}/.lock
ps aux | grep narratio

Safe fix:

  • wait for active process; remove stale lock only if no process is active.

Restore current pointer/manifest missing

Symptom:

  • restore fails reading remote current state.

Likely cause:

  • publish commit did not complete.
  • current/run_id.txt or current/manifest.json is missing.

Diagnostics:

narratio session restore 2026-04-04 --dry-run

Safe fix:

  • republish from a healthy local session state.

Restore conflict without --force

Symptom:

  • restore reports conflict and exits.

Likely cause:

  • local durable file differs from remote restore source.

Safe fix:

  • inspect with --dry-run.
  • rerun with --force only when remote should overwrite local.

Secrets or credentials failure

Symptom:

  • startup fails loading secrets dir, or storage/tool auth fails at runtime.

Likely cause:

  • invalid pipeline.secrets.env_dir.
  • missing credential env vars.

Diagnostics:

ls -la /path/to/secrets_dir
env | grep -E 'AUDITA|OBJECT_STORAGE|AWS|SCRIPTORIUM'

Safe fix:

  • fix path/permissions/env vars; keep secret values out of YAML.

S3 audio prepare failure

Symptom:

  • prepare fails in S3 mode (list/download/no files/backend error).

Likely cause:

  • bad session.inputs.audio_s3.prefix.
  • no .flac objects at prefix.
  • bad storage credentials/config.
  • mixed local+S3 audio config.

Diagnostics:

narratio run-stage prepare 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml

Safe fix:

  • configure exactly one audio mode and verify storage access.

Publish output or current-pointer failure

Symptom:

  • publish fails on required output source missing, upload error, or commit-marker write failure.

Likely cause:

  • required source file not produced.
  • storage upload failed before current/run_id.txt write.

Diagnostics:

narratio session status 2026-04-04
narratio run-stage publish 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml

Safe fix:

  • rerun upstream stages to regenerate required outputs.
  • adjust pipeline.publish.outputs source/dest rules.
  • retry after storage issue is fixed.