narratio/docs/troubleshooting.md

Troubleshooting

Purpose

Canonical operator troubleshooting guide for recurring implemented Narratio failures.

Config file discovery failure

Symptom:

• run, resume, run-stage, session plan, or session restore fails with config/session not found.

Likely Cause:

• pipeline.yml or session.yml is missing from system discovery paths.
• the selected campaign ID does not exist under pipeline.campaigns.root.
• a local working-directory config file was not passed explicitly.

Diagnostics:

ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
ls -l /usr/local/etc/narratio/session.yml /etc/narratio/session.yml

Safe Fix:

• pass explicit --config, --campaign <id>, --campaign-file <path>, and --session as appropriate.
• or place files in documented discovery paths and set pipeline.campaigns.default_campaign_id.

Links:

• docs/config.md
• docs/cli.md

Templated session file rejected

Symptom:

• load fails with a message that session.yml must be concrete.

Likely Cause:

• a template authoring file such as session.template.yml was passed to --session or uploaded as remote session.yml.
• session.yml still contains {{ ... }} placeholders.

Diagnostics:

narratio session plan 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session ./session.yml

Safe Fix:

• generate concrete YAML with narratio session init.
• pass the generated concrete session.yml to downstream commands or upload it through session init --remote.

Links:

• docs/config.md

Strict YAML decode or validation failure

Symptom:

• config load fails with unknown field or validation error.

Likely Cause:

• typo/stale field name.
• missing required fields or invalid constraints.

Diagnostics:

narratio session plan 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml

Safe Fix:

• align fields/values to canonical config reference and examples.

Links:

• docs/config.md
• examples/

--artifacts selection failure

Symptom:

• run/resume/run-stage fails with invalid or unknown artifact selection.

Likely Cause:

• --artifacts contains blank names or unknown artifact keys.
• pipeline.scriptorium.artifacts missing while using --artifacts.

Diagnostics:

narratio run 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --artifacts player_handout

Safe Fix:

• use configured artifact keys only.
• ensure pipeline.scriptorium.artifacts is defined.

Links:

• docs/cli.md
• docs/config.md

run-stage --artifacts on unsupported stage

Symptom:

• run-stage fails because --artifacts is only supported for analyze and archive.

Likely Cause:

• --artifacts was used with a stage other than analyze or archive.

Diagnostics:

narratio run-stage polish 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --artifacts session_recap

Safe Fix:

• use --artifacts only with run-stage analyze ... or run-stage archive ....

Links:

• docs/cli.md

Configured artifact dependency/input validation failure

Symptom:

• config validation fails for depends_on, narratio.artifact.<name> source, or artifact output path.

Likely Cause:

• narratio.artifact.<name> source missing matching depends_on key.
• dependency references unknown artifact key.
• dependency self-reference or enabled dependency cycle.
• artifact output path missing/invalid/outside artifacts/ root.

Diagnostics:

narratio session plan 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml

Safe Fix:

• ensure artifact-to-artifact inputs have explicit depends_on entries using artifact keys.
• ensure referenced artifacts exist and define valid output_path values.
• keep output paths relative and under artifacts/.

Links:

• docs/config.md
• docs/internal/stage-analyze.md

Required configured artifact input unavailable at analyze time

Symptom:

• analyze fails because configured input source is unavailable.

Likely Cause:

• required upstream configured artifact was not selected/executed this run.
• non-executable dependency output file is missing or invalid on disk.

Diagnostics:

narratio session status 2026-04-04
narratio run-stage analyze 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --artifacts player_handout

Safe Fix:

• run analyze with needed artifacts selected.
• or ensure dependency output file exists at configured path and is valid.

Links:

• docs/operations.md
• docs/config.md

Manifest/status path failure

Symptom:

• session status fails because config/session state is missing, unreadable, or invalid.

Likely Cause:

• wrong session ID.
• wrong config/campaign/session file selected.
• manifest removed after cleanup.

Diagnostics:

narratio session status 2026-04-04

Safe Fix:

• use the same session ID and config files that will be used for run, resume, or run-stage.

Links:

• docs/cli.md
• docs/operations.md

Session lock conflict (.lock)

Symptom:

• run, resume, run-stage, or session restore fails with lock conflict for session workdir.

Likely Cause:

• another Narratio process is running same session.
• stale lock from interrupted prior run.

Diagnostics:

ls -l {workspace.root}/work/{campaign}/{session_id}/.lock
cat {workspace.root}/work/{campaign}/{session_id}/.lock
ps aux | grep narratio

Safe Fix:

• wait for active process to finish.
• if no process is active, remove only stale session .lock file.

Links:

• docs/operations.md
• docs/internal/workspace.md

Restore remote current pointer or manifest missing

Symptom:

• session restore fails with remote current pointer or current manifest errors.

Likely Cause:

• current/run_id.txt was never published.
• current/manifest.json is missing for the session prefix.
• archive commit did not complete.

Diagnostics:

narratio session restore 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --dry-run

Safe Fix:

• verify archive stage succeeded for the target session.
• rerun/archive from a healthy source workspace so current pointers are published.

Links:

• docs/operations.md
• docs/internal/stage-archive.md

Restore manifest identity mismatch

Symptom:

• session restore fails because remote manifest session or campaign does not match requested values.

Likely Cause:

• wrong positional session ID or wrong session config selected.
• archive prefix points to a different campaign/session.

Diagnostics:

narratio session restore 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --dry-run

Safe Fix:

• use the correct session config and positional session ID.
• verify campaign/session identity in local config before restore.

Links:

• docs/config.md
• docs/operations.md

Restore conflict without --force

Symptom:

• session restore fails with restore conflict and conflict counts.

Likely Cause:

• local durable file differs from remote file for one or more planned restore paths.

Diagnostics:

narratio session restore 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml --dry-run

Safe Fix:

• review planned conflicts.
• rerun with --force only when remote state should overwrite local state.

Links:

• docs/cli.md
• docs/operations.md

Restore report expectations

Symptom:

• operator expects restore report file but does not find one.

Likely Cause:

• restore was executed in --dry-run mode.
• restore failed before report persistence path (for example lock acquisition failure).

Diagnostics:

ls -l {workspace.root}/work/{campaign}/{session_id}/reports/restore-latest.json

Safe Fix:

• run non-dry-run restore for durable report output.
• resolve lock or early preflight failures and retry.

Links:

• docs/operations.md

Secrets env-dir or credential-env failure

Symptom:

• startup fails loading secrets directory, or stage fails due to missing credential env vars.

Likely Cause:

• invalid pipeline.secrets.env_dir path/permissions.
• required credential env var unset/empty.

Diagnostics:

ls -la /path/to/secrets_dir
env | grep -E 'AUDITA|OBJECT_STORAGE|AWS|SCRIPTORIUM'

Safe Fix:

• fix secrets directory and credential env vars.
• keep secret values out of YAML.

Links:

• docs/config.md

S3-audio prepare failure

Symptom:

• prepare fails in S3 mode (listing/downloading/no audio/backend error).

Likely Cause:

• wrong session.inputs.audio_s3.prefix.
• no .flac files at resolved prefix.
• invalid/missing object-store credentials or backend config.
• mixed local+S3 audio input config.

Diagnostics:

narratio run-stage prepare 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml

Safe Fix:

• configure exactly one audio source mode.
• verify .flac files and storage access.

Links:

• docs/config.md
• docs/operations.md

Archive promotion/current-pointer failure

Symptom:

• archive fails on required promotion source missing or pointer write failure.

Likely Cause:

• required promoted file absent (including analyze outputs not generated for this run).
• storage upload failed before current/run_id.txt commit marker write.

Diagnostics:

narratio session status 2026-04-04
narratio run-stage archive 2026-04-04 --config /path/to/pipeline.yml --campaign-file /path/to/campaign.yml --session /path/to/session.yml

Safe Fix:

• rerun or resume upstream stages to generate required files.
• adjust promotion source/dest rules to match artifacts that must exist.
• retry after storage issue is resolved.

Links:

• docs/operations.md
• docs/config.md
• docs/internal/stage-archive.md