narratio/docs/troubleshooting.md

Troubleshooting

Purpose

Canonical operator troubleshooting guide for recurring implemented Narratio failures.

Config file discovery failure

Symptom:

• run, plan, resume, or run-stage fails saying config/session file was not found.

Likely Cause:

• pipeline.yml or session.yml is missing from default search paths.
• Wrong working directory when relying on ./session.yml.

Diagnostics:

pwd
ls -l ./session.yml
ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml

Safe Fix:

• Pass explicit paths with --config and --session.
• Or place files in documented discovery paths.

Links:

• docs/config.md
• docs/cli.md

Session template rendering failure

Symptom:

• Load fails with unresolved template placeholder or session_id mismatch.

Likely Cause:

• session.yml contains {{session_id}}/{{ session_id }} but --session-id was omitted.
• Provided --session-id does not match rendered session_id.

Diagnostics:

narratio plan --session ./session.yml --session-id 2026-04-04

Safe Fix:

• Always pass --session-id when using template placeholders.
• Ensure rendered session_id equals intended run session id.

Links:

• docs/config.md

Strict YAML decode or validation failure

Symptom:

• Config load fails with unknown field, missing required field, invalid duration, or invalid cross-field constraint.

Likely Cause:

• YAML key typo or stale field name.
• Required fields missing.
• Invalid value format (for example duration/URL/env var name).

Diagnostics:

narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04

Safe Fix:

• Correct fields/values to match canonical reference and examples.
• Validate against docs/examples/ shapes.

Links:

• docs/config.md
• docs/examples/

Manifest/status path failure

Symptom:

• status fails because manifest path is missing, unreadable, or invalid.

Likely Cause:

• Wrong manifest path.
• Manifest removed after cleanup.
• Trying to run status without --manifest.

Diagnostics:

narratio status --manifest /path/to/manifest.json
ls -l /path/to/manifest.json

Safe Fix:

• Use manifest path printed by run, resume, or run-stage output.
• Re-run with correct session/config if inspecting a different session.

Links:

• docs/cli.md
• docs/operations.md

Session lock conflict (.lock)

Symptom:

• Run fails with lock conflict indicating session workdir is already locked.

Likely Cause:

• Another Narratio process is actively running the same session.
• Prior run exited unexpectedly and left a stale lock file.

Diagnostics:

ls -l {workspace.root}/work/{campaign}/{session_id}/.lock
cat {workspace.root}/work/{campaign}/{session_id}/.lock
ps aux | grep narratio

Safe Fix:

• If another run is active, wait for it to finish.
• If no process is active and lock is stale, remove only that session .lock file and retry.

Links:

• docs/operations.md
• docs/internal/workspace.md

Secrets env-dir or credential env failure

Symptom:

• Startup fails loading secrets directory, or a stage fails because required credential env var is missing.

Likely Cause:

• pipeline.secrets.env_dir path is wrong/unreadable.
• Credential env var referenced in config is unset or empty.

Diagnostics:

ls -la /path/to/secrets_dir
env | grep -E 'AUDITA|OBJECT_STORAGE|AWS|SCRIPTORIUM'

Safe Fix:

• Fix pipeline.secrets.env_dir path/permissions.
• Ensure required env vars are set to non-empty values.
• Keep secrets out of YAML; use env references only.

Links:

• docs/config.md
• docs/operations.md

S3-audio prepare failure

Symptom:

• prepare fails in S3 mode (no audio found, list/download failure, backend missing, path conflict).

Likely Cause:

• Wrong session.inputs.audio_s3.prefix.
• No .flac files at expected prefix.
• Missing or invalid S3 backend credentials/config.
• Conflicting audio-source settings (audio_s3 plus local audio fields).

Diagnostics:

narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 prepare

Safe Fix:

• Ensure audio_s3 is the only audio source configured for that session.
• Confirm .flac objects exist under the resolved session audio prefix.
• Fix S3 storage configuration and credentials.

Links:

• docs/config.md
• docs/operations.md
• docs/internal/stage-prepare.md

Archive prerequisite or promotion/current-pointer failure

Symptom:

• archive fails due to prerequisite stage status, missing required promotion source, or pointer write failure.

Likely Cause:

• One or more prerequisite stages are not succeeded.
• Required promoted artifact does not exist.
• Remote upload failure before current/run_id.txt write.

Diagnostics:

narratio status --manifest /path/to/manifest.json
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 archive

Safe Fix:

• Resume or rerun failed upstream stage(s).
• Ensure required promoted artifact paths exist locally before archive.
• Retry archive after storage/connectivity issue is resolved.

Links:

• docs/operations.md
• docs/config.md
• docs/internal/stage-archive.md

run-stage invalid stage name or invalid flags

Symptom:

• run-stage fails with unknown stage or invalid flag/argument usage.

Likely Cause:

• Stage name typo.
• Missing positional stage argument.
• Unsupported/incorrect flag syntax.

Diagnostics:

narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 normalize

Safe Fix:

• Use only supported stage names.
• Provide exactly one positional stage argument.
• Align flags to documented command reference.

Links:

• docs/cli.md
• docs/operations.md