224 lines
5.0 KiB
Markdown
224 lines
5.0 KiB
Markdown
# Troubleshooting
|
|
|
|
## Purpose
|
|
Canonical operator troubleshooting guide for recurring Narratio failures.
|
|
|
|
## Config discovery failure
|
|
|
|
Symptom:
|
|
- command fails because `pipeline.yml`, `campaign.yml`, or `session.yml` was not found.
|
|
|
|
Likely cause:
|
|
- missing files in discovery paths.
|
|
- missing/incorrect campaign selection.
|
|
- local file exists but was not passed explicitly.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
|
|
ls -l /usr/local/etc/narratio/session.yml /etc/narratio/session.yml
|
|
```
|
|
|
|
Safe fix:
|
|
- pass explicit `--config`, `--campaign` or `--campaign-file`, and `--session`.
|
|
|
|
## Templated session file rejected
|
|
|
|
Symptom:
|
|
- load fails because `session.yml` must be concrete.
|
|
|
|
Likely cause:
|
|
- template placeholders (`{{ ... }}`) still present in loaded session config.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session plan 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
|
```
|
|
|
|
Safe fix:
|
|
- generate concrete session YAML via `narratio session init`.
|
|
|
|
## Strict decode or validation failure
|
|
|
|
Symptom:
|
|
- unknown field or invalid value error during config load.
|
|
|
|
Likely cause:
|
|
- typo, stale field name, or invalid value.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session plan 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
|
```
|
|
|
|
Safe fix:
|
|
- align config with [docs/config.md](./config.md) and maintained examples.
|
|
|
|
## `--artifacts` selection failure
|
|
|
|
Symptom:
|
|
- command fails on unknown/invalid selected artifact key.
|
|
|
|
Likely cause:
|
|
- artifact key not defined in `pipeline.scriptorium.artifacts`.
|
|
- empty token in `--artifacts` input.
|
|
|
|
Safe fix:
|
|
- use only configured artifact keys.
|
|
|
|
## `run-stage --artifacts` unsupported stage
|
|
|
|
Symptom:
|
|
- `run-stage` rejects `--artifacts` for the selected stage.
|
|
|
|
Likely cause:
|
|
- `--artifacts` used with a stage other than `analyze` or `publish`.
|
|
|
|
Safe fix:
|
|
- use `--artifacts` only with `run-stage analyze ...` or `run-stage publish ...`.
|
|
|
|
## Previous-session input unavailable
|
|
|
|
Symptom:
|
|
- analyze fails on required previous-session artifact input.
|
|
|
|
Likely cause:
|
|
- `previous/**` cache not hydrated for this session.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session status 2026-04-04
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
```bash
|
|
narratio run-stage prepare 2026-04-04 --force
|
|
```
|
|
|
|
Or rehydrate from remote current state:
|
|
|
|
```bash
|
|
narratio session restore 2026-04-04
|
|
```
|
|
|
|
## Session lock conflict (`.lock`)
|
|
|
|
Symptom:
|
|
- command fails with lock conflict.
|
|
|
|
Likely cause:
|
|
- another process is running for the same session.
|
|
- stale lock file from interrupted command.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
ls -l {workspace.root}/work/{campaign}/{session_id}/.lock
|
|
ps aux | grep narratio
|
|
```
|
|
|
|
Safe fix:
|
|
- wait for active process; remove stale lock only if no process is active.
|
|
|
|
## Restore current pointer/manifest missing
|
|
|
|
Symptom:
|
|
- restore fails reading remote current state.
|
|
|
|
Likely cause:
|
|
- publish commit did not complete.
|
|
- `current/run_id.txt` or `current/manifest.json` is missing.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session restore 2026-04-04 --dry-run
|
|
```
|
|
|
|
Safe fix:
|
|
- republish from a healthy local session state.
|
|
|
|
## Restore conflict without `--force`
|
|
|
|
Symptom:
|
|
- restore reports conflict and exits.
|
|
|
|
Likely cause:
|
|
- local durable file differs from remote restore source.
|
|
|
|
Safe fix:
|
|
- inspect with `--dry-run`.
|
|
- rerun with `--force` only when remote should overwrite local.
|
|
|
|
## Secrets or credentials failure
|
|
|
|
Symptom:
|
|
- startup fails loading secrets dir, or storage/tool auth fails at runtime.
|
|
|
|
Likely cause:
|
|
- invalid `pipeline.secrets.env_dir`.
|
|
- missing credential env vars.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
ls -la /path/to/secrets_dir
|
|
env | grep -E 'AUDITA|OBJECT_STORAGE|AWS|SCRIPTORIUM'
|
|
```
|
|
|
|
Safe fix:
|
|
- fix path/permissions/env vars; keep secret values out of YAML.
|
|
|
|
## S3 audio prepare failure
|
|
|
|
Symptom:
|
|
- prepare fails in S3 mode (list/download/no files/backend error).
|
|
|
|
Likely cause:
|
|
- bad `session.inputs.audio_s3.prefix`.
|
|
- no `.flac` objects at prefix.
|
|
- bad storage credentials/config.
|
|
- mixed local+S3 audio config.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio run-stage prepare 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
|
```
|
|
|
|
Safe fix:
|
|
- configure exactly one audio mode and verify storage access.
|
|
|
|
## Publish output or current-pointer failure
|
|
|
|
Symptom:
|
|
- publish fails on required output source missing, upload error, or commit-marker write failure.
|
|
|
|
Likely cause:
|
|
- required source file not produced.
|
|
- storage upload failed before `current/run_id.txt` write.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session status 2026-04-04
|
|
narratio run-stage publish 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
|
```
|
|
|
|
Safe fix:
|
|
- rerun upstream stages to regenerate required outputs.
|
|
- adjust `pipeline.publish.outputs` source/dest rules.
|
|
- retry after storage issue is fixed.
|
|
|
|
## Helpful Links
|
|
|
|
- [docs/config.md](./config.md)
|
|
- [docs/cli.md](./cli.md)
|
|
- [docs/operations.md](./operations.md)
|
|
- [docs/internal/stage-publish.md](./internal/stage-publish.md)
|