narratio/docs/troubleshooting.md

# Troubleshooting

## Purpose
Canonical operator troubleshooting guide for recurring implemented Narratio failures.

## Config file discovery failure

Symptom:
- `run`, `plan`, `resume`, `run-stage`, or `restore` fails with config/session not found.

Likely Cause:
- `pipeline.yml` or `session.yml` is missing from discovery paths.
- wrong working directory when relying on `./session.yml`.

Diagnostics:

```bash
pwd
ls -l ./session.yml
ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
```

Safe Fix:
- pass explicit `--config` and `--session`.
- or place files in documented discovery paths.

Links:
- [docs/config.md](./config.md)
- [docs/cli.md](./cli.md)

## Session template rendering failure

Symptom:
- load fails with unresolved placeholder or `session_id` mismatch.

Likely Cause:
- templated `session.yml` used without `--session-id`.
- rendered `session_id` differs from passed `--session-id`.

Diagnostics:

```bash
narratio plan --session ./session.yml --session-id 2026-04-04
```

Safe Fix:
- pass `--session-id` when template placeholders are present.
- ensure rendered `session_id` matches intended run session id.

Links:
- [docs/config.md](./config.md)

## Strict YAML decode or validation failure

Symptom:
- config load fails with unknown field or validation error.

Likely Cause:
- typo/stale field name.
- missing required fields or invalid constraints.

Diagnostics:

```bash
narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04
```

Safe Fix:
- align fields/values to canonical config reference and examples.

Links:
- [docs/config.md](./config.md)
- [examples/](../examples/)

## `--artifacts` selection failure

Symptom:
- `run`/`resume`/`run-stage` fails with invalid or unknown artifact selection.

Likely Cause:
- `--artifacts` contains blank names or unknown artifact keys.
- `pipeline.scriptorium.artifacts` missing while using `--artifacts`.

Diagnostics:

```bash
narratio run --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout
```

Safe Fix:
- use configured artifact keys only.
- ensure `pipeline.scriptorium.artifacts` is defined.

Links:
- [docs/cli.md](./cli.md)
- [docs/config.md](./config.md)

## `run-stage --artifacts` on non-analyze stage

Symptom:
- `run-stage` fails with `--artifacts is only supported for stage "analyze"`.

Likely Cause:
- `--artifacts` was used with a non-`analyze` stage.

Diagnostics:

```bash
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts session_recap polish
```

Safe Fix:
- use `--artifacts` only with `run-stage ... analyze`.

Links:
- [docs/cli.md](./cli.md)

## Configured artifact dependency/input validation failure

Symptom:
- config validation fails for `depends_on`, `narratio.artifact.<name>` source, or artifact output path.

Likely Cause:
- `narratio.artifact.<name>` source missing matching `depends_on` key.
- dependency references unknown artifact key.
- dependency self-reference or enabled dependency cycle.
- artifact output path missing/invalid/outside `artifacts/` root.

Diagnostics:

```bash
narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04
```

Safe Fix:
- ensure artifact-to-artifact inputs have explicit `depends_on` entries using artifact keys.
- ensure referenced artifacts exist and define valid `output_path` values.
- keep output paths relative and under `artifacts/`.

Links:
- [docs/config.md](./config.md)
- [docs/internal/stage-analyze.md](./internal/stage-analyze.md)

## Required configured artifact input unavailable at analyze time

Symptom:
- analyze fails because configured input source is unavailable.

Likely Cause:
- required upstream configured artifact was not selected/executed this run.
- non-executable dependency output file is missing or invalid on disk.

Diagnostics:

```bash
narratio status --manifest /path/to/manifest.json
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout analyze
```

Safe Fix:
- run analyze with needed artifacts selected.
- or ensure dependency output file exists at configured path and is valid.

Links:
- [docs/operations.md](./operations.md)
- [docs/config.md](./config.md)

## Manifest/status path failure

Symptom:
- `status` fails because manifest path is missing, unreadable, or invalid.

Likely Cause:
- wrong manifest path.
- manifest removed after cleanup.
- `--manifest` omitted.

Diagnostics:

```bash
narratio status --manifest /path/to/manifest.json
ls -l /path/to/manifest.json
```

Safe Fix:
- use manifest path printed by `run`, `resume`, or `run-stage`.

Links:
- [docs/cli.md](./cli.md)
- [docs/operations.md](./operations.md)

## Session lock conflict (`.lock`)

Symptom:
- `run`, `resume`, `run-stage`, or `restore` fails with lock conflict for session workdir.

Likely Cause:
- another Narratio process is running same session.
- stale lock from interrupted prior run.

Diagnostics:

```bash
ls -l {workspace.root}/work/{campaign}/{session_id}/.lock
cat {workspace.root}/work/{campaign}/{session_id}/.lock
ps aux | grep narratio
```

Safe Fix:
- wait for active process to finish.
- if no process is active, remove only stale session `.lock` file.

Links:
- [docs/operations.md](./operations.md)
- [docs/internal/workspace.md](./internal/workspace.md)

## Restore remote current pointer or manifest missing

Symptom:
- `restore` fails with remote current pointer or current manifest errors.

Likely Cause:
- `current/run_id.txt` was never published.
- `current/manifest.json` is missing for the session prefix.
- archive commit did not complete.

Diagnostics:

```bash
narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
```

Safe Fix:
- verify archive stage succeeded for the target session.
- rerun/archive from a healthy source workspace so current pointers are published.

Links:
- [docs/operations.md](./operations.md)
- [docs/internal/stage-archive.md](./internal/stage-archive.md)

## Restore manifest identity mismatch

Symptom:
- `restore` fails because remote manifest session or campaign does not match requested values.

Likely Cause:
- wrong `--session-id` or wrong session config selected.
- archive prefix points to a different campaign/session.

Diagnostics:

```bash
narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
```

Safe Fix:
- use the correct session config and `--session-id`.
- verify campaign/session identity in local config before restore.

Links:
- [docs/config.md](./config.md)
- [docs/operations.md](./operations.md)

## Restore conflict without `--force`

Symptom:
- `restore` fails with `restore conflict` and conflict counts.

Likely Cause:
- local durable file differs from remote file for one or more planned restore paths.

Diagnostics:

```bash
narratio restore --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --dry-run
```

Safe Fix:
- review planned conflicts.
- rerun with `--force` only when remote state should overwrite local state.

Links:
- [docs/cli.md](./cli.md)
- [docs/operations.md](./operations.md)

## Restore report expectations

Symptom:
- operator expects restore report file but does not find one.

Likely Cause:
- restore was executed in `--dry-run` mode.
- restore failed before report persistence path (for example lock acquisition failure).

Diagnostics:

```bash
ls -l {workspace.root}/work/{campaign}/{session_id}/reports/restore-latest.json
```

Safe Fix:
- run non-dry-run restore for durable report output.
- resolve lock or early preflight failures and retry.

Links:
- [docs/operations.md](./operations.md)

## Secrets env-dir or credential-env failure

Symptom:
- startup fails loading secrets directory, or stage fails due to missing credential env vars.

Likely Cause:
- invalid `pipeline.secrets.env_dir` path/permissions.
- required credential env var unset/empty.

Diagnostics:

```bash
ls -la /path/to/secrets_dir
env | grep -E 'AUDITA|OBJECT_STORAGE|AWS|SCRIPTORIUM'
```

Safe Fix:
- fix secrets directory and credential env vars.
- keep secret values out of YAML.

Links:
- [docs/config.md](./config.md)

## S3-audio prepare failure

Symptom:
- `prepare` fails in S3 mode (listing/downloading/no audio/backend error).

Likely Cause:
- wrong `session.inputs.audio_s3.prefix`.
- no `.flac` files at resolved prefix.
- invalid/missing object-store credentials or backend config.
- mixed local+S3 audio input config.

Diagnostics:

```bash
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 prepare
```

Safe Fix:
- configure exactly one audio source mode.
- verify `.flac` files and storage access.

Links:
- [docs/config.md](./config.md)
- [docs/operations.md](./operations.md)

## Archive promotion/current-pointer failure

Symptom:
- archive fails on required promotion source missing or pointer write failure.

Likely Cause:
- required promoted file absent (including analyze outputs not generated for this run).
- storage upload failed before `current/run_id.txt` commit marker write.

Diagnostics:

```bash
narratio status --manifest /path/to/manifest.json
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 archive
```

Safe Fix:
- rerun or resume upstream stages to generate required files.
- adjust promotion `source`/`dest` rules to match artifacts that must exist.
- retry after storage issue is resolved.

Links:
- [docs/operations.md](./operations.md)
- [docs/config.md](./config.md)
- [docs/internal/stage-archive.md](./internal/stage-archive.md)