Complete documentation rebuild
This commit is contained in:
229
docs/troubleshooting.md
Normal file
229
docs/troubleshooting.md
Normal file
@@ -0,0 +1,229 @@
|
||||
# Troubleshooting
|
||||
|
||||
## Purpose
|
||||
Canonical operator troubleshooting guide for recurring implemented Narratio failures.
|
||||
|
||||
## Config file discovery failure
|
||||
|
||||
Symptom:
|
||||
- `run`, `plan`, `resume`, or `run-stage` fails saying config/session file was not found.
|
||||
|
||||
Likely Cause:
|
||||
- `pipeline.yml` or `session.yml` is missing from default search paths.
|
||||
- Wrong working directory when relying on `./session.yml`.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
pwd
|
||||
ls -l ./session.yml
|
||||
ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- Pass explicit paths with `--config` and `--session`.
|
||||
- Or place files in documented discovery paths.
|
||||
|
||||
Links:
|
||||
- [docs/config.md](./config.md)
|
||||
- [docs/cli.md](./cli.md)
|
||||
|
||||
## Session template rendering failure
|
||||
|
||||
Symptom:
|
||||
- Load fails with unresolved template placeholder or `session_id` mismatch.
|
||||
|
||||
Likely Cause:
|
||||
- `session.yml` contains `{{session_id}}`/`{{ session_id }}` but `--session-id` was omitted.
|
||||
- Provided `--session-id` does not match rendered `session_id`.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio plan --session ./session.yml --session-id 2026-04-04
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- Always pass `--session-id` when using template placeholders.
|
||||
- Ensure rendered `session_id` equals intended run session id.
|
||||
|
||||
Links:
|
||||
- [docs/config.md](./config.md)
|
||||
|
||||
## Strict YAML decode or validation failure
|
||||
|
||||
Symptom:
|
||||
- Config load fails with unknown field, missing required field, invalid duration, or invalid cross-field constraint.
|
||||
|
||||
Likely Cause:
|
||||
- YAML key typo or stale field name.
|
||||
- Required fields missing.
|
||||
- Invalid value format (for example duration/URL/env var name).
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- Correct fields/values to match canonical reference and examples.
|
||||
- Validate against `docs/examples/` shapes.
|
||||
|
||||
Links:
|
||||
- [docs/config.md](./config.md)
|
||||
- [docs/examples/](./examples/)
|
||||
|
||||
## Manifest/status path failure
|
||||
|
||||
Symptom:
|
||||
- `status` fails because manifest path is missing, unreadable, or invalid.
|
||||
|
||||
Likely Cause:
|
||||
- Wrong manifest path.
|
||||
- Manifest removed after cleanup.
|
||||
- Trying to run `status` without `--manifest`.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio status --manifest /path/to/manifest.json
|
||||
ls -l /path/to/manifest.json
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- Use manifest path printed by `run`, `resume`, or `run-stage` output.
|
||||
- Re-run with correct session/config if inspecting a different session.
|
||||
|
||||
Links:
|
||||
- [docs/cli.md](./cli.md)
|
||||
- [docs/operations.md](./operations.md)
|
||||
|
||||
## Session lock conflict (`.lock`)
|
||||
|
||||
Symptom:
|
||||
- Run fails with lock conflict indicating session workdir is already locked.
|
||||
|
||||
Likely Cause:
|
||||
- Another Narratio process is actively running the same session.
|
||||
- Prior run exited unexpectedly and left a stale lock file.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
ls -l {workspace.root}/work/{campaign}/{session_id}/.lock
|
||||
cat {workspace.root}/work/{campaign}/{session_id}/.lock
|
||||
ps aux | grep narratio
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- If another run is active, wait for it to finish.
|
||||
- If no process is active and lock is stale, remove only that session `.lock` file and retry.
|
||||
|
||||
Links:
|
||||
- [docs/operations.md](./operations.md)
|
||||
- [docs/internal/workspace.md](./internal/workspace.md)
|
||||
|
||||
## Secrets env-dir or credential env failure
|
||||
|
||||
Symptom:
|
||||
- Startup fails loading secrets directory, or a stage fails because required credential env var is missing.
|
||||
|
||||
Likely Cause:
|
||||
- `pipeline.secrets.env_dir` path is wrong/unreadable.
|
||||
- Credential env var referenced in config is unset or empty.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
ls -la /path/to/secrets_dir
|
||||
env | grep -E 'AUDITA|OBJECT_STORAGE|AWS|SCRIPTORIUM'
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- Fix `pipeline.secrets.env_dir` path/permissions.
|
||||
- Ensure required env vars are set to non-empty values.
|
||||
- Keep secrets out of YAML; use env references only.
|
||||
|
||||
Links:
|
||||
- [docs/config.md](./config.md)
|
||||
- [docs/operations.md](./operations.md)
|
||||
|
||||
## S3-audio prepare failure
|
||||
|
||||
Symptom:
|
||||
- `prepare` fails in S3 mode (no audio found, list/download failure, backend missing, path conflict).
|
||||
|
||||
Likely Cause:
|
||||
- Wrong `session.inputs.audio_s3.prefix`.
|
||||
- No `.flac` files at expected prefix.
|
||||
- Missing or invalid S3 backend credentials/config.
|
||||
- Conflicting audio-source settings (`audio_s3` plus local audio fields).
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 prepare
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- Ensure `audio_s3` is the only audio source configured for that session.
|
||||
- Confirm `.flac` objects exist under the resolved session audio prefix.
|
||||
- Fix S3 storage configuration and credentials.
|
||||
|
||||
Links:
|
||||
- [docs/config.md](./config.md)
|
||||
- [docs/operations.md](./operations.md)
|
||||
- [docs/internal/stage-prepare.md](./internal/stage-prepare.md)
|
||||
|
||||
## Archive prerequisite or promotion/current-pointer failure
|
||||
|
||||
Symptom:
|
||||
- `archive` fails due to prerequisite stage status, missing required promotion source, or pointer write failure.
|
||||
|
||||
Likely Cause:
|
||||
- One or more prerequisite stages are not `succeeded`.
|
||||
- Required promoted artifact does not exist.
|
||||
- Remote upload failure before `current/run_id.txt` write.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio status --manifest /path/to/manifest.json
|
||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 archive
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- Resume or rerun failed upstream stage(s).
|
||||
- Ensure required promoted artifact paths exist locally before archive.
|
||||
- Retry archive after storage/connectivity issue is resolved.
|
||||
|
||||
Links:
|
||||
- [docs/operations.md](./operations.md)
|
||||
- [docs/config.md](./config.md)
|
||||
- [docs/internal/stage-archive.md](./internal/stage-archive.md)
|
||||
|
||||
## `run-stage` invalid stage name or invalid flags
|
||||
|
||||
Symptom:
|
||||
- `run-stage` fails with unknown stage or invalid flag/argument usage.
|
||||
|
||||
Likely Cause:
|
||||
- Stage name typo.
|
||||
- Missing positional stage argument.
|
||||
- Unsupported/incorrect flag syntax.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 normalize
|
||||
```
|
||||
|
||||
Safe Fix:
|
||||
- Use only supported stage names.
|
||||
- Provide exactly one positional stage argument.
|
||||
- Align flags to documented command reference.
|
||||
|
||||
Links:
|
||||
- [docs/cli.md](./cli.md)
|
||||
- [docs/operations.md](./operations.md)
|
||||
Reference in New Issue
Block a user