Rewrite user and operator documentation for current CLI and config behavior
This commit is contained in:
@@ -1,35 +1,58 @@
|
||||
# Troubleshooting
|
||||
|
||||
## Purpose
|
||||
Canonical operator troubleshooting guide for recurring Narratio failures.
|
||||
Operational diagnosis guide for common Narratio failures.
|
||||
|
||||
## Config discovery failure
|
||||
## Config file not found
|
||||
|
||||
Symptom:
|
||||
- command fails because `pipeline.yml`, `campaign.yml`, or `session.yml` was not found.
|
||||
|
||||
Likely cause:
|
||||
- missing files in discovery paths.
|
||||
- missing/incorrect campaign selection.
|
||||
- local file exists but was not passed explicitly.
|
||||
- command fails to resolve `pipeline.yml`, `campaign.yml`, or `session.yml`.
|
||||
|
||||
Likely causes:
|
||||
|
||||
- missing files in default search paths;
|
||||
- wrong campaign selection;
|
||||
- omitted explicit flags.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
|
||||
ls -l /usr/local/etc/narratio/session.yml /etc/narratio/session.yml
|
||||
narratio session plan 2026-04-04
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
|
||||
- pass explicit `--config`, `--campaign` or `--campaign-file`, and `--session`.
|
||||
|
||||
## Templated session file rejected
|
||||
## Session template placeholders rejected
|
||||
|
||||
Symptom:
|
||||
- load fails because `session.yml` must be concrete.
|
||||
|
||||
- load error says session file must be concrete or contains `{{ ... }}` placeholders.
|
||||
|
||||
Likely cause:
|
||||
- template placeholders (`{{ ... }}`) still present in loaded session config.
|
||||
|
||||
- using template content as runtime session config.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio session validate 2026-04-04 --session /path/session.yml
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
|
||||
- generate concrete session YAML with `narratio session init`.
|
||||
|
||||
## Strict decode or schema validation failure
|
||||
|
||||
Symptom:
|
||||
|
||||
- unknown field / invalid value error during config load.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- stale field name, typo, invalid enum, or invalid duration/path format.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
@@ -38,82 +61,79 @@ narratio session plan 2026-04-04 --config /path/pipeline.yml --campaign-file /pa
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
- generate concrete session YAML via `narratio session init`.
|
||||
|
||||
## Strict decode or validation failure
|
||||
- align config with [docs/config.md](./config.md) and maintained files under `examples/`.
|
||||
|
||||
## Audio mode conflict
|
||||
|
||||
Symptom:
|
||||
- unknown field or invalid value error during config load.
|
||||
|
||||
- validation fails on session audio configuration.
|
||||
|
||||
Likely cause:
|
||||
- typo, stale field name, or invalid value.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio session plan 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
- align config with [docs/config.md](./config.md) and maintained examples.
|
||||
|
||||
## `--artifacts` selection failure
|
||||
|
||||
Symptom:
|
||||
- command fails on unknown/invalid selected artifact key.
|
||||
|
||||
Likely cause:
|
||||
- artifact key not defined in `pipeline.scriptorium.artifacts`.
|
||||
- empty token in `--artifacts` input.
|
||||
|
||||
Safe fix:
|
||||
- use only configured artifact keys.
|
||||
|
||||
## `run-stage --artifacts` unsupported stage
|
||||
|
||||
Symptom:
|
||||
- `run-stage` rejects `--artifacts` for the selected stage.
|
||||
|
||||
Likely cause:
|
||||
- `--artifacts` used with a stage other than `analyze` or `publish`.
|
||||
|
||||
Safe fix:
|
||||
- use `--artifacts` only with `run-stage analyze ...` or `run-stage publish ...`.
|
||||
|
||||
## Previous-session input unavailable
|
||||
|
||||
Symptom:
|
||||
- analyze fails on required previous-session artifact input.
|
||||
|
||||
Likely cause:
|
||||
- `previous/**` cache not hydrated for this session.
|
||||
|
||||
- configured both local and S3 session audio inputs.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- use local mode (`audio_dir` or `audio_files`) or S3 mode (`audio_s3.prefix`), not both.
|
||||
|
||||
## `--artifacts` selection error
|
||||
|
||||
Symptom:
|
||||
|
||||
- unknown artifact key or invalid `--artifacts` usage.
|
||||
|
||||
Likely causes:
|
||||
|
||||
- key not defined in `pipeline.scriptorium.artifacts`;
|
||||
- empty list entry (for example trailing comma);
|
||||
- `run-stage` used with non-`analyze`/`publish` target.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- provide only configured keys and use `--artifacts` with supported commands/stages.
|
||||
|
||||
## Previous-session artifact input missing
|
||||
|
||||
Symptom:
|
||||
|
||||
- prepare/analyze fails due to missing required previous-session artifact cache input.
|
||||
|
||||
Likely causes:
|
||||
|
||||
- missing `session.previous_session_id`;
|
||||
- previous artifact not restored/published for source session.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio session validate 2026-04-04
|
||||
narratio session status 2026-04-04
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
|
||||
```bash
|
||||
narratio run-stage prepare 2026-04-04 --force
|
||||
narratio session restore 2026-04-04
|
||||
```
|
||||
|
||||
Or rehydrate from remote current state:
|
||||
or rerun prepare after correcting session config:
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04
|
||||
narratio run-stage prepare 2026-04-04 --force
|
||||
```
|
||||
|
||||
## Session lock conflict (`.lock`)
|
||||
|
||||
Symptom:
|
||||
- command fails with lock conflict.
|
||||
|
||||
Likely cause:
|
||||
- another process is running for the same session.
|
||||
- stale lock file from interrupted command.
|
||||
- command fails acquiring session lock.
|
||||
|
||||
Likely causes:
|
||||
|
||||
- another process is running for the same session;
|
||||
- stale lock left by interrupted process.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
@@ -123,16 +143,19 @@ ps aux | grep narratio
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
- wait for active process; remove stale lock only if no process is active.
|
||||
|
||||
## Restore current pointer/manifest missing
|
||||
- wait for active process completion;
|
||||
- remove stale lock only after confirming no live process owns it.
|
||||
|
||||
## Restore conflict without `--force`
|
||||
|
||||
Symptom:
|
||||
- restore fails reading remote current state.
|
||||
|
||||
- restore fails with conflict count.
|
||||
|
||||
Likely cause:
|
||||
- publish commit did not complete.
|
||||
- `current/run_id.txt` or `current/manifest.json` is missing.
|
||||
|
||||
- local durable files differ from remote restore sources.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
@@ -141,83 +164,110 @@ narratio session restore 2026-04-04 --dry-run
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
- republish from a healthy local session state.
|
||||
|
||||
## Restore conflict without `--force`
|
||||
- review conflicts;
|
||||
- rerun with `--force` only when remote state should overwrite local.
|
||||
|
||||
## Restore current-state discovery failure
|
||||
|
||||
Symptom:
|
||||
- restore reports conflict and exits.
|
||||
|
||||
Likely cause:
|
||||
- local durable file differs from remote restore source.
|
||||
- restore cannot find current pointer or current manifest.
|
||||
|
||||
Safe fix:
|
||||
- inspect with `--dry-run`.
|
||||
- rerun with `--force` only when remote should overwrite local.
|
||||
Likely causes:
|
||||
|
||||
## Secrets or credentials failure
|
||||
|
||||
Symptom:
|
||||
- startup fails loading secrets dir, or storage/tool auth fails at runtime.
|
||||
|
||||
Likely cause:
|
||||
- invalid `pipeline.secrets.env_dir`.
|
||||
- missing credential env vars.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
ls -la /path/to/secrets_dir
|
||||
env | grep -E 'AUDITA|OBJECT_STORAGE|AWS|SCRIPTORIUM'
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
- fix path/permissions/env vars; keep secret values out of YAML.
|
||||
|
||||
## S3 audio prepare failure
|
||||
|
||||
Symptom:
|
||||
- prepare fails in S3 mode (list/download/no files/backend error).
|
||||
|
||||
Likely cause:
|
||||
- bad `session.inputs.audio_s3.prefix`.
|
||||
- no `.flac` objects at prefix.
|
||||
- bad storage credentials/config.
|
||||
- mixed local+S3 audio config.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio run-stage prepare 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
- configure exactly one audio mode and verify storage access.
|
||||
|
||||
## Publish output or current-pointer failure
|
||||
|
||||
Symptom:
|
||||
- publish fails on required output source missing, upload error, or commit-marker write failure.
|
||||
|
||||
Likely cause:
|
||||
- required source file not produced.
|
||||
- storage upload failed before `current/run_id.txt` write.
|
||||
- no committed publish current state;
|
||||
- storage credentials or connectivity failure.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio session status 2026-04-04
|
||||
narratio run-stage publish 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
||||
narratio session restore 2026-04-04 --dry-run
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
- rerun upstream stages to regenerate required outputs.
|
||||
- adjust `pipeline.publish.outputs` source/dest rules.
|
||||
- retry after storage issue is fixed.
|
||||
|
||||
## Helpful Links
|
||||
- resolve storage/auth issue;
|
||||
- republish from healthy local state if current pointer is missing.
|
||||
|
||||
## Publish output failure
|
||||
|
||||
Symptom:
|
||||
|
||||
- publish fails on missing required source, upload error, or commit write.
|
||||
|
||||
Likely causes:
|
||||
|
||||
- required source file not produced;
|
||||
- lock/state expectations mismatch;
|
||||
- remote storage failure.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio session artifacts 2026-04-04 --remote
|
||||
narratio session status 2026-04-04
|
||||
narratio run-stage publish 2026-04-04 --force
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
|
||||
- regenerate missing sources by rerunning prerequisite stages;
|
||||
- correct publish source/destination rules;
|
||||
- retry after storage failure is resolved.
|
||||
|
||||
## Secrets or storage credential failure
|
||||
|
||||
Symptom:
|
||||
|
||||
- object-store command fails at initialization/auth.
|
||||
|
||||
Likely causes:
|
||||
|
||||
- invalid `pipeline.secrets.env_dir`;
|
||||
- missing credential environment variables;
|
||||
- invalid S3 endpoint/bucket settings.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
ls -la /path/to/secrets_dir
|
||||
env | grep -E 'OBJECT_STORAGE|AWS|AUDITA|SCRIPTORIUM'
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
|
||||
- correct secret-file path and permissions;
|
||||
- provide required env vars;
|
||||
- keep secret values out of YAML.
|
||||
|
||||
## S3 audio prepare failure
|
||||
|
||||
Symptom:
|
||||
|
||||
- prepare fails listing/downloading session S3 audio.
|
||||
|
||||
Likely causes:
|
||||
|
||||
- incorrect `session.inputs.audio_s3.prefix`;
|
||||
- no matching `.flac` objects;
|
||||
- storage connectivity or permissions failure.
|
||||
|
||||
Diagnostics:
|
||||
|
||||
```bash
|
||||
narratio run-stage prepare 2026-04-04 --force
|
||||
```
|
||||
|
||||
Safe fix:
|
||||
|
||||
- verify prefix contents and storage access;
|
||||
- keep session audio mode consistent.
|
||||
|
||||
## References
|
||||
|
||||
- [docs/config.md](./config.md)
|
||||
- [docs/cli.md](./cli.md)
|
||||
- [docs/config.md](./config.md)
|
||||
- [docs/operations.md](./operations.md)
|
||||
- [docs/internal/stage-publish.md](./internal/stage-publish.md)
|
||||
|
||||
Reference in New Issue
Block a user