481 lines
11 KiB
Markdown
481 lines
11 KiB
Markdown
# Troubleshooting
|
|
|
|
Operational diagnosis guide for common Narratio failures.
|
|
|
|
## Config file not found
|
|
|
|
Symptom:
|
|
|
|
- command fails to resolve `pipeline.yml`, `campaign.yml`, or `session.yml`.
|
|
|
|
Likely causes:
|
|
|
|
- missing files in default search paths;
|
|
- wrong campaign selection;
|
|
- omitted explicit flags.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session plan 2026-04-04
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- pass explicit `--config`, `--campaign` or `--campaign-file`, and `--session`.
|
|
|
|
Relevant reference: [Configuration discovery](./config.md#discovery-and-selection).
|
|
|
|
## Session template placeholders rejected
|
|
|
|
Symptom:
|
|
|
|
- load error says session file must be concrete or contains `{{ ... }}` placeholders.
|
|
|
|
Likely cause:
|
|
|
|
- using template content as runtime session config.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session validate 2026-04-04 --session /path/session.yml
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- generate concrete session YAML with `narratio session init`.
|
|
|
|
Relevant reference: [Operations: Session Initialization](./operations.md#session-initialization).
|
|
|
|
## Strict decode or schema validation failure
|
|
|
|
Symptom:
|
|
|
|
- unknown field / invalid value error during config load.
|
|
|
|
Likely cause:
|
|
|
|
- stale field name, typo, invalid enum, or invalid duration/path format.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session plan 2026-04-04 --config /path/pipeline.yml --campaign-file /path/campaign.yml --session /path/session.yml
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- align config with [Configuration](./config.md) and the
|
|
[maintained examples](../examples/README.md).
|
|
|
|
Relevant reference: [Configuration](./config.md).
|
|
|
|
## Audio mode conflict
|
|
|
|
Symptom:
|
|
|
|
- validation fails on session audio configuration.
|
|
|
|
Likely cause:
|
|
|
|
- configured both local and S3 session audio inputs.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session validate 2026-04-04
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- use local mode (`audio_dir` or `audio_files`) or S3 mode (`audio_s3.prefix`), not both.
|
|
|
|
Relevant reference: [Session configuration](./config.md#session).
|
|
|
|
## `--artifacts` selection error
|
|
|
|
Symptom:
|
|
|
|
- unknown artifact key or invalid `--artifacts` usage.
|
|
|
|
Likely causes:
|
|
|
|
- key not defined in `pipeline.scriptorium.artifacts`;
|
|
- empty list entry (for example trailing comma);
|
|
- `run-stage` used with non-`analyze`/`publish` target.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session artifacts 2026-04-04
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- provide only configured keys and use `--artifacts` with supported commands/stages.
|
|
|
|
Relevant reference: [CLI artifact selection](./cli.md).
|
|
|
|
## Notarius executable missing
|
|
|
|
Symptom:
|
|
|
|
- extraction fails while resolving or starting the Notarius executable.
|
|
|
|
Likely causes:
|
|
|
|
- `pipeline.notarius.binary` is not installed, executable, or on `PATH`;
|
|
- a configured executable path is wrong.
|
|
|
|
Safe fix:
|
|
|
|
- install a compatible Notarius release or correct the binary setting, then
|
|
rerun extraction.
|
|
|
|
Relevant references: [Notarius configuration](./config.md#notarius-output-entries)
|
|
and [Notarius integration](./integrations/notarius.md).
|
|
|
|
## Notarius exits nonzero
|
|
|
|
Symptom:
|
|
|
|
- extraction reports a Notarius exit error instead of a receipt.
|
|
|
|
Diagnostics:
|
|
|
|
- inspect `runs/{run_id}/extract/notarius.stderr.log`; stdout is reserved for
|
|
the receipt and is not merged with diagnostics.
|
|
|
|
Safe fix:
|
|
|
|
- correct the reported Notarius pipeline, input, provider, or configuration
|
|
failure and rerun extraction. Do not edit a staged output bundle into place.
|
|
|
|
After a failed replacement, an older immutable bundle may still exist even
|
|
though the current session manifest has no successful extraction payload. This
|
|
is expected audit state, not a signal to relink the old bundle manually.
|
|
|
|
Relevant reference: [Operations: Extraction Workflow](./operations.md#extraction-workflow).
|
|
|
|
## Atomic Notarius promotion unsupported
|
|
|
|
Symptom:
|
|
|
|
- extraction fails with `atomic no-replace directory promotion is unsupported`
|
|
before a durable bundle or temporary promotion tree is created.
|
|
|
|
Likely cause:
|
|
|
|
- Narratio is running on an operating system other than Linux, macOS, or
|
|
Windows, where the required atomic no-replace directory primitive has not
|
|
been implemented and verified.
|
|
|
|
Safe fix:
|
|
|
|
- run extraction on Linux, macOS, or Windows. Do not replace the atomic commit
|
|
with a manual copy or move; the session manifest must never observe a partial
|
|
or overwritten bundle.
|
|
|
|
This is an extraction-specific platform boundary, not a support statement for
|
|
unrelated Narratio workflows. See
|
|
[Operations: Extraction Workflow](./operations.md#extraction-workflow).
|
|
|
|
## Notarius receipt or index incompatible
|
|
|
|
Symptom:
|
|
|
|
- extraction rejects the receipt schema, pipeline identity, bundle/index path,
|
|
lane descriptor, or payload path even though Notarius exited successfully.
|
|
|
|
Likely causes:
|
|
|
|
- Narratio and Notarius versions disagree on their consumer contract;
|
|
- the configured pipeline or lane constraints are stale;
|
|
- output paths escape the bundle or traverse symlinks.
|
|
|
|
Safe fix:
|
|
|
|
- compare installed Notarius output with the canonical Notarius contracts,
|
|
including receipt `index_file: index.json` and index management names
|
|
`manifest.json`, `rejected.json`, and `warnings.json`; align
|
|
`pipeline.notarius` constraints and rerun. Do not bypass confinement or schema
|
|
checks.
|
|
|
|
Relevant reference: [Notarius integration](./integrations/notarius.md).
|
|
|
|
## Required Notarius lane rejected or missing
|
|
|
|
Symptom:
|
|
|
|
- extraction fails because a configured lane is rejected, missing, duplicated,
|
|
or incompatible, including after a zero exit.
|
|
|
|
Safe fix:
|
|
|
|
- inspect the Notarius diagnostic log and bundle rejection/warning information;
|
|
- correct the Notarius module or the exact declared lane contract;
|
|
- remove an output declaration only if downstream consumers genuinely no longer
|
|
require that source, then rerun extraction.
|
|
|
|
Every configured output is required. Narratio does not promote a partial result.
|
|
|
|
## Extraction resume invalidated
|
|
|
|
Symptom:
|
|
|
|
- a previously successful extraction runs again during ordinary continuation.
|
|
|
|
Likely causes:
|
|
|
|
- the executable/config path, pipeline ID, timeout, working directory, or
|
|
configured output contracts changed;
|
|
- the durable bundle, index, lane set, provenance, regular-file status, or
|
|
checksum no longer validates.
|
|
|
|
Safe fix:
|
|
|
|
- allow the automatic rerun after verifying the current configuration. Treat
|
|
an unsafe path or symlink error as filesystem corruption or tampering and
|
|
investigate it rather than replacing files manually.
|
|
|
|
## Notarius transitive configuration changed
|
|
|
|
Symptom:
|
|
|
|
- Notarius profiles, prompts, modules, imported files, or references changed,
|
|
but Narratio still considers the previous extraction resumable.
|
|
|
|
Safe fix:
|
|
|
|
```bash
|
|
narratio run-stage extract 2026-04-04 --force
|
|
```
|
|
|
|
Narratio fingerprints its invocation contract, not the contents of transitive
|
|
Notarius inputs. Always force extraction after changing them; downstream
|
|
successful stages are then marked stale normally.
|
|
|
|
Relevant reference: [Operations: Extraction Workflow](./operations.md#extraction-workflow).
|
|
|
|
## Previous-session artifact input missing
|
|
|
|
Symptom:
|
|
|
|
- prepare/analyze fails due to missing required previous-session artifact cache input.
|
|
|
|
Likely causes:
|
|
|
|
- missing `session.previous_session_id`;
|
|
- previous artifact not restored/published for source session.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session validate 2026-04-04
|
|
narratio session status 2026-04-04
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
```bash
|
|
narratio session restore 2026-04-04
|
|
```
|
|
|
|
or rerun prepare after correcting session config:
|
|
|
|
```bash
|
|
narratio run-stage prepare 2026-04-04 --force
|
|
```
|
|
|
|
Relevant reference: [Operations: Restore Workflow](./operations.md#restore-workflow).
|
|
|
|
## Session lock conflict (`.lock`)
|
|
|
|
Symptom:
|
|
|
|
- command fails acquiring session lock.
|
|
|
|
Likely causes:
|
|
|
|
- another process is running for the same session;
|
|
- a process still holds the operating-system lock while it is shutting down.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
ls -l {workspace.root}/work/{campaign}/{session_id}/.lock
|
|
ps aux | grep narratio
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- wait for active process completion;
|
|
- retry after an interrupted holder has exited; the kernel releases its lock
|
|
even though the `.lock` metadata file remains for inspection.
|
|
|
|
Relevant reference: [Operations: Local State Layout](./operations.md#local-state-layout).
|
|
|
|
## Restore conflict without `--force`
|
|
|
|
Symptom:
|
|
|
|
- restore fails with conflict count.
|
|
|
|
Likely cause:
|
|
|
|
- local durable files differ from remote restore sources.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session restore 2026-04-04 --dry-run
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- review conflicts;
|
|
- rerun with `--force` only when remote state should overwrite local.
|
|
|
|
Relevant reference: [Operations: Restore Workflow](./operations.md#restore-workflow).
|
|
|
|
## Restore current-state discovery failure
|
|
|
|
Symptom:
|
|
|
|
- restore cannot find current pointer or current manifest.
|
|
|
|
Likely causes:
|
|
|
|
- no committed publish current state;
|
|
- storage credentials or connectivity failure.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session status 2026-04-04
|
|
narratio session restore 2026-04-04 --dry-run
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- resolve storage/auth issue;
|
|
- republish from healthy local state if current pointer is missing.
|
|
|
|
Relevant reference: [Operations: Publish Workflow](./operations.md#publish-workflow).
|
|
|
|
## Publish output failure
|
|
|
|
Symptom:
|
|
|
|
- publish fails on missing required source, upload error, or commit write.
|
|
|
|
Likely causes:
|
|
|
|
- required source file not produced;
|
|
- lock/state expectations mismatch;
|
|
- remote storage failure.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session artifacts 2026-04-04 --remote
|
|
narratio session status 2026-04-04
|
|
narratio run-stage publish 2026-04-04 --force
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- regenerate missing sources by rerunning prerequisite stages;
|
|
- correct publish source/destination rules;
|
|
- retry after storage failure is resolved.
|
|
|
|
Relevant reference: [Publish configuration](./config.md#publish-configuration-summary).
|
|
|
|
## Render markdown source missing
|
|
|
|
Symptom:
|
|
|
|
- analyze or publish fails because `narratio.transcript.final_markdown` or `narratio.transcript.final_trimmed_markdown` is unavailable.
|
|
|
|
Likely causes:
|
|
|
|
- render stage was not executed after transcript changes;
|
|
- render stage failed before producing canonical markdown outputs.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session status 2026-04-04
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- rerun render and then retry downstream stage(s):
|
|
|
|
```bash
|
|
narratio run-stage render 2026-04-04 --force
|
|
narratio run-stage analyze 2026-04-04 --force
|
|
```
|
|
|
|
Relevant reference: [Operations: Stage Execution](./operations.md#stage-execution-and-continuation-behavior).
|
|
|
|
## Secrets or storage credential failure
|
|
|
|
Symptom:
|
|
|
|
- object-store command fails at initialization/auth.
|
|
|
|
Likely causes:
|
|
|
|
- invalid `pipeline.secrets.env_dir`;
|
|
- missing credential environment variables;
|
|
- invalid S3 endpoint/bucket settings.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
ls -la /path/to/secrets_dir
|
|
env | sed 's/=.*//' | grep -E 'OBJECT_STORAGE|AWS|AUDITA|SCRIPTORIUM'
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- correct secret-file path and permissions;
|
|
- provide required env vars;
|
|
- keep secret values out of YAML.
|
|
|
|
Relevant reference: [Secrets](./config.md#secrets-handling).
|
|
|
|
## S3 audio prepare failure
|
|
|
|
Symptom:
|
|
|
|
- prepare fails listing/downloading session S3 audio.
|
|
|
|
Likely causes:
|
|
|
|
- incorrect `session.inputs.audio_s3.prefix`;
|
|
- no matching `.flac` objects;
|
|
- storage connectivity or permissions failure.
|
|
|
|
Diagnostics:
|
|
|
|
```bash
|
|
narratio session validate 2026-04-04
|
|
```
|
|
|
|
Safe fix:
|
|
|
|
- verify prefix contents and storage access;
|
|
- keep session audio mode consistent.
|
|
|
|
Relevant reference: [Operations](./operations.md).
|
|
|
|
## References
|
|
|
|
- [docs/cli.md](./cli.md)
|
|
- [docs/config.md](./config.md)
|
|
- [docs/operations.md](./operations.md)
|
|
- [docs/internal/stage-publish.md](./internal/stage-publish.md)
|