380 lines
12 KiB
Markdown
380 lines
12 KiB
Markdown
# Troubleshooting
|
|
|
|
This guide maps common implemented failure modes to inspection steps and fixes.
|
|
For command syntax, see [CLI Reference](cli.md). For YAML fields and
|
|
environment overrides, see [Configuration](config.md). For output and
|
|
diagnostics layout, see [Operations](operations.md).
|
|
|
|
## Config File Not Found
|
|
|
|
Symptom:
|
|
|
|
```text
|
|
notarius: config file not found; pass --config or set NOTARIUS_CONFIG
|
|
```
|
|
|
|
Fix:
|
|
|
|
- Pass `--config path/to/config.yml`.
|
|
- Or set `NOTARIUS_CONFIG` to a readable file.
|
|
- Or install a config at `/usr/local/etc/notarius/config.yml`.
|
|
|
|
If the message says the config path is a directory or is not available, correct
|
|
the path or file permissions.
|
|
|
|
## Unsupported Or Invalid Config
|
|
|
|
Symptoms include:
|
|
|
|
- `unsupported config version`
|
|
- `config version is required`
|
|
- `field <name> not found`
|
|
- `total LLM concurrency must be greater than zero`
|
|
- `diagnostics retention "<value>" is not supported`
|
|
|
|
Fix:
|
|
|
|
- Use `version: 2`.
|
|
- Remove unknown YAML fields.
|
|
- Validate with:
|
|
|
|
```sh
|
|
go run ./cmd/notarius config validate --config path/to/config.yml
|
|
```
|
|
|
|
## Unknown Pipeline
|
|
|
|
Symptom:
|
|
|
|
```text
|
|
notarius: pipeline "..." is not configured
|
|
```
|
|
|
|
Fix:
|
|
|
|
- List configured pipeline IDs:
|
|
|
|
```sh
|
|
go run ./cmd/notarius pipelines list --config path/to/config.yml
|
|
```
|
|
|
|
- Use one of those IDs in `notarius run <pipeline-id>`.
|
|
- Check indentation under the top-level `pipelines` map.
|
|
|
|
## Unknown Or Incompatible Module
|
|
|
|
Symptoms mention a module key, pipeline slot, lane, capability, or `not
|
|
registered`.
|
|
|
|
Fix:
|
|
|
|
- Validate the pipeline against the production module catalog:
|
|
|
|
```sh
|
|
go run ./cmd/notarius config validate \
|
|
--config path/to/config.yml \
|
|
--pipeline dnd-session
|
|
```
|
|
|
|
- Use only implemented production module keys listed in
|
|
[Configuration](config.md#implemented-production-modules).
|
|
- Check that artifact lanes include an `extract` binding.
|
|
|
|
## Invalid `--only`
|
|
|
|
Symptoms include:
|
|
|
|
- `--only must contain comma-separated non-empty artifact lane IDs`
|
|
- `--only requires --pipeline`
|
|
- `selected artifact lane`
|
|
|
|
Fix:
|
|
|
|
- Use comma-separated lane IDs with no empty entries:
|
|
|
|
```sh
|
|
go run ./cmd/notarius run dnd-session \
|
|
--config path/to/config.yml \
|
|
--input path/to/input.json \
|
|
--only spells
|
|
```
|
|
|
|
- For `config validate`, include `--pipeline` when using `--only`.
|
|
- Confirm the lane ID exists under `pipelines.<id>.artifacts`.
|
|
|
|
## Reference Binding Failure
|
|
|
|
Symptoms include:
|
|
|
|
- `reference slot "..." is not declared`
|
|
- `reference slot "..." is declared by multiple selected targets`
|
|
- `required reference slot "..." is not bound`
|
|
- `--reference must use slot=path`
|
|
- `--without-reference must use a reference selector without =path`
|
|
- `read "...": no such file`
|
|
- `must be UTF-8 text`
|
|
- `media type "..." is not accepted`
|
|
- `is ... bytes, limit ...`
|
|
|
|
Fix:
|
|
|
|
- Confirm the selected chunker, extractor, merger, or normalizer declares the
|
|
slot. The implemented `dnd/scenes` chunker and `dnd/spells` extractor declare
|
|
optional `players`, `party`, and `glossary` slots, plus `roster` as a
|
|
deprecated compatibility alias for `party`.
|
|
- Use a specific selector when more than one selected target declares the same
|
|
slot: `chunk.context=./context.txt`,
|
|
`spells.extract.context=./extract-context.txt`,
|
|
`spells.merge.context=./merge-context.txt`, or
|
|
`spells.normalize.context=./normalize-context.txt`.
|
|
- `lane.slot=path` is valid only when exactly one selected extractor, merger,
|
|
or normalizer in that lane declares the slot. If more than one does, use
|
|
`lane.extract.slot=path`, `lane.merge.slot=path`, or
|
|
`lane.normalize.slot=path`.
|
|
- Use `--without-reference selector` to remove optional config bindings; do not
|
|
pass an empty `--reference selector=`.
|
|
- Check whether a path came from config or CLI. Config paths are relative to
|
|
the config file. CLI reference paths are relative to the current working
|
|
directory.
|
|
- Ensure the file is readable UTF-8 text and within any byte limit declared by
|
|
the declaring module.
|
|
- If the declaring module narrows accepted media types, use a file extension that
|
|
infers an accepted type such as `text/markdown`, `application/yaml`, or
|
|
`application/json`.
|
|
Unknown extensions infer `application/octet-stream`.
|
|
- If diagnostics are retained, inspect `resolved-pipeline.json`,
|
|
`resolved-references.json`, and `error.log`.
|
|
|
|
## Seriatim Input Validation Failure
|
|
|
|
Symptoms include `seriatim input`, `parse JSON`, `segments must not be empty`,
|
|
or validation errors naming a segment field.
|
|
|
|
Fix:
|
|
|
|
- Compare the input to
|
|
[examples/seriatim-minimal-transcript.json](../examples/seriatim-minimal-transcript.json).
|
|
- Ensure the JSON has a `metadata` object and a non-empty `segments` array.
|
|
- Each segment needs a non-empty `id`, non-empty `speaker`, non-empty `text`,
|
|
non-negative numeric `start`, and non-negative numeric `end`.
|
|
- Segment IDs must be unique and must not contain leading or trailing
|
|
whitespace.
|
|
- `end` must be greater than or equal to `start`.
|
|
|
|
## Scriptorium Profile Source Failure
|
|
|
|
Symptoms include:
|
|
|
|
- `scriptorium profile_dir and profile_file are mutually exclusive`
|
|
- `scriptorium.profile_dir must not be empty when set`
|
|
- `scriptorium.profile_file must not be empty when set`
|
|
- `profile load`
|
|
- `profile not found`
|
|
|
|
Fix:
|
|
|
|
- Configure at most one of `scriptorium.profile_dir` or
|
|
`scriptorium.profile_file`.
|
|
- Confirm the selected Scriptorium profile ID exists in the configured profile
|
|
source or Scriptorium built-in profiles.
|
|
- If using `--llm-profile`, pass a Scriptorium profile ID, not a removed
|
|
Notarius profile ID.
|
|
- Validate the config and selected pipeline:
|
|
|
|
```sh
|
|
go run ./cmd/notarius config validate \
|
|
--config path/to/config.yml \
|
|
--pipeline dnd-session
|
|
```
|
|
|
|
## LLM Profile Override Failure
|
|
|
|
Symptom:
|
|
|
|
```text
|
|
notarius: LLM profile override "..." is not configured
|
|
```
|
|
|
|
Fix:
|
|
|
|
- Add the profile to the configured Scriptorium profile source.
|
|
- Or use an existing Scriptorium profile ID with `--llm-profile`.
|
|
|
|
Use `--llm-profile <id>` when one run should force every LLM-backed binding to
|
|
the same Scriptorium profile. The override applies to effective chunk, extract,
|
|
merge, and normalize bindings.
|
|
|
|
## Missing API Key Environment Variable
|
|
|
|
Symptoms include:
|
|
|
|
- `api_key_env`
|
|
- `unset environment variable`
|
|
- provider authentication failures after selecting a profile that needs a key
|
|
|
|
Fix:
|
|
|
|
- Check the selected Scriptorium profile's `api_key_env` field.
|
|
- Set that environment variable before running Notarius.
|
|
- Do not put raw API keys in Notarius config or file-backed Scriptorium
|
|
profiles.
|
|
|
|
## Prompt Or Structured Output Failure
|
|
|
|
Symptoms include:
|
|
|
|
- `prompt not found`
|
|
- `prompt render`
|
|
- `schema`
|
|
- `validation`
|
|
- `decode structured output`
|
|
|
|
Fix:
|
|
|
|
- Ensure production modules register their embedded Scriptorium prompt and
|
|
schema assets.
|
|
- If the error names a profile, select a Scriptorium profile that is available
|
|
through the configured profile source or built-in catalog.
|
|
- If the error names generated output validation, retry with a model that
|
|
follows JSON schema instructions reliably.
|
|
- Inspect retained diagnostics `error.log`, `resolved-pipeline.json`, and
|
|
`run-manifest.json` when available. Prompt text, source text, reference
|
|
content, raw schema JSON, and secrets are not written to default diagnostics.
|
|
- Provider errors are redacted for bearer tokens and configured API key values.
|
|
|
|
## Scene Chunking Failure
|
|
|
|
Symptoms include:
|
|
|
|
- `dnd scenes chunker`
|
|
- `malformed structured output`
|
|
- `boundary_caveats`
|
|
- `start_unit_id`
|
|
- `end_unit_id`
|
|
- `gap`
|
|
- `overlap`
|
|
- `final scene`
|
|
- `complete structured output`
|
|
|
|
Fix:
|
|
|
|
- Validate the pipeline configuration and confirm the input module provides a
|
|
transcript source when using `chunk: dnd/scenes`.
|
|
- Confirm the selected Scriptorium profile has a working endpoint, model, and
|
|
credentials.
|
|
- Inspect retained diagnostics for the run error and resolved pipeline.
|
|
- If the error names malformed structured output, retry with a model that
|
|
follows structured response schemas reliably.
|
|
- If the error names `boundary_caveats`, check for blank or whitespace-only
|
|
caveat text in the scene response.
|
|
- Scene boundaries must resolve to valid source units, cover the full source
|
|
document, be contiguous, and not overlap.
|
|
|
|
## Session ID
|
|
|
|
Symptom: external logs or provider traces cannot be correlated with a Notarius
|
|
run.
|
|
|
|
Fix:
|
|
|
|
- Pass `--session-id <id>` to `notarius run`.
|
|
- Use a stable, non-secret identifier from the external orchestrator.
|
|
|
|
## Output Write Failure
|
|
|
|
Symptoms include:
|
|
|
|
- `create output directory`
|
|
- `write output file`
|
|
- `output file name must`
|
|
- `unsupported media type`
|
|
- `invalid JSON`
|
|
|
|
Fix:
|
|
|
|
- Ensure `--output-dir` points to a directory path or a path that can be
|
|
created.
|
|
- Check filesystem permissions and available disk space.
|
|
- The production JSON output encoder writes lane payloads under `lanes/` and
|
|
accepts only valid `application/json` normalized outputs. If an error names an
|
|
unsupported media type or invalid JSON, inspect the lane's merge and normalize
|
|
module output.
|
|
- If diagnostics were retained, inspect `run-report.json`, `run-manifest.json`,
|
|
and `error.log`.
|
|
|
|
The CLI rejects unsafe logical output paths before writing files.
|
|
|
|
## Raw Output Rejection
|
|
|
|
Symptoms include a successful run with:
|
|
|
|
- `validation_status` set to `rejected`;
|
|
- non-empty `rejected.json`;
|
|
- `rejected_outputs` entries in `manifest.json`.
|
|
|
|
Explanation and fixes:
|
|
|
|
- Validator rejection is a non-fatal run outcome. Rejected module outputs do not
|
|
pass to the next pipeline stage.
|
|
- Check `rejected.json` for the stage, lane, module, chunk, validator, reason,
|
|
message, and attempt count.
|
|
- Check `manifest.json` `validator_chains` to see the exact resolved validators
|
|
and order used for the rejected validation point. The production `dnd/spells`
|
|
extractor runs JSON syntax, JSON schema, D&D spell shape, source-reference,
|
|
and source-relatedness validators by default.
|
|
- If the configured chain is not what you expected, inspect the selected
|
|
binding in config. Omitted `validators` uses production defaults,
|
|
`validators: []` disables validators for that binding, and a non-empty list
|
|
replaces the default chain in configured order.
|
|
- Run `notarius config validate --pipeline <id>` to catch unknown validator keys
|
|
and invalid validator `llm_profile` usage before running the pipeline.
|
|
- Increase a module binding's `retries` only when re-running the same module
|
|
input can reasonably produce an acceptable output.
|
|
- If rejection is deterministic, fix the source input, module configuration, or
|
|
validation policy rather than adding retries.
|
|
|
|
Common production D&D spell validator reasons:
|
|
|
|
- `invalid_json`: the raw output is not valid JSON.
|
|
- `json_schema_invalid`: the raw JSON does not match the spell response schema.
|
|
- `invalid_spell_shape`: required spell-cast fields are missing or malformed.
|
|
- `invalid_source_refs`: source references are missing or do not point to valid
|
|
source units.
|
|
- `spell_not_near_source`: warning-only; the spell name was not found near the
|
|
cited source text.
|
|
|
|
## Retry Exhaustion
|
|
|
|
Symptoms include:
|
|
|
|
- errors containing `failed after ... attempt(s)`;
|
|
- rejected output records with `attempt_count` greater than `1`.
|
|
|
|
Fix:
|
|
|
|
- `retries` is the number of extra attempts after the first attempt for chunk,
|
|
extract, merge, and normalize bindings.
|
|
- Framework-level errors after the last attempt fail the run.
|
|
- Validator rejections after the last attempt are recorded as rejected outputs.
|
|
- Check retained `error.log`, `run-manifest.json`, and `rejected.json` for the
|
|
operation, module key, lane, chunk, and attempt count.
|
|
|
|
## Diagnostics Directory Surprise
|
|
|
|
Symptom: the diagnostics directory is missing after a successful run.
|
|
|
|
Fix:
|
|
|
|
- Check `diagnostics.retention`.
|
|
- With `auto`, successful runs without warnings are removed.
|
|
- Use `diagnostics.retention: always` when every diagnostics run directory
|
|
should be kept.
|
|
- Use `--diagnostics-dir` to override the configured work directory for a run.
|
|
|
|
Symptom: diagnostics exist even with `retention: never`.
|
|
|
|
Explanation:
|
|
|
|
- Failed runs are retained so that `error.log` and available context can be
|
|
inspected.
|