# Configuration This is the canonical reference for implemented Notarius configuration. Notarius reads YAML config files with `version: 3`. File configuration is applied over built-in defaults, then environment overrides are applied. Explicit CLI overrides are applied last where the command supports them. ## Discovery Commands that load configuration use this order: 1. an explicit path supplied through the CLI, when provided; 2. `NOTARIUS_CONFIG`, when set to a non-empty path; 3. `/usr/local/etc/notarius/config.yml`. If none is available, the command fails with a config file not found error. The explicit-path option is defined in the [CLI reference](cli.md). ## Maintained Examples - [Minimal D&D spell configuration](../examples/dnd-spells.config.yml) - [Production-oriented D&D spell configuration](../examples/dnd-spells-production.config.yml) Both are complete version 3 files. The fragments below illustrate individual fields and are not alternate complete configurations. ## Top-Level Fields - `version`: required. The only supported value is `3`. - `scriptorium`: optional Scriptorium profile source settings. - `pipelines`: optional map of pipeline IDs to pipeline definitions. - `concurrency`: optional global concurrency settings. - `output`: optional durable output placement. - `cache`: optional chunk-plan and checkpoint cache placement. - `debug`: optional debug-bundle placement. It does not enable debug capture. Unknown YAML fields are rejected. The removed top-level `llm_profiles` field is rejected; execution profiles now come from Scriptorium. ## Defaults Built-in defaults: - `concurrency.total_llm`: `1` - `concurrency.stage_workers.extract`: effective `concurrency.total_llm` - `output.directory`: `./notarius-output` - `cache.chunk_plans.mode`: `auto` - `cache.chunk_plans.directory`: unset, selecting `/notarius/chunk-plans` - `cache.checkpoints.enabled`: `false` - `cache.checkpoints.directory`: unset, selecting `/notarius/checkpoints` - `debug.directory`: `./notarius-debug` No pipelines are built in. A run requires a configured pipeline. If `scriptorium` is omitted, Notarius uses Scriptorium's built-in profile catalog. Prompt definitions may also name default profile IDs. The current D&D scene and spell prompts default to the built-in `mistral-small-3` profile when a module binding does not set `llm_profile`. That built-in profile reads its credential from `OPENROUTER_API_KEY`. ## Scriptorium Profiles `scriptorium` fields: - `profile_dir`: optional directory containing Scriptorium profile YAML files. - `profile_file`: optional Scriptorium profile YAML file. `profile_dir` and `profile_file` are mutually exclusive. Custom profiles overlay Scriptorium built-in profiles by profile ID. Scriptorium profile files use Scriptorium's profile schema. A minimal profile looks like: ```yaml id: local-fast endpoint: http://127.0.0.1:8080/v1 model: your-model api_key_env: SCRIPTORIUM_API_KEY timeout_seconds: 180 ``` Notarius does not accept raw API keys in Notarius config. For file-backed Scriptorium profiles, store the environment variable name in `api_key_env` and set that variable in the run environment. Scriptorium rejects raw `api_key` fields in profile YAML. ## Environment Overrides These environment variables are applied after the config file: - `NOTARIUS_CONFIG`: config discovery path. - `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency. - `NOTARIUS_STAGE_WORKERS_EXTRACT`: integer extract worker limit. - `NOTARIUS_OUTPUT_DIR`: durable output root. - `NOTARIUS_CACHE_CHUNK_PLANS_MODE`: chunk-plan cache mode. - `NOTARIUS_CACHE_CHUNK_PLANS_DIR`: chunk-plan cache root. - `NOTARIUS_CACHE_CHECKPOINTS_DIR`: checkpoint cache root. - `NOTARIUS_DEBUG_DIR`: debug-bundle root. Integer environment values must parse as base-10 integers. Directory overrides must be non-empty after trimming. Cache-directory fields in a file may be empty, which deliberately selects the corresponding per-user default. The removed `NOTARIUS_LLM_DEFAULT_*` variables are not read. Configure provider endpoint, model, and credential environment variable names through Scriptorium profiles. ## Concurrency `concurrency` fields: - `total_llm`: positive integer ceiling on concurrent provider calls. - `stage_workers`: optional map of framework worker limits. The only supported key is `extract`. `stage_workers.extract` defaults to the effective `total_llm` value after file and environment precedence. It must be between `1` and `total_llm`, inclusive. Unknown or empty stage-worker keys are rejected. The environment override `NOTARIUS_STAGE_WORKERS_EXTRACT` takes precedence over the file value, as does `NOTARIUS_TOTAL_LLM_CONCURRENCY` for the global ceiling. The worker value is present in effective and redacted configuration. It bounds the fixed run-wide extract pool and its bounded dispatch queue. Extract jobs are submitted by source chunk and then resolved lane; `total_llm` independently bounds actual provider calls made by extracts, retries, and validators. ## Pipelines A pipeline selects implementations for the fixed workflow defined by [Architecture](policy/architecture.md#system-shape). Pipeline fields: - `input`: required module binding. - `chunk`: optional module binding. Default module is `generic`. - `artifacts`: required for pipeline resolution. It maps artifact lane IDs to lane definitions. - `output`: optional module binding. Default module is `json`. - `references`: optional map of reference slot names to reference paths. These bindings are defaults for eligible pipeline targets that declare the matching slot. Artifact lane fields: - `extract`: required module binding. - `merge`: optional module binding. Default module is `appendorder`. - `normalize`: optional module binding. Default module is `noop`. - `validators`: deprecated lane-level validator list. Non-empty lists are rejected; use `extract.validators`, `merge.validators`, or `normalize.validators`. - `references`: optional compatibility alias for extractor reference bindings. Lane bindings override pipeline-level bindings for the same slot. Commands that resolve a pipeline fail for unknown or incompatible module keys. See [CLI Reference](cli.md) for command syntax. Reference bindings are validated against reference slots declared by eligible chunk, extract, merge, and normalize targets during pipeline resolution. Required slots must be bound after config defaults, target-local references, lane-level compatibility bindings, and command-line reference overrides are applied. Config-relative paths are resolved relative to the config file; command-line reference paths are resolved relative to the current working directory. Bound files must be UTF-8 text. Reference media types are inferred from file extensions and checked when a module restricts accepted types; unknown extensions use `application/octet-stream`. See [CLI Reference](cli.md#run) for command-line selectors and [Operations](operations.md) for recorded provenance and sensitive-data handling. Pipeline-level `references` are defaults. They are valid when at least one eligible target in the full configured pipeline declares the slot, including chunk, extractor, merger, and normalizer targets. During a run, they apply only to the selected targets that declare the slot: ```yaml pipelines: dnd-session: input: seriatim references: players: ./campaign/players.txt party: ./campaign/party-roster.txt glossary: ./campaign/glossary.txt artifacts: spells: extract: dnd/spells ``` Extractor binding `references` are the canonical lane-local location. The legacy lane-level `references` field remains supported as an alias; when both bind the same slot, `extract.references` wins: ```yaml pipelines: dnd-session: input: seriatim references: glossary: ./campaign/glossary.txt artifacts: spells: references: roster: ./campaign/legacy-roster.txt extract: module: dnd/spells references: party: ./campaign/session-party.txt ``` `chunk.references`, `merge.references`, and `normalize.references` are accepted in object-form bindings. They override pipeline-level defaults for slots declared by that target module. Extractor-local references apply only to the extractor, merger-local references apply only to the merger, and normalizer-local references apply only to the normalizer. Target-local reference fields use the same map shape at: - `pipelines..chunk.references` - `pipelines..artifacts..extract.references` - `pipelines..artifacts..merge.references` - `pipelines..artifacts..normalize.references` Each binding is valid only when that target module declares the slot. ## Module Bindings Every module binding may use shorthand: ```yaml input: seriatim ``` or object form: ```yaml chunk: module: dnd/scenes llm_profile: local-fast ``` Binding fields: - `module`: module key. - `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the Scriptorium prompt default select the profile. - `retries`: non-negative retry count for extra runtime attempts after the first attempt. Default: `0`. Supported on `chunk`, `extract`, `merge`, and `normalize` bindings. - `options`: optional module-specific settings. - `references`: optional reference bindings. Supported only for `chunk`, `extract`, `merge`, and `normalize` bindings. `input` and `output` bindings reject this field during validation. - `validators`: optional stage-local validator chain override. Supported only for `chunk`, `extract`, `merge`, and `normalize` bindings. Omit the field to use the production default chain; set `validators: []` to force an empty chain; set a non-empty list to use exactly those validators in configured order. During resolution, each selected module's registered option validator runs. Production input, chunk, and output bindings reject unknown or invalid options with the affected binding context. Validator bindings use the same shorthand or object module-binding form, but only these fields are supported: - `module`: validator key. - `llm_profile`: optional Scriptorium profile ID for LLM-backed validators. - `options`: optional validator-specific settings. Validator bindings reject `references`, `retries`, and nested `validators`. During resolution, deterministic validators reject explicit `llm_profile` values. Configured LLM-backed validators with explicit `llm_profile` values are validated against the configured Scriptorium profile source. Deterministic production validators do not call the LLM and must not set `llm_profile`. ## Implemented Production Modules | Slot | Key | Notes | | --- | --- | --- | | input | `seriatim` | Reads Seriatim transcript JSON. | | chunk | `generic` | Splits source units into ordered chunks. | | chunk | `dnd/scenes` | Uses an LLM to split transcript source units into D&D scenes. | | extract | `dnd/spells` | Extracts typed D&D spell-list artifacts. | | merge | `appendorder` | Combines typed artifacts in chunk order. | | normalize | `noop` | Passes merged typed artifacts through unchanged. | | output | `json` | Produces JSON output files for normalized `application/json` lanes. | ## Implemented Production Validators | Key | Execution | Notes | | --- | --- | --- | | `generic/always_accept` | deterministic | Accepts returned module output. | | `generic/always_reject` | deterministic | Rejects returned module output with reason `always_reject`. | | `generic/valid_json` | deterministic | Rejects payloads that are not syntactically valid JSON. | | `generic/valid_json_schema` | deterministic | Rejects invalid JSON or JSON that does not conform to the module response schema. | | `extract/dnd/spells/shape` | deterministic | Rejects malformed D&D spell-list artifacts. | | `extract/dnd/spells/source_refs` | deterministic | Rejects missing or invalid D&D spell source references. | | `extract/dnd/spells/source_relatedness` | deterministic | Emits warnings when a spell name is not found near its cited source text. | The production default chain for the `dnd/spells` extractor is: ```yaml validators: - generic/valid_json - generic/valid_json_schema - extract/dnd/spells/shape - extract/dnd/spells/source_refs - extract/dnd/spells/source_relatedness ``` No other production module currently has a default validator chain. Empty chains approve output by default. The `generic` chunker accepts: - `max_units`: positive integer, default `50`; - `overlap_units`: non-negative integer, default `0`, and must be less than `max_units`. The `dnd/scenes` chunker requires transcript source capabilities, calls the configured structured LLM runtime, and does not accept module options. It declares optional `players`, `party`, and `glossary` references for scene disambiguation, and accepts `roster` as a deprecated compatibility alias for `party`. The `dnd/spells` extractor declares optional campaign reference slots: - `players` - `party` - `glossary` - `roster` as a deprecated compatibility alias for `party` These campaign slots accept UTF-8 plain text, Markdown, YAML, or JSON reference files. The extractor also declares an optional `spell_catalog` slot that accepts one UTF-8 `application/json` overlay bundle no larger than 1 MiB. The slot does not allow multiple files. Its format is defined in the [spell-catalog overlay contract](integrations/dnd-spell-catalog-overlays.md). The extractor uses campaign references only as supporting disambiguation material; spell casts still must be present in the source transcript. ## State Surfaces The `output`, `cache`, and `debug` top-level fields select independent physical roots. Their layout, permissions, lifecycle, and sensitive-data handling are defined in [Operations](operations.md). ```yaml output: directory: ./notarius-output cache: chunk_plans: directory: "" mode: auto checkpoints: enabled: false directory: "" debug: directory: ./notarius-debug ``` `output.directory` is the durable output root. Its precedence is `--output-dir`, `NOTARIUS_OUTPUT_DIR`, the file value, then the default. `cache.chunk_plans.mode` accepts `auto`, `bypass`, or `refresh`. Its precedence is `--chunk_cache`, `NOTARIUS_CACHE_CHUNK_PLANS_MODE`, the file value, then `auto`. `auto` reuses a valid source-addressed plan and regenerates missing or invalid records; `bypass` performs no plan-cache I/O; `refresh` regenerates and publishes a plan after chunk validation. `cache.chunk_plans.directory` and `cache.checkpoints.directory` each name an exact cache-family root. Their precedence is the corresponding environment variable, the file value, then the family-specific per-user default. There is no CLI cache-root override. The defaults are `/notarius/chunk-plans` and `/notarius/checkpoints`; on Unix, `os.UserCacheDir` ordinarily uses an absolute `$XDG_CACHE_HOME` or falls back to `$HOME/.cache`. A relative `XDG_CACHE_HOME` is an error. `cache.checkpoints.enabled` defaults to `false`. When `true`, every run records checkpoint transitions and reusable approved results. When `false`, Notarius does not resolve or create the checkpoint root, and `--resume` is rejected. The `--resume` flag authorizes loading compatible checkpoints; it does not control recording. `debug.directory` chooses a root but never enables debug capture. Its precedence is `--debug-dir`, `NOTARIUS_DEBUG_DIR`, the file value, then the default. Only `--debug` requests a bundle; `--debug-dir` is valid only with `--debug`. Every supplied file, environment, and CLI value is validated even when a higher-precedence value wins. ## Version 2 To Version 3 Migration Version 2 files are rejected. Move each setting to the surface it controls and remove obsolete enablement and retention controls. This complete before/after example preserves an existing chunk-plan cache and checkpoint directory while choosing an output and debug root explicitly. ```yaml # Version 2 (no longer accepted) version: 2 workspace: directory: /srv/notarius/state resume: enabled: true debug: enabled: true chunk_cache: directory: /srv/notarius/chunk-plans mode: auto diagnostics: retention: always diagnostics: work_dir: /srv/notarius/inspection ``` ```yaml # Version 3 version: 3 output: directory: /srv/notarius/output cache: chunk_plans: directory: /srv/notarius/chunk-plans mode: auto checkpoints: enabled: true directory: /srv/notarius/state/checkpoints debug: directory: /srv/notarius/debug ``` Run the migrated configuration with `--resume` when checkpoint reuse is wanted, and with `--debug` when a debug bundle is wanted. Enabled checkpoint recording occurs with or without `--resume`. The removed fields are `workspace.directory`, `workspace.resume.enabled`, `workspace.debug.enabled`, `workspace.chunk_cache.mode`, `workspace.chunk_cache.directory`, `workspace.diagnostics.enabled`, `workspace.diagnostics.retention`, `diagnostics.work_dir`, and `diagnostics.retention`. The removed environment variables are `NOTARIUS_WORKSPACE_DIR`, `NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED`, `NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION`, `NOTARIUS_WORKSPACE_RESUME_ENABLED`, `NOTARIUS_WORKSPACE_DEBUG_ENABLED`, `NOTARIUS_WORKSPACE_CHUNK_CACHE_MODE`, `NOTARIUS_WORKSPACE_CHUNK_CACHE_DIR`, `NOTARIUS_WORK_DIR`, and `NOTARIUS_DIAGNOSTICS_RETENTION`. The chunk-cache variables are replaced by `NOTARIUS_CACHE_CHUNK_PLANS_MODE` and `NOTARIUS_CACHE_CHUNK_PLANS_DIR`; the former shared directory has no direct replacement. ## Validation Configuration validation checks: - supported config version and known YAML fields; - mutually exclusive `scriptorium.profile_dir` and `scriptorium.profile_file`; - non-empty, non-duplicated IDs after trimming; - positive global LLM concurrency; - supported stage-worker keys and an effective extract worker count in the inclusive range `1..concurrency.total_llm`; - non-empty output and debug directories; - a supported chunk-cache mode, Boolean checkpoint enablement, and state-surface directories without NUL bytes; - stale removed fields such as `llm_profiles`. Pipeline resolution additionally checks: - the pipeline ID exists; - at least one artifact lane is declared and selected; - lanes selected through the CLI exist in the resolved pipeline; - required module keys are present; - module keys are registered for the expected slot; - module capability requirements are satisfied; - non-empty validator overrides reference registered validator keys; - deterministic validators do not set `llm_profile`; - LLM-backed validators with explicit `llm_profile` values reference configured Scriptorium profile IDs; - bound reference slots are declared by selected chunk, extractor, merger, or normalizer targets; - required reference slots are bound for selected targets.