# Configuration This is the canonical reference for implemented Notarius configuration. Notarius reads YAML config files with `version: 2`. File config is applied over built-in defaults, then environment overrides are applied. ## Discovery Commands that accept `--config` load configuration in this order: 1. the `--config` path, when provided; 2. `NOTARIUS_CONFIG`, when set to a non-empty path; 3. `/usr/local/etc/notarius/config.yml`. If none is available, the command fails with a config file not found error. ## Minimal Example ```yaml version: 2 pipelines: dnd-session: input: seriatim references: party: ./dnd-spells-roster.txt glossary: ./dnd-spells-glossary.txt chunk: module: generic options: max_units: 50 artifacts: spells: extract: dnd/spells ``` The maintained fixture is [examples/dnd-spells.config.yml](../examples/dnd-spells.config.yml). ## Top-Level Fields - `version`: required. The only supported value is `2`. - `scriptorium`: optional Scriptorium profile source settings. - `pipelines`: optional map of pipeline IDs to pipeline definitions. - `concurrency`: optional global concurrency settings. - `workspace`: optional workspace settings for Notarius-owned local state. - `diagnostics`: optional diagnostics settings. Unknown YAML fields are rejected. The removed top-level `llm_profiles` field is rejected; execution profiles now come from Scriptorium. ## Defaults Built-in defaults: ```yaml concurrency: total_llm: 1 diagnostics: work_dir: /tmp/notarius retention: auto workspace: diagnostics: enabled: true resume: enabled: false debug: enabled: false ``` `workspace.directory` is unset by default. Without a workspace directory, diagnostics continue to use `/tmp/notarius`, and checkpoint and debug workspace features have no storage root. No pipelines are built in. A run requires a configured pipeline. If `scriptorium` is omitted, Notarius uses Scriptorium's built-in profile catalog. Prompt definitions may also name default profile IDs. The current D&D scene and spell prompts use Scriptorium prompt defaults when a module binding does not set `llm_profile`. ## Scriptorium Profiles `scriptorium` fields: - `profile_dir`: optional directory containing Scriptorium profile YAML files. - `profile_file`: optional Scriptorium profile YAML file. `profile_dir` and `profile_file` are mutually exclusive. Custom profiles overlay Scriptorium built-in profiles by profile ID. Scriptorium profile files use Scriptorium's profile schema. A minimal profile looks like: ```yaml id: local-fast endpoint: http://127.0.0.1:8080/v1 model: your-model api_key_env: SCRIPTORIUM_API_KEY timeout_seconds: 180 ``` Notarius does not accept raw API keys in Notarius config. For file-backed Scriptorium profiles, store the environment variable name in `api_key_env` and set that variable in the run environment. Scriptorium rejects raw `api_key` fields in profile YAML. ## Environment Overrides These environment variables are applied after the config file: - `NOTARIUS_CONFIG`: config discovery path. - `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency. - `NOTARIUS_WORKSPACE_DIR`: workspace directory. - `NOTARIUS_WORKSPACE_DIAGNOSTICS_ENABLED`: boolean diagnostics enablement. - `NOTARIUS_WORKSPACE_DIAGNOSTICS_RETENTION`: workspace diagnostics retention mode. - `NOTARIUS_WORKSPACE_RESUME_ENABLED`: boolean resume checkpointing enablement. - `NOTARIUS_WORKSPACE_DEBUG_ENABLED`: boolean debug artifact enablement. - `NOTARIUS_WORK_DIR`: deprecated diagnostics work directory compatibility override. - `NOTARIUS_DIAGNOSTICS_RETENTION`: deprecated diagnostics retention compatibility override. Integer environment values must parse as base-10 integers. Boolean environment values must parse as Go booleans such as `true`, `false`, `1`, or `0`. The removed `NOTARIUS_LLM_DEFAULT_*` variables are not read. Configure provider endpoint, model, and credential environment variable names through Scriptorium profiles. ## Pipelines A pipeline defines the fixed Notarius workflow: ```text input -> chunk -> extract -> merge -> normalize -> output ``` Pipeline fields: - `input`: required module binding. - `chunk`: optional module binding. Default module is `generic`. - `artifacts`: required for pipeline resolution. It maps artifact lane IDs to lane definitions. - `output`: optional module binding. Default module is `json`. - `references`: optional map of reference slot names to reference paths. These bindings are defaults for eligible pipeline targets that declare the matching slot. Artifact lane fields: - `extract`: required module binding. - `merge`: optional module binding. Default module is `appendorder`. - `normalize`: optional module binding. Default module is `noop`. - `validators`: deprecated lane-level validator list. Non-empty lists are rejected; use `extract.validators`, `merge.validators`, or `normalize.validators`. - `references`: optional compatibility alias for extractor reference bindings. Lane bindings override pipeline-level bindings for the same slot. `notarius run` and `notarius config validate --pipeline` resolve the pipeline against the production module catalog and fail fast for unknown or incompatible module keys. Reference bindings are validated against reference slots declared by eligible chunk, extract, merge, and normalize targets during pipeline resolution. Required slots must be bound after config defaults, target-local references, lane-level compatibility bindings, and run-time `--reference` or `--without-reference` overrides are applied. Config-relative paths are resolved relative to the config file; CLI reference paths are resolved relative to the current working directory. Materialized bound files must be UTF-8 text. Materialized reference provenance is recorded for chunk, extractor, merger, and normalizer targets, and runtime reference content is passed to the target that declares the slot. Reference media types are inferred from file extensions, recorded as canonical base media types, and checked only when a module declares `AcceptedMediaTypes`; unknown extensions are recorded as `application/octet-stream`. Reference content is not written to diagnostics, logs, errors, or manifests. Pipeline-level `references` are defaults. They are valid when at least one eligible target in the full configured pipeline declares the slot, including chunk, extractor, merger, and normalizer targets. During a run, they apply only to the selected targets that declare the slot: ```yaml pipelines: dnd-session: input: seriatim references: players: ./campaign/players.txt party: ./campaign/party-roster.txt glossary: ./campaign/glossary.txt artifacts: spells: extract: dnd/spells ``` Extractor binding `references` are the canonical lane-local location. The legacy lane-level `references` field remains supported as an alias; when both bind the same slot, `extract.references` wins: ```yaml pipelines: dnd-session: input: seriatim references: glossary: ./campaign/glossary.txt artifacts: spells: references: roster: ./campaign/legacy-roster.txt extract: module: dnd/spells references: party: ./campaign/session-party.txt ``` `chunk.references`, `merge.references`, and `normalize.references` are accepted in object-form bindings. They override pipeline-level defaults for slots declared by that target module. Extractor-local references apply only to the extractor, merger-local references apply only to the merger, and normalizer-local references apply only to the normalizer. Target-local reference fields use the same map shape at: - `pipelines..chunk.references` - `pipelines..artifacts..extract.references` - `pipelines..artifacts..merge.references` - `pipelines..artifacts..normalize.references` Each binding is valid only when that target module declares the slot. ## Module Bindings Every module binding may use shorthand: ```yaml input: seriatim ``` or object form: ```yaml chunk: module: dnd/scenes llm_profile: local-fast ``` Binding fields: - `module`: module key. - `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the Scriptorium prompt default select the profile. - `retries`: non-negative retry count for extra runtime attempts after the first attempt. The runner applies retries to `chunk`, `extract`, `merge`, and `normalize` bindings. - `options`: optional module-specific settings. - `references`: optional reference bindings. Supported only for `chunk`, `extract`, `merge`, and `normalize` bindings. `input` and `output` bindings reject this field during validation. - `validators`: optional stage-local validator chain override. Supported only for `chunk`, `extract`, `merge`, and `normalize` bindings. Omit the field to use the production default chain; set `validators: []` to force an empty chain; set a non-empty list to use exactly those validators in configured order. Validator bindings use the same shorthand or object module-binding form, but only these fields are supported: - `module`: validator key. - `llm_profile`: optional Scriptorium profile ID for LLM-backed validators. - `options`: optional validator-specific settings. Validator bindings reject `references`, `retries`, and nested `validators`. During resolution, deterministic validators reject explicit `llm_profile` values. The `--llm-profile` run flag overrides every effective LLM-capable module binding to use one Scriptorium profile ID: chunk, every selected lane extract, merge, and normalize binding. It does not override validator-specific `llm_profile` values. Configured LLM-backed validators with explicit `llm_profile` values are validated against the configured Scriptorium profile source. Deterministic production validators do not call the LLM and must not set `llm_profile`. ## Implemented Production Modules | Slot | Key | Notes | | --- | --- | --- | | input | `seriatim` | Reads Seriatim transcript JSON. | | chunk | `generic` | Splits source units into ordered chunks. | | chunk | `dnd/scenes` | Uses an LLM to split transcript source units into D&D scenes. | | extract | `dnd/spells` | Extracts D&D spell raw outputs. | | merge | `appendorder` | Merges JSON raw extract outputs in chunk order. | | normalize | `noop` | Passes merged raw outputs through unchanged. | | output | `json` | Produces JSON output files for normalized `application/json` lanes. | ## Implemented Production Validators | Key | Execution | Notes | | --- | --- | --- | | `generic/always_accept` | deterministic | Accepts returned module output. | | `generic/always_reject` | deterministic | Rejects returned module output with reason `always_reject`. | | `generic/valid_json` | deterministic | Rejects payloads that are not syntactically valid JSON. | | `generic/valid_json_schema` | deterministic | Rejects invalid JSON or JSON that does not conform to the module response schema. | | `extract/dnd/spells/shape` | deterministic | Rejects malformed D&D spell-cast JSON payloads. | | `extract/dnd/spells/source_refs` | deterministic | Rejects missing or invalid D&D spell source references. | | `extract/dnd/spells/source_relatedness` | deterministic | Emits warnings when a spell name is not found near its cited source text. | The production default chain for the `dnd/spells` extractor is: ```yaml validators: - generic/valid_json - generic/valid_json_schema - extract/dnd/spells/shape - extract/dnd/spells/source_refs - extract/dnd/spells/source_relatedness ``` No other production module currently has a default validator chain. Empty chains approve output by default. The `generic` chunker accepts: - `max_units`: positive integer, default `50`; - `overlap_units`: non-negative integer, default `0`, and must be less than `max_units`. The `dnd/scenes` chunker requires transcript source capabilities, calls the configured structured LLM runtime, and does not accept module options. It declares optional `players`, `party`, and `glossary` references for scene disambiguation, and accepts `roster` as a deprecated compatibility alias for `party`. The `dnd/spells` extractor declares optional reference slots: - `players` - `party` - `glossary` - `roster` as a deprecated compatibility alias for `party` Both modules accept UTF-8 plain text, Markdown, YAML, or JSON reference files. The extractor uses references only as supporting disambiguation material; spell casts still must be present in the source transcript. ## Workspace `workspace` fields: - `directory`: optional workspace root for Notarius-owned local state. - `diagnostics.enabled`: set to `false` to skip diagnostics run directories and diagnostics artifact writes. Default: `true`. - `diagnostics.retention`: `auto`, `always`, or `never`. - `resume.enabled`: boolean resume checkpointing setting. Default: `false`. - `debug.enabled`: boolean debug artifact setting. Default: `false`. Use `/var/lib/notarius` as the standard production workspace directory. For local development, prefer a project-local ignored path such as `./.notarius/workspace`. ```yaml workspace: directory: /var/lib/notarius diagnostics: enabled: true retention: auto resume: enabled: false debug: enabled: false ``` When `workspace.directory` is set, diagnostics are written under `/diagnostics/`. When both `workspace.directory` and `workspace.resume.enabled` are set, runs write stage-owned checkpoint artifacts under `/checkpoints/`. `notarius run --resume` can reuse valid checkpoints from a compatible invocation. Checkpoints may contain source text, intermediate raw outputs, rejected outputs, metadata, and warnings. Protect the workspace as sensitive local state. When both `workspace.directory` and `workspace.debug.enabled` are set, runs write per-invocation debug artifacts under `/debug//`. Debug artifacts may contain source material, reference material, prompt inputs, model outputs, validation payloads, and other sensitive content. Debug is disabled by default. `workspace.resume.enabled` and `workspace.debug.enabled` are independent. Enabling one does not enable the other. ## Diagnostics Preferred workspace diagnostics fields: - `workspace.directory`: workspace root for Notarius-owned local state. - `workspace.diagnostics.enabled`: set to `false` to skip creating diagnostics run directories and diagnostics artifacts. Default: `true`. - `workspace.diagnostics.retention`: `auto`, `always`, or `never`. When `workspace.directory` is set, diagnostics use `/diagnostics` as their work directory. `workspace.diagnostics.retention` overrides legacy diagnostics retention when set. `diagnostics` fields: - `work_dir`: deprecated compatibility directory for per-run diagnostics. Default: `/tmp/notarius`. - `retention`: deprecated compatibility retention mode. `auto`, `always`, or `never`. Empty uses `auto`. Existing `diagnostics.work_dir`, `diagnostics.retention`, `NOTARIUS_WORK_DIR`, and `NOTARIUS_DIAGNOSTICS_RETENTION` inputs remain supported for compatibility. New configuration should use `workspace.directory` and `workspace.diagnostics.retention` instead. `auto` retains diagnostics for failed runs and successful runs with warnings. `always` retains diagnostics for every run. `never` removes diagnostics for successful runs without regard to warnings; failed runs are retained. The `--diagnostics-dir` run flag overrides the effective diagnostics work directory for that invocation. It affects diagnostics only and does not change the workspace directory. ## Validation Configuration validation checks: - supported config version and known YAML fields; - mutually exclusive `scriptorium.profile_dir` and `scriptorium.profile_file`; - non-empty, non-duplicated IDs after trimming; - positive global LLM concurrency; - supported diagnostics retention and non-empty work directory; - stale removed fields such as `llm_profiles`. Pipeline resolution additionally checks: - the pipeline ID exists; - at least one artifact lane is declared and selected; - selected lanes exist when `--only` is used; - required module keys are present; - module keys are registered for the expected slot; - module capability requirements are satisfied; - non-empty validator overrides reference registered validator keys; - deterministic validators do not set `llm_profile`; - LLM-backed validators with explicit `llm_profile` values reference configured Scriptorium profile IDs; - bound reference slots are declared by selected chunk, extractor, merger, or normalizer targets; - required reference slots are bound for selected targets.