notarius/docs/config.md

# Configuration

This is the canonical reference for Notarius configuration. Configuration files
are YAML and must declare version 3. They select pipelines and their modules;
the [CLI reference](cli.md) owns invocation syntax, and
[Operations](operations.md) owns run-state procedures.

## Configuration Discovery And Precedence

Commands that load configuration choose a file in this order:

1. a non-empty **--config** CLI value;
2. a non-empty **NOTARIUS_CONFIG** environment value;
3. the installed default file at **/usr/local/etc/notarius/config.yml**, when
   it exists.

The command fails if none of these paths provides a configuration file.

For configuration values, precedence is:

1. built-in defaults;
2. the selected YAML file;
3. supported operational environment variables; and
4. the CLI run overrides that apply to a command.

Environment variables do not provide a second configuration schema. They only
override the fields listed below.

## Maintained Examples

- [Minimal D&D configuration](../examples/dnd-minimal.config.yml) is a
  single-lane Seriatim-to-spell pipeline.
- [Complete D&D configuration](../examples/dnd-complete.config.yml) uses
  ordered steps, all implemented D&D lanes, generated references, state
  settings, and bounded LLM concurrency.

Use these complete files as starting points rather than combining the
illustrative fragments in this reference.

## File Shape And Defaults

Unknown fields, duplicate mapping keys, empty identifiers, and identifiers that
become duplicates after trimming whitespace are rejected. Every top-level field
other than **version** is optional.

| Field | Type | Default | Rules |
| --- | --- | --- | --- |
| **version** | integer | none | Required; must be 3. |
| **scriptorium** | object | none | Profile source configuration. |
| **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. |
| **concurrency** | object | see below | Global LLM and extraction limits. |
| **output** | object | see below | Published output settings. |
| **cache** | object | see below | Chunk-plan and checkpoint settings. |
| **debug** | object | see below | Debug-bundle root only; it does not enable capture. |

Built-in defaults are:

| Field | Default |
| --- | --- |
| **concurrency.total_llm** | 1 |
| **concurrency.stage_workers.extract** | Effective **total_llm** |
| **output.directory** | **./notarius-output** |
| **cache.chunk_plans.mode** | **auto** |
| **cache.chunk_plans.directory** | Empty, selecting the per-user chunk-plan root |
| **cache.checkpoints.enabled** | false |
| **cache.checkpoints.directory** | Empty, selecting the per-user checkpoint root |
| **debug.directory** | **./notarius-debug** |

An empty cache directory in YAML deliberately selects the corresponding
per-user root. An explicit empty output or debug directory is invalid.

## Scriptorium Profiles

The optional **scriptorium** object selects one source of profile definitions:

| Field | Type | Rules |
| --- | --- | --- |
| **profile_dir** | string | Non-empty directory containing profile files. |
| **profile_file** | string | Non-empty profile file. |

Set at most one of these fields. Profile IDs used by a binding must be available
from the selected Scriptorium profile source when the pipeline is resolved.
Keep credentials out of this file: configure a profile to read its credential
from an environment variable, then set that environment variable only in the
run environment.

## Operational Environment Variables

These variables are applied after YAML values:

| Variable | Overrides | Rules |
| --- | --- | --- |
| **NOTARIUS_TOTAL_LLM_CONCURRENCY** | **concurrency.total_llm** | Integer. |
| **NOTARIUS_STAGE_WORKERS_EXTRACT** | **concurrency.stage_workers.extract** | Integer. |
| **NOTARIUS_OUTPUT_DIR** | **output.directory** | Non-empty path. |
| **NOTARIUS_CACHE_CHUNK_PLANS_MODE** | **cache.chunk_plans.mode** | **auto**, **bypass**, or **refresh**. |
| **NOTARIUS_CACHE_CHUNK_PLANS_DIR** | **cache.chunk_plans.directory** | Non-empty path. |
| **NOTARIUS_CACHE_CHECKPOINTS_DIR** | **cache.checkpoints.directory** | Non-empty path. |
| **NOTARIUS_DEBUG_DIR** | **debug.directory** | Non-empty path. |

Integer values are trimmed then parsed as base-10 integers. Directory and
output values reject NUL characters. **NOTARIUS_CONFIG** participates only in
configuration discovery.

## Concurrency, Output, Cache, And Debug

~~~yaml
concurrency:
  total_llm: 2
  stage_workers:
    extract: 2
output:
  directory: ./notarius-output
cache:
  chunk_plans:
    mode: auto
    directory: ./notarius-cache/chunk-plans
  checkpoints:
    enabled: true
    directory: ./notarius-cache/checkpoints
debug:
  directory: ./notarius-debug
~~~

**concurrency.total_llm** must be greater than zero. The only supported
**concurrency.stage_workers** key is **extract**; its value must be from 1
through **total_llm**. When omitted, it is recalculated from the effective
**total_llm** after YAML and environment precedence.

**cache.chunk_plans.mode** accepts **auto**, **bypass**, or **refresh**.
**cache.checkpoints.enabled** is a boolean. The CLI can override the output
directory and chunk-plan mode for one run; see [CLI reference](cli.md#run).

## Pipelines

Each **pipelines** entry has a unique, non-empty ID and the following shape:

~~~yaml
pipelines:
  dnd-session:
    input: seriatim
    chunk: generic
    output: json
    artifacts:
      spells:
        extract: dnd/spells
        merge: appendorder
        normalize: dnd/spells
~~~

| Field | Type | Default | Rules |
| --- | --- | --- | --- |
| **input** | module binding | none | Required. |
| **chunk** | module binding | **generic** | Optional. |
| **output** | module binding | **json** | Optional. |
| **artifacts** | map | none | Compact single-step lane map. |
| **steps** | list | none | Ordered lane definitions. Mutually exclusive with **artifacts**. |
| **references** | map | none | External reference defaults for eligible targets. |

Use either **artifacts** or **steps**. The compact **artifacts** form is an
implicit single step. An explicit **steps** list must be non-empty; every step
needs a unique non-empty **id**, an **artifacts** map, and may have
**references**. A lane ID must not appear more than once in a pipeline,
including across explicit steps.

A lane has these fields:

| Field | Type | Default | Rules |
| --- | --- | --- | --- |
| **extract** | module binding | none | Required. |
| **merge** | module binding | **appendorder** | Optional. |
| **normalize** | module binding | **noop** | Optional. |
| **references** | map | none | Supported compatibility alias for **extract.references**. |
| **validators** | list | none | Non-empty lane-level lists are rejected. Set validator overrides on a binding instead. |

The lane-level **references** alias remains accepted. When the alias and
**extract.references** bind the same slot, **extract.references** wins. Use
the binding-local form in new configurations.

## Module Bindings And Validators

Use a module key directly when no other binding fields are needed:

~~~yaml
input: seriatim
~~~

Use an object for fields:

~~~yaml
extract:
  module: dnd/spells
  llm_profile: gemini-2-flash
  retries: 2
  references:
    spell_catalog: ./dnd-spell-catalog.json
~~~

| Binding field | Type | Default | Rules |
| --- | --- | --- | --- |
| **module** | string | none | Required for an object binding. Must be a registered compatible key. |
| **llm_profile** | string | none | Optional non-empty Scriptorium profile ID. |
| **retries** | integer | 0 | Non-negative additional attempts for chunk, extract, merge, and normalize bindings. |
| **options** | object | none | Must satisfy the selected module. |
| **references** | map | none | Valid only on chunk, extract, merge, and normalize bindings. |
| **validators** | list | production chain | Valid only on chunk, extract, merge, and normalize bindings. |

Omitting **validators** uses the registered chain. **validators: []** selects
an empty chain; a non-empty list replaces the chain in the listed order.
Validator bindings accept only **module**, **llm_profile**, and **options**.
They reject **references**, **retries**, and nested **validators**. Deterministic
validators reject an explicit **llm_profile**.

The **json** output module accepts one option:

~~~yaml
output:
  module: json
  options:
    include_chunk_map: true
~~~

**include_chunk_map** is a boolean and defaults to false. It adds the accepted
chunk map when one exists; its wire format is defined in the
[chunk-map contract](integrations/chunk-map.md).

## References And Ordered Handoffs

Reference maps bind named slots that the selected target declares. A scalar is
an external path. Pipeline-level maps accept only external paths; step-local
and binding-local maps may also select a normalized artifact from an earlier
step:

~~~yaml
steps:
  - id: describe-session
    artifacts:
      npcs:
        extract: dnd/npcs
        normalize: dnd/npcs
  - id: extract-events
    references:
      npcs:
        artifact:
          step: describe-session
          lane: npcs
    artifacts:
      spells:
        extract: dnd/spells
        normalize: dnd/spells
~~~

An artifact selector contains only **step** and **lane**. The producer must be
an earlier step and the selected artifact must be compatible with the consumer
slot. A generated binding supplies one accepted normalized artifact; it does
not name a file. A configured generated dependency remains required even when
that consumer slot is otherwise optional.

Pipeline references are defaults. A matching step-local or binding-local
external path overrides a pipeline default. Required slots must be bound after
these configuration values and any CLI reference overrides are applied.
Reference paths in YAML are resolved relative to the configuration file.

### D&D Reference Slots

The following slot names are accepted by the implemented D&D modules when the
selected target declares them:

| Slot | Source and use |
| --- | --- |
| **party** | Optional text campaign context. This is the canonical party-roster spelling. |
| **roster** | Accepted compatibility alias for **party**. |
| **players** | Optional text player context. |
| **glossary** | Optional text campaign glossary. |
| **spell_catalog** | Optional JSON spell-catalog overlay for spell extraction and normalization. See [spell-catalog overlays](integrations/dnd-spell-catalog-overlays.md). |
| **npcs** | Normalized NPC registry. Optional for spells and combat turns; required for NPC interactions. |
| **scene_descriptions** | Required normalized scene-description artifact for combat-turn extraction. |

Scene descriptions accept **party**, **players**, and **glossary**, but not
**roster**. NPC interactions require **npcs** for both extraction and
normalization. Combat turns require **scene_descriptions** for extraction; the
normalized combat-turn module may use optional **npcs**. The complete example
shows generated **npcs** and **scene_descriptions** bindings.

## Production Module Keys

| Kind | Keys |
| --- | --- |
| Input | **seriatim** |
| Chunk | **generic**, **dnd/scenes** |
| Extract | **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions** |
| Merge | **appendorder** |
| Normalize | **noop**, **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions** |
| Output | **json** |

The D&D artifact contracts define each emitted schema:
[spells](integrations/dnd-spell-artifacts.md),
[NPCs](integrations/dnd-npc-artifacts.md),
[NPC interactions](integrations/dnd-npc-interaction-artifacts.md),
[combat turns](integrations/dnd-combat-turn-artifacts.md),
[item events](integrations/dnd-item-event-artifacts.md), and
[scene descriptions](integrations/dnd-scene-description-artifacts.md).

## Production Validator Keys And Default Chains

Available validator keys are:

| Family | Keys |
| --- | --- |
| Generic | **generic/always_accept**, **generic/always_reject**, **generic/valid_json**, **generic/valid_json_schema** |
| Spells | **extract/dnd/spells/shape**, **extract/dnd/spells/catalog**, **extract/dnd/spells/source_refs**, **extract/dnd/spells/source_relatedness** |
| NPCs | **extract/dnd/npcs/shape**, **extract/dnd/npcs/source_refs**, **extract/dnd/npcs/source_relatedness**, **normalize/dnd/npcs/identity** |
| Combat turns | **extract/dnd/combat-turns/shape**, **extract/dnd/combat-turns/source_refs**, **extract/dnd/combat-turns/source_relatedness**, **normalize/dnd/combat-turns/invariants** |
| Item events | **extract/dnd/item-events/shape**, **extract/dnd/item-events/source_refs**, **extract/dnd/item-events/source_relatedness**, **normalize/dnd/item-events/invariants** |
| NPC interactions | **extract/dnd/npc-interactions/shape**, **extract/dnd/npc-interactions/registry**, **extract/dnd/npc-interactions/source_refs**, **extract/dnd/npc-interactions/source_relatedness**, **normalize/dnd/npc-interactions/invariants** |
| Scene descriptions | **extract/dnd/scene-descriptions/shape**, **extract/dnd/scene-descriptions/source_refs**, **extract/dnd/scene-descriptions/source_relatedness**, **normalize/dnd/scene-descriptions/invariants** |

When no override is configured, production D&D bindings use the following
ordered chains. Each row lists extract then normalize; spell chains are the
same at both stages.

| Lane | Extract | Normalize |
| --- | --- | --- |
| Spells | generic/valid_json, extract/dnd/spells/shape, extract/dnd/spells/catalog, extract/dnd/spells/source_refs, generic/valid_json_schema, extract/dnd/spells/source_relatedness | Same as extract |
| NPCs | generic/valid_json, extract/dnd/npcs/shape, extract/dnd/npcs/source_refs, generic/valid_json_schema, extract/dnd/npcs/source_relatedness | generic/valid_json, extract/dnd/npcs/shape, normalize/dnd/npcs/identity, extract/dnd/npcs/source_refs, generic/valid_json_schema, extract/dnd/npcs/source_relatedness |
| Combat turns | generic/valid_json, extract/dnd/combat-turns/shape, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness | generic/valid_json, extract/dnd/combat-turns/shape, normalize/dnd/combat-turns/invariants, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness |
| Item events | generic/valid_json, extract/dnd/item-events/shape, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness | generic/valid_json, extract/dnd/item-events/shape, normalize/dnd/item-events/invariants, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness |
| NPC interactions | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, normalize/dnd/npc-interactions/invariants, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness |
| Scene descriptions | generic/valid_json, extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness | generic/valid_json, extract/dnd/scene-descriptions/shape, normalize/dnd/scene-descriptions/invariants, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness |

Chains are only registered for the D&D extract and normalize modules shown
above; select an explicit override when a different compatible chain is
required.

## Validation

Validate a file and one pipeline before running it:

~~~sh
go run ./cmd/notarius config validate \
  --config examples/dnd-minimal.config.yml \
  --pipeline dnd-session
~~~

Configuration validation rejects invalid YAML, unsupported fields, invalid
defaults or environment overrides, incompatible module keys, unknown options,
invalid reference bindings, missing required reference slots, invalid validator
overrides, and incompatible generated artifact handoffs. Use
[pipelines list](cli.md#pipelines-list) to inspect configured IDs.