notarius/docs/config.md

# Configuration

This is the canonical reference for Notarius configuration. Configuration files
are YAML and must declare version 4. They select pipelines and their modules;
the [CLI reference](cli.md) owns invocation syntax, and
[Operations](operations.md) owns run-state procedures.

## Configuration Discovery And Precedence

Commands that load configuration choose a file in this order:

1. a non-empty **--config** CLI value;
2. a non-empty **NOTARIUS_CONFIG** environment value;
3. the installed default file at **/usr/local/etc/notarius/config.yml**, when
   it exists.

The command fails if none of these paths provides a configuration file.

For configuration values, precedence is:

1. built-in defaults;
2. the selected YAML file;
3. supported operational environment variables; and
4. the CLI run overrides that apply to a command.

Environment variables do not provide a second configuration schema. They only
override the fields listed below.

## Maintained Examples

- [Minimal D&D configuration](../examples/dnd-minimal.config.yml) is a
  single-lane Seriatim-to-spell pipeline.
- [Complete D&D configuration](../examples/dnd-complete.config.yml) uses
  ordered steps, all implemented D&D lanes, generated references, state
  settings, and bounded LLM concurrency.

Use these complete files as starting points rather than combining the
illustrative fragments in this reference.

## File Shape And Defaults

Unknown fields, duplicate mapping keys, empty identifiers, and identifiers that
become duplicates after trimming whitespace are rejected. Every top-level field
other than **version** is optional.

| Field | Type | Default | Rules |
| --- | --- | --- | --- |
| **version** | integer | none | Required; must be 4. |
| **promptkit** | object | none | Profile source and optional local-backend configuration. |
| **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. |
| **concurrency** | object | see below | Global LLM and extraction limits. |
| **output** | object | see below | Published output settings. |
| **cache** | object | see below | Chunk-plan and checkpoint settings. |
| **debug** | object | see below | Debug-bundle root only; it does not enable capture. |

Built-in defaults are:

| Field | Default |
| --- | --- |
| **concurrency.total_llm** | 1 |
| **concurrency.stage_workers.extract** | Effective **total_llm** |
| **output.directory** | **./notarius-output** |
| **cache.chunk_plans.mode** | **auto** |
| **cache.chunk_plans.directory** | Empty, selecting the per-user chunk-plan root |
| **cache.checkpoints.enabled** | false |
| **cache.checkpoints.directory** | Empty, selecting the per-user checkpoint root |
| **debug.directory** | **./notarius-debug** |

An empty cache directory in YAML deliberately selects the corresponding
per-user root. An explicit empty output or debug directory is invalid.

## PromptKit Profiles

The optional **promptkit** object selects one source of profile definitions and
may register one conventional local OpenAI-compatible backend:

~~~yaml
version: 4

promptkit:
  profile_dir: ./profiles
  # profile_file: ./profiles.yml
  local_backend:
    endpoint: http://localhost:8000/v1
    concurrency_limit: 2
~~~

| Field | Type | Rules |
| --- | --- | --- |
| **profile_dir** | string | Non-empty directory containing profile files. |
| **profile_file** | string | Non-empty profile file. |
| **local_backend** | object | Optional registration for the conventional PromptKit backend ID **local**. |
| **local_backend.endpoint** | string | Required when **local_backend** is present; absolute HTTP or HTTPS URL with a host. |
| **local_backend.concurrency_limit** | integer | Optional non-negative limit; defaults to 0. |

Set at most one of **profile_dir** and **profile_file**. Profile IDs used by a
binding must be available from the selected PromptKit profile source when the
pipeline is resolved. The optional local registration may coexist with either
profile source or with PromptKit's built-in profiles.

When **local_backend** is present, its endpoint is trimmed and must use HTTP or
HTTPS case-insensitively, be absolute, and have a non-empty host. URL paths are
allowed. User information, queries, and fragments are rejected. A zero
**concurrency_limit** leaves the local backend unrestricted inside PromptKit;
a positive value limits simultaneous local generations. The application-wide
**concurrency.total_llm** limit still applies in both cases. Neither local
backend field has an environment override. Omitting **local_backend** registers
nothing and preserves existing built-in and endpoint-only profile behavior.

A file-backed PromptKit profile selects the registration by its case-sensitive
backend ID:

~~~yaml
id: local-summary
backend: local
model: example-model
~~~

Keep credentials out of the local-backend object. A PromptKit profile may name
its credential environment variable through `api_key_env`; set that variable
only in the run environment. PromptKit owns the
[pinned profile-file format](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.3.0/docs/formats.md).
The [PromptKit upstream boundary](integrations/pkg-promptkit.md) identifies the
supported package API, and [Operations](operations.md#operational-limits)
describes the effective concurrency layers.

## Migrating Version 3 Configuration

Version 3 files are not decoded or rewritten. Change **version: 3** to
**version: 4** and rename the top-level **scriptorium:** section to
**promptkit:**. Version 4 decoding is strict, so a remaining **scriptorium**
field is rejected as unknown.

## Operational Environment Variables

These variables are applied after YAML values:

| Variable | Overrides | Rules |
| --- | --- | --- |
| **NOTARIUS_TOTAL_LLM_CONCURRENCY** | **concurrency.total_llm** | Integer. |
| **NOTARIUS_STAGE_WORKERS_EXTRACT** | **concurrency.stage_workers.extract** | Integer. |
| **NOTARIUS_OUTPUT_DIR** | **output.directory** | Non-empty path. |
| **NOTARIUS_CACHE_CHUNK_PLANS_MODE** | **cache.chunk_plans.mode** | **auto**, **bypass**, or **refresh**. |
| **NOTARIUS_CACHE_CHUNK_PLANS_DIR** | **cache.chunk_plans.directory** | Non-empty path. |
| **NOTARIUS_CACHE_CHECKPOINTS_DIR** | **cache.checkpoints.directory** | Non-empty path. |
| **NOTARIUS_DEBUG_DIR** | **debug.directory** | Non-empty path. |

Integer values are trimmed then parsed as base-10 integers. Directory and
output values reject NUL characters. **NOTARIUS_CONFIG** participates only in
configuration discovery.

## Concurrency, Output, Cache, And Debug

~~~yaml
concurrency:
  total_llm: 2
  stage_workers:
    extract: 2
output:
  directory: ./notarius-output
cache:
  chunk_plans:
    mode: auto
    directory: ./notarius-cache/chunk-plans
  checkpoints:
    enabled: true
    directory: ./notarius-cache/checkpoints
debug:
  directory: ./notarius-debug
~~~

**concurrency.total_llm** must be greater than zero. The only supported
**concurrency.stage_workers** key is **extract**; its value must be from 1
through **total_llm**. When omitted, it is recalculated from the effective
**total_llm** after YAML and environment precedence.

**cache.chunk_plans.mode** accepts **auto**, **bypass**, or **refresh**.
**cache.checkpoints.enabled** is a boolean. The CLI can override the output
directory and chunk-plan mode for one run; see [CLI reference](cli.md#run).

## Pipelines

Each **pipelines** entry has a unique, non-empty ID and the following shape:

~~~yaml
pipelines:
  dnd-session:
    input: seriatim
    chunk: generic
    output: json
    artifacts:
      spells:
        extract: dnd/spells
        merge: appendorder
        normalize: dnd/spells
~~~

| Field | Type | Default | Rules |
| --- | --- | --- | --- |
| **input** | module binding | none | Required. |
| **chunk** | module binding | **generic** | Optional. |
| **output** | module binding | **json** | Optional. |
| **artifacts** | map | none | Compact single-step lane map. |
| **steps** | list | none | Ordered lane definitions. Mutually exclusive with **artifacts**. |
| **references** | map | none | External reference defaults for eligible targets. |

Use either **artifacts** or **steps**. The compact **artifacts** form is an
implicit single step. An explicit **steps** list must be non-empty; every step
needs a unique non-empty **id**, an **artifacts** map, and may have
**references**. A lane ID must not appear more than once in a pipeline,
including across explicit steps.

A lane has these fields:

| Field | Type | Default | Rules |
| --- | --- | --- | --- |
| **extract** | module binding | none | Required. |
| **merge** | module binding | **appendorder** | Optional. |
| **normalize** | module binding | **noop** | Optional. |
| **references** | map | none | Supported compatibility alias for **extract.references**. |
| **validators** | list | none | Non-empty lane-level lists are rejected. Set validator overrides on a binding instead. |

The lane-level **references** alias remains accepted. When the alias and
**extract.references** bind the same slot, **extract.references** wins. Use
the binding-local form in new configurations.

## Module Bindings And Validators

Use a module key directly when no other binding fields are needed:

~~~yaml
input: seriatim
~~~

Use an object for fields:

~~~yaml
extract:
  module: dnd/spells
  llm_profile: gemini-2-flash
  retries: 2
  references:
    spell_catalog: ./dnd-spell-catalog.json
~~~

| Binding field | Type | Default | Rules |
| --- | --- | --- | --- |
| **module** | string | none | Required for an object binding. Must be a registered compatible key. |
| **llm_profile** | string | none | Optional non-empty PromptKit profile ID. |
| **retries** | integer | 0 | Non-negative additional attempts for chunk, extract, merge, and normalize bindings. |
| **options** | object | none | Must satisfy the selected module. |
| **references** | map | none | Valid only on chunk, extract, merge, and normalize bindings. |
| **validators** | list | production chain | Valid only on chunk, extract, merge, and normalize bindings. |

Omitting **validators** uses the registered chain. **validators: []** selects
an empty chain; a non-empty list replaces the chain in the listed order.
Validator bindings accept only **module**, **llm_profile**, and **options**.
They reject **references**, **retries**, and nested **validators**. Deterministic
validators reject an explicit **llm_profile**.

The **json** output module accepts optional **include_chunk_map** and
**evidence_context** settings:

~~~yaml
output:
  module: json
  options:
    include_chunk_map: true
    evidence_context:
      enabled: true
      window_units: 3
      lanes:
        - npcs
        - spells
~~~

**include_chunk_map** is a boolean and defaults to false. It adds the accepted
chunk map when one exists; its wire format is defined in the
[chunk-map contract](integrations/chunk-map.md).

Omitting **evidence_context** disables evidence publication. When present, it
is an object with these strict fields:

| Field | Type | Rules |
| --- | --- | --- |
| **enabled** | boolean | Required. `false` permits no other evidence fields. |
| **lanes** | array of strings | Required and non-empty when enabled. Each value is trimmed and must be unique; every value must name a configured pipeline lane. |
| **window_units** | non-negative integer | Optional when enabled; defaults to 3. Zero retains only directly cited units. |

Unknown outer or nested option fields are rejected, as are incompatible YAML
types. The allowlist remains valid when a run uses lane filtering: a configured
lane that is not active for that invocation simply contributes no evidence.
Evidence publication is opt-in because it can persist source text and metadata.
Its payload contract is [Published Evidence Context](integrations/evidence-context.md).

## References And Ordered Handoffs

Reference maps bind named slots that the selected target declares. A scalar is
an external path. Pipeline-level maps accept only external paths; step-local
and binding-local maps may also select a normalized artifact from an earlier
step:

~~~yaml
steps:
  - id: describe-session
    artifacts:
      npcs:
        extract: dnd/npcs
        normalize: dnd/npcs
  - id: extract-events
    references:
      npcs:
        artifact:
          step: describe-session
          lane: npcs
    artifacts:
      spells:
        extract: dnd/spells
        normalize: dnd/spells
~~~

An artifact selector contains only **step** and **lane**. The producer must be
an earlier step and the selected artifact must be compatible with the consumer
slot. A generated binding supplies one accepted normalized artifact; it does
not name a file. A configured generated dependency remains required even when
that consumer slot is otherwise optional.

Pipeline references are defaults. A matching step-local or binding-local
external path overrides a pipeline default. Required slots must be bound after
these configuration values and any CLI reference overrides are applied.
Reference paths in YAML are resolved relative to the configuration file.

### D&D Reference Slots

The following slot names are accepted by the implemented D&D modules when the
selected target declares them:

| Slot | Source and use |
| --- | --- |
| **party** | Optional text campaign context. This is the canonical party-roster spelling. |
| **roster** | Accepted compatibility alias for **party**. |
| **players** | Optional text player context. |
| **glossary** | Optional text campaign glossary. |
| **spell_catalog** | Optional JSON spell-catalog overlay for spell extraction and normalization. See [spell-catalog overlays](integrations/dnd-spell-catalog-overlays.md). |
| **npcs** | Normalized NPC registry. Optional for spells and combat turns; required for NPC interactions. |
| **scene_descriptions** | Required normalized scene-description artifact for combat-turn extraction. |

Scene descriptions accept **party**, **players**, and **glossary**, but not
**roster**. NPC interactions require **npcs** for both extraction and
normalization. Combat turns require **scene_descriptions** for extraction; the
normalized combat-turn module may use optional **npcs**. The complete example
shows generated **npcs** and **scene_descriptions** bindings.

## Production Module Keys

| Kind | Keys |
| --- | --- |
| Input | **seriatim** |
| Chunk | **generic**, **dnd/scenes** |
| Extract | **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions** |
| Merge | **appendorder** |
| Normalize | **noop**, **dnd/spells**, **dnd/npcs**, **dnd/combat-turns**, **dnd/item-events**, **dnd/npc-interactions**, **dnd/scene-descriptions** |
| Output | **json** |

The D&D artifact contracts define each emitted schema:
[spells](integrations/dnd-spell-artifacts.md),
[NPCs](integrations/dnd-npc-artifacts.md),
[NPC interactions](integrations/dnd-npc-interaction-artifacts.md),
[combat turns](integrations/dnd-combat-turn-artifacts.md),
[item events](integrations/dnd-item-event-artifacts.md), and
[scene descriptions](integrations/dnd-scene-description-artifacts.md).

## Production Validator Keys And Default Chains

Available validator keys are:

| Family | Keys |
| --- | --- |
| Generic | **generic/always_accept**, **generic/always_reject**, **generic/valid_json**, **generic/valid_json_schema** |
| Spells | **extract/dnd/spells/shape**, **extract/dnd/spells/catalog**, **extract/dnd/spells/source_refs**, **extract/dnd/spells/source_relatedness** |
| NPCs | **extract/dnd/npcs/shape**, **extract/dnd/npcs/source_refs**, **extract/dnd/npcs/source_relatedness**, **normalize/dnd/npcs/identity** |
| Combat turns | **extract/dnd/combat-turns/shape**, **extract/dnd/combat-turns/source_refs**, **extract/dnd/combat-turns/source_relatedness**, **normalize/dnd/combat-turns/invariants** |
| Item events | **extract/dnd/item-events/shape**, **extract/dnd/item-events/source_refs**, **extract/dnd/item-events/source_relatedness**, **normalize/dnd/item-events/invariants** |
| NPC interactions | **extract/dnd/npc-interactions/shape**, **extract/dnd/npc-interactions/registry**, **extract/dnd/npc-interactions/source_refs**, **extract/dnd/npc-interactions/source_relatedness**, **normalize/dnd/npc-interactions/invariants** |
| Scene descriptions | **extract/dnd/scene-descriptions/shape**, **extract/dnd/scene-descriptions/source_refs**, **extract/dnd/scene-descriptions/source_relatedness**, **normalize/dnd/scene-descriptions/invariants** |

When no override is configured, production D&D bindings use the following
ordered chains. Each row lists extract then normalize; spell chains are the
same at both stages.

| Lane | Extract | Normalize |
| --- | --- | --- |
| Spells | generic/valid_json, extract/dnd/spells/shape, extract/dnd/spells/catalog, extract/dnd/spells/source_refs, generic/valid_json_schema, extract/dnd/spells/source_relatedness | Same as extract |
| NPCs | generic/valid_json, extract/dnd/npcs/shape, extract/dnd/npcs/source_refs, generic/valid_json_schema, extract/dnd/npcs/source_relatedness | generic/valid_json, extract/dnd/npcs/shape, normalize/dnd/npcs/identity, extract/dnd/npcs/source_refs, generic/valid_json_schema, extract/dnd/npcs/source_relatedness |
| Combat turns | generic/valid_json, extract/dnd/combat-turns/shape, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness | generic/valid_json, extract/dnd/combat-turns/shape, normalize/dnd/combat-turns/invariants, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness |
| Item events | generic/valid_json, extract/dnd/item-events/shape, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness | generic/valid_json, extract/dnd/item-events/shape, normalize/dnd/item-events/invariants, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness |
| NPC interactions | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness | generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, normalize/dnd/npc-interactions/invariants, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness |
| Scene descriptions | generic/valid_json, extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness | generic/valid_json, extract/dnd/scene-descriptions/shape, normalize/dnd/scene-descriptions/invariants, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness |

Chains are only registered for the D&D extract and normalize modules shown
above; select an explicit override when a different compatible chain is
required.

## Validation

Validate a file and one pipeline before running it:

~~~sh
go run ./cmd/notarius config validate \
  --config examples/dnd-minimal.config.yml \
  --pipeline dnd-session
~~~

Configuration validation rejects invalid YAML, unsupported fields, invalid
defaults or environment overrides, incompatible module keys, unknown options,
invalid reference bindings, missing required reference slots, invalid validator
overrides, and incompatible generated artifact handoffs. Use
[pipelines list](cli.md#pipelines-list) to inspect configured IDs.