notarius/docs/config.md

Configuration

This is the canonical reference for Notarius configuration. Configuration files are YAML and must declare version 3. They select pipelines and their modules; the CLI reference owns invocation syntax, and Operations owns run-state procedures.

Configuration Discovery And Precedence

Commands that load configuration choose a file in this order:

1. a non-empty --config CLI value;
2. a non-empty NOTARIUS_CONFIG environment value;
3. the installed default file at /usr/local/etc/notarius/config.yml, when it exists.

The command fails if none of these paths provides a configuration file.

For configuration values, precedence is:

1. built-in defaults;
2. the selected YAML file;
3. supported operational environment variables; and
4. the CLI run overrides that apply to a command.

Environment variables do not provide a second configuration schema. They only override the fields listed below.

Maintained Examples

• Minimal D&D configuration is a single-lane Seriatim-to-spell pipeline.
• Complete D&D configuration uses ordered steps, all implemented D&D lanes, generated references, state settings, and bounded LLM concurrency.

Use these complete files as starting points rather than combining the illustrative fragments in this reference.

File Shape And Defaults

Unknown fields, duplicate mapping keys, empty identifiers, and identifiers that become duplicates after trimming whitespace are rejected. Every top-level field other than version is optional.

Field	Type	Default	Rules
version	integer	none	Required; must be 3.
scriptorium	object	none	Profile source configuration.
pipelines	map	empty	Maps pipeline IDs to pipeline definitions.
concurrency	object	see below	Global LLM and extraction limits.
output	object	see below	Published output settings.
cache	object	see below	Chunk-plan and checkpoint settings.
debug	object	see below	Debug-bundle root only; it does not enable capture.

Built-in defaults are:

Field	Default
concurrency.total_llm	1
concurrency.stage_workers.extract	Effective total_llm
output.directory	./notarius-output
cache.chunk_plans.mode	auto
cache.chunk_plans.directory	Empty, selecting the per-user chunk-plan root
cache.checkpoints.enabled	false
cache.checkpoints.directory	Empty, selecting the per-user checkpoint root
debug.directory	./notarius-debug

An empty cache directory in YAML deliberately selects the corresponding per-user root. An explicit empty output or debug directory is invalid.

Scriptorium Profiles

The optional scriptorium object selects one source of profile definitions:

Field	Type	Rules
profile_dir	string	Non-empty directory containing profile files.
profile_file	string	Non-empty profile file.

Set at most one of these fields. Profile IDs used by a binding must be available from the selected Scriptorium profile source when the pipeline is resolved. Keep credentials out of this file: configure a profile to read its credential from an environment variable, then set that environment variable only in the run environment.

Operational Environment Variables

These variables are applied after YAML values:

Variable	Overrides	Rules
NOTARIUS_TOTAL_LLM_CONCURRENCY	concurrency.total_llm	Integer.
NOTARIUS_STAGE_WORKERS_EXTRACT	concurrency.stage_workers.extract	Integer.
NOTARIUS_OUTPUT_DIR	output.directory	Non-empty path.
NOTARIUS_CACHE_CHUNK_PLANS_MODE	cache.chunk_plans.mode	auto, bypass, or refresh.
NOTARIUS_CACHE_CHUNK_PLANS_DIR	cache.chunk_plans.directory	Non-empty path.
NOTARIUS_CACHE_CHECKPOINTS_DIR	cache.checkpoints.directory	Non-empty path.
NOTARIUS_DEBUG_DIR	debug.directory	Non-empty path.

Integer values are trimmed then parsed as base-10 integers. Directory and output values reject NUL characters. NOTARIUS_CONFIG participates only in configuration discovery.

Concurrency, Output, Cache, And Debug

concurrency:
  total_llm: 2
  stage_workers:
    extract: 2
output:
  directory: ./notarius-output
cache:
  chunk_plans:
    mode: auto
    directory: ./notarius-cache/chunk-plans
  checkpoints:
    enabled: true
    directory: ./notarius-cache/checkpoints
debug:
  directory: ./notarius-debug

concurrency.total_llm must be greater than zero. The only supported concurrency.stage_workers key is extract; its value must be from 1 through total_llm. When omitted, it is recalculated from the effective total_llm after YAML and environment precedence.

cache.chunk_plans.mode accepts auto, bypass, or refresh. cache.checkpoints.enabled is a boolean. The CLI can override the output directory and chunk-plan mode for one run; see CLI reference.

Pipelines

Each pipelines entry has a unique, non-empty ID and the following shape:

pipelines:
  dnd-session:
    input: seriatim
    chunk: generic
    output: json
    artifacts:
      spells:
        extract: dnd/spells
        merge: appendorder
        normalize: dnd/spells

Field	Type	Default	Rules
input	module binding	none	Required.
chunk	module binding	generic	Optional.
output	module binding	json	Optional.
artifacts	map	none	Compact single-step lane map.
steps	list	none	Ordered lane definitions. Mutually exclusive with artifacts.
references	map	none	External reference defaults for eligible targets.

Use either artifacts or steps. The compact artifacts form is an implicit single step. An explicit steps list must be non-empty; every step needs a unique non-empty id, an artifacts map, and may have references. A lane ID must not appear more than once in a pipeline, including across explicit steps.

A lane has these fields:

Field	Type	Default	Rules
extract	module binding	none	Required.
merge	module binding	appendorder	Optional.
normalize	module binding	noop	Optional.
references	map	none	Supported compatibility alias for extract.references.
validators	list	none	Non-empty lane-level lists are rejected. Set validator overrides on a binding instead.

The lane-level references alias remains accepted. When the alias and extract.references bind the same slot, extract.references wins. Use the binding-local form in new configurations.

Module Bindings And Validators

Use a module key directly when no other binding fields are needed:

input: seriatim

Use an object for fields:

extract:
  module: dnd/spells
  llm_profile: gemini-2-flash
  retries: 2
  references:
    spell_catalog: ./dnd-spell-catalog.json

Binding field	Type	Default	Rules
module	string	none	Required for an object binding. Must be a registered compatible key.
llm_profile	string	none	Optional non-empty Scriptorium profile ID.
retries	integer	0	Non-negative additional attempts for chunk, extract, merge, and normalize bindings.
options	object	none	Must satisfy the selected module.
references	map	none	Valid only on chunk, extract, merge, and normalize bindings.
validators	list	production chain	Valid only on chunk, extract, merge, and normalize bindings.

Omitting validators uses the registered chain. validators: [] selects an empty chain; a non-empty list replaces the chain in the listed order. Validator bindings accept only module, llm_profile, and options. They reject references, retries, and nested validators. Deterministic validators reject an explicit llm_profile.

The json output module accepts optional include_chunk_map and evidence_context settings:

output:
  module: json
  options:
    include_chunk_map: true
    evidence_context:
      enabled: true
      window_units: 3
      lanes:
        - npcs
        - spells

include_chunk_map is a boolean and defaults to false. It adds the accepted chunk map when one exists; its wire format is defined in the chunk-map contract.

Omitting evidence_context disables evidence publication. When present, it is an object with these strict fields:

Field	Type	Rules
enabled	boolean	Required. false permits no other evidence fields.
lanes	array of strings	Required and non-empty when enabled. Each value is trimmed and must be unique; every value must name a configured pipeline lane.
window_units	non-negative integer	Optional when enabled; defaults to 3. Zero retains only directly cited units.

Unknown outer or nested option fields are rejected, as are incompatible YAML types. The allowlist remains valid when a run uses lane filtering: a configured lane that is not active for that invocation simply contributes no evidence. Evidence publication is opt-in because it can persist source text and metadata. Its payload contract is Published Evidence Context.

References And Ordered Handoffs

Reference maps bind named slots that the selected target declares. A scalar is an external path. Pipeline-level maps accept only external paths; step-local and binding-local maps may also select a normalized artifact from an earlier step:

steps:
  - id: describe-session
    artifacts:
      npcs:
        extract: dnd/npcs
        normalize: dnd/npcs
  - id: extract-events
    references:
      npcs:
        artifact:
          step: describe-session
          lane: npcs
    artifacts:
      spells:
        extract: dnd/spells
        normalize: dnd/spells

An artifact selector contains only step and lane. The producer must be an earlier step and the selected artifact must be compatible with the consumer slot. A generated binding supplies one accepted normalized artifact; it does not name a file. A configured generated dependency remains required even when that consumer slot is otherwise optional.

Pipeline references are defaults. A matching step-local or binding-local external path overrides a pipeline default. Required slots must be bound after these configuration values and any CLI reference overrides are applied. Reference paths in YAML are resolved relative to the configuration file.

D&D Reference Slots

The following slot names are accepted by the implemented D&D modules when the selected target declares them:

Slot	Source and use
party	Optional text campaign context. This is the canonical party-roster spelling.
roster	Accepted compatibility alias for party.
players	Optional text player context.
glossary	Optional text campaign glossary.
spell_catalog	Optional JSON spell-catalog overlay for spell extraction and normalization. See spell-catalog overlays.
npcs	Normalized NPC registry. Optional for spells and combat turns; required for NPC interactions.
scene_descriptions	Required normalized scene-description artifact for combat-turn extraction.

Scene descriptions accept party, players, and glossary, but not roster. NPC interactions require npcs for both extraction and normalization. Combat turns require scene_descriptions for extraction; the normalized combat-turn module may use optional npcs. The complete example shows generated npcs and scene_descriptions bindings.

Production Module Keys

Kind	Keys
Input	seriatim
Chunk	generic, dnd/scenes
Extract	dnd/spells, dnd/npcs, dnd/combat-turns, dnd/item-events, dnd/npc-interactions, dnd/scene-descriptions
Merge	appendorder
Normalize	noop, dnd/spells, dnd/npcs, dnd/combat-turns, dnd/item-events, dnd/npc-interactions, dnd/scene-descriptions
Output	json

The D&D artifact contracts define each emitted schema: spells, NPCs, NPC interactions, combat turns, item events, and scene descriptions.

Production Validator Keys And Default Chains

Available validator keys are:

Family	Keys
Generic	generic/always_accept, generic/always_reject, generic/valid_json, generic/valid_json_schema
Spells	extract/dnd/spells/shape, extract/dnd/spells/catalog, extract/dnd/spells/source_refs, extract/dnd/spells/source_relatedness
NPCs	extract/dnd/npcs/shape, extract/dnd/npcs/source_refs, extract/dnd/npcs/source_relatedness, normalize/dnd/npcs/identity
Combat turns	extract/dnd/combat-turns/shape, extract/dnd/combat-turns/source_refs, extract/dnd/combat-turns/source_relatedness, normalize/dnd/combat-turns/invariants
Item events	extract/dnd/item-events/shape, extract/dnd/item-events/source_refs, extract/dnd/item-events/source_relatedness, normalize/dnd/item-events/invariants
NPC interactions	extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, extract/dnd/npc-interactions/source_refs, extract/dnd/npc-interactions/source_relatedness, normalize/dnd/npc-interactions/invariants
Scene descriptions	extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, extract/dnd/scene-descriptions/source_relatedness, normalize/dnd/scene-descriptions/invariants

When no override is configured, production D&D bindings use the following ordered chains. Each row lists extract then normalize; spell chains are the same at both stages.

Lane	Extract	Normalize
Spells	generic/valid_json, extract/dnd/spells/shape, extract/dnd/spells/catalog, extract/dnd/spells/source_refs, generic/valid_json_schema, extract/dnd/spells/source_relatedness	Same as extract
NPCs	generic/valid_json, extract/dnd/npcs/shape, extract/dnd/npcs/source_refs, generic/valid_json_schema, extract/dnd/npcs/source_relatedness	generic/valid_json, extract/dnd/npcs/shape, normalize/dnd/npcs/identity, extract/dnd/npcs/source_refs, generic/valid_json_schema, extract/dnd/npcs/source_relatedness
Combat turns	generic/valid_json, extract/dnd/combat-turns/shape, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness	generic/valid_json, extract/dnd/combat-turns/shape, normalize/dnd/combat-turns/invariants, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness
Item events	generic/valid_json, extract/dnd/item-events/shape, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness	generic/valid_json, extract/dnd/item-events/shape, normalize/dnd/item-events/invariants, extract/dnd/item-events/source_refs, generic/valid_json_schema, extract/dnd/item-events/source_relatedness
NPC interactions	generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness	generic/valid_json, extract/dnd/npc-interactions/shape, extract/dnd/npc-interactions/registry, normalize/dnd/npc-interactions/invariants, extract/dnd/npc-interactions/source_refs, generic/valid_json_schema, extract/dnd/npc-interactions/source_relatedness
Scene descriptions	generic/valid_json, extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness	generic/valid_json, extract/dnd/scene-descriptions/shape, normalize/dnd/scene-descriptions/invariants, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness

Chains are only registered for the D&D extract and normalize modules shown above; select an explicit override when a different compatible chain is required.

Validation

Validate a file and one pipeline before running it:

go run ./cmd/notarius config validate \
  --config examples/dnd-minimal.config.yml \
  --pipeline dnd-session

Configuration validation rejects invalid YAML, unsupported fields, invalid defaults or environment overrides, incompatible module keys, unknown options, invalid reference bindings, missing required reference slots, invalid validator overrides, and incompatible generated artifact handoffs. Use pipelines list to inspect configured IDs.