26 KiB
Configuration
This is the canonical reference for Notarius configuration. Configuration files are YAML and must declare version 4. They select pipelines and their modules; the CLI reference owns invocation syntax, and Operations owns run-state procedures.
Configuration Discovery And Precedence
Commands that load configuration choose a file in this order:
- a non-empty --config CLI value;
- a non-empty NOTARIUS_CONFIG environment value;
- the installed default file at /usr/local/etc/notarius/config.yml, when it exists.
The command fails if none of these paths provides a configuration file.
For configuration values, precedence is:
- built-in defaults;
- the selected YAML file;
- supported operational environment variables; and
- the CLI run overrides that apply to a command.
Environment variables do not provide a second configuration schema. They only override the fields listed below.
Maintained Examples
- Minimal D&D configuration is a single-lane Seriatim-to-spell pipeline.
- Complete D&D configuration uses ordered steps, all implemented D&D lanes, generated references, state settings, bounded LLM concurrency, and the maintained operator profile.
Use these complete files as starting points rather than combining the illustrative fragments in this reference.
File Shape And Defaults
Unknown fields, duplicate mapping keys, empty identifiers, and identifiers that become duplicates after trimming whitespace are rejected. Every top-level field other than version is optional.
| Field | Type | Default | Rules |
|---|---|---|---|
| version | integer | none | Required; must be 4. |
| promptkit | object | none | Profile source and optional local-backend configuration. |
| pipelines | map | empty | Maps pipeline IDs to pipeline definitions. |
| concurrency | object | see below | Global LLM and extraction limits. |
| output | object | see below | Published output settings. |
| cache | object | see below | Chunk-plan and checkpoint settings. |
| debug | object | see below | Debug-bundle root only; it does not enable capture. |
Built-in defaults are:
| Field | Default |
|---|---|
| concurrency.total_llm | 16 |
| concurrency.stage_workers.extract | Effective total_llm |
| output.directory | ./notarius-output |
| cache.chunk_plans.mode | auto |
| cache.chunk_plans.directory | Empty, selecting the per-user chunk-plan root |
| cache.checkpoints.enabled | false |
| cache.checkpoints.directory | Empty, selecting the per-user checkpoint root |
| debug.directory | ./notarius-debug |
An empty cache directory in YAML deliberately selects the corresponding per-user root. An explicit empty output or debug directory is invalid.
PromptKit Profiles
The optional promptkit object selects one source of profile definitions and may register one conventional local OpenAI-compatible backend:
version: 4
promptkit:
profile_dir: ./profiles
# profile_file: ./profiles.yml
local_backend:
endpoint: http://localhost:8000/v1
concurrency_limit: 2
| Field | Type | Rules |
|---|---|---|
| profile_dir | string | Non-empty directory containing profile files. |
| profile_file | string | Non-empty profile file. |
| local_backend | object | Optional registration for the conventional PromptKit backend ID local. |
| local_backend.endpoint | string | Required when local_backend is present; absolute HTTP or HTTPS URL with a host. |
| local_backend.concurrency_limit | integer | Optional non-negative limit; defaults to 0. |
Set at most one of profile_dir and profile_file. Relative values use
the process working directory, not the configuration file's directory. The
complete example's ./examples/profiles/dnd-extraction.yml value is therefore
valid when Notarius is launched from the repository root; use an absolute path
for services and containers.
An operator source is optional. For a requested ID, PromptKit checks the
configured operator source first, then Notarius's embedded fallback profiles,
then its own built-in catalog. A matching profile is complete: it replaces a
lower-precedence definition rather than merging with it. The maintained
dnd-extraction operator profile
is a secret-free deployment artifact; production, development, and local
deployments can each provide a complete definition with that same workload ID.
Use workload-oriented IDs for new profiles instead of model names.
Operations owns the deployment
workflow and credential-handling guidance.
When local_backend is present, its endpoint is trimmed and must use HTTP or HTTPS case-insensitively, be absolute, and have a non-empty host. URL paths are allowed. User information, queries, and fragments are rejected. A zero concurrency_limit leaves the local backend unrestricted inside PromptKit; a positive value limits simultaneous local generations. The application-wide concurrency.total_llm limit still applies in both cases. Neither local backend field has an environment override. Omitting local_backend registers nothing and preserves existing built-in and endpoint-only profile behavior.
A file-backed PromptKit profile selects the registration by its case-sensitive backend ID:
id: local-summary
backend: local
model: example-model
Keep credentials out of the local-backend object. A PromptKit profile may name
its credential environment variable through api_key_env; set that variable
only in the run environment. PromptKit owns the
pinned profile-file format.
The PromptKit upstream boundary identifies the
supported package API, and Operations
describes the effective concurrency layers.
notarius config validate --pipeline <id> resolves the selected pipeline and
inspects every explicit effective profile without contacting a provider or
requiring credential values. It rejects absent, malformed, or incompatible
profiles before a run prepares modules. Credential availability is checked only
when a generation is prepared.
Migrating Version 3 Configuration
Version 3 files are not decoded or rewritten. Change version: 3 to version: 4 and rename the top-level scriptorium: section to promptkit:. Version 4 decoding is strict, so a remaining scriptorium field is rejected as unknown.
Operational Environment Variables
These variables are applied after YAML values:
| Variable | Overrides | Rules |
|---|---|---|
| NOTARIUS_TOTAL_LLM_CONCURRENCY | concurrency.total_llm | Integer. |
| NOTARIUS_STAGE_WORKERS_EXTRACT | concurrency.stage_workers.extract | Integer. |
| NOTARIUS_OUTPUT_DIR | output.directory | Non-empty path. |
| NOTARIUS_CACHE_CHUNK_PLANS_MODE | cache.chunk_plans.mode | auto, bypass, or refresh. |
| NOTARIUS_CACHE_CHUNK_PLANS_DIR | cache.chunk_plans.directory | Non-empty path. |
| NOTARIUS_CACHE_CHECKPOINTS_DIR | cache.checkpoints.directory | Non-empty path. |
| NOTARIUS_DEBUG_DIR | debug.directory | Non-empty path. |
Integer values are trimmed then parsed as base-10 integers. Directory and output values reject NUL characters. NOTARIUS_CONFIG participates only in configuration discovery.
Concurrency, Output, Cache, And Debug
concurrency:
total_llm: 2
stage_workers:
extract: 2
output:
directory: ./notarius-output
cache:
chunk_plans:
mode: auto
directory: ./notarius-cache/chunk-plans
checkpoints:
enabled: true
directory: ./notarius-cache/checkpoints
debug:
directory: ./notarius-debug
concurrency.total_llm must be greater than zero. The only supported concurrency.stage_workers key is extract; its value must be from 1 through total_llm. When omitted, it is recalculated from the effective total_llm after YAML and environment precedence.
cache.chunk_plans.mode accepts auto, bypass, or refresh. cache.checkpoints.enabled is a boolean. The CLI can override the output directory and chunk-plan mode for one run; see CLI reference.
Pipelines
Each pipelines entry has a unique, non-empty ID and the following shape:
pipelines:
dnd-session:
llm_profile: dnd-extraction
input: seriatim
chunk: generic
output: json
artifacts:
spells:
extract: dnd/spells
merge: appendorder
normalize: dnd/spells
| Field | Type | Default | Rules |
|---|---|---|---|
| llm_profile | string | none | Optional non-empty default PromptKit profile ID for selected LLM-backed bindings and validators. An explicitly present blank value is invalid. |
| input | module binding | none | Required. |
| chunk | module binding | generic | Optional. |
| output | module binding | json | Optional. |
| artifacts | map | none | Compact single-step lane map. |
| steps | list | none | Ordered lane definitions. Mutually exclusive with artifacts. |
| references | map | none | External reference defaults for eligible targets. |
Use either artifacts or steps. The compact artifacts form is an implicit single step. An explicit steps list must be non-empty; every step needs a unique non-empty id, an artifacts map, and may have references. A lane ID must not appear more than once in a pipeline, including across explicit steps.
For each selected LLM-backed binding or validator, profile selection occurs
after module, validator, and --only lane selection. It uses the
run-level --llm-profile value first, then the binding's llm_profile,
then the pipeline's llm_profile, and finally the PromptKit default.
Deterministic bindings do not receive these defaults or run overrides.
A lane has these fields:
| Field | Type | Default | Rules |
|---|---|---|---|
| extract | module binding | none | Required. |
| merge | module binding | appendorder | Optional. |
| normalize | module binding | noop | Optional. |
| references | map | none | Supported compatibility alias for extract.references. |
| validators | list | none | Non-empty lane-level lists are rejected. Set validator overrides on a binding instead. |
The lane-level references alias remains accepted. When the alias and extract.references bind the same slot, extract.references wins. Use the binding-local form in new configurations.
Module Bindings And Validators
Use a module key directly when no other binding fields are needed:
input: seriatim
Use an object for fields:
extract:
module: dnd/spells
llm_profile: dnd-extraction
retries: 2
references:
spell_catalog: ./dnd-spell-catalog.json
| Binding field | Type | Default | Rules |
|---|---|---|---|
| module | string | none | Required for an object binding. Must be a registered compatible key. |
| llm_profile | string | none | Optional non-empty PromptKit profile ID for an LLM-backed binding. It overrides the pipeline default unless the run supplies --llm-profile. |
| retries | integer | 0 | Non-negative additional attempts for chunk, extract, merge, and normalize bindings. |
| options | object | none | Must satisfy the selected module. |
| references | map | none | Valid only on chunk, extract, merge, and normalize bindings. |
| validators | list | production chain | Valid only on chunk, extract, merge, and normalize bindings. |
Omitting validators uses the registered chain. validators: [] selects an empty chain; a non-empty list replaces the chain in the listed order. Validator bindings accept only module, llm_profile, and options. They reject references, retries, and nested validators. Deterministic validators reject an explicit llm_profile. Deterministic module bindings also reject an explicit llm_profile.
The json output module accepts optional include_chunk_map and evidence_context settings:
output:
module: json
options:
include_chunk_map: true
evidence_context:
enabled: true
window_units: 3
lanes:
- npc_registry
- spells
include_chunk_map is a boolean and defaults to false. It adds the accepted chunk map when one exists; its wire format is defined in the chunk-map contract.
Omitting evidence_context disables evidence publication. When present, it is an object with these strict fields:
| Field | Type | Rules |
|---|---|---|
| enabled | boolean | Required. false permits no other evidence fields. |
| lanes | array of strings | Required and non-empty when enabled. Each value is trimmed and must be unique; every value must name a configured pipeline lane. |
| window_units | non-negative integer | Optional when enabled; defaults to 3. Zero retains only directly cited units. |
Unknown outer or nested option fields are rejected, as are incompatible YAML types. The allowlist remains valid when a run uses lane filtering: a configured lane that is not active for that invocation simply contributes no evidence. Evidence publication is opt-in because it can persist source text and metadata. Its payload contract is Published Evidence Context.
References And Ordered Handoffs
Reference maps bind named slots that the selected target declares. A scalar is an external path. Pipeline-level maps accept only external paths; step-local and binding-local maps may also select a normalized artifact from an earlier step:
steps:
- id: describe-session
artifacts:
npc_registry:
extract: dnd/npc-registry
normalize: dnd/npc-registry
- id: extract-events
references:
npc_registry:
artifact:
step: describe-session
lane: npc_registry
artifacts:
spells:
extract: dnd/spells
normalize: dnd/spells
An artifact selector contains only step and lane. The producer must be an earlier step and the selected artifact must be compatible with the consumer slot. A generated binding supplies one accepted normalized artifact; it does not name a file. A configured generated dependency remains required even when that consumer slot is otherwise optional.
Pipeline references are defaults. A matching step-local or binding-local external path overrides a pipeline default. Required slots must be bound after these configuration values and any CLI reference overrides are applied. Reference paths in YAML are resolved relative to the configuration file.
D&D Reference Slots
The following slot names are accepted by the implemented D&D modules when the selected target declares them:
| Slot | Source and use |
|---|---|
| party | Optional text campaign context. This is the canonical party-roster spelling. |
| roster | Accepted compatibility alias for party. |
| players | Optional text player context. |
| glossary | Optional text campaign glossary. |
| spell_catalog | Optional JSON spell-catalog overlay for spell extraction and normalization. See spell-catalog overlays. |
| location_registry | Required normalized location registry for location-occurrence extraction and normalization. |
| npc_registry | Normalized NPC registry. Optional for spells and combat turns; required for NPC occurrences and enemy-event extraction and normalization. |
| scene_descriptions | Required normalized scene-description artifact for combat-turn and enemy-event extraction. |
| combat_turns | Required normalized combat-turn artifact for enemy-event extraction. |
| npc_occurrences | Required normalized NPC-occurrence artifact for enemy-event extraction. |
Location-occurrence and enemy-event artifact slots have the following exact binding contracts. Durable semantics and wire shapes remain in their location-occurrence and enemy-event contracts.
| Slot | Accepted artifact kind | Media type | Maximum size | Required stage |
|---|---|---|---|---|
npc_registry |
dnd/npc-registry |
application/json |
1,048,576 bytes | extract and normalize |
scene_descriptions |
dnd/scene-description-list |
application/json |
1,048,576 bytes | extract only |
combat_turns |
dnd/combat-turn-list |
application/json |
1,048,576 bytes | extract only |
npc_occurrences |
dnd/npc-occurrence-list |
application/json |
1,048,576 bytes | extract only |
location_registry |
dnd/location-registry |
application/json |
1,048,576 bytes | location-occurrence extract and normalize |
Scene descriptions accept party, players, and glossary, but not roster. NPC occurrences require npc_registry for both extraction and normalization. Combat turns require scene_descriptions for extraction; the normalized combat-turn module may use optional npc_registry. Location occurrences require location_registry for extraction and normalization. Enemy-event extraction requires all four of its JSON artifact slots; its normalizer requires npc_registry. The complete example shows the ordered generated bindings.
Production Module Keys
| Kind | Keys |
|---|---|
| Input | seriatim |
| Chunk | generic, dnd/scenes |
| Extract | dnd/spells, dnd/npc-registry, dnd/combat-turns, dnd/item-occurrences, dnd/item-registry, dnd/npc-occurrences, dnd/scene-descriptions, dnd/enemy-events, dnd/location-registry, dnd/location-occurrences |
| Merge | appendorder |
| Normalize | noop, dnd/spells, dnd/npc-registry, dnd/combat-turns, dnd/item-occurrences, dnd/item-registry, dnd/npc-occurrences, dnd/scene-descriptions, dnd/enemy-events, dnd/location-registry, dnd/location-occurrences |
| Output | json |
dnd/location-registry and dnd/item-registry extraction and normalization
are llm_backed; their normalizers may use the pipeline's selected LLM profile
for bounded duplicate proposals. dnd/location-occurrences and
dnd/item-occurrences extraction are llm_backed, while their normalizers are
deterministic. The complete example binds each registry in an earlier step
before its occurrence consumer.
The D&D artifact contracts define each emitted schema: spells, NPC registry, NPC occurrences, combat turns, item occurrences, item registry, scene descriptions, and enemy events, location registry, and location occurrences.
Production Validator Keys And Default Chains
Available validator keys are:
| Family | Keys |
|---|---|
| Generic | generic/always_accept, generic/always_reject, generic/valid_json, generic/valid_json_schema |
| Spells | extract/dnd/spells/shape, extract/dnd/spells/catalog, extract/dnd/spells/source_refs, extract/dnd/spells/source_relatedness |
| NPC registry | extract/dnd/npc-registry/shape, extract/dnd/npc-registry/source_refs, extract/dnd/npc-registry/source_relatedness, normalize/dnd/npc-registry/identity |
| Combat turns | extract/dnd/combat-turns/shape, extract/dnd/combat-turns/source_refs, extract/dnd/combat-turns/source_relatedness, normalize/dnd/combat-turns/invariants |
| Item occurrences | extract/dnd/item-occurrences/shape, extract/dnd/item-occurrences/registry, extract/dnd/item-occurrences/source_refs, extract/dnd/item-occurrences/source_relatedness, normalize/dnd/item-occurrences/invariants |
| Item registry | extract/dnd/item-registry/shape, extract/dnd/item-registry/source_refs, extract/dnd/item-registry/source_relatedness, normalize/dnd/item-registry/identity |
| NPC occurrences | extract/dnd/npc-occurrences/shape, extract/dnd/npc-occurrences/registry, extract/dnd/npc-occurrences/source_refs, extract/dnd/npc-occurrences/source_relatedness, normalize/dnd/npc-occurrences/invariants |
| Scene descriptions | extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, extract/dnd/scene-descriptions/source_relatedness, normalize/dnd/scene-descriptions/invariants |
| Enemy events | extract/dnd/enemy-events/shape, extract/dnd/enemy-events/engagements, extract/dnd/enemy-events/source_refs, extract/dnd/enemy-events/source_relatedness, normalize/dnd/enemy-events/invariants |
| Location registry | extract/dnd/location-registry/shape, extract/dnd/location-registry/source_refs, extract/dnd/location-registry/source_relatedness, normalize/dnd/location-registry/identity |
| Location occurrences | extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, extract/dnd/location-occurrences/source_refs, extract/dnd/location-occurrences/source_relatedness, normalize/dnd/location-occurrences/invariants |
When no override is configured, production D&D bindings use the following ordered chains. Each row lists extract then normalize; spell chains are the same at both stages.
| Lane | Extract | Normalize |
|---|---|---|
| Spells | generic/valid_json, extract/dnd/spells/shape, extract/dnd/spells/catalog, extract/dnd/spells/source_refs, generic/valid_json_schema, extract/dnd/spells/source_relatedness | Same as extract |
| NPC registry | generic/valid_json, extract/dnd/npc-registry/shape, extract/dnd/npc-registry/source_refs, generic/valid_json_schema, extract/dnd/npc-registry/source_relatedness | generic/valid_json, extract/dnd/npc-registry/shape, normalize/dnd/npc-registry/identity, extract/dnd/npc-registry/source_refs, generic/valid_json_schema, extract/dnd/npc-registry/source_relatedness |
| Combat turns | generic/valid_json, extract/dnd/combat-turns/shape, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness | generic/valid_json, extract/dnd/combat-turns/shape, normalize/dnd/combat-turns/invariants, extract/dnd/combat-turns/source_refs, generic/valid_json_schema, extract/dnd/combat-turns/source_relatedness |
| Item occurrences | generic/valid_json, extract/dnd/item-occurrences/shape, extract/dnd/item-occurrences/registry, extract/dnd/item-occurrences/source_refs, generic/valid_json_schema, extract/dnd/item-occurrences/source_relatedness | generic/valid_json, extract/dnd/item-occurrences/shape, extract/dnd/item-occurrences/registry, normalize/dnd/item-occurrences/invariants, extract/dnd/item-occurrences/source_refs, generic/valid_json_schema, extract/dnd/item-occurrences/source_relatedness |
| Item registry | generic/valid_json, extract/dnd/item-registry/shape, extract/dnd/item-registry/source_refs, generic/valid_json_schema, extract/dnd/item-registry/source_relatedness | generic/valid_json, extract/dnd/item-registry/shape, normalize/dnd/item-registry/identity, extract/dnd/item-registry/source_refs, generic/valid_json_schema, extract/dnd/item-registry/source_relatedness |
| NPC occurrences | generic/valid_json, extract/dnd/npc-occurrences/shape, extract/dnd/npc-occurrences/registry, extract/dnd/npc-occurrences/source_refs, generic/valid_json_schema, extract/dnd/npc-occurrences/source_relatedness | generic/valid_json, extract/dnd/npc-occurrences/shape, extract/dnd/npc-occurrences/registry, normalize/dnd/npc-occurrences/invariants, extract/dnd/npc-occurrences/source_refs, generic/valid_json_schema, extract/dnd/npc-occurrences/source_relatedness |
| Scene descriptions | generic/valid_json, extract/dnd/scene-descriptions/shape, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness | generic/valid_json, extract/dnd/scene-descriptions/shape, normalize/dnd/scene-descriptions/invariants, extract/dnd/scene-descriptions/source_refs, generic/valid_json_schema, extract/dnd/scene-descriptions/source_relatedness |
| Enemy events | generic/valid_json, extract/dnd/enemy-events/shape, extract/dnd/enemy-events/engagements, extract/dnd/enemy-events/source_refs, generic/valid_json_schema, extract/dnd/enemy-events/source_relatedness | generic/valid_json, extract/dnd/enemy-events/shape, normalize/dnd/enemy-events/invariants, extract/dnd/enemy-events/source_refs, generic/valid_json_schema, extract/dnd/enemy-events/source_relatedness |
| Location registry | generic/valid_json, extract/dnd/location-registry/shape, extract/dnd/location-registry/source_refs, generic/valid_json_schema, extract/dnd/location-registry/source_relatedness | generic/valid_json, extract/dnd/location-registry/shape, normalize/dnd/location-registry/identity, extract/dnd/location-registry/source_refs, generic/valid_json_schema, extract/dnd/location-registry/source_relatedness |
| Location occurrences | generic/valid_json, extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, extract/dnd/location-occurrences/source_refs, generic/valid_json_schema, extract/dnd/location-occurrences/source_relatedness | generic/valid_json, extract/dnd/location-occurrences/shape, extract/dnd/location-occurrences/registry, normalize/dnd/location-occurrences/invariants, extract/dnd/location-occurrences/source_refs, generic/valid_json_schema, extract/dnd/location-occurrences/source_relatedness |
Chains are only registered for the D&D extract and normalize modules shown above; select an explicit override when a different compatible chain is required.
Validation
Validate a file and one pipeline before running it:
go run ./cmd/notarius config validate \
--config examples/dnd-minimal.config.yml \
--pipeline dnd-session
Configuration validation rejects invalid YAML, unsupported fields, invalid defaults or environment overrides, incompatible module keys, unknown options, invalid reference bindings, missing required reference slots, invalid validator overrides, and incompatible generated artifact handoffs. Use pipelines list to inspect configured IDs.