296 lines
9.7 KiB
Markdown
296 lines
9.7 KiB
Markdown
# Configuration
|
|
|
|
This is the canonical reference for implemented Notarius configuration.
|
|
|
|
Notarius reads YAML config files with `version: 1`. File config is applied over
|
|
built-in defaults, then environment overrides are applied.
|
|
|
|
## Discovery
|
|
|
|
Commands that accept `--config` load configuration in this order:
|
|
|
|
1. the `--config` path, when provided;
|
|
2. `NOTARIUS_CONFIG`, when set to a non-empty path;
|
|
3. `/usr/local/etc/notarius/config.yml`.
|
|
|
|
If none is available, the command fails with a config file not found error.
|
|
|
|
## Minimal Example
|
|
|
|
```yaml
|
|
version: 1
|
|
llm_profiles:
|
|
default:
|
|
provider: openai-compatible
|
|
base_url: http://127.0.0.1:8080/v1
|
|
model: your-model
|
|
pipelines:
|
|
dnd-session:
|
|
input: seriatim
|
|
references:
|
|
roster: ./dnd-spells-roster.txt
|
|
glossary: ./dnd-spells-glossary.txt
|
|
chunk:
|
|
module: generic
|
|
options:
|
|
max_units: 50
|
|
artifacts:
|
|
spells:
|
|
extract: dnd/spells
|
|
```
|
|
|
|
The maintained fixture is [examples/dnd-spells.config.yml](../examples/dnd-spells.config.yml).
|
|
|
|
## Top-Level Fields
|
|
|
|
- `version`: required. The only supported value is `1`.
|
|
- `llm_profiles`: optional map of LLM profile IDs to profile settings.
|
|
- `pipelines`: optional map of pipeline IDs to pipeline definitions.
|
|
- `concurrency`: optional global concurrency settings.
|
|
- `diagnostics`: optional diagnostics settings.
|
|
|
|
Unknown YAML fields are rejected.
|
|
|
|
## Defaults
|
|
|
|
Built-in defaults:
|
|
|
|
```yaml
|
|
llm_profiles:
|
|
default:
|
|
provider: openai-compatible
|
|
timeout: 600
|
|
max_retries: 3
|
|
max_concurrency: 1
|
|
concurrency:
|
|
total_llm: 1
|
|
diagnostics:
|
|
work_dir: /tmp/notarius
|
|
retention: auto
|
|
```
|
|
|
|
No pipelines are built in. A run requires a configured pipeline.
|
|
|
|
## LLM Profiles
|
|
|
|
Each `llm_profiles` entry may contain:
|
|
|
|
- `provider`: optional provider key. Empty means `openai-compatible`; any other
|
|
non-empty value must be `openai-compatible`.
|
|
- `base_url`: provider base URL. Required for actual LLM calls.
|
|
- `model`: provider model name. Required for actual LLM calls.
|
|
- `api_key_env`: environment variable name to read for the API key.
|
|
- `timeout`: request timeout as whole seconds or a Go-style duration string such
|
|
as `10m`.
|
|
- `max_retries`: retry count for provider calls. Must be zero or greater.
|
|
- `max_concurrency`: per-profile LLM concurrency. Must be zero or greater; when
|
|
zero, Notarius uses `concurrency.total_llm`.
|
|
|
|
Raw API keys are not accepted as file config fields. Use `api_key_env` or an
|
|
environment override.
|
|
|
|
## Environment Overrides
|
|
|
|
These environment variables are applied after the config file:
|
|
|
|
- `NOTARIUS_CONFIG`: config discovery path.
|
|
- `NOTARIUS_LLM_DEFAULT_API_KEY`: API key for the `default` LLM profile.
|
|
- `NOTARIUS_LLM_DEFAULT_BASE_URL`: base URL for the `default` LLM profile.
|
|
- `NOTARIUS_LLM_DEFAULT_MODEL`: model for the `default` LLM profile.
|
|
- `NOTARIUS_LLM_DEFAULT_TIMEOUT_SECONDS`: integer timeout seconds for the
|
|
`default` LLM profile.
|
|
- `NOTARIUS_LLM_DEFAULT_MAX_RETRIES`: integer retry count for the `default` LLM
|
|
profile.
|
|
- `NOTARIUS_LLM_DEFAULT_MAX_CONCURRENCY`: integer max concurrency for the
|
|
`default` LLM profile.
|
|
- `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency.
|
|
- `NOTARIUS_WORK_DIR`: diagnostics work directory.
|
|
- `NOTARIUS_DIAGNOSTICS_RETENTION`: diagnostics retention mode.
|
|
|
|
Integer environment values must parse as base-10 integers.
|
|
|
|
## Pipelines
|
|
|
|
A pipeline defines the fixed Notarius workflow:
|
|
|
|
```text
|
|
input -> chunk -> extract -> merge -> normalize -> output
|
|
```
|
|
|
|
Pipeline fields:
|
|
|
|
- `input`: required module binding.
|
|
- `chunk`: optional module binding. Default module is `generic`.
|
|
- `artifacts`: required for pipeline resolution. It maps artifact lane IDs to
|
|
lane definitions.
|
|
- `output`: optional module binding. Default module is `json`.
|
|
- `references`: optional map of reference slot names to reference paths. These
|
|
bindings are defaults for eligible pipeline targets that declare the matching
|
|
slot.
|
|
|
|
Artifact lane fields:
|
|
|
|
- `extract`: required module binding.
|
|
- `merge`: optional module binding. Default module is `appendorder`.
|
|
- `normalize`: optional module binding. Default module is `noop`.
|
|
- `validators`: optional list of module bindings. The production CLI currently
|
|
does not register validator modules.
|
|
- `references`: optional compatibility alias for extractor reference bindings.
|
|
Lane bindings override pipeline-level bindings for the same slot.
|
|
|
|
`notarius run` and `notarius config validate --pipeline` resolve the pipeline
|
|
against the production module catalog and fail fast for unknown or incompatible
|
|
module keys.
|
|
|
|
Reference bindings are validated against reference slots declared by eligible
|
|
chunk, extract, and normalize targets during pipeline resolution. Required slots
|
|
must be bound after config defaults, target-local references, lane-level
|
|
compatibility bindings, and run-time `--reference` or `--without-reference`
|
|
overrides are applied. Config-relative paths are resolved relative to the config
|
|
file; CLI reference paths are resolved relative to the current working
|
|
directory. Materialized bound files must be UTF-8 text. Materialized reference
|
|
provenance is recorded for chunk, extractor, and normalizer targets, and runtime
|
|
reference content is passed to the target that declares the slot. Reference
|
|
media types are inferred from file extensions, recorded as canonical base media
|
|
types, and checked only when a module declares
|
|
`AcceptedMediaTypes`; unknown extensions are recorded as
|
|
`application/octet-stream`. Reference content is not written to diagnostics,
|
|
logs, errors, or manifests.
|
|
|
|
Pipeline-level `references` are defaults. They are valid when at least one
|
|
eligible target in the full configured pipeline declares the slot, including
|
|
chunk, extractor, and normalizer targets. During a run, they apply only to the
|
|
selected targets that declare the slot:
|
|
|
|
```yaml
|
|
pipelines:
|
|
dnd-session:
|
|
input: seriatim
|
|
references:
|
|
roster: ./campaign/party-roster.txt
|
|
glossary: ./campaign/glossary.txt
|
|
artifacts:
|
|
spells:
|
|
extract: dnd/spells
|
|
```
|
|
|
|
Extractor binding `references` are the canonical lane-local location. The
|
|
legacy lane-level `references` field remains supported as an alias; when both
|
|
bind the same slot, `extract.references` wins:
|
|
|
|
```yaml
|
|
pipelines:
|
|
dnd-session:
|
|
input: seriatim
|
|
references:
|
|
glossary: ./campaign/glossary.txt
|
|
artifacts:
|
|
spells:
|
|
references:
|
|
roster: ./campaign/legacy-roster.txt
|
|
extract:
|
|
module: dnd/spells
|
|
references:
|
|
roster: ./campaign/session-roster.txt
|
|
```
|
|
|
|
`chunk.references` and `normalize.references` are accepted in object-form
|
|
bindings. They override pipeline-level defaults for slots declared by the chunk
|
|
or normalizer module. Extractor-local references apply only to the extractor,
|
|
and normalizer-local references apply only to the normalizer.
|
|
|
|
## Module Bindings
|
|
|
|
Every module binding may use shorthand:
|
|
|
|
```yaml
|
|
input: seriatim
|
|
```
|
|
|
|
or object form:
|
|
|
|
```yaml
|
|
chunk:
|
|
module: generic
|
|
llm_profile: default
|
|
options:
|
|
max_units: 50
|
|
```
|
|
|
|
Binding fields:
|
|
|
|
- `module`: module key.
|
|
- `llm_profile`: optional LLM profile ID. Empty means `default`.
|
|
- `options`: optional module-specific settings.
|
|
- `references`: optional reference bindings. Supported only for `chunk`,
|
|
`extract`, and `normalize` bindings. `input`, `merge`, validator, and
|
|
`output` bindings reject this field during validation.
|
|
|
|
The `--llm-profile` run flag overrides every effective module binding to use
|
|
one configured profile.
|
|
|
|
## Implemented Production Modules
|
|
|
|
| Slot | Key | Notes |
|
|
| --- | --- | --- |
|
|
| input | `seriatim` | Reads Seriatim transcript JSON. |
|
|
| chunk | `generic` | Splits source units into ordered chunks. |
|
|
| chunk | `dnd/scenes` | Uses an LLM to split transcript source units into D&D scenes. |
|
|
| extract | `dnd/spells` | Extracts `dnd.spell_cast` artifacts. |
|
|
| merge | `appendorder` | Keeps candidates in append order. |
|
|
| normalize | `noop` | Passes merged artifacts through unchanged. |
|
|
| output | `json` | Produces JSON output files. |
|
|
|
|
The `generic` chunker accepts:
|
|
|
|
- `max_units`: positive integer, default `50`;
|
|
- `overlap_units`: non-negative integer, default `0`, and must be less than
|
|
`max_units`.
|
|
|
|
The `dnd/scenes` chunker requires transcript source capabilities, calls the
|
|
configured structured LLM provider, and does not accept module options.
|
|
|
|
The `dnd/spells` extractor declares optional text reference slots:
|
|
|
|
- `roster`
|
|
- `glossary`
|
|
|
|
The extractor uses these references only as supporting disambiguation material;
|
|
spell casts still must be present in the source transcript.
|
|
|
|
## Diagnostics
|
|
|
|
`diagnostics` fields:
|
|
|
|
- `work_dir`: directory for per-run diagnostics. Default: `/tmp/notarius`.
|
|
- `retention`: `auto`, `always`, or `never`. Empty uses `auto`.
|
|
|
|
`auto` retains diagnostics for failed runs and successful runs with warnings.
|
|
`always` retains diagnostics for every run. `never` removes diagnostics for
|
|
successful runs without regard to warnings; failed runs are retained.
|
|
|
|
The `--diagnostics-dir` run flag overrides `diagnostics.work_dir` for that
|
|
invocation.
|
|
|
|
## Validation
|
|
|
|
Configuration validation checks:
|
|
|
|
- supported config version and known YAML fields;
|
|
- non-empty, non-duplicated IDs after trimming;
|
|
- supported LLM provider and non-negative profile limits;
|
|
- positive global LLM concurrency;
|
|
- supported diagnostics retention and non-empty work directory;
|
|
- module binding LLM profiles refer to configured profiles.
|
|
|
|
Pipeline resolution additionally checks:
|
|
|
|
- the pipeline ID exists;
|
|
- at least one artifact lane is declared and selected;
|
|
- selected lanes exist when `--only` is used;
|
|
- required module keys are present;
|
|
- module keys are registered for the expected slot;
|
|
- module capability requirements are satisfied.
|
|
- bound reference slots are declared by selected lane extractors;
|
|
- required reference slots are bound for selected lanes.
|