eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Switch commands to ConfigKit configuration

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-13 17:12:56 +00:00
parent d809085406
commit 92a6a4985a
23 changed files with 393 additions and 331 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
docs/cli.md
View File

@@ -12,8 +12,8 @@ defined in [Operations](operations.md).
notarius help
notarius --version
notarius run <pipeline-id> --input path/to/source.json [--json] [flags]
notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list [--config path/to/config.yml] [--json]
notarius config validate [--config path/to/config.yml] [--config-profile name] [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius pipelines list [--config path/to/config.yml] [--config-profile name] [--json]
~~~
Running Notarius without arguments, or with **help**, **--help**, or **-h**,
@@ -42,6 +42,7 @@ pipeline ID and **--input** are required.
| Flag | Meaning |
| --- | --- |
| **--config path** | Use this configuration file. When omitted, configuration discovery applies; see [Configuration](config.md). |
| **--config-profile name** | Select a ConfigKit configuration overlay. Empty and **default** select the composed base configuration. |
| **--input path** | Source input file to process. Required. |
| **--output-dir path** | Override the configured output root for this run. |
| **--json** | Write the successful run-result receipt as JSON to standard output. |
@@ -154,10 +155,10 @@ OPENROUTER_API_KEY=your-api-key \
## config validate
~~~
notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b]
notarius config validate [--config path/to/config.yml] [--config-profile name] [--pipeline pipeline-id] [--only lane-a,lane-b]
~~~
This command loads and validates a configuration. With **--pipeline**, it also
This command loads and validates the selected composed configuration. With **--pipeline**, it also
resolves that pipeline against the production module catalog. **--only** selects
lanes during that resolution and requires **--pipeline**.
@@ -183,7 +184,7 @@ validation; it cannot run a provider-backed pipeline.
## pipelines list
~~~
notarius pipelines list [--config path/to/config.yml] [--json]
notarius pipelines list [--config path/to/config.yml] [--config-profile name] [--json]
~~~
This command lists configured pipeline IDs in sorted order. By default, it

142
docs/config.md
View File

@@ -1,7 +1,7 @@
# Configuration
This is the canonical reference for Notarius configuration. Configuration files
are YAML and must declare version 4. They select pipelines and their modules;
are YAML and must declare version 5. They select pipelines and their modules;
the [CLI reference](cli.md) owns invocation syntax, and
[Operations](operations.md) owns run-state procedures.
@@ -19,12 +19,15 @@ The command fails if none of these paths provides a configuration file.
For configuration values, precedence is:
1. built-in defaults;
2. the selected YAML file;
3. supported operational environment variables; and
4. the CLI run overrides that apply to a command.
2. composed YAML sources, including imports;
3. the selected ConfigKit profile; and
4. explicitly supplied configuration-valued run flags.
Environment variables do not provide a second configuration schema. They only
override the fields listed below.
`--config-profile` selects a ConfigKit overlay from `profiles/<name>.yml`
beneath the selected configuration root. Empty and `default` select no overlay.
Configuration profiles are distinct from PromptKit LLM profiles selected by
`llm_profile` or `--llm-profile`. The retired operational environment variables
have no effect; `NOTARIUS_CONFIG` remains a source-discovery input.
## Maintained Examples
@@ -46,7 +49,7 @@ other than **version** is optional.
| Field | Type | Default | Rules |
| --- | --- | --- | --- |
| **version** | integer | none | Required; must be 4. |
| **version** | integer or quoted string | none | Required; must be 5. Maintained files use `version: 5`. |
| **promptkit** | object | none | Profile source and optional local-backend configuration. |
| **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. |
| **concurrency** | object | see below | Global LLM and extraction limits. |
@@ -73,13 +76,26 @@ Selecting a chunk-plan or checkpoint directory does not configure filesystem
access policy. Cache permissions come from deployment provisioning, process
umask, setgid inheritance, and default ACLs; see [Operations](operations.md#cache-provisioning).
Every nonempty relative configured destination or reference path resolves from
the canonical directory of the file that supplies its surviving declaration.
This includes imported fragments and selected configuration profiles. Built-in
relative destinations use the selected main configuration file's directory;
explicit `--output-dir` and `--debug-dir` values use the invocation working
directory. Absolute values remain unchanged.
The root file can use ConfigKit imports to compose disjoint definitions. Imports
and selected profiles remain under the configuration root; they do not merge or
append a pipeline's `steps` sequence. A profile replaces sequences as defined
by ConfigKit, so changing an individual ordered binding requires supplying the
whole replacement sequence.
## PromptKit Profiles
The optional **promptkit** object selects one source of profile definitions and
may register one conventional local OpenAI-compatible backend:
~~~yaml
version: 4
version: 5
promptkit:
profile_dir: ./profiles
@@ -97,11 +113,12 @@ promptkit:
| **local_backend.endpoint** | string | Required when **local_backend** is present; absolute HTTP or HTTPS URL with a host. |
| **local_backend.concurrency_limit** | integer | Optional non-negative limit; defaults to 0. |
Set at most one of **profile_dir** and **profile_file**. Relative values use
the process working directory, not the configuration file's directory. The
complete example's `./examples/profiles/dnd-extraction.yml` value is therefore
valid when Notarius is launched from the repository root; use an absolute path
for services and containers.
Set at most one of **profile_dir** and **profile_file**. Every nonempty relative
configured filesystem path is resolved from the canonical directory of the
file that supplies it. The complete example's
`./profiles/dnd-extraction.yml` value therefore resolves beside that example.
Use absolute paths for services and containers when their deployment layout is
not configuration-relative.
An operator source is optional. For a requested ID, PromptKit checks the
configured operator source first, then Notarius's embedded fallback profiles,
@@ -150,30 +167,15 @@ requiring credential values. It rejects absent, malformed, or incompatible
profiles before a run prepares modules. Credential availability is checked only
when a generation is prepared.
## Migrating Version 3 Configuration
## Version 5 Migration
Version 3 files are not decoded or rewritten. Change **version: 3** to
**version: 4** and rename the top-level **scriptorium:** section to
**promptkit:**. Version 4 decoding is strict, so a remaining **scriptorium**
field is rejected as unknown.
## Operational Environment Variables
These variables are applied after YAML values:
| Variable | Overrides | Rules |
| --- | --- | --- |
| **NOTARIUS_TOTAL_LLM_CONCURRENCY** | **concurrency.total_llm** | Integer. |
| **NOTARIUS_STAGE_WORKERS_EXTRACT** | **concurrency.stage_workers.extract** | Integer. |
| **NOTARIUS_OUTPUT_DIR** | **output.directory** | Non-empty path. |
| **NOTARIUS_CACHE_CHUNK_PLANS_MODE** | **cache.chunk_plans.mode** | **auto**, **bypass**, or **refresh**. |
| **NOTARIUS_CACHE_CHUNK_PLANS_DIR** | **cache.chunk_plans.directory** | Non-empty path. |
| **NOTARIUS_CACHE_CHECKPOINTS_DIR** | **cache.checkpoints.directory** | Non-empty path. |
| **NOTARIUS_DEBUG_DIR** | **debug.directory** | Non-empty path. |
Integer values are trimmed then parsed as base-10 integers. Directory and
output values reject NUL characters. **NOTARIUS_CONFIG** participates only in
configuration discovery.
Version 3 and version 4 files are rejected and are not rewritten at load time.
Use `version: 5`, rename the former `scriptorium` section to `promptkit`, and
convert every module binding to an object containing `module`. Convert every
external reference to `{path: ...}` and move any lane-level references to the
corresponding extraction binding. Put durable operational values in YAML or
use the documented explicit run flags; the former field-level environment
variables are not read.
## Concurrency, Output, Cache, And Debug
@@ -198,13 +200,14 @@ debug:
**concurrency.total_llm** must be greater than zero. The only supported
**concurrency.stage_workers** key is **extract**; its value must be from 1
through **total_llm**. When omitted, it is recalculated from the effective
**total_llm** after YAML and environment precedence.
**total_llm** after imports, profile selection, and explicit ConfigKit
overrides.
**cache.chunk_plans.mode** accepts **auto**, **bypass**, or **refresh**.
**cache.checkpoints.enabled** is a boolean. The CLI can override the output
directory and chunk-plan mode for one run; see [CLI reference](cli.md#run).
Cache directory fields select only locations; Notarius has no cache permission
field or environment override.
field or operational environment override.
## Pipelines
@@ -214,14 +217,20 @@ Each **pipelines** entry has a unique, non-empty ID and the following shape:
pipelines:
dnd-session:
llm_profile: dnd-extraction
input: seriatim
chunk: generic
output: json
input:
module: seriatim
chunk:
module: generic
output:
module: json
artifacts:
spells:
extract: dnd/spells
merge: appendorder
normalize: dnd/spells
extract:
module: dnd/spells
merge:
module: appendorder
normalize:
module: dnd/spells
~~~
| Field | Type | Default | Rules |
@@ -291,19 +300,20 @@ A lane has these fields:
| **extract** | module binding | none | Required. |
| **merge** | module binding | **appendorder** | Optional. |
| **normalize** | module binding | **noop** | Optional. |
| **references** | map | none | Supported compatibility alias for **extract.references**. |
| **references** | map | none | Not supported at lane scope; place references on the eligible binding. |
| **validators** | list | none | Non-empty lane-level lists are rejected. Set validator overrides on a binding instead. |
The lane-level **references** alias remains accepted. When the alias and
**extract.references** bind the same slot, **extract.references** wins. Use
the binding-local form in new configurations.
Lane-level **references** are not supported. Put a lane extraction reference
under **extract.references**.
## Module Bindings And Validators
Use a module key directly when no other binding fields are needed:
Every module and validator binding uses an object, even when it has no fields
beyond its module key:
~~~yaml
input: seriatim
input:
module: seriatim
~~~
Use an object for fields:
@@ -314,7 +324,8 @@ extract:
llm_profile: dnd-extraction
retries: 2
references:
spell_catalog: ./dnd-spell-catalog.json
spell_catalog:
path: ./dnd-spell-catalog.json
~~~
| Binding field | Type | Default | Rules |
@@ -386,18 +397,20 @@ When enabled, it publishes the selected source-unit excerpt defined by the
## References And Ordered Handoffs
Reference maps bind named slots that the selected target declares. A scalar is
an external path. Pipeline-level maps accept only external paths; step-local
and binding-local maps may also select a normalized artifact from an earlier
step:
Reference maps bind named slots that the selected target declares. An external
reference is an object with `path`. Pipeline-level maps accept only external
paths; step-local and binding-local maps may also select a normalized artifact
from an earlier step:
~~~yaml
steps:
- id: describe-session
artifacts:
npc-registry:
extract: dnd/npc-registry
normalize: dnd/npc-registry
extract:
module: dnd/npc-registry
normalize:
module: dnd/npc-registry
- id: extract-events
references:
npc_registry:
@@ -406,8 +419,10 @@ steps:
lane: npc-registry
artifacts:
spells:
extract: dnd/spells
normalize: dnd/spells
extract:
module: dnd/spells
normalize:
module: dnd/spells
~~~
An artifact selector contains only **step** and **lane**. The producer must be
@@ -422,9 +437,10 @@ bindings are then operational overrides of configured external paths; their
pipeline, lane, and binding scopes and precedence are defined by the
[CLI reference](cli.md#reference-selectors). A CLI file reference cannot
replace a configured generated artifact handoff. Required slots must be bound
after configuration and CLI reference actions are applied. Reference paths in
YAML are resolved relative to the configuration file; CLI reference paths are
resolved relative to the process working directory.
after configuration and CLI reference actions are applied. A YAML reference
path resolves from the file that supplies its surviving declaration, including
an imported fragment or selected profile. CLI reference paths resolve relative
to the process working directory.
### D&D Reference Slots
@@ -573,7 +589,7 @@ go run ./cmd/notarius config validate \
~~~
Configuration validation rejects invalid YAML, unsupported fields, invalid
defaults or environment overrides, incompatible module keys, unknown options,
defaults or explicit CLI overrides, incompatible module keys, unknown options,
invalid reference bindings, missing required reference slots, invalid validator
overrides, and incompatible generated artifact handoffs. Use
[pipelines list](cli.md#pipelines-list) to inspect configured IDs.

12
docs/consumers/dnd-pipeline.md
View File

@@ -23,16 +23,14 @@ A deployment must provide its own PromptKit profile and campaign reference
files. Use absolute paths for service and subprocess deployments. In
particular, observe these different resolution rules:
- reference paths in YAML are resolved relative to the Notarius configuration
file;
- reference and PromptKit profile paths in YAML are resolved relative to the
file that supplies their surviving declaration;
- reference paths passed with `--reference` are resolved relative to the
Notarius process working directory; and
- `promptkit.profile_file` is resolved relative to the Notarius process working
directory.
Notarius process working directory.
Do not copy the repository example's relative profile path into a deployment
without also controlling that working directory. The complete path and profile
rules are defined in [Configuration](../config.md).
without preserving its configuration-relative layout. The complete path and
profile rules are defined in [Configuration](../config.md).
Preflight the deployed configuration before processing sessions and whenever
it changes:

6
docs/consumers/subprocess.md
View File

@@ -15,9 +15,12 @@ configured lane inventory, and downstream acceptance checklist, see
Optionally preflight a selected configuration and pipeline before work starts:
```sh
notarius config validate --config /path/to/notarius.yml --pipeline pipeline-id
notarius config validate --config /path/to/notarius.yml --config-profile production --pipeline pipeline-id
```
Omit `--config-profile` when the composed base configuration is the intended
selection; it is independent from any `--llm-profile` run override.
Invoke the run with explicit paths and machine-readable output. Capture
standard output and standard error separately; do not combine them before
processing the result.
@@ -25,6 +28,7 @@ processing the result.
```sh
notarius run pipeline-id \
--config /path/to/notarius.yml \
--config-profile production \
--input /path/to/source.json \
--output-dir /path/to/output-root \
--json

16
docs/operations.md
View File

@@ -126,11 +126,11 @@ notarius config validate --config /etc/notarius/config.yml --pipeline dnd-sessio
An unset optional `api_key_env` reaches the provider without authorization and
may receive a 401 or 403 response.
Profile paths are currently resolved from the process working directory, not
from the configuration file. The complete example's
`./examples/profiles/dnd-extraction.yml` path is valid for a repository-root
invocation only. Use absolute paths such as
`/etc/notarius/profiles/dnd-extraction.yml` for services and containers.
Configured PromptKit profile paths resolve from the file that supplies their
surviving declaration. The complete example's
`./profiles/dnd-extraction.yml` path is therefore relative to that example.
Use absolute paths such as `/etc/notarius/profiles/dnd-extraction.yml` for
services and containers when that is clearer for the deployment.
## Run Lifecycle
@@ -308,7 +308,8 @@ Only a [debug-enabled run](cli.md#run) creates a bundle:
trace/
~~~
The summary contains redacted invocation and resolution information plus run,
The summary contains redacted invocation and resolution information, including
`effective-config.yml` and the separately redacted resolved pipeline, plus run,
final grouped diagnostic, checkpoint, chunk-plan, and terminal reporting artifacts. Attempt
terminal records contain bounded attempt kinds, validator outcomes, policy,
decision, PromptKit repair count, and usage; they do not contain assistant
@@ -318,7 +319,8 @@ content. Neither surface is a cache input. Do not treat a debug bundle as safe
to share merely because its configuration summary is redacted. Invocation
metadata omits reasoning effort when it is inherited, records the replacement
value when one is supplied, and records an empty value when inherited reasoning
was explicitly cleared.
was explicitly cleared. It records `config_profile` only when a non-base
configuration overlay was selected.
Notarius never creates debug state without an explicit request and never
automatically deletes a requested bundle. If allocation succeeds, the command

2
docs/roadmap/implementation.md
View File

@@ -484,6 +484,8 @@ incompatible generated references.
## Stage 10 — Switch Run And The Maintained V5 Configurations
**Status:** Completed 2026-09-13.
**Goal:** Complete production cutover and make all runs use one ConfigKit load.
**Work:**