Finish Scriptorium runtime documentation

This commit is contained in:
2026-07-05 18:34:21 +00:00
parent c9fbb331e2
commit 31d70a2dd7
15 changed files with 221 additions and 755 deletions

View File

@@ -2,7 +2,7 @@
This is the canonical reference for implemented Notarius configuration.
Notarius reads YAML config files with `version: 1`. File config is applied over
Notarius reads YAML config files with `version: 2`. File config is applied over
built-in defaults, then environment overrides are applied.
## Discovery
@@ -18,12 +18,7 @@ If none is available, the command fails with a config file not found error.
## Minimal Example
```yaml
version: 1
llm_profiles:
default:
provider: openai-compatible
base_url: http://127.0.0.1:8080/v1
model: your-model
version: 2
pipelines:
dnd-session:
input: seriatim
@@ -43,25 +38,20 @@ The maintained fixture is [examples/dnd-spells.config.yml](../examples/dnd-spell
## Top-Level Fields
- `version`: required. The only supported value is `1`.
- `llm_profiles`: optional map of LLM profile IDs to profile settings.
- `version`: required. The only supported value is `2`.
- `scriptorium`: optional Scriptorium profile source settings.
- `pipelines`: optional map of pipeline IDs to pipeline definitions.
- `concurrency`: optional global concurrency settings.
- `diagnostics`: optional diagnostics settings.
Unknown YAML fields are rejected.
Unknown YAML fields are rejected. The removed top-level `llm_profiles` field is
rejected; execution profiles now come from Scriptorium.
## Defaults
Built-in defaults:
```yaml
llm_profiles:
default:
provider: openai-compatible
timeout: 600
max_retries: 3
max_concurrency: 1
concurrency:
total_llm: 1
diagnostics:
@@ -71,44 +61,52 @@ diagnostics:
No pipelines are built in. A run requires a configured pipeline.
## LLM Profiles
If `scriptorium` is omitted, Notarius uses Scriptorium's built-in profile
catalog. Prompt definitions may also name default profile IDs. The current D&D
scene and spell prompts use Scriptorium prompt defaults when a module binding
does not set `llm_profile`.
Each `llm_profiles` entry may contain:
## Scriptorium Profiles
- `provider`: optional provider key. Empty means `openai-compatible`; any other
non-empty value must be `openai-compatible`.
- `base_url`: provider base URL. Required for actual LLM calls.
- `model`: provider model name. Required for actual LLM calls.
- `api_key_env`: environment variable name to read for the API key.
- `timeout`: request timeout as whole seconds or a Go-style duration string such
as `10m`.
- `max_retries`: retry count for provider calls. Must be zero or greater.
- `max_concurrency`: per-profile LLM concurrency. Must be zero or greater; when
zero, Notarius uses `concurrency.total_llm`.
`scriptorium` fields:
Raw API keys are not accepted as file config fields. Use `api_key_env` or an
environment override.
- `profile_dir`: optional directory containing Scriptorium profile YAML files.
- `profile_file`: optional Scriptorium profile YAML file.
`profile_dir` and `profile_file` are mutually exclusive. Custom profiles
overlay Scriptorium built-in profiles by profile ID.
Scriptorium profile files use Scriptorium's profile schema. A minimal profile
looks like:
```yaml
id: local-fast
endpoint: http://127.0.0.1:8080/v1
model: your-model
api_key_env: SCRIPTORIUM_API_KEY
timeout_seconds: 180
```
Notarius does not accept raw API keys in Notarius config. For file-backed
Scriptorium profiles, store the environment variable name in `api_key_env` and
set that variable in the run environment. Scriptorium rejects raw `api_key`
fields in profile YAML.
## Environment Overrides
These environment variables are applied after the config file:
- `NOTARIUS_CONFIG`: config discovery path.
- `NOTARIUS_LLM_DEFAULT_API_KEY`: API key for the `default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_BASE_URL`: base URL for the `default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_MODEL`: model for the `default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_TIMEOUT_SECONDS`: integer timeout seconds for the
`default` LLM profile.
- `NOTARIUS_LLM_DEFAULT_MAX_RETRIES`: integer retry count for the `default` LLM
profile.
- `NOTARIUS_LLM_DEFAULT_MAX_CONCURRENCY`: integer max concurrency for the
`default` LLM profile.
- `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency.
- `NOTARIUS_WORK_DIR`: diagnostics work directory.
- `NOTARIUS_DIAGNOSTICS_RETENTION`: diagnostics retention mode.
Integer environment values must parse as base-10 integers.
The removed `NOTARIUS_LLM_DEFAULT_*` variables are not read. Configure provider
endpoint, model, and credential environment variable names through Scriptorium
profiles.
## Pipelines
A pipeline defines the fixed Notarius workflow:
@@ -152,10 +150,9 @@ directory. Materialized bound files must be UTF-8 text. Materialized reference
provenance is recorded for chunk, extractor, and normalizer targets, and runtime
reference content is passed to the target that declares the slot. Reference
media types are inferred from file extensions, recorded as canonical base media
types, and checked only when a module declares
`AcceptedMediaTypes`; unknown extensions are recorded as
`application/octet-stream`. Reference content is not written to diagnostics,
logs, errors, or manifests.
types, and checked only when a module declares `AcceptedMediaTypes`; unknown
extensions are recorded as `application/octet-stream`. Reference content is not
written to diagnostics, logs, errors, or manifests.
Pipeline-level `references` are defaults. They are valid when at least one
eligible target in the full configured pipeline declares the slot, including
@@ -199,7 +196,7 @@ bindings. They override pipeline-level defaults for slots declared by the chunk
or normalizer module. Extractor-local references apply only to the extractor,
and normalizer-local references apply only to the normalizer.
Stage-local reference fields use the same map shape at:
Target-local reference fields use the same map shape at:
- `pipelines.<id>.chunk.references`
- `pipelines.<id>.artifacts.<lane>.extract.references`
@@ -219,23 +216,22 @@ or object form:
```yaml
chunk:
module: generic
llm_profile: default
options:
max_units: 50
module: dnd/scenes
llm_profile: local-fast
```
Binding fields:
- `module`: module key.
- `llm_profile`: optional LLM profile ID. Empty means `default`.
- `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the
Scriptorium prompt default select the profile.
- `options`: optional module-specific settings.
- `references`: optional reference bindings. Supported only for `chunk`,
`extract`, and `normalize` bindings. `input`, `merge`, validator, and
`output` bindings reject this field during validation.
The `--llm-profile` run flag overrides every effective module binding to use
one configured profile.
The `--llm-profile` run flag overrides every effective LLM-capable module
binding to use one Scriptorium profile ID.
## Implemented Production Modules
@@ -256,7 +252,7 @@ The `generic` chunker accepts:
`max_units`.
The `dnd/scenes` chunker requires transcript source capabilities, calls the
configured structured LLM provider, and does not accept module options.
configured structured LLM runtime, and does not accept module options.
The `dnd/spells` extractor declares optional text reference slots:
@@ -285,11 +281,11 @@ invocation.
Configuration validation checks:
- supported config version and known YAML fields;
- mutually exclusive `scriptorium.profile_dir` and `scriptorium.profile_file`;
- non-empty, non-duplicated IDs after trimming;
- supported LLM provider and non-negative profile limits;
- positive global LLM concurrency;
- supported diagnostics retention and non-empty work directory;
- module binding LLM profiles refer to configured profiles.
- stale removed fields such as `llm_profiles`.
Pipeline resolution additionally checks:
@@ -298,7 +294,7 @@ Pipeline resolution additionally checks:
- selected lanes exist when `--only` is used;
- required module keys are present;
- module keys are registered for the expected slot;
- module capability requirements are satisfied.
- module capability requirements are satisfied;
- bound reference slots are declared by selected chunk, extractor, or
normalizer targets;
- required reference slots are bound for selected targets.