Finish Scriptorium runtime documentation
This commit is contained in:
106
docs/config.md
106
docs/config.md
@@ -2,7 +2,7 @@
|
||||
|
||||
This is the canonical reference for implemented Notarius configuration.
|
||||
|
||||
Notarius reads YAML config files with `version: 1`. File config is applied over
|
||||
Notarius reads YAML config files with `version: 2`. File config is applied over
|
||||
built-in defaults, then environment overrides are applied.
|
||||
|
||||
## Discovery
|
||||
@@ -18,12 +18,7 @@ If none is available, the command fails with a config file not found error.
|
||||
## Minimal Example
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
llm_profiles:
|
||||
default:
|
||||
provider: openai-compatible
|
||||
base_url: http://127.0.0.1:8080/v1
|
||||
model: your-model
|
||||
version: 2
|
||||
pipelines:
|
||||
dnd-session:
|
||||
input: seriatim
|
||||
@@ -43,25 +38,20 @@ The maintained fixture is [examples/dnd-spells.config.yml](../examples/dnd-spell
|
||||
|
||||
## Top-Level Fields
|
||||
|
||||
- `version`: required. The only supported value is `1`.
|
||||
- `llm_profiles`: optional map of LLM profile IDs to profile settings.
|
||||
- `version`: required. The only supported value is `2`.
|
||||
- `scriptorium`: optional Scriptorium profile source settings.
|
||||
- `pipelines`: optional map of pipeline IDs to pipeline definitions.
|
||||
- `concurrency`: optional global concurrency settings.
|
||||
- `diagnostics`: optional diagnostics settings.
|
||||
|
||||
Unknown YAML fields are rejected.
|
||||
Unknown YAML fields are rejected. The removed top-level `llm_profiles` field is
|
||||
rejected; execution profiles now come from Scriptorium.
|
||||
|
||||
## Defaults
|
||||
|
||||
Built-in defaults:
|
||||
|
||||
```yaml
|
||||
llm_profiles:
|
||||
default:
|
||||
provider: openai-compatible
|
||||
timeout: 600
|
||||
max_retries: 3
|
||||
max_concurrency: 1
|
||||
concurrency:
|
||||
total_llm: 1
|
||||
diagnostics:
|
||||
@@ -71,44 +61,52 @@ diagnostics:
|
||||
|
||||
No pipelines are built in. A run requires a configured pipeline.
|
||||
|
||||
## LLM Profiles
|
||||
If `scriptorium` is omitted, Notarius uses Scriptorium's built-in profile
|
||||
catalog. Prompt definitions may also name default profile IDs. The current D&D
|
||||
scene and spell prompts use Scriptorium prompt defaults when a module binding
|
||||
does not set `llm_profile`.
|
||||
|
||||
Each `llm_profiles` entry may contain:
|
||||
## Scriptorium Profiles
|
||||
|
||||
- `provider`: optional provider key. Empty means `openai-compatible`; any other
|
||||
non-empty value must be `openai-compatible`.
|
||||
- `base_url`: provider base URL. Required for actual LLM calls.
|
||||
- `model`: provider model name. Required for actual LLM calls.
|
||||
- `api_key_env`: environment variable name to read for the API key.
|
||||
- `timeout`: request timeout as whole seconds or a Go-style duration string such
|
||||
as `10m`.
|
||||
- `max_retries`: retry count for provider calls. Must be zero or greater.
|
||||
- `max_concurrency`: per-profile LLM concurrency. Must be zero or greater; when
|
||||
zero, Notarius uses `concurrency.total_llm`.
|
||||
`scriptorium` fields:
|
||||
|
||||
Raw API keys are not accepted as file config fields. Use `api_key_env` or an
|
||||
environment override.
|
||||
- `profile_dir`: optional directory containing Scriptorium profile YAML files.
|
||||
- `profile_file`: optional Scriptorium profile YAML file.
|
||||
|
||||
`profile_dir` and `profile_file` are mutually exclusive. Custom profiles
|
||||
overlay Scriptorium built-in profiles by profile ID.
|
||||
|
||||
Scriptorium profile files use Scriptorium's profile schema. A minimal profile
|
||||
looks like:
|
||||
|
||||
```yaml
|
||||
id: local-fast
|
||||
endpoint: http://127.0.0.1:8080/v1
|
||||
model: your-model
|
||||
api_key_env: SCRIPTORIUM_API_KEY
|
||||
timeout_seconds: 180
|
||||
```
|
||||
|
||||
Notarius does not accept raw API keys in Notarius config. For file-backed
|
||||
Scriptorium profiles, store the environment variable name in `api_key_env` and
|
||||
set that variable in the run environment. Scriptorium rejects raw `api_key`
|
||||
fields in profile YAML.
|
||||
|
||||
## Environment Overrides
|
||||
|
||||
These environment variables are applied after the config file:
|
||||
|
||||
- `NOTARIUS_CONFIG`: config discovery path.
|
||||
- `NOTARIUS_LLM_DEFAULT_API_KEY`: API key for the `default` LLM profile.
|
||||
- `NOTARIUS_LLM_DEFAULT_BASE_URL`: base URL for the `default` LLM profile.
|
||||
- `NOTARIUS_LLM_DEFAULT_MODEL`: model for the `default` LLM profile.
|
||||
- `NOTARIUS_LLM_DEFAULT_TIMEOUT_SECONDS`: integer timeout seconds for the
|
||||
`default` LLM profile.
|
||||
- `NOTARIUS_LLM_DEFAULT_MAX_RETRIES`: integer retry count for the `default` LLM
|
||||
profile.
|
||||
- `NOTARIUS_LLM_DEFAULT_MAX_CONCURRENCY`: integer max concurrency for the
|
||||
`default` LLM profile.
|
||||
- `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency.
|
||||
- `NOTARIUS_WORK_DIR`: diagnostics work directory.
|
||||
- `NOTARIUS_DIAGNOSTICS_RETENTION`: diagnostics retention mode.
|
||||
|
||||
Integer environment values must parse as base-10 integers.
|
||||
|
||||
The removed `NOTARIUS_LLM_DEFAULT_*` variables are not read. Configure provider
|
||||
endpoint, model, and credential environment variable names through Scriptorium
|
||||
profiles.
|
||||
|
||||
## Pipelines
|
||||
|
||||
A pipeline defines the fixed Notarius workflow:
|
||||
@@ -152,10 +150,9 @@ directory. Materialized bound files must be UTF-8 text. Materialized reference
|
||||
provenance is recorded for chunk, extractor, and normalizer targets, and runtime
|
||||
reference content is passed to the target that declares the slot. Reference
|
||||
media types are inferred from file extensions, recorded as canonical base media
|
||||
types, and checked only when a module declares
|
||||
`AcceptedMediaTypes`; unknown extensions are recorded as
|
||||
`application/octet-stream`. Reference content is not written to diagnostics,
|
||||
logs, errors, or manifests.
|
||||
types, and checked only when a module declares `AcceptedMediaTypes`; unknown
|
||||
extensions are recorded as `application/octet-stream`. Reference content is not
|
||||
written to diagnostics, logs, errors, or manifests.
|
||||
|
||||
Pipeline-level `references` are defaults. They are valid when at least one
|
||||
eligible target in the full configured pipeline declares the slot, including
|
||||
@@ -199,7 +196,7 @@ bindings. They override pipeline-level defaults for slots declared by the chunk
|
||||
or normalizer module. Extractor-local references apply only to the extractor,
|
||||
and normalizer-local references apply only to the normalizer.
|
||||
|
||||
Stage-local reference fields use the same map shape at:
|
||||
Target-local reference fields use the same map shape at:
|
||||
|
||||
- `pipelines.<id>.chunk.references`
|
||||
- `pipelines.<id>.artifacts.<lane>.extract.references`
|
||||
@@ -219,23 +216,22 @@ or object form:
|
||||
|
||||
```yaml
|
||||
chunk:
|
||||
module: generic
|
||||
llm_profile: default
|
||||
options:
|
||||
max_units: 50
|
||||
module: dnd/scenes
|
||||
llm_profile: local-fast
|
||||
```
|
||||
|
||||
Binding fields:
|
||||
|
||||
- `module`: module key.
|
||||
- `llm_profile`: optional LLM profile ID. Empty means `default`.
|
||||
- `llm_profile`: optional Scriptorium profile ID. Empty or omitted lets the
|
||||
Scriptorium prompt default select the profile.
|
||||
- `options`: optional module-specific settings.
|
||||
- `references`: optional reference bindings. Supported only for `chunk`,
|
||||
`extract`, and `normalize` bindings. `input`, `merge`, validator, and
|
||||
`output` bindings reject this field during validation.
|
||||
|
||||
The `--llm-profile` run flag overrides every effective module binding to use
|
||||
one configured profile.
|
||||
The `--llm-profile` run flag overrides every effective LLM-capable module
|
||||
binding to use one Scriptorium profile ID.
|
||||
|
||||
## Implemented Production Modules
|
||||
|
||||
@@ -256,7 +252,7 @@ The `generic` chunker accepts:
|
||||
`max_units`.
|
||||
|
||||
The `dnd/scenes` chunker requires transcript source capabilities, calls the
|
||||
configured structured LLM provider, and does not accept module options.
|
||||
configured structured LLM runtime, and does not accept module options.
|
||||
|
||||
The `dnd/spells` extractor declares optional text reference slots:
|
||||
|
||||
@@ -285,11 +281,11 @@ invocation.
|
||||
Configuration validation checks:
|
||||
|
||||
- supported config version and known YAML fields;
|
||||
- mutually exclusive `scriptorium.profile_dir` and `scriptorium.profile_file`;
|
||||
- non-empty, non-duplicated IDs after trimming;
|
||||
- supported LLM provider and non-negative profile limits;
|
||||
- positive global LLM concurrency;
|
||||
- supported diagnostics retention and non-empty work directory;
|
||||
- module binding LLM profiles refer to configured profiles.
|
||||
- stale removed fields such as `llm_profiles`.
|
||||
|
||||
Pipeline resolution additionally checks:
|
||||
|
||||
@@ -298,7 +294,7 @@ Pipeline resolution additionally checks:
|
||||
- selected lanes exist when `--only` is used;
|
||||
- required module keys are present;
|
||||
- module keys are registered for the expected slot;
|
||||
- module capability requirements are satisfied.
|
||||
- module capability requirements are satisfied;
|
||||
- bound reference slots are declared by selected chunk, extractor, or
|
||||
normalizer targets;
|
||||
- required reference slots are bound for selected targets.
|
||||
|
||||
Reference in New Issue
Block a user