Add user documentation for Notarius CLI and config
This commit is contained in:
29
README.md
29
README.md
@@ -1,2 +1,29 @@
|
||||
# go-application-template
|
||||
# Notarius
|
||||
|
||||
Notarius is a Go CLI for extracting structured artifacts from source material
|
||||
with explicit, configurable pipeline modules.
|
||||
|
||||
The current implementation reads Seriatim minimal transcript JSON, chunks the
|
||||
source units, extracts D&D spell-cast artifacts with an OpenAI-compatible LLM,
|
||||
and writes JSON output plus diagnostics for each run.
|
||||
|
||||
```sh
|
||||
NOTARIUS_LLM_DEFAULT_BASE_URL=http://127.0.0.1:8080/v1 \
|
||||
NOTARIUS_LLM_DEFAULT_MODEL=your-model \
|
||||
go run ./cmd/notarius run dnd-session \
|
||||
--config examples/dnd-spells.config.yml \
|
||||
--input examples/seriatim-minimal-transcript.json
|
||||
```
|
||||
|
||||
If the provider requires authentication, set
|
||||
`NOTARIUS_LLM_DEFAULT_API_KEY` in the environment before running the command.
|
||||
Outputs are written under `./notarius-output/<run-id>/` unless `--output-dir`
|
||||
is provided.
|
||||
|
||||
Useful references:
|
||||
|
||||
- [CLI reference](docs/cli.md)
|
||||
- [Configuration reference](docs/config.md)
|
||||
- [Seriatim input contract](docs/integrations/seriatim.md)
|
||||
- [Maintained example config](examples/dnd-spells.config.yml)
|
||||
- [Maintained example input](examples/seriatim-minimal-transcript.json)
|
||||
|
||||
126
docs/cli.md
Normal file
126
docs/cli.md
Normal file
@@ -0,0 +1,126 @@
|
||||
# CLI Reference
|
||||
|
||||
This is the canonical reference for the implemented Notarius command-line
|
||||
interface.
|
||||
|
||||
## Quick Run
|
||||
|
||||
```sh
|
||||
NOTARIUS_LLM_DEFAULT_BASE_URL=http://127.0.0.1:8080/v1 \
|
||||
NOTARIUS_LLM_DEFAULT_MODEL=your-model \
|
||||
go run ./cmd/notarius run dnd-session \
|
||||
--config examples/dnd-spells.config.yml \
|
||||
--input examples/seriatim-minimal-transcript.json
|
||||
```
|
||||
|
||||
Set `NOTARIUS_LLM_DEFAULT_API_KEY` if the OpenAI-compatible provider requires
|
||||
a bearer token.
|
||||
|
||||
## Commands
|
||||
|
||||
```text
|
||||
notarius help
|
||||
notarius run <pipeline-id> --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b]
|
||||
notarius config validate --config path/to/config.yml [--pipeline pipeline-id] [--only lane-a,lane-b]
|
||||
notarius pipelines list --config path/to/config.yml [--json]
|
||||
```
|
||||
|
||||
Running `notarius` with no arguments, `notarius help`, `notarius --help`, or
|
||||
`notarius -h` prints usage and exits successfully.
|
||||
|
||||
## `run`
|
||||
|
||||
`notarius run <pipeline-id>` executes a configured pipeline against one input
|
||||
file.
|
||||
|
||||
Flags:
|
||||
|
||||
- `--input path`: required source input file.
|
||||
- `--config path`: config file path. If omitted, Notarius checks
|
||||
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
|
||||
- `--only lane-a,lane-b`: run only the named artifact lanes. Values are
|
||||
comma-separated and must be non-empty.
|
||||
- `--output-dir path`: output root. The run writes to `<path>/<run-id>/`.
|
||||
Defaults to `./notarius-output`.
|
||||
- `--diagnostics-dir path`: diagnostics work directory override for this
|
||||
invocation.
|
||||
- `--llm-profile id`: override every effective module binding to use one LLM
|
||||
profile.
|
||||
|
||||
On success, the command prints the completed pipeline ID, approved and rejected
|
||||
artifact counts, and the output directory. If the run completes with warnings,
|
||||
the warning count is printed to stderr.
|
||||
|
||||
The current `run` command requires the resolved pipeline to use exactly one
|
||||
distinct LLM profile after defaults and overrides are applied.
|
||||
|
||||
## `config validate`
|
||||
|
||||
`notarius config validate` loads and validates configuration.
|
||||
|
||||
Flags:
|
||||
|
||||
- `--config path`: config file path. If omitted, discovery uses
|
||||
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
|
||||
- `--pipeline pipeline-id`: additionally resolve one configured pipeline against
|
||||
the production module catalog.
|
||||
- `--only lane-a,lane-b`: validate resolution for selected artifact lanes. This
|
||||
flag requires `--pipeline`.
|
||||
|
||||
Examples:
|
||||
|
||||
```sh
|
||||
go run ./cmd/notarius config validate \
|
||||
--config examples/dnd-spells.config.yml
|
||||
|
||||
go run ./cmd/notarius config validate \
|
||||
--config examples/dnd-spells.config.yml \
|
||||
--pipeline dnd-session \
|
||||
--only spells
|
||||
```
|
||||
|
||||
## `pipelines list`
|
||||
|
||||
`notarius pipelines list` prints configured pipeline IDs in sorted order.
|
||||
|
||||
Flags:
|
||||
|
||||
- `--config path`: config file path. If omitted, discovery uses
|
||||
`NOTARIUS_CONFIG`, then `/usr/local/etc/notarius/config.yml`.
|
||||
- `--json`: print `{"pipelines":[...]}` instead of one ID per line.
|
||||
|
||||
Examples:
|
||||
|
||||
```sh
|
||||
go run ./cmd/notarius pipelines list \
|
||||
--config examples/dnd-spells.config.yml
|
||||
|
||||
go run ./cmd/notarius pipelines list \
|
||||
--config examples/dnd-spells.config.yml \
|
||||
--json
|
||||
```
|
||||
|
||||
## Exit Codes
|
||||
|
||||
- `0`: command succeeded.
|
||||
- `1`: command syntax was valid, but loading config, resolving modules, running
|
||||
the pipeline, calling the provider, writing output, or writing diagnostics
|
||||
failed.
|
||||
- `2`: command syntax was invalid, a command was unknown, a required argument
|
||||
was missing, or a flag value was malformed.
|
||||
|
||||
## Implemented Production Pipeline Modules
|
||||
|
||||
The production CLI currently registers these module keys:
|
||||
|
||||
- input: `seriatim`
|
||||
- chunk: `generic`
|
||||
- extract: `dnd/spells`
|
||||
- merge: `appendorder`
|
||||
- normalize: `noop`
|
||||
- output: `json`
|
||||
|
||||
The production CLI does not currently register validator modules.
|
||||
|
||||
For YAML structure, defaults, environment overrides, and module binding syntax,
|
||||
see [Configuration](config.md).
|
||||
213
docs/config.md
Normal file
213
docs/config.md
Normal file
@@ -0,0 +1,213 @@
|
||||
# Configuration
|
||||
|
||||
This is the canonical reference for implemented Notarius configuration.
|
||||
|
||||
Notarius reads YAML config files with `version: 1`. File config is applied over
|
||||
built-in defaults, then environment overrides are applied.
|
||||
|
||||
## Discovery
|
||||
|
||||
Commands that accept `--config` load configuration in this order:
|
||||
|
||||
1. the `--config` path, when provided;
|
||||
2. `NOTARIUS_CONFIG`, when set to a non-empty path;
|
||||
3. `/usr/local/etc/notarius/config.yml`.
|
||||
|
||||
If none is available, the command fails with a config file not found error.
|
||||
|
||||
## Minimal Example
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
llm_profiles:
|
||||
default:
|
||||
provider: openai-compatible
|
||||
base_url: http://127.0.0.1:8080/v1
|
||||
model: your-model
|
||||
pipelines:
|
||||
dnd-session:
|
||||
input: seriatim
|
||||
chunk:
|
||||
module: generic
|
||||
options:
|
||||
max_units: 50
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
```
|
||||
|
||||
The maintained fixture is [examples/dnd-spells.config.yml](../examples/dnd-spells.config.yml).
|
||||
|
||||
## Top-Level Fields
|
||||
|
||||
- `version`: required. The only supported value is `1`.
|
||||
- `llm_profiles`: optional map of LLM profile IDs to profile settings.
|
||||
- `pipelines`: optional map of pipeline IDs to pipeline definitions.
|
||||
- `concurrency`: optional global concurrency settings.
|
||||
- `diagnostics`: optional diagnostics settings.
|
||||
|
||||
Unknown YAML fields are rejected.
|
||||
|
||||
## Defaults
|
||||
|
||||
Built-in defaults:
|
||||
|
||||
```yaml
|
||||
llm_profiles:
|
||||
default:
|
||||
provider: openai-compatible
|
||||
timeout: 600
|
||||
max_retries: 3
|
||||
max_concurrency: 1
|
||||
concurrency:
|
||||
total_llm: 1
|
||||
diagnostics:
|
||||
work_dir: /tmp/notarius
|
||||
retention: auto
|
||||
```
|
||||
|
||||
No pipelines are built in. A run requires a configured pipeline.
|
||||
|
||||
## LLM Profiles
|
||||
|
||||
Each `llm_profiles` entry may contain:
|
||||
|
||||
- `provider`: optional provider key. Empty means `openai-compatible`; any other
|
||||
non-empty value must be `openai-compatible`.
|
||||
- `base_url`: provider base URL. Required for actual LLM calls.
|
||||
- `model`: provider model name. Required for actual LLM calls.
|
||||
- `api_key_env`: environment variable name to read for the API key.
|
||||
- `timeout`: request timeout as whole seconds or a Go-style duration string such
|
||||
as `10m`.
|
||||
- `max_retries`: retry count for provider calls. Must be zero or greater.
|
||||
- `max_concurrency`: per-profile LLM concurrency. Must be zero or greater; when
|
||||
zero, Notarius uses `concurrency.total_llm`.
|
||||
|
||||
Raw API keys are not accepted as file config fields. Use `api_key_env` or an
|
||||
environment override.
|
||||
|
||||
## Environment Overrides
|
||||
|
||||
These environment variables are applied after the config file:
|
||||
|
||||
- `NOTARIUS_CONFIG`: config discovery path.
|
||||
- `NOTARIUS_LLM_DEFAULT_API_KEY`: API key for the `default` LLM profile.
|
||||
- `NOTARIUS_LLM_DEFAULT_BASE_URL`: base URL for the `default` LLM profile.
|
||||
- `NOTARIUS_LLM_DEFAULT_MODEL`: model for the `default` LLM profile.
|
||||
- `NOTARIUS_LLM_DEFAULT_TIMEOUT_SECONDS`: integer timeout seconds for the
|
||||
`default` LLM profile.
|
||||
- `NOTARIUS_LLM_DEFAULT_MAX_RETRIES`: integer retry count for the `default` LLM
|
||||
profile.
|
||||
- `NOTARIUS_LLM_DEFAULT_MAX_CONCURRENCY`: integer max concurrency for the
|
||||
`default` LLM profile.
|
||||
- `NOTARIUS_TOTAL_LLM_CONCURRENCY`: integer global LLM concurrency.
|
||||
- `NOTARIUS_WORK_DIR`: diagnostics work directory.
|
||||
- `NOTARIUS_DIAGNOSTICS_RETENTION`: diagnostics retention mode.
|
||||
|
||||
Integer environment values must parse as base-10 integers.
|
||||
|
||||
## Pipelines
|
||||
|
||||
A pipeline defines the fixed Notarius workflow:
|
||||
|
||||
```text
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
|
||||
Pipeline fields:
|
||||
|
||||
- `input`: required module binding.
|
||||
- `chunk`: optional module binding. Default module is `generic`.
|
||||
- `artifacts`: required for pipeline resolution. It maps artifact lane IDs to
|
||||
lane definitions.
|
||||
- `output`: optional module binding. Default module is `json`.
|
||||
|
||||
Artifact lane fields:
|
||||
|
||||
- `extract`: required module binding.
|
||||
- `merge`: optional module binding. Default module is `appendorder`.
|
||||
- `normalize`: optional module binding. Default module is `noop`.
|
||||
- `validators`: optional list of module bindings. The production CLI currently
|
||||
does not register validator modules.
|
||||
|
||||
`notarius run` and `notarius config validate --pipeline` resolve the pipeline
|
||||
against the production module catalog and fail fast for unknown or incompatible
|
||||
module keys.
|
||||
|
||||
## Module Bindings
|
||||
|
||||
Every module binding may use shorthand:
|
||||
|
||||
```yaml
|
||||
input: seriatim
|
||||
```
|
||||
|
||||
or object form:
|
||||
|
||||
```yaml
|
||||
chunk:
|
||||
module: generic
|
||||
llm_profile: default
|
||||
options:
|
||||
max_units: 50
|
||||
```
|
||||
|
||||
Binding fields:
|
||||
|
||||
- `module`: module key.
|
||||
- `llm_profile`: optional LLM profile ID. Empty means `default`.
|
||||
- `options`: optional module-specific settings.
|
||||
|
||||
The `--llm-profile` run flag overrides every effective module binding to use
|
||||
one configured profile.
|
||||
|
||||
## Implemented Production Modules
|
||||
|
||||
| Slot | Key | Notes |
|
||||
| --- | --- | --- |
|
||||
| input | `seriatim` | Reads Seriatim minimal transcript JSON. |
|
||||
| chunk | `generic` | Splits source units into ordered chunks. |
|
||||
| extract | `dnd/spells` | Extracts `dnd.spell_cast` artifacts. |
|
||||
| merge | `appendorder` | Keeps candidates in append order. |
|
||||
| normalize | `noop` | Passes merged artifacts through unchanged. |
|
||||
| output | `json` | Produces JSON output files. |
|
||||
|
||||
The `generic` chunker accepts:
|
||||
|
||||
- `max_units`: positive integer, default `50`;
|
||||
- `overlap_units`: non-negative integer, default `0`, and must be less than
|
||||
`max_units`.
|
||||
|
||||
## Diagnostics
|
||||
|
||||
`diagnostics` fields:
|
||||
|
||||
- `work_dir`: directory for per-run diagnostics. Default: `/tmp/notarius`.
|
||||
- `retention`: `auto`, `always`, or `never`. Empty uses `auto`.
|
||||
|
||||
`auto` retains diagnostics for failed runs and successful runs with warnings.
|
||||
`always` retains diagnostics for every run. `never` removes diagnostics for
|
||||
successful runs without regard to warnings; failed runs are retained.
|
||||
|
||||
The `--diagnostics-dir` run flag overrides `diagnostics.work_dir` for that
|
||||
invocation.
|
||||
|
||||
## Validation
|
||||
|
||||
Configuration validation checks:
|
||||
|
||||
- supported config version and known YAML fields;
|
||||
- non-empty, non-duplicated IDs after trimming;
|
||||
- supported LLM provider and non-negative profile limits;
|
||||
- positive global LLM concurrency;
|
||||
- supported diagnostics retention and non-empty work directory;
|
||||
- module binding LLM profiles refer to configured profiles.
|
||||
|
||||
Pipeline resolution additionally checks:
|
||||
|
||||
- the pipeline ID exists;
|
||||
- at least one artifact lane is declared and selected;
|
||||
- selected lanes exist when `--only` is used;
|
||||
- required module keys are present;
|
||||
- module keys are registered for the expected slot;
|
||||
- module capability requirements are satisfied.
|
||||
Reference in New Issue
Block a user