From ee43add75cc44870d933f4ecf8824af35a42023b Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 26 Jul 2026 13:11:40 +0000 Subject: [PATCH] Rewrite CLI orientation and reference --- README.md | 63 +++++---- docs/cli.md | 338 ++++++++++++++------------------------------ docs/development.md | 5 +- 3 files changed, 142 insertions(+), 264 deletions(-) diff --git a/README.md b/README.md index 01bb58b..f1dcd1b 100644 --- a/README.md +++ b/README.md @@ -1,34 +1,43 @@ # Notarius -Notarius is a Go CLI for extracting structured artifacts from source material -with explicit, configurable pipeline modules. +Notarius is a Go CLI for turning source material into structured artifacts with +configured extraction pipelines. The implemented D&D workflow reads Seriatim +transcript JSON and can produce scene descriptions, item and currency events, +NPC identities, combat turns, NPC interactions, and spell casts. -The current implementation reads Seriatim transcript JSON, chunks the source -units, extracts D&D spell-cast artifacts with a Scriptorium-backed LLM runtime, -and writes JSON output. Add `--debug` when a per-run inspection bundle is -needed. +## Quickstart -```sh -OPENROUTER_API_KEY=... \ -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-minimal.config.yml \ - --input examples/seriatim-minimal-transcript.json -``` +Provide an OpenRouter API key through the environment, then run the maintained +minimal example: -This invocation uses the maintained example configuration and input. See the -configuration and operations references for profile selection, credentials, and -run artifacts. +~~~ +OPENROUTER_API_KEY=your-api-key \ + go run ./cmd/notarius run dnd-session \ + --config examples/dnd-minimal.config.yml \ + --input examples/seriatim-minimal-transcript.json +~~~ -Useful references: +The command publishes a JSON output bundle. Its command syntax and exit +behavior are documented in the [CLI reference](docs/cli.md); configuration, +credentials, and module selection are owned by the +[configuration reference](docs/config.md). -- [CLI reference](docs/cli.md) -- [Configuration reference](docs/config.md) -- [Operations](docs/operations.md) -- [Seriatim input contract](docs/integrations/seriatim.md) -- [JSON output contract](docs/integrations/json-output.md) -- [D&D spell artifact contract](docs/integrations/dnd-spell-artifacts.md) -- [Developer guide](docs/development.md) -- [Internal implementation docs](docs/internal/overview.md) -- [Minimal D&D configuration](examples/dnd-minimal.config.yml) -- [Complete D&D configuration](examples/dnd-complete.config.yml) -- [Maintained example input](examples/seriatim-minimal-transcript.json) +For the complete ordered D&D workflow, use +[the complete configuration](examples/dnd-complete.config.yml) with +[its synthetic transcript](examples/dnd-complete-transcript.json). It +demonstrates all implemented D&D lanes and the supporting campaign references. + +## Documentation + +- [CLI reference](docs/cli.md) — commands, flags, output streams, and exits. +- [Configuration reference](docs/config.md) — configuration files, profiles, + validation, and module selection. +- [Operations](docs/operations.md) — output, state, recovery, and debug + handling. +- [Integration contracts](docs/integrations/) — Seriatim input and published + artifact formats. +- [Internal overview](docs/internal/overview.md) — implemented component map + for maintainers. +- [Developer guide](docs/development.md) — contributor orientation and + validation guidance. +- [Future work](docs/roadmap/future.md) — unimplemented ideas and priorities. diff --git a/docs/cli.md b/docs/cli.md index b58152b..8090525 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -1,278 +1,146 @@ # CLI Reference This is the canonical reference for the implemented Notarius command-line -interface. +interface. For the shortest successful run, see the [README](../README.md). +Configuration fields, discovery rules, and selectable module keys are defined +in [Configuration](config.md); runtime state and recovery procedures are +defined in [Operations](operations.md). -For the minimal end-to-end invocation, see the [README](../README.md). +## Command Summary -## Commands - -```text +~~~ notarius help -notarius run --input path/to/source.json [--config path/to/config.yml] [--only lane-a,lane-b] [--chunk_cache auto|bypass|refresh] [--output-dir path] [--resume] [--recompute-step step-id] [--debug [--debug-dir path]] [--llm-profile id] [--session-id id] [--reference selector=path] [--without-reference selector] +notarius run --input path/to/source.json [flags] notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b] notarius pipelines list [--config path/to/config.yml] [--json] -``` +~~~ -Running `notarius` with no arguments, `notarius help`, `notarius --help`, or -`notarius -h` prints usage and exits successfully. +Running Notarius without arguments, or with **help**, **--help**, or **-h**, +writes the command summary to standard output and exits with status 0. -## `run` +## run -`notarius run ` executes a configured pipeline against one input -file. +~~~ +notarius run --input path/to/source.json [flags] +~~~ -Flags: +The **run** command executes the named pipeline for one input file. The +pipeline ID and **--input** are required. -- `--input path`: required source input file. -- `--config path`: config file path. If omitted, Notarius uses the discovery - rules in [Configuration](config.md#discovery). -- `--only lane-a,lane-b`: run only the named artifact lanes. Values are - comma-separated and must be non-empty. This retains its existing behavior for - implicit single-step pipelines; explicit multi-step pipelines reject it - rather than inferring dependency closure. -- `--resume`: request checkpoint reuse for this invocation. Checkpoint recording - must be enabled in configuration. See - [Operations](operations.md#checkpoint-cache) for prerequisites and reuse - behavior. -- `--recompute-step step-id`: with `--resume` and checkpoint recording enabled, - force the named ordered step and every transitive dependent lane to execute. - Compatible required predecessors and unrelated lanes remain reusable. The - value may identify an explicit step or the implicit single-step ID `default`; - it cannot be combined with `--only`. -- `--chunk_cache auto|bypass|refresh`: select chunk-plan reuse for this - invocation. `auto` reuses a valid plan by canonical source digest, `bypass` - performs no plan-cache I/O, and `refresh` regenerates and replaces a valid - plan only after chunk validation succeeds. See - [Configuration](config.md#state-surfaces) for the persistent setting, precedence, - and cache-root selection. -- `--output-dir path`: output root. Defaults to `./notarius-output`. -- `--debug`: allocate and retain one debug bundle for this invocation. -- `--debug-dir path`: debug-bundle root override. This flag requires `--debug`. -- `--llm-profile id`: override every effective LLM-capable pipeline module - binding with one Scriptorium profile ID. Validator-specific profiles are not - overridden. -- `--session-id id`: pass a stable prompt session identifier through LLM-backed - module calls. -- `--reference selector=path`: bind a reference path to a chunk, extractor, - merger, or normalizer reference slot. Repeatable. -- `--without-reference selector`: remove a configured optional reference binding. - Repeatable. It accepts the same selector forms as `--reference`, without - `=path`. +| Flag | Meaning | +| --- | --- | +| **--config path** | Use this configuration file. When omitted, configuration discovery applies; see [Configuration](config.md). | +| **--input path** | Source input file to process. Required. | +| **--output-dir path** | Override the configured output root for this run. | +| **--chunk_cache auto\|bypass\|refresh** | Override chunk-plan cache handling for this run. | +| **--resume** | Reuse compatible recorded checkpoints when checkpoint recording is enabled. | +| **--recompute-step step-id** | With **--resume**, recompute the selected ordered step and its dependent lanes. It cannot be combined with **--only**. | +| **--debug** | Retain a debug bundle for this run. | +| **--debug-dir path** | Override the debug-bundle root. Requires **--debug**. | +| **--only lane-a,lane-b** | Run only the selected comma-separated artifact lanes when that selection is valid for the configured pipeline. | +| **--llm-profile id** | Override effective LLM-capable module bindings with one configured profile. | +| **--session-id id** | Supply a non-empty prompt session identifier to LLM-backed module calls. | +| **--reference selector=path** | Add or replace a file reference binding. Repeatable. | +| **--without-reference selector** | Remove a configured optional reference binding. Repeatable. | -On success, the command prints the completed pipeline ID, normalized output and -rejected output counts, and the output directory. A debug-enabled run also -prints `debug=`. If the run completes with warnings, the warning -count is printed to stderr. +**--chunk_cache** accepts only **auto**, **bypass**, or **refresh**. +**--debug-dir**, **--output-dir**, **--session-id**, and +**--recompute-step** reject explicit empty values. **--recompute-step** +requires **--resume**; checkpoint requirements and reuse behavior are +documented in [Operations](operations.md). -Reference flags are external file bindings resolved against selected chunk, -extractor, merger, and normalizer targets before the run starts. Generated -artifact bindings are configured in ordered steps and cannot be introduced by a -CLI path flag. Flat slot names are accepted only -when exactly one selected target declares that slot. For configured reference -bindings, precedence, path resolution, and validation, see -[Configuration](config.md#pipelines). +### Reference selectors -`--reference` binds or replaces one slot for one selected target. Selectors are: +Use **--reference** only for a reference slot declared by the selected +configured target. The accepted selector forms are: -- `slot=path`: valid when exactly one selected target declares `slot`; -- `chunk.slot=path`: target the chunker; -- `merge.slot=path`: valid when exactly one selected merger declares `slot`; -- `lane.slot=path`: valid when exactly one selected extractor, merger, or - normalizer in that lane declares `slot`; -- `lane.extract.slot=path`: target a lane extractor; -- `lane.merge.slot=path`: target a lane merger; -- `lane.normalize.slot=path`: target a lane normalizer. +| Form | Target | +| --- | --- | +| slot=path | The unique selected target that declares slot. | +| chunk.slot=path | The chunker. | +| merge.slot=path | The unique selected merger that declares slot. | +| lane.slot=path | The unique extractor, merger, or normalizer in lane that declares slot. | +| lane.extract.slot=path | The extractor in lane. | +| lane.merge.slot=path | The merger in lane. | +| lane.normalize.slot=path | The normalizer in lane. | -Use `slot=path` when the selected targets declare the slot unambiguously: +**--without-reference** uses the same selector forms without =path. Slot +names, requiredness, and configured bindings are part of the +[configuration contract](config.md). -```sh -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-minimal.config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --reference roster=./campaign-roster.txt -``` +### Run output -Use an explicit selector when multiple selected targets declare the same slot or -when you want to target a specific target: +On success, standard output contains the completed pipeline ID, counts of +normalized and rejected outputs, and the output directory. A debug-enabled run +also prints its debug-bundle path to standard output. A successful run with +warnings reports the warning count to standard error. The published JSON +envelope is defined by the [JSON output contract](integrations/json-output.md). -```sh -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-minimal.config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --reference spells.extract.glossary=./campaign-glossary.txt -``` +Example: -For the maintained ordered D&D workflow, use the explicit pipeline. The first -step produces normalized NPC and scene-description artifacts; the second step -receives the NPC artifact in memory for spell extraction, combat extraction, and -combat normalization, and receives the required scene-description artifact for -combat eligibility: +~~~ +OPENROUTER_API_KEY=your-api-key \ + go run ./cmd/notarius run dnd-session \ + --config examples/dnd-minimal.config.yml \ + --input examples/seriatim-minimal-transcript.json +~~~ -```sh -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-complete.config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --output-dir ./npc-grounded-output -``` +## config validate -The generated NPC content remains contextual grounding, not spell or combat -evidence. It is represented in manifests and debug summaries by bounded -identity and producer provenance, not by payload content or a filesystem path. -The scene-description artifact is control context: combat extraction calls its -LLM only for an exact `combat` scene match. See the -[D&D combat-turn artifact contract](integrations/dnd-combat-turn-artifacts.md) -for the resulting empty-output and warning behavior. +~~~ +notarius config validate [--config path/to/config.yml] [--pipeline pipeline-id] [--only lane-a,lane-b] +~~~ -The same grammar can target chunk, merge, and normalize slots when the configured -modules declare them: +This command loads and validates a configuration. With **--pipeline**, it also +resolves that pipeline against the production module catalog. **--only** selects +lanes during that resolution and requires **--pipeline**. -```sh -go run ./cmd/notarius run dnd-session \ - --config path/to/config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --reference chunk.scene_guide=./campaign-scenes.txt \ - --reference spells.merge.merge_notes=./merge-notes.txt \ - --reference spells.normalize.normalization_notes=./normalization-notes.txt -``` - -Use `--without-reference` to remove a configured optional binding for a run: - -```sh -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-minimal.config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --without-reference glossary -``` - -Use `--session-id` when an external orchestrator needs all prompt calls from one -run to share an identifier: - -```sh -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-minimal.config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --session-id campaign-17-session-04 -``` - -When `cache.checkpoints.enabled` is `true`, runs record checkpoints whether or -not `--resume` is present. Add the resume flag to load and reuse compatible -recorded work; using it while checkpoint recording is disabled is an error: - -```sh -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-minimal.config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --resume -``` - -To selectively rerun one ordered step and its dependent lanes, use the step ID -from the configuration. The selected step and dependents are reported as -`forced_recompute`; reusable predecessors are reported as `reused`: - -```sh -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-complete.config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --resume --recompute-step extract-events -``` - -Checkpoint decisions use these categories: `reused`, `executed`, -`forced_recompute`, and `dependency_invalidated`. The reason code and bounded -detail identify the decision without exposing reference content, local paths, -or secrets. `--recompute-step` requires checkpoint recording and `--resume`; -unknown step IDs, empty values, and combinations with `--only` are rejected. -The operator meanings of checkpoint reason codes are maintained in -[Operations](operations.md#resume-and-selective-recompute). - -Use `--debug` to retain the redacted summary and trace bundle for one run. The -bundle is allocated before pipeline resolution; once allocated, its path is -also printed to stderr if the command fails. Debug-write failures cause exit -code `1`. - -```sh -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-minimal.config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --debug --debug-dir ./notarius-debug -``` - -Use `refresh` when intentionally replacing the cached plan for the same source: - -```sh -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-minimal.config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --chunk_cache refresh -``` - -Use `bypass` for a one-off run that must not inspect or create plan-cache state: - -```sh -go run ./cmd/notarius run dnd-session \ - --config examples/dnd-minimal.config.yml \ - --input examples/seriatim-minimal-transcript.json \ - --chunk_cache bypass -``` - -`--diagnostics-dir` has been removed. For checkpoint behavior, durable output, -debug-bundle lifecycle, and failure inspection, see [Operations](operations.md). - -## `config validate` - -`notarius config validate` loads and validates configuration. - -Flags: - -- `--config path`: config file path. If omitted, Notarius uses the discovery - rules in [Configuration](config.md#discovery). -- `--pipeline pipeline-id`: additionally resolve one configured pipeline against - the production module catalog. -- `--only lane-a,lane-b`: validate resolution for selected artifact lanes. This - flag requires `--pipeline`. +Success is written to standard output as either config "" is valid or +config "" is valid for pipeline "". Examples: -```sh -go run ./cmd/notarius config validate \ - --config examples/dnd-minimal.config.yml - +~~~ go run ./cmd/notarius config validate \ --config examples/dnd-minimal.config.yml \ - --pipeline dnd-session \ - --only spells -``` + --pipeline dnd-session -## `pipelines list` +OPENROUTER_API_KEY=validation-placeholder \ + go run ./cmd/notarius config validate \ + --config examples/dnd-complete.config.yml \ + --pipeline dnd-session +~~~ -`notarius pipelines list` prints configured pipeline IDs in sorted order. +The placeholder in the second command is sufficient only for offline +validation; it cannot run a provider-backed pipeline. -Flags: +## pipelines list -- `--config path`: config file path. If omitted, Notarius uses the discovery - rules in [Configuration](config.md#discovery). -- `--json`: print `{"pipelines":[...]}` instead of one ID per line. +~~~ +notarius pipelines list [--config path/to/config.yml] [--json] +~~~ -Examples: +This command lists configured pipeline IDs in sorted order. By default, it +writes one ID per line to standard output. **--json** writes an object shaped as +{"pipelines":[...]} instead. -```sh +~~~ go run ./cmd/notarius pipelines list \ --config examples/dnd-minimal.config.yml +~~~ -go run ./cmd/notarius pipelines list \ - --config examples/dnd-minimal.config.yml \ - --json -``` +## Output Streams And Exit Statuses -## Exit Codes +Successful commands write their primary result to standard output. Warnings and +errors are written to standard error. -- `0`: command succeeded. -- `1`: command syntax was valid, but loading config, resolving modules, running - the pipeline, calling the provider, writing output, or writing a requested - debug bundle failed. -- `2`: command syntax was invalid, a command was unknown, a required argument - was missing, or a flag value was malformed. +| Status | Meaning | +| --- | --- | +| 0 | The command completed successfully, including root help. | +| 1 | Command syntax was valid but configuration loading or validation, pipeline resolution or execution, provider use, output, or requested debug handling failed. | +| 2 | The command or flag syntax was invalid, including unknown commands, missing required arguments, invalid flag values, or invalid flag combinations. | -For YAML structure, defaults, Scriptorium profile sources, environment -overrides, and selectable module and validator keys, see -[Configuration](config.md). +The root help spellings are the supported help path. Invoking **--help** on +**run**, **config validate**, or **pipelines list** is handled by the flag +parser as a usage error: it writes an error to standard error and exits with +status 2. diff --git a/docs/development.md b/docs/development.md index 19bd8cb..b4ba597 100644 --- a/docs/development.md +++ b/docs/development.md @@ -17,11 +17,12 @@ implemented component map. | Application shape, package boundaries, contracts, dependency direction, runtime guarantees, or safety properties | [Architecture](policy/architecture.md) and relevant [ADRs](adr/) | Architecture defines the intended system and its invariants; ADRs preserve significant decision rationale. | | Any documentation addition or revision | [Documentation Policy](policy/documentation.md) | It defines canonical homes, audiences, current-behavior rules, and maintenance requirements. | | Adding, changing, reviewing, or deleting tests | [Testing Policy](policy/testing.md) | It defines risk-based sufficiency, durable test boundaries, test-double guidance, and criteria for retaining tests. | +| CLI composition or command behavior | [CLI Reference](cli.md) | It owns public command syntax; focused CLI internals are documented separately as they are introduced. | +| Configuration loading, resolution, or user-visible configuration behavior | [Configuration](config.md) | It owns the configuration contract; focused configuration internals are documented separately as they are introduced. | | Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. | -| Production modules or validators | [Module Internals](internal/modules.md) | It documents implemented module contracts, capabilities, assets, and registration. | +| Production modules or validators | [Module Internals](internal/modules.md) and [D&D integration contracts](integrations/) | The generic module guide owns extension mechanics; D&D artifact contracts own durable output shapes. | | LLM clients, prompts, schemas, profiles, or scheduling | [LLM Runtime](internal/llm.md) | It documents the transport boundary and Scriptorium integration. | | Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. | -| CLI or user-visible configuration behavior | [CLI Reference](cli.md) and [Configuration](config.md) | These are the canonical user and operator references. | | External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. | | Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. |