From b69ba96811a3cddbea0ef3e1f44bb138b210498d Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Tue, 26 May 2026 03:21:11 +0000 Subject: [PATCH] Rewrite README and add canonical CLI/config documentation --- README.md | 447 +---------------------------- docs/cli.md | 139 +++++++++ docs/config.md | 206 +++++++++++++ docs/config/config-yml.md | 49 +--- docs/config/profile-definitions.md | 42 +-- docs/config/prompt-definitions.md | 74 +---- docs/config/schema-definitions.md | 83 +----- 7 files changed, 370 insertions(+), 670 deletions(-) create mode 100644 docs/cli.md create mode 100644 docs/config.md diff --git a/README.md b/README.md index 86ad3f4..1113e1f 100644 --- a/README.md +++ b/README.md @@ -1,449 +1,28 @@ # scriptorium -Scriptorium is a generic prompt execution engine. +Scriptorium is a config-driven prompt execution engine. -It takes: -- a prompt definition -- a selected or default execution profile -- named input artifacts -- template variables -- optional runtime overrides +It separates prompt definitions (what to generate) from execution profiles (how to call an OpenAI-compatible model endpoint), then runs or renders a prepared request from named input artifacts. -It returns: -- for `run`: generated artifact, validation result, metadata -- for `render`: prepared/rendered prompt data (no model output) +## Quickstart -## Prompt vs Profile +From the repository root: -Scriptorium separates **what** to do (Prompt) from **how** to do it (Profile). - -### Prompt Definition -Defines the task logic and output contract. -- Task description and version. -- Message templates (system, user, etc.). -- Required and optional input artifacts. -- Output format and validation rules. -- Repair settings for structured output. -- Optional `default_profile` for convenience. - -### Execution Profile -Defines the runtime environment and model settings. -- LLM endpoint (URL). -- Model name. -- Generation parameters: `temperature`, `max_tokens`, `top_p`. -- Runtime settings: `timeout`, `reasoning_effort`. -- API key source via `api_key_env`. - -Callers can explicitly provide a `profile_id` to override the prompt's `default_profile`. - -## Precedence - -Scriptorium uses two precedence layers: - -### Application Configuration Precedence - -For application-level adapter settings (for example prompt/profile/schema directories, server address, and render output default), precedence is: - -1. **CLI Flags** -2. **`config.yml`** -3. **Built-in application defaults** - -Application config loading behavior: -- Default config path: `/etc/scriptorium/config.yml` -- Override path: `--config ` (supported by `run`, `render`, and `serve`) -- If `--config` is provided, the file must exist and be valid. -- If `--config` is omitted, missing `/etc/scriptorium/config.yml` is allowed. - -### Runtime Model Precedence - -When resolving runtime model settings, Scriptorium follows this precedence model (highest to lowest): - -1. **Runtime Overrides**: Provided via CLI flags or HTTP request `model` object. -2. **Execution Profile**: Settings defined in the selected profile. -3. **Application Defaults**: Built-in fallback values. - -### Profile Selection Logic -The engine determines which profile to use in this order: -1. Explicit `profile_id` (via `--profile` or HTTP request). -2. The `default_profile` named in the Prompt Definition. -3. Error: If neither is provided and no default exists. - -## API Key Policy - -To ensure security, Scriptorium does not support raw API keys in configuration files, CLI arguments, or HTTP requests. - -- **`api_key_env`**: Profiles and overrides specify the name of an environment variable (e.g., `SCRIPTORIUM_API_KEY`). -- **Runtime Resolution**: The value of the environment variable is read directly from the process environment at runtime. -- **Zero Leakage**: API key values are never included in metadata, logs, or response bodies. - -## CLI Usage - -Available CLI commands: -- `scriptorium run` -- `scriptorium render` -- `scriptorium serve` - -All commands accept `--config `. - -`prompt_dir` and `profile_dir` may be supplied by CLI flags or `config.yml`: -- `--prompt-dir` or `config.yml` `prompt_dir` -- `--profile-dir` or `config.yml` `profile_dir` - -`schema_dir` and `serve` `addr` may also be supplied by `config.yml` where applicable: -- `--schema-dir` or `config.yml` `schema_dir` -- `--addr` or `config.yml` `server.addr` - -### `scriptorium run` - -Runs a single prompt execution. - -**Required Flags:** -- `--prompt`: The prompt ID to execute. -- `--input`: Input mapping `name=path` (repeatable). - -**Required Effective Settings:** -- Prompt directory: `--prompt-dir` or `config.yml` `prompt_dir` -- Profile directory: `--profile-dir` or `config.yml` `profile_dir` - -**Optional Flags:** -- `--config`: Application config file path. Default discovery path is `/etc/scriptorium/config.yml`. -- `--prompt-dir`: Override prompt directory from config. -- `--profile-dir`: Override profile directory from config. -- `--profile`: Override the prompt's default profile. -- `--var`: Template variable `name=value` (repeatable). -- `--out`: Write output to a file instead of stdout. -- `--llm-base-url`: Override endpoint. -- `--model`: Override model name. -- `--api-key-env`: Override API key environment variable name. -- `--temperature`: Override temperature. -- `--max-tokens`: Override max tokens. -- `--top-p`: Override top_p. -- `--timeout`: Override request timeout (e.g., `30s`, `1m`). -- `--schema-dir`: Base directory for validation schemas. - -**Examples:** - -Using `config.yml` for prompt/profile directories: ```bash -scriptorium run \ - --prompt generic.markdown_summary \ - --input transcript=./examples/fixtures/transcript.md -``` - -Overriding config directories explicitly: -```bash -scriptorium run \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --prompt generic.markdown_summary \ - --input transcript=./examples/fixtures/transcript.md -``` - -Overriding the profile: -```bash -scriptorium run \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --prompt generic.markdown_summary \ - --profile local-quality \ - --input transcript=./examples/fixtures/transcript.md -``` - -Overriding model and runtime values: -```bash -scriptorium run \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --prompt generic.markdown_summary \ - --model gpt-4o \ - --temperature 0.7 \ - --input transcript=./examples/fixtures/transcript.md -``` - -Using a local OpenAI-compatible vLLM endpoint: -```bash -scriptorium run \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --prompt generic.markdown_summary \ - --llm-base-url http://localhost:8000/v1 \ - --model meta-llama-3-8b \ - --input transcript=./examples/fixtures/transcript.md -``` - -### `scriptorium render` - -Prepares and renders a prompt without calling the LLM. - -`render` uses the same prompt/profile/input/variable/runtime override resolution as `run`: -- Profile selection precedence: `--profile` -> prompt `default_profile` -> error. -- Runtime precedence: CLI runtime overrides -> selected profile -> built-in defaults. - -`render` is useful for debugging: -- prompt template rendering -- input mappings -- selected profile behavior -- runtime override behavior - -`render` does not: -- call the LLM -- validate model output -- perform repair -- expose resolved API key values - -It may include `api_key_env` names where relevant. - -**Required Flags:** -- `--prompt`: The prompt ID to render. -- `--input`: Input mapping `name=path` (repeatable). - -**Required Effective Settings:** -- Prompt directory: `--prompt-dir` or `config.yml` `prompt_dir` -- Profile directory: `--profile-dir` or `config.yml` `profile_dir` - -**Optional Flags:** -- `--config`: Application config file path. Default discovery path is `/etc/scriptorium/config.yml`. -- `--prompt-dir`: Override prompt directory from config. -- `--profile-dir`: Override profile directory from config. -- `--profile`: Override the prompt's default profile. -- `--var`: Template variable `name=value` (repeatable). -- `--out`: Write output to a file instead of stdout. -- `--format`: Render output format (`text` or `json`). Default: `text`. -- `--llm-base-url`: Runtime override for endpoint. -- `--model`: Runtime override for model name. -- `--api-key-env`: Runtime override for API key environment variable name. -- `--temperature`: Runtime override for temperature. -- `--max-tokens`: Runtime override for max tokens. -- `--top-p`: Runtime override for top_p. -- `--timeout`: Runtime override for timeout (e.g., `30s`, `1m`). - -**Render Output Formats:** -- `text`: Human-readable output (default). -- `json`: Machine-readable structured output. - -Render formatting is modular; additional output formats can be added later without changing prepare/run core logic. - -**Examples:** - -Default text output using `config.yml` directories: -```bash -scriptorium render \ - --prompt generic.markdown_summary \ - --input transcript=./examples/fixtures/transcript.md -``` - -Explicit config path: -```bash -scriptorium render \ +go run ./cmd/scriptorium render \ --config ./examples/config.yml \ --prompt generic.markdown_summary \ - --input transcript=./examples/fixtures/transcript.md -``` - -Explicit directory overrides: -```bash -scriptorium render \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --prompt generic.markdown_summary \ - --input transcript=./examples/fixtures/transcript.md -``` - -Explicit JSON output: -```bash -scriptorium render \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --prompt generic.markdown_summary \ --input transcript=./examples/fixtures/transcript.md \ + --input glossary=./examples/fixtures/glossary.yml \ --format json ``` -Using prompt `default_profile` (omit `--profile`): -```bash -scriptorium render \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --prompt generic.markdown_summary \ - --input transcript=./examples/fixtures/transcript.md -``` +This command renders the prepared prompt and effective runtime settings without calling an LLM. -Overriding profile selection: -```bash -scriptorium render \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --prompt generic.markdown_summary \ - --profile local-quality \ - --input transcript=./examples/fixtures/transcript.md -``` +## Documentation -Overriding runtime settings: -```bash -scriptorium render \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --prompt generic.markdown_summary \ - --input transcript=./examples/fixtures/transcript.md \ - --llm-base-url http://localhost:8000/v1 \ - --model gpt-4o-mini \ - --temperature 0.2 \ - --max-tokens 800 \ - --top-p 1.0 \ - --timeout 45s -``` - -Writing rendered output to a file: -```bash -scriptorium render \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --prompt generic.markdown_summary \ - --input transcript=./examples/fixtures/transcript.md \ - --format text \ - --out ./rendered_prompt.txt -``` - -### `scriptorium serve` - -Starts the HTTP API. - -**Required Effective Settings:** -- Prompt directory: `--prompt-dir` or `config.yml` `prompt_dir` -- Profile directory: `--profile-dir` or `config.yml` `profile_dir` - -**Optional Flags:** -- `--config`: Application config file path. Default discovery path is `/etc/scriptorium/config.yml`. -- `--addr`: Listen address (default `:8080`). -- `--schema-dir`: Base directory for validation schemas. - -**Examples:** - -Using `config.yml`: -```bash -scriptorium serve -``` - -Overriding config for local use: -```bash -scriptorium serve \ - --prompt-dir ./prompts \ - --profile-dir ./profiles \ - --addr :9090 -``` - -## HTTP API - -### `POST /v1/runs` - -Executes a prompt. No built-in authentication is provided; deploy behind a trusted gateway. - -**Request Body:** -```json -{ - "prompt_id": "generic.structured_events", - "profile_id": "local-quality", - "include_raw_output": false, - "inputs": { - "transcript": {"type": "file", "uri": "./examples/fixtures/transcript.md"} - }, - "vars": { - "session_date": "2026-05-04" - }, - "model": { - "endpoint": "http://localhost:8000/v1", - "model": "gpt-4o-mini", - "temperature": 0.0 - } -} -``` - -`profile_id` is optional. If omitted, Scriptorium uses the prompt's `default_profile`. If neither is available, the run fails. - -**Response:** -Returns a `200 OK` with the generated artifact, validation results, and metadata including the `prompt_id` and the `selected_profile_id`. - -**Validation Failures:** -If the model output fails validation (e.g., invalid JSON), the API returns `200 OK` with `validation.status = "failed"`. - -**Raw Output Exposure:** -- `raw_model_output` is omitted by default. -- Set `include_raw_output: true` in the request to include it in the response. -- Raw output is preserved internally in run results regardless of HTTP exposure. - -## Prompt Definition Authoring - -Prompts are defined in YAML. - -### Canonical Shape -```yaml -id: generic.structured_events -version: "1.0.0" -description: "Extracts structured events from a transcript" -default_profile: local-quality - -inputs: - - name: transcript - required: true - content_type: text/markdown - description: "The raw session transcript" - - name: glossary - required: false - content_type: application/yaml - description: "Optional glossary terms" - -messages: - - role: system - content: "You are a helpful assistant." - - role: user - content_file: messages/extract_events.tmpl - -output: - format: json - validation_mode: json_schema - schema_path: structured_events.schema.json - repair_attempts: 2 -``` - -**Key Features:** -- **Inline vs File**: Use `content` for short prompts or `content_file` for larger templates. Exactly one must be set per message. -- **Path Resolution**: `content_file` paths are resolved relative to the prompt YAML file. -- **Inputs**: Mark inputs as `required` to ensure the runner fails early if they are missing. -- **Input Metadata**: `content_type` is currently descriptive metadata and not enforced yet. -- **Validation**: Support `none`, `basic`, `json`, and `json_schema`. -- **Repair**: `repair_attempts` enables bounded retries to fix structured output. - -## Execution Profile Authoring - -Profiles are defined in YAML. - -### Canonical Shape -```yaml -id: local-quality -endpoint: http://localhost:8000/v1 -model: gpt-4o -temperature: 0.0 -max_tokens: 4096 -top_p: 1.0 -timeout_seconds: 300 -reasoning_effort: high -api_key_env: SCRIPTORIUM_API_KEY -``` - -**Constraints:** -- **No Raw Keys**: Do not include actual API keys. Only specify the environment variable name in `api_key_env`. -- **Local Profiles**: For local endpoints that don't require auth, `api_key_env` can be omitted. - -## Examples - -- **Prompt Definitions**: `prompts/` -- **Execution Profiles**: `profiles/` -- **Schemas**: `schemas/` -- **Fixtures**: `examples/fixtures/` - -## Build and Test - -```bash -go build -o scriptorium ./cmd/scriptorium -go test ./... -``` +- [CLI reference](docs/cli.md) +- [Configuration reference](docs/config.md) +- [Narratio subprocess integration](docs/integrations/narratio.md) +- [Architecture policy](docs/policy/architecture.md) +- [Documentation roadmap](docs/roadmap/documentation.md) diff --git a/docs/cli.md b/docs/cli.md new file mode 100644 index 0000000..b5bbe5e --- /dev/null +++ b/docs/cli.md @@ -0,0 +1,139 @@ +# CLI Reference + +## Shortest Useful Command + +```bash +go run ./cmd/scriptorium render \ + --config ./examples/config.yml \ + --prompt generic.markdown_summary \ + --input transcript=./examples/fixtures/transcript.md \ + --input glossary=./examples/fixtures/glossary.yml +``` + +`render` prepares and formats the prompt without calling an LLM. + +## Command Overview + +- `scriptorium run`: prepare prompt, call the configured LLM, write generated output, print a run summary. +- `scriptorium render`: prepare prompt only; write prepared-run output as `text` or `json`. +- `scriptorium serve`: start the HTTP server. + +## Common Argument Rules + +- `--config` is supported by `run`, `render`, and `serve`. +- `run` and `render` require: + - `--prompt` + - at least one `--input` + - an effective `prompt_dir` and `profile_dir` (from flags or config) +- `serve` requires an effective `prompt_dir` and `profile_dir` (from flags or config). +- Positional arguments are rejected. + +## Flag Reference + +### `scriptorium run` + +- `--config `: app config file path. +- `--prompt-dir `: prompt definition directory. +- `--profile-dir `: profile definition directory. +- `--schema-dir `: schema base directory for `json_schema` validation. +- `--prompt `: prompt ID to execute. Required. +- `--prompt-id `: deprecated alias for `--prompt`. +- `--profile `: explicit profile override. +- `--profile-id `: deprecated alias for `--profile`. +- `--input name=path`: input mapping (repeatable, comma-separated accepted). +- `--var name=value`: template variable mapping (repeatable, comma-separated accepted). +- `--out `: write artifact body to file instead of stdout. +- `--llm-base-url `: runtime endpoint override. +- `--model `: runtime model override. +- `--api-key-env `: runtime API key environment-variable name override. +- `--temperature `: runtime temperature override. +- `--max-tokens `: runtime max tokens override. +- `--top-p `: runtime top-p override. +- `--timeout `: runtime timeout override (Go duration syntax, for example `30s`, `2m`). + +### `scriptorium render` + +- Supports the same flags as `run`, except: + - no `--schema-dir` flag. +- Adds: + - `--format text|json`: prepared-run output format. + +Notes: +- `render` still resolves profile and runtime settings. +- `render` still validates that `api_key_env` exists if the selected profile or overrides require it. + +### `scriptorium serve` + +- `--config `: app config file path. +- `--addr `: HTTP listen address. +- `--prompt-dir `: prompt definition directory. +- `--profile-dir `: profile definition directory. +- `--schema-dir `: schema base directory for `json_schema` validation. + +Notes: +- `serve` does not accept runtime model override flags such as `--model` or `--llm-base-url`. + +## Input And Variable Syntax + +- `--input name=path` maps prompt input names to local file paths. +- `--var name=value` maps template variable names to values. +- Both flags can be repeated. +- Both flags also support comma-separated batches, for example: + - `--input transcript=./t.md,glossary=./g.yml` + - `--var session_id=42,session_date=2026-05-04` + +## Output Behavior + +`run`: +- Writes generated artifact content to stdout by default. +- Writes generated artifact content to `--out` when provided. +- Prints run summary metadata to stderr on success. +- Prints errors to stderr on failure. + +`render`: +- Writes prepared-run output to stdout by default. +- Writes prepared-run output to `--out` when provided. +- Does not print a success summary line. + +`serve`: +- Logs startup and server errors to stderr. + +## Exit Codes + +- `0`: success. +- `1`: runtime/parse/config/load/render/generation/output-write error. +- `2`: `run` completed, output was generated, but validation status is `failed`. + +When `run` exits `2`, output may already be written to stdout or `--out`. + +## Common Workflows + +Render prompt inputs and template variables as JSON: + +```bash +go run ./cmd/scriptorium render \ + --config ./examples/config.yml \ + --prompt generic.markdown_summary \ + --input transcript=./examples/fixtures/transcript.md \ + --input glossary=./examples/fixtures/glossary.yml \ + --var session_date=2026-05-04 \ + --format json +``` + +Run a prompt with profile override and file output: + +```bash +go run ./cmd/scriptorium run \ + --config ./examples/config.yml \ + --prompt generic.markdown_summary \ + --profile local-fast \ + --input transcript=./examples/fixtures/transcript.md \ + --input glossary=./examples/fixtures/glossary.yml \ + --out ./out/summary.md +``` + +Start the HTTP server with explicit config: + +```bash +go run ./cmd/scriptorium serve --config ./examples/config.yml +``` diff --git a/docs/config.md b/docs/config.md new file mode 100644 index 0000000..798be04 --- /dev/null +++ b/docs/config.md @@ -0,0 +1,206 @@ +# Configuration Reference + +## Config Discovery And Precedence + +Application settings are loaded in this order: + +1. Built-in defaults +2. `config.yml` values +3. CLI overrides + +When `--config` is not provided, Scriptorium searches for config files in this order: + +1. `/usr/local/etc/scriptorium/config.yml` +2. `/etc/scriptorium/config.yml` + +If neither file exists, Scriptorium continues with built-in defaults. + +When `--config ` is provided, that file is required. + +## Minimal App Config + +```yaml +prompt_dir: ./prompts +profile_dir: ./profiles +``` + +This is enough to use `run` and `render` when prompt/profile files are valid. + +## Production-Oriented App Config + +```yaml +prompt_dir: /opt/scriptorium/prompts +profile_dir: /opt/scriptorium/profiles +schema_dir: /opt/scriptorium/schemas + +server: + addr: 127.0.0.1:8080 + +defaults: + render_format: text +``` + +## App Config File (`config.yml`) + +Top-level fields: + +- `prompt_dir` (optional): default prompt definition directory. +- `profile_dir` (optional): default profile definition directory. +- `schema_dir` (optional): base directory for schema files used by `json_schema` validation. +- `server.addr` (optional): default listen address for `serve`. +- `defaults.render_format` (optional): default `render` output format (`text` or `json`). + +Built-in defaults: + +- `schema_dir`: `.` +- `server.addr`: `:8080` +- `defaults.render_format`: `text` + +Validation behavior: + +- Config decoding is strict; unknown YAML fields are rejected. +- Raw API key fields are not supported in `config.yml`. + +## Prompt Definition Files + +Prompt definitions are YAML files in `prompt_dir`. + +Example: + +```yaml +id: generic.structured_events +version: "1.0.0" +default_profile: local-quality +description: Produce structured event JSON from a transcript. + +inputs: + - name: transcript + required: true + content_type: text/markdown + description: Source transcript content + - name: glossary + required: false + content_type: text/yaml + description: Optional glossary context + +messages: + - role: system + content_file: ./generic.structured_events.system.md + - role: user + content_file: ./generic.structured_events.user.md + +output: + format: json + validation_mode: json_schema + schema_path: structured_events.schema.json + repair_attempts: 0 +``` + +Field reference: + +- `id` (required): prompt identifier. +- `version` (required): prompt version. +- `default_profile` (optional): profile ID used when request does not provide `profile_id`. +- `description` (optional): prompt description. +- `inputs` (optional list): expected named inputs. +- `messages` (required list): prompt message templates. +- `output` (required object): output contract. + +`inputs[]` fields: + +- `name` (required) +- `required` (optional, boolean) +- `content_type` (optional metadata) +- `description` (optional) + +`messages[]` fields: + +- `role` (required) +- `content` or `content_file` (exactly one is required) + +Message rules: + +- Repeated roles are allowed. +- `content_file` is resolved relative to the prompt YAML file location. +- Prompt decoding is strict; unknown YAML fields are rejected. + +`output` fields: + +- `format` (required): `text`, `markdown`, or `json`. +- `validation_mode` (required): `none`, `basic`, `json`, or `json_schema`. +- `schema_path` (required when `validation_mode: json_schema`). +- `repair_attempts` (required): integer `>= 0`. + +Repair behavior boundary: + +- `repair_attempts` is part of the prompt contract. +- CLI and HTTP currently construct the runner without a repairer, so normal `run`/`serve` execution does not perform output repair attempts. + +## Profile Definition Files + +Execution profiles are YAML files in `profile_dir`. + +Example: + +```yaml +id: local-fast +endpoint: http://localhost:8000/v1 +model: gpt-4o-mini +temperature: 0.2 +max_tokens: 500 +top_p: 1.0 +timeout_seconds: 90 +api_key_env: SCRIPTORIUM_API_KEY +``` + +Field reference: + +- `id` (required) +- `endpoint` (required) +- `model` (required) +- `temperature` (optional): range `0..2` +- `max_tokens` (optional): `>= 0` +- `top_p` (optional): range `0..1` +- `timeout_seconds` (optional): `>= 0` +- `reasoning_effort` (optional) +- `api_key_env` (optional) +- `extra_params` (optional map of strings) + +Profile rules: + +- Profile decoding is strict; unknown YAML fields are rejected. +- Raw `api_key` is rejected; use `api_key_env`. +- If `api_key_env` is set, that environment variable must be set when preparing/running. + +Current outbound request behavior: + +- The OpenAI-compatible client currently serializes: `model`, `messages`, `temperature`, `max_tokens`, `top_p`, and optional `response_format` for `json_schema` prompts. +- `reasoning_effort` and `extra_params` are parsed and carried in effective settings, but are not currently serialized into outbound chat-completions requests. + +## Schema Behavior + +Schemas are JSON files, typically in `schema_dir`. + +Rules: + +- `output.validation_mode: json_schema` requires `output.schema_path`. +- Relative `schema_path` values resolve from `schema_dir`. +- Absolute `schema_path` values are used directly. +- Missing or invalid schema documents cause runtime validation errors. +- Invalid generated JSON causes validation status `failed` (not a runtime error). + +Supported artifact reference types for request inputs are `file` and `inline`. + +## Secrets Handling + +- Keep secret values in environment variables. +- Store only environment-variable names in profile `api_key_env`. +- Do not put raw API keys in config, prompts, profiles, CLI flags, or HTTP request bodies. + +## Maintained Examples + +- App config: `examples/config.yml` +- Prompt examples: `prompts/` +- Profile examples: `profiles/` +- Schema examples: `schemas/` +- Input fixtures: `examples/fixtures/` diff --git a/docs/config/config-yml.md b/docs/config/config-yml.md index 937d5d3..b20c7cd 100644 --- a/docs/config/config-yml.md +++ b/docs/config/config-yml.md @@ -1,48 +1,5 @@ -# Main `config.yml` +# Moved -`config.yml` defines application-level defaults used by CLI commands. +This page has moved. -By default, Scriptorium looks for `/usr/local/etc/scriptorium/config.yml` and, if not present, then for `/etc/scriptorium/config.yml`. You can also pass `--config PATH`. - -## Complete Example - -```yaml -prompt_dir: ./prompts -profile_dir: ./profiles -schema_dir: ./schemas - -server: - addr: :8080 - -defaults: - render_format: text -``` - -## Available Options - -- `prompt_dir` (optional): Default directory for prompt definition YAML files. -- `profile_dir` (optional): Default directory for execution profile YAML files. -- `schema_dir` (optional): Base directory for JSON schema files used by `json_schema` validation. - -### `server` - -- `addr` (optional): HTTP server listen address for `scriptorium serve`. - -### `defaults` - -- `render_format` (optional): Default output format for `scriptorium render`. -- Allowed values: `text`, `json`. - -## Precedence - -For run/render/serve settings, precedence is: - -1. Explicit CLI flags -2. `config.yml` -3. Built-in defaults - -## Notes and Rules - -- Unknown YAML fields fail to load (strict decoding). -- This file does not accept API keys. -- `config.yml` sets directory/server defaults only; prompt/profile content remains in their own files. +Use the canonical configuration reference: [../config.md](../config.md). diff --git a/docs/config/profile-definitions.md b/docs/config/profile-definitions.md index 23de3a2..b20c7cd 100644 --- a/docs/config/profile-definitions.md +++ b/docs/config/profile-definitions.md @@ -1,41 +1,5 @@ -# Execution Profile Definitions +# Moved -Execution Profiles define **how** Scriptorium calls an LLM endpoint. +This page has moved. -A profile file is YAML, typically stored under `profiles/`, for example `profiles/local-quality.yaml`. - -## Complete Example - -```yaml -id: local-quality -endpoint: http://localhost:8000/v1 -model: gpt-4.1 -temperature: 0.0 -max_tokens: 1200 -top_p: 1.0 -timeout_seconds: 180 -reasoning_effort: medium -api_key_env: SCRIPTORIUM_API_KEY -extra_params: - provider: openrouter - route: fallback -``` - -## Available Options - -- `id` (required): Unique profile identifier used by `--profile` or prompt `default_profile`. -- `endpoint` (required): OpenAI-compatible base URL, usually ending in `/v1`. -- `model` (required): Model name to request at that endpoint. -- `temperature` (optional): Sampling temperature. Valid range is `0` to `2`. -- `max_tokens` (optional): Max completion tokens. Must be `>= 0`. -- `top_p` (optional): Nucleus sampling parameter. Valid range is `0` to `1`. -- `timeout_seconds` (optional): Request timeout in seconds. Must be `>= 0`. -- `reasoning_effort` (optional): Provider/model-specific reasoning level string. -- `api_key_env` (optional): Environment variable name that holds the API key. -- `extra_params` (optional): String key/value map for provider-specific parameters. - -## Notes and Rules - -- Raw API keys are not supported. Do **not** add `api_key` fields. -- Unknown YAML fields fail to load (strict decoding). -- If `api_key_env` is set, the environment variable must be present when the run executes. +Use the canonical configuration reference: [../config.md](../config.md). diff --git a/docs/config/prompt-definitions.md b/docs/config/prompt-definitions.md index aa1681c..b20c7cd 100644 --- a/docs/config/prompt-definitions.md +++ b/docs/config/prompt-definitions.md @@ -1,73 +1,5 @@ -# Prompt Definition Files +# Moved -Prompt Definitions define **what** Scriptorium should do. +This page has moved. -A prompt file is YAML, typically stored under `prompts/`, for example `prompts/generic.structured_events.yaml`. - -## Complete Example - -```yaml -id: generic.structured_events -version: "1.0.0" -default_profile: local-quality -description: Extract events from a transcript into structured JSON. - -inputs: - - name: transcript - required: true - content_type: text/markdown - description: Source transcript - - name: glossary - required: false - content_type: text/yaml - description: Optional glossary context - -messages: - - role: system - content: | - You are a structured extraction assistant. - Return only JSON. - - role: user - content_file: ./generic.structured_events.user.md - -output: - format: json - validation_mode: json_schema - schema_path: structured_events.schema.json - repair_attempts: 1 -``` - -## Available Options - -- `id` (required): Prompt identifier used by `--prompt` / `prompt_id`. -- `version` (required): Prompt version string. -- `default_profile` (optional): Execution profile ID used when no explicit profile is provided. -- `description` (optional): Human-readable description. - -### `inputs[]` - -- `name` (required): Logical input name referenced in templates via `{{input "name"}}`. -- `required` (optional): If `true`, run fails when input is missing. -- `content_type` (optional): Metadata only (not enforced yet). -- `description` (optional): Human-readable input description. - -### `messages[]` - -- `role` (required): Message role such as `system` or `user`. -- `content` (optional): Inline Go-template message body. -- `content_file` (optional): Path to a template file. - -Each message must set **exactly one** of `content` or `content_file`. - -### `output` - -- `format` (required): One of `text`, `markdown`, `json`. -- `validation_mode` (required): One of `none`, `basic`, `json`, `json_schema`. -- `schema_path` (required when `validation_mode: json_schema`): Path to JSON Schema file. -- `repair_attempts` (required): Number of bounded repair retries (`>= 0`). - -## Notes and Rules - -- Unknown YAML fields fail to load (strict decoding). -- `content_file` paths are resolved relative to the prompt YAML file. -- For `json_schema` validation mode, Scriptorium also sends provider-level structured output requests automatically. +Use the canonical configuration reference: [../config.md](../config.md). diff --git a/docs/config/schema-definitions.md b/docs/config/schema-definitions.md index f314387..b20c7cd 100644 --- a/docs/config/schema-definitions.md +++ b/docs/config/schema-definitions.md @@ -1,82 +1,5 @@ -# JSON Schema Definition Files +# Moved -Schema definition files describe the expected JSON output contract for prompts that use: +This page has moved. -- `output.format: json` -- `output.validation_mode: json_schema` - -Schema files are JSON, typically stored under `schemas/`, for example `schemas/structured_events.schema.json`. - -## Complete Example - -```json -{ - "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://example.com/schemas/structured-events.schema.json", - "title": "Structured Events", - "description": "Expected shape for extracted event output", - "type": "object", - "properties": { - "summary": { - "type": "string", - "minLength": 1, - "description": "High-level session summary" - }, - "events": { - "type": "array", - "items": { - "type": "object", - "properties": { - "title": { "type": "string" }, - "type": { - "type": "string", - "enum": ["discovery", "combat", "social", "travel", "downtime", "other"] - }, - "notes": { "type": "string" } - }, - "required": ["title", "type"], - "additionalProperties": false - }, - "minItems": 0 - } - }, - "required": ["summary", "events"], - "additionalProperties": false, - "$defs": { - "nonEmptyString": { - "type": "string", - "minLength": 1 - } - } -} -``` - -## Available Options - -Scriptorium does not define custom schema keywords. It expects a valid JSON Schema document and passes it to the validator/provider. - -Commonly used JSON Schema options include: - -- `$schema`: Draft identifier URI. -- `$id`: Schema identifier URI. -- `title`: Human-readable schema title. -- `description`: Human-readable schema description. -- `type`: Expected JSON type (`object`, `array`, `string`, etc.). -- `properties`: Object field definitions. -- `required`: Required object fields. -- `additionalProperties`: Whether undeclared fields are allowed. -- `items`: Array item schema. -- `enum`: Allowed literal values. -- `const`: Single allowed literal value. -- `oneOf`, `anyOf`, `allOf`: Composition rules. -- `minimum`, `maximum`: Numeric bounds. -- `minLength`, `maxLength`, `pattern`: String constraints. -- `minItems`, `maxItems`: Array constraints. -- `$defs`: Reusable local definitions. -- `$ref`: Reference to another schema/definition. - -## Notes and Rules - -- Schema path comes from prompt `output.schema_path` and is resolved relative to `schema_dir`. -- If schema loading fails for `json_schema` mode, the run fails before the LLM request. -- Keep schemas strict (`additionalProperties: false`) when you want predictable output shape. +Use the canonical configuration reference: [../config.md](../config.md).