scriptorium/README.md

# scriptorium

Scriptorium is a generic prompt execution engine.

It takes:
- a prompt definition
- a selected or default execution profile
- named input artifacts
- template variables
- optional runtime overrides

It returns:
- for `run`: generated artifact, validation result, metadata
- for `render`: prepared/rendered prompt data (no model output)

## Prompt vs Profile

Scriptorium separates **what** to do (Prompt) from **how** to do it (Profile).

### Prompt Definition
Defines the task logic and output contract.
- Task description and version.
- Message templates (system, user, etc.).
- Required and optional input artifacts.
- Output format and validation rules.
- Repair settings for structured output.
- Optional `default_profile` for convenience.

### Execution Profile
Defines the runtime environment and model settings.
- LLM endpoint (URL).
- Model name.
- Generation parameters: `temperature`, `max_tokens`, `top_p`.
- Runtime settings: `timeout`, `reasoning_effort`.
- API key source via `api_key_env`.

Callers can explicitly provide a `profile_id` to override the prompt's `default_profile`.

## Precedence

When resolving runtime settings, Scriptorium follows this precedence model (highest to lowest):

1. **Runtime Overrides**: Provided via CLI flags or HTTP request `model` object.
2. **Execution Profile**: Settings defined in the selected profile.
3. **Application Defaults**: Built-in fallback values.

### Profile Selection Logic
The engine determines which profile to use in this order:
1. Explicit `profile_id` (via `--profile` or HTTP request).
2. The `default_profile` named in the Prompt Definition.
3. Error: If neither is provided and no default exists.

## API Key Policy

To ensure security, Scriptorium does not support raw API keys in configuration files, CLI arguments, or HTTP requests.

- **`api_key_env`**: Profiles and overrides specify the name of an environment variable (e.g., `SCRIPTORIUM_API_KEY`).
- **Runtime Resolution**: The value of the environment variable is read directly from the process environment at runtime.
- **Zero Leakage**: API key values are never included in metadata, logs, or response bodies.

## CLI Usage

Available CLI commands:
- `scriptorium run`
- `scriptorium render`
- `scriptorium serve`

### `scriptorium run`

Runs a single prompt execution.

**Required Flags:**
- `--prompt-dir`: Directory containing prompt YAML files.
- `--profile-dir`: Directory containing profile YAML files.
- `--prompt`: The prompt ID to execute.
- `--input`: Input mapping `name=path` (repeatable).

**Optional Flags:**
- `--profile`: Override the prompt's default profile.
- `--var`: Template variable `name=value` (repeatable).
- `--out`: Write output to a file instead of stdout.
- `--llm-base-url`: Override endpoint.
- `--model`: Override model name.
- `--api-key-env`: Override API key environment variable name.
- `--temperature`: Override temperature.
- `--max-tokens`: Override max tokens.
- `--top-p`: Override top_p.
- `--timeout`: Override request timeout (e.g., `30s`, `1m`).
- `--schema-dir`: Base directory for validation schemas.

**Examples:**

Using the prompt's `default_profile`:
```bash
scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md
```

Overriding the profile:
```bash
scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --profile local-quality \
  --input transcript=./examples/fixtures/transcript.md
```

Overriding model and runtime values:
```bash
scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --model gpt-4o \
  --temperature 0.7 \
  --input transcript=./examples/fixtures/transcript.md
```

Using a local OpenAI-compatible vLLM endpoint:
```bash
scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --llm-base-url http://localhost:8000/v1 \
  --model meta-llama-3-8b \
  --input transcript=./examples/fixtures/transcript.md
```

### `scriptorium render`

Prepares and renders a prompt without calling the LLM.

`render` uses the same prompt/profile/input/variable/runtime override resolution as `run`:
- Profile selection precedence: `--profile` -> prompt `default_profile` -> error.
- Runtime precedence: CLI runtime overrides -> selected profile -> built-in defaults.

`render` is useful for debugging:
- prompt template rendering
- input mappings
- selected profile behavior
- runtime override behavior

`render` does not:
- call the LLM
- validate model output
- perform repair
- expose resolved API key values

It may include `api_key_env` names where relevant.

**Required Flags:**
- `--prompt-dir`: Directory containing prompt YAML files.
- `--profile-dir`: Directory containing profile YAML files.
- `--prompt`: The prompt ID to render.
- `--input`: Input mapping `name=path` (repeatable).

**Optional Flags:**
- `--profile`: Override the prompt's default profile.
- `--var`: Template variable `name=value` (repeatable).
- `--out`: Write output to a file instead of stdout.
- `--format`: Render output format (`text` or `json`). Default: `text`.
- `--llm-base-url`: Runtime override for endpoint.
- `--model`: Runtime override for model name.
- `--api-key-env`: Runtime override for API key environment variable name.
- `--temperature`: Runtime override for temperature.
- `--max-tokens`: Runtime override for max tokens.
- `--top-p`: Runtime override for top_p.
- `--timeout`: Runtime override for timeout (e.g., `30s`, `1m`).

**Render Output Formats:**
- `text`: Human-readable output (default).
- `json`: Machine-readable structured output.

Render formatting is modular; additional output formats can be added later without changing prepare/run core logic.

**Examples:**

Default text output:
```bash
scriptorium render \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md
```

Explicit JSON output:
```bash
scriptorium render \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md \
  --format json
```

Using prompt `default_profile` (omit `--profile`):
```bash
scriptorium render \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md
```

Overriding profile selection:
```bash
scriptorium render \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --profile local-quality \
  --input transcript=./examples/fixtures/transcript.md
```

Overriding runtime settings:
```bash
scriptorium render \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md \
  --llm-base-url http://localhost:8000/v1 \
  --model gpt-4o-mini \
  --temperature 0.2 \
  --max-tokens 800 \
  --top-p 1.0 \
  --timeout 45s
```

Writing rendered output to a file:
```bash
scriptorium render \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md \
  --format text \
  --out ./rendered_prompt.txt
```

### `scriptorium serve`

Starts the HTTP API.

**Required Flags:**
- `--prompt-dir`: Directory containing prompt YAML files.
- `--profile-dir`: Directory containing profile YAML files.

**Optional Flags:**
- `--addr`: Listen address (default `:8080`).
- `--schema-dir`: Base directory for validation schemas.

## HTTP API

### `POST /v1/runs`

Executes a prompt. No built-in authentication is provided; deploy behind a trusted gateway.

**Request Body:**
```json
{
  "prompt_id": "generic.structured_events",
  "profile_id": "local-quality",
  "include_raw_output": false,
  "inputs": {
    "transcript": {"type": "file", "uri": "./examples/fixtures/transcript.md"}
  },
  "vars": {
    "session_date": "2026-05-04"
  },
  "model": {
    "endpoint": "http://localhost:8000/v1",
    "model": "gpt-4o-mini",
    "temperature": 0.0
  }
}
```

`profile_id` is optional. If omitted, Scriptorium uses the prompt's `default_profile`. If neither is available, the run fails.

**Response:**
Returns a `200 OK` with the generated artifact, validation results, and metadata including the `prompt_id` and the `selected_profile_id`.

**Validation Failures:**
If the model output fails validation (e.g., invalid JSON), the API returns `200 OK` with `validation.status = "failed"`.

**Raw Output Exposure:**
- `raw_model_output` is omitted by default.
- Set `include_raw_output: true` in the request to include it in the response.
- Raw output is preserved internally in run results regardless of HTTP exposure.

## Prompt Definition Authoring

Prompts are defined in YAML.

### Canonical Shape
```yaml
id: generic.structured_events
version: "1.0.0"
description: "Extracts structured events from a transcript"
default_profile: local-quality

inputs:
  - name: transcript
    required: true
    content_type: text/markdown
    description: "The raw session transcript"
  - name: glossary
    required: false
    content_type: application/yaml
    description: "Optional glossary terms"

messages:
  - role: system
    content: "You are a helpful assistant."
  - role: user
    content_file: messages/extract_events.tmpl

output:
  format: json
  validation_mode: json_schema
  schema_path: structured_events.schema.json
  repair_attempts: 2
```

**Key Features:**
- **Inline vs File**: Use `content` for short prompts or `content_file` for larger templates. Exactly one must be set per message.
- **Path Resolution**: `content_file` paths are resolved relative to the prompt YAML file.
- **Inputs**: Mark inputs as `required` to ensure the runner fails early if they are missing.
- **Input Metadata**: `content_type` is currently descriptive metadata and not enforced yet.
- **Validation**: Support `none`, `basic`, `json`, and `json_schema`.
- **Repair**: `repair_attempts` enables bounded retries to fix structured output.

## Execution Profile Authoring

Profiles are defined in YAML.

### Canonical Shape
```yaml
id: local-quality
endpoint: http://localhost:8000/v1
model: gpt-4o
temperature: 0.0
max_tokens: 4096
top_p: 1.0
timeout_seconds: 300
reasoning_effort: high
api_key_env: SCRIPTORIUM_API_KEY
```

**Constraints:**
- **No Raw Keys**: Do not include actual API keys. Only specify the environment variable name in `api_key_env`.
- **Local Profiles**: For local endpoints that don't require auth, `api_key_env` can be omitted.

## Examples

- **Prompt Definitions**: `prompts/`
- **Execution Profiles**: `profiles/`
- **Schemas**: `schemas/`
- **Fixtures**: `examples/fixtures/`

## Build and Test

```bash
go build -o scriptorium ./cmd/scriptorium
go test ./...
```