scriptorium/README.md

scriptorium

Scriptorium is a generic prompt execution engine.

It takes:

• a prompt definition
• a selected or default execution profile
• named input artifacts
• template variables
• optional runtime overrides

It returns:

• generated artifact
• validation result
• metadata

Prompt vs Profile

Scriptorium separates what to do (Prompt) from how to do it (Profile).

Prompt Definition

Defines the task logic and output contract.

• Task description and version.
• Message templates (system, user, etc.).
• Required and optional input artifacts.
• Output format and validation rules.
• Repair settings for structured output.
• Optional default_profile for convenience.

Execution Profile

Defines the runtime environment and model settings.

• LLM endpoint (URL).
• Model name.
• Generation parameters: temperature, max_tokens, top_p.
• Runtime settings: timeout, reasoning_effort.
• API key source via api_key_env.

Callers can explicitly provide a profile_id to override the prompt's default_profile.

Precedence

When resolving runtime settings, Scriptorium follows this precedence model (highest to lowest):

1. Runtime Overrides: Provided via CLI flags or HTTP request model object.
2. Execution Profile: Settings defined in the selected profile.
3. Application Defaults: Built-in fallback values.

Profile Selection Logic

The engine determines which profile to use in this order:

1. Explicit profile_id (via --profile or HTTP request).
2. The default_profile named in the Prompt Definition.
3. Error: If neither is provided and no default exists.

API Key Policy

To ensure security, Scriptorium does not support raw API keys in configuration files, CLI arguments, or HTTP requests.

• api_key_env: Profiles and overrides specify the name of an environment variable (e.g., SCRIPTORIUM_API_KEY).
• Runtime Resolution: The value of the environment variable is read directly from the process environment at runtime.
• Zero Leakage: API key values are never included in metadata, logs, or response bodies.

CLI Usage

scriptorium run

Runs a single prompt execution.

Required Flags:

• --prompt-dir: Directory containing prompt YAML files.
• --profile-dir: Directory containing profile YAML files.
• --prompt: The prompt ID to execute.
• --input: Input mapping name=path (repeatable).

Optional Flags:

• --profile: Override the prompt's default profile.
• --var: Template variable name=value (repeatable).
• --out: Write output to a file instead of stdout.
• --llm-base-url: Override endpoint.
• --model: Override model name.
• --api-key-env: Override API key environment variable name.
• --temperature: Override temperature.
• --max-tokens: Override max tokens.
• --top-p: Override top_p.
• --timeout: Override request timeout (e.g., 30s, 1m).

Examples:

Using the prompt's default_profile:

export SCRIPTORIUM_API_KEY="sk-..."
scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md

Overriding the profile:

scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --profile local-quality \
  --input transcript=./examples/fixtures/transcript.md

Overriding model and runtime values:

scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --model gpt-4o \
  --temperature 0.7 \
  --input transcript=./examples/fixtures/transcript.md

Using a local OpenAI-compatible vLLM endpoint:

scriptorium run \
  --prompt-dir ./prompts \
  --profile-dir ./profiles \
  --prompt generic.markdown_summary \
  --llm-base-url http://localhost:8000/v1 \
  --model meta-llama-3-8b \
  --input transcript=./examples/fixtures/transcript.md

scriptorium serve

Starts the HTTP API.

Required Flags:

• --prompt-dir: Directory containing prompt YAML files.
• --profile-dir: Directory containing profile YAML files.

Optional Flags:

• --addr: Listen address (default :8080).
• --schema-dir: Base directory for validation schemas.

HTTP API

POST /v1/runs

Executes a prompt. No built-in authentication is provided; deploy behind a trusted gateway.

Request Body:

{
  "prompt_id": "generic.structured_events",
  "profile_id": "local-quality",
  "include_raw_output": false,
  "inputs": {
    "transcript": {"type": "file", "uri": "./examples/fixtures/transcript.md"}
  },
  "vars": {
    "session_date": "2026-05-04"
  },
  "model": {
    "endpoint": "http://localhost:8000/v1",
    "model": "gpt-4o-mini",
    "temperature": 0.0
  }
}

profile_id is optional. If omitted, Scriptorium uses the prompt's default_profile. If neither is available, the run fails.

Response: Returns a 200 OK with the generated artifact, validation results, and metadata including the prompt_id and the selected_profile_id.

Validation Failures: If the model output fails validation (e.g., invalid JSON), the API returns 200 OK with validation.status = "failed".

Raw Output Exposure:

• raw_model_output is omitted by default.
• Set include_raw_output: true in the request to include it in the response.
• Raw output is preserved internally in run results regardless of HTTP exposure.

Prompt Definition Authoring

Prompts are defined in YAML.

Canonical Shape

id: generic.structured_events
version: "1.0.0"
description: "Extracts structured events from a transcript"
default_profile: local-quality

inputs:
  - name: transcript
    required: true
    content_type: text/markdown
    description: "The raw session transcript"
  - name: glossary
    required: false
    content_type: application/yaml
    description: "Optional glossary terms"

messages:
  - role: system
    content: "You are a helpful assistant."
  - role: user
    content_file: messages/extract_events.tmpl

output:
  format: json
  validation_mode: json_schema
  schema_path: structured_events.schema.json
  repair_attempts: 2

Key Features:

• Inline vs File: Use content for short prompts or content_file for larger templates. Exactly one must be set per message.
• Path Resolution: content_file paths are resolved relative to the prompt YAML file.
• Inputs: Mark inputs as required to ensure the runner fails early if they are missing.
• Input Metadata: content_type is currently descriptive metadata and not enforced yet.
• Validation: Support none, basic, json, and json_schema.
• Repair: repair_attempts enables bounded retries to fix structured output.

Execution Profile Authoring

Profiles are defined in YAML.

Canonical Shape

id: local-quality
endpoint: http://localhost:8000/v1
model: gpt-4o
temperature: 0.0
max_tokens: 4096
top_p: 1.0
timeout_seconds: 300
reasoning_effort: high
api_key_env: SCRIPTORIUM_API_KEY

Constraints:

• No Raw Keys: Do not include actual API keys. Only specify the environment variable name in api_key_env.
• Local Profiles: For local endpoints that don't require auth, api_key_env can be omitted.

Examples

• Prompt Definitions: prompts/
• Execution Profiles: profiles/
• Schemas: schemas/
• Fixtures: examples/fixtures/
• Local Experimentation: local-test/

Build and Test

go build -o scriptorium ./cmd/scriptorium
go test ./...