eric/scriptorium
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Updated documentation to reflect current build state

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-05 08:45:10 -05:00
parent ca4d939fdc
commit f67e2d48e1
2 changed files with 435 additions and 751 deletions
Download Patch File Download Diff File Expand all files Collapse all files

334
README.md
View File

@@ -2,24 +2,87 @@
Scriptorium is a generic prompt-profile execution engine written in Go.
It loads a prompt profile, resolves named input artifacts, renders a prompt, calls an OpenAI-compatible LLM endpoint, validates output, and returns an artifact plus metadata.
Given named input artifacts and a prompt profile, Scriptorium:
## Relationship to Narratio
1. Loads the profile.
2. Resolves input artifact references.
3. Renders prompt messages from templates.
4. Calls an OpenAI-compatible LLM endpoint.
5. Validates output if configured.
6. Optionally performs bounded structured-output repair.
7. Returns a generated artifact plus run metadata.
In the broader workflow, Narratio handles pipeline orchestration (transcription, cleanup, storage, notifications). Scriptorium handles only prompt-profile execution for a single run.
## Where Scriptorium Fits
## Repository Example Assets
Scriptorium is not an orchestrator.
- Profiles: `profiles/`
- Schemas: `schemas/`
- Tiny fixtures: `examples/fixtures/`
In the D&D workflow:
Included profiles:
- `generic.markdown_summary`
- `dnd.session_recap` (example content only; no D&D-specific Go logic)
- `generic.structured_events` (JSON + JSON Schema validation)
- Narratio orchestrates the full pipeline.
- WhisperX transcribes audio.
- Seriatim merges transcripts.
- Audita polishes transcripts.
- Scriptorium generates final artifacts from prepared inputs.
## Run a Local Profile (CLI)
D&D-specific behavior belongs in profiles, schemas, fixtures, and caller inputs, not in core Go logic.
## Core Concepts
- Prompt profile: YAML config that defines templates, model defaults, output format, and validation behavior.
- Named inputs: logical input names (for example `transcript`, `glossary`) mapped to artifact references.
- Artifact refs: currently `file` and `inline` are supported by readers used in v1 flows.
- Template variables: key/value vars provided at run time and accessed in templates as `{{.var_name}}`.
- Model target: endpoint/model and generation parameters (`temperature`, `max_tokens`, `top_p`, `timeout_seconds`).
- Output format: `text`, `markdown`, or `json`.
- Validation mode: `none`, `basic`, `json`, `json_schema`.
- Repair attempts: bounded retries for structured modes (`json`, `json_schema`) when output validation fails.
- Run metadata: IDs/hashes/model/timing/usage/validation details for auditability.
## Build and Test
Build:
```bash
go build -o scriptorium ./cmd/scriptorium
```
Run tests:
```bash
go test ./...
```
Run CLI locally:
```bash
go run ./cmd/scriptorium run --help
```
## CLI Usage
### `scriptorium run`
Required flags:
- `--profile-dir`
- `--profile-id`
- `--input` (repeatable `name=path`)
Optional flags:
- `--var` (repeatable `name=value`)
- `--out`
- `--llm-base-url`
- `--llm-api-key`
- `--model`
- `--temperature`
- `--max-tokens`
- `--schema-dir`
- `--timeout`
If `--llm-base-url` and/or `--model` are omitted, profile `model_defaults` must provide them.
Markdown summary example:
```bash
go run ./cmd/scriptorium run \
@@ -30,81 +93,222 @@ go run ./cmd/scriptorium run \
--out ./out.md
```
This relies on `model_defaults.endpoint` and `model_defaults.model` in the selected profile.
You can override either at runtime with `--llm-base-url` and/or `--model`.
For schema-validated JSON output:
Same run with explicit local OpenAI-compatible endpoint (for example vLLM):
```bash
go run ./cmd/scriptorium run \
--profile-dir ./profiles \
--profile-id generic.structured_events \
--profile-id generic.markdown_summary \
--input transcript=./examples/fixtures/transcript.md \
--input glossary=./examples/fixtures/glossary.yml \
--llm-base-url http://localhost:8000/v1 \
--model gpt-4o-mini \
--schema-dir ./schemas \
--out ./events.json
--out ./out.md
```
## Start Local HTTP API
Passing template variables:
```bash
go run ./cmd/scriptorium serve \
--addr :8080 \
go run ./cmd/scriptorium run \
--profile-dir ./profiles \
--schema-dir ./schemas \
--llm-base-url http://localhost:8000/v1 \
--model gpt-4o-mini
--profile-id generic.markdown_summary \
--input transcript=./examples/fixtures/transcript.md \
--input glossary=./examples/fixtures/glossary.yml \
--var session_date=2026-05-04 \
--var facilitator="Eris" \
--out ./out.md
```
## Call `POST /v1/runs`
Output behavior:
```bash
curl -sS http://localhost:8080/v1/runs \
-H 'Content-Type: application/json' \
-d '{
"profile_id": "generic.structured_events",
"inputs": {
"transcript": {"type": "file", "uri": "./examples/fixtures/transcript.md"},
"glossary": {"type": "file", "uri": "./examples/fixtures/glossary.yml"}
},
"model": {"model": "gpt-4o-mini"}
}'
- Artifact content goes to stdout unless `--out` is set.
- Summaries and errors are written to stderr.
- Exit code `2` indicates run succeeded but validation status is `failed`.
### `scriptorium serve`
Starts HTTP API.
Required flags:
- `--profile-dir`
- `--llm-base-url`
Common optional flags:
- `--addr` (default `:8080`)
- `--schema-dir` (default `.`)
- `--llm-api-key`
- `--model`
- `--timeout` (default `10m`)
## HTTP API
Run endpoint:
- `POST /v1/runs`
Request example:
```json
{
"profile_id": "generic.structured_events",
"profile_version": "1.0.0",
"inputs": {
"transcript": {"type": "file", "uri": "./examples/fixtures/transcript.md"},
"glossary": {"type": "file", "uri": "./examples/fixtures/glossary.yml"}
},
"vars": {
"session_date": "2026-05-04"
},
"model": {
"endpoint": "http://localhost:8000/v1",
"model": "gpt-4o-mini",
"temperature": 0.0,
"max_tokens": 600,
"top_p": 1.0,
"timeout_seconds": 120
}
}
```
Response shape:
- `artifact`
- `validation`
- `metadata`
- `raw_model_output`
`metadata` includes stable audit fields such as run/profile IDs, profile hash, effective model params, prompt/input hashes, timing, usage, and validation summary.
```json
{
"artifact": {
"name": "output",
"content_type": "application/json",
"body": "{...}",
"uri": "",
"size": 123,
"hash": "..."
},
"validation": {
"status": "passed",
"mode": "json_schema",
"errors": [],
"schema_path": "structured_events.schema.json",
"repair_attempts": 0,
"is_valid": true
},
"metadata": {
"run_id": "xxxxxxxx-xxxx-4xxx-8xxx-xxxxxxxxxxxx",
"profile_id": "generic.structured_events",
"profile_version": "1.0.0",
"profile_hash": "...",
"model_name": "gpt-4o-mini",
"endpoint": "http://localhost:8000/v1",
"model_params": {
"endpoint": "http://localhost:8000/v1",
"model": "gpt-4o-mini",
"temperature": 0,
"max_tokens": 600,
"top_p": 1,
"timeout_seconds": 120
},
"input_hashes": {"transcript": "...", "glossary": "..."},
"prompt_hash": "...",
"usage": {"prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30},
"start_time": "...",
"end_time": "...",
"duration_ms": 1523,
"validation_mode": "json_schema",
"validation_status": "passed",
"repair_attempts_used": 0
},
"raw_model_output": "{...}"
}
```
## Add a New Prompt Profile
Validation content failures are returned as successful run responses (`200`) with `validation.status = "failed"`; raw model output is preserved in `raw_model_output`.
1. Add a YAML file under `profiles/` with:
- `id`, `version`, `expected_inputs`, `templates`, `model_defaults`, `output_format`, `validation`
- optional model timeout via `model_defaults.timeout_seconds` (per-run LLM timeout override)
2. Ensure endpoint/model are available from either:
- profile defaults (`model_defaults.endpoint`, `model_defaults.model`), or
- request overrides (`--llm-base-url`, `--model`, or HTTP `model.endpoint`/`model.model`).
3. Use template helpers such as `{{input "transcript"}}` and template vars like `{{.session_date}}`.
4. For structured JSON output, set:
- `output_format: json`
- `validation.validation_mode: json_schema`
- `validation.schema_path: <schema file>`
5. Place schema files in `schemas/` and pass `--schema-dir ./schemas` for CLI/serve.
6. `validation.repair_attempts` is bounded and applies only to structured modes (`json`, `json_schema`).
Error response shape:
## Validation Behavior
```json
{
"error": {
"code": "artifact_read_failed",
"message": "failed to read input artifact"
}
}
```
Validation modes currently implemented:
- `none`
- `basic` (non-empty output)
- `json` (must parse as JSON)
- `json_schema` (must parse JSON and satisfy schema)
## Prompt Profile Authoring
Important behavior:
- Validation content failures are returned as structured run results (`validation.status = failed`) and preserve `raw_model_output`.
- Validation runtime/configuration failures are treated as run errors.
### Minimal Markdown profile
```yaml
id: generic.markdown_summary
version: "1.0.0"
expected_inputs:
- transcript
templates:
- role: system
content: "You are a concise assistant."
- role: user
content: |
Summarize:
{{input "transcript"}}
model_defaults:
endpoint: http://localhost:8000/v1
model: gpt-4o-mini
temperature: 0.2
max_tokens: 700
output_format: markdown
validation:
validation_mode: basic
```
### Structured JSON profile with schema validation
```yaml
id: generic.structured_events
version: "1.0.0"
expected_inputs:
- transcript
templates:
- role: system
content: "Return only JSON."
- role: user
content: |
Extract events from:
{{input "transcript"}}
model_defaults:
endpoint: http://localhost:8000/v1
model: gpt-4o-mini
output_format: json
validation:
format: json
validation_mode: json_schema
schema_path: structured_events.schema.json
repair_attempts: 1
```
`repair_attempts` is bounded. Repair is attempted only for structured validation modes.
## Validation Modes
Supported modes:
- `none`: skipped validation result.
- `basic`: fails if output is empty/whitespace.
- `json`: output must parse as JSON.
- `json_schema`: output must parse as JSON and satisfy the configured schema.
Validation failures caused by output content are represented in `validation` and do not discard raw model output.
## Repository Examples
- Profiles: `profiles/`
- Schemas: `schemas/`
- Fixtures: `examples/fixtures/`
- Local experimentation: `local-test/`
## Development Notes
- Core is generic and follows a ports-and-adapters style.
- Domain/usecase packages do not depend on HTTP/CLI/wire types.
- To add a new LLM adapter: implement `internal/llm.Client`.
- To add a new artifact reader: implement/extend `internal/artifact.Reader` routing.
- To add a new validation mode: extend `internal/validate` and keep run semantics stable.