eric/scriptorium
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Expand consumer integration documentation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-05 03:09:32 +00:00
parent 879cb021b2
commit 07ac7e54c5
3 changed files with 327 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files

125
docs/consumers/api.md
View File

@@ -1,13 +1,122 @@
# Consumer API Overview
# Consumer Integration Overview
Scriptorium can be used by consumers through three implemented surfaces:
This guide is for applications that call Scriptorium from another codebase.
- CLI commands, documented in [CLI reference](../cli.md).
- HTTP `POST /v1/runs`, documented in [HTTP API reference](../api.md).
- Go package `gitea.maximumdirect.net/eric/scriptorium`, documented in [pkg-scriptorium](pkg-scriptorium.md).
Scriptorium exposes three integration surfaces:
The Go package is the typed in-process API. It prepares prompts, runs prompts, accepts file or inline artifacts, supports per-request execution overrides, and exposes stable public errors for `errors.Is`.
| Surface | Use when |
| --- | --- |
| Go package | The consumer is Go, needs typed requests/results, or wants injected LLM clients for tests. |
| CLI subprocess | The consumer wants process isolation or is not written in Go. |
| HTTP API | The consumer needs a service boundary or remote access to `POST /v1/runs`. |
Use the Go package when the caller is a Go program that wants typed requests/results, context cancellation, repeated calls without subprocess overhead, or fake LLM injection for tests. Use the CLI or HTTP surfaces when process isolation, language neutrality, or an HTTP boundary is preferred.
Canonical references:
Raw API key values are not accepted in public payloads and are not returned in prepared or run results. Execution profiles may reference an environment variable name through `api_key_env`.
- Go package: [Package scriptorium](pkg-scriptorium.md)
- CLI subprocess: [Subprocess integration](../integrations/subprocess.md)
- HTTP: [HTTP API reference](../api.md)
- File formats: [Configuration reference](../config.md)
## Required Deployment Inputs
Every integration needs operators to provide:
- prompt definitions;
- profile definitions or built-in profile IDs;
- schema files when prompts use `json_schema`;
- input artifacts or inline input bodies;
- API-key environment variables or direct per-request keys where supported.
Raw API keys do not belong in config, prompt files, profile YAML, CLI
arguments, or HTTP request bodies.
## Recommended Workflow
Use the Go package when:
- the consumer is a Go application;
- the application needs `context.Context` cancellation;
- repeated calls should avoid subprocess startup;
- tests need a fake LLM client;
- direct per-request `RunRequest.APIKey` is required.
Use the CLI subprocess when:
- the consumer is not Go;
- process isolation is useful;
- stdout/stderr separation and exit codes are enough;
- the consumer already manages local files and environment variables.
Use HTTP when:
- Scriptorium should run as a service;
- multiple clients need a shared prompt/profile deployment;
- clients can reach a trusted, protected HTTP boundary.
## Minimal Go Example
```go
engine, err := scriptorium.NewEngine(scriptorium.Config{
PromptDir: "./examples/prompts",
ProfileDir: "./examples/profiles",
SchemaDir: "./examples/schemas",
})
if err != nil {
return err
}
prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{
PromptID: "generic.markdown_summary",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.File("./examples/fixtures/transcript.md"),
"glossary": scriptorium.File("./examples/fixtures/glossary.yml"),
},
})
if err != nil {
return err
}
_ = prepared.Messages
```
Run the maintained package example:
```bash
go run ./examples/go-library/prepare
```
## Subprocess Workflow
Invoke `scriptorium render` for preflight and `scriptorium run` for generation.
Capture stdout and stderr separately. Treat exit code `2` from `run` as a
completed generation with failed validation.
See [Subprocess integration](../integrations/subprocess.md) for the stable
invocation contract.
## HTTP Workflow
Run `scriptorium serve` behind trusted controls and send JSON requests to
`POST /v1/runs`.
Do not duplicate endpoint schemas in consumers. Use the [HTTP API
reference](../api.md) as the authoritative contract.
## Consumer Responsibilities
Consumers are responsible for:
- selecting prompt/profile IDs as deployment configuration;
- supplying all required inputs and vars;
- protecting generated artifacts and rendered prompts as sensitive data;
- deciding whether to keep output when validation fails;
- implementing retries only when another model call is acceptable.
Scriptorium does not persist run state. Retrying a failed or timed-out request
can produce different output and can incur another provider request.
## Status Behavior
- Go package methods return typed results or errors that support `errors.Is`.
- CLI `run` exits `2` when generation succeeds but validation fails.
- HTTP returns `200 OK` for generated-content validation failures and exposes the failed status in the response body.
- Runtime validation failures are errors.

181
docs/consumers/pkg-scriptorium.md
View File

@@ -6,7 +6,22 @@ Import path:
import "gitea.maximumdirect.net/eric/scriptorium"
```
The root package is a public facade over Scriptorium's prompt execution use case. It keeps `internal/*` packages private while exposing typed construction, preparation, execution, inputs, results, and errors.
The root package is the public Go facade for Scriptorium's prompt prepare/run
workflow. It exposes typed requests, results, source options, injected LLM
clients, and stable public errors while keeping `internal/*` packages private.
## Intended Use Cases
Use the package when a Go application needs:
- in-process prompt preparation or execution;
- typed request/result structs;
- direct `context.Context` cancellation;
- injected/fake LLM clients for tests;
- direct per-request `RunRequest.APIKey`.
Use [Subprocess integration](../integrations/subprocess.md) or the [HTTP API](../api.md)
when a process or service boundary is preferred.
## Construct An Engine
@@ -21,21 +36,56 @@ if err != nil {
}
```
`PromptDir` is required unless an explicit prompt source option is supplied. `ProfileDir` is optional; omit it to use built-in profiles only, or set it to overlay custom profiles above built-ins. `SchemaDir` defaults to the built-in schema directory. `Timeout` and `HTTPClient` configure the default OpenAI-compatible client used by `Run` when no custom LLM client is supplied.
`Config` fields:
## Asset Sources
| Field | Description |
| --- | --- |
| `PromptDir` | Prompt definition directory. Required unless `WithPromptFS` or `WithPromptFile` is used. |
| `ProfileDir` | Optional custom profile directory overlaid above built-in profiles. |
| `SchemaDir` | Schema directory. Defaults to `.` when empty. |
| `Timeout` | Default timeout for the built-in OpenAI-compatible client. |
| `HTTPClient` | Optional HTTP client for the built-in OpenAI-compatible client. |
Directory fields on `Config` remain the compatibility path. Explicit source options override the matching directory field:
`NewEngine` accepts `nil` options and ignores them. Invalid construction wraps
`ErrInvalidConfig`.
- `WithPromptFS(fsys, root)` and `WithPromptFile(path)`
- `WithProfileFS(fsys, root)` and `WithProfileFile(path)`
- `WithSchemaFS(fsys, root)` and `WithSchemaFile(path)`
## Source Options
Prompt and profile sources load standard Scriptorium YAML with the same strict validation as directory sources. For `WithPromptFS`, the configured root is a containment boundary: prompt `content_file` paths resolve relative to the prompt file and must remain inside that root. Profile options overlay custom profiles above built-ins. For `WithSchemaFS`, prompt `schema_path` values resolve inside the configured root. Absolute paths and relative traversal outside those `fs.FS` roots are rejected. Schema file options expose the file by its base name.
Directory fields are the compatibility path. Explicit source options override
the matching directory field.
Prompt sources:
- `WithPromptFS(fsys, root)`
- `WithPromptFile(path)`
Profile sources:
- `WithProfileFS(fsys, root)`
- `WithProfileFile(path)`
- `WithProfiles(profiles...)`
Schema sources:
- `WithSchemaFS(fsys, root)`
- `WithSchemaFile(path)`
LLM source:
- `WithLLMClient(client)`
Source behavior:
- Prompt and profile YAML use the same strict rules as directory loading.
- Prompt `content_file` values resolve relative to the prompt file.
- `fs.FS` roots are containment boundaries for prompt content files and schema paths.
- File options expose the selected file by its base name.
- Profile source precedence is in-memory profiles, then explicit profile file/FS/directory source, then built-ins.
- `WithLLMClient(nil)` returns `ErrInvalidConfig`.
## In-Memory Profiles
Use `WithProfiles` when the consuming application already has profile settings in typed Go configuration:
Use `WithProfiles` when the application already has typed model settings:
```go
profile := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
@@ -48,13 +98,33 @@ profile := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfi
engine, err := scriptorium.NewEngine(cfg, scriptorium.WithProfiles(profile))
```
In-memory profiles have highest precedence, followed by configured profile file/FS/directory sources, then built-in profiles. Duplicate IDs in one `WithProfiles` call return `ErrInvalidConfig`.
`Profile` and `OpenAICompatibleProfileConfig` include:
`Profile` and `OpenAICompatibleProfileConfig` include endpoint, model, numeric defaults, service tier, reasoning effort, `APIKeyRequired`, and JSON-compatible `ExtraParams`. `WithProfiles` validates `ExtraParams` and returns `ErrInvalidConfig` for unsupported values such as functions, channels, non-string map keys, non-finite floats, or cyclic values. Raw API-key fields are not accepted. When `APIKeyRequired` is true, pass the secret with `RunRequest.APIKey`.
- `ID`
- `Endpoint`
- `Model`
- `Temperature`
- `MaxTokens`
- `TopP`
- `TimeoutSeconds`
- `ServiceTier`
- `ReasoningEffort`
- `APIKeyRequired`
- `ExtraParams`
## Prepare A Prompt
`WithProfiles` rejects duplicate IDs in one call. In-memory profiles do not
store raw keys. When `APIKeyRequired` is true, pass the secret on each request
with `RunRequest.APIKey`.
`Prepare` resolves the prompt definition, profile, inputs, variables, output contract, structured-output metadata, and rendered messages without calling an LLM.
`ExtraParams` must be JSON-compatible: strings, booleans, finite numbers,
objects with string keys, arrays/slices, and nil. Unsupported values, non-string
map keys, non-finite floats, and cycles return `ErrInvalidConfig` for profiles
or `ErrInvalidRequest` for request overrides.
## Prepare Workflow
`Prepare` resolves prompt/profile/input/schema state and renders messages
without calling an LLM.
```go
prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{
@@ -67,18 +137,19 @@ prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{
if err != nil {
return err
}
_ = prepared.Messages
_ = prepared.EffectiveModelParams
```
Input helpers:
`PreparedRun` includes prompt ID/version/hash, selected profile, effective
model params, output contract, structured-output metadata, input hashes,
rendered prompt hash, rendered messages, and timing fields. It does not include
raw API-key values, model output, validation results, or internal target
presence metadata.
- `scriptorium.File(path)` loads an input artifact from a file.
- `scriptorium.Inline(body)` passes inline input content.
- `scriptorium.InlineWithURI(uri, body)` passes inline content with URI metadata.
## Run Workflow
## Run A Prompt
`Run` prepares the prompt, calls the configured LLM client, builds the output artifact, and validates the output.
`Run` calls `Prepare`, invokes the configured LLM client, builds the output
artifact, and validates the output.
```go
result, err := engine.Run(ctx, scriptorium.RunRequest{
@@ -95,13 +166,25 @@ if err != nil {
_ = result.Artifact
```
`RunResult` includes the run ID, output artifact, raw output, validation result, prompt/profile/model metadata, effective model parameters, input hashes, token/cache usage, and timing fields. Validation content failures return a successful `RunResult` with failed validation status. Runtime validation errors return `ErrValidation`.
`RunResult` includes run ID, output artifact, raw output, validation result,
prompt/profile/model metadata, effective model params, input hashes, usage, and
timing fields.
For the public Go API, pass provider credentials with `RunRequest.APIKey`. The value is request-scoped, uses `json:"-"`, is preferred over profile `api_key_env` by the default OpenAI-compatible client, and is not included in `PreparedRun` or `RunResult` JSON. Normal Go string formatting of `RunRequest` reports only whether a direct key is set. Do not store raw keys in config, prompt files, or profile YAML.
Generated-content validation failures return a successful `RunResult` with
`Validation.Status == ValidationFailed`. Runtime/schema validation errors
return an error that matches `ErrValidation`.
Avoid logging raw request structs with reflection-based debug dumpers; exported fields remain visible to tools that bypass `String` and `GoString` methods.
## Inputs
## Inject An LLM Client
Input helpers:
- `File(path)`: file-backed artifact reference.
- `Inline(body)`: inline artifact body.
- `InlineWithURI(uri, body)`: inline artifact body with URI metadata.
Input map keys must match the prompt's expected input names.
## Injected LLM Clients
Use `WithLLMClient` for tests or custom model integrations:
@@ -118,11 +201,34 @@ func (fakeLLM) Generate(ctx context.Context, req scriptorium.GenerateRequest) (*
engine, err := scriptorium.NewEngine(cfg, scriptorium.WithLLMClient(fakeLLM{}))
```
The injected client receives the rendered prompt, effective execution target, target presence metadata for explicit numeric overrides, structured-output spec, and request API key when provided. `GenerateRequest.APIKey` also uses `json:"-"`, and normal Go string formatting reports only whether a direct key is set. Custom and fake clients should avoid logging or serializing it. `WithLLMClient(nil)` returns `ErrInvalidConfig`.
Injected clients receive:
## Request Overrides
- rendered prompt;
- effective execution target;
- numeric target presence metadata;
- structured-output spec when applicable;
- direct request API key when provided.
`RunRequest.Execution` accepts per-request overrides. Numeric override fields are pointers so explicit zero values are preserved:
Custom clients should not log raw prompts or API keys by default.
## Overrides And API Keys
`RunRequest` fields:
| Field | Description |
| --- | --- |
| `PromptID` | Prompt ID. |
| `PromptVersion` | Optional prompt version filter. |
| `ProfileID` | Optional profile override. |
| `APIKey` | Direct per-request API key. |
| `Inputs` | Input artifact references. |
| `Vars` | Template variables. |
| `Execution` | Per-request model overrides. |
| `Validation` | Per-request output contract override. |
| `Metadata` | Request metadata reserved for callers. |
`RunRequest.Execution` uses pointer fields for numeric values so explicit zero
overrides are preserved:
```go
zero := 0
@@ -131,11 +237,19 @@ req.Execution = &scriptorium.ExecutionTargetOverride{
}
```
`ExecutionTargetOverride.ExtraParams` accepts JSON-compatible values and copies typed maps/slices so later caller mutation does not affect the run. Unsupported values, non-string map keys, non-finite floats, and cycles return `ErrInvalidRequest`.
Direct `RunRequest.APIKey` takes precedence over profile `api_key_env` for the
default OpenAI-compatible client. It is request-scoped, uses `json:"-"`, and is
not included in `PreparedRun` or `RunResult` JSON. Normal Go string formatting
of `RunRequest` and `GenerateRequest` reports only whether a direct key is set.
Raw API keys do not belong in profile YAML, in-memory profiles, or app config.
Avoid reflection-based debug dumps of request structs because exported fields
remain visible to tools that bypass `String` and `GoString`.
## Errors
Public methods wrap context while preserving stable sentinel checks with `errors.Is`:
Public methods wrap context while preserving stable sentinel checks with
`errors.Is`:
- `ErrInvalidConfig`
- `ErrInvalidRequest`
@@ -158,8 +272,13 @@ if errors.Is(err, scriptorium.ErrPromptNotFound) {
## Examples
Run the prepare-only example from the repository root:
Run the maintained prepare-only example from the repository root:
```bash
go run ./examples/go-library/prepare
```
See also:
- [Configuration reference](../config.md)
- [Consumer integration overview](api.md)

104
docs/integrations/subprocess.md
View File

@@ -1,59 +1,67 @@
# Subprocess Integration
## Purpose
This document defines the supported subprocess contract for downstream
applications invoking Scriptorium through the public CLI.
This document defines the supported subprocess contract for downstream applications invoking Scriptorium through the public CLI.
This is a CLI contract, not an internal Go package integration.
This is a CLI contract. Go callers that want an in-process typed API should use
the [package guide](../consumers/pkg-scriptorium.md).
## Supported Commands
Downstream applications should invoke:
- `scriptorium run`
- `scriptorium render`
- `scriptorium render` for preflight/debug output without LLM execution.
- `scriptorium run` for generation.
Use `run` for generation.
Use `render` for preflight/debug output without LLM execution.
`scriptorium serve` is an HTTP service command, not the recommended subprocess
contract for per-request execution.
## Recommended Invocation Shapes
Run:
```bash
scriptorium run \
--prompt <prompt_id> \
--input transcript=<path> \
--out <artifact_path>
```
Render:
```bash
scriptorium render \
--config <config_path> \
--prompt <prompt_id> \
--input transcript=<path> \
--format json
```
Run:
```bash
scriptorium run \
--config <config_path> \
--prompt <prompt_id> \
--input transcript=<path> \
--out <artifact_path>
```
Callers may add:
- `--config <path>`
- `--profile <profile_id>`
- repeatable `--input name=path`
- repeatable `--var name=value`
- runtime overrides when explicitly needed (`--model`, `--llm-base-url`, `--timeout`, etc.)
- runtime overrides when explicitly needed, such as `--model`, `--llm-base-url`, `--api-key-env`, and `--timeout`
Do not pass raw API keys as command arguments.
## Config And Directory Behavior
Callers can rely on resolved app config or pass explicit paths.
- default config search order:
1. `/usr/local/etc/scriptorium/config.yml`
2. `/etc/scriptorium/config.yml`
- explicit `--config` requires file existence and valid syntax
- CLI flags override config values
Default config search order:
1. `/usr/local/etc/scriptorium/config.yml`
2. `/etc/scriptorium/config.yml`
Rules:
- Explicit `--config` requires file existence and valid syntax.
- CLI flags override config values.
- `run` and `render` require an effective `prompt_dir`.
- `profile_dir` is optional because built-in profiles are available.
## Profile Selection
@@ -63,52 +71,60 @@ Profile selection follows runner behavior:
2. prompt `default_profile`
3. error if neither is available
Callers should treat prompt/profile IDs as deployment configuration, not hardcoded logic.
Treat prompt and profile IDs as deployment configuration, not hardcoded business
logic.
## Input And Variable Contract
- Inputs use repeated `--input name=path`.
- Input names must match prompt definition input names.
- Variables use repeated `--var name=value` for small metadata values.
- Variables use repeated `--var name=value`.
- Both flags also accept comma-separated mappings.
- Prefer file inputs for large content.
CLI inputs are file references. HTTP-only `inline` references are documented in
the [HTTP API reference](../api.md).
## Environment Contract
- Pass through required API-key environment variables referenced by `api_key_env`.
- Never pass raw API keys via CLI arguments.
- Keep subprocess environment scoped to required variables.
- Keep subprocess environments scoped to required variables.
- Use `--api-key-env` only to name an environment variable.
- Never pass raw API keys via argv.
## Output And Error Handling
## Stdout And Stderr
`run`:
- stdout: artifact body unless `--out` is used
- `--out`: writes artifact to file
- stderr: success summary and errors
- stdout: generated artifact body unless `--out` is used.
- stderr: success summary and errors.
`render`:
- stdout: prepared-run output unless `--out` is used
- stderr: errors
- stdout: prepared-run output unless `--out` is used.
- stderr: errors.
Callers should capture stdout and stderr separately.
Capture stdout and stderr separately. Do not parse stderr as a stable data
format beyond exit status handling.
## Exit Status Contract
- `0`: success
- `1`: parse/config/load/render/generation/IO/runtime error
- `2`: run completed but validation failed
- `0`: success.
- `1`: parse, config, load, render, generation, IO, or runtime error.
- `2`: `run` completed and output was written, but validation failed.
A `run` exit code `2` can still produce output (stdout or `--out`).
A `run` exit code `2` can still produce output on stdout or at `--out`.
Consumers must decide whether to keep or discard that output.
## Security Notes
- Treat generated artifacts and stderr logs as potentially sensitive.
- Avoid logging full rendered prompts by default in production contexts.
- Treat generated artifacts, rendered prompts, stdout, and stderr as potentially sensitive.
- Use controlled output paths and access controls for persisted artifacts.
- Avoid logging full rendered prompts or generated artifacts by default.
## Canonical References
- CLI behavior: [CLI reference](../cli.md)
- Config behavior: [Configuration reference](../config.md)
- Operations and failure handling: [Operations guide](../operations.md), [Troubleshooting](../troubleshooting.md)
- Config and file formats: [Configuration reference](../config.md)
- Operations: [Operations guide](../operations.md)
- Troubleshooting: [Troubleshooting](../troubleshooting.md)