diff --git a/docs/consumers/api.md b/docs/consumers/api.md index f7fa87a..e7e21fd 100644 --- a/docs/consumers/api.md +++ b/docs/consumers/api.md @@ -1,13 +1,122 @@ -# Consumer API Overview +# Consumer Integration Overview -Scriptorium can be used by consumers through three implemented surfaces: +This guide is for applications that call Scriptorium from another codebase. -- CLI commands, documented in [CLI reference](../cli.md). -- HTTP `POST /v1/runs`, documented in [HTTP API reference](../api.md). -- Go package `gitea.maximumdirect.net/eric/scriptorium`, documented in [pkg-scriptorium](pkg-scriptorium.md). +Scriptorium exposes three integration surfaces: -The Go package is the typed in-process API. It prepares prompts, runs prompts, accepts file or inline artifacts, supports per-request execution overrides, and exposes stable public errors for `errors.Is`. +| Surface | Use when | +| --- | --- | +| Go package | The consumer is Go, needs typed requests/results, or wants injected LLM clients for tests. | +| CLI subprocess | The consumer wants process isolation or is not written in Go. | +| HTTP API | The consumer needs a service boundary or remote access to `POST /v1/runs`. | -Use the Go package when the caller is a Go program that wants typed requests/results, context cancellation, repeated calls without subprocess overhead, or fake LLM injection for tests. Use the CLI or HTTP surfaces when process isolation, language neutrality, or an HTTP boundary is preferred. +Canonical references: -Raw API key values are not accepted in public payloads and are not returned in prepared or run results. Execution profiles may reference an environment variable name through `api_key_env`. +- Go package: [Package scriptorium](pkg-scriptorium.md) +- CLI subprocess: [Subprocess integration](../integrations/subprocess.md) +- HTTP: [HTTP API reference](../api.md) +- File formats: [Configuration reference](../config.md) + +## Required Deployment Inputs + +Every integration needs operators to provide: + +- prompt definitions; +- profile definitions or built-in profile IDs; +- schema files when prompts use `json_schema`; +- input artifacts or inline input bodies; +- API-key environment variables or direct per-request keys where supported. + +Raw API keys do not belong in config, prompt files, profile YAML, CLI +arguments, or HTTP request bodies. + +## Recommended Workflow + +Use the Go package when: + +- the consumer is a Go application; +- the application needs `context.Context` cancellation; +- repeated calls should avoid subprocess startup; +- tests need a fake LLM client; +- direct per-request `RunRequest.APIKey` is required. + +Use the CLI subprocess when: + +- the consumer is not Go; +- process isolation is useful; +- stdout/stderr separation and exit codes are enough; +- the consumer already manages local files and environment variables. + +Use HTTP when: + +- Scriptorium should run as a service; +- multiple clients need a shared prompt/profile deployment; +- clients can reach a trusted, protected HTTP boundary. + +## Minimal Go Example + +```go +engine, err := scriptorium.NewEngine(scriptorium.Config{ + PromptDir: "./examples/prompts", + ProfileDir: "./examples/profiles", + SchemaDir: "./examples/schemas", +}) +if err != nil { + return err +} + +prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{ + PromptID: "generic.markdown_summary", + Inputs: map[string]scriptorium.ArtifactRef{ + "transcript": scriptorium.File("./examples/fixtures/transcript.md"), + "glossary": scriptorium.File("./examples/fixtures/glossary.yml"), + }, +}) +if err != nil { + return err +} +_ = prepared.Messages +``` + +Run the maintained package example: + +```bash +go run ./examples/go-library/prepare +``` + +## Subprocess Workflow + +Invoke `scriptorium render` for preflight and `scriptorium run` for generation. +Capture stdout and stderr separately. Treat exit code `2` from `run` as a +completed generation with failed validation. + +See [Subprocess integration](../integrations/subprocess.md) for the stable +invocation contract. + +## HTTP Workflow + +Run `scriptorium serve` behind trusted controls and send JSON requests to +`POST /v1/runs`. + +Do not duplicate endpoint schemas in consumers. Use the [HTTP API +reference](../api.md) as the authoritative contract. + +## Consumer Responsibilities + +Consumers are responsible for: + +- selecting prompt/profile IDs as deployment configuration; +- supplying all required inputs and vars; +- protecting generated artifacts and rendered prompts as sensitive data; +- deciding whether to keep output when validation fails; +- implementing retries only when another model call is acceptable. + +Scriptorium does not persist run state. Retrying a failed or timed-out request +can produce different output and can incur another provider request. + +## Status Behavior + +- Go package methods return typed results or errors that support `errors.Is`. +- CLI `run` exits `2` when generation succeeds but validation fails. +- HTTP returns `200 OK` for generated-content validation failures and exposes the failed status in the response body. +- Runtime validation failures are errors. diff --git a/docs/consumers/pkg-scriptorium.md b/docs/consumers/pkg-scriptorium.md index 696398e..1b43cc7 100644 --- a/docs/consumers/pkg-scriptorium.md +++ b/docs/consumers/pkg-scriptorium.md @@ -6,7 +6,22 @@ Import path: import "gitea.maximumdirect.net/eric/scriptorium" ``` -The root package is a public facade over Scriptorium's prompt execution use case. It keeps `internal/*` packages private while exposing typed construction, preparation, execution, inputs, results, and errors. +The root package is the public Go facade for Scriptorium's prompt prepare/run +workflow. It exposes typed requests, results, source options, injected LLM +clients, and stable public errors while keeping `internal/*` packages private. + +## Intended Use Cases + +Use the package when a Go application needs: + +- in-process prompt preparation or execution; +- typed request/result structs; +- direct `context.Context` cancellation; +- injected/fake LLM clients for tests; +- direct per-request `RunRequest.APIKey`. + +Use [Subprocess integration](../integrations/subprocess.md) or the [HTTP API](../api.md) +when a process or service boundary is preferred. ## Construct An Engine @@ -21,21 +36,56 @@ if err != nil { } ``` -`PromptDir` is required unless an explicit prompt source option is supplied. `ProfileDir` is optional; omit it to use built-in profiles only, or set it to overlay custom profiles above built-ins. `SchemaDir` defaults to the built-in schema directory. `Timeout` and `HTTPClient` configure the default OpenAI-compatible client used by `Run` when no custom LLM client is supplied. +`Config` fields: -## Asset Sources +| Field | Description | +| --- | --- | +| `PromptDir` | Prompt definition directory. Required unless `WithPromptFS` or `WithPromptFile` is used. | +| `ProfileDir` | Optional custom profile directory overlaid above built-in profiles. | +| `SchemaDir` | Schema directory. Defaults to `.` when empty. | +| `Timeout` | Default timeout for the built-in OpenAI-compatible client. | +| `HTTPClient` | Optional HTTP client for the built-in OpenAI-compatible client. | -Directory fields on `Config` remain the compatibility path. Explicit source options override the matching directory field: +`NewEngine` accepts `nil` options and ignores them. Invalid construction wraps +`ErrInvalidConfig`. -- `WithPromptFS(fsys, root)` and `WithPromptFile(path)` -- `WithProfileFS(fsys, root)` and `WithProfileFile(path)` -- `WithSchemaFS(fsys, root)` and `WithSchemaFile(path)` +## Source Options -Prompt and profile sources load standard Scriptorium YAML with the same strict validation as directory sources. For `WithPromptFS`, the configured root is a containment boundary: prompt `content_file` paths resolve relative to the prompt file and must remain inside that root. Profile options overlay custom profiles above built-ins. For `WithSchemaFS`, prompt `schema_path` values resolve inside the configured root. Absolute paths and relative traversal outside those `fs.FS` roots are rejected. Schema file options expose the file by its base name. +Directory fields are the compatibility path. Explicit source options override +the matching directory field. + +Prompt sources: + +- `WithPromptFS(fsys, root)` +- `WithPromptFile(path)` + +Profile sources: + +- `WithProfileFS(fsys, root)` +- `WithProfileFile(path)` +- `WithProfiles(profiles...)` + +Schema sources: + +- `WithSchemaFS(fsys, root)` +- `WithSchemaFile(path)` + +LLM source: + +- `WithLLMClient(client)` + +Source behavior: + +- Prompt and profile YAML use the same strict rules as directory loading. +- Prompt `content_file` values resolve relative to the prompt file. +- `fs.FS` roots are containment boundaries for prompt content files and schema paths. +- File options expose the selected file by its base name. +- Profile source precedence is in-memory profiles, then explicit profile file/FS/directory source, then built-ins. +- `WithLLMClient(nil)` returns `ErrInvalidConfig`. ## In-Memory Profiles -Use `WithProfiles` when the consuming application already has profile settings in typed Go configuration: +Use `WithProfiles` when the application already has typed model settings: ```go profile := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{ @@ -48,13 +98,33 @@ profile := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfi engine, err := scriptorium.NewEngine(cfg, scriptorium.WithProfiles(profile)) ``` -In-memory profiles have highest precedence, followed by configured profile file/FS/directory sources, then built-in profiles. Duplicate IDs in one `WithProfiles` call return `ErrInvalidConfig`. +`Profile` and `OpenAICompatibleProfileConfig` include: -`Profile` and `OpenAICompatibleProfileConfig` include endpoint, model, numeric defaults, service tier, reasoning effort, `APIKeyRequired`, and JSON-compatible `ExtraParams`. `WithProfiles` validates `ExtraParams` and returns `ErrInvalidConfig` for unsupported values such as functions, channels, non-string map keys, non-finite floats, or cyclic values. Raw API-key fields are not accepted. When `APIKeyRequired` is true, pass the secret with `RunRequest.APIKey`. +- `ID` +- `Endpoint` +- `Model` +- `Temperature` +- `MaxTokens` +- `TopP` +- `TimeoutSeconds` +- `ServiceTier` +- `ReasoningEffort` +- `APIKeyRequired` +- `ExtraParams` -## Prepare A Prompt +`WithProfiles` rejects duplicate IDs in one call. In-memory profiles do not +store raw keys. When `APIKeyRequired` is true, pass the secret on each request +with `RunRequest.APIKey`. -`Prepare` resolves the prompt definition, profile, inputs, variables, output contract, structured-output metadata, and rendered messages without calling an LLM. +`ExtraParams` must be JSON-compatible: strings, booleans, finite numbers, +objects with string keys, arrays/slices, and nil. Unsupported values, non-string +map keys, non-finite floats, and cycles return `ErrInvalidConfig` for profiles +or `ErrInvalidRequest` for request overrides. + +## Prepare Workflow + +`Prepare` resolves prompt/profile/input/schema state and renders messages +without calling an LLM. ```go prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{ @@ -67,18 +137,19 @@ prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{ if err != nil { return err } -_ = prepared.Messages +_ = prepared.EffectiveModelParams ``` -Input helpers: +`PreparedRun` includes prompt ID/version/hash, selected profile, effective +model params, output contract, structured-output metadata, input hashes, +rendered prompt hash, rendered messages, and timing fields. It does not include +raw API-key values, model output, validation results, or internal target +presence metadata. -- `scriptorium.File(path)` loads an input artifact from a file. -- `scriptorium.Inline(body)` passes inline input content. -- `scriptorium.InlineWithURI(uri, body)` passes inline content with URI metadata. +## Run Workflow -## Run A Prompt - -`Run` prepares the prompt, calls the configured LLM client, builds the output artifact, and validates the output. +`Run` calls `Prepare`, invokes the configured LLM client, builds the output +artifact, and validates the output. ```go result, err := engine.Run(ctx, scriptorium.RunRequest{ @@ -95,13 +166,25 @@ if err != nil { _ = result.Artifact ``` -`RunResult` includes the run ID, output artifact, raw output, validation result, prompt/profile/model metadata, effective model parameters, input hashes, token/cache usage, and timing fields. Validation content failures return a successful `RunResult` with failed validation status. Runtime validation errors return `ErrValidation`. +`RunResult` includes run ID, output artifact, raw output, validation result, +prompt/profile/model metadata, effective model params, input hashes, usage, and +timing fields. -For the public Go API, pass provider credentials with `RunRequest.APIKey`. The value is request-scoped, uses `json:"-"`, is preferred over profile `api_key_env` by the default OpenAI-compatible client, and is not included in `PreparedRun` or `RunResult` JSON. Normal Go string formatting of `RunRequest` reports only whether a direct key is set. Do not store raw keys in config, prompt files, or profile YAML. +Generated-content validation failures return a successful `RunResult` with +`Validation.Status == ValidationFailed`. Runtime/schema validation errors +return an error that matches `ErrValidation`. -Avoid logging raw request structs with reflection-based debug dumpers; exported fields remain visible to tools that bypass `String` and `GoString` methods. +## Inputs -## Inject An LLM Client +Input helpers: + +- `File(path)`: file-backed artifact reference. +- `Inline(body)`: inline artifact body. +- `InlineWithURI(uri, body)`: inline artifact body with URI metadata. + +Input map keys must match the prompt's expected input names. + +## Injected LLM Clients Use `WithLLMClient` for tests or custom model integrations: @@ -118,11 +201,34 @@ func (fakeLLM) Generate(ctx context.Context, req scriptorium.GenerateRequest) (* engine, err := scriptorium.NewEngine(cfg, scriptorium.WithLLMClient(fakeLLM{})) ``` -The injected client receives the rendered prompt, effective execution target, target presence metadata for explicit numeric overrides, structured-output spec, and request API key when provided. `GenerateRequest.APIKey` also uses `json:"-"`, and normal Go string formatting reports only whether a direct key is set. Custom and fake clients should avoid logging or serializing it. `WithLLMClient(nil)` returns `ErrInvalidConfig`. +Injected clients receive: -## Request Overrides +- rendered prompt; +- effective execution target; +- numeric target presence metadata; +- structured-output spec when applicable; +- direct request API key when provided. -`RunRequest.Execution` accepts per-request overrides. Numeric override fields are pointers so explicit zero values are preserved: +Custom clients should not log raw prompts or API keys by default. + +## Overrides And API Keys + +`RunRequest` fields: + +| Field | Description | +| --- | --- | +| `PromptID` | Prompt ID. | +| `PromptVersion` | Optional prompt version filter. | +| `ProfileID` | Optional profile override. | +| `APIKey` | Direct per-request API key. | +| `Inputs` | Input artifact references. | +| `Vars` | Template variables. | +| `Execution` | Per-request model overrides. | +| `Validation` | Per-request output contract override. | +| `Metadata` | Request metadata reserved for callers. | + +`RunRequest.Execution` uses pointer fields for numeric values so explicit zero +overrides are preserved: ```go zero := 0 @@ -131,11 +237,19 @@ req.Execution = &scriptorium.ExecutionTargetOverride{ } ``` -`ExecutionTargetOverride.ExtraParams` accepts JSON-compatible values and copies typed maps/slices so later caller mutation does not affect the run. Unsupported values, non-string map keys, non-finite floats, and cycles return `ErrInvalidRequest`. +Direct `RunRequest.APIKey` takes precedence over profile `api_key_env` for the +default OpenAI-compatible client. It is request-scoped, uses `json:"-"`, and is +not included in `PreparedRun` or `RunResult` JSON. Normal Go string formatting +of `RunRequest` and `GenerateRequest` reports only whether a direct key is set. + +Raw API keys do not belong in profile YAML, in-memory profiles, or app config. +Avoid reflection-based debug dumps of request structs because exported fields +remain visible to tools that bypass `String` and `GoString`. ## Errors -Public methods wrap context while preserving stable sentinel checks with `errors.Is`: +Public methods wrap context while preserving stable sentinel checks with +`errors.Is`: - `ErrInvalidConfig` - `ErrInvalidRequest` @@ -158,8 +272,13 @@ if errors.Is(err, scriptorium.ErrPromptNotFound) { ## Examples -Run the prepare-only example from the repository root: +Run the maintained prepare-only example from the repository root: ```bash go run ./examples/go-library/prepare ``` + +See also: + +- [Configuration reference](../config.md) +- [Consumer integration overview](api.md) diff --git a/docs/integrations/subprocess.md b/docs/integrations/subprocess.md index f1f15af..8b6eeed 100644 --- a/docs/integrations/subprocess.md +++ b/docs/integrations/subprocess.md @@ -1,59 +1,67 @@ # Subprocess Integration -## Purpose +This document defines the supported subprocess contract for downstream +applications invoking Scriptorium through the public CLI. -This document defines the supported subprocess contract for downstream applications invoking Scriptorium through the public CLI. - -This is a CLI contract, not an internal Go package integration. +This is a CLI contract. Go callers that want an in-process typed API should use +the [package guide](../consumers/pkg-scriptorium.md). ## Supported Commands Downstream applications should invoke: -- `scriptorium run` -- `scriptorium render` +- `scriptorium render` for preflight/debug output without LLM execution. +- `scriptorium run` for generation. -Use `run` for generation. - -Use `render` for preflight/debug output without LLM execution. +`scriptorium serve` is an HTTP service command, not the recommended subprocess +contract for per-request execution. ## Recommended Invocation Shapes -Run: - -```bash -scriptorium run \ - --prompt \ - --input transcript= \ - --out -``` - Render: ```bash scriptorium render \ + --config \ --prompt \ --input transcript= \ --format json ``` +Run: + +```bash +scriptorium run \ + --config \ + --prompt \ + --input transcript= \ + --out +``` + Callers may add: -- `--config ` - `--profile ` - repeatable `--input name=path` - repeatable `--var name=value` -- runtime overrides when explicitly needed (`--model`, `--llm-base-url`, `--timeout`, etc.) +- runtime overrides when explicitly needed, such as `--model`, `--llm-base-url`, `--api-key-env`, and `--timeout` + +Do not pass raw API keys as command arguments. ## Config And Directory Behavior Callers can rely on resolved app config or pass explicit paths. -- default config search order: - 1. `/usr/local/etc/scriptorium/config.yml` - 2. `/etc/scriptorium/config.yml` -- explicit `--config` requires file existence and valid syntax -- CLI flags override config values +Default config search order: + +1. `/usr/local/etc/scriptorium/config.yml` +2. `/etc/scriptorium/config.yml` + +Rules: + +- Explicit `--config` requires file existence and valid syntax. +- CLI flags override config values. +- `run` and `render` require an effective `prompt_dir`. +- `profile_dir` is optional because built-in profiles are available. ## Profile Selection @@ -63,52 +71,60 @@ Profile selection follows runner behavior: 2. prompt `default_profile` 3. error if neither is available -Callers should treat prompt/profile IDs as deployment configuration, not hardcoded logic. +Treat prompt and profile IDs as deployment configuration, not hardcoded business +logic. ## Input And Variable Contract - Inputs use repeated `--input name=path`. - Input names must match prompt definition input names. -- Variables use repeated `--var name=value` for small metadata values. +- Variables use repeated `--var name=value`. +- Both flags also accept comma-separated mappings. - Prefer file inputs for large content. +CLI inputs are file references. HTTP-only `inline` references are documented in +the [HTTP API reference](../api.md). + ## Environment Contract - Pass through required API-key environment variables referenced by `api_key_env`. -- Never pass raw API keys via CLI arguments. -- Keep subprocess environment scoped to required variables. +- Keep subprocess environments scoped to required variables. +- Use `--api-key-env` only to name an environment variable. +- Never pass raw API keys via argv. -## Output And Error Handling +## Stdout And Stderr `run`: -- stdout: artifact body unless `--out` is used -- `--out`: writes artifact to file -- stderr: success summary and errors +- stdout: generated artifact body unless `--out` is used. +- stderr: success summary and errors. `render`: -- stdout: prepared-run output unless `--out` is used -- stderr: errors +- stdout: prepared-run output unless `--out` is used. +- stderr: errors. -Callers should capture stdout and stderr separately. +Capture stdout and stderr separately. Do not parse stderr as a stable data +format beyond exit status handling. ## Exit Status Contract -- `0`: success -- `1`: parse/config/load/render/generation/IO/runtime error -- `2`: run completed but validation failed +- `0`: success. +- `1`: parse, config, load, render, generation, IO, or runtime error. +- `2`: `run` completed and output was written, but validation failed. -A `run` exit code `2` can still produce output (stdout or `--out`). +A `run` exit code `2` can still produce output on stdout or at `--out`. +Consumers must decide whether to keep or discard that output. ## Security Notes -- Treat generated artifacts and stderr logs as potentially sensitive. -- Avoid logging full rendered prompts by default in production contexts. +- Treat generated artifacts, rendered prompts, stdout, and stderr as potentially sensitive. - Use controlled output paths and access controls for persisted artifacts. +- Avoid logging full rendered prompts or generated artifacts by default. ## Canonical References - CLI behavior: [CLI reference](../cli.md) -- Config behavior: [Configuration reference](../config.md) -- Operations and failure handling: [Operations guide](../operations.md), [Troubleshooting](../troubleshooting.md) +- Config and file formats: [Configuration reference](../config.md) +- Operations: [Operations guide](../operations.md) +- Troubleshooting: [Troubleshooting](../troubleshooting.md)