eric/scriptorium
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Add request API key support

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-04 16:55:01 +00:00
parent 32e2433628
commit 3ad247039b
13 changed files with 284 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
convert.go
View File

@@ -11,6 +11,7 @@ func toDomainRunRequest(req RunRequest) domain.RunRequest {
PromptID: req.PromptID,
PromptVersion: req.PromptVersion,
ProfileID: req.ProfileID,
APIKey: req.APIKey,
Inputs: toDomainArtifactRefMap(req.Inputs),
Vars: copyStringMap(req.Vars),
Execution: toDomainExecutionTargetOverride(req.Execution),
@@ -72,6 +73,7 @@ func fromDomainGenerateRequest(req domain.GenerateRequest) GenerateRequest {
Target: fromDomainExecutionTarget(req.Target),
TargetPresence: fromDomainExecutionTargetPresence(req.TargetPresence),
StructuredOutput: fromDomainStructuredOutputSpec(req.StructuredOutput),
APIKey: req.Target.APIKey,
}
}

5
docs/consumers/pkg-scriptorium.md
View File

@@ -54,6 +54,7 @@ Input helpers:
```go
result, err := engine.Run(ctx, scriptorium.RunRequest{
PromptID: "generic.markdown_summary",
APIKey: apiKey,
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.File("./examples/fixtures/transcript.md"),
"glossary": scriptorium.File("./examples/fixtures/glossary.yml"),
@@ -67,6 +68,8 @@ _ = result.Artifact
`RunResult` includes the run ID, output artifact, raw output, validation result, prompt/profile/model metadata, effective model parameters, input hashes, token/cache usage, and timing fields. Validation content failures return a successful `RunResult` with failed validation status. Runtime validation errors return `ErrValidation`.
For the public Go API, pass provider credentials with `RunRequest.APIKey`. The value is request-scoped, uses `json:"-"`, is preferred over profile `api_key_env` by the default OpenAI-compatible client, and is not included in `PreparedRun` or `RunResult` JSON. Do not store raw keys in config, prompt files, or profile YAML.
## Inject An LLM Client
Use `WithLLMClient` for tests or custom model integrations:
@@ -84,7 +87,7 @@ func (fakeLLM) Generate(ctx context.Context, req scriptorium.GenerateRequest) (*
engine, err := scriptorium.NewEngine(cfg, scriptorium.WithLLMClient(fakeLLM{}))
```
The injected client receives the rendered prompt, effective execution target, target presence metadata for explicit numeric overrides, and structured-output spec. `WithLLMClient(nil)` returns `ErrInvalidConfig`.
The injected client receives the rendered prompt, effective execution target, target presence metadata for explicit numeric overrides, structured-output spec, and request API key when provided. `GenerateRequest.APIKey` also uses `json:"-"`; custom and fake clients should avoid logging or serializing it. `WithLLMClient(nil)` returns `ErrInvalidConfig`.
## Request Overrides

3
docs/internal/adapters.md
View File

@@ -37,6 +37,7 @@ Public library facade:
- Input: typed `scriptorium.RunRequest` values.
- Output: typed `PreparedRun` and `RunResult` values plus public sentinel errors.
- Custom LLM behavior is injected with `WithLLMClient`; otherwise the default OpenAI-compatible client is used.
- `RunRequest.APIKey` is a request-scoped Go value only; it is converted into internal execution state for LLM generation and stripped from public result types.
- Public types are facade types converted at the package boundary; internal domain types remain internal.
Filesystem repositories:
@@ -60,6 +61,7 @@ LLM adapter:
- Input: `domain.GenerateRequest`.
- Output: `domain.GenerateResponse`.
- Direct API-key values are preferred when present; otherwise `api_key_env` is resolved from the process environment.
Validator:
@@ -123,6 +125,7 @@ LLM adapter:
- compatible cache usage response fields are parsed into domain token usage.
- non-2xx responses map to request failure errors.
- malformed responses (including missing/empty first choice content) are errors.
- direct API-key values are never serialized in provider request bodies.
Validator:

10
docs/internal/runner.md
View File

@@ -104,9 +104,10 @@ Validation content failures are not run errors:
- selected profile values
- request overrides
- request numeric overrides are presence-aware, so omitted values preserve the current effective value and explicit zero values override it
6. verify required `api_key_env` environment variable:
- missing/empty env value returns an invalid request with `ErrAPIKeyEnvMissing`
- only the environment-variable name is retained; secret value is never returned
6. verify credentials when the effective target names `api_key_env`:
- a request-scoped direct API key satisfies the credential requirement
- otherwise a missing/empty env value returns an invalid request with `ErrAPIKeyEnvMissing`
- only the environment-variable name is returned in public output; secret values are never returned
7. resolve output contract and structured-output schema payload when `json_schema` mode is active.
8. read input artifacts.
9. render prompt messages, including any normalized message cache-control metadata.
@@ -122,7 +123,8 @@ Runtime target notes:
- The OpenAI-compatible client serializes non-empty `reasoning_effort` as a top-level provider request field.
- The OpenAI-compatible client flattens `extra_params` into provider-specific top-level JSON request fields.
- Empty `extra_params` keys, reserved outbound field names, and values that cannot be JSON-encoded fail before the provider request.
- Resolved API-key values are never stored in `PreparedRun`, `RunResult`, logs, or HTTP responses.
- Resolved API-key values are never serialized in prepared/run output, public results, logs, or HTTP responses.
- Public direct API-key values are carried only far enough to call the configured LLM client and are excluded from JSON/YAML serialization.
## Run Flow

169
engine_test.go
View File

@@ -4,6 +4,8 @@ import (
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"reflect"
@@ -205,6 +207,7 @@ func TestRunSucceedsWithInjectedLLMClient(t *testing.T) {
}
func TestRunPassesPreparedRequestToInjectedLLMClient(t *testing.T) {
const directKey = "direct-injected-key"
fake := &fakeLLMClient{
response: &scriptorium.GenerateResponse{Content: "ok"},
}
@@ -214,6 +217,7 @@ func TestRunPassesPreparedRequestToInjectedLLMClient(t *testing.T) {
_, err := engine.Run(context.Background(), scriptorium.RunRequest{
PromptID: "generic.markdown_summary",
APIKey: directKey,
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.Inline("Rin opens the gate."),
"glossary": scriptorium.Inline("gate: A guarded passage."),
@@ -244,6 +248,159 @@ func TestRunPassesPreparedRequestToInjectedLLMClient(t *testing.T) {
if req.StructuredOutput != nil {
t.Fatalf("did not expect structured output for markdown prompt: %+v", req.StructuredOutput)
}
if req.APIKey != directKey {
t.Fatalf("expected direct key on injected generate request")
}
payload, err := json.Marshal(req)
if err != nil {
t.Fatalf("expected generate request to marshal, got %v", err)
}
if strings.Contains(string(payload), directKey) {
t.Fatalf("generate request JSON leaked direct API key: %s", payload)
}
}
func TestRunUsesDirectAPIKeyWithDefaultLLMClient(t *testing.T) {
const directKey = "direct-public-key"
const missingEnv = "SCRIPTORIUM_PUBLIC_DIRECT_MISSING"
t.Setenv(missingEnv, "")
var gotAuth string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotAuth = r.Header.Get("Authorization")
if r.URL.Path != "/v1/chat/completions" {
t.Errorf("unexpected path: %s", r.URL.Path)
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{
"choices": [{"message": {"role": "assistant", "content": "# Summary\n\nDone."}}],
"usage": {"prompt_tokens": 3, "completion_tokens": 4, "total_tokens": 7}
}`))
}))
defer server.Close()
profileDir := t.TempDir()
writePublicProfileFileWithAPIKeyEnv(t, profileDir, "direct-auth", server.URL+"/v1", "test-model", missingEnv)
engine, err := scriptorium.NewEngine(scriptorium.Config{
PromptDir: "./examples/prompts",
ProfileDir: profileDir,
SchemaDir: "./examples/schemas",
})
if err != nil {
t.Fatalf("expected engine construction to succeed, got %v", err)
}
result, err := engine.Run(context.Background(), scriptorium.RunRequest{
PromptID: "generic.markdown_summary",
ProfileID: "direct-auth",
APIKey: directKey,
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.Inline("Rin opens the gate."),
"glossary": scriptorium.Inline("gate: A guarded passage."),
},
})
if err != nil {
t.Fatalf("expected run with direct API key to succeed, got %v", err)
}
if gotAuth != "Bearer "+directKey {
t.Fatalf("unexpected Authorization header: %q", gotAuth)
}
if result.Usage.TotalTokens != 7 {
t.Fatalf("unexpected usage: %+v", result.Usage)
}
payload, err := json.Marshal(result)
if err != nil {
t.Fatalf("expected run result to marshal, got %v", err)
}
if strings.Contains(string(payload), directKey) {
t.Fatalf("run result JSON leaked direct API key: %s", payload)
}
}
func TestPrepareDirectAPIKeyBypassesMissingEnvWithoutLeakingOrHashing(t *testing.T) {
const missingEnv = "SCRIPTORIUM_PUBLIC_PREPARE_MISSING"
const firstKey = "first-direct-key"
const secondKey = "second-direct-key"
t.Setenv(missingEnv, "")
profileDir := t.TempDir()
writePublicProfileFileWithAPIKeyEnv(t, profileDir, "direct-prepare", "http://localhost:8000/v1", "test-model", missingEnv)
engine, err := scriptorium.NewEngine(scriptorium.Config{
PromptDir: "./examples/prompts",
ProfileDir: profileDir,
SchemaDir: "./examples/schemas",
})
if err != nil {
t.Fatalf("expected engine construction to succeed, got %v", err)
}
baseReq := scriptorium.RunRequest{
PromptID: "generic.markdown_summary",
ProfileID: "direct-prepare",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.Inline("Rin opens the gate."),
"glossary": scriptorium.Inline("gate: A guarded passage."),
},
}
firstReq := baseReq
firstReq.APIKey = firstKey
firstPrepared, err := engine.Prepare(context.Background(), firstReq)
if err != nil {
t.Fatalf("expected prepare with direct API key to succeed, got %v", err)
}
secondReq := baseReq
secondReq.APIKey = secondKey
secondPrepared, err := engine.Prepare(context.Background(), secondReq)
if err != nil {
t.Fatalf("expected prepare with alternate direct API key to succeed, got %v", err)
}
if firstPrepared.PromptHash != secondPrepared.PromptHash {
t.Fatalf("direct API keys changed prompt hash: %q vs %q", firstPrepared.PromptHash, secondPrepared.PromptHash)
}
if firstPrepared.RenderedPromptHash != secondPrepared.RenderedPromptHash {
t.Fatalf("direct API keys changed rendered prompt hash: %q vs %q", firstPrepared.RenderedPromptHash, secondPrepared.RenderedPromptHash)
}
payload, err := json.Marshal(firstPrepared)
if err != nil {
t.Fatalf("expected prepared run to marshal, got %v", err)
}
if strings.Contains(string(payload), firstKey) {
t.Fatalf("prepared run JSON leaked direct API key: %s", payload)
}
}
func TestMissingCredentialsFailClearlyWhenProfileRequiresAuth(t *testing.T) {
const missingEnv = "SCRIPTORIUM_PUBLIC_AUTH_MISSING"
t.Setenv(missingEnv, "")
profileDir := t.TempDir()
writePublicProfileFileWithAPIKeyEnv(t, profileDir, "requires-auth", "http://localhost:8000/v1", "test-model", missingEnv)
engine, err := scriptorium.NewEngine(scriptorium.Config{
PromptDir: "./examples/prompts",
ProfileDir: profileDir,
SchemaDir: "./examples/schemas",
})
if err != nil {
t.Fatalf("expected engine construction to succeed, got %v", err)
}
_, err = engine.Prepare(context.Background(), scriptorium.RunRequest{
PromptID: "generic.markdown_summary",
ProfileID: "requires-auth",
Inputs: map[string]scriptorium.ArtifactRef{
"transcript": scriptorium.Inline("Rin opens the gate."),
"glossary": scriptorium.Inline("gate: A guarded passage."),
},
})
if !errors.Is(err, scriptorium.ErrInvalidRequest) {
t.Fatalf("expected invalid request for missing credentials, got %v", err)
}
if err == nil || !strings.Contains(err.Error(), missingEnv) {
t.Fatalf("expected missing env name in error, got %v", err)
}
}
func TestRunValidationFailureReturnsResult(t *testing.T) {
@@ -693,6 +850,18 @@ model: ` + model + `
}
}
func writePublicProfileFileWithAPIKeyEnv(t *testing.T, dir, id, endpoint, model, apiKeyEnv string) {
t.Helper()
data := `id: ` + id + `
endpoint: ` + endpoint + `
model: ` + model + `
api_key_env: ` + apiKeyEnv + `
`
if err := os.WriteFile(filepath.Join(dir, id+".yaml"), []byte(data), 0o644); err != nil {
t.Fatalf("failed to write profile fixture: %v", err)
}
}
type fakeLLMClient struct {
response *scriptorium.GenerateResponse
err error

2
internal/domain/domain.go
View File

@@ -63,6 +63,7 @@ type RunRequest struct {
PromptID string
PromptVersion string
ProfileID string
APIKey string `json:"-" yaml:"-"`
Inputs map[string]ArtifactRef
Vars map[string]string
Execution *ExecutionTargetOverride
@@ -207,6 +208,7 @@ type ExecutionTarget struct {
ServiceTier string `yaml:"service_tier" json:"service_tier"`
ReasoningEffort string `yaml:"reasoning_effort" json:"reasoning_effort"`
APIKeyEnv string `yaml:"api_key_env" json:"api_key_env"`
APIKey string `yaml:"-" json:"-"`
ExtraParams map[string]any `yaml:"extra_params" json:"extra_params"`
}

1
internal/domain/prepared_run_test.go
View File

@@ -20,6 +20,7 @@ func TestPreparedRunJSONDoesNotIncludeSecretValues(t *testing.T) {
Endpoint: "http://llm/v1",
Model: "gpt-test",
APIKeyEnv: envName,
APIKey: secret,
},
InputHashes: map[string]string{"transcript": "hash-1"},
RenderedPromptHash: "rendered-hash",

28
internal/format/prepared_run_test.go
View File

@@ -92,6 +92,20 @@ func TestTextFormatterDoesNotIncludeResolvedAPIKeyValue(t *testing.T) {
}
}
func TestTextFormatterDoesNotIncludeDirectAPIKeyValue(t *testing.T) {
const directKey = "direct-format-key"
prepared := samplePreparedRun()
prepared.EffectiveModelParams.APIKey = directKey
out, err := FormatPreparedRun(prepared, PreparedRunFormatText)
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
if strings.Contains(string(out), directKey) {
t.Fatalf("text output should not include direct api key value: %s", out)
}
}
func TestTextFormatterIncludesMessageCacheControlBeforeContent(t *testing.T) {
prepared := samplePreparedRun()
prepared.Messages = []domain.RenderedMessage{
@@ -269,6 +283,20 @@ func TestJSONFormatterDoesNotIncludeResolvedAPIKeyValue(t *testing.T) {
}
}
func TestJSONFormatterDoesNotIncludeDirectAPIKeyValue(t *testing.T) {
const directKey = "direct-format-key"
prepared := samplePreparedRun()
prepared.EffectiveModelParams.APIKey = directKey
out, err := FormatPreparedRun(prepared, PreparedRunFormatJSON)
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
if strings.Contains(string(out), directKey) {
t.Fatalf("json output should not include direct api key value: %s", out)
}
}
func TestParsePreparedRunOutputFormatRecognizesSupportedNames(t *testing.T) {
tests := []struct {
name string

4
internal/llm/openai_compatible_client.go
View File

@@ -105,7 +105,9 @@ func (c *OpenAICompatibleClient) Generate(ctx context.Context, req domain.Genera
return nil, fmt.Errorf("%w: failed to create request: %v", ErrRequestFailed, err)
}
httpReq.Header.Set("Content-Type", "application/json")
if envName := strings.TrimSpace(req.Target.APIKeyEnv); envName != "" {
if apiKey := strings.TrimSpace(req.Target.APIKey); apiKey != "" {
httpReq.Header.Set("Authorization", "Bearer "+apiKey)
} else if envName := strings.TrimSpace(req.Target.APIKeyEnv); envName != "" {
apiKey := strings.TrimSpace(os.Getenv(envName))
if apiKey == "" {
return nil, fmt.Errorf("%w: api key environment variable %q is not set", ErrInvalidRequest, envName)

32
internal/llm/openai_compatible_client_test.go
View File

@@ -148,6 +148,38 @@ func TestOpenAICompatibleClientGenerateSuccess(t *testing.T) {
}
}
func TestOpenAICompatibleClientDirectAPIKeyPreferredOverEnv(t *testing.T) {
const directKey = "direct-llm-key"
t.Setenv("SCRIPTORIUM_TEST_API_KEY", "env-key")
var gotAuth string
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotAuth = r.Header.Get("Authorization")
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
}))
defer ts.Close()
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: ts.URL + "/v1"})
if err != nil {
t.Fatal(err)
}
_, err = client.Generate(context.Background(), domain.GenerateRequest{
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
Target: domain.ExecutionTarget{
Model: "model",
APIKeyEnv: "SCRIPTORIUM_TEST_API_KEY",
APIKey: directKey,
},
})
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
if gotAuth != "Bearer "+directKey {
t.Fatalf("unexpected Authorization header: %q", gotAuth)
}
}
func TestOpenAICompatibleClientSerializesCacheControlledMessageAsContentBlock(t *testing.T) {
var observedBody map[string]any
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

8
internal/usecase/runner.go
View File

@@ -195,13 +195,14 @@ func (r *Runner) Prepare(ctx context.Context, req domain.RunRequest) (*domain.Pr
if err != nil {
return nil, fmt.Errorf("%w: %w", ErrInvalidRequest, err)
}
effectiveModel.APIKey = req.APIKey
if strings.TrimSpace(effectiveModel.Endpoint) == "" {
return nil, fmt.Errorf("%w: execution endpoint is required", ErrInvalidRequest)
}
if strings.TrimSpace(effectiveModel.Model) == "" {
return nil, fmt.Errorf("%w: execution model is required", ErrInvalidRequest)
}
if err := validateAPIKeyEnv(effectiveModel.APIKeyEnv); err != nil {
if err := validateAPIKey(effectiveModel.APIKeyEnv, effectiveModel.APIKey); err != nil {
return nil, fmt.Errorf("%w: %w", ErrInvalidRequest, err)
}
@@ -434,7 +435,10 @@ func resolveExecutionTarget(profileValue *domain.ExecutionProfile, override *dom
return out, presence, nil
}
func validateAPIKeyEnv(apiKeyEnv string) error {
func validateAPIKey(apiKeyEnv string, apiKey string) error {
if strings.TrimSpace(apiKey) != "" {
return nil
}
envName := strings.TrimSpace(apiKeyEnv)
if envName == "" {
return nil

26
internal/usecase/runner_test.go
View File

@@ -1179,6 +1179,32 @@ func TestRunnerRunAPIKeyEnvMissingEnvironmentValueFailsClearly(t *testing.T) {
}
}
func TestRunnerRunDirectAPIKeyBypassesMissingEnvAndReachesLLM(t *testing.T) {
const directKey = "direct-runner-key"
promptRepo := &fakePromptRepo{def: promptDef(domain.FormatText, domain.ValidationNone, 0)}
execRepo := &fakeExecutionProfileRepo{profiles: map[string]*domain.ExecutionProfile{
"exec": {ID: "exec", Endpoint: "http://profile/v1", Model: "profile-model", APIKeyEnv: "SCRIPTORIUM_MISSING_KEY"},
}}
llmClient := &fakeLLM{resp: &domain.GenerateResponse{Content: "ok"}}
runner := NewRunner(promptRepo, execRepo, defaultArtifactReader(), defaultRenderer(), llmClient, nil)
_, err := runner.Run(context.Background(), domain.RunRequest{
PromptID: "p",
ProfileID: "exec",
APIKey: directKey,
Inputs: singleInputRef(),
})
if err != nil {
t.Fatalf("expected no error, got %v", err)
}
if llmClient.lastReq.Target.APIKey != directKey {
t.Fatalf("expected direct API key to reach LLM request")
}
if llmClient.lastReq.Target.APIKeyEnv != "SCRIPTORIUM_MISSING_KEY" {
t.Fatalf("expected api_key_env name to remain on target, got %q", llmClient.lastReq.Target.APIKeyEnv)
}
}
func TestRunnerRunRuntimeAPIKeyEnvOverrideWorks(t *testing.T) {
const envName = "SCRIPTORIUM_RUNTIME_API_KEY"
t.Setenv(envName, "runtime-secret")

2
types.go
View File

@@ -60,6 +60,7 @@ type RunRequest struct {
PromptID string
PromptVersion string
ProfileID string
APIKey string `json:"-"`
Inputs map[string]ArtifactRef
Vars map[string]string
Execution *ExecutionTargetOverride
@@ -232,6 +233,7 @@ type GenerateRequest struct {
Target ExecutionTarget `json:"target"`
TargetPresence ExecutionTargetPresence `json:"target_presence"`
StructuredOutput *StructuredOutputSpec `json:"structured_output,omitempty"`
APIKey string `json:"-"`
}
// GenerateResponse is returned by an injected LLM client.