Compare commits
15 Commits
39485d87f6
...
v0.11.1
| Author | SHA1 | Date | |
|---|---|---|---|
| 33698903be | |||
| 90b76ddad3 | |||
| d5b3d1e061 | |||
| 41083de46a | |||
| 07ac7e54c5 | |||
| 879cb021b2 | |||
| 574f88bd6a | |||
| d5d7a222a4 | |||
| aabd89aea7 | |||
| 9189cbfc22 | |||
| 872c166ed7 | |||
| 6742def4d3 | |||
| 1b39f82117 | |||
| f7d821067f | |||
| a16f66cbc7 |
2
LICENSE
2
LICENSE
@@ -1,4 +1,4 @@
|
|||||||
Copyright (c) 2026 eric.
|
Copyright (c) 2026 Eric Rakestraw.
|
||||||
|
|
||||||
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
||||||
|
|
||||||
|
|||||||
15
README.md
15
README.md
@@ -1,8 +1,12 @@
|
|||||||
# scriptorium
|
# scriptorium
|
||||||
|
|
||||||
Scriptorium is a config-driven prompt execution engine.
|
Scriptorium is a narrow prompt-execution application for rendering prompt
|
||||||
|
requests, running them against OpenAI-compatible chat-completions endpoints, and
|
||||||
|
serving the same run workflow over HTTP.
|
||||||
|
|
||||||
It separates prompt definitions (what to generate) from execution profiles (how to call an OpenAI-compatible model endpoint), then runs or renders a prepared request from named input artifacts.
|
It keeps prompt definitions, execution profiles, schemas, and input artifacts as
|
||||||
|
separate files so prompts can be reviewed and reused without baking model
|
||||||
|
runtime settings into application code.
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
@@ -23,16 +27,19 @@ This command renders the prepared prompt and effective runtime settings without
|
|||||||
|
|
||||||
- [CLI reference](docs/cli.md)
|
- [CLI reference](docs/cli.md)
|
||||||
- [Configuration reference](docs/config.md)
|
- [Configuration reference](docs/config.md)
|
||||||
|
- [HTTP API reference](docs/api.md)
|
||||||
- [Operations guide](docs/operations.md)
|
- [Operations guide](docs/operations.md)
|
||||||
- [Troubleshooting](docs/troubleshooting.md)
|
- [Troubleshooting](docs/troubleshooting.md)
|
||||||
|
- [Consumer integration overview](docs/consumers/api.md)
|
||||||
- [Go library package](docs/consumers/pkg-scriptorium.md)
|
- [Go library package](docs/consumers/pkg-scriptorium.md)
|
||||||
- [HTTP API integration](docs/integrations/http-api.md)
|
- [Subprocess integration](docs/integrations/subprocess.md)
|
||||||
- [OpenAI-compatible chat integration](docs/integrations/openai-compatible-chat.md)
|
- [OpenAI-compatible chat integration](docs/integrations/openai-compatible-chat.md)
|
||||||
- [Narratio subprocess integration](docs/integrations/narratio.md)
|
|
||||||
- [Architecture policy](docs/policy/architecture.md)
|
- [Architecture policy](docs/policy/architecture.md)
|
||||||
|
|
||||||
## Examples
|
## Examples
|
||||||
|
|
||||||
|
- `examples/config.yml`
|
||||||
|
- `examples/config.full.yml`
|
||||||
- `examples/render-markdown-summary.sh`
|
- `examples/render-markdown-summary.sh`
|
||||||
- `examples/http-run.json`
|
- `examples/http-run.json`
|
||||||
- `examples/go-library/prepare`
|
- `examples/go-library/prepare`
|
||||||
|
|||||||
297
docs/api.md
Normal file
297
docs/api.md
Normal file
@@ -0,0 +1,297 @@
|
|||||||
|
# HTTP API Reference
|
||||||
|
|
||||||
|
This is the canonical public HTTP contract for Scriptorium.
|
||||||
|
|
||||||
|
Implemented route:
|
||||||
|
|
||||||
|
- `POST /v1/runs`
|
||||||
|
|
||||||
|
For CLI behavior, see [CLI reference](cli.md). For config and prompt/profile
|
||||||
|
file formats, see [Configuration reference](config.md).
|
||||||
|
|
||||||
|
The maintained request-shape example is `examples/http-run.json`. It requires a
|
||||||
|
running `serve` process with an artifact root that can read the referenced
|
||||||
|
files, plus a reachable model endpoint for full execution.
|
||||||
|
|
||||||
|
## Base URL And Deployment
|
||||||
|
|
||||||
|
`scriptorium serve` listens on `server.addr` or `serve --addr`. The default is
|
||||||
|
`:8080`.
|
||||||
|
|
||||||
|
The route path is always:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/v1/runs
|
||||||
|
```
|
||||||
|
|
||||||
|
The HTTP adapter has no built-in authentication or authorization. Deploy it
|
||||||
|
behind trusted network and authentication controls.
|
||||||
|
|
||||||
|
## Media Types
|
||||||
|
|
||||||
|
- Request body: JSON object.
|
||||||
|
- Response body: JSON object.
|
||||||
|
- Response `Content-Type`: `application/json`.
|
||||||
|
|
||||||
|
Requests are decoded as JSON regardless of the request `Content-Type` header.
|
||||||
|
There are no shared query parameters.
|
||||||
|
|
||||||
|
## Request Limits
|
||||||
|
|
||||||
|
HTTP limits are configured through `server.*` config fields or `serve` flags:
|
||||||
|
|
||||||
|
- `server.max_request_bytes`: encoded JSON request body limit, including inline input bodies.
|
||||||
|
- `server.max_artifact_bytes`: file artifact limit for HTTP `file` input references.
|
||||||
|
- `server.max_response_bytes`: encoded JSON response limit, including artifact body and optional raw output.
|
||||||
|
|
||||||
|
Each limit defaults to `16777216` bytes. `0` disables that limit.
|
||||||
|
|
||||||
|
## `POST /v1/runs`
|
||||||
|
|
||||||
|
Runs one prompt request and returns the generated artifact, validation result,
|
||||||
|
and metadata.
|
||||||
|
|
||||||
|
### Request Body
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"prompt_id": "generic.markdown_summary",
|
||||||
|
"profile_id": "local-fast",
|
||||||
|
"prompt_version": "1.0.0",
|
||||||
|
"inputs": {
|
||||||
|
"transcript": {
|
||||||
|
"type": "file",
|
||||||
|
"uri": "./examples/fixtures/transcript.md"
|
||||||
|
},
|
||||||
|
"glossary": {
|
||||||
|
"type": "inline",
|
||||||
|
"body": "party:\n - Rin"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"vars": {
|
||||||
|
"session_date": "2026-05-04"
|
||||||
|
},
|
||||||
|
"model": {
|
||||||
|
"endpoint": "http://localhost:8000/v1",
|
||||||
|
"model": "gpt-4o-mini",
|
||||||
|
"temperature": 0,
|
||||||
|
"max_tokens": 800,
|
||||||
|
"top_p": 1,
|
||||||
|
"timeout_seconds": 120,
|
||||||
|
"service_tier": "priority",
|
||||||
|
"reasoning_effort": "medium",
|
||||||
|
"api_key_env": "SCRIPTORIUM_API_KEY",
|
||||||
|
"extra_params": {
|
||||||
|
"provider_option": "enabled"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"include_raw_output": false
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Request fields:
|
||||||
|
|
||||||
|
| Field | Required | Description |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `prompt_id` | yes | Prompt ID. Must not be blank. |
|
||||||
|
| `prompt_version` | no | Prompt version filter. |
|
||||||
|
| `profile_id` | no | Execution profile ID. If omitted, the prompt must define `default_profile`. |
|
||||||
|
| `inputs` | yes | Object mapping prompt input names to input references. Must contain at least one entry. |
|
||||||
|
| `vars` | no | Object mapping template variable names to string values. |
|
||||||
|
| `model` | no | Runtime model override object. |
|
||||||
|
| `include_raw_output` | no | When `true`, include `raw_model_output` in the response. |
|
||||||
|
|
||||||
|
Input reference fields:
|
||||||
|
|
||||||
|
| Field | Required | Description |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `type` | yes | `file` or `inline`. |
|
||||||
|
| `uri` | for `file` | File URI/path. |
|
||||||
|
| `body` | for `inline` | Inline artifact body. |
|
||||||
|
|
||||||
|
HTTP `file` references require `server.artifact_root` or `serve
|
||||||
|
--artifact-root`. Relative file URIs resolve against that root. Absolute file
|
||||||
|
URIs are accepted only when lexically inside the root. Relative traversal and
|
||||||
|
absolute paths outside the root return `400 artifact_not_allowed`.
|
||||||
|
|
||||||
|
The containment check is lexical and does not resolve symlinks. Symlinks inside
|
||||||
|
the artifact root are followed by the operating system, including symlinks that
|
||||||
|
point outside the root. Keep the artifact root narrow and not writable by
|
||||||
|
untrusted users.
|
||||||
|
|
||||||
|
Model override fields:
|
||||||
|
|
||||||
|
| Field | Description |
|
||||||
|
| --- | --- |
|
||||||
|
| `endpoint` | Runtime endpoint override. |
|
||||||
|
| `model` | Runtime model override. |
|
||||||
|
| `temperature` | Number in range `0..2`. Explicit `0` is an override. |
|
||||||
|
| `max_tokens` | Integer greater than or equal to `0`. Explicit `0` is an override. |
|
||||||
|
| `top_p` | Number in range `0..1`. Explicit `0` is an override. |
|
||||||
|
| `timeout_seconds` | Integer greater than or equal to `0`. Explicit `0` disables the outbound client timeout. |
|
||||||
|
| `service_tier` | Provider-specific request tier. |
|
||||||
|
| `reasoning_effort` | Provider-specific reasoning setting. |
|
||||||
|
| `api_key_env` | Name of an environment variable containing the API key. |
|
||||||
|
| `extra_params` | JSON-compatible provider-specific top-level request fields. |
|
||||||
|
|
||||||
|
Raw API-key values are not accepted in HTTP payloads. A field such as
|
||||||
|
`api_key` is rejected as unknown JSON.
|
||||||
|
|
||||||
|
`extra_params` keys must not be empty and must not collide with reserved
|
||||||
|
outbound fields: `model`, `session_id`, `messages`, `temperature`,
|
||||||
|
`max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or
|
||||||
|
`response_format`.
|
||||||
|
|
||||||
|
### Strict JSON Rules
|
||||||
|
|
||||||
|
Request decoding is strict:
|
||||||
|
|
||||||
|
- malformed JSON returns `400 invalid_json`
|
||||||
|
- unknown request fields return `400 invalid_json`
|
||||||
|
- unknown `inputs` item fields return `400 invalid_json`
|
||||||
|
- unknown `model` fields return `400 invalid_json`
|
||||||
|
- trailing JSON tokens after the request object return `400 invalid_json`
|
||||||
|
- request bodies above the configured limit return `413 request_too_large`
|
||||||
|
|
||||||
|
### Success Response
|
||||||
|
|
||||||
|
Status: `200 OK`
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"artifact": {
|
||||||
|
"name": "output",
|
||||||
|
"content_type": "text/markdown",
|
||||||
|
"body": "Generated content",
|
||||||
|
"size": 17,
|
||||||
|
"hash": "..."
|
||||||
|
},
|
||||||
|
"validation": {
|
||||||
|
"status": "passed",
|
||||||
|
"mode": "basic",
|
||||||
|
"repair_attempts": 0,
|
||||||
|
"is_valid": true
|
||||||
|
},
|
||||||
|
"metadata": {
|
||||||
|
"run_id": "...",
|
||||||
|
"prompt_id": "generic.markdown_summary",
|
||||||
|
"prompt_version": "1.0.0",
|
||||||
|
"prompt_hash": "...",
|
||||||
|
"rendered_prompt_hash": "...",
|
||||||
|
"selected_profile_id": "local-fast",
|
||||||
|
"model_name": "gpt-4o-mini",
|
||||||
|
"endpoint": "http://localhost:8000/v1",
|
||||||
|
"model_params": {
|
||||||
|
"endpoint": "http://localhost:8000/v1",
|
||||||
|
"model": "gpt-4o-mini",
|
||||||
|
"temperature": 0.2,
|
||||||
|
"max_tokens": 500,
|
||||||
|
"top_p": 1,
|
||||||
|
"timeout_seconds": 90
|
||||||
|
},
|
||||||
|
"input_hashes": {
|
||||||
|
"transcript": "..."
|
||||||
|
},
|
||||||
|
"usage": {
|
||||||
|
"prompt_tokens": 11,
|
||||||
|
"completion_tokens": 22,
|
||||||
|
"total_tokens": 33,
|
||||||
|
"cached_tokens": 0,
|
||||||
|
"cache_write_tokens": 0
|
||||||
|
},
|
||||||
|
"start_time": "2026-05-04T12:00:00Z",
|
||||||
|
"end_time": "2026-05-04T12:00:01Z",
|
||||||
|
"duration_ms": 1000,
|
||||||
|
"validation_mode": "basic",
|
||||||
|
"validation_status": "passed",
|
||||||
|
"repair_attempts_used": 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Response fields:
|
||||||
|
|
||||||
|
- `artifact`: generated output artifact.
|
||||||
|
- `validation`: validation result for the generated artifact.
|
||||||
|
- `metadata`: run and effective runtime metadata.
|
||||||
|
- `raw_model_output`: omitted unless `include_raw_output` is `true`.
|
||||||
|
|
||||||
|
`artifact.uri` is omitted when empty. `validation.errors` and
|
||||||
|
`validation.schema_path` are omitted when empty. `model_params.service_tier`,
|
||||||
|
`model_params.reasoning_effort`, `model_params.api_key_env`, and
|
||||||
|
`model_params.extra_params` are omitted when empty.
|
||||||
|
|
||||||
|
`metadata.usage.cached_tokens` and `metadata.usage.cache_write_tokens` are
|
||||||
|
always present as numbers. They are `0` when the provider omits compatible cache
|
||||||
|
usage fields or reports no cache activity.
|
||||||
|
|
||||||
|
### Validation Failure Response
|
||||||
|
|
||||||
|
Generated-content validation failures still return `200 OK`.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"validation": {
|
||||||
|
"status": "failed",
|
||||||
|
"mode": "json",
|
||||||
|
"errors": ["invalid JSON: ..."],
|
||||||
|
"repair_attempts": 0,
|
||||||
|
"is_valid": false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The response still includes `artifact` and `metadata`.
|
||||||
|
|
||||||
|
## Error Responses
|
||||||
|
|
||||||
|
Error body shape:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"error": {
|
||||||
|
"code": "invalid_request",
|
||||||
|
"message": "prompt_id is required"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Current status/code mapping:
|
||||||
|
|
||||||
|
| Status | Code | Meaning |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `400` | `invalid_json` | Malformed JSON, unknown JSON field, or trailing JSON token. |
|
||||||
|
| `400` | `invalid_request` | Missing/invalid request fields or invalid runtime overrides. |
|
||||||
|
| `400` | `profile_required` | No `profile_id` and prompt has no `default_profile`. |
|
||||||
|
| `400` | `prompt_load_failed` | Prompt definition YAML/contract failed to load. |
|
||||||
|
| `400` | `profile_load_failed` | Profile YAML/contract failed to load, including raw `api_key`. |
|
||||||
|
| `400` | `artifact_not_allowed` | HTTP file refs are disabled or requested path is outside artifact root. |
|
||||||
|
| `400` | `artifact_read_failed` | Input artifact could not be read or input ref was unsupported/invalid. |
|
||||||
|
| `400` | `prompt_render_failed` | Prompt template rendering failed. |
|
||||||
|
| `400` | `api_key_env_missing` | Selected `api_key_env` variable is unset or empty. |
|
||||||
|
| `404` | `not_found` | Route path is unknown. |
|
||||||
|
| `404` | `prompt_not_found` | Prompt ID/version was not found. |
|
||||||
|
| `404` | `profile_not_found` | Profile ID was not found. |
|
||||||
|
| `405` | `method_not_allowed` | Method is not `POST` on `/v1/runs`. |
|
||||||
|
| `413` | `request_too_large` | Encoded JSON request body exceeds configured request limit. |
|
||||||
|
| `413` | `artifact_too_large` | HTTP file input artifact exceeds configured artifact limit. |
|
||||||
|
| `413` | `response_too_large` | Encoded JSON response exceeds configured response limit. |
|
||||||
|
| `500` | `validation_runtime_failed` | Validator runtime/schema loading failed. |
|
||||||
|
| `500` | `internal_error` | Unclassified server error. |
|
||||||
|
| `502` | `llm_failed` | Outbound model request failed. |
|
||||||
|
|
||||||
|
HTTP error messages are intentionally concise and do not include sensitive
|
||||||
|
internal causes.
|
||||||
|
|
||||||
|
## Retry And Idempotency
|
||||||
|
|
||||||
|
Scriptorium does not provide idempotency keys, pagination, caching headers, or
|
||||||
|
rate limiting.
|
||||||
|
|
||||||
|
Clients may retry transport failures or `5xx` responses when their surrounding
|
||||||
|
workflow can tolerate another model call. A retry can generate different output
|
||||||
|
and incur another provider request.
|
||||||
|
|
||||||
|
## Example File
|
||||||
|
|
||||||
|
- `examples/http-run.json`
|
||||||
181
docs/cli.md
181
docs/cli.md
@@ -10,124 +10,195 @@ go run ./cmd/scriptorium render \
|
|||||||
--input glossary=./examples/fixtures/glossary.yml
|
--input glossary=./examples/fixtures/glossary.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
`render` prepares and formats the prompt without calling an LLM.
|
`render` prepares the prompt, loads input artifacts, resolves the execution
|
||||||
|
profile, and prints the prepared request without calling an LLM.
|
||||||
|
|
||||||
## Command Overview
|
## Command Overview
|
||||||
|
|
||||||
- `scriptorium run`: prepare prompt, call the configured LLM, write generated output, print a run summary.
|
- `scriptorium run`: prepare a prompt, call the configured LLM, write generated output, and print a run summary.
|
||||||
- `scriptorium render`: prepare prompt only; write prepared-run output as `text` or `json`.
|
- `scriptorium render`: prepare a prompt only; write prepared-run output as `text` or `json`.
|
||||||
- `scriptorium serve`: start the HTTP server.
|
- `scriptorium serve`: start the HTTP server for `POST /v1/runs`.
|
||||||
|
|
||||||
Integration references:
|
Canonical related references:
|
||||||
|
|
||||||
- [HTTP contract](integrations/http-api.md)
|
- [Configuration reference](config.md)
|
||||||
- [Narratio subprocess contract](integrations/narratio.md)
|
- [HTTP API reference](api.md)
|
||||||
|
- [Subprocess integration](integrations/subprocess.md)
|
||||||
|
|
||||||
## Common Argument Rules
|
## Common Rules
|
||||||
|
|
||||||
- `--config` is supported by `run`, `render`, and `serve`.
|
- `--config` is supported by `run`, `render`, and `serve`.
|
||||||
- `run` and `render` require:
|
|
||||||
- `--prompt`
|
|
||||||
- at least one `--input`
|
|
||||||
- an effective `prompt_dir` from flags or config
|
|
||||||
- `serve` requires an effective `prompt_dir` from flags or config.
|
|
||||||
- `profile_dir` is optional. If omitted, only built-in profiles are available; if provided, custom profiles override built-ins with the same ID.
|
|
||||||
- Built-in profile IDs are listed in the [configuration reference](config.md#profile-definition-files).
|
|
||||||
- Positional arguments are rejected.
|
- Positional arguments are rejected.
|
||||||
- Prompt cache control is configured in prompt YAML (`messages[].cache_control`), not with CLI flags.
|
- `run` and `render` require `--prompt`, at least one `--input`, and an effective `prompt_dir`.
|
||||||
- Provider-specific `reasoning_effort` and `extra_params` are configured in profile YAML or HTTP model overrides, not with CLI flags.
|
- `serve` requires an effective `prompt_dir`.
|
||||||
|
- `profile_dir` is optional. Without it, only built-in profiles are available.
|
||||||
|
- If `profile_dir` is set, custom profiles override built-in profiles with the same ID.
|
||||||
|
- Prompt cache control, `session_id`, structured output, and provider-specific profile fields are configured in YAML, not with CLI flags.
|
||||||
|
|
||||||
|
Config precedence is:
|
||||||
|
|
||||||
|
1. built-in defaults
|
||||||
|
2. config file values
|
||||||
|
3. CLI flags
|
||||||
|
|
||||||
## Flag Reference
|
## Flag Reference
|
||||||
|
|
||||||
### `scriptorium run`
|
### `scriptorium run`
|
||||||
|
|
||||||
- `--config <path>`: app config file path.
|
```bash
|
||||||
|
scriptorium run [flags]
|
||||||
|
```
|
||||||
|
|
||||||
|
Required through flags or config:
|
||||||
|
|
||||||
- `--prompt-dir <dir>`: prompt definition directory.
|
- `--prompt-dir <dir>`: prompt definition directory.
|
||||||
|
|
||||||
|
Required as flags:
|
||||||
|
|
||||||
|
- `--prompt <id>`: prompt ID to execute.
|
||||||
|
- `--input name=path`: input file mapping. Repeat or use comma-separated mappings.
|
||||||
|
|
||||||
|
Optional flags:
|
||||||
|
|
||||||
|
- `--config <path>`: application config file.
|
||||||
- `--profile-dir <dir>`: custom profile definition directory.
|
- `--profile-dir <dir>`: custom profile definition directory.
|
||||||
- `--schema-dir <dir>`: schema base directory for `json_schema` validation.
|
- `--schema-dir <dir>`: schema base directory for `json_schema` validation.
|
||||||
- `--prompt <id>`: prompt ID to execute. Required.
|
- `--profile <id>`: execution profile override. If omitted, the prompt `default_profile` is used.
|
||||||
- `--prompt-id <id>`: deprecated alias for `--prompt`.
|
- `--var name=value`: template variable mapping. Repeat or use comma-separated mappings.
|
||||||
- `--profile <id>`: explicit profile override.
|
- `--out <path>`: write generated artifact body to a file instead of stdout.
|
||||||
- `--profile-id <id>`: deprecated alias for `--profile`.
|
|
||||||
- `--input name=path`: input mapping (repeatable, comma-separated accepted).
|
|
||||||
- `--var name=value`: template variable mapping (repeatable, comma-separated accepted).
|
|
||||||
- `--out <path>`: write artifact body to file instead of stdout.
|
|
||||||
- `--llm-base-url <url>`: runtime endpoint override.
|
- `--llm-base-url <url>`: runtime endpoint override.
|
||||||
- `--model <name>`: runtime model override.
|
- `--model <name>`: runtime model override.
|
||||||
- `--api-key-env <name>`: runtime API key environment-variable name override.
|
- `--api-key-env <name>`: runtime API-key environment variable name override.
|
||||||
- `--temperature <float>`: runtime temperature override.
|
- `--temperature <float>`: runtime temperature override.
|
||||||
- `--max-tokens <int>`: runtime max tokens override.
|
- `--max-tokens <int>`: runtime max tokens override.
|
||||||
- `--top-p <float>`: runtime top-p override.
|
- `--top-p <float>`: runtime top-p override.
|
||||||
- `--timeout <duration>`: runtime timeout override (Go duration syntax, for example `30s`, `2m`).
|
- `--timeout <duration>`: runtime timeout override using Go duration syntax, such as `30s` or `2m`.
|
||||||
|
|
||||||
Numeric runtime override flags are presence-aware:
|
Deprecated aliases:
|
||||||
|
|
||||||
- omitted numeric flags preserve the selected profile/default value
|
- `--prompt-id <id>`: alias for `--prompt`.
|
||||||
- explicit zero values override the selected profile/default value (`--temperature 0`, `--max-tokens 0`, `--top-p 0`, `--timeout 0s`)
|
- `--profile-id <id>`: alias for `--profile`.
|
||||||
|
|
||||||
|
Runtime override notes:
|
||||||
|
|
||||||
|
- Omitted numeric override flags preserve the selected profile/default value.
|
||||||
|
- Explicit zero values override the selected profile/default value.
|
||||||
|
- `--timeout 0s` disables the outbound HTTP client timeout for that request.
|
||||||
|
- There is no raw API-key flag; use `--api-key-env`.
|
||||||
|
|
||||||
### `scriptorium render`
|
### `scriptorium render`
|
||||||
|
|
||||||
- Supports the same flags as `run`, except:
|
```bash
|
||||||
- no `--schema-dir` flag.
|
scriptorium render [flags]
|
||||||
- Adds:
|
```
|
||||||
- `--format text|json`: prepared-run output format.
|
|
||||||
|
Required through flags or config:
|
||||||
|
|
||||||
|
- `--prompt-dir <dir>`: prompt definition directory.
|
||||||
|
|
||||||
|
Required as flags:
|
||||||
|
|
||||||
|
- `--prompt <id>`: prompt ID to render.
|
||||||
|
- `--input name=path`: input file mapping. Repeat or use comma-separated mappings.
|
||||||
|
|
||||||
|
Optional flags:
|
||||||
|
|
||||||
|
- `--config <path>`: application config file.
|
||||||
|
- `--prompt-dir <dir>`: prompt definition directory.
|
||||||
|
- `--profile-dir <dir>`: custom profile definition directory.
|
||||||
|
- `--profile <id>`: execution profile override.
|
||||||
|
- `--var name=value`: template variable mapping. Repeat or use comma-separated mappings.
|
||||||
|
- `--out <path>`: write prepared-run output to a file instead of stdout.
|
||||||
|
- `--llm-base-url <url>`: runtime endpoint override for the prepared request.
|
||||||
|
- `--model <name>`: runtime model override for the prepared request.
|
||||||
|
- `--api-key-env <name>`: runtime API-key environment variable name override.
|
||||||
|
- `--temperature <float>`: runtime temperature override.
|
||||||
|
- `--max-tokens <int>`: runtime max tokens override.
|
||||||
|
- `--top-p <float>`: runtime top-p override.
|
||||||
|
- `--timeout <duration>`: runtime timeout override using Go duration syntax.
|
||||||
|
- `--format text|json`: prepared-run output format. Defaults to config `defaults.render_format`, then `text`.
|
||||||
|
|
||||||
|
Deprecated aliases:
|
||||||
|
|
||||||
|
- `--prompt-id <id>`: alias for `--prompt`.
|
||||||
|
- `--profile-id <id>`: alias for `--profile`.
|
||||||
|
|
||||||
Notes:
|
Notes:
|
||||||
- `render` still resolves profile and runtime settings.
|
|
||||||
- `render` still validates that `api_key_env` exists if the selected profile or overrides require it.
|
- `render` resolves profiles, loads schemas for `json_schema` prompts, and validates `api_key_env`.
|
||||||
|
- `render` does not accept `--schema-dir`; use config `schema_dir` for render-time schema lookup.
|
||||||
|
- `render` does not call the LLM.
|
||||||
|
|
||||||
### `scriptorium serve`
|
### `scriptorium serve`
|
||||||
|
|
||||||
- `--config <path>`: app config file path.
|
```bash
|
||||||
|
scriptorium serve [flags]
|
||||||
|
```
|
||||||
|
|
||||||
|
Required through flags or config:
|
||||||
|
|
||||||
|
- `--prompt-dir <dir>`: prompt definition directory.
|
||||||
|
|
||||||
|
Optional flags:
|
||||||
|
|
||||||
|
- `--config <path>`: application config file.
|
||||||
- `--addr <listen-address>`: HTTP listen address.
|
- `--addr <listen-address>`: HTTP listen address.
|
||||||
- `--prompt-dir <dir>`: prompt definition directory.
|
- `--prompt-dir <dir>`: prompt definition directory.
|
||||||
- `--profile-dir <dir>`: custom profile definition directory.
|
- `--profile-dir <dir>`: custom profile definition directory.
|
||||||
- `--schema-dir <dir>`: schema base directory for `json_schema` validation.
|
- `--schema-dir <dir>`: schema base directory for `json_schema` validation.
|
||||||
- `--artifact-root <dir>`: base directory for HTTP `file` input references.
|
- `--artifact-root <dir>`: base directory for HTTP `file` input references.
|
||||||
|
- `--max-request-bytes <n>`: maximum HTTP request body bytes; `0` disables the limit.
|
||||||
|
- `--max-artifact-bytes <n>`: maximum HTTP file artifact bytes; `0` disables the limit.
|
||||||
|
- `--max-response-bytes <n>`: maximum encoded HTTP response body bytes; `0` disables the limit.
|
||||||
|
|
||||||
Notes:
|
Notes:
|
||||||
|
|
||||||
- `serve` does not accept runtime model override flags such as `--model` or `--llm-base-url`.
|
- `serve` does not accept runtime model override flags such as `--model` or `--llm-base-url`.
|
||||||
- HTTP `file` input references are rejected unless an artifact root is configured through `server.artifact_root` or `--artifact-root`.
|
- HTTP request fields and error codes are documented in the [HTTP API reference](api.md).
|
||||||
- `--artifact-root` affects only `serve`; `run` and `render` file input paths are unchanged.
|
- HTTP `file` input references are rejected unless an artifact root is configured.
|
||||||
|
- HTTP size-limit flags affect only `serve`.
|
||||||
|
|
||||||
## Input And Variable Syntax
|
## Input And Variable Syntax
|
||||||
|
|
||||||
- `--input name=path` maps prompt input names to local file paths.
|
- `--input name=path` maps prompt input names to local file paths.
|
||||||
- `--var name=value` maps template variable names to values.
|
- `--var name=value` maps prompt template variables to string values.
|
||||||
- If a prompt defines `session_id: "{{ .session_id }}"`, pass the OpenRouter sticky-routing value with `--var session_id=<value>`.
|
|
||||||
- Both flags can be repeated.
|
- Both flags can be repeated.
|
||||||
- Both flags also support comma-separated batches, for example:
|
- Both flags also accept comma-separated mappings, such as `--input transcript=./t.md,glossary=./g.yml`.
|
||||||
- `--input transcript=./t.md,glossary=./g.yml`
|
- Values may contain `=` after the first separator, such as `--var note=a=b=c`.
|
||||||
- `--var session_id=42,session_date=2026-05-04`
|
- Empty names and empty values are rejected.
|
||||||
|
|
||||||
|
CLI `run` and `render` convert every `--input` mapping to a `file` artifact
|
||||||
|
reference. HTTP also supports `inline` input references; see [HTTP API
|
||||||
|
reference](api.md).
|
||||||
|
|
||||||
## Output Behavior
|
## Output Behavior
|
||||||
|
|
||||||
`run`:
|
`run`:
|
||||||
|
|
||||||
- Writes generated artifact content to stdout by default.
|
- Writes generated artifact content to stdout by default.
|
||||||
- Writes generated artifact content to `--out` when provided.
|
- Writes generated artifact content to `--out` when provided.
|
||||||
- Prints run summary metadata to stderr on success.
|
- Prints a success summary to stderr.
|
||||||
- Appends `cached_tokens=<n> cache_write_tokens=<n>` to the summary only when the provider reports non-zero cache usage.
|
|
||||||
- Prints errors to stderr on failure.
|
- Prints errors to stderr on failure.
|
||||||
|
|
||||||
`render`:
|
`render`:
|
||||||
|
|
||||||
- Writes prepared-run output to stdout by default.
|
- Writes prepared-run output to stdout by default.
|
||||||
- Writes prepared-run output to `--out` when provided.
|
- Writes prepared-run output to `--out` when provided.
|
||||||
- Does not print a success summary line.
|
- Does not print a success summary.
|
||||||
|
|
||||||
`serve`:
|
`serve`:
|
||||||
|
|
||||||
- Logs startup and server errors to stderr.
|
- Logs startup and server errors to stderr.
|
||||||
|
|
||||||
## Exit Codes
|
## Exit Codes
|
||||||
|
|
||||||
- `0`: success.
|
- `0`: success.
|
||||||
- `1`: runtime/parse/config/load/render/generation/output-write error.
|
- `1`: parse, config, load, render, generation, output-write, or runtime error.
|
||||||
- `2`: `run` completed, output was generated, but validation status is `failed`.
|
- `2`: `run` completed and wrote output, but validation status is `failed`.
|
||||||
|
|
||||||
When `run` exits `2`, output may already be written to stdout or `--out`.
|
|
||||||
|
|
||||||
## Common Workflows
|
## Common Workflows
|
||||||
|
|
||||||
Render prompt inputs and template variables as JSON:
|
Render prompt inputs and variables as JSON:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
go run ./cmd/scriptorium render \
|
go run ./cmd/scriptorium render \
|
||||||
@@ -139,7 +210,7 @@ go run ./cmd/scriptorium render \
|
|||||||
--format json
|
--format json
|
||||||
```
|
```
|
||||||
|
|
||||||
Run a prompt with profile override and file output:
|
Run a prompt with an explicit profile and file output:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
go run ./cmd/scriptorium run \
|
go run ./cmd/scriptorium run \
|
||||||
@@ -151,12 +222,12 @@ go run ./cmd/scriptorium run \
|
|||||||
--out ./summary.md
|
--out ./summary.md
|
||||||
```
|
```
|
||||||
|
|
||||||
Start the HTTP server with explicit config:
|
Start the HTTP server with example config:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
go run ./cmd/scriptorium serve --config ./examples/config.yml
|
go run ./cmd/scriptorium serve --config ./examples/config.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
Copyable example script:
|
Copyable maintained script:
|
||||||
|
|
||||||
- `examples/render-markdown-summary.sh`
|
- `examples/render-markdown-summary.sh`
|
||||||
|
|||||||
277
docs/config.md
277
docs/config.md
@@ -2,30 +2,32 @@
|
|||||||
|
|
||||||
## Config Discovery And Precedence
|
## Config Discovery And Precedence
|
||||||
|
|
||||||
Application settings are loaded in this order:
|
Application settings are resolved in this order:
|
||||||
|
|
||||||
1. Built-in defaults
|
1. built-in defaults
|
||||||
2. `config.yml` values
|
2. `config.yml` values
|
||||||
3. CLI overrides
|
3. CLI overrides
|
||||||
|
|
||||||
When `--config` is not provided, Scriptorium searches for config files in this order:
|
When `--config` is omitted, Scriptorium searches:
|
||||||
|
|
||||||
1. `/usr/local/etc/scriptorium/config.yml`
|
1. `/usr/local/etc/scriptorium/config.yml`
|
||||||
2. `/etc/scriptorium/config.yml`
|
2. `/etc/scriptorium/config.yml`
|
||||||
|
|
||||||
If neither file exists, Scriptorium continues with built-in defaults.
|
If neither file exists, Scriptorium uses built-in defaults. When
|
||||||
|
`--config <path>` is provided, that file must exist and decode successfully.
|
||||||
|
|
||||||
When `--config <path>` is provided, that file is required.
|
## Minimal Working Config
|
||||||
|
|
||||||
## Minimal App Config
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
prompt_dir: ./examples/prompts
|
prompt_dir: ./examples/prompts
|
||||||
```
|
```
|
||||||
|
|
||||||
This is enough to use `run` and `render` when prompts select built-in profiles.
|
This is enough for `run` and `render` when selected prompts use built-in
|
||||||
|
profiles. Set `profile_dir` when prompts or requests use custom profiles.
|
||||||
|
|
||||||
## Production-Oriented App Config
|
The maintained repository example is `examples/config.yml`.
|
||||||
|
|
||||||
|
## Production-Oriented Config
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
prompt_dir: /opt/scriptorium/prompts
|
prompt_dir: /opt/scriptorium/prompts
|
||||||
@@ -35,48 +37,56 @@ schema_dir: /opt/scriptorium/schemas
|
|||||||
server:
|
server:
|
||||||
addr: 127.0.0.1:8080
|
addr: 127.0.0.1:8080
|
||||||
artifact_root: /var/lib/scriptorium/artifacts
|
artifact_root: /var/lib/scriptorium/artifacts
|
||||||
|
max_request_bytes: 16777216
|
||||||
|
max_artifact_bytes: 16777216
|
||||||
|
max_response_bytes: 16777216
|
||||||
|
|
||||||
defaults:
|
defaults:
|
||||||
render_format: text
|
render_format: text
|
||||||
```
|
```
|
||||||
|
|
||||||
## App Config File (`config.yml`)
|
The maintained full example is `examples/config.full.yml`.
|
||||||
|
|
||||||
|
## App Config Reference
|
||||||
|
|
||||||
Top-level fields:
|
Top-level fields:
|
||||||
|
|
||||||
- `prompt_dir` (optional): default prompt definition directory.
|
| Field | Default | Description |
|
||||||
- `profile_dir` (optional): default custom profile definition directory.
|
| --- | --- | --- |
|
||||||
- `schema_dir` (optional): base directory for schema files used by `json_schema` validation.
|
| `prompt_dir` | unset | Directory containing prompt definition YAML files. Required effectively by `run`, `render`, and `serve`. |
|
||||||
- `server.addr` (optional): default listen address for `serve`.
|
| `profile_dir` | unset | Directory containing custom profile YAML files. Built-in profiles remain available when unset. |
|
||||||
- `server.artifact_root` (optional): base directory for HTTP `file` input references.
|
| `schema_dir` | `.` | Base directory for relative JSON Schema paths. |
|
||||||
- `defaults.render_format` (optional): default `render` output format (`text` or `json`).
|
| `server` | `{}` | HTTP service settings used by `serve`. |
|
||||||
|
| `defaults` | `{}` | Adapter defaults. |
|
||||||
|
|
||||||
Built-in defaults:
|
`server` fields:
|
||||||
|
|
||||||
- `schema_dir`: `.`
|
| Field | Default | Description |
|
||||||
- `server.addr`: `:8080`
|
| --- | --- | --- |
|
||||||
- `server.artifact_root`: unset; HTTP `file` input references are rejected until configured.
|
| `server.addr` | `:8080` | Listen address for `serve`. |
|
||||||
- `defaults.render_format`: `text`
|
| `server.artifact_root` | unset | Base directory for HTTP `file` input references. Without it, HTTP file refs are rejected. |
|
||||||
|
| `server.max_request_bytes` | `16777216` | Maximum encoded HTTP request body bytes. `0` disables the limit. |
|
||||||
|
| `server.max_artifact_bytes` | `16777216` | Maximum HTTP file artifact bytes. `0` disables the limit. |
|
||||||
|
| `server.max_response_bytes` | `16777216` | Maximum encoded HTTP response bytes. `0` disables the limit. |
|
||||||
|
|
||||||
Validation behavior:
|
`defaults` fields:
|
||||||
|
|
||||||
- Config decoding is strict; unknown YAML fields are rejected.
|
| Field | Default | Description |
|
||||||
- Raw API key fields are not supported in `config.yml`.
|
| --- | --- | --- |
|
||||||
|
| `defaults.render_format` | `text` | Default `render` output format: `text` or `json`. |
|
||||||
|
|
||||||
HTTP artifact root behavior:
|
Config rules:
|
||||||
|
|
||||||
- `server.artifact_root` applies only to `serve`.
|
- YAML decoding is strict; unknown fields are rejected.
|
||||||
- HTTP `inline` input references work without an artifact root.
|
- HTTP size limits must be greater than or equal to `0`.
|
||||||
- HTTP `file` input references are resolved against `server.artifact_root` and must stay inside it.
|
- Empty string config values are ignored.
|
||||||
- Relative traversal and absolute paths outside the root are rejected.
|
- Raw API key fields are not supported in app config.
|
||||||
- Symlinks inside the root are followed by the operating system; do not make the artifact root writable by untrusted users.
|
|
||||||
- CLI `run` and `render` file inputs keep their normal direct filesystem path behavior.
|
|
||||||
|
|
||||||
## Prompt Definition Files
|
## Prompt Definition Files
|
||||||
|
|
||||||
Prompt definitions are YAML files anywhere under `prompt_dir`, including nested subdirectories.
|
Prompt definitions are YAML files anywhere under `prompt_dir`. Nested
|
||||||
|
directories are organizational; callers select prompts by YAML `id`, not file
|
||||||
Subdirectories are organizational only. Callers still select prompts by the YAML `id`, not by file path. For example, `prompts/dnd/recap.yaml` may still declare `id: dnd.recap`, and callers use `--prompt dnd.recap`.
|
path.
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
|
|
||||||
@@ -109,84 +119,76 @@ output:
|
|||||||
repair_attempts: 0
|
repair_attempts: 0
|
||||||
```
|
```
|
||||||
|
|
||||||
Field reference:
|
Prompt fields:
|
||||||
|
|
||||||
- `id` (required): prompt identifier.
|
| Field | Required | Description |
|
||||||
- `version` (required): prompt version.
|
| --- | --- | --- |
|
||||||
- `default_profile` (optional): profile ID used when request does not provide `profile_id`.
|
| `id` | yes | Prompt identifier used by `--prompt` and HTTP `prompt_id`. |
|
||||||
- `description` (optional): prompt description.
|
| `version` | yes | Prompt version. |
|
||||||
- `session_id` (optional): Go-template string for OpenRouter sticky-routing `session_id`; rendered from request vars.
|
| `default_profile` | no | Profile ID used when a request does not provide a profile. |
|
||||||
- `inputs` (optional list): expected named inputs.
|
| `description` | no | Human-readable description. |
|
||||||
- `messages` (required list): prompt message templates.
|
| `session_id` | no | Go-template string rendered from request vars and forwarded as provider `session_id` when non-empty. |
|
||||||
- `output` (required object): output contract.
|
| `inputs` | no | Named input declarations. |
|
||||||
|
| `messages` | yes | Chat message templates. |
|
||||||
|
| `output` | yes | Output format and validation contract. |
|
||||||
|
|
||||||
`inputs[]` fields:
|
`inputs[]` fields:
|
||||||
|
|
||||||
- `name` (required)
|
- `name` (required)
|
||||||
- `required` (optional, boolean)
|
- `required` (optional boolean)
|
||||||
- `content_type` (optional metadata)
|
- `content_type` (optional metadata)
|
||||||
- `description` (optional)
|
- `description` (optional)
|
||||||
|
|
||||||
`messages[]` fields:
|
`messages[]` fields:
|
||||||
|
|
||||||
- `role` (required)
|
- `role` (required)
|
||||||
- `content` or `content_file` (exactly one is required)
|
- exactly one of `content` or `content_file`
|
||||||
- `cache_control` (optional object): provider prompt-cache metadata for this message
|
- `cache_control` (optional)
|
||||||
|
|
||||||
Message rules:
|
Message rules:
|
||||||
|
|
||||||
|
- `content_file` resolves relative to the prompt YAML file location.
|
||||||
- Repeated roles are allowed.
|
- Repeated roles are allowed.
|
||||||
- `content_file` is resolved relative to the prompt YAML file location.
|
- Prompt YAML decoding is strict.
|
||||||
- Nested prompt files keep the same relative `content_file` behavior; `./recap.user.md` next to `dnd/recap.yaml` resolves from `dnd/`.
|
- Duplicate input names are invalid.
|
||||||
- Prompt decoding is strict; unknown YAML fields are rejected.
|
- Duplicate prompt IDs are invalid for a requested ID/version.
|
||||||
- Duplicate prompt IDs are invalid. If multiple files declare the requested prompt ID, Scriptorium fails instead of choosing one.
|
|
||||||
|
|
||||||
`messages[].cache_control` fields:
|
`messages[].cache_control` fields:
|
||||||
|
|
||||||
- `type` (required when `cache_control` is present): currently only `ephemeral`.
|
| Field | Required | Supported values |
|
||||||
- `ttl` (optional): currently only `1h`; omitted from outbound requests when unset.
|
| --- | --- | --- |
|
||||||
|
| `type` | yes | `ephemeral` |
|
||||||
|
| `ttl` | no | `1h` |
|
||||||
|
|
||||||
Example cache-controlled message:
|
`session_id` behavior:
|
||||||
|
|
||||||
```yaml
|
- Rendered with the same variable context as message templates.
|
||||||
messages:
|
- Trimmed and omitted when empty.
|
||||||
- role: system
|
- Rejected when longer than 256 Unicode code points.
|
||||||
content_file: ./stable_context.md
|
- CLI callers pass variables with `--var`; HTTP callers use `vars`.
|
||||||
cache_control:
|
|
||||||
type: ephemeral
|
|
||||||
ttl: 1h
|
|
||||||
- role: user
|
|
||||||
content: |
|
|
||||||
{{input "transcript"}}
|
|
||||||
```
|
|
||||||
|
|
||||||
Use cache control on stable reusable prompt content. Dynamic per-run inputs before the cache-controlled message change the provider cache key.
|
|
||||||
|
|
||||||
Example prompt-level session ID:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
session_id: "{{ .session_id }}"
|
|
||||||
```
|
|
||||||
|
|
||||||
When configured, `session_id` is rendered with the same variable context as messages. The rendered value is trimmed, omitted when empty, and rejected if longer than 256 characters. CLI callers pass the value through `--var session_id=<value>`; HTTP callers pass it through `"vars": {"session_id": "<value>"}`.
|
|
||||||
|
|
||||||
`output` fields:
|
`output` fields:
|
||||||
|
|
||||||
- `format` (required): `text`, `markdown`, or `json`.
|
| Field | Required | Supported values |
|
||||||
- `validation_mode` (required): `none`, `basic`, `json`, or `json_schema`.
|
| --- | --- | --- |
|
||||||
- `schema_path` (required when `validation_mode: json_schema`).
|
| `format` | yes | `text`, `markdown`, `json` |
|
||||||
- `repair_attempts` (required): integer `>= 0`.
|
| `validation_mode` | yes | `none`, `basic`, `json`, `json_schema` |
|
||||||
|
| `schema_path` | only for `json_schema` | Relative to `schema_dir` unless absolute. |
|
||||||
|
| `repair_attempts` | yes | Integer greater than or equal to `0`. |
|
||||||
|
|
||||||
Repair behavior boundary:
|
Repair boundary:
|
||||||
|
|
||||||
- `repair_attempts` is part of the prompt contract.
|
- `repair_attempts` is part of the prompt contract.
|
||||||
- CLI and HTTP currently construct the runner without a repairer, so normal `run`/`serve` execution does not perform output repair attempts.
|
- The current CLI and HTTP wiring constructs the runner without a repairer, so normal `run` and `serve` execution does not perform repair attempts.
|
||||||
|
|
||||||
## Profile Definition Files
|
## Profile Definition Files
|
||||||
|
|
||||||
Scriptorium includes built-in execution profiles. Custom execution profiles are YAML files anywhere under `profile_dir`, including nested subdirectories.
|
Execution profiles are YAML files anywhere under `profile_dir`. Nested
|
||||||
|
directories are organizational; callers select profiles by YAML `id`, not file
|
||||||
|
path.
|
||||||
|
|
||||||
Subdirectories are organizational only. Callers still select profiles by the YAML `id`, not by file path. For example, `profiles/local/local-quality.yaml` may still declare `id: local-quality`, and callers use `--profile local-quality`.
|
Scriptorium also ships built-in profiles. Custom profiles override built-ins
|
||||||
|
with the same ID.
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
|
|
||||||
@@ -203,36 +205,44 @@ service_tier: priority
|
|||||||
reasoning_effort: medium
|
reasoning_effort: medium
|
||||||
extra_params:
|
extra_params:
|
||||||
provider_route: primary
|
provider_route: primary
|
||||||
provider_options:
|
|
||||||
retry_budget: 2
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Field reference:
|
Profile fields:
|
||||||
|
|
||||||
- `id` (required)
|
| Field | Required | Description |
|
||||||
- `endpoint` (required)
|
| --- | --- | --- |
|
||||||
- `model` (required)
|
| `id` | yes | Profile identifier. |
|
||||||
- `temperature` (optional): range `0..2`
|
| `endpoint` | yes | OpenAI-compatible base URL including `/v1`. |
|
||||||
- `max_tokens` (optional): `>= 0`
|
| `model` | yes | Provider model name. |
|
||||||
- `top_p` (optional): range `0..1`
|
| `temperature` | no | Range `0..2`. |
|
||||||
- `timeout_seconds` (optional): `>= 0`
|
| `max_tokens` | no | Integer greater than or equal to `0`. |
|
||||||
- `service_tier` (optional): provider-specific request tier such as OpenRouter `flex` or `priority`
|
| `top_p` | no | Range `0..1`. |
|
||||||
- `reasoning_effort` (optional): serialized as top-level `reasoning_effort` in outbound chat-completions requests
|
| `timeout_seconds` | no | Integer greater than or equal to `0`. |
|
||||||
- `api_key_env` (optional)
|
| `service_tier` | no | Provider-specific request tier. |
|
||||||
- `extra_params` (optional map): JSON-compatible provider-specific parameters. Values may be strings, numbers, booleans, objects, or arrays.
|
| `reasoning_effort` | no | Provider-specific reasoning setting. |
|
||||||
|
| `api_key_env` | no | Environment variable name containing the API key. |
|
||||||
|
| `extra_params` | no | JSON-compatible provider-specific top-level request fields. |
|
||||||
|
|
||||||
|
Execution defaults before profile/request overrides:
|
||||||
|
|
||||||
|
| Field | Default |
|
||||||
|
| --- | --- |
|
||||||
|
| `temperature` | `0.0` |
|
||||||
|
| `max_tokens` | `0` |
|
||||||
|
| `top_p` | `1.0` |
|
||||||
|
| `timeout_seconds` | `600` |
|
||||||
|
|
||||||
Profile rules:
|
Profile rules:
|
||||||
|
|
||||||
- `profile_dir` is optional. If omitted, only built-in profiles are available.
|
- Profile YAML decoding is strict.
|
||||||
- If `profile_dir` is set, custom profiles from that directory override built-in profiles with the same `id`.
|
- Duplicate custom profile IDs are invalid.
|
||||||
- Duplicate IDs within the custom profile directory are invalid. Matching IDs across custom and built-in profiles are valid override behavior.
|
- Matching custom and built-in IDs are valid override behavior.
|
||||||
- Profile decoding is strict; unknown YAML fields are rejected.
|
|
||||||
- Raw `api_key` is rejected; use `api_key_env`.
|
- Raw `api_key` is rejected; use `api_key_env`.
|
||||||
- If `api_key_env` is set, that environment variable must be set when preparing/running.
|
- If `api_key_env` is set, the named environment variable must be set before `run`, `render`, or HTTP execution can prepare the request.
|
||||||
- Duplicate profile IDs are invalid. If multiple files declare the requested profile ID, Scriptorium fails instead of choosing one.
|
- Profile numeric fields merge by non-zero value. Request overrides are presence-aware, so explicit zero values are supported through CLI flags or HTTP model overrides.
|
||||||
- `extra_params` keys must not be empty and must not collide with reserved outbound request fields: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or `response_format`.
|
- `extra_params` keys must not be empty and must not collide with reserved outbound fields: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or `response_format`.
|
||||||
|
|
||||||
Built-in profile IDs:
|
Built-in profile catalog:
|
||||||
|
|
||||||
| Provider | ID | Model | API key env |
|
| Provider | ID | Model | API key env |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
@@ -260,56 +270,55 @@ Built-in profile IDs:
|
|||||||
| openai | `gpt-5-mini` | `openai/gpt-5.4-mini` | `OPENROUTER_API_KEY` |
|
| openai | `gpt-5-mini` | `openai/gpt-5.4-mini` | `OPENROUTER_API_KEY` |
|
||||||
| openai | `gpt-5-nano` | `openai/gpt-5.4-nano` | `OPENROUTER_API_KEY` |
|
| openai | `gpt-5-nano` | `openai/gpt-5.4-nano` | `OPENROUTER_API_KEY` |
|
||||||
|
|
||||||
Current outbound request behavior:
|
|
||||||
|
|
||||||
- The OpenAI-compatible client currently serializes: `model`, optional `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, optional `response_format` for `json_schema` prompts, and `extra_params`.
|
|
||||||
- `extra_params` are flattened into provider-specific top-level JSON request fields. They are not wrapped in an `extra_params` object on the outbound provider request.
|
|
||||||
- Messages without `cache_control` serialize with string `content`.
|
|
||||||
- Messages with `cache_control` serialize as a single text content-block array containing `cache_control`.
|
|
||||||
|
|
||||||
## Schema Behavior
|
## Schema Behavior
|
||||||
|
|
||||||
Schemas are JSON files, typically in `schema_dir`.
|
Schemas are JSON files, typically under `schema_dir`.
|
||||||
|
|
||||||
Rules:
|
Rules:
|
||||||
|
|
||||||
- `output.validation_mode: json_schema` requires `output.schema_path`.
|
- `output.validation_mode: json_schema` requires `output.schema_path`.
|
||||||
- Relative `schema_path` values resolve from `schema_dir`, including explicit nested paths such as `dnd/structured_events.schema.json`.
|
- Relative `schema_path` values resolve from `schema_dir`.
|
||||||
- Absolute `schema_path` values are used directly.
|
- Absolute `schema_path` values are used directly.
|
||||||
- Scriptorium does not recursively search schemas by basename; nested schemas must be referenced by their relative path.
|
- Nested schemas must be referenced by relative path; schemas are not searched recursively by basename.
|
||||||
- Missing or invalid schema documents cause runtime validation errors.
|
- Missing or invalid schema documents are runtime validation errors.
|
||||||
- Invalid generated JSON causes validation status `failed` (not a runtime error).
|
- Invalid generated JSON produces validation status `failed`, not a runtime error.
|
||||||
|
|
||||||
Supported artifact reference types for request inputs are `file` and `inline`.
|
## Artifact References
|
||||||
For HTTP `serve`, `file` references require `server.artifact_root` and must stay
|
|
||||||
inside that root. CLI `run` and `render` file inputs are not restricted by
|
Supported request input artifact reference types are:
|
||||||
`server.artifact_root`.
|
|
||||||
|
- `file`
|
||||||
|
- `inline`
|
||||||
|
|
||||||
|
CLI `run` and `render` create `file` references from `--input name=path`.
|
||||||
|
|
||||||
|
HTTP `file` references require `server.artifact_root` or `serve
|
||||||
|
--artifact-root`. Relative file URIs resolve under that root. Absolute paths
|
||||||
|
and relative traversal outside the root are rejected by lexical checks. Symlinks
|
||||||
|
inside the root are followed by the operating system, including symlinks that
|
||||||
|
point outside the root.
|
||||||
|
|
||||||
|
HTTP `inline` references do not require an artifact root.
|
||||||
|
|
||||||
## Secrets Handling
|
## Secrets Handling
|
||||||
|
|
||||||
- Keep secret values in environment variables.
|
- Keep secret values in environment variables.
|
||||||
- Store only environment-variable names in profile `api_key_env`.
|
- Store only environment-variable names in `api_key_env`.
|
||||||
- Do not put raw API keys in config, prompts, profiles, CLI flags, or HTTP request bodies.
|
- Do not put raw API keys in config, prompts, profiles, CLI arguments, examples, or HTTP request bodies.
|
||||||
|
|
||||||
## Maintained Examples
|
## Maintained Examples
|
||||||
|
|
||||||
- App config: `examples/config.yml`
|
- Minimal app config: `examples/config.yml`
|
||||||
|
- Full app config: `examples/config.full.yml`
|
||||||
- Prompt examples: `examples/prompts/`
|
- Prompt examples: `examples/prompts/`
|
||||||
- Custom profile examples: `examples/profiles/`
|
- Custom profile examples: `examples/profiles/`
|
||||||
- Schema examples: `examples/schemas/`
|
- Schema examples: `examples/schemas/`
|
||||||
- Input fixtures: `examples/fixtures/`
|
- Input fixtures: `examples/fixtures/`
|
||||||
- Render example script: `examples/render-markdown-summary.sh`
|
- Render script: `examples/render-markdown-summary.sh`
|
||||||
- HTTP request example: `examples/http-run.json`
|
- HTTP request-shape example: `examples/http-run.json`
|
||||||
|
|
||||||
Example organizational layout:
|
|
||||||
|
|
||||||
```text
|
|
||||||
examples/prompts/dnd/recap.yaml
|
|
||||||
examples/profiles/local/local-quality.yaml
|
|
||||||
examples/schemas/dnd/structured_events.schema.json
|
|
||||||
```
|
|
||||||
|
|
||||||
## Integration References
|
## Integration References
|
||||||
|
|
||||||
- [Inbound HTTP contract](integrations/http-api.md)
|
- [CLI reference](cli.md)
|
||||||
|
- [HTTP API reference](api.md)
|
||||||
- [Outbound OpenAI-compatible contract](integrations/openai-compatible-chat.md)
|
- [Outbound OpenAI-compatible contract](integrations/openai-compatible-chat.md)
|
||||||
|
|||||||
@@ -1,13 +1,122 @@
|
|||||||
# Consumer API Overview
|
# Consumer Integration Overview
|
||||||
|
|
||||||
Scriptorium can be used by consumers through three implemented surfaces:
|
This guide is for applications that call Scriptorium from another codebase.
|
||||||
|
|
||||||
- CLI commands, documented in [CLI reference](../cli.md).
|
Scriptorium exposes three integration surfaces:
|
||||||
- HTTP `POST /v1/runs`, documented in [HTTP API integration](../integrations/http-api.md).
|
|
||||||
- Go package `gitea.maximumdirect.net/eric/scriptorium`, documented in [pkg-scriptorium](pkg-scriptorium.md).
|
|
||||||
|
|
||||||
The Go package is the typed in-process API. It prepares prompts, runs prompts, accepts file or inline artifacts, supports per-request execution overrides, and exposes stable public errors for `errors.Is`.
|
| Surface | Use when |
|
||||||
|
| --- | --- |
|
||||||
|
| Go package | The consumer is Go, needs typed requests/results, or wants injected LLM clients for tests. |
|
||||||
|
| CLI subprocess | The consumer wants process isolation or is not written in Go. |
|
||||||
|
| HTTP API | The consumer needs a service boundary or remote access to `POST /v1/runs`. |
|
||||||
|
|
||||||
Use the Go package when the caller is a Go program that wants typed requests/results, context cancellation, repeated calls without subprocess overhead, or fake LLM injection for tests. Use the CLI or HTTP surfaces when process isolation, language neutrality, or an HTTP boundary is preferred.
|
Canonical references:
|
||||||
|
|
||||||
Raw API key values are not accepted in public payloads and are not returned in prepared or run results. Execution profiles may reference an environment variable name through `api_key_env`.
|
- Go package: [Package scriptorium](pkg-scriptorium.md)
|
||||||
|
- CLI subprocess: [Subprocess integration](../integrations/subprocess.md)
|
||||||
|
- HTTP: [HTTP API reference](../api.md)
|
||||||
|
- File formats: [Configuration reference](../config.md)
|
||||||
|
|
||||||
|
## Required Deployment Inputs
|
||||||
|
|
||||||
|
Every integration needs operators to provide:
|
||||||
|
|
||||||
|
- prompt definitions;
|
||||||
|
- profile definitions or built-in profile IDs;
|
||||||
|
- schema files when prompts use `json_schema`;
|
||||||
|
- input artifacts or inline input bodies;
|
||||||
|
- API-key environment variables or direct per-request keys where supported.
|
||||||
|
|
||||||
|
Raw API keys do not belong in config, prompt files, profile YAML, CLI
|
||||||
|
arguments, or HTTP request bodies.
|
||||||
|
|
||||||
|
## Recommended Workflow
|
||||||
|
|
||||||
|
Use the Go package when:
|
||||||
|
|
||||||
|
- the consumer is a Go application;
|
||||||
|
- the application needs `context.Context` cancellation;
|
||||||
|
- repeated calls should avoid subprocess startup;
|
||||||
|
- tests need a fake LLM client;
|
||||||
|
- direct per-request `RunRequest.APIKey` is required.
|
||||||
|
|
||||||
|
Use the CLI subprocess when:
|
||||||
|
|
||||||
|
- the consumer is not Go;
|
||||||
|
- process isolation is useful;
|
||||||
|
- stdout/stderr separation and exit codes are enough;
|
||||||
|
- the consumer already manages local files and environment variables.
|
||||||
|
|
||||||
|
Use HTTP when:
|
||||||
|
|
||||||
|
- Scriptorium should run as a service;
|
||||||
|
- multiple clients need a shared prompt/profile deployment;
|
||||||
|
- clients can reach a trusted, protected HTTP boundary.
|
||||||
|
|
||||||
|
## Minimal Go Example
|
||||||
|
|
||||||
|
```go
|
||||||
|
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||||
|
PromptDir: "./examples/prompts",
|
||||||
|
ProfileDir: "./examples/profiles",
|
||||||
|
SchemaDir: "./examples/schemas",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{
|
||||||
|
PromptID: "generic.markdown_summary",
|
||||||
|
Inputs: map[string]scriptorium.ArtifactRef{
|
||||||
|
"transcript": scriptorium.File("./examples/fixtures/transcript.md"),
|
||||||
|
"glossary": scriptorium.File("./examples/fixtures/glossary.yml"),
|
||||||
|
},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_ = prepared.Messages
|
||||||
|
```
|
||||||
|
|
||||||
|
Run the maintained package example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go run ./examples/go-library/prepare
|
||||||
|
```
|
||||||
|
|
||||||
|
## Subprocess Workflow
|
||||||
|
|
||||||
|
Invoke `scriptorium render` for preflight and `scriptorium run` for generation.
|
||||||
|
Capture stdout and stderr separately. Treat exit code `2` from `run` as a
|
||||||
|
completed generation with failed validation.
|
||||||
|
|
||||||
|
See [Subprocess integration](../integrations/subprocess.md) for the stable
|
||||||
|
invocation contract.
|
||||||
|
|
||||||
|
## HTTP Workflow
|
||||||
|
|
||||||
|
Run `scriptorium serve` behind trusted controls and send JSON requests to
|
||||||
|
`POST /v1/runs`.
|
||||||
|
|
||||||
|
Do not duplicate endpoint schemas in consumers. Use the [HTTP API
|
||||||
|
reference](../api.md) as the authoritative contract.
|
||||||
|
|
||||||
|
## Consumer Responsibilities
|
||||||
|
|
||||||
|
Consumers are responsible for:
|
||||||
|
|
||||||
|
- selecting prompt/profile IDs as deployment configuration;
|
||||||
|
- supplying all required inputs and vars;
|
||||||
|
- protecting generated artifacts and rendered prompts as sensitive data;
|
||||||
|
- deciding whether to keep output when validation fails;
|
||||||
|
- implementing retries only when another model call is acceptable.
|
||||||
|
|
||||||
|
Scriptorium does not persist run state. Retrying a failed or timed-out request
|
||||||
|
can produce different output and can incur another provider request.
|
||||||
|
|
||||||
|
## Status Behavior
|
||||||
|
|
||||||
|
- Go package methods return typed results or errors that support `errors.Is`.
|
||||||
|
- CLI `run` exits `2` when generation succeeds but validation fails.
|
||||||
|
- HTTP returns `200 OK` for generated-content validation failures and exposes the failed status in the response body.
|
||||||
|
- Runtime validation failures are errors.
|
||||||
|
|||||||
@@ -6,7 +6,22 @@ Import path:
|
|||||||
import "gitea.maximumdirect.net/eric/scriptorium"
|
import "gitea.maximumdirect.net/eric/scriptorium"
|
||||||
```
|
```
|
||||||
|
|
||||||
The root package is a public facade over Scriptorium's prompt execution use case. It keeps `internal/*` packages private while exposing typed construction, preparation, execution, inputs, results, and errors.
|
The root package is the public Go facade for Scriptorium's prompt prepare/run
|
||||||
|
workflow. It exposes typed requests, results, source options, injected LLM
|
||||||
|
clients, and stable public errors while keeping `internal/*` packages private.
|
||||||
|
|
||||||
|
## Intended Use Cases
|
||||||
|
|
||||||
|
Use the package when a Go application needs:
|
||||||
|
|
||||||
|
- in-process prompt preparation or execution;
|
||||||
|
- typed request/result structs;
|
||||||
|
- direct `context.Context` cancellation;
|
||||||
|
- injected/fake LLM clients for tests;
|
||||||
|
- direct per-request `RunRequest.APIKey`.
|
||||||
|
|
||||||
|
Use [Subprocess integration](../integrations/subprocess.md) or the [HTTP API](../api.md)
|
||||||
|
when a process or service boundary is preferred.
|
||||||
|
|
||||||
## Construct An Engine
|
## Construct An Engine
|
||||||
|
|
||||||
@@ -21,21 +36,56 @@ if err != nil {
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
`PromptDir` is required unless an explicit prompt source option is supplied. `ProfileDir` is optional; omit it to use built-in profiles only, or set it to overlay custom profiles above built-ins. `SchemaDir` defaults to the built-in schema directory. `Timeout` and `HTTPClient` configure the default OpenAI-compatible client used by `Run` when no custom LLM client is supplied.
|
`Config` fields:
|
||||||
|
|
||||||
## Asset Sources
|
| Field | Description |
|
||||||
|
| --- | --- |
|
||||||
|
| `PromptDir` | Prompt definition directory. Required unless `WithPromptFS` or `WithPromptFile` is used. |
|
||||||
|
| `ProfileDir` | Optional custom profile directory overlaid above built-in profiles. |
|
||||||
|
| `SchemaDir` | Schema directory. Defaults to `.` when empty. |
|
||||||
|
| `Timeout` | Default timeout for the built-in OpenAI-compatible client. |
|
||||||
|
| `HTTPClient` | Optional HTTP client for the built-in OpenAI-compatible client. |
|
||||||
|
|
||||||
Directory fields on `Config` remain the compatibility path. Explicit source options override the matching directory field:
|
`NewEngine` accepts `nil` options and ignores them. Invalid construction wraps
|
||||||
|
`ErrInvalidConfig`.
|
||||||
|
|
||||||
- `WithPromptFS(fsys, root)` and `WithPromptFile(path)`
|
## Source Options
|
||||||
- `WithProfileFS(fsys, root)` and `WithProfileFile(path)`
|
|
||||||
- `WithSchemaFS(fsys, root)` and `WithSchemaFile(path)`
|
|
||||||
|
|
||||||
Prompt and profile sources load standard Scriptorium YAML with the same strict validation as directory sources. Prompt `content_file` paths resolve relative to the prompt file in the same source. Profile options overlay custom profiles above built-ins. Schema `fs.FS` sources preserve prompt `schema_path` semantics; schema file options expose the file by its base name.
|
Directory fields are the compatibility path. Explicit source options override
|
||||||
|
the matching directory field.
|
||||||
|
|
||||||
|
Prompt sources:
|
||||||
|
|
||||||
|
- `WithPromptFS(fsys, root)`
|
||||||
|
- `WithPromptFile(path)`
|
||||||
|
|
||||||
|
Profile sources:
|
||||||
|
|
||||||
|
- `WithProfileFS(fsys, root)`
|
||||||
|
- `WithProfileFile(path)`
|
||||||
|
- `WithProfiles(profiles...)`
|
||||||
|
|
||||||
|
Schema sources:
|
||||||
|
|
||||||
|
- `WithSchemaFS(fsys, root)`
|
||||||
|
- `WithSchemaFile(path)`
|
||||||
|
|
||||||
|
LLM source:
|
||||||
|
|
||||||
|
- `WithLLMClient(client)`
|
||||||
|
|
||||||
|
Source behavior:
|
||||||
|
|
||||||
|
- Prompt and profile YAML use the same strict rules as directory loading.
|
||||||
|
- Prompt `content_file` values resolve relative to the prompt file.
|
||||||
|
- `fs.FS` roots are containment boundaries for prompt content files and schema paths.
|
||||||
|
- File options expose the selected file by its base name.
|
||||||
|
- Profile source precedence is in-memory profiles, then explicit profile file/FS/directory source, then built-ins.
|
||||||
|
- `WithLLMClient(nil)` returns `ErrInvalidConfig`.
|
||||||
|
|
||||||
## In-Memory Profiles
|
## In-Memory Profiles
|
||||||
|
|
||||||
Use `WithProfiles` when the consuming application already has profile settings in typed Go configuration:
|
Use `WithProfiles` when the application already has typed model settings:
|
||||||
|
|
||||||
```go
|
```go
|
||||||
profile := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
|
profile := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
|
||||||
@@ -48,13 +98,33 @@ profile := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfi
|
|||||||
engine, err := scriptorium.NewEngine(cfg, scriptorium.WithProfiles(profile))
|
engine, err := scriptorium.NewEngine(cfg, scriptorium.WithProfiles(profile))
|
||||||
```
|
```
|
||||||
|
|
||||||
In-memory profiles have highest precedence, followed by configured profile file/FS/directory sources, then built-in profiles. Duplicate IDs in one `WithProfiles` call return `ErrInvalidConfig`.
|
`Profile` and `OpenAICompatibleProfileConfig` include:
|
||||||
|
|
||||||
`Profile` and `OpenAICompatibleProfileConfig` include endpoint, model, numeric defaults, service tier, reasoning effort, `APIKeyRequired`, and JSON-compatible `ExtraParams`. `WithProfiles` validates `ExtraParams` and returns `ErrInvalidConfig` for unsupported values such as functions, channels, non-string map keys, non-finite floats, or cyclic values. Raw API-key fields are not accepted. When `APIKeyRequired` is true, pass the secret with `RunRequest.APIKey`.
|
- `ID`
|
||||||
|
- `Endpoint`
|
||||||
|
- `Model`
|
||||||
|
- `Temperature`
|
||||||
|
- `MaxTokens`
|
||||||
|
- `TopP`
|
||||||
|
- `TimeoutSeconds`
|
||||||
|
- `ServiceTier`
|
||||||
|
- `ReasoningEffort`
|
||||||
|
- `APIKeyRequired`
|
||||||
|
- `ExtraParams`
|
||||||
|
|
||||||
## Prepare A Prompt
|
`WithProfiles` rejects duplicate IDs in one call. In-memory profiles do not
|
||||||
|
store raw keys. When `APIKeyRequired` is true, pass the secret on each request
|
||||||
|
with `RunRequest.APIKey`.
|
||||||
|
|
||||||
`Prepare` resolves the prompt definition, profile, inputs, variables, output contract, structured-output metadata, and rendered messages without calling an LLM.
|
`ExtraParams` must be JSON-compatible: strings, booleans, finite numbers,
|
||||||
|
objects with string keys, arrays/slices, and nil. Unsupported values, non-string
|
||||||
|
map keys, non-finite floats, and cycles return `ErrInvalidConfig` for profiles
|
||||||
|
or `ErrInvalidRequest` for request overrides.
|
||||||
|
|
||||||
|
## Prepare Workflow
|
||||||
|
|
||||||
|
`Prepare` resolves prompt/profile/input/schema state and renders messages
|
||||||
|
without calling an LLM.
|
||||||
|
|
||||||
```go
|
```go
|
||||||
prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{
|
prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{
|
||||||
@@ -67,18 +137,19 @@ prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
_ = prepared.Messages
|
_ = prepared.EffectiveModelParams
|
||||||
```
|
```
|
||||||
|
|
||||||
Input helpers:
|
`PreparedRun` includes prompt ID/version/hash, selected profile, effective
|
||||||
|
model params, output contract, structured-output metadata, input hashes,
|
||||||
|
rendered prompt hash, rendered messages, and timing fields. It does not include
|
||||||
|
raw API-key values, model output, validation results, or internal target
|
||||||
|
presence metadata.
|
||||||
|
|
||||||
- `scriptorium.File(path)` loads an input artifact from a file.
|
## Run Workflow
|
||||||
- `scriptorium.Inline(body)` passes inline input content.
|
|
||||||
- `scriptorium.InlineWithURI(uri, body)` passes inline content with URI metadata.
|
|
||||||
|
|
||||||
## Run A Prompt
|
`Run` calls `Prepare`, invokes the configured LLM client, builds the output
|
||||||
|
artifact, and validates the output.
|
||||||
`Run` prepares the prompt, calls the configured LLM client, builds the output artifact, and validates the output.
|
|
||||||
|
|
||||||
```go
|
```go
|
||||||
result, err := engine.Run(ctx, scriptorium.RunRequest{
|
result, err := engine.Run(ctx, scriptorium.RunRequest{
|
||||||
@@ -95,13 +166,25 @@ if err != nil {
|
|||||||
_ = result.Artifact
|
_ = result.Artifact
|
||||||
```
|
```
|
||||||
|
|
||||||
`RunResult` includes the run ID, output artifact, raw output, validation result, prompt/profile/model metadata, effective model parameters, input hashes, token/cache usage, and timing fields. Validation content failures return a successful `RunResult` with failed validation status. Runtime validation errors return `ErrValidation`.
|
`RunResult` includes run ID, output artifact, raw output, validation result,
|
||||||
|
prompt/profile/model metadata, effective model params, input hashes, usage, and
|
||||||
|
timing fields.
|
||||||
|
|
||||||
For the public Go API, pass provider credentials with `RunRequest.APIKey`. The value is request-scoped, uses `json:"-"`, is preferred over profile `api_key_env` by the default OpenAI-compatible client, and is not included in `PreparedRun` or `RunResult` JSON. Normal Go string formatting of `RunRequest` reports only whether a direct key is set. Do not store raw keys in config, prompt files, or profile YAML.
|
Generated-content validation failures return a successful `RunResult` with
|
||||||
|
`Validation.Status == ValidationFailed`. Runtime/schema validation errors
|
||||||
|
return an error that matches `ErrValidation`.
|
||||||
|
|
||||||
Avoid logging raw request structs with reflection-based debug dumpers; exported fields remain visible to tools that bypass `String` and `GoString` methods.
|
## Inputs
|
||||||
|
|
||||||
## Inject An LLM Client
|
Input helpers:
|
||||||
|
|
||||||
|
- `File(path)`: file-backed artifact reference.
|
||||||
|
- `Inline(body)`: inline artifact body.
|
||||||
|
- `InlineWithURI(uri, body)`: inline artifact body with URI metadata.
|
||||||
|
|
||||||
|
Input map keys must match the prompt's expected input names.
|
||||||
|
|
||||||
|
## Injected LLM Clients
|
||||||
|
|
||||||
Use `WithLLMClient` for tests or custom model integrations:
|
Use `WithLLMClient` for tests or custom model integrations:
|
||||||
|
|
||||||
@@ -118,11 +201,34 @@ func (fakeLLM) Generate(ctx context.Context, req scriptorium.GenerateRequest) (*
|
|||||||
engine, err := scriptorium.NewEngine(cfg, scriptorium.WithLLMClient(fakeLLM{}))
|
engine, err := scriptorium.NewEngine(cfg, scriptorium.WithLLMClient(fakeLLM{}))
|
||||||
```
|
```
|
||||||
|
|
||||||
The injected client receives the rendered prompt, effective execution target, target presence metadata for explicit numeric overrides, structured-output spec, and request API key when provided. `GenerateRequest.APIKey` also uses `json:"-"`, and normal Go string formatting reports only whether a direct key is set. Custom and fake clients should avoid logging or serializing it. `WithLLMClient(nil)` returns `ErrInvalidConfig`.
|
Injected clients receive:
|
||||||
|
|
||||||
## Request Overrides
|
- rendered prompt;
|
||||||
|
- effective execution target;
|
||||||
|
- numeric target presence metadata;
|
||||||
|
- structured-output spec when applicable;
|
||||||
|
- direct request API key when provided.
|
||||||
|
|
||||||
`RunRequest.Execution` accepts per-request overrides. Numeric override fields are pointers so explicit zero values are preserved:
|
Custom clients should not log raw prompts or API keys by default.
|
||||||
|
|
||||||
|
## Overrides And API Keys
|
||||||
|
|
||||||
|
`RunRequest` fields:
|
||||||
|
|
||||||
|
| Field | Description |
|
||||||
|
| --- | --- |
|
||||||
|
| `PromptID` | Prompt ID. |
|
||||||
|
| `PromptVersion` | Optional prompt version filter. |
|
||||||
|
| `ProfileID` | Optional profile override. |
|
||||||
|
| `APIKey` | Direct per-request API key. |
|
||||||
|
| `Inputs` | Input artifact references. |
|
||||||
|
| `Vars` | Template variables. |
|
||||||
|
| `Execution` | Per-request model overrides. |
|
||||||
|
| `Validation` | Per-request output contract override. |
|
||||||
|
| `Metadata` | Request metadata reserved for callers. |
|
||||||
|
|
||||||
|
`RunRequest.Execution` uses pointer fields for numeric values so explicit zero
|
||||||
|
overrides are preserved:
|
||||||
|
|
||||||
```go
|
```go
|
||||||
zero := 0
|
zero := 0
|
||||||
@@ -131,11 +237,19 @@ req.Execution = &scriptorium.ExecutionTargetOverride{
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
`ExecutionTargetOverride.ExtraParams` accepts JSON-compatible values and copies typed maps/slices so later caller mutation does not affect the run. Unsupported values, non-string map keys, non-finite floats, and cycles return `ErrInvalidRequest`.
|
Direct `RunRequest.APIKey` takes precedence over profile `api_key_env` for the
|
||||||
|
default OpenAI-compatible client. It is request-scoped, uses `json:"-"`, and is
|
||||||
|
not included in `PreparedRun` or `RunResult` JSON. Normal Go string formatting
|
||||||
|
of `RunRequest` and `GenerateRequest` reports only whether a direct key is set.
|
||||||
|
|
||||||
|
Raw API keys do not belong in profile YAML, in-memory profiles, or app config.
|
||||||
|
Avoid reflection-based debug dumps of request structs because exported fields
|
||||||
|
remain visible to tools that bypass `String` and `GoString`.
|
||||||
|
|
||||||
## Errors
|
## Errors
|
||||||
|
|
||||||
Public methods wrap context while preserving stable sentinel checks with `errors.Is`:
|
Public methods wrap context while preserving stable sentinel checks with
|
||||||
|
`errors.Is`:
|
||||||
|
|
||||||
- `ErrInvalidConfig`
|
- `ErrInvalidConfig`
|
||||||
- `ErrInvalidRequest`
|
- `ErrInvalidRequest`
|
||||||
@@ -158,8 +272,13 @@ if errors.Is(err, scriptorium.ErrPromptNotFound) {
|
|||||||
|
|
||||||
## Examples
|
## Examples
|
||||||
|
|
||||||
Run the prepare-only example from the repository root:
|
Run the maintained prepare-only example from the repository root:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
go run ./examples/go-library/prepare
|
go run ./examples/go-library/prepare
|
||||||
```
|
```
|
||||||
|
|
||||||
|
See also:
|
||||||
|
|
||||||
|
- [Configuration reference](../config.md)
|
||||||
|
- [Consumer integration overview](api.md)
|
||||||
|
|||||||
@@ -1,217 +0,0 @@
|
|||||||
# HTTP API Integration
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
This document defines the implemented inbound HTTP contract for Scriptorium.
|
|
||||||
|
|
||||||
Current scope is only:
|
|
||||||
|
|
||||||
- `POST /v1/runs`
|
|
||||||
|
|
||||||
For CLI behavior, see the [CLI reference](../cli.md).
|
|
||||||
|
|
||||||
## Endpoint
|
|
||||||
|
|
||||||
- Method: `POST`
|
|
||||||
- Path: `/v1/runs`
|
|
||||||
- Content type: JSON request/response
|
|
||||||
|
|
||||||
Route behavior:
|
|
||||||
|
|
||||||
- unknown path: `404 not_found`
|
|
||||||
- unsupported method on `/v1/runs`: `405 method_not_allowed`
|
|
||||||
|
|
||||||
Copyable request example file:
|
|
||||||
|
|
||||||
- `examples/http-run.json`
|
|
||||||
|
|
||||||
## Request Body
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"prompt_id": "generic.structured_events",
|
|
||||||
"profile_id": "local-quality",
|
|
||||||
"prompt_version": "1.0.0",
|
|
||||||
"inputs": {
|
|
||||||
"transcript": {"type": "file", "uri": "./examples/fixtures/transcript.md"},
|
|
||||||
"glossary": {"type": "inline", "body": "party:\n - Rin"}
|
|
||||||
},
|
|
||||||
"vars": {
|
|
||||||
"session_date": "2026-05-04"
|
|
||||||
},
|
|
||||||
"model": {
|
|
||||||
"endpoint": "http://localhost:8000/v1",
|
|
||||||
"model": "gpt-4o-mini",
|
|
||||||
"temperature": 0.0,
|
|
||||||
"max_tokens": 800,
|
|
||||||
"top_p": 1.0,
|
|
||||||
"timeout_seconds": 120,
|
|
||||||
"service_tier": "priority",
|
|
||||||
"reasoning_effort": "medium",
|
|
||||||
"api_key_env": "SCRIPTORIUM_API_KEY",
|
|
||||||
"extra_params": {
|
|
||||||
"route": "primary",
|
|
||||||
"provider_options": {
|
|
||||||
"retry_budget": 2
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"include_raw_output": false
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Required fields:
|
|
||||||
|
|
||||||
- `prompt_id`
|
|
||||||
- `inputs` (must contain at least one named input)
|
|
||||||
|
|
||||||
Input reference types currently supported by runtime artifact loading:
|
|
||||||
|
|
||||||
- `file`
|
|
||||||
- `inline`
|
|
||||||
|
|
||||||
HTTP `file` references require `server.artifact_root` or `serve --artifact-root`.
|
|
||||||
Relative file URIs resolve inside that root. Absolute file URIs are accepted
|
|
||||||
only when they remain inside the root. Requests that escape the root, including
|
|
||||||
`..` traversal and absolute paths outside the root, return
|
|
||||||
`400 artifact_not_allowed`. `inline` references do not require an artifact root.
|
|
||||||
|
|
||||||
Model override notes:
|
|
||||||
|
|
||||||
- Numeric model override fields distinguish omitted values from explicit zero values. For example, omitting `temperature` preserves the selected profile/default value, while `"temperature": 0` explicitly sets the effective temperature to zero.
|
|
||||||
- `extra_params` accepts JSON-compatible values: strings, numbers, booleans, objects, and arrays.
|
|
||||||
- `extra_params` are passed through effective model metadata and flattened into top-level provider request fields by the OpenAI-compatible client.
|
|
||||||
- `extra_params` keys must not be empty and must not collide with reserved outbound fields: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or `response_format`.
|
|
||||||
- Raw API-key values are not accepted. Use `api_key_env` to name an environment variable.
|
|
||||||
|
|
||||||
## Strict JSON Rules
|
|
||||||
|
|
||||||
Request decoding uses strict JSON field checks:
|
|
||||||
|
|
||||||
- unknown request fields are rejected with `400 invalid_json`
|
|
||||||
- unknown `model` fields are rejected with `400 invalid_json`
|
|
||||||
- raw API-key payload fields such as `api_key` are rejected as unknown fields
|
|
||||||
|
|
||||||
## Success Response
|
|
||||||
|
|
||||||
Status: `200 OK`
|
|
||||||
|
|
||||||
Response shape:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"artifact": {
|
|
||||||
"name": "output",
|
|
||||||
"content_type": "application/json",
|
|
||||||
"body": "{\"summary\":\"...\"}",
|
|
||||||
"uri": "",
|
|
||||||
"size": 123,
|
|
||||||
"hash": "..."
|
|
||||||
},
|
|
||||||
"validation": {
|
|
||||||
"status": "passed",
|
|
||||||
"mode": "json_schema",
|
|
||||||
"errors": [],
|
|
||||||
"schema_path": "structured_events.schema.json",
|
|
||||||
"repair_attempts": 0,
|
|
||||||
"is_valid": true
|
|
||||||
},
|
|
||||||
"metadata": {
|
|
||||||
"run_id": "...",
|
|
||||||
"prompt_id": "generic.structured_events",
|
|
||||||
"prompt_version": "1.0.0",
|
|
||||||
"prompt_hash": "...",
|
|
||||||
"rendered_prompt_hash": "...",
|
|
||||||
"selected_profile_id": "local-quality",
|
|
||||||
"model_name": "gpt-4o-mini",
|
|
||||||
"endpoint": "http://localhost:8000/v1",
|
|
||||||
"model_params": {
|
|
||||||
"endpoint": "http://localhost:8000/v1",
|
|
||||||
"model": "gpt-4o-mini",
|
|
||||||
"temperature": 0,
|
|
||||||
"max_tokens": 800,
|
|
||||||
"top_p": 1,
|
|
||||||
"timeout_seconds": 120,
|
|
||||||
"service_tier": "priority",
|
|
||||||
"reasoning_effort": "medium",
|
|
||||||
"api_key_env": "SCRIPTORIUM_API_KEY",
|
|
||||||
"extra_params": {
|
|
||||||
"route": "primary",
|
|
||||||
"provider_options": {
|
|
||||||
"retry_budget": 2
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"input_hashes": {
|
|
||||||
"transcript": "..."
|
|
||||||
},
|
|
||||||
"usage": {
|
|
||||||
"prompt_tokens": 11,
|
|
||||||
"completion_tokens": 22,
|
|
||||||
"total_tokens": 33,
|
|
||||||
"cached_tokens": 0,
|
|
||||||
"cache_write_tokens": 0
|
|
||||||
},
|
|
||||||
"start_time": "2026-05-04T12:00:00Z",
|
|
||||||
"end_time": "2026-05-04T12:00:01Z",
|
|
||||||
"duration_ms": 1000,
|
|
||||||
"validation_mode": "json_schema",
|
|
||||||
"validation_status": "passed",
|
|
||||||
"repair_attempts_used": 0
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
`raw_model_output` is omitted by default.
|
|
||||||
|
|
||||||
`metadata.usage.cached_tokens` and `metadata.usage.cache_write_tokens` are always present as numbers. They are `0` when the provider omits compatible cache usage fields or reports no cache activity.
|
|
||||||
|
|
||||||
To include it, send:
|
|
||||||
|
|
||||||
- `"include_raw_output": true`
|
|
||||||
|
|
||||||
## Validation Failure Behavior
|
|
||||||
|
|
||||||
Validation content failures do not map to HTTP error status.
|
|
||||||
|
|
||||||
Behavior:
|
|
||||||
|
|
||||||
- status remains `200 OK`
|
|
||||||
- `validation.status` is `failed`
|
|
||||||
- validation errors are returned in `validation.errors`
|
|
||||||
|
|
||||||
## Error Responses
|
|
||||||
|
|
||||||
Error body shape:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"error": {
|
|
||||||
"code": "invalid_request",
|
|
||||||
"message": "prompt_id is required"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Current error mapping (non-exhaustive):
|
|
||||||
|
|
||||||
- `400 invalid_json`: malformed JSON or unknown JSON fields
|
|
||||||
- `400 invalid_request`: missing/invalid request fields
|
|
||||||
- `400 profile_required`: no explicit `profile_id` and prompt has no `default_profile`
|
|
||||||
- `400 prompt_load_failed`: prompt definition invalid/unloadable
|
|
||||||
- `400 profile_load_failed`: profile invalid/unloadable
|
|
||||||
- `400 artifact_not_allowed`: file input artifact is outside the configured artifact root or file refs are not enabled
|
|
||||||
- `400 artifact_read_failed`: input artifact loading failed
|
|
||||||
- `400 prompt_render_failed`: template render failed
|
|
||||||
- `400 api_key_env_missing`: named API-key environment variable is missing
|
|
||||||
- `404 prompt_not_found`
|
|
||||||
- `404 profile_not_found`
|
|
||||||
- `502 llm_failed`: outbound model request failed
|
|
||||||
- `500 validation_runtime_failed`: validator runtime/schema-load failure
|
|
||||||
- `500 internal_error`
|
|
||||||
|
|
||||||
## Security And Deployment Note
|
|
||||||
|
|
||||||
The HTTP adapter has no built-in authentication or authorization.
|
|
||||||
|
|
||||||
Deploy behind trusted controls (for example authenticated gateway/reverse proxy and network boundaries).
|
|
||||||
@@ -1,114 +0,0 @@
|
|||||||
# Narratio Subprocess Integration
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
This document defines the supported subprocess contract for Narratio invoking Scriptorium through the public CLI.
|
|
||||||
|
|
||||||
This is a CLI contract, not an internal Go package integration.
|
|
||||||
|
|
||||||
## Supported Commands
|
|
||||||
|
|
||||||
Narratio should invoke:
|
|
||||||
|
|
||||||
- `scriptorium run`
|
|
||||||
- `scriptorium render`
|
|
||||||
|
|
||||||
Use `run` for generation.
|
|
||||||
|
|
||||||
Use `render` for preflight/debug output without LLM execution.
|
|
||||||
|
|
||||||
## Recommended Invocation Shapes
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
scriptorium run \
|
|
||||||
--prompt <prompt_id> \
|
|
||||||
--input transcript=<path> \
|
|
||||||
--out <artifact_path>
|
|
||||||
```
|
|
||||||
|
|
||||||
Render:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
scriptorium render \
|
|
||||||
--prompt <prompt_id> \
|
|
||||||
--input transcript=<path> \
|
|
||||||
--format json
|
|
||||||
```
|
|
||||||
|
|
||||||
Narratio may add:
|
|
||||||
|
|
||||||
- `--config <path>`
|
|
||||||
- `--profile <profile_id>`
|
|
||||||
- repeatable `--input name=path`
|
|
||||||
- repeatable `--var name=value`
|
|
||||||
- runtime overrides when explicitly needed (`--model`, `--llm-base-url`, `--timeout`, etc.)
|
|
||||||
|
|
||||||
## Config And Directory Behavior
|
|
||||||
|
|
||||||
Narratio can rely on resolved app config or pass explicit paths.
|
|
||||||
|
|
||||||
- default config search order:
|
|
||||||
1. `/usr/local/etc/scriptorium/config.yml`
|
|
||||||
2. `/etc/scriptorium/config.yml`
|
|
||||||
- explicit `--config` requires file existence and valid syntax
|
|
||||||
- CLI flags override config values
|
|
||||||
|
|
||||||
## Profile Selection
|
|
||||||
|
|
||||||
Profile selection follows runner behavior:
|
|
||||||
|
|
||||||
1. explicit `--profile`
|
|
||||||
2. prompt `default_profile`
|
|
||||||
3. error if neither is available
|
|
||||||
|
|
||||||
Narratio should treat prompt/profile IDs as deployment configuration, not hardcoded logic.
|
|
||||||
|
|
||||||
## Input And Variable Contract
|
|
||||||
|
|
||||||
- Inputs use repeated `--input name=path`.
|
|
||||||
- Input names must match prompt definition input names.
|
|
||||||
- Variables use repeated `--var name=value` for small metadata values.
|
|
||||||
- Prefer file inputs for large content.
|
|
||||||
|
|
||||||
## Environment Contract
|
|
||||||
|
|
||||||
- Pass through required API-key environment variables referenced by `api_key_env`.
|
|
||||||
- Never pass raw API keys via CLI arguments.
|
|
||||||
- Keep subprocess environment scoped to required variables.
|
|
||||||
|
|
||||||
## Output And Error Handling
|
|
||||||
|
|
||||||
`run`:
|
|
||||||
|
|
||||||
- stdout: artifact body unless `--out` is used
|
|
||||||
- `--out`: writes artifact to file
|
|
||||||
- stderr: success summary and errors
|
|
||||||
|
|
||||||
`render`:
|
|
||||||
|
|
||||||
- stdout: prepared-run output unless `--out` is used
|
|
||||||
- stderr: errors
|
|
||||||
|
|
||||||
Narratio should capture stdout and stderr separately.
|
|
||||||
|
|
||||||
## Exit Status Contract
|
|
||||||
|
|
||||||
- `0`: success
|
|
||||||
- `1`: parse/config/load/render/generation/IO/runtime error
|
|
||||||
- `2`: run completed but validation failed
|
|
||||||
|
|
||||||
A `run` exit code `2` can still produce output (stdout or `--out`).
|
|
||||||
|
|
||||||
## Security Notes
|
|
||||||
|
|
||||||
- Treat generated artifacts and stderr logs as potentially sensitive.
|
|
||||||
- Avoid logging full rendered prompts by default in production contexts.
|
|
||||||
- Use controlled output paths and access controls for persisted artifacts.
|
|
||||||
|
|
||||||
## Canonical References
|
|
||||||
|
|
||||||
- CLI behavior: [CLI reference](../cli.md)
|
|
||||||
- Config behavior: [Configuration reference](../config.md)
|
|
||||||
- Operations and failure handling: [Operations guide](../operations.md), [Troubleshooting](../troubleshooting.md)
|
|
||||||
@@ -177,7 +177,7 @@ Malformed responses return `ErrMalformedResponse`.
|
|||||||
## Error Handling
|
## Error Handling
|
||||||
|
|
||||||
- network/request-construction failures: `ErrRequestFailed`
|
- network/request-construction failures: `ErrRequestFailed`
|
||||||
- non-2xx HTTP status: `ErrUnexpectedStatus` (includes status code and trimmed response body snippet)
|
- non-2xx HTTP status: `ErrUnexpectedStatus` (includes status code; provider response bodies are not included)
|
||||||
- malformed response shape/content: `ErrMalformedResponse`
|
- malformed response shape/content: `ErrMalformedResponse`
|
||||||
|
|
||||||
## Unsupported Or Non-Serialized Fields
|
## Unsupported Or Non-Serialized Fields
|
||||||
|
|||||||
130
docs/integrations/subprocess.md
Normal file
130
docs/integrations/subprocess.md
Normal file
@@ -0,0 +1,130 @@
|
|||||||
|
# Subprocess Integration
|
||||||
|
|
||||||
|
This document defines the supported subprocess contract for downstream
|
||||||
|
applications invoking Scriptorium through the public CLI.
|
||||||
|
|
||||||
|
This is a CLI contract. Go callers that want an in-process typed API should use
|
||||||
|
the [package guide](../consumers/pkg-scriptorium.md).
|
||||||
|
|
||||||
|
## Supported Commands
|
||||||
|
|
||||||
|
Downstream applications should invoke:
|
||||||
|
|
||||||
|
- `scriptorium render` for preflight/debug output without LLM execution.
|
||||||
|
- `scriptorium run` for generation.
|
||||||
|
|
||||||
|
`scriptorium serve` is an HTTP service command, not the recommended subprocess
|
||||||
|
contract for per-request execution.
|
||||||
|
|
||||||
|
## Recommended Invocation Shapes
|
||||||
|
|
||||||
|
Render:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
scriptorium render \
|
||||||
|
--config <config_path> \
|
||||||
|
--prompt <prompt_id> \
|
||||||
|
--input transcript=<path> \
|
||||||
|
--format json
|
||||||
|
```
|
||||||
|
|
||||||
|
Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
scriptorium run \
|
||||||
|
--config <config_path> \
|
||||||
|
--prompt <prompt_id> \
|
||||||
|
--input transcript=<path> \
|
||||||
|
--out <artifact_path>
|
||||||
|
```
|
||||||
|
|
||||||
|
Callers may add:
|
||||||
|
|
||||||
|
- `--profile <profile_id>`
|
||||||
|
- repeatable `--input name=path`
|
||||||
|
- repeatable `--var name=value`
|
||||||
|
- runtime overrides when explicitly needed, such as `--model`, `--llm-base-url`, `--api-key-env`, and `--timeout`
|
||||||
|
|
||||||
|
Do not pass raw API keys as command arguments.
|
||||||
|
|
||||||
|
## Config And Directory Behavior
|
||||||
|
|
||||||
|
Callers can rely on resolved app config or pass explicit paths.
|
||||||
|
|
||||||
|
Default config search order:
|
||||||
|
|
||||||
|
1. `/usr/local/etc/scriptorium/config.yml`
|
||||||
|
2. `/etc/scriptorium/config.yml`
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- Explicit `--config` requires file existence and valid syntax.
|
||||||
|
- CLI flags override config values.
|
||||||
|
- `run` and `render` require an effective `prompt_dir`.
|
||||||
|
- `profile_dir` is optional because built-in profiles are available.
|
||||||
|
|
||||||
|
## Profile Selection
|
||||||
|
|
||||||
|
Profile selection follows runner behavior:
|
||||||
|
|
||||||
|
1. explicit `--profile`
|
||||||
|
2. prompt `default_profile`
|
||||||
|
3. error if neither is available
|
||||||
|
|
||||||
|
Treat prompt and profile IDs as deployment configuration, not hardcoded business
|
||||||
|
logic.
|
||||||
|
|
||||||
|
## Input And Variable Contract
|
||||||
|
|
||||||
|
- Inputs use repeated `--input name=path`.
|
||||||
|
- Input names must match prompt definition input names.
|
||||||
|
- Variables use repeated `--var name=value`.
|
||||||
|
- Both flags also accept comma-separated mappings.
|
||||||
|
- Prefer file inputs for large content.
|
||||||
|
|
||||||
|
CLI inputs are file references. HTTP-only `inline` references are documented in
|
||||||
|
the [HTTP API reference](../api.md).
|
||||||
|
|
||||||
|
## Environment Contract
|
||||||
|
|
||||||
|
- Pass through required API-key environment variables referenced by `api_key_env`.
|
||||||
|
- Keep subprocess environments scoped to required variables.
|
||||||
|
- Use `--api-key-env` only to name an environment variable.
|
||||||
|
- Never pass raw API keys via argv.
|
||||||
|
|
||||||
|
## Stdout And Stderr
|
||||||
|
|
||||||
|
`run`:
|
||||||
|
|
||||||
|
- stdout: generated artifact body unless `--out` is used.
|
||||||
|
- stderr: success summary and errors.
|
||||||
|
|
||||||
|
`render`:
|
||||||
|
|
||||||
|
- stdout: prepared-run output unless `--out` is used.
|
||||||
|
- stderr: errors.
|
||||||
|
|
||||||
|
Capture stdout and stderr separately. Do not parse stderr as a stable data
|
||||||
|
format beyond exit status handling.
|
||||||
|
|
||||||
|
## Exit Status Contract
|
||||||
|
|
||||||
|
- `0`: success.
|
||||||
|
- `1`: parse, config, load, render, generation, IO, or runtime error.
|
||||||
|
- `2`: `run` completed and output was written, but validation failed.
|
||||||
|
|
||||||
|
A `run` exit code `2` can still produce output on stdout or at `--out`.
|
||||||
|
Consumers must decide whether to keep or discard that output.
|
||||||
|
|
||||||
|
## Security Notes
|
||||||
|
|
||||||
|
- Treat generated artifacts, rendered prompts, stdout, and stderr as potentially sensitive.
|
||||||
|
- Use controlled output paths and access controls for persisted artifacts.
|
||||||
|
- Avoid logging full rendered prompts or generated artifacts by default.
|
||||||
|
|
||||||
|
## Canonical References
|
||||||
|
|
||||||
|
- CLI behavior: [CLI reference](../cli.md)
|
||||||
|
- Config and file formats: [Configuration reference](../config.md)
|
||||||
|
- Operations: [Operations guide](../operations.md)
|
||||||
|
- Troubleshooting: [Troubleshooting](../troubleshooting.md)
|
||||||
@@ -1,156 +1,81 @@
|
|||||||
# Adapter And Repository Internals
|
# Adapter Internals
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
This document describes implemented adapter/repository boundaries and their current behavior.
|
Adapters translate external interfaces into domain requests and translate domain results back out. They wire dependencies, apply app config, and own IO concerns, but they do not make runner decisions.
|
||||||
|
|
||||||
|
Source-loading behavior belongs in `docs/internal/sources.md`. User-facing CLI, HTTP, and package contracts belong in `docs/cli.md`, `docs/api.md`, and `docs/consumers/pkg-scriptorium.md`.
|
||||||
|
|
||||||
## Adapter Map
|
## Adapter Map
|
||||||
|
|
||||||
- `internal/adapter/cli`: CLI command parsing, app wiring, stdout/stderr handling, exit codes.
|
- `cmd/scriptorium`: process entrypoint.
|
||||||
- `internal/adapter/http`: HTTP request/response mapping for `POST /v1/runs`.
|
- `internal/adapter/cli`: command parsing, config handoff, runner construction, stdout/stderr, exit codes.
|
||||||
- root package `scriptorium`: public Go library facade for preparing and running prompt requests.
|
- `internal/adapter/http`: `POST /v1/runs` request/response mapping and HTTP error/status mapping.
|
||||||
- `internal/promptdef`: filesystem and `fs.FS` prompt-definition repositories.
|
- root package `scriptorium`: public Go facade over internal runner types and dependencies.
|
||||||
- `internal/profile`: filesystem, `fs.FS`, and overlay execution-profile repositories.
|
|
||||||
- `internal/filecatalog`: shared YAML discovery and display-path helpers for prompt/profile repositories.
|
Supporting implementation packages used during adapter wiring:
|
||||||
- `internal/profile/builtin`: embedded built-in execution-profile repository.
|
|
||||||
- `internal/artifact`: input artifact reader.
|
- `internal/config`
|
||||||
- `internal/prompt`: Go-template renderer.
|
- `internal/defaults`
|
||||||
- `internal/llm`: OpenAI-compatible LLM client implementation.
|
- `internal/format`
|
||||||
- `internal/validate`: filesystem and `fs.FS` output validators.
|
- `internal/llm`
|
||||||
- `internal/format`: prepared-run formatters for `render` output.
|
- `internal/prompt`
|
||||||
|
|
||||||
## Inputs And Outputs
|
## Inputs And Outputs
|
||||||
|
|
||||||
CLI adapter:
|
CLI adapter:
|
||||||
|
|
||||||
- Input: process args, filesystem config/assets, environment.
|
- Input: process args, optional config file, filesystem sources, environment variables.
|
||||||
- Output: exit code, stdout artifact/prepared output, stderr summaries/errors.
|
- Output: process exit code, stdout artifact/prepared output, stderr summaries and errors.
|
||||||
- `run` summaries include cache usage counters only when either parsed cache counter is non-zero.
|
|
||||||
|
|
||||||
HTTP adapter:
|
HTTP adapter:
|
||||||
|
|
||||||
- Input: JSON request body (`runRequestDTO`).
|
- Input: HTTP request method/path/headers/body for `POST /v1/runs`.
|
||||||
- Output: JSON success/error body with mapped status codes.
|
- Output: JSON success or error body with mapped status code.
|
||||||
- Success metadata includes token usage plus cache usage counters.
|
|
||||||
|
|
||||||
Public library facade:
|
Public Go facade:
|
||||||
|
|
||||||
- Input: typed `scriptorium.RunRequest` values.
|
- Input: typed `scriptorium.Config`, `Option`, and `RunRequest` values.
|
||||||
- Output: typed `PreparedRun` and `RunResult` values plus public sentinel errors.
|
- Output: typed `PreparedRun` and `RunResult` values plus public sentinel errors.
|
||||||
- Custom LLM behavior is injected with `WithLLMClient`; otherwise the default OpenAI-compatible client is used.
|
|
||||||
- `RunRequest.APIKey` is a request-scoped Go value only; it is converted into internal execution state for LLM generation and stripped from public result types.
|
|
||||||
- Prompt, profile, and schema source options can use directories, single files, or `fs.FS` roots. Explicit source options override the matching `Config` directory field.
|
|
||||||
- Public types are facade types converted at the package boundary; internal domain types remain internal.
|
|
||||||
|
|
||||||
Prompt/profile repositories:
|
|
||||||
|
|
||||||
- Input: prompt/profile YAML files under configured directories or `fs.FS` roots.
|
|
||||||
- Output: normalized domain definitions/profiles or typed errors.
|
|
||||||
- Shared YAML catalog helpers provide recursive discovery, extension filtering, deterministic ordering, file stems, and `fs.FS` display paths.
|
|
||||||
- Single-file public sources are represented as `fs.FS` roots containing one YAML file; lookup still uses YAML `id` values.
|
|
||||||
|
|
||||||
Profile repository composition:
|
|
||||||
|
|
||||||
- Built-in profiles are embedded and loaded through the same profile validation rules as filesystem profiles.
|
|
||||||
- When no custom profile directory is configured, the runner receives the built-in profile repository.
|
|
||||||
- When a custom profile directory/file/`fs.FS` source is configured, the runner receives an overlay repository with custom profiles as primary and built-ins as fallback.
|
|
||||||
- Overlay lookup falls back only after custom profile-not-found errors; custom load/validation/raw-key errors are returned directly.
|
|
||||||
|
|
||||||
Artifact reader:
|
|
||||||
|
|
||||||
- Input: `domain.ArtifactRef`.
|
|
||||||
- Output: loaded `domain.Artifact`.
|
|
||||||
|
|
||||||
LLM adapter:
|
|
||||||
|
|
||||||
- Input: `domain.GenerateRequest`.
|
|
||||||
- Output: `domain.GenerateResponse`.
|
|
||||||
- Direct API-key values are preferred when present; otherwise `api_key_env` is resolved from the process environment.
|
|
||||||
|
|
||||||
Validator:
|
|
||||||
|
|
||||||
- Input: artifact body + output contract.
|
|
||||||
- Output: validation result or runtime validation error.
|
|
||||||
- Schema documents may be loaded from a directory, single file, or `fs.FS` root in the public package. CLI and HTTP continue to use directory-backed schema loading.
|
|
||||||
|
|
||||||
## Boundaries
|
## Boundaries
|
||||||
|
|
||||||
- Adapters convert external representations to domain requests and back.
|
- Adapters convert external shapes to `domain.RunRequest` and back.
|
||||||
- Use-case decisions remain in `internal/usecase`.
|
- Runner orchestration remains in `internal/usecase`.
|
||||||
- External dependency details stay scoped to adapter packages.
|
- Prompt/profile/schema/artifact source rules remain in repository, validator, and artifact packages.
|
||||||
|
- LLM provider request serialization remains in `internal/llm`.
|
||||||
|
- Public package types are facade types; internal domain types do not leak across the package boundary.
|
||||||
|
|
||||||
## Config Fields Used
|
## Config Fields Used
|
||||||
|
|
||||||
Primary app settings consumed by adapters:
|
Adapter app settings:
|
||||||
|
|
||||||
- `prompt_dir`
|
- `prompt_dir`
|
||||||
- `profile_dir` (optional custom profile source)
|
- `profile_dir`
|
||||||
- `schema_dir`
|
- `schema_dir`
|
||||||
- `server.addr`
|
- `server.addr`
|
||||||
- `server.artifact_root` (HTTP `serve` file input root)
|
- `server.artifact_root`
|
||||||
|
- `server.max_request_bytes`
|
||||||
|
- `server.max_artifact_bytes`
|
||||||
|
- `server.max_response_bytes`
|
||||||
- `defaults.render_format`
|
- `defaults.render_format`
|
||||||
|
|
||||||
Execution profile/request settings used through runner:
|
Execution request/profile settings passed through the runner:
|
||||||
|
|
||||||
- `endpoint`, `model`, `temperature`, `max_tokens`, `top_p`, `timeout_seconds`, `service_tier`, `api_key_env`, `reasoning_effort`, `extra_params`
|
- `endpoint`
|
||||||
- CLI and HTTP request adapters preserve caller intent for numeric runtime overrides. Omitted values remain absent; explicit zero values are mapped as explicit overrides.
|
- `model`
|
||||||
- HTTP `extra_params` accepts JSON-compatible values and maps them to domain request overrides without provider-specific adapter logic.
|
- `temperature`
|
||||||
|
- `max_tokens`
|
||||||
|
- `top_p`
|
||||||
|
- `timeout_seconds`
|
||||||
|
- `service_tier`
|
||||||
|
- `api_key_env`
|
||||||
|
- `reasoning_effort`
|
||||||
|
- `extra_params`
|
||||||
|
|
||||||
## External Dependencies
|
CLI and HTTP preserve numeric override presence so omitted values and explicit zero values remain distinct.
|
||||||
|
|
||||||
- YAML decoding: `gopkg.in/yaml.v3` (strict known-fields mode in config/prompt/profile loaders).
|
## CLI Adapter
|
||||||
- JSON Schema validation: `github.com/santhosh-tekuri/jsonschema/v6`.
|
|
||||||
- HTTP client/server: Go standard library.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
|
|
||||||
Strict decoding and input checks:
|
|
||||||
|
|
||||||
- config/prompt/profile loaders reject unknown YAML fields.
|
|
||||||
- prompt/profile repositories scan nested subdirectories recursively.
|
|
||||||
- prompt/profile lookup uses YAML `id` values; subdirectory paths are organizational only.
|
|
||||||
- prompt `content_file` paths resolve relative to the prompt YAML file within the same source.
|
|
||||||
- duplicate prompt/profile IDs are invalid and fail instead of using first-match behavior.
|
|
||||||
- duplicate profile IDs across custom and built-in sources are allowed; the custom source overrides the built-in profile.
|
|
||||||
- HTTP DTO decoder rejects unknown JSON fields.
|
|
||||||
- raw API key payload fields are rejected by strict decoding in profile/http paths.
|
|
||||||
|
|
||||||
Artifact refs:
|
|
||||||
|
|
||||||
- Supported reference types: `inline`, `file`.
|
|
||||||
- Unsupported types return `ErrUnsupportedRefType`.
|
|
||||||
- CLI `run` and `render` use direct filesystem file reads for `file` references.
|
|
||||||
- HTTP `serve` uses a restricted artifact reader: `inline` references work without a root, while `file` references require `server.artifact_root` or `--artifact-root` and must stay inside that root.
|
|
||||||
- HTTP file paths are resolved with clean absolute paths and containment checks, not string-prefix checks.
|
|
||||||
- Symlinks inside the root are followed by the operating system; the configured root must not be writable by untrusted users.
|
|
||||||
|
|
||||||
LLM adapter:
|
|
||||||
|
|
||||||
- endpoint appends `/chat/completions`.
|
|
||||||
- rendered messages without cache control serialize with string `content`.
|
|
||||||
- rendered messages with cache control serialize as one text content block with `cache_control`.
|
|
||||||
- non-empty `reasoning_effort` serializes as a top-level provider request field.
|
|
||||||
- `extra_params` flatten into provider-specific top-level JSON request fields.
|
|
||||||
- reserved `extra_params` keys are rejected before the provider call: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, and `response_format`.
|
|
||||||
- empty `extra_params` keys and values that cannot be JSON-encoded are rejected before the provider call.
|
|
||||||
- compatible cache usage response fields are parsed into domain token usage.
|
|
||||||
- non-2xx responses map to request failure errors.
|
|
||||||
- malformed responses (including missing/empty first choice content) are errors.
|
|
||||||
- direct API-key values are never serialized in provider request bodies.
|
|
||||||
|
|
||||||
Validator:
|
|
||||||
|
|
||||||
- `basic`, `json`, `json_schema` content failures return `ValidationFailed` results.
|
|
||||||
- schema load/compile/path failures are runtime errors.
|
|
||||||
- schema lookup uses explicit `schema_path` values relative to `schema_dir`; it does not recursively search by basename.
|
|
||||||
|
|
||||||
HTTP error mapping:
|
|
||||||
|
|
||||||
- maps domain/use-case errors to stable HTTP code + error code/message.
|
|
||||||
- distinguishes missing profile selection and missing `api_key_env` variable using stable use-case sentinel errors.
|
|
||||||
- avoids returning internal wrapped-cause details in response payload.
|
|
||||||
|
|
||||||
## CLI Adapter Semantics
|
|
||||||
|
|
||||||
Implemented commands:
|
Implemented commands:
|
||||||
|
|
||||||
@@ -158,30 +83,73 @@ Implemented commands:
|
|||||||
- `render`
|
- `render`
|
||||||
- `serve`
|
- `serve`
|
||||||
|
|
||||||
Behavior highlights:
|
Behavior:
|
||||||
|
|
||||||
- `run` exit `2` indicates validation failed after generation.
|
- `run` constructs a runner with direct filesystem artifact reading and calls `Runner.Run`.
|
||||||
- `render` does not call the LLM.
|
- `render` constructs a runner and calls `Runner.Prepare`; it does not call the LLM.
|
||||||
- `serve` exposes HTTP handler only; no built-in auth.
|
- `serve` constructs a restricted artifact reader and HTTP handler, then starts an unauthenticated HTTP server.
|
||||||
- `render` supports `--format text|json`; `render` does not expose `--schema-dir`.
|
- `run` exits `2` when generation succeeds but validation fails.
|
||||||
- deprecated aliases `--prompt-id` and `--profile-id` are still accepted.
|
- parse, runtime, and output-write errors exit `1`.
|
||||||
|
- deprecated `--prompt-id` and `--profile-id` aliases are accepted.
|
||||||
|
|
||||||
## Tests To Inspect Before Changing
|
## HTTP Adapter
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
|
||||||
|
- Accepts only `POST /v1/runs`.
|
||||||
|
- Decodes JSON strictly and rejects unknown fields and trailing JSON tokens.
|
||||||
|
- Rejects empty `prompt_id` and empty `inputs` before calling the runner.
|
||||||
|
- Does not accept raw API key values in the request body.
|
||||||
|
- Returns validation failures as `200` responses with failed validation details.
|
||||||
|
- Maps request-body, artifact, and encoded-response size failures to `413`.
|
||||||
|
- Maps domain and repository errors to stable error codes without returning wrapped internal cause text.
|
||||||
|
|
||||||
|
The HTTP adapter has no built-in authentication or authorization. Deployment controls must be provided outside the process.
|
||||||
|
|
||||||
|
## Public Go Facade
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
|
||||||
|
- `NewEngine` wires the same default runner components as CLI/HTTP unless options override them.
|
||||||
|
- Prompt, profile, and schema sources may come from directories, single files, or `fs.FS` roots.
|
||||||
|
- `WithProfiles` adds in-memory profiles ahead of file-backed and built-in profiles.
|
||||||
|
- `WithLLMClient` injects custom model behavior.
|
||||||
|
- `RunRequest.APIKey` is request-scoped and direct; it is used only for generation and is stripped from public results.
|
||||||
|
- internal errors are mapped to public sentinels in `errors.go`.
|
||||||
|
|
||||||
|
## Failure Behavior
|
||||||
|
|
||||||
|
Adapters should:
|
||||||
|
|
||||||
|
- keep external error payloads concise and stable.
|
||||||
|
- avoid leaking raw secret values.
|
||||||
|
- use sentinels and typed errors for mapping.
|
||||||
|
- preserve strict external input decoding.
|
||||||
|
- keep validation content failures distinct from runtime errors.
|
||||||
|
|
||||||
|
CLI writes human-readable summaries to stderr. HTTP writes JSON error envelopes. The public Go facade returns typed errors.
|
||||||
|
|
||||||
|
## State And Manifests
|
||||||
|
|
||||||
|
Adapters do not add durable run state.
|
||||||
|
|
||||||
|
- No adapter writes run manifests.
|
||||||
|
- No adapter implements checkpoint, skip, or resume behavior.
|
||||||
|
- CLI output files are caller-selected artifacts, not internal state.
|
||||||
|
|
||||||
|
## Tests To Inspect
|
||||||
|
|
||||||
- `internal/adapter/cli/run_test.go`
|
- `internal/adapter/cli/run_test.go`
|
||||||
- `internal/adapter/http/handler_test.go`
|
- `internal/adapter/http/handler_test.go`
|
||||||
- `internal/promptdef/repository_test.go`
|
- `engine_test.go`
|
||||||
- `internal/profile/repository_test.go`
|
|
||||||
- `internal/artifact/reader_test.go`
|
|
||||||
- `internal/prompt/renderer_test.go`
|
|
||||||
- `internal/llm/openai_compatible_client_test.go`
|
|
||||||
- `internal/validate/standard_validator_test.go`
|
|
||||||
- `internal/format/prepared_run_test.go`
|
- `internal/format/prepared_run_test.go`
|
||||||
|
- `internal/llm/openai_compatible_client_test.go`
|
||||||
|
|
||||||
## Architectural Invariants
|
## Architectural Invariants
|
||||||
|
|
||||||
- Adapter packages do not own runner decision logic.
|
- Adapter packages stay thin and translation-focused.
|
||||||
- External request/response strictness is part of contract stability.
|
- App config is resolved before dependency construction.
|
||||||
- Prepared-render output never includes resolved API key values.
|
- External input strictness is part of contract stability.
|
||||||
- Outbound OpenAI-compatible request includes currently serialized first-class fields (`model`, optional `session_id`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `service_tier`, optional `reasoning_effort`, optional `response_format`) plus validated `extra_params` flattened as provider-specific top-level fields.
|
- CLI and HTTP construct runners without a repairer.
|
||||||
- Outbound cache control is message-level only; no top-level cache-control field is serialized.
|
- HTTP endpoint details remain canonical in `docs/api.md`.
|
||||||
|
- Public Go package details remain canonical in `docs/consumers/pkg-scriptorium.md`.
|
||||||
|
|||||||
@@ -2,29 +2,32 @@
|
|||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
`internal/usecase.Runner` is the core use case orchestrator for prompt preparation and execution.
|
`internal/usecase.Runner` is the core prompt-execution orchestrator. It prepares prompt requests, calls the configured LLM client for `Run`, validates generated output, and returns domain results.
|
||||||
|
|
||||||
It owns request validation, prompt/profile resolution, runtime-parameter merge, artifact loading, prompt rendering, structured-output setup, LLM invocation, output validation, and result metadata.
|
Transport parsing, DTOs, CLI output, HTTP status mapping, and public package type conversion belong outside the runner.
|
||||||
|
|
||||||
## Inputs And Outputs
|
## Inputs And Outputs
|
||||||
|
|
||||||
Primary input type:
|
Primary inputs:
|
||||||
|
|
||||||
- `domain.RunRequest`
|
- `domain.RunRequest`
|
||||||
|
- repositories/readers/renderers/validators injected at construction
|
||||||
|
- `context.Context` for cancellation
|
||||||
|
|
||||||
Primary output types:
|
Primary outputs:
|
||||||
|
|
||||||
- `domain.PreparedRun` from `Prepare`
|
- `domain.PreparedRun` from `Prepare`
|
||||||
- `domain.RunResult` from `Run`
|
- `domain.RunResult` from `Run`
|
||||||
|
- wrapped sentinel errors for adapter mapping
|
||||||
|
|
||||||
LLM boundary types:
|
LLM boundary types:
|
||||||
|
|
||||||
- `domain.GenerateRequest`
|
- `domain.GenerateRequest`
|
||||||
- `domain.GenerateResponse`
|
- `domain.GenerateResponse`
|
||||||
|
|
||||||
## Boundaries
|
## Dependencies
|
||||||
|
|
||||||
`Runner` coordinates the following interfaces:
|
`Runner` depends on package interfaces instead of concrete adapter types:
|
||||||
|
|
||||||
- `promptdef.Repository`
|
- `promptdef.Repository`
|
||||||
- `profile.Repository`
|
- `profile.Repository`
|
||||||
@@ -34,136 +37,110 @@ LLM boundary types:
|
|||||||
- `validate.Validator`
|
- `validate.Validator`
|
||||||
- optional `usecase.OutputRepairer`
|
- optional `usecase.OutputRepairer`
|
||||||
|
|
||||||
Transport concerns (CLI flags, HTTP DTO parsing, status-code mapping) stay outside runner.
|
The CLI, HTTP adapter, and public Go package construct these dependencies and pass them in.
|
||||||
|
|
||||||
## Config Fields Used
|
## Config Fields
|
||||||
|
|
||||||
`Runner` does not read app config files directly.
|
`Runner` does not read app config files. Effective behavior is determined by injected dependencies and the `domain.RunRequest`.
|
||||||
|
|
||||||
It receives fully constructed repositories/readers/validators from adapters. Effective behavior depends on adapter wiring, including:
|
Adapter wiring commonly reflects these app config fields:
|
||||||
|
|
||||||
- prompt/profile directories
|
- `prompt_dir`
|
||||||
- schema base directory
|
- `profile_dir`
|
||||||
- selected profile/runtime overrides in request
|
- `schema_dir`
|
||||||
|
- `server.artifact_root`
|
||||||
|
- HTTP request/artifact/response size limits
|
||||||
|
|
||||||
## External Adapters Used
|
Runtime model settings are resolved from the selected profile plus request overrides.
|
||||||
|
|
||||||
`Runner` works with adapter implementations via interfaces. Current wiring from CLI/HTTP uses:
|
|
||||||
|
|
||||||
- filesystem prompt/profile repositories
|
|
||||||
- composite artifact reader
|
|
||||||
- Go-template prompt renderer
|
|
||||||
- OpenAI-compatible LLM client
|
|
||||||
- standard validator
|
|
||||||
|
|
||||||
## State And Resume Behavior
|
|
||||||
|
|
||||||
`Runner` is stateless across requests.
|
|
||||||
|
|
||||||
- No durable run-state storage.
|
|
||||||
- No built-in resume/skip checkpoints.
|
|
||||||
- Each `Run`/`Prepare` executes from request inputs and current repositories.
|
|
||||||
|
|
||||||
## Failure Behavior
|
|
||||||
|
|
||||||
Primary runner error classes:
|
|
||||||
|
|
||||||
- `ErrInvalidRequest`: invalid run request envelope.
|
|
||||||
- `ErrProfileRequired`: specific invalid-request reason when neither request `profile_id` nor prompt `default_profile` is available.
|
|
||||||
- `ErrAPIKeyEnvMissing`: specific invalid-request reason when `api_key_env` is set but the named environment variable is unset/empty.
|
|
||||||
- `ErrPromptLoad`: prompt-definition repository load failures.
|
|
||||||
- `ErrProfileLoad`: execution-profile repository load failures.
|
|
||||||
- `ErrArtifactLoad`: artifact read failures.
|
|
||||||
- `ErrPromptRender`: template render failures.
|
|
||||||
- `ErrLLMGenerate`: outbound model request failures.
|
|
||||||
- `ErrValidation`: validation runtime failures (including structured-output schema load/compile failures).
|
|
||||||
|
|
||||||
Reason sentinel behavior:
|
|
||||||
|
|
||||||
- `ErrProfileRequired` and `ErrAPIKeyEnvMissing` are wrapped with `ErrInvalidRequest`.
|
|
||||||
- Adapters can use `errors.Is` for stable reason mapping without matching runner prose.
|
|
||||||
|
|
||||||
Validation content failures are not run errors:
|
|
||||||
|
|
||||||
- `Run` can succeed with `Validation.Status == failed`.
|
|
||||||
- CLI maps this to exit code `2`.
|
|
||||||
- HTTP returns `200` with failed validation details.
|
|
||||||
|
|
||||||
## Prepare Flow
|
## Prepare Flow
|
||||||
|
|
||||||
`Prepare` performs:
|
`Prepare`:
|
||||||
|
|
||||||
1. validate request basics (prompt ID present).
|
1. requires a non-empty prompt ID.
|
||||||
2. load prompt definition by ID/version.
|
2. loads the prompt definition and computes its hash.
|
||||||
3. select profile ID:
|
3. selects the profile from request `profile_id`, then prompt `default_profile`.
|
||||||
- explicit request profile ID
|
4. loads the selected execution profile.
|
||||||
- prompt `default_profile`
|
5. merges built-in execution defaults, profile values, and request overrides.
|
||||||
- otherwise return an invalid request with `ErrProfileRequired`
|
6. applies request-scoped direct API key values for public Go callers.
|
||||||
4. load execution profile.
|
7. validates endpoint, model, and credential requirements.
|
||||||
5. merge effective runtime target:
|
8. resolves the output contract and JSON Schema document when required.
|
||||||
- built-in execution defaults
|
9. reads input artifacts.
|
||||||
- selected profile values
|
10. renders prompt messages and hashes the rendered prompt.
|
||||||
- request overrides
|
11. returns a prepared run without calling the LLM.
|
||||||
- request numeric overrides are presence-aware, so omitted values preserve the current effective value and explicit zero values override it
|
|
||||||
6. verify credentials when the effective target names `api_key_env`:
|
|
||||||
- a request-scoped direct API key satisfies the credential requirement
|
|
||||||
- otherwise a missing/empty env value returns an invalid request with `ErrAPIKeyEnvMissing`
|
|
||||||
- only the environment-variable name is returned in public output; secret values are never returned
|
|
||||||
7. resolve output contract and structured-output schema payload when `json_schema` mode is active.
|
|
||||||
8. read input artifacts.
|
|
||||||
9. render prompt messages, including any normalized message cache-control metadata.
|
|
||||||
10. compute prompt/input/render hashes and return `PreparedRun`.
|
|
||||||
|
|
||||||
`rendered_prompt_hash` includes cache-control metadata when present because it affects the outbound provider request. Prompts without cache control keep the role/content hash behavior.
|
Numeric request overrides are presence-aware: omitted values preserve the current effective value, while explicit zero values are real overrides.
|
||||||
|
|
||||||
`Prepare` does not call the LLM.
|
|
||||||
|
|
||||||
Runtime target notes:
|
|
||||||
|
|
||||||
- Profile `extra_params` and request `extra_params` carry JSON-compatible values through prepared output, run metadata, and `domain.GenerateRequest.Target`.
|
|
||||||
- The OpenAI-compatible client serializes non-empty `reasoning_effort` as a top-level provider request field.
|
|
||||||
- The OpenAI-compatible client flattens `extra_params` into provider-specific top-level JSON request fields.
|
|
||||||
- Empty `extra_params` keys, reserved outbound field names, and values that cannot be JSON-encoded fail before the provider request.
|
|
||||||
- Resolved API-key values are never serialized in prepared/run output, public results, logs, or HTTP responses.
|
|
||||||
- Public direct API-key values are carried only far enough to call the configured LLM client and are excluded from JSON/YAML serialization.
|
|
||||||
|
|
||||||
## Run Flow
|
## Run Flow
|
||||||
|
|
||||||
`Run` performs:
|
`Run`:
|
||||||
|
|
||||||
1. generate run ID.
|
1. creates a run ID and start timestamp.
|
||||||
2. call `Prepare`.
|
2. calls `Prepare`.
|
||||||
3. call LLM with prepared messages/effective target/structured-output spec.
|
3. calls the injected LLM client with rendered messages, effective target, target presence, and structured-output settings.
|
||||||
4. build output artifact content type from output format.
|
4. builds the output artifact.
|
||||||
5. validate output.
|
5. validates the output.
|
||||||
6. optionally attempt bounded repair when repairer is injected and contract allows it.
|
6. optionally attempts bounded repair when a repairer is injected and the contract permits repair.
|
||||||
7. return `RunResult` with artifact, raw output, validation, hashes, profile/model metadata, token/cache usage, and timestamps.
|
7. returns the run result with artifact, raw output, validation, hashes, selected profile/model metadata, usage, and timing.
|
||||||
|
|
||||||
## Repair Hook Boundary
|
`Run` must reuse `Prepare`; prepare logic should not be duplicated elsewhere.
|
||||||
|
|
||||||
Repair attempts occur only when all are true:
|
## Validation And Repair
|
||||||
|
|
||||||
- repairer is injected
|
Validation content failures are returned as successful run results with `Validation.Status == failed`. They are not runtime errors.
|
||||||
- `repair_attempts > 0`
|
|
||||||
|
Validation runtime failures, such as schema load or compile errors, return `ErrValidation`.
|
||||||
|
|
||||||
|
Repair attempts occur only when all conditions are true:
|
||||||
|
|
||||||
|
- a repairer is injected
|
||||||
|
- `repair_attempts` is greater than zero
|
||||||
- validation status is `failed`
|
- validation status is `failed`
|
||||||
- validation mode is `json` or `json_schema`
|
- validation mode is `json` or `json_schema`
|
||||||
|
|
||||||
Current production wiring boundary:
|
CLI and HTTP wiring call `usecase.NewRunner(...)`, which does not inject a repairer. Normal CLI and HTTP execution therefore does not repair invalid output.
|
||||||
|
|
||||||
- CLI and HTTP adapters call `usecase.NewRunner(...)` (no repairer argument).
|
## Failure Behavior
|
||||||
- Therefore normal CLI/HTTP execution does not perform repair attempts today.
|
|
||||||
|
|
||||||
## Tests To Inspect Before Changing
|
Stable runner sentinels include:
|
||||||
|
|
||||||
|
- `ErrInvalidRequest`
|
||||||
|
- `ErrProfileRequired`
|
||||||
|
- `ErrAPIKeyEnvMissing`
|
||||||
|
- `ErrAPIKeyRequired`
|
||||||
|
- `ErrPromptLoad`
|
||||||
|
- `ErrProfileLoad`
|
||||||
|
- `ErrArtifactLoad`
|
||||||
|
- `ErrPromptRender`
|
||||||
|
- `ErrLLMGenerate`
|
||||||
|
- `ErrValidation`
|
||||||
|
|
||||||
|
Adapters should use `errors.Is` against sentinels and lower-level repository errors instead of matching message text.
|
||||||
|
|
||||||
|
Secret values must not appear in prepared output, run results, logs, HTTP responses, or serialized public package results. The effective API-key environment-variable name may appear.
|
||||||
|
|
||||||
|
## State And Manifests
|
||||||
|
|
||||||
|
The runner is stateless across requests.
|
||||||
|
|
||||||
|
- No durable run store.
|
||||||
|
- No manifest files.
|
||||||
|
- No checkpoint, skip, or resume behavior.
|
||||||
|
- Recovery is a new request after correcting inputs, config, or environment.
|
||||||
|
|
||||||
|
## Tests To Inspect
|
||||||
|
|
||||||
- `internal/usecase/runner_test.go`
|
- `internal/usecase/runner_test.go`
|
||||||
- `internal/usecase/integration_test.go`
|
- `internal/usecase/integration_test.go`
|
||||||
|
- `engine_test.go`
|
||||||
- `internal/adapter/cli/run_test.go`
|
- `internal/adapter/cli/run_test.go`
|
||||||
- `internal/adapter/http/handler_test.go`
|
- `internal/adapter/http/handler_test.go`
|
||||||
|
|
||||||
## Architectural Invariants
|
## Architectural Invariants
|
||||||
|
|
||||||
- `Run` reuses `Prepare`; prepare logic is not duplicated.
|
- Use-case decisions stay in `internal/usecase`.
|
||||||
- Effective API-key environment-variable name may appear; resolved secret value must not.
|
- `Run` reuses `Prepare`.
|
||||||
- Structured-output schema document must load before LLM call for `json_schema` mode.
|
- Prompt/profile/artifact/schema loading remains behind injected boundaries.
|
||||||
|
- Validation content failures are result state; validation runtime failures are errors.
|
||||||
- Repair loops are bounded by `repair_attempts` and repairer presence.
|
- Repair loops are bounded by `repair_attempts` and repairer presence.
|
||||||
- Runner stays transport-agnostic.
|
- Resolved secret values are never serialized or emitted.
|
||||||
|
|||||||
157
docs/internal/sources.md
Normal file
157
docs/internal/sources.md
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
# Source Internals
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
This document covers implemented prompt, profile, schema, artifact, and catalog source behavior. It is for developers changing loaders or source wiring.
|
||||||
|
|
||||||
|
Full user-facing YAML and config reference material belongs in `docs/config.md`.
|
||||||
|
|
||||||
|
## Prompt Definition Sources
|
||||||
|
|
||||||
|
`internal/promptdef` provides directory-backed and `fs.FS` repositories.
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
|
||||||
|
- recursively scans `.yaml` and `.yml` files.
|
||||||
|
- decodes YAML with known-fields checking.
|
||||||
|
- looks up prompts by YAML `id`, not by path.
|
||||||
|
- optionally filters by prompt `version`.
|
||||||
|
- rejects duplicate matching prompt IDs.
|
||||||
|
- requires `id`, `version`, and at least one message.
|
||||||
|
- requires each message to set exactly one of `content` or `content_file`.
|
||||||
|
- resolves filesystem `content_file` values relative to the prompt YAML file.
|
||||||
|
- resolves `fs.FS` `content_file` values inside the configured source root.
|
||||||
|
- permits prompt subdirectories only as organization; they are not part of prompt identity.
|
||||||
|
|
||||||
|
For `fs.FS` roots, absolute paths and relative traversal outside the source root are rejected by catalog path helpers.
|
||||||
|
|
||||||
|
## Profile Sources
|
||||||
|
|
||||||
|
`internal/profile` provides directory-backed, `fs.FS`, and overlay repositories. `internal/profile/builtin` embeds built-in profile YAML assets and exposes them through the same repository interface.
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
|
||||||
|
- recursively scans `.yaml` and `.yml` files.
|
||||||
|
- decodes YAML with known-fields checking.
|
||||||
|
- looks up profiles by YAML `id`, not by path.
|
||||||
|
- rejects duplicate IDs inside the same source.
|
||||||
|
- rejects raw `api_key` fields in YAML; file-backed profiles must use `api_key_env`.
|
||||||
|
- validates required `endpoint` and `model` values.
|
||||||
|
- validates numeric profile ranges.
|
||||||
|
|
||||||
|
Overlay behavior:
|
||||||
|
|
||||||
|
- custom profiles are primary.
|
||||||
|
- built-in profiles are fallback.
|
||||||
|
- fallback occurs only after a primary `ErrProfileNotFound`.
|
||||||
|
- primary validation, YAML, duplicate, and raw-key errors are returned directly.
|
||||||
|
- duplicate IDs across custom and built-in sources are allowed because the custom profile overrides the built-in one.
|
||||||
|
|
||||||
|
The public Go facade can add in-memory profiles ahead of file-backed and built-in profiles.
|
||||||
|
|
||||||
|
## Schema Sources
|
||||||
|
|
||||||
|
`internal/validate` provides:
|
||||||
|
|
||||||
|
- `StandardValidator` for filesystem paths.
|
||||||
|
- `FSValidator` for `fs.FS` roots and single-file public schema sources.
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
|
||||||
|
- `json_schema` validation requires a non-empty `schema_path`.
|
||||||
|
- filesystem schema paths resolve relative to `schema_dir` unless absolute.
|
||||||
|
- directory-backed schema lookup uses the explicit `schema_path`; it does not search recursively by basename.
|
||||||
|
- `fs.FS` schema paths must remain inside the configured source root.
|
||||||
|
- single-file schema sources match by the configured file base name.
|
||||||
|
- schema documents are loaded before the LLM call for structured output.
|
||||||
|
- JSON parse failures are validation content failures.
|
||||||
|
- schema access, decode, registration, and compile failures are runtime validation errors.
|
||||||
|
|
||||||
|
## Artifact Sources
|
||||||
|
|
||||||
|
`internal/artifact` supports two input artifact reference types:
|
||||||
|
|
||||||
|
- `inline`
|
||||||
|
- `file`
|
||||||
|
|
||||||
|
Inline behavior:
|
||||||
|
|
||||||
|
- requires a non-empty body.
|
||||||
|
- produces text/plain artifacts.
|
||||||
|
- hashes the body bytes.
|
||||||
|
|
||||||
|
Direct file behavior:
|
||||||
|
|
||||||
|
- used by CLI `run`, CLI `render`, and the public Go facade.
|
||||||
|
- requires a non-empty URI.
|
||||||
|
- reads from the process filesystem without HTTP artifact-root restrictions.
|
||||||
|
- infers content type from file extension, defaulting to text/plain.
|
||||||
|
|
||||||
|
Restricted file behavior:
|
||||||
|
|
||||||
|
- used by HTTP `serve`.
|
||||||
|
- allows inline artifacts even when no artifact root is configured.
|
||||||
|
- denies file artifacts when no artifact root is configured.
|
||||||
|
- resolves relative file URIs against `server.artifact_root`.
|
||||||
|
- accepts absolute file URIs only when they pass containment checks.
|
||||||
|
- applies `server.max_artifact_bytes` when configured.
|
||||||
|
|
||||||
|
Restricted containment is lexical. It cleans paths and checks the relative path against the configured root; it does not resolve symlinks. Symlinks inside the root are followed by the operating system, including symlinks that target files outside the root.
|
||||||
|
|
||||||
|
## Catalog Helpers
|
||||||
|
|
||||||
|
`internal/filecatalog` centralizes shared source helpers:
|
||||||
|
|
||||||
|
- recursive YAML discovery for filesystem and `fs.FS` roots.
|
||||||
|
- deterministic sorting.
|
||||||
|
- `.yaml` and `.yml` filtering.
|
||||||
|
- display paths for diagnostics.
|
||||||
|
- YAML file stems.
|
||||||
|
- `fs.FS` root cleaning and containment checks.
|
||||||
|
|
||||||
|
Repository code should use these helpers instead of reimplementing path traversal and containment rules.
|
||||||
|
|
||||||
|
## Failure Behavior
|
||||||
|
|
||||||
|
Common source failures:
|
||||||
|
|
||||||
|
- missing prompt/profile/schema/artifact files.
|
||||||
|
- invalid YAML or JSON.
|
||||||
|
- unknown YAML fields.
|
||||||
|
- duplicate prompt or profile IDs.
|
||||||
|
- prompt/profile validation errors.
|
||||||
|
- raw API key fields in profile YAML.
|
||||||
|
- unsupported artifact reference type.
|
||||||
|
- missing inline body or file URI.
|
||||||
|
- artifact outside HTTP root.
|
||||||
|
- artifact exceeding HTTP size limit.
|
||||||
|
- schema load or compile failure.
|
||||||
|
|
||||||
|
Prompt/profile repository lookup errors are mapped by adapters separately from runtime runner errors. Validation content failures remain result state; source and schema runtime failures return errors.
|
||||||
|
|
||||||
|
## State And Manifests
|
||||||
|
|
||||||
|
Source packages do not persist run state.
|
||||||
|
|
||||||
|
- No manifests are read or written.
|
||||||
|
- No source package implements skip or resume behavior.
|
||||||
|
- Source reads reflect the current filesystem or `fs.FS` state for each request.
|
||||||
|
|
||||||
|
## Tests To Inspect
|
||||||
|
|
||||||
|
- `internal/promptdef/repository_test.go`
|
||||||
|
- `internal/profile/repository_test.go`
|
||||||
|
- `internal/profile/builtin/repository_test.go`
|
||||||
|
- `internal/artifact/reader_test.go`
|
||||||
|
- `internal/validate/standard_validator_test.go`
|
||||||
|
- `internal/usecase/integration_test.go`
|
||||||
|
- `engine_test.go`
|
||||||
|
|
||||||
|
## Architectural Invariants
|
||||||
|
|
||||||
|
- Prompt/profile identity comes from YAML `id`.
|
||||||
|
- External YAML decoding remains strict.
|
||||||
|
- File-backed profile YAML never accepts raw API key values.
|
||||||
|
- Built-in profiles are fallback, not a replacement for custom source validation.
|
||||||
|
- HTTP file artifacts remain rooted by lexical containment.
|
||||||
|
- Schema runtime failures remain errors, while JSON/schema content mismatches remain validation results.
|
||||||
@@ -2,128 +2,163 @@
|
|||||||
|
|
||||||
## Scope
|
## Scope
|
||||||
|
|
||||||
This document covers day-to-day operation of the CLI and HTTP service for currently implemented behavior.
|
This guide covers operating the implemented CLI commands and HTTP service. It
|
||||||
|
does not replace the [CLI reference](cli.md), [Configuration reference](config.md),
|
||||||
For command syntax, see [CLI reference](cli.md). For file formats and defaults, see [Configuration reference](config.md).
|
or [HTTP API reference](api.md).
|
||||||
|
|
||||||
## Operational Model
|
## Operational Model
|
||||||
|
|
||||||
Scriptorium executes one request at a time per CLI invocation or HTTP request.
|
Scriptorium executes one prompt request per CLI invocation or HTTP request.
|
||||||
|
|
||||||
Important boundaries:
|
Important boundaries:
|
||||||
|
|
||||||
- No durable run state is stored.
|
- No durable run state is stored.
|
||||||
- No built-in resume, checkpoint, archive, or backup workflow exists.
|
- No manifest, archive, checkpoint, or built-in backup workflow is written.
|
||||||
- Recovery is rerun-based: fix inputs/config, then rerun.
|
- No built-in resume behavior exists.
|
||||||
|
- Recovery is rerun-based: correct inputs, config, or environment, then run again.
|
||||||
|
|
||||||
## Filesystem Layout And Config
|
## Filesystem Layout
|
||||||
|
|
||||||
Scriptorium depends on:
|
Operational deployments usually provide:
|
||||||
|
|
||||||
- prompt definition files (`prompt_dir`)
|
- `prompt_dir`: prompt definition YAML files and adjacent `content_file` templates.
|
||||||
- execution profile files (`profile_dir`)
|
- `profile_dir`: optional custom profile YAML files.
|
||||||
- optional JSON schemas (`schema_dir`)
|
- `schema_dir`: optional JSON Schema files.
|
||||||
|
- `server.artifact_root`: optional HTTP file-input root for `serve`.
|
||||||
|
|
||||||
Config discovery order when `--config` is omitted:
|
Keep these directories readable by the Scriptorium process. Keep
|
||||||
|
`server.artifact_root` narrow and not writable by untrusted users.
|
||||||
1. `/usr/local/etc/scriptorium/config.yml`
|
|
||||||
2. `/etc/scriptorium/config.yml`
|
|
||||||
|
|
||||||
If neither exists, built-in defaults are used. If `--config <path>` is provided, that file must exist and parse successfully.
|
|
||||||
|
|
||||||
Built-in defaults relevant to operations:
|
|
||||||
|
|
||||||
- `schema_dir: .`
|
|
||||||
- `server.addr: :8080`
|
|
||||||
- `server.artifact_root`: unset; HTTP `file` input references are rejected until configured
|
|
||||||
- `defaults.render_format: text`
|
|
||||||
|
|
||||||
## Normal CLI Workflow
|
## Normal CLI Workflow
|
||||||
|
|
||||||
Use `render` first when you need to verify prompt resolution and runtime settings without calling a model.
|
Use `render` before `run` when changing prompt/profile/input wiring:
|
||||||
|
|
||||||
Use `run` for generation.
|
```bash
|
||||||
|
go run ./cmd/scriptorium render \
|
||||||
|
--config ./examples/config.yml \
|
||||||
|
--prompt generic.markdown_summary \
|
||||||
|
--input transcript=./examples/fixtures/transcript.md \
|
||||||
|
--input glossary=./examples/fixtures/glossary.yml \
|
||||||
|
--format json
|
||||||
|
```
|
||||||
|
|
||||||
Typical sequence:
|
Use `run` for generation after preflight:
|
||||||
|
|
||||||
1. Confirm prompt/profile directories resolve through config or flags.
|
```bash
|
||||||
2. Confirm required input files exist and map to prompt input names.
|
go run ./cmd/scriptorium run \
|
||||||
3. Confirm required API-key environment variables are set.
|
--config ./examples/config.yml \
|
||||||
4. Confirm the selected profile's model endpoint is reachable from the process environment.
|
--prompt generic.markdown_summary \
|
||||||
5. Run `render` for preflight when changing prompt/profile/input wiring.
|
--input transcript=./examples/fixtures/transcript.md \
|
||||||
6. Run `run` for actual generation.
|
--input glossary=./examples/fixtures/glossary.yml \
|
||||||
|
--out ./summary.md
|
||||||
|
```
|
||||||
|
|
||||||
## Secrets Handling
|
Before production runs, confirm:
|
||||||
|
|
||||||
Raw API keys are not accepted in config files, profile files as `api_key`, CLI flags, or HTTP request bodies.
|
- the effective config path is the intended one;
|
||||||
|
- prompt/profile/schema directories are readable;
|
||||||
Operational pattern:
|
- input file paths exist and match prompt input names;
|
||||||
|
- required API-key environment variables are set;
|
||||||
- Set environment variables that hold secret values.
|
- the selected model endpoint is reachable from the process environment.
|
||||||
- Set profile `api_key_env` (or runtime override `api_key_env`) to the environment variable name.
|
|
||||||
- Keep process environments scoped to only required variables.
|
|
||||||
|
|
||||||
## HTTP Service Operation
|
## HTTP Service Operation
|
||||||
|
|
||||||
Start service with:
|
Start the service with:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
go run ./cmd/scriptorium serve --config ./examples/config.yml
|
go run ./cmd/scriptorium serve --config ./examples/config.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
Current inbound API behavior:
|
The implemented HTTP route is `POST /v1/runs`; request and response fields are
|
||||||
|
defined in the [HTTP API reference](api.md).
|
||||||
|
|
||||||
- Route: `POST /v1/runs`
|
The maintained HTTP request-shape example is `examples/http-run.json`.
|
||||||
- JSON request parsing rejects unknown fields.
|
|
||||||
- Validation content failures still return `200 OK` with `validation.status: "failed"`.
|
|
||||||
- `inline` input references work without filesystem configuration.
|
|
||||||
- `file` input references require `server.artifact_root` or `serve --artifact-root`; relative paths resolve inside that root and paths outside it are rejected.
|
|
||||||
|
|
||||||
Security caveat:
|
HTTP service notes:
|
||||||
|
|
||||||
|
- Unknown JSON fields are rejected.
|
||||||
|
- `inline` input references work without an artifact root.
|
||||||
|
- `file` input references require `server.artifact_root` or `serve --artifact-root`.
|
||||||
|
- Request bodies, HTTP file input artifacts, and encoded JSON responses are size-limited.
|
||||||
|
- Validation content failures return `200 OK` with `validation.status: "failed"`.
|
||||||
|
|
||||||
|
Security boundary:
|
||||||
|
|
||||||
- `serve` has no built-in authentication or authorization.
|
- `serve` has no built-in authentication or authorization.
|
||||||
- Deploy only behind trusted controls (private network boundary, authenticated reverse proxy, API gateway, or equivalent).
|
- Put it behind trusted controls such as a private network, authenticated reverse proxy, or API gateway.
|
||||||
- Keep the HTTP artifact root as narrow as practical and do not make it writable by untrusted users.
|
- Do not expose an artifact root containing unrelated sensitive files.
|
||||||
|
- Symlinks inside the artifact root are followed by the operating system.
|
||||||
|
|
||||||
|
## Secrets Handling
|
||||||
|
|
||||||
|
Raw API keys are not accepted in app config, profiles, CLI flags, or HTTP
|
||||||
|
request bodies.
|
||||||
|
|
||||||
|
Use this pattern:
|
||||||
|
|
||||||
|
1. Set an environment variable containing the secret value.
|
||||||
|
2. Store only the variable name in profile `api_key_env` or request override `api_key_env`.
|
||||||
|
3. Scope the process environment to the minimum required variables.
|
||||||
|
|
||||||
## Output, Logs, And Exit Codes
|
## Output, Logs, And Exit Codes
|
||||||
|
|
||||||
`run` command:
|
`run`:
|
||||||
|
|
||||||
- Generated artifact body goes to stdout by default.
|
- stdout: generated artifact body unless `--out` is used.
|
||||||
- `--out` writes generated artifact to a file.
|
- stderr: summary on success, errors on failure.
|
||||||
- Summary metadata line is written to stderr on success.
|
- exit `2`: generation completed and output was written, but validation failed.
|
||||||
- Exit code `2` means generation completed but validation failed.
|
|
||||||
|
|
||||||
`render` command:
|
`render`:
|
||||||
|
|
||||||
- Prepared-run output goes to stdout by default.
|
- stdout: prepared-run output unless `--out` is used.
|
||||||
- `--out` writes prepared-run output to a file.
|
- stderr: errors.
|
||||||
- Exit code is `0` on success and `1` on failure.
|
- exit `0` on success, `1` on failure.
|
||||||
|
|
||||||
`serve` command:
|
`serve`:
|
||||||
|
|
||||||
- Startup and server errors are written to stderr.
|
- stderr: startup and server errors.
|
||||||
|
- HTTP response body: JSON success or error envelope.
|
||||||
|
|
||||||
## Validation Behavior In Operations
|
## Validation Behavior
|
||||||
|
|
||||||
Validation modes (`none`, `basic`, `json`, `json_schema`) are defined by prompt output contract.
|
Prompt `output.validation_mode` controls validation:
|
||||||
|
|
||||||
Operational interpretation:
|
- `none`: skipped.
|
||||||
|
- `basic`: output body must not be empty.
|
||||||
|
- `json`: output body must parse as JSON.
|
||||||
|
- `json_schema`: output body must parse as JSON and satisfy the configured schema.
|
||||||
|
|
||||||
- Validation runtime errors are hard failures (`run` exit `1`; HTTP error response).
|
Runtime/schema failures are hard failures (`run` exit `1`, HTTP error).
|
||||||
- Validation content failures are soft failures (`run` exit `2`; HTTP `200` with failed status).
|
Generated-content validation failures are soft failures (`run` exit `2`, HTTP
|
||||||
|
`200 OK` with failed validation status).
|
||||||
|
|
||||||
A failed validation run can still produce output. Decide whether to keep or discard that output in your surrounding workflow.
|
## Size Limits
|
||||||
|
|
||||||
## Safe Recovery Steps
|
Defaults are documented in [Configuration reference](config.md). Operationally:
|
||||||
|
|
||||||
For failed runs or requests:
|
- Keep default HTTP limits unless larger payloads are measured and expected.
|
||||||
|
- Prefer `inline` HTTP inputs for small payloads.
|
||||||
|
- Prefer `file` HTTP inputs for larger local artifacts under a controlled artifact root.
|
||||||
|
- Increase `server.max_response_bytes` when generated artifacts or requested raw output are expected to be large.
|
||||||
|
- Use `0` only when another trusted layer enforces size limits.
|
||||||
|
|
||||||
1. Capture stderr output or HTTP error code/message.
|
## Maintained Examples
|
||||||
2. Confirm config path and directory settings.
|
|
||||||
3. Verify prompt/profile IDs and input mappings.
|
- `examples/config.yml`
|
||||||
4. Verify API-key environment-variable presence when required.
|
- `examples/config.full.yml`
|
||||||
5. Reproduce with `render --format json` when prompt/profile/input resolution is uncertain.
|
- `examples/render-markdown-summary.sh`
|
||||||
|
- `examples/http-run.json`
|
||||||
|
|
||||||
|
## Safe Recovery
|
||||||
|
|
||||||
|
For failed CLI commands or HTTP requests:
|
||||||
|
|
||||||
|
1. Capture stderr or the HTTP error `code` and `message`.
|
||||||
|
2. Confirm config path and effective directory settings.
|
||||||
|
3. Verify prompt ID, profile ID, schema path, and input mappings.
|
||||||
|
4. Verify required API-key environment variables.
|
||||||
|
5. Reproduce with `render --format json` when pre-LLM resolution is uncertain.
|
||||||
6. Rerun after correction.
|
6. Rerun after correction.
|
||||||
|
|
||||||
Because Scriptorium does not persist run state, rerun is the canonical recovery path.
|
Because Scriptorium does not persist run state, rerun is the supported recovery
|
||||||
|
path.
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ Scriptorium is a narrow prompt-execution application with three entry paths:
|
|||||||
- CLI `run`
|
- CLI `run`
|
||||||
- CLI `render`
|
- CLI `render`
|
||||||
- HTTP `POST /v1/runs` through `serve`
|
- HTTP `POST /v1/runs` through `serve`
|
||||||
|
- public Go package `gitea.maximumdirect.net/eric/scriptorium`
|
||||||
|
|
||||||
Domain behavior is centralized in `internal/usecase` and `internal/domain`.
|
Domain behavior is centralized in `internal/usecase` and `internal/domain`.
|
||||||
|
|
||||||
@@ -26,6 +27,7 @@ Domain behavior is centralized in `internal/usecase` and `internal/domain`.
|
|||||||
|
|
||||||
Current package map:
|
Current package map:
|
||||||
|
|
||||||
|
- root package `scriptorium`: public Go facade over engine construction, source options, request/result types, and error mapping.
|
||||||
- `cmd/scriptorium`: process entrypoint.
|
- `cmd/scriptorium`: process entrypoint.
|
||||||
- `internal/adapter/cli`: command parsing, app wiring for CLI commands, output behavior.
|
- `internal/adapter/cli`: command parsing, app wiring for CLI commands, output behavior.
|
||||||
- `internal/adapter/http`: HTTP DTO mapping and error/status mapping.
|
- `internal/adapter/http`: HTTP DTO mapping and error/status mapping.
|
||||||
@@ -34,7 +36,9 @@ Current package map:
|
|||||||
- `internal/domain`: core request/result and contract types.
|
- `internal/domain`: core request/result and contract types.
|
||||||
- `internal/usecase`: `Runner` prepare/run orchestration and repair-hook boundary.
|
- `internal/usecase`: `Runner` prepare/run orchestration and repair-hook boundary.
|
||||||
- `internal/promptdef`: filesystem prompt-definition repository.
|
- `internal/promptdef`: filesystem prompt-definition repository.
|
||||||
- `internal/profile`: filesystem execution-profile repository.
|
- `internal/profile`: filesystem, `fs.FS`, and overlay execution-profile repositories.
|
||||||
|
- `internal/profile/builtin`: embedded built-in execution profiles.
|
||||||
|
- `internal/filecatalog`: shared YAML discovery and `fs.FS` source helpers.
|
||||||
- `internal/artifact`: artifact reference readers.
|
- `internal/artifact`: artifact reference readers.
|
||||||
- `internal/prompt`: template renderer.
|
- `internal/prompt`: template renderer.
|
||||||
- `internal/llm`: provider-neutral LLM client interface and OpenAI-compatible implementation.
|
- `internal/llm`: provider-neutral LLM client interface and OpenAI-compatible implementation.
|
||||||
@@ -45,6 +49,7 @@ Detailed component behavior is documented in:
|
|||||||
|
|
||||||
- `docs/internal/runner.md`
|
- `docs/internal/runner.md`
|
||||||
- `docs/internal/adapters.md`
|
- `docs/internal/adapters.md`
|
||||||
|
- `docs/internal/sources.md`
|
||||||
|
|
||||||
## Configuration And Precedence
|
## Configuration And Precedence
|
||||||
|
|
||||||
@@ -69,9 +74,10 @@ Scriptorium has no durable run-state store.
|
|||||||
|
|
||||||
Current external contracts:
|
Current external contracts:
|
||||||
|
|
||||||
- inbound HTTP contract: `POST /v1/runs`
|
- inbound HTTP contract: `POST /v1/runs`, documented canonically in `docs/api.md`
|
||||||
- outbound model contract: OpenAI-compatible chat completions subset
|
- outbound model contract: OpenAI-compatible chat completions subset
|
||||||
- subprocess contract for integrators: CLI `run`/`render`
|
- subprocess contract for integrators: CLI `run`/`render`
|
||||||
|
- public Go package contract: `docs/consumers/pkg-scriptorium.md`
|
||||||
|
|
||||||
Integration docs belong under `docs/integrations/`.
|
Integration docs belong under `docs/integrations/`.
|
||||||
|
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ This document defines contributor workflow for Scriptorium.
|
|||||||
|
|
||||||
## Repository Layout
|
## Repository Layout
|
||||||
|
|
||||||
|
- root package `scriptorium`: public Go facade, options, types, and error mapping.
|
||||||
- `cmd/scriptorium`: application entrypoint.
|
- `cmd/scriptorium`: application entrypoint.
|
||||||
- `internal/domain`: core contracts.
|
- `internal/domain`: core contracts.
|
||||||
- `internal/usecase`: runner orchestration.
|
- `internal/usecase`: runner orchestration.
|
||||||
@@ -13,6 +14,8 @@ This document defines contributor workflow for Scriptorium.
|
|||||||
- `internal/defaults`: default constants.
|
- `internal/defaults`: default constants.
|
||||||
- `internal/promptdef`: prompt-definition repository.
|
- `internal/promptdef`: prompt-definition repository.
|
||||||
- `internal/profile`: execution-profile repository.
|
- `internal/profile`: execution-profile repository.
|
||||||
|
- `internal/profile/builtin`: embedded built-in execution profiles.
|
||||||
|
- `internal/filecatalog`: shared source discovery and path helpers.
|
||||||
- `internal/artifact`: artifact readers.
|
- `internal/artifact`: artifact readers.
|
||||||
- `internal/prompt`: prompt rendering.
|
- `internal/prompt`: prompt rendering.
|
||||||
- `internal/llm`: LLM client interface and OpenAI-compatible implementation.
|
- `internal/llm`: LLM client interface and OpenAI-compatible implementation.
|
||||||
@@ -38,7 +41,9 @@ go test ./...
|
|||||||
Targeted test runs commonly used during changes:
|
Targeted test runs commonly used during changes:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
go test .
|
||||||
go test ./internal/adapter/cli ./internal/adapter/http ./internal/usecase
|
go test ./internal/adapter/cli ./internal/adapter/http ./internal/usecase
|
||||||
|
go test ./internal/...
|
||||||
```
|
```
|
||||||
|
|
||||||
## Coding Conventions
|
## Coding Conventions
|
||||||
@@ -82,7 +87,8 @@ go test ./internal/adapter/cli ./internal/adapter/http ./internal/usecase
|
|||||||
3. Keep business decisions in `internal/usecase`.
|
3. Keep business decisions in `internal/usecase`.
|
||||||
4. Add focused adapter tests for mapping, parse, and error behavior.
|
4. Add focused adapter tests for mapping, parse, and error behavior.
|
||||||
5. Document the new/changed boundary in `docs/internal/adapters.md`.
|
5. Document the new/changed boundary in `docs/internal/adapters.md`.
|
||||||
6. If external contract changes, update `docs/integrations/` in the same change.
|
6. If source-loading behavior changes, update `docs/internal/sources.md`.
|
||||||
|
7. If an external contract changes, update the canonical public or integration doc in the same change.
|
||||||
|
|
||||||
## How To Update Prompt/Profile/Schema Assets
|
## How To Update Prompt/Profile/Schema Assets
|
||||||
|
|
||||||
@@ -99,5 +105,6 @@ When behavior changes:
|
|||||||
2. Keep non-roadmap docs limited to implemented behavior.
|
2. Keep non-roadmap docs limited to implemented behavior.
|
||||||
3. Update links after file moves/renames.
|
3. Update links after file moves/renames.
|
||||||
4. Re-run relevant tests and smoke commands.
|
4. Re-run relevant tests and smoke commands.
|
||||||
|
5. For internal boundary docs, check references with `rg "docs/internal|internal/sources" docs/policy docs/internal`.
|
||||||
|
|
||||||
Docs work is complete only when code/tests/examples/docs agree.
|
Docs work is complete only when code/tests/examples/docs agree.
|
||||||
|
|||||||
@@ -1,324 +0,0 @@
|
|||||||
# Full Codebase Cleanup And Hardening Plan
|
|
||||||
|
|
||||||
This document is a decision-complete implementation plan for the current full-codebase audit findings. It is a planning document only. Implementation should proceed in stages and should not change unrelated behavior.
|
|
||||||
|
|
||||||
The goal is to polish and harden the public and internal code paths without expanding Scriptorium's scope. Keep the existing package boundaries unless a stage explicitly calls for a helper extraction.
|
|
||||||
|
|
||||||
## Guiding Decisions
|
|
||||||
|
|
||||||
- Treat configured `fs.FS` roots as real containment boundaries for public source options.
|
|
||||||
- Keep local directory-backed CLI behavior compatible unless this plan explicitly names a change.
|
|
||||||
- Keep HTTP `serve` minimal, but safe by default against accidental large request, file, and response bodies.
|
|
||||||
- Do not add HTTP authentication in this cleanup pass.
|
|
||||||
- Do not add new dependencies.
|
|
||||||
- Document only implemented behavior outside `docs/roadmap/`.
|
|
||||||
|
|
||||||
## Stage 1: Enforce Source-Root Containment For `fs.FS` Prompt And Schema Sources
|
|
||||||
|
|
||||||
Problem:
|
|
||||||
|
|
||||||
Public source options such as `WithPromptFS(fsys, root)` and `WithSchemaFS(fsys, root)` describe `root` as the source boundary, but prompt `content_file` and schema path resolution can clean `..` paths above that root when the supplied `fs.FS` is broader than the configured root.
|
|
||||||
|
|
||||||
Decision:
|
|
||||||
|
|
||||||
For public `fs.FS` source options, the configured root is a containment boundary. Prompt `content_file` paths and schema paths must resolve inside that root. Absolute paths and relative traversal that escape the root are invalid.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
1. Add a small shared internal helper for `fs.FS` path resolution.
|
|
||||||
- Prefer `internal/filecatalog` if the helper naturally belongs with existing clean/display path utilities.
|
|
||||||
- Inputs should include a source root and a user path.
|
|
||||||
- It should trim whitespace, clean slash paths with `path.Clean`, reject empty paths where the caller requires a file, reject absolute paths, and reject any path whose clean form escapes the clean root.
|
|
||||||
- Use path-component checks, not string-prefix checks alone.
|
|
||||||
- Return both the resolved `fs.FS` path and a display path when useful for errors.
|
|
||||||
2. Update `internal/promptdef` `fsRepository` content-file loading.
|
|
||||||
- `content_file: ./local.tmpl` beside the prompt should continue to work.
|
|
||||||
- Nested prompt files should keep the existing relative-to-prompt-file behavior.
|
|
||||||
- `content_file` values that escape the configured `WithPromptFS` root should return a prompt-load error.
|
|
||||||
3. Update `internal/validate` `FSValidator` schema resolution.
|
|
||||||
- `WithSchemaFS(fsys, root)` should allow schema paths inside `root`.
|
|
||||||
- `WithSchemaFile(path)` should keep the existing single-file behavior: prompt `schema_path` must match the selected file's base name.
|
|
||||||
- Schema paths that escape the configured root should return validation/schema-load errors.
|
|
||||||
4. Preserve directory-backed compatibility unless a failing test reveals an inconsistency that must be fixed.
|
|
||||||
- `WithPromptFile(path)` should continue resolving `content_file` values relative to the selected prompt file's directory.
|
|
||||||
- `Config.SchemaDir` / CLI `--schema-dir` should keep documented behavior, including absolute `schema_path` support, because these are operator-controlled local filesystem paths.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Add `internal/promptdef` tests for `fs.FS` `content_file` traversal:
|
|
||||||
- sibling file inside root succeeds;
|
|
||||||
- nested file inside root succeeds;
|
|
||||||
- `../outside.tmpl` from a prompt under the root is rejected;
|
|
||||||
- absolute-style `/outside.tmpl` is rejected.
|
|
||||||
- Add public package tests through `WithPromptFS` proving escaped `content_file` returns `ErrPromptLoad`.
|
|
||||||
- Add `internal/validate` tests for `WithSchemaFS` traversal:
|
|
||||||
- schema inside root succeeds;
|
|
||||||
- `../outside.schema.json` is rejected;
|
|
||||||
- absolute-style paths are rejected.
|
|
||||||
- Keep existing `WithSchemaFile` tests passing.
|
|
||||||
|
|
||||||
Documentation:
|
|
||||||
|
|
||||||
- Update `docs/consumers/pkg-scriptorium.md` to state that `WithPromptFS` and `WithSchemaFS` roots are containment boundaries.
|
|
||||||
- Update `docs/config.md` only if directory-backed behavior changes. Otherwise leave its local-directory schema-path behavior intact.
|
|
||||||
- Update `docs/internal/adapters.md` if shared source-resolution behavior is documented there.
|
|
||||||
|
|
||||||
## Stage 2: Add HTTP Request, Artifact, And Response Size Limits
|
|
||||||
|
|
||||||
Problem:
|
|
||||||
|
|
||||||
HTTP `serve` decodes request bodies directly from `r.Body`, reads file artifacts fully into memory, and serializes generated artifact bodies fully into the response. This is acceptable for trusted small local use, but it is not hardened against accidental or hostile large inputs.
|
|
||||||
|
|
||||||
Decision:
|
|
||||||
|
|
||||||
Add configurable HTTP size limits with conservative defaults. Limits apply only to HTTP `serve`; CLI `run` and `render` keep existing direct filesystem behavior.
|
|
||||||
|
|
||||||
Default limits:
|
|
||||||
|
|
||||||
- `server.max_request_bytes`: 16 MiB.
|
|
||||||
- `server.max_artifact_bytes`: 16 MiB.
|
|
||||||
- `server.max_response_bytes`: 16 MiB.
|
|
||||||
|
|
||||||
Use `0` to disable a specific limit only where this is consistent with existing config style. Negative values are invalid config.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
1. Add default constants in `internal/defaults`.
|
|
||||||
2. Extend app config in `internal/config`.
|
|
||||||
- Add `server.max_request_bytes`.
|
|
||||||
- Add `server.max_artifact_bytes`.
|
|
||||||
- Add `server.max_response_bytes`.
|
|
||||||
- Apply built-in defaults, config-file values, and CLI overrides according to existing precedence.
|
|
||||||
- Reject negative values.
|
|
||||||
3. Add `serve` CLI overrides.
|
|
||||||
- `--max-request-bytes`
|
|
||||||
- `--max-artifact-bytes`
|
|
||||||
- `--max-response-bytes`
|
|
||||||
- Keep these flags scoped to `serve`.
|
|
||||||
4. Extend HTTP handler construction.
|
|
||||||
- Add `httpadapter.HandlerOptions` with request and response limit fields.
|
|
||||||
- Keep `httpadapter.NewHandler(runner)` as a default constructor for existing tests and callers.
|
|
||||||
- Add `httpadapter.NewHandlerWithOptions(runner, options)` for `serve` wiring.
|
|
||||||
- `NewHandler(runner)` should apply built-in defaults.
|
|
||||||
- `NewHandlerWithOptions(runner, options)` should use the supplied values exactly, so `0` means disabled after config validation.
|
|
||||||
5. Limit request decoding.
|
|
||||||
- Wrap `r.Body` with `http.MaxBytesReader` when `max_request_bytes > 0`.
|
|
||||||
- Return `413 request_too_large` when decoding fails due to size.
|
|
||||||
- Continue returning `400 invalid_json` for malformed JSON.
|
|
||||||
- Ensure the decoder rejects trailing JSON tokens if it does not already.
|
|
||||||
6. Limit HTTP file artifact reads.
|
|
||||||
- Add a max-bytes option to the restricted HTTP artifact reader.
|
|
||||||
- Use `os.Open`, `Stat`, and `io.LimitReader` or equivalent instead of unbounded `os.ReadFile` for restricted HTTP file reads.
|
|
||||||
- If a file exceeds the configured limit, return a specific artifact error that maps to `413 artifact_too_large`.
|
|
||||||
- Keep inline artifact bodies covered by the request-body limit.
|
|
||||||
7. Limit HTTP response artifact bodies.
|
|
||||||
- Build the response DTO, marshal it to JSON bytes, and compare the final encoded response size against `max_response_bytes` when the limit is positive.
|
|
||||||
- Return `413 response_too_large` when the encoded response exceeds the configured limit.
|
|
||||||
- Do not truncate successful artifacts silently.
|
|
||||||
- Apply the same encoded-response check when `include_raw_output` is true.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Config tests:
|
|
||||||
- defaults are applied;
|
|
||||||
- config file values load;
|
|
||||||
- CLI overrides win;
|
|
||||||
- negative values are rejected.
|
|
||||||
- CLI tests:
|
|
||||||
- `serve` parses the three flags;
|
|
||||||
- `run` and `render` do not gain these flags.
|
|
||||||
- HTTP tests:
|
|
||||||
- oversized request body returns `413 request_too_large`;
|
|
||||||
- malformed JSON below the limit still returns `400 invalid_json`;
|
|
||||||
- successful request below the limit still works;
|
|
||||||
- oversized generated artifact returns the configured too-large error;
|
|
||||||
- `include_raw_output` does not bypass response limits.
|
|
||||||
- Artifact tests:
|
|
||||||
- restricted file reader accepts files at or below the limit;
|
|
||||||
- restricted file reader rejects files above the limit;
|
|
||||||
- unlimited mode with `0` keeps existing behavior.
|
|
||||||
|
|
||||||
Documentation:
|
|
||||||
|
|
||||||
- Update `docs/config.md` with the new server limit fields and defaults.
|
|
||||||
- Update `docs/cli.md` with the new `serve` flags.
|
|
||||||
- Update `docs/integrations/http-api.md` with `413` errors.
|
|
||||||
- Update `docs/operations.md` with sizing guidance.
|
|
||||||
- Update `docs/troubleshooting.md` with common size-limit failures.
|
|
||||||
- Update `docs/internal/adapters.md` with the HTTP limit boundary.
|
|
||||||
|
|
||||||
## Stage 3: Harden `OpenAICompatibleProfile` ExtraParams Copying
|
|
||||||
|
|
||||||
Problem:
|
|
||||||
|
|
||||||
`OpenAICompatibleProfile` currently deep-copies `ExtraParams` through the general internal copy helper. Cyclic caller-provided maps can recurse indefinitely before `WithProfiles` can validate and return `ErrInvalidConfig`.
|
|
||||||
|
|
||||||
Decision:
|
|
||||||
|
|
||||||
`OpenAICompatibleProfile` is a convenience constructor, not a validator. It must not recursively walk caller-provided `ExtraParams`. Validation and safe deep copying belong in `WithProfiles` through the existing public JSON validation path.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
1. Change `OpenAICompatibleProfile` to use a shallow map copy for `ExtraParams`.
|
|
||||||
- Copy only the top-level `map[string]any`.
|
|
||||||
- Do not recursively copy nested values.
|
|
||||||
- Do not call `copyAnyMap` from this constructor.
|
|
||||||
2. Keep `WithProfiles` validation and deep-copy behavior unchanged.
|
|
||||||
- Unsupported values, non-finite numbers, non-string map keys, and cycles should still return `ErrInvalidConfig`.
|
|
||||||
3. Review `copyAnyMap` call sites.
|
|
||||||
- Keep it for trusted internal-to-public conversions where values come from already-decoded JSON-like data.
|
|
||||||
- Do not use it for untrusted public caller input before validation.
|
|
||||||
4. Add a short comment near the constructor if needed to clarify that recursive validation is intentionally deferred.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Add a public package test where `OpenAICompatibleProfile` receives cyclic `ExtraParams`.
|
|
||||||
- The constructor must return promptly.
|
|
||||||
- `NewEngine(..., WithProfiles(profile))` must return `ErrInvalidConfig`.
|
|
||||||
- Add a test proving non-cyclic nested `ExtraParams` still work through `WithProfiles`.
|
|
||||||
- Keep existing mutation-isolation tests passing.
|
|
||||||
|
|
||||||
Documentation:
|
|
||||||
|
|
||||||
- No user-facing behavior change is required if existing docs already state that `WithProfiles` validates `ExtraParams`.
|
|
||||||
- Update docs only if constructor behavior is currently described as validating or deep-copying recursively.
|
|
||||||
|
|
||||||
## Stage 4: Redact Provider Non-2xx Response Bodies From Default Errors
|
|
||||||
|
|
||||||
Problem:
|
|
||||||
|
|
||||||
The OpenAI-compatible client includes a trimmed provider response body snippet in `ErrUnexpectedStatus`. CLI and library callers may log this error. Provider error bodies can include prompt fragments, schema details, request IDs, or other sensitive operational data.
|
|
||||||
|
|
||||||
Decision:
|
|
||||||
|
|
||||||
Default errors should include the provider status code but not the response body. Do not add a debug mode in this pass unless an existing debug/logging surface already supports it.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
1. Change the non-2xx error returned by `internal/llm.OpenAICompatibleClient`.
|
|
||||||
- Keep wrapping `ErrUnexpectedStatus`.
|
|
||||||
- Include `status=<code>`.
|
|
||||||
- Do not include response body text.
|
|
||||||
2. Drain and close the response body safely enough for normal HTTP client reuse.
|
|
||||||
- It is acceptable to read and discard a small bounded amount if needed.
|
|
||||||
- Do not store or return the discarded content.
|
|
||||||
3. Review tests that assert the old body-snippet behavior and update them.
|
|
||||||
4. Review CLI, HTTP, and public package error mapping.
|
|
||||||
- HTTP should remain generic and not leak provider details.
|
|
||||||
- CLI/library errors should retain enough status context to diagnose provider failures.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Update `internal/llm` non-2xx tests:
|
|
||||||
- `errors.Is(err, ErrUnexpectedStatus)` remains true;
|
|
||||||
- the status code appears in the error string;
|
|
||||||
- the provider response body does not appear in the error string.
|
|
||||||
- Add a regression test with a body containing distinctive sensitive-looking text and assert it is absent.
|
|
||||||
|
|
||||||
Documentation:
|
|
||||||
|
|
||||||
- Update `docs/integrations/openai-compatible-chat.md` to remove the claim that `ErrUnexpectedStatus` includes a response body snippet.
|
|
||||||
- Update troubleshooting docs only if they currently instruct users to inspect provider body snippets.
|
|
||||||
|
|
||||||
## Stage 5: Clarify Artifact-Root Symlink Semantics
|
|
||||||
|
|
||||||
Problem:
|
|
||||||
|
|
||||||
The restricted HTTP artifact reader uses lexical path containment before reading the file. Symlinks inside the artifact root are followed by the operating system. This is documented, but the phrase "must stay inside the root" can be overread as a strict realpath guarantee.
|
|
||||||
|
|
||||||
Decision:
|
|
||||||
|
|
||||||
Keep symlink-following behavior for this cleanup pass, but make the code and docs explicit that containment is lexical and relies on the artifact root not being writable by untrusted users. This avoids a potentially breaking change for deployments that intentionally use symlinks.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
1. Rename or comment the restricted reader's path-resolution helper to make the lexical nature clear.
|
|
||||||
2. Add tests documenting current symlink behavior where the platform supports symlinks.
|
|
||||||
- A symlink inside the root to a file outside the root is followed.
|
|
||||||
- The test should skip cleanly if symlink creation is unavailable.
|
|
||||||
3. Keep traversal rejection tests for `..` and absolute paths outside the root.
|
|
||||||
4. Do not introduce `filepath.EvalSymlinks` in this pass.
|
|
||||||
|
|
||||||
Documentation:
|
|
||||||
|
|
||||||
- Update `docs/config.md`, `docs/operations.md`, `docs/integrations/http-api.md`, and `docs/internal/adapters.md` to say:
|
|
||||||
- lexical traversal outside the root is rejected;
|
|
||||||
- symlinks inside the root are followed;
|
|
||||||
- the artifact root must not be writable by untrusted users.
|
|
||||||
- Avoid wording that implies strict realpath containment unless the implementation changes to enforce it.
|
|
||||||
|
|
||||||
Future option:
|
|
||||||
|
|
||||||
If strict filesystem containment becomes required, add an opt-in or replacement mode that resolves both the configured root and requested file with `filepath.EvalSymlinks` before reading, rejects symlink escapes, and documents any compatibility impact.
|
|
||||||
|
|
||||||
## Stage 6: Align CLI Help And Documentation
|
|
||||||
|
|
||||||
Problem:
|
|
||||||
|
|
||||||
The `serve` usage text omits `--artifact-root`, even though the flag exists. New limit flags from Stage 2 also need to appear consistently in CLI help and docs.
|
|
||||||
|
|
||||||
Decision:
|
|
||||||
|
|
||||||
Keep CLI help concise but complete for supported flags.
|
|
||||||
|
|
||||||
Implementation:
|
|
||||||
|
|
||||||
1. Update `printUsage` in `internal/adapter/cli`.
|
|
||||||
- Include `--artifact-root DIR` in the `serve` usage line.
|
|
||||||
- Include the new size-limit flags from Stage 2.
|
|
||||||
- Keep the line readable; splitting long usage text into multiple lines is acceptable if tests are updated.
|
|
||||||
2. Update CLI tests that assert usage output.
|
|
||||||
3. Confirm `docs/cli.md` matches actual flags.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- Add or update CLI usage tests to assert that `serve` help mentions:
|
|
||||||
- `--artifact-root`;
|
|
||||||
- `--max-request-bytes`;
|
|
||||||
- `--max-artifact-bytes`;
|
|
||||||
- `--max-response-bytes`.
|
|
||||||
|
|
||||||
Documentation:
|
|
||||||
|
|
||||||
- Update `docs/cli.md` and any command examples affected by line wrapping or flag additions.
|
|
||||||
|
|
||||||
## Stage 7: Final Verification
|
|
||||||
|
|
||||||
Run the full verification set after all stages:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
go test ./...
|
|
||||||
go vet ./...
|
|
||||||
go run ./examples/go-library/prepare
|
|
||||||
go run ./cmd/scriptorium render \
|
|
||||||
--config ./examples/config.yml \
|
|
||||||
--prompt generic.markdown_summary \
|
|
||||||
--input transcript=./examples/fixtures/transcript.md \
|
|
||||||
--input glossary=./examples/fixtures/glossary.yml \
|
|
||||||
--format json
|
|
||||||
```
|
|
||||||
|
|
||||||
Also run targeted packages while implementing each stage:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
go test ./internal/promptdef ./internal/validate ./internal/filecatalog
|
|
||||||
go test ./internal/artifact ./internal/adapter/http ./internal/adapter/cli ./internal/config
|
|
||||||
go test .
|
|
||||||
```
|
|
||||||
|
|
||||||
## Non-Goals
|
|
||||||
|
|
||||||
- Do not collapse internal packages merely to reduce package count.
|
|
||||||
- Do not add HTTP authentication or authorization.
|
|
||||||
- Do not remove HTTP file inputs.
|
|
||||||
- Do not change public request/result type names or method signatures.
|
|
||||||
- Do not change CLI `run` or `render` local file-input behavior.
|
|
||||||
- Do not silently truncate request, artifact, provider, or response bodies.
|
|
||||||
- Do not accept raw API keys through config files, profile files, CLI flags, or HTTP payloads.
|
|
||||||
|
|
||||||
## Assumptions
|
|
||||||
|
|
||||||
- Public `fs.FS` roots are intended to be narrower than the supplied filesystem and should therefore be enforced.
|
|
||||||
- The initial HTTP size-limit defaults are intentionally conservative and can be tuned by operators.
|
|
||||||
- Provider response-body diagnostics are less important than safe default error handling.
|
|
||||||
- Symlink compatibility is more important than strict realpath containment for the immediate cleanup pass, provided documentation is explicit.
|
|
||||||
@@ -1,19 +1,25 @@
|
|||||||
# Troubleshooting
|
# Troubleshooting
|
||||||
|
|
||||||
This guide lists recurring implemented failure modes and safe fixes.
|
This guide lists common implemented failure modes and safe fixes.
|
||||||
|
|
||||||
For command syntax, see [CLI reference](cli.md). For configuration and file formats, see [Configuration reference](config.md). For operational behavior, see [Operations guide](operations.md).
|
Canonical references:
|
||||||
|
|
||||||
## Missing Or Invalid Config File
|
- [CLI reference](cli.md)
|
||||||
|
- [Configuration reference](config.md)
|
||||||
|
- [HTTP API reference](api.md)
|
||||||
|
- [Operations guide](operations.md)
|
||||||
|
|
||||||
|
## Missing Or Invalid Config
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|
||||||
- CLI errors such as `application config error: config file not found` or `invalid config YAML`.
|
- CLI error includes `application config error`, `config file not found`, `invalid config YAML`, or `invalid config`.
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
- `--config` points to a missing file.
|
- `--config` points to a missing file.
|
||||||
- Config YAML has syntax errors or unknown fields.
|
- YAML syntax is invalid.
|
||||||
|
- Config contains unknown fields or negative HTTP size limits.
|
||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
@@ -23,40 +29,34 @@ go run ./cmd/scriptorium render --config /path/to/config.yml --prompt generic.ma
|
|||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Correct file path.
|
- Correct the config path.
|
||||||
- Remove unknown fields.
|
|
||||||
- Fix YAML syntax.
|
- Fix YAML syntax.
|
||||||
- Keep secrets out of config.
|
- Remove unknown fields.
|
||||||
|
- Keep raw secrets out of config.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [Configuration reference](config.md), [CLI reference](cli.md)
|
||||||
|
|
||||||
- [Configuration reference](config.md)
|
## Missing Prompt Directory
|
||||||
- [CLI reference](cli.md)
|
|
||||||
|
|
||||||
## Missing Prompt Directory Settings
|
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|
||||||
- CLI parse errors saying prompt directory is required.
|
- CLI parse error says the prompt directory is required.
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
- Neither CLI flags nor config provide an effective `prompt_dir`.
|
- Neither config nor CLI flags provide an effective `prompt_dir`.
|
||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
- Run the failing command with explicit `--prompt-dir` once to verify.
|
- Re-run once with explicit `--prompt-dir`.
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Set `prompt_dir` in config, or always pass `--prompt-dir`.
|
- Set `prompt_dir` in config or pass `--prompt-dir`.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [Configuration reference](config.md), [CLI reference](cli.md)
|
||||||
|
|
||||||
- [Configuration reference](config.md)
|
## Unknown Flags
|
||||||
- [CLI reference](cli.md)
|
|
||||||
|
|
||||||
## Unknown Or Unsupported Flags
|
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|
||||||
@@ -64,33 +64,33 @@ Symptom:
|
|||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
- Typo or command mismatch (for example, `serve` with runtime model override flags).
|
- Typo.
|
||||||
|
- Flag is valid for another command.
|
||||||
|
- `serve` was given runtime model override flags.
|
||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
- Compare command against the command-specific flag list.
|
- Compare the command with the command-specific flag list.
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Remove unsupported flags.
|
- Remove unsupported flags.
|
||||||
- Use `run`/`render` for runtime model overrides.
|
- Use `run` or `render` for runtime model overrides.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [CLI reference](cli.md)
|
||||||
|
|
||||||
- [CLI reference](cli.md)
|
## Prompt Load Failures
|
||||||
|
|
||||||
## Prompt Definition Load Failures
|
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|
||||||
- CLI run/render error from prompt loading.
|
- CLI run/render fails during prompt loading.
|
||||||
- HTTP `404 prompt_not_found` or `400 prompt_load_failed`.
|
- HTTP returns `404 prompt_not_found` or `400 prompt_load_failed`.
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
- Prompt ID not found.
|
- Prompt ID/version does not exist.
|
||||||
- Invalid prompt YAML.
|
- Prompt YAML is invalid or has unknown fields.
|
||||||
- Invalid prompt contract (for example bad validation mode, message content/content_file rule violation, missing schema path for `json_schema`).
|
- Prompt contract is invalid, such as missing messages, invalid output mode, bad `content_file`, or missing `schema_path` for `json_schema`.
|
||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
@@ -100,28 +100,25 @@ go run ./cmd/scriptorium render --config ./examples/config.yml --prompt <prompt-
|
|||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Correct prompt ID.
|
- Correct prompt ID/version.
|
||||||
- Fix prompt YAML and contract fields.
|
- Fix prompt YAML and referenced `content_file` paths.
|
||||||
- Ensure referenced `content_file` paths exist.
|
- Fix output contract fields.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [Configuration reference](config.md), [CLI reference](cli.md)
|
||||||
|
|
||||||
- [Configuration reference](config.md)
|
## Profile Load Failures
|
||||||
- [CLI reference](cli.md)
|
|
||||||
|
|
||||||
## Profile Definition Load Failures
|
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|
||||||
- CLI run/render error from profile loading.
|
- CLI run/render fails during profile loading.
|
||||||
- HTTP `404 profile_not_found` or `400 profile_load_failed`.
|
- HTTP returns `404 profile_not_found`, `400 profile_load_failed`, or `400 profile_required`.
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
- Profile ID missing/not found.
|
- Profile ID does not exist.
|
||||||
- Invalid profile YAML.
|
- Request omitted profile and prompt has no `default_profile`.
|
||||||
- Invalid profile values.
|
- Profile YAML is invalid or has unknown fields.
|
||||||
- Raw `api_key` field present (rejected).
|
- Profile contains raw `api_key`.
|
||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
@@ -131,85 +128,52 @@ go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.
|
|||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Correct profile ID.
|
- Correct profile ID or prompt `default_profile`.
|
||||||
- Fix profile YAML and value ranges.
|
- Fix profile YAML and value ranges.
|
||||||
- Replace `api_key` with `api_key_env`.
|
- Replace raw `api_key` with `api_key_env`.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [Configuration reference](config.md), [CLI reference](cli.md)
|
||||||
|
|
||||||
- [Configuration reference](config.md)
|
## Input Artifact Failures
|
||||||
- [CLI reference](cli.md)
|
|
||||||
|
|
||||||
## Input Artifact Read Failures
|
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|
||||||
- CLI run/render error reading input artifacts.
|
- CLI run/render fails while reading inputs.
|
||||||
- HTTP `400 artifact_read_failed`.
|
- HTTP returns `400 artifact_read_failed`, `400 artifact_not_allowed`, or `413 artifact_too_large`.
|
||||||
- HTTP `400 artifact_not_allowed`.
|
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
- File path in input mapping does not exist or is unreadable.
|
- Input file path is missing or unreadable.
|
||||||
- Unsupported artifact reference type in HTTP request.
|
- HTTP input type is unsupported or missing required fields.
|
||||||
- HTTP `file` input references are disabled because no artifact root is configured.
|
- HTTP file refs are disabled because no artifact root is configured.
|
||||||
- HTTP `file` input path escapes the configured artifact root.
|
- HTTP file path is lexically outside the artifact root.
|
||||||
|
- HTTP file input exceeds `server.max_artifact_bytes`.
|
||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
- Verify every mapped file path exists and is readable by the process.
|
- Verify each input path exists and is readable by the process.
|
||||||
- For HTTP, verify each input uses supported `type` values.
|
- For HTTP, verify input refs use `file` or `inline`.
|
||||||
- For HTTP `file` inputs, verify `server.artifact_root` or `serve --artifact-root` is configured and the requested path stays inside that root.
|
- For HTTP file refs, verify the artifact root and compare file size to `server.max_artifact_bytes`.
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Correct file paths and permissions.
|
- Correct paths and permissions.
|
||||||
- Use supported input types (`file`, `inline`).
|
- Configure a narrow artifact root for HTTP file refs.
|
||||||
- Configure a narrow HTTP artifact root when HTTP file inputs are required.
|
- Use relative paths under the artifact root or switch to `inline`.
|
||||||
- Use relative paths under the artifact root, or switch to `inline` inputs.
|
- Increase `server.max_artifact_bytes` only for expected larger inputs.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [HTTP API reference](api.md), [Configuration reference](config.md)
|
||||||
|
|
||||||
- [CLI reference](cli.md)
|
|
||||||
- [Configuration reference](config.md)
|
|
||||||
- [HTTP API integration](integrations/http-api.md)
|
|
||||||
|
|
||||||
## Prompt Template Render Failures
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
|
|
||||||
- CLI run/render error from prompt rendering.
|
|
||||||
- HTTP `400 prompt_render_failed`.
|
|
||||||
|
|
||||||
Likely cause:
|
|
||||||
|
|
||||||
- Template references missing input names.
|
|
||||||
- Template syntax or data reference issues.
|
|
||||||
|
|
||||||
Diagnostic step:
|
|
||||||
|
|
||||||
- Run `render --format json` with the same prompt, inputs, vars, and profile selection.
|
|
||||||
|
|
||||||
Safe fix:
|
|
||||||
|
|
||||||
- Align template `{{input "name"}}` references with actual input mappings.
|
|
||||||
- Fix template syntax and variable names.
|
|
||||||
|
|
||||||
Relevant links:
|
|
||||||
|
|
||||||
- [CLI reference](cli.md)
|
|
||||||
- [Configuration reference](config.md)
|
|
||||||
|
|
||||||
## Missing API-Key Environment Variable
|
## Missing API-Key Environment Variable
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|
||||||
- CLI run/render invalid request error about missing API-key environment variable.
|
- CLI render/run fails with an API-key environment error.
|
||||||
- HTTP `400 api_key_env_missing`.
|
- HTTP returns `400 api_key_env_missing`.
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
- Selected profile or override sets `api_key_env`, but that environment variable is unset/empty.
|
- Selected profile or runtime override sets `api_key_env`, but the environment variable is unset or empty.
|
||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
@@ -219,85 +183,68 @@ printenv SCRIPTORIUM_API_KEY
|
|||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Set the required environment variable before invoking CLI/service.
|
- Set the required environment variable before starting the CLI command or HTTP service.
|
||||||
- Or use a profile that does not require API key auth for the target endpoint.
|
- Or use a profile that does not require provider API-key auth.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [Configuration reference](config.md), [Operations guide](operations.md)
|
||||||
|
|
||||||
- [Configuration reference](config.md)
|
## Prompt Template Render Failures
|
||||||
- [Operations guide](operations.md)
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- CLI render/run fails during prompt rendering.
|
||||||
|
- HTTP returns `400 prompt_render_failed`.
|
||||||
|
|
||||||
|
Likely cause:
|
||||||
|
|
||||||
|
- Template references an input that was not supplied.
|
||||||
|
- Template syntax or variable reference is invalid.
|
||||||
|
|
||||||
|
Diagnostic step:
|
||||||
|
|
||||||
|
- Run `render --format json` with the same prompt, inputs, vars, and profile.
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- Align `{{input "name"}}` references with request input names.
|
||||||
|
- Fix template syntax and variable names.
|
||||||
|
|
||||||
|
Relevant links: [Configuration reference](config.md), [CLI reference](cli.md)
|
||||||
|
|
||||||
## LLM Request Failures
|
## LLM Request Failures
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|
||||||
- CLI `run` fails with LLM generation errors.
|
- CLI `run` fails during generation.
|
||||||
- HTTP returns `502 llm_failed`.
|
- HTTP returns `502 llm_failed`.
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
- Endpoint unreachable.
|
- Endpoint is unreachable.
|
||||||
- Non-2xx response from provider.
|
- Provider returns non-2xx.
|
||||||
- Timeout.
|
- Request times out.
|
||||||
- Malformed provider response.
|
- Provider response is malformed.
|
||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
- Confirm endpoint URL and model in selected profile/overrides.
|
- Run `render` first to confirm pre-LLM preparation works.
|
||||||
- Retry with `render` first to confirm pre-LLM preparation works.
|
- Check selected endpoint/model in prepared output.
|
||||||
- Check provider/network logs for non-2xx responses and timeouts.
|
- Check network/provider logs for timeout or non-2xx details.
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Correct endpoint/model settings.
|
- Correct endpoint/model/profile settings.
|
||||||
- Adjust timeout if needed.
|
- Adjust timeout when appropriate.
|
||||||
- Resolve provider-side or network issues.
|
- Resolve provider or network issue.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [Operations guide](operations.md), [Configuration reference](config.md)
|
||||||
|
|
||||||
- [CLI reference](cli.md)
|
## Validation Failed
|
||||||
- [Configuration reference](config.md)
|
|
||||||
- [Operations guide](operations.md)
|
|
||||||
|
|
||||||
## Prompt Cache Misses Or No Cache Usage
|
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|
||||||
- CLI run summary omits `cached_tokens` / `cache_write_tokens`.
|
- CLI `run` exits `2`.
|
||||||
- HTTP `metadata.usage.cached_tokens` and `metadata.usage.cache_write_tokens` are both `0`.
|
- HTTP returns `200 OK` with `validation.status` set to `failed`.
|
||||||
- Provider cost or latency does not improve after repeated similar runs.
|
|
||||||
|
|
||||||
Likely cause:
|
|
||||||
|
|
||||||
- The selected prompt has no `messages[].cache_control`.
|
|
||||||
- Dynamic per-run input appears before the cache-controlled message and changes the provider cache key.
|
|
||||||
- The provider does not support the serialized cache-control shape for the selected model.
|
|
||||||
- The provider imposes minimum token thresholds or cache-breakpoint limits.
|
|
||||||
|
|
||||||
Diagnostic step:
|
|
||||||
|
|
||||||
- Run `render --format json` and verify the intended rendered message includes `cache_control`.
|
|
||||||
- Confirm stable reusable context appears before the cache-controlled message, with dynamic input after it.
|
|
||||||
- Check provider docs/logs for model support, minimum token thresholds, and breakpoint limits.
|
|
||||||
|
|
||||||
Safe fix:
|
|
||||||
|
|
||||||
- Move stable reusable context before the cache-controlled message.
|
|
||||||
- Move highly dynamic input after the cache breakpoint.
|
|
||||||
- Keep `cache_control.type: ephemeral` and, when using `ttl`, set `ttl: 1h`.
|
|
||||||
- Use CLI cache counters or HTTP cache usage fields to verify cache reads/writes after rerunning.
|
|
||||||
|
|
||||||
Relevant links:
|
|
||||||
|
|
||||||
- [Configuration reference](config.md)
|
|
||||||
- [OpenAI-compatible chat integration](integrations/openai-compatible-chat.md)
|
|
||||||
|
|
||||||
## Validation Status Failed (`run` Exit 2 Or HTTP 200 With Failed Status)
|
|
||||||
|
|
||||||
Symptom:
|
|
||||||
|
|
||||||
- CLI exits with code `2`.
|
|
||||||
- HTTP returns `200`, but `validation.status` is `failed`.
|
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
@@ -305,18 +252,15 @@ Likely cause:
|
|||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
- Inspect validation mode and validation errors in CLI summary/HTTP response.
|
- Inspect validation errors in CLI stderr or the HTTP response.
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Refine prompt constraints.
|
- Refine prompt instructions.
|
||||||
- Tighten schema or adjust model/profile settings.
|
- Adjust schema or model/profile settings.
|
||||||
- Rerun after correction.
|
- Rerun after correction.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [Operations guide](operations.md), [HTTP API reference](api.md)
|
||||||
|
|
||||||
- [Configuration reference](config.md)
|
|
||||||
- [Operations guide](operations.md)
|
|
||||||
|
|
||||||
## Validation Runtime Failure
|
## Validation Runtime Failure
|
||||||
|
|
||||||
@@ -327,48 +271,91 @@ Symptom:
|
|||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
- `json_schema` schema file missing/inaccessible.
|
- `json_schema` schema file is missing or unreadable.
|
||||||
- Invalid schema JSON document.
|
- Schema JSON is invalid.
|
||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
- Verify `schema_dir` and `output.schema_path` resolution.
|
- Verify `schema_dir` and prompt `output.schema_path`.
|
||||||
- Check schema file readability and valid JSON syntax.
|
- Check schema file readability and JSON syntax.
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Correct schema path.
|
- Correct schema path or permissions.
|
||||||
- Fix schema JSON content.
|
- Fix schema JSON.
|
||||||
- Rerun.
|
- Rerun.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [Configuration reference](config.md), [Operations guide](operations.md)
|
||||||
|
|
||||||
- [Configuration reference](config.md)
|
## HTTP JSON Or Request Contract Errors
|
||||||
- [Operations guide](operations.md)
|
|
||||||
|
|
||||||
## HTTP Request Parsing/Contract Errors
|
|
||||||
|
|
||||||
Symptom:
|
Symptom:
|
||||||
|
|
||||||
- HTTP `400 invalid_json` or `400 invalid_request`.
|
- HTTP returns `400 invalid_json` or `400 invalid_request`.
|
||||||
|
|
||||||
Likely cause:
|
Likely cause:
|
||||||
|
|
||||||
- Malformed JSON body.
|
- JSON body is malformed.
|
||||||
- Unknown JSON fields.
|
- Request has unknown fields or trailing JSON tokens.
|
||||||
- Missing required `prompt_id` or `inputs`.
|
- Required `prompt_id` or `inputs` is missing.
|
||||||
|
- Runtime override values are out of range.
|
||||||
|
- `extra_params` collides with reserved outbound fields.
|
||||||
|
|
||||||
Diagnostic step:
|
Diagnostic step:
|
||||||
|
|
||||||
- Revalidate request JSON.
|
- Revalidate request JSON and compare fields with the API reference.
|
||||||
- Confirm required request fields are present.
|
|
||||||
|
|
||||||
Safe fix:
|
Safe fix:
|
||||||
|
|
||||||
- Send valid JSON with only supported fields.
|
- Send one JSON object with only supported fields.
|
||||||
- Ensure `prompt_id` and at least one input mapping are included.
|
- Include `prompt_id` and at least one input.
|
||||||
|
- Use valid model override ranges.
|
||||||
|
- Remove reserved `extra_params` keys.
|
||||||
|
|
||||||
Relevant links:
|
Relevant links: [HTTP API reference](api.md)
|
||||||
|
|
||||||
- [Operations guide](operations.md)
|
## HTTP Size Limit Errors
|
||||||
- [CLI reference](cli.md)
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- HTTP returns `413 request_too_large`, `413 artifact_too_large`, or `413 response_too_large`.
|
||||||
|
|
||||||
|
Likely cause:
|
||||||
|
|
||||||
|
- JSON request body exceeds `server.max_request_bytes`.
|
||||||
|
- HTTP file input exceeds `server.max_artifact_bytes`.
|
||||||
|
- Encoded JSON response exceeds `server.max_response_bytes`.
|
||||||
|
|
||||||
|
Diagnostic step:
|
||||||
|
|
||||||
|
- Compare request, file input, and expected response sizes with configured limits.
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- Use smaller inline inputs or switch to file inputs under the artifact root.
|
||||||
|
- Reduce generated output size.
|
||||||
|
- Omit `include_raw_output`.
|
||||||
|
- Increase limits only when the deployment expects larger payloads.
|
||||||
|
|
||||||
|
Relevant links: [HTTP API reference](api.md), [Operations guide](operations.md)
|
||||||
|
|
||||||
|
## HTTP Route Or Method Errors
|
||||||
|
|
||||||
|
Symptom:
|
||||||
|
|
||||||
|
- HTTP returns `404 not_found` or `405 method_not_allowed`.
|
||||||
|
|
||||||
|
Likely cause:
|
||||||
|
|
||||||
|
- Path is not `/v1/runs`.
|
||||||
|
- Method on `/v1/runs` is not `POST`.
|
||||||
|
|
||||||
|
Diagnostic step:
|
||||||
|
|
||||||
|
- Check the request URL and method.
|
||||||
|
|
||||||
|
Safe fix:
|
||||||
|
|
||||||
|
- Send `POST /v1/runs`.
|
||||||
|
|
||||||
|
Relevant links: [HTTP API reference](api.md)
|
||||||
|
|||||||
107
engine_test.go
107
engine_test.go
@@ -881,6 +881,38 @@ output:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestPrepareWithPromptFSRejectsEscapedContentFile(t *testing.T) {
|
||||||
|
promptFS := fstest.MapFS{
|
||||||
|
"assets/prompts/fs-escape.yaml": &fstest.MapFile{Data: []byte(`
|
||||||
|
id: fs.escape
|
||||||
|
version: "1.0.0"
|
||||||
|
default_profile: local-fast
|
||||||
|
messages:
|
||||||
|
- role: user
|
||||||
|
content_file: ../outside.tmpl
|
||||||
|
output:
|
||||||
|
format: text
|
||||||
|
validation_mode: none
|
||||||
|
repair_attempts: 0
|
||||||
|
`)},
|
||||||
|
"assets/outside.tmpl": &fstest.MapFile{Data: []byte(`Outside root.`)},
|
||||||
|
}
|
||||||
|
|
||||||
|
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||||
|
PromptDir: t.TempDir(),
|
||||||
|
ProfileDir: "./examples/profiles",
|
||||||
|
SchemaDir: "./examples/schemas",
|
||||||
|
}, scriptorium.WithPromptFS(promptFS, "assets/prompts"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected engine construction to succeed, got %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err = engine.Prepare(context.Background(), scriptorium.RunRequest{PromptID: "fs.escape"})
|
||||||
|
if !errors.Is(err, scriptorium.ErrPromptLoad) {
|
||||||
|
t.Fatalf("expected ErrPromptLoad, got %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestPrepareWorksWithPromptFile(t *testing.T) {
|
func TestPrepareWorksWithPromptFile(t *testing.T) {
|
||||||
promptDir := t.TempDir()
|
promptDir := t.TempDir()
|
||||||
promptPath := filepath.Join(promptDir, "single.yaml")
|
promptPath := filepath.Join(promptDir, "single.yaml")
|
||||||
@@ -893,7 +925,7 @@ inputs:
|
|||||||
required: true
|
required: true
|
||||||
messages:
|
messages:
|
||||||
- role: user
|
- role: user
|
||||||
content: "Summarize {{input \"transcript\"}} from file."
|
content_file: ./single.tmpl
|
||||||
output:
|
output:
|
||||||
format: text
|
format: text
|
||||||
validation_mode: none
|
validation_mode: none
|
||||||
@@ -901,6 +933,9 @@ output:
|
|||||||
`), 0o644); err != nil {
|
`), 0o644); err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(promptDir, "single.tmpl"), []byte(`Summarize {{input "transcript"}} from file.`), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||||
ProfileDir: "./examples/profiles",
|
ProfileDir: "./examples/profiles",
|
||||||
@@ -922,6 +957,9 @@ output:
|
|||||||
if prepared.PromptID != "single.file.prompt" {
|
if prepared.PromptID != "single.file.prompt" {
|
||||||
t.Fatalf("unexpected prompt id: %q", prepared.PromptID)
|
t.Fatalf("unexpected prompt id: %q", prepared.PromptID)
|
||||||
}
|
}
|
||||||
|
if len(prepared.Messages) != 1 || !strings.Contains(prepared.Messages[0].Content, "from file") {
|
||||||
|
t.Fatalf("expected content_file body from prompt file, got %+v", prepared.Messages)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPrepareWorksWithProfileFSOverBuiltIns(t *testing.T) {
|
func TestPrepareWorksWithProfileFSOverBuiltIns(t *testing.T) {
|
||||||
@@ -1116,6 +1154,73 @@ func TestOpenAICompatibleProfileRunsThroughNormalProfilePath(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestOpenAICompatibleProfileDefersExtraParamsValidation(t *testing.T) {
|
||||||
|
cyclic := map[string]any{}
|
||||||
|
cyclic["self"] = cyclic
|
||||||
|
|
||||||
|
prof := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
|
||||||
|
ID: "cyclic-template-profile",
|
||||||
|
Endpoint: "http://cyclic-template/v1",
|
||||||
|
Model: "cyclic-template-model",
|
||||||
|
ExtraParams: cyclic,
|
||||||
|
})
|
||||||
|
|
||||||
|
_, err := scriptorium.NewEngine(scriptorium.Config{PromptDir: "./examples/prompts"},
|
||||||
|
scriptorium.WithProfiles(prof),
|
||||||
|
)
|
||||||
|
if !errors.Is(err, scriptorium.ErrInvalidConfig) {
|
||||||
|
t.Fatalf("expected ErrInvalidConfig, got %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOpenAICompatibleProfileNestedExtraParamsRunThroughWithProfiles(t *testing.T) {
|
||||||
|
fake := &fakeLLMClient{response: &scriptorium.GenerateResponse{Content: "ok"}}
|
||||||
|
nested := map[string]any{
|
||||||
|
"labels": map[string]string{"route": "primary"},
|
||||||
|
"ids": []int{1, 2, 3},
|
||||||
|
}
|
||||||
|
extraParams := map[string]any{
|
||||||
|
"nested": nested,
|
||||||
|
}
|
||||||
|
prof := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
|
||||||
|
ID: "nested-template-profile",
|
||||||
|
Endpoint: "http://nested-template/v1",
|
||||||
|
Model: "nested-template-model",
|
||||||
|
ExtraParams: extraParams,
|
||||||
|
})
|
||||||
|
extraParams["added"] = "mutated-after-construction"
|
||||||
|
|
||||||
|
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||||
|
PromptDir: "./examples/prompts",
|
||||||
|
SchemaDir: "./examples/schemas",
|
||||||
|
}, scriptorium.WithProfiles(prof), scriptorium.WithLLMClient(fake))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected engine construction to succeed, got %v", err)
|
||||||
|
}
|
||||||
|
nested["added"] = "mutated-after-construction"
|
||||||
|
|
||||||
|
_, err = engine.Run(context.Background(), scriptorium.RunRequest{
|
||||||
|
PromptID: "generic.markdown_summary",
|
||||||
|
ProfileID: "nested-template-profile",
|
||||||
|
Inputs: map[string]scriptorium.ArtifactRef{
|
||||||
|
"transcript": scriptorium.Inline("Rin opens the gate."),
|
||||||
|
"glossary": scriptorium.Inline("gate: A guarded passage."),
|
||||||
|
},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected run to succeed, got %v", err)
|
||||||
|
}
|
||||||
|
want := map[string]any{
|
||||||
|
"nested": map[string]any{
|
||||||
|
"labels": map[string]string{"route": "primary"},
|
||||||
|
"ids": []int{1, 2, 3},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(fake.requests[0].Target.ExtraParams, want) {
|
||||||
|
t.Fatalf("unexpected extra params:\ngot=%#v\nwant=%#v", fake.requests[0].Target.ExtraParams, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestInMemoryProfileAPIKeyRequiredBehavior(t *testing.T) {
|
func TestInMemoryProfileAPIKeyRequiredBehavior(t *testing.T) {
|
||||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||||
PromptDir: "./examples/prompts",
|
PromptDir: "./examples/prompts",
|
||||||
|
|||||||
13
examples/config.full.yml
Normal file
13
examples/config.full.yml
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
prompt_dir: ./examples/prompts
|
||||||
|
profile_dir: ./examples/profiles
|
||||||
|
schema_dir: ./examples/schemas
|
||||||
|
|
||||||
|
server:
|
||||||
|
addr: 127.0.0.1:8080
|
||||||
|
artifact_root: .
|
||||||
|
max_request_bytes: 16777216
|
||||||
|
max_artifact_bytes: 16777216
|
||||||
|
max_response_bytes: 16777216
|
||||||
|
|
||||||
|
defaults:
|
||||||
|
render_format: text
|
||||||
@@ -79,6 +79,9 @@ type serveConfig struct {
|
|||||||
profileDir string
|
profileDir string
|
||||||
schemaDir string
|
schemaDir string
|
||||||
artifactRoot string
|
artifactRoot string
|
||||||
|
maxRequestBytes int64
|
||||||
|
maxArtifactBytes int64
|
||||||
|
maxResponseBytes int64
|
||||||
}
|
}
|
||||||
|
|
||||||
type commonCommandSettings struct {
|
type commonCommandSettings struct {
|
||||||
@@ -87,6 +90,9 @@ type commonCommandSettings struct {
|
|||||||
schemaDir string
|
schemaDir string
|
||||||
serverAddr string
|
serverAddr string
|
||||||
artifactRoot string
|
artifactRoot string
|
||||||
|
maxRequestBytes int64
|
||||||
|
maxArtifactBytes int64
|
||||||
|
maxResponseBytes int64
|
||||||
defaultRenderFormat renderformat.PreparedRunOutputFormat
|
defaultRenderFormat renderformat.PreparedRunOutputFormat
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -204,7 +210,7 @@ func serveCommand(args []string, stderr io.Writer) int {
|
|||||||
return ExitRuntimeError
|
return ExitRuntimeError
|
||||||
}
|
}
|
||||||
|
|
||||||
artifactReader, err := artifactadapter.NewRestrictedCompositeReader(cfg.artifactRoot)
|
artifactReader, err := artifactadapter.NewRestrictedCompositeReaderWithLimit(cfg.artifactRoot, cfg.maxArtifactBytes)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintf(stderr, "artifact root error: %v\n", err)
|
fmt.Fprintf(stderr, "artifact root error: %v\n", err)
|
||||||
return ExitRuntimeError
|
return ExitRuntimeError
|
||||||
@@ -212,7 +218,10 @@ func serveCommand(args []string, stderr io.Writer) int {
|
|||||||
|
|
||||||
runner := newRunnerWithArtifactReader(cfg.promptDir, cfg.profileDir, cfg.schemaDir, llmClient, artifactReader)
|
runner := newRunnerWithArtifactReader(cfg.promptDir, cfg.profileDir, cfg.schemaDir, llmClient, artifactReader)
|
||||||
|
|
||||||
h := httpadapter.NewHandler(runner)
|
h := httpadapter.NewHandlerWithOptions(runner, httpadapter.HandlerOptions{
|
||||||
|
MaxRequestBytes: cfg.maxRequestBytes,
|
||||||
|
MaxResponseBytes: cfg.maxResponseBytes,
|
||||||
|
})
|
||||||
srv := &http.Server{
|
srv := &http.Server{
|
||||||
Addr: cfg.addr,
|
Addr: cfg.addr,
|
||||||
Handler: h,
|
Handler: h,
|
||||||
@@ -290,6 +299,9 @@ func parseServeArgs(args []string) (*serveConfig, error) {
|
|||||||
fs.StringVar(&cfg.profileDir, "profile-dir", "", "directory containing execution profile YAML files")
|
fs.StringVar(&cfg.profileDir, "profile-dir", "", "directory containing execution profile YAML files")
|
||||||
fs.StringVar(&cfg.schemaDir, "schema-dir", "", "base directory for validation schemas")
|
fs.StringVar(&cfg.schemaDir, "schema-dir", "", "base directory for validation schemas")
|
||||||
fs.StringVar(&cfg.artifactRoot, "artifact-root", "", "base directory for HTTP file input artifacts")
|
fs.StringVar(&cfg.artifactRoot, "artifact-root", "", "base directory for HTTP file input artifacts")
|
||||||
|
fs.Int64Var(&cfg.maxRequestBytes, "max-request-bytes", 0, "maximum HTTP request body bytes; 0 disables the limit")
|
||||||
|
fs.Int64Var(&cfg.maxArtifactBytes, "max-artifact-bytes", 0, "maximum HTTP file artifact bytes; 0 disables the limit")
|
||||||
|
fs.Int64Var(&cfg.maxResponseBytes, "max-response-bytes", 0, "maximum HTTP response body bytes; 0 disables the limit")
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
if err := fs.Parse(args); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -304,6 +316,9 @@ func parseServeArgs(args []string) (*serveConfig, error) {
|
|||||||
SchemaDir: cfg.schemaDirIfSet(fs),
|
SchemaDir: cfg.schemaDirIfSet(fs),
|
||||||
ServerAddr: cfg.addrIfSet(fs),
|
ServerAddr: cfg.addrIfSet(fs),
|
||||||
ArtifactRoot: cfg.artifactRootIfSet(fs),
|
ArtifactRoot: cfg.artifactRootIfSet(fs),
|
||||||
|
MaxRequestBytes: cfg.maxRequestBytesIfSet(fs),
|
||||||
|
MaxArtifactBytes: cfg.maxArtifactBytesIfSet(fs),
|
||||||
|
MaxResponseBytes: cfg.maxResponseBytesIfSet(fs),
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -314,6 +329,9 @@ func parseServeArgs(args []string) (*serveConfig, error) {
|
|||||||
cfg.schemaDir = settings.schemaDir
|
cfg.schemaDir = settings.schemaDir
|
||||||
cfg.addr = settings.serverAddr
|
cfg.addr = settings.serverAddr
|
||||||
cfg.artifactRoot = settings.artifactRoot
|
cfg.artifactRoot = settings.artifactRoot
|
||||||
|
cfg.maxRequestBytes = settings.maxRequestBytes
|
||||||
|
cfg.maxArtifactBytes = settings.maxArtifactBytes
|
||||||
|
cfg.maxResponseBytes = settings.maxResponseBytes
|
||||||
|
|
||||||
if err := validateRequiredLibraryDirs(cfg.promptDir); err != nil {
|
if err := validateRequiredLibraryDirs(cfg.promptDir); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -450,6 +468,27 @@ func (c *serveConfig) artifactRootIfSet(fs *flag.FlagSet) string {
|
|||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (c *serveConfig) maxRequestBytesIfSet(fs *flag.FlagSet) *int64 {
|
||||||
|
if flagWasSet(fs, "max-request-bytes") {
|
||||||
|
return &c.maxRequestBytes
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *serveConfig) maxArtifactBytesIfSet(fs *flag.FlagSet) *int64 {
|
||||||
|
if flagWasSet(fs, "max-artifact-bytes") {
|
||||||
|
return &c.maxArtifactBytes
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *serveConfig) maxResponseBytesIfSet(fs *flag.FlagSet) *int64 {
|
||||||
|
if flagWasSet(fs, "max-response-bytes") {
|
||||||
|
return &c.maxResponseBytes
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
func registerConfigPathFlag(fs *flag.FlagSet, target *string) {
|
func registerConfigPathFlag(fs *flag.FlagSet, target *string) {
|
||||||
fs.StringVar(
|
fs.StringVar(
|
||||||
target,
|
target,
|
||||||
@@ -488,6 +527,9 @@ func resolveCommonSettings(fs *flag.FlagSet, configPath string, overrides appcon
|
|||||||
schemaDir: settings.SchemaDir,
|
schemaDir: settings.SchemaDir,
|
||||||
serverAddr: settings.ServerAddr,
|
serverAddr: settings.ServerAddr,
|
||||||
artifactRoot: settings.ArtifactRoot,
|
artifactRoot: settings.ArtifactRoot,
|
||||||
|
maxRequestBytes: settings.MaxRequestBytes,
|
||||||
|
maxArtifactBytes: settings.MaxArtifactBytes,
|
||||||
|
maxResponseBytes: settings.MaxResponseBytes,
|
||||||
defaultRenderFormat: settings.DefaultRenderFormat,
|
defaultRenderFormat: settings.DefaultRenderFormat,
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
@@ -666,5 +708,5 @@ func printUsage(w io.Writer) {
|
|||||||
fmt.Fprintln(w, "usage: scriptorium <run|render|serve> ...")
|
fmt.Fprintln(w, "usage: scriptorium <run|render|serve> ...")
|
||||||
fmt.Fprintln(w, " run: scriptorium run [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID --input name=path [--input ...] [--profile ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--var k=v] [--out path] [--timeout 10m]")
|
fmt.Fprintln(w, " run: scriptorium run [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID --input name=path [--input ...] [--profile ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--var k=v] [--out path] [--timeout 10m]")
|
||||||
fmt.Fprintln(w, " render: scriptorium render [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID --input name=path [--input ...] [--profile ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--var k=v] [--format text|json] [--out path] [--timeout 10m]")
|
fmt.Fprintln(w, " render: scriptorium render [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID --input name=path [--input ...] [--profile ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--var k=v] [--format text|json] [--out path] [--timeout 10m]")
|
||||||
fmt.Fprintf(w, " serve: scriptorium serve [--config PATH] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR]\n", defaults.HTTPAddrDefault)
|
fmt.Fprintf(w, " serve: scriptorium serve [--config PATH] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR] [--artifact-root DIR] [--max-request-bytes N] [--max-artifact-bytes N] [--max-response-bytes N]\n", defaults.HTTPAddrDefault)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -192,6 +192,26 @@ func TestParseServeArgsRejectsRuntimeOverrideFlags(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestUsageIncludesServeFileAndSizeLimitFlags(t *testing.T) {
|
||||||
|
var stderr bytes.Buffer
|
||||||
|
code := Run(nil, io.Discard, &stderr)
|
||||||
|
if code != ExitRuntimeError {
|
||||||
|
t.Fatalf("expected usage path to return runtime error, got %d", code)
|
||||||
|
}
|
||||||
|
|
||||||
|
usage := stderr.String()
|
||||||
|
for _, want := range []string{
|
||||||
|
"--artifact-root",
|
||||||
|
"--max-request-bytes",
|
||||||
|
"--max-artifact-bytes",
|
||||||
|
"--max-response-bytes",
|
||||||
|
} {
|
||||||
|
if !strings.Contains(usage, want) {
|
||||||
|
t.Fatalf("expected usage to include %q, got %q", want, usage)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestParseRunArgsTimeout(t *testing.T) {
|
func TestParseRunArgsTimeout(t *testing.T) {
|
||||||
cfg, err := parseRunArgs([]string{
|
cfg, err := parseRunArgs([]string{
|
||||||
"--prompt-dir", "./prompts",
|
"--prompt-dir", "./prompts",
|
||||||
@@ -436,12 +456,18 @@ schema_dir: ./from-config/schemas
|
|||||||
server:
|
server:
|
||||||
addr: 127.0.0.1:9000
|
addr: 127.0.0.1:9000
|
||||||
artifact_root: ./from-config/artifacts
|
artifact_root: ./from-config/artifacts
|
||||||
|
max_request_bytes: 1024
|
||||||
|
max_artifact_bytes: 2048
|
||||||
|
max_response_bytes: 4096
|
||||||
`)
|
`)
|
||||||
|
|
||||||
cfg, err := parseServeArgs([]string{
|
cfg, err := parseServeArgs([]string{
|
||||||
"--config", configPath,
|
"--config", configPath,
|
||||||
"--addr", ":7777",
|
"--addr", ":7777",
|
||||||
"--artifact-root", "./from-cli/artifacts",
|
"--artifact-root", "./from-cli/artifacts",
|
||||||
|
"--max-request-bytes", "0",
|
||||||
|
"--max-artifact-bytes", "8192",
|
||||||
|
"--max-response-bytes", "16384",
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("expected valid args, got %v", err)
|
t.Fatalf("expected valid args, got %v", err)
|
||||||
@@ -462,6 +488,15 @@ server:
|
|||||||
if cfg.artifactRoot != filepath.Clean("./from-cli/artifacts") {
|
if cfg.artifactRoot != filepath.Clean("./from-cli/artifacts") {
|
||||||
t.Fatalf("expected CLI artifact root override, got %q", cfg.artifactRoot)
|
t.Fatalf("expected CLI artifact root override, got %q", cfg.artifactRoot)
|
||||||
}
|
}
|
||||||
|
if cfg.maxRequestBytes != 0 {
|
||||||
|
t.Fatalf("expected CLI max request bytes override, got %d", cfg.maxRequestBytes)
|
||||||
|
}
|
||||||
|
if cfg.maxArtifactBytes != 8192 {
|
||||||
|
t.Fatalf("expected CLI max artifact bytes override, got %d", cfg.maxArtifactBytes)
|
||||||
|
}
|
||||||
|
if cfg.maxResponseBytes != 16384 {
|
||||||
|
t.Fatalf("expected CLI max response bytes override, got %d", cfg.maxResponseBytes)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestParseServeArgsWithConfigProvidesRequiredDirectoriesAndAddr(t *testing.T) {
|
func TestParseServeArgsWithConfigProvidesRequiredDirectoriesAndAddr(t *testing.T) {
|
||||||
@@ -472,6 +507,9 @@ schema_dir: ./from-config/schemas
|
|||||||
server:
|
server:
|
||||||
addr: 127.0.0.1:9000
|
addr: 127.0.0.1:9000
|
||||||
artifact_root: ./from-config/artifacts
|
artifact_root: ./from-config/artifacts
|
||||||
|
max_request_bytes: 1024
|
||||||
|
max_artifact_bytes: 2048
|
||||||
|
max_response_bytes: 4096
|
||||||
`)
|
`)
|
||||||
|
|
||||||
cfg, err := parseServeArgs([]string{
|
cfg, err := parseServeArgs([]string{
|
||||||
@@ -496,6 +534,72 @@ server:
|
|||||||
if cfg.artifactRoot != filepath.Clean("./from-config/artifacts") {
|
if cfg.artifactRoot != filepath.Clean("./from-config/artifacts") {
|
||||||
t.Fatalf("expected artifact root from config, got %q", cfg.artifactRoot)
|
t.Fatalf("expected artifact root from config, got %q", cfg.artifactRoot)
|
||||||
}
|
}
|
||||||
|
if cfg.maxRequestBytes != 1024 {
|
||||||
|
t.Fatalf("expected max request bytes from config, got %d", cfg.maxRequestBytes)
|
||||||
|
}
|
||||||
|
if cfg.maxArtifactBytes != 2048 {
|
||||||
|
t.Fatalf("expected max artifact bytes from config, got %d", cfg.maxArtifactBytes)
|
||||||
|
}
|
||||||
|
if cfg.maxResponseBytes != 4096 {
|
||||||
|
t.Fatalf("expected max response bytes from config, got %d", cfg.maxResponseBytes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseServeArgsRejectsNegativeSizeLimits(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
flag string
|
||||||
|
}{
|
||||||
|
{name: "request", flag: "--max-request-bytes"},
|
||||||
|
{name: "artifact", flag: "--max-artifact-bytes"},
|
||||||
|
{name: "response", flag: "--max-response-bytes"},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
_, err := parseServeArgs([]string{
|
||||||
|
"--prompt-dir", "./prompts",
|
||||||
|
tc.flag, "-1",
|
||||||
|
})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected negative size limit error")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunAndRenderRejectServeSizeLimitFlags(t *testing.T) {
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
parse func([]string) error
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "run",
|
||||||
|
parse: func(args []string) error {
|
||||||
|
_, err := parseRunArgs(args)
|
||||||
|
return err
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "render",
|
||||||
|
parse: func(args []string) error {
|
||||||
|
_, err := parseRenderArgs(args)
|
||||||
|
return err
|
||||||
|
},
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
err := tc.parse([]string{
|
||||||
|
"--prompt-dir", "./prompts",
|
||||||
|
"--prompt", "p",
|
||||||
|
"--input", "a=b",
|
||||||
|
"--max-request-bytes", "1024",
|
||||||
|
})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected unsupported flag error")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRunAndRenderBuildEquivalentRuntimeOverrideRequestsForSharedFlags(t *testing.T) {
|
func TestRunAndRenderBuildEquivalentRuntimeOverrideRequestsForSharedFlags(t *testing.T) {
|
||||||
|
|||||||
@@ -4,10 +4,12 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/scriptorium/internal/artifact"
|
"gitea.maximumdirect.net/eric/scriptorium/internal/artifact"
|
||||||
|
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
|
||||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||||
"gitea.maximumdirect.net/eric/scriptorium/internal/profile"
|
"gitea.maximumdirect.net/eric/scriptorium/internal/profile"
|
||||||
"gitea.maximumdirect.net/eric/scriptorium/internal/promptdef"
|
"gitea.maximumdirect.net/eric/scriptorium/internal/promptdef"
|
||||||
@@ -20,10 +22,23 @@ type Runner interface {
|
|||||||
|
|
||||||
type Handler struct {
|
type Handler struct {
|
||||||
runner Runner
|
runner Runner
|
||||||
|
options HandlerOptions
|
||||||
|
}
|
||||||
|
|
||||||
|
type HandlerOptions struct {
|
||||||
|
MaxRequestBytes int64
|
||||||
|
MaxResponseBytes int64
|
||||||
}
|
}
|
||||||
|
|
||||||
func NewHandler(runner Runner) *Handler {
|
func NewHandler(runner Runner) *Handler {
|
||||||
return &Handler{runner: runner}
|
return NewHandlerWithOptions(runner, HandlerOptions{
|
||||||
|
MaxRequestBytes: defaults.HTTPMaxRequestBytesDefault,
|
||||||
|
MaxResponseBytes: defaults.HTTPMaxResponseBytesDefault,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewHandlerWithOptions(runner Runner, options HandlerOptions) *Handler {
|
||||||
|
return &Handler{runner: runner, options: options}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||||
@@ -37,9 +52,26 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var req runRequestDTO
|
var req runRequestDTO
|
||||||
dec := json.NewDecoder(r.Body)
|
body := r.Body
|
||||||
|
if h.options.MaxRequestBytes > 0 {
|
||||||
|
body = http.MaxBytesReader(w, r.Body, h.options.MaxRequestBytes)
|
||||||
|
}
|
||||||
|
dec := json.NewDecoder(body)
|
||||||
dec.DisallowUnknownFields()
|
dec.DisallowUnknownFields()
|
||||||
if err := dec.Decode(&req); err != nil {
|
if err := dec.Decode(&req); err != nil {
|
||||||
|
if isRequestTooLarge(err) {
|
||||||
|
writeError(w, http.StatusRequestEntityTooLarge, "request_too_large", "request body is too large")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeError(w, http.StatusBadRequest, "invalid_json", "invalid JSON request body")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var trailing any
|
||||||
|
if err := dec.Decode(&trailing); err != io.EOF {
|
||||||
|
if isRequestTooLarge(err) {
|
||||||
|
writeError(w, http.StatusRequestEntityTooLarge, "request_too_large", "request body is too large")
|
||||||
|
return
|
||||||
|
}
|
||||||
writeError(w, http.StatusBadRequest, "invalid_json", "invalid JSON request body")
|
writeError(w, http.StatusBadRequest, "invalid_json", "invalid JSON request body")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -121,7 +153,7 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
raw := res.RawOutput
|
raw := res.RawOutput
|
||||||
resp.RawModelOutput = &raw
|
resp.RawModelOutput = &raw
|
||||||
}
|
}
|
||||||
writeJSON(w, http.StatusOK, resp)
|
writeLimitedJSON(w, http.StatusOK, resp, h.options.MaxResponseBytes)
|
||||||
}
|
}
|
||||||
|
|
||||||
func executionTargetOverrideFromModelOverrideDTO(dto *modelOverrideRequestDTO) *domain.ExecutionTargetOverride {
|
func executionTargetOverrideFromModelOverrideDTO(dto *modelOverrideRequestDTO) *domain.ExecutionTargetOverride {
|
||||||
@@ -190,6 +222,8 @@ func mapRunError(err error) (int, string, string) {
|
|||||||
return http.StatusBadRequest, "profile_load_failed", "failed to load execution profile"
|
return http.StatusBadRequest, "profile_load_failed", "failed to load execution profile"
|
||||||
case errors.Is(err, artifact.ErrFileNotAllowed), errors.Is(err, artifact.ErrFileOutsideRoot):
|
case errors.Is(err, artifact.ErrFileNotAllowed), errors.Is(err, artifact.ErrFileOutsideRoot):
|
||||||
return http.StatusBadRequest, "artifact_not_allowed", "file input artifact is not allowed"
|
return http.StatusBadRequest, "artifact_not_allowed", "file input artifact is not allowed"
|
||||||
|
case errors.Is(err, artifact.ErrFileTooLarge):
|
||||||
|
return http.StatusRequestEntityTooLarge, "artifact_too_large", "file input artifact is too large"
|
||||||
case errors.Is(err, usecase.ErrArtifactLoad):
|
case errors.Is(err, usecase.ErrArtifactLoad):
|
||||||
return http.StatusBadRequest, "artifact_read_failed", "failed to read input artifact"
|
return http.StatusBadRequest, "artifact_read_failed", "failed to read input artifact"
|
||||||
case errors.Is(err, usecase.ErrPromptRender):
|
case errors.Is(err, usecase.ErrPromptRender):
|
||||||
@@ -204,9 +238,23 @@ func mapRunError(err error) (int, string, string) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func writeJSON(w http.ResponseWriter, status int, v any) {
|
func writeJSON(w http.ResponseWriter, status int, v any) {
|
||||||
|
writeLimitedJSON(w, status, v, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeLimitedJSON(w http.ResponseWriter, status int, v any, maxBytes int64) {
|
||||||
|
data, err := json.Marshal(v)
|
||||||
|
if err != nil {
|
||||||
|
writeError(w, http.StatusInternalServerError, "internal_error", "internal server error")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
data = append(data, '\n')
|
||||||
|
if maxBytes > 0 && int64(len(data)) > maxBytes {
|
||||||
|
writeError(w, http.StatusRequestEntityTooLarge, "response_too_large", "response body is too large")
|
||||||
|
return
|
||||||
|
}
|
||||||
w.Header().Set("Content-Type", "application/json")
|
w.Header().Set("Content-Type", "application/json")
|
||||||
w.WriteHeader(status)
|
w.WriteHeader(status)
|
||||||
_ = json.NewEncoder(w).Encode(v)
|
_, _ = w.Write(data)
|
||||||
}
|
}
|
||||||
|
|
||||||
func writeError(w http.ResponseWriter, status int, code, message string) {
|
func writeError(w http.ResponseWriter, status int, code, message string) {
|
||||||
@@ -217,3 +265,8 @@ func writeError(w http.ResponseWriter, status int, code, message string) {
|
|||||||
},
|
},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func isRequestTooLarge(err error) bool {
|
||||||
|
var maxBytesErr *http.MaxBytesError
|
||||||
|
return errors.As(err, &maxBytesErr)
|
||||||
|
}
|
||||||
|
|||||||
@@ -243,6 +243,24 @@ func TestHandlerFileRefsUnderArtifactRootWork(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestHandlerFileRefsAboveArtifactLimitAreRejected(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(root, "large.txt"), []byte("123456"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
h := newArtifactRootHandlerWithLimit(t, root, 5)
|
||||||
|
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{
|
||||||
|
"prompt_id":"p",
|
||||||
|
"inputs":{"x":{"type":"file","uri":"large.txt"}}
|
||||||
|
}`))
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
h.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assertHTTPErrorCode(t, w, http.StatusRequestEntityTooLarge, "artifact_too_large")
|
||||||
|
}
|
||||||
|
|
||||||
func TestHandlerFileRefsOutsideArtifactRootAreRejected(t *testing.T) {
|
func TestHandlerFileRefsOutsideArtifactRootAreRejected(t *testing.T) {
|
||||||
root := t.TempDir()
|
root := t.TempDir()
|
||||||
outside := t.TempDir()
|
outside := t.TempDir()
|
||||||
@@ -576,6 +594,69 @@ func TestHandlerInvalidJSON(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestHandlerRejectsTrailingJSON(t *testing.T) {
|
||||||
|
h := NewHandler(&fakeRunner{})
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p","inputs":{"x":{"type":"file","uri":"a"}}} {}`))
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
h.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assertHTTPErrorCode(t, w, http.StatusBadRequest, "invalid_json")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHandlerRequestTooLarge(t *testing.T) {
|
||||||
|
h := NewHandlerWithOptions(&fakeRunner{}, HandlerOptions{MaxRequestBytes: 12})
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p","inputs":{"x":{"type":"file","uri":"a"}}}`))
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
h.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assertHTTPErrorCode(t, w, http.StatusRequestEntityTooLarge, "request_too_large")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHandlerMalformedJSONBelowLimitStillBadRequest(t *testing.T) {
|
||||||
|
h := NewHandlerWithOptions(&fakeRunner{}, HandlerOptions{MaxRequestBytes: 1024})
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString("{"))
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
h.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assertHTTPErrorCode(t, w, http.StatusBadRequest, "invalid_json")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHandlerResponseTooLarge(t *testing.T) {
|
||||||
|
h := NewHandlerWithOptions(&fakeRunner{result: &domain.RunResult{
|
||||||
|
Artifact: domain.Artifact{Body: []byte(strings.Repeat("x", 128))},
|
||||||
|
Validation: domain.ValidationResult{Status: domain.ValidationPassed, Mode: domain.ValidationBasic, IsValid: true},
|
||||||
|
EffectiveModelParams: domain.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
|
||||||
|
}}, HandlerOptions{MaxRequestBytes: 1024, MaxResponseBytes: 64})
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p","inputs":{"x":{"type":"file","uri":"a"}}}`))
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
h.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assertHTTPErrorCode(t, w, http.StatusRequestEntityTooLarge, "response_too_large")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHandlerRawOutputDoesNotBypassResponseLimit(t *testing.T) {
|
||||||
|
h := NewHandlerWithOptions(&fakeRunner{result: &domain.RunResult{
|
||||||
|
Artifact: domain.Artifact{Body: []byte("ok")},
|
||||||
|
RawOutput: strings.Repeat("raw", 80),
|
||||||
|
Validation: domain.ValidationResult{Status: domain.ValidationPassed, Mode: domain.ValidationBasic, IsValid: true},
|
||||||
|
EffectiveModelParams: domain.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
|
||||||
|
}}, HandlerOptions{MaxRequestBytes: 1024, MaxResponseBytes: 128})
|
||||||
|
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{
|
||||||
|
"prompt_id":"p",
|
||||||
|
"inputs":{"x":{"type":"file","uri":"a"}},
|
||||||
|
"include_raw_output":true
|
||||||
|
}`))
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
|
h.ServeHTTP(w, req)
|
||||||
|
|
||||||
|
assertHTTPErrorCode(t, w, http.StatusRequestEntityTooLarge, "response_too_large")
|
||||||
|
}
|
||||||
|
|
||||||
func TestHandlerMissingPromptID(t *testing.T) {
|
func TestHandlerMissingPromptID(t *testing.T) {
|
||||||
h := NewHandler(&fakeRunner{})
|
h := NewHandler(&fakeRunner{})
|
||||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"inputs":{"x":{"type":"file","uri":"a"}}}`))
|
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"inputs":{"x":{"type":"file","uri":"a"}}}`))
|
||||||
@@ -770,7 +851,13 @@ func wrap(stage error, cause error) error {
|
|||||||
func newArtifactRootHandler(t *testing.T, root string) *Handler {
|
func newArtifactRootHandler(t *testing.T, root string) *Handler {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
reader, err := artifact.NewRestrictedCompositeReader(root)
|
return newArtifactRootHandlerWithLimit(t, root, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func newArtifactRootHandlerWithLimit(t *testing.T, root string, maxArtifactBytes int64) *Handler {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
reader, err := artifact.NewRestrictedCompositeReaderWithLimit(root, maxArtifactBytes)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("expected restricted artifact reader: %v", err)
|
t.Fatalf("expected restricted artifact reader: %v", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
|
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
|
||||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||||
|
"io"
|
||||||
"mime"
|
"mime"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
@@ -19,6 +20,7 @@ var (
|
|||||||
ErrMissingFilePath = errors.New("missing file path for file artifact")
|
ErrMissingFilePath = errors.New("missing file path for file artifact")
|
||||||
ErrFileNotAllowed = errors.New("file artifact references are not allowed")
|
ErrFileNotAllowed = errors.New("file artifact references are not allowed")
|
||||||
ErrFileOutsideRoot = errors.New("file artifact path is outside artifact root")
|
ErrFileOutsideRoot = errors.New("file artifact path is outside artifact root")
|
||||||
|
ErrFileTooLarge = errors.New("file artifact exceeds size limit")
|
||||||
)
|
)
|
||||||
|
|
||||||
// Reader resolves artifact references into actual artifacts.
|
// Reader resolves artifact references into actual artifacts.
|
||||||
@@ -40,7 +42,11 @@ func NewCompositeReader() Reader {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func NewRestrictedCompositeReader(root string) (Reader, error) {
|
func NewRestrictedCompositeReader(root string) (Reader, error) {
|
||||||
fileReader, err := newRestrictedFileReader(root)
|
return NewRestrictedCompositeReaderWithLimit(root, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewRestrictedCompositeReaderWithLimit(root string, maxBytes int64) (Reader, error) {
|
||||||
|
fileReader, err := newRestrictedFileReader(root, maxBytes)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -123,9 +129,13 @@ func (r deniedFileReader) Read(ctx context.Context, ref domain.ArtifactRef) (*do
|
|||||||
|
|
||||||
type restrictedFileReader struct {
|
type restrictedFileReader struct {
|
||||||
root string
|
root string
|
||||||
|
maxBytes int64
|
||||||
}
|
}
|
||||||
|
|
||||||
func newRestrictedFileReader(root string) (Reader, error) {
|
func newRestrictedFileReader(root string, maxBytes int64) (Reader, error) {
|
||||||
|
if maxBytes < 0 {
|
||||||
|
return nil, fmt.Errorf("artifact size limit must be greater than or equal to 0")
|
||||||
|
}
|
||||||
cleanRoot := strings.TrimSpace(root)
|
cleanRoot := strings.TrimSpace(root)
|
||||||
if cleanRoot == "" {
|
if cleanRoot == "" {
|
||||||
return deniedFileReader{}, nil
|
return deniedFileReader{}, nil
|
||||||
@@ -134,7 +144,7 @@ func newRestrictedFileReader(root string) (Reader, error) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("resolve artifact root: %w", err)
|
return nil, fmt.Errorf("resolve artifact root: %w", err)
|
||||||
}
|
}
|
||||||
return &restrictedFileReader{root: absRoot}, nil
|
return &restrictedFileReader{root: absRoot, maxBytes: maxBytes}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (r *restrictedFileReader) Read(ctx context.Context, ref domain.ArtifactRef) (*domain.Artifact, error) {
|
func (r *restrictedFileReader) Read(ctx context.Context, ref domain.ArtifactRef) (*domain.Artifact, error) {
|
||||||
@@ -148,14 +158,15 @@ func (r *restrictedFileReader) Read(ctx context.Context, ref domain.ArtifactRef)
|
|||||||
return nil, ErrMissingFilePath
|
return nil, ErrMissingFilePath
|
||||||
}
|
}
|
||||||
|
|
||||||
path, err := r.resolve(ref.URI)
|
path, err := r.resolveLexicalPath(ref.URI)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
return readFileArtifact(path)
|
return readFileArtifactWithLimit(path, r.maxBytes)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (r *restrictedFileReader) resolve(rawPath string) (string, error) {
|
// resolveLexicalPath checks cleaned path containment without resolving symlinks.
|
||||||
|
func (r *restrictedFileReader) resolveLexicalPath(rawPath string) (string, error) {
|
||||||
cleanPath := filepath.Clean(strings.TrimSpace(rawPath))
|
cleanPath := filepath.Clean(strings.TrimSpace(rawPath))
|
||||||
var candidate string
|
var candidate string
|
||||||
if filepath.IsAbs(cleanPath) {
|
if filepath.IsAbs(cleanPath) {
|
||||||
@@ -181,10 +192,39 @@ func (r *restrictedFileReader) resolve(rawPath string) (string, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func readFileArtifact(path string) (*domain.Artifact, error) {
|
func readFileArtifact(path string) (*domain.Artifact, error) {
|
||||||
data, err := os.ReadFile(path)
|
return readFileArtifactWithLimit(path, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func readFileArtifactWithLimit(path string, maxBytes int64) (*domain.Artifact, error) {
|
||||||
|
if maxBytes < 0 {
|
||||||
|
return nil, fmt.Errorf("file size limit must be greater than or equal to 0")
|
||||||
|
}
|
||||||
|
|
||||||
|
file, err := os.Open(path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("failed to read file %s: %w", path, err)
|
return nil, fmt.Errorf("failed to read file %s: %w", path, err)
|
||||||
}
|
}
|
||||||
|
defer file.Close()
|
||||||
|
|
||||||
|
info, err := file.Stat()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to stat file %s: %w", path, err)
|
||||||
|
}
|
||||||
|
if maxBytes > 0 && info.Size() > maxBytes {
|
||||||
|
return nil, ErrFileTooLarge
|
||||||
|
}
|
||||||
|
|
||||||
|
var reader io.Reader = file
|
||||||
|
if maxBytes > 0 {
|
||||||
|
reader = io.LimitReader(file, maxBytes+1)
|
||||||
|
}
|
||||||
|
data, err := io.ReadAll(reader)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to read file %s: %w", path, err)
|
||||||
|
}
|
||||||
|
if maxBytes > 0 && int64(len(data)) > maxBytes {
|
||||||
|
return nil, ErrFileTooLarge
|
||||||
|
}
|
||||||
|
|
||||||
contentType := mime.TypeByExtension(filepath.Ext(path))
|
contentType := mime.TypeByExtension(filepath.Ext(path))
|
||||||
if contentType == "" {
|
if contentType == "" {
|
||||||
|
|||||||
@@ -112,6 +112,34 @@ func TestRestrictedCompositeReader(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRestrictedCompositeReaderFollowsSymlinkInsideRoot(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
root := t.TempDir()
|
||||||
|
outside := t.TempDir()
|
||||||
|
|
||||||
|
target := filepath.Join(outside, "linked.txt")
|
||||||
|
if err := os.WriteFile(target, []byte("linked outside root"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
link := filepath.Join(root, "linked.txt")
|
||||||
|
if err := os.Symlink(target, link); err != nil {
|
||||||
|
t.Skipf("symlink creation unavailable: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
reader, err := NewRestrictedCompositeReader(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected restricted reader construction, got %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
art, err := reader.Read(ctx, domain.ArtifactRef{Type: domain.ArtifactRefFile, URI: "linked.txt"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected symlink inside root to be followed, got %v", err)
|
||||||
|
}
|
||||||
|
if string(art.Body) != "linked outside root" {
|
||||||
|
t.Fatalf("unexpected artifact body: %q", string(art.Body))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestRestrictedCompositeReaderWithoutRootDeniesFileRefs(t *testing.T) {
|
func TestRestrictedCompositeReaderWithoutRootDeniesFileRefs(t *testing.T) {
|
||||||
reader, err := NewRestrictedCompositeReader("")
|
reader, err := NewRestrictedCompositeReader("")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -132,6 +160,54 @@ func TestRestrictedCompositeReaderWithoutRootDeniesFileRefs(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRestrictedCompositeReaderFileSizeLimit(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
root := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(root, "exact.txt"), []byte("12345"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(root, "large.txt"), []byte("123456"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
reader, err := NewRestrictedCompositeReaderWithLimit(root, 5)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected restricted reader construction, got %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
art, err := reader.Read(ctx, domain.ArtifactRef{Type: domain.ArtifactRefFile, URI: "exact.txt"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected file at limit to succeed, got %v", err)
|
||||||
|
}
|
||||||
|
if string(art.Body) != "12345" {
|
||||||
|
t.Fatalf("unexpected artifact body: %q", string(art.Body))
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err = reader.Read(ctx, domain.ArtifactRef{Type: domain.ArtifactRefFile, URI: "large.txt"})
|
||||||
|
if !errors.Is(err, ErrFileTooLarge) {
|
||||||
|
t.Fatalf("expected ErrFileTooLarge, got %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRestrictedCompositeReaderFileSizeLimitZeroDisablesLimit(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(root, "large.txt"), []byte("123456"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
reader, err := NewRestrictedCompositeReaderWithLimit(root, 0)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected restricted reader construction, got %v", err)
|
||||||
|
}
|
||||||
|
art, err := reader.Read(context.Background(), domain.ArtifactRef{Type: domain.ArtifactRefFile, URI: "large.txt"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected unlimited reader to succeed, got %v", err)
|
||||||
|
}
|
||||||
|
if string(art.Body) != "123456" {
|
||||||
|
t.Fatalf("unexpected artifact body: %q", string(art.Body))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestFileReader_Read(t *testing.T) {
|
func TestFileReader_Read(t *testing.T) {
|
||||||
content := []byte("test file content")
|
content := []byte("test file content")
|
||||||
tmpFile, err := os.CreateTemp("", "artifact_test_*.txt")
|
tmpFile, err := os.CreateTemp("", "artifact_test_*.txt")
|
||||||
|
|||||||
@@ -42,6 +42,9 @@ type Config struct {
|
|||||||
type ServerConfig struct {
|
type ServerConfig struct {
|
||||||
Addr string `yaml:"addr"`
|
Addr string `yaml:"addr"`
|
||||||
ArtifactRoot string `yaml:"artifact_root"`
|
ArtifactRoot string `yaml:"artifact_root"`
|
||||||
|
MaxRequestBytes *int64 `yaml:"max_request_bytes"`
|
||||||
|
MaxArtifactBytes *int64 `yaml:"max_artifact_bytes"`
|
||||||
|
MaxResponseBytes *int64 `yaml:"max_response_bytes"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type DefaultsConfig struct {
|
type DefaultsConfig struct {
|
||||||
@@ -55,6 +58,9 @@ type AppSettings struct {
|
|||||||
SchemaDir string
|
SchemaDir string
|
||||||
ServerAddr string
|
ServerAddr string
|
||||||
ArtifactRoot string
|
ArtifactRoot string
|
||||||
|
MaxRequestBytes int64
|
||||||
|
MaxArtifactBytes int64
|
||||||
|
MaxResponseBytes int64
|
||||||
DefaultRenderFormat renderformat.PreparedRunOutputFormat
|
DefaultRenderFormat renderformat.PreparedRunOutputFormat
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -65,6 +71,9 @@ type CLIOverrides struct {
|
|||||||
SchemaDir string
|
SchemaDir string
|
||||||
ServerAddr string
|
ServerAddr string
|
||||||
ArtifactRoot string
|
ArtifactRoot string
|
||||||
|
MaxRequestBytes *int64
|
||||||
|
MaxArtifactBytes *int64
|
||||||
|
MaxResponseBytes *int64
|
||||||
RenderFormat string
|
RenderFormat string
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -73,6 +82,9 @@ func BuiltInDefaults() AppSettings {
|
|||||||
return AppSettings{
|
return AppSettings{
|
||||||
SchemaDir: defaults.SchemaDirDefault,
|
SchemaDir: defaults.SchemaDirDefault,
|
||||||
ServerAddr: defaults.HTTPAddrDefault,
|
ServerAddr: defaults.HTTPAddrDefault,
|
||||||
|
MaxRequestBytes: defaults.HTTPMaxRequestBytesDefault,
|
||||||
|
MaxArtifactBytes: defaults.HTTPMaxArtifactBytesDefault,
|
||||||
|
MaxResponseBytes: defaults.HTTPMaxResponseBytesDefault,
|
||||||
DefaultRenderFormat: renderformat.DefaultPreparedRunOutputFormat,
|
DefaultRenderFormat: renderformat.DefaultPreparedRunOutputFormat,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -148,6 +160,24 @@ func ApplyCLIOverrides(base AppSettings, overrides CLIOverrides) (AppSettings, e
|
|||||||
if v := strings.TrimSpace(overrides.ArtifactRoot); v != "" {
|
if v := strings.TrimSpace(overrides.ArtifactRoot); v != "" {
|
||||||
out.ArtifactRoot = filepath.Clean(v)
|
out.ArtifactRoot = filepath.Clean(v)
|
||||||
}
|
}
|
||||||
|
if overrides.MaxRequestBytes != nil {
|
||||||
|
if *overrides.MaxRequestBytes < 0 {
|
||||||
|
return AppSettings{}, fmt.Errorf("%w: server.max_request_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||||
|
}
|
||||||
|
out.MaxRequestBytes = *overrides.MaxRequestBytes
|
||||||
|
}
|
||||||
|
if overrides.MaxArtifactBytes != nil {
|
||||||
|
if *overrides.MaxArtifactBytes < 0 {
|
||||||
|
return AppSettings{}, fmt.Errorf("%w: server.max_artifact_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||||
|
}
|
||||||
|
out.MaxArtifactBytes = *overrides.MaxArtifactBytes
|
||||||
|
}
|
||||||
|
if overrides.MaxResponseBytes != nil {
|
||||||
|
if *overrides.MaxResponseBytes < 0 {
|
||||||
|
return AppSettings{}, fmt.Errorf("%w: server.max_response_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||||
|
}
|
||||||
|
out.MaxResponseBytes = *overrides.MaxResponseBytes
|
||||||
|
}
|
||||||
if rawFormat := strings.TrimSpace(overrides.RenderFormat); rawFormat != "" {
|
if rawFormat := strings.TrimSpace(overrides.RenderFormat); rawFormat != "" {
|
||||||
parsed, err := renderformat.ParsePreparedRunOutputFormat(rawFormat)
|
parsed, err := renderformat.ParsePreparedRunOutputFormat(rawFormat)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -190,6 +220,24 @@ func applyConfig(base AppSettings, cfg Config) (AppSettings, error) {
|
|||||||
if v := strings.TrimSpace(cfg.Server.ArtifactRoot); v != "" {
|
if v := strings.TrimSpace(cfg.Server.ArtifactRoot); v != "" {
|
||||||
out.ArtifactRoot = filepath.Clean(v)
|
out.ArtifactRoot = filepath.Clean(v)
|
||||||
}
|
}
|
||||||
|
if cfg.Server.MaxRequestBytes != nil {
|
||||||
|
if *cfg.Server.MaxRequestBytes < 0 {
|
||||||
|
return AppSettings{}, fmt.Errorf("%w: server.max_request_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||||
|
}
|
||||||
|
out.MaxRequestBytes = *cfg.Server.MaxRequestBytes
|
||||||
|
}
|
||||||
|
if cfg.Server.MaxArtifactBytes != nil {
|
||||||
|
if *cfg.Server.MaxArtifactBytes < 0 {
|
||||||
|
return AppSettings{}, fmt.Errorf("%w: server.max_artifact_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||||
|
}
|
||||||
|
out.MaxArtifactBytes = *cfg.Server.MaxArtifactBytes
|
||||||
|
}
|
||||||
|
if cfg.Server.MaxResponseBytes != nil {
|
||||||
|
if *cfg.Server.MaxResponseBytes < 0 {
|
||||||
|
return AppSettings{}, fmt.Errorf("%w: server.max_response_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||||
|
}
|
||||||
|
out.MaxResponseBytes = *cfg.Server.MaxResponseBytes
|
||||||
|
}
|
||||||
if rawFormat := strings.TrimSpace(cfg.Defaults.RenderFormat); rawFormat != "" {
|
if rawFormat := strings.TrimSpace(cfg.Defaults.RenderFormat); rawFormat != "" {
|
||||||
parsed, err := renderformat.ParsePreparedRunOutputFormat(rawFormat)
|
parsed, err := renderformat.ParsePreparedRunOutputFormat(rawFormat)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"path/filepath"
|
"path/filepath"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
|
||||||
renderformat "gitea.maximumdirect.net/eric/scriptorium/internal/format"
|
renderformat "gitea.maximumdirect.net/eric/scriptorium/internal/format"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -24,6 +25,20 @@ func TestLoadConfigMissingImplicitPathUsesBuiltInDefaults(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestBuiltInDefaultsIncludeHTTPSizeLimits(t *testing.T) {
|
||||||
|
got := BuiltInDefaults()
|
||||||
|
|
||||||
|
if got.MaxRequestBytes != defaults.HTTPMaxRequestBytesDefault {
|
||||||
|
t.Fatalf("unexpected max request bytes: %d", got.MaxRequestBytes)
|
||||||
|
}
|
||||||
|
if got.MaxArtifactBytes != defaults.HTTPMaxArtifactBytesDefault {
|
||||||
|
t.Fatalf("unexpected max artifact bytes: %d", got.MaxArtifactBytes)
|
||||||
|
}
|
||||||
|
if got.MaxResponseBytes != defaults.HTTPMaxResponseBytesDefault {
|
||||||
|
t.Fatalf("unexpected max response bytes: %d", got.MaxResponseBytes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestLoadConfigMissingExplicitPathReturnsError(t *testing.T) {
|
func TestLoadConfigMissingExplicitPathReturnsError(t *testing.T) {
|
||||||
tmp := t.TempDir()
|
tmp := t.TempDir()
|
||||||
missing := filepath.Join(tmp, "missing.yml")
|
missing := filepath.Join(tmp, "missing.yml")
|
||||||
@@ -93,6 +108,9 @@ schema_dir: ./schemas
|
|||||||
server:
|
server:
|
||||||
addr: 127.0.0.1:9090
|
addr: 127.0.0.1:9090
|
||||||
artifact_root: ./artifacts
|
artifact_root: ./artifacts
|
||||||
|
max_request_bytes: 1024
|
||||||
|
max_artifact_bytes: 2048
|
||||||
|
max_response_bytes: 4096
|
||||||
defaults:
|
defaults:
|
||||||
render_format: json
|
render_format: json
|
||||||
`)
|
`)
|
||||||
@@ -117,11 +135,58 @@ defaults:
|
|||||||
if got.ArtifactRoot != filepath.Clean("./artifacts") {
|
if got.ArtifactRoot != filepath.Clean("./artifacts") {
|
||||||
t.Fatalf("unexpected server.artifact_root: %q", got.ArtifactRoot)
|
t.Fatalf("unexpected server.artifact_root: %q", got.ArtifactRoot)
|
||||||
}
|
}
|
||||||
|
if got.MaxRequestBytes != 1024 {
|
||||||
|
t.Fatalf("unexpected server.max_request_bytes: %d", got.MaxRequestBytes)
|
||||||
|
}
|
||||||
|
if got.MaxArtifactBytes != 2048 {
|
||||||
|
t.Fatalf("unexpected server.max_artifact_bytes: %d", got.MaxArtifactBytes)
|
||||||
|
}
|
||||||
|
if got.MaxResponseBytes != 4096 {
|
||||||
|
t.Fatalf("unexpected server.max_response_bytes: %d", got.MaxResponseBytes)
|
||||||
|
}
|
||||||
if got.DefaultRenderFormat != renderformat.PreparedRunFormatJSON {
|
if got.DefaultRenderFormat != renderformat.PreparedRunFormatJSON {
|
||||||
t.Fatalf("unexpected defaults.render_format: %q", got.DefaultRenderFormat)
|
t.Fatalf("unexpected defaults.render_format: %q", got.DefaultRenderFormat)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestLoadConfigAcceptsZeroHTTPSizeLimits(t *testing.T) {
|
||||||
|
path := writeConfigFile(t, "config.yml", `
|
||||||
|
server:
|
||||||
|
max_request_bytes: 0
|
||||||
|
max_artifact_bytes: 0
|
||||||
|
max_response_bytes: 0
|
||||||
|
`)
|
||||||
|
|
||||||
|
got, err := LoadConfig(path, true)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected no error, got %v", err)
|
||||||
|
}
|
||||||
|
if got.MaxRequestBytes != 0 || got.MaxArtifactBytes != 0 || got.MaxResponseBytes != 0 {
|
||||||
|
t.Fatalf("expected zero limits to be preserved, got request=%d artifact=%d response=%d", got.MaxRequestBytes, got.MaxArtifactBytes, got.MaxResponseBytes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadConfigRejectsNegativeHTTPSizeLimits(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
body string
|
||||||
|
}{
|
||||||
|
{name: "request", body: "server:\n max_request_bytes: -1\n"},
|
||||||
|
{name: "artifact", body: "server:\n max_artifact_bytes: -1\n"},
|
||||||
|
{name: "response", body: "server:\n max_response_bytes: -1\n"},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
path := writeConfigFile(t, "config.yml", tc.body)
|
||||||
|
_, err := LoadConfig(path, true)
|
||||||
|
if !errors.Is(err, ErrInvalidConfig) {
|
||||||
|
t.Fatalf("expected ErrInvalidConfig, got %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestLoadConfigEmptyFileResolvesToBuiltInDefaults(t *testing.T) {
|
func TestLoadConfigEmptyFileResolvesToBuiltInDefaults(t *testing.T) {
|
||||||
path := writeConfigFile(t, "config.yml", "")
|
path := writeConfigFile(t, "config.yml", "")
|
||||||
|
|
||||||
@@ -190,8 +255,14 @@ func TestApplyCLIOverridesAppliesPrecedence(t *testing.T) {
|
|||||||
SchemaDir: "/from/config/schemas",
|
SchemaDir: "/from/config/schemas",
|
||||||
ServerAddr: ":1234",
|
ServerAddr: ":1234",
|
||||||
ArtifactRoot: "/from/config/artifacts",
|
ArtifactRoot: "/from/config/artifacts",
|
||||||
|
MaxRequestBytes: 111,
|
||||||
|
MaxArtifactBytes: 222,
|
||||||
|
MaxResponseBytes: 333,
|
||||||
DefaultRenderFormat: renderformat.PreparedRunFormatJSON,
|
DefaultRenderFormat: renderformat.PreparedRunFormatJSON,
|
||||||
}
|
}
|
||||||
|
maxRequestBytes := int64(0)
|
||||||
|
maxArtifactBytes := int64(444)
|
||||||
|
maxResponseBytes := int64(555)
|
||||||
|
|
||||||
got, err := ApplyCLIOverrides(base, CLIOverrides{
|
got, err := ApplyCLIOverrides(base, CLIOverrides{
|
||||||
PromptDir: "./prompts-cli",
|
PromptDir: "./prompts-cli",
|
||||||
@@ -199,6 +270,9 @@ func TestApplyCLIOverridesAppliesPrecedence(t *testing.T) {
|
|||||||
SchemaDir: "./schemas-cli",
|
SchemaDir: "./schemas-cli",
|
||||||
ServerAddr: ":8081",
|
ServerAddr: ":8081",
|
||||||
ArtifactRoot: "./artifacts-cli",
|
ArtifactRoot: "./artifacts-cli",
|
||||||
|
MaxRequestBytes: &maxRequestBytes,
|
||||||
|
MaxArtifactBytes: &maxArtifactBytes,
|
||||||
|
MaxResponseBytes: &maxResponseBytes,
|
||||||
RenderFormat: "text",
|
RenderFormat: "text",
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -220,11 +294,42 @@ func TestApplyCLIOverridesAppliesPrecedence(t *testing.T) {
|
|||||||
if got.ArtifactRoot != filepath.Clean("./artifacts-cli") {
|
if got.ArtifactRoot != filepath.Clean("./artifacts-cli") {
|
||||||
t.Fatalf("unexpected artifact root: %q", got.ArtifactRoot)
|
t.Fatalf("unexpected artifact root: %q", got.ArtifactRoot)
|
||||||
}
|
}
|
||||||
|
if got.MaxRequestBytes != 0 {
|
||||||
|
t.Fatalf("unexpected max request bytes: %d", got.MaxRequestBytes)
|
||||||
|
}
|
||||||
|
if got.MaxArtifactBytes != 444 {
|
||||||
|
t.Fatalf("unexpected max artifact bytes: %d", got.MaxArtifactBytes)
|
||||||
|
}
|
||||||
|
if got.MaxResponseBytes != 555 {
|
||||||
|
t.Fatalf("unexpected max response bytes: %d", got.MaxResponseBytes)
|
||||||
|
}
|
||||||
if got.DefaultRenderFormat != renderformat.PreparedRunFormatText {
|
if got.DefaultRenderFormat != renderformat.PreparedRunFormatText {
|
||||||
t.Fatalf("unexpected render format: %q", got.DefaultRenderFormat)
|
t.Fatalf("unexpected render format: %q", got.DefaultRenderFormat)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestApplyCLIOverridesRejectsNegativeHTTPSizeLimits(t *testing.T) {
|
||||||
|
negative := int64(-1)
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
overrides CLIOverrides
|
||||||
|
}{
|
||||||
|
{name: "request", overrides: CLIOverrides{MaxRequestBytes: &negative}},
|
||||||
|
{name: "artifact", overrides: CLIOverrides{MaxArtifactBytes: &negative}},
|
||||||
|
{name: "response", overrides: CLIOverrides{MaxResponseBytes: &negative}},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
_, err := ApplyCLIOverrides(BuiltInDefaults(), tc.overrides)
|
||||||
|
if !errors.Is(err, ErrInvalidConfig) {
|
||||||
|
t.Fatalf("expected ErrInvalidConfig, got %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestApplyCLIOverridesInvalidRenderFormatReturnsError(t *testing.T) {
|
func TestApplyCLIOverridesInvalidRenderFormatReturnsError(t *testing.T) {
|
||||||
_, err := ApplyCLIOverrides(BuiltInDefaults(), CLIOverrides{RenderFormat: "yaml"})
|
_, err := ApplyCLIOverrides(BuiltInDefaults(), CLIOverrides{RenderFormat: "yaml"})
|
||||||
if err == nil {
|
if err == nil {
|
||||||
|
|||||||
@@ -14,6 +14,9 @@ const (
|
|||||||
ContentTypeTextMarkdown = "text/markdown"
|
ContentTypeTextMarkdown = "text/markdown"
|
||||||
ContentTypeApplicationJSON = "application/json"
|
ContentTypeApplicationJSON = "application/json"
|
||||||
OpenAIChatCompletionsPath = "/chat/completions"
|
OpenAIChatCompletionsPath = "/chat/completions"
|
||||||
|
HTTPMaxRequestBytesDefault = 16 * 1024 * 1024
|
||||||
|
HTTPMaxArtifactBytesDefault = 16 * 1024 * 1024
|
||||||
|
HTTPMaxResponseBytesDefault = 16 * 1024 * 1024
|
||||||
|
|
||||||
ExecutionDefaultTemperature = 0.0
|
ExecutionDefaultTemperature = 0.0
|
||||||
ExecutionDefaultMaxTokens = 0
|
ExecutionDefaultMaxTokens = 0
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package filecatalog
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"fmt"
|
||||||
"io/fs"
|
"io/fs"
|
||||||
"os"
|
"os"
|
||||||
"path"
|
"path"
|
||||||
@@ -93,6 +94,42 @@ func DisplayPath(root string, name string) string {
|
|||||||
return cleanName
|
return cleanName
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ResolveFSPath resolves userPath from baseDir and keeps it inside root.
|
||||||
|
func ResolveFSPath(root string, baseDir string, userPath string) (string, string, error) {
|
||||||
|
cleanRoot := CleanFSRoot(root)
|
||||||
|
cleanBase := path.Clean(strings.TrimSpace(baseDir))
|
||||||
|
if cleanBase == "" {
|
||||||
|
cleanBase = cleanRoot
|
||||||
|
}
|
||||||
|
if !containsFSPath(cleanRoot, cleanBase) {
|
||||||
|
return "", "", fmt.Errorf("base path %q is outside source root %q", cleanBase, cleanRoot)
|
||||||
|
}
|
||||||
|
|
||||||
|
cleanUserPath := strings.TrimSpace(userPath)
|
||||||
|
if cleanUserPath == "" {
|
||||||
|
return "", "", fmt.Errorf("path is required")
|
||||||
|
}
|
||||||
|
cleanUserPath = path.Clean(cleanUserPath)
|
||||||
|
if path.IsAbs(cleanUserPath) {
|
||||||
|
return "", "", fmt.Errorf("path %q must be relative", userPath)
|
||||||
|
}
|
||||||
|
|
||||||
|
resolved := path.Clean(path.Join(cleanBase, cleanUserPath))
|
||||||
|
if !containsFSPath(cleanRoot, resolved) {
|
||||||
|
return "", "", fmt.Errorf("path %q escapes source root %q", userPath, cleanRoot)
|
||||||
|
}
|
||||||
|
return resolved, DisplayPath(cleanRoot, resolved), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func containsFSPath(root string, name string) bool {
|
||||||
|
root = CleanFSRoot(root)
|
||||||
|
name = path.Clean(name)
|
||||||
|
if root == "." {
|
||||||
|
return name == "." || (name != ".." && !strings.HasPrefix(name, "../"))
|
||||||
|
}
|
||||||
|
return name == root || strings.HasPrefix(name, strings.TrimSuffix(root, "/")+"/")
|
||||||
|
}
|
||||||
|
|
||||||
// Stem strips .yaml or .yml from a file name.
|
// Stem strips .yaml or .yml from a file name.
|
||||||
func Stem(name string) string {
|
func Stem(name string) string {
|
||||||
name = strings.TrimSuffix(name, ".yaml")
|
name = strings.TrimSuffix(name, ".yaml")
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"reflect"
|
"reflect"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"testing/fstest"
|
"testing/fstest"
|
||||||
)
|
)
|
||||||
@@ -131,6 +132,92 @@ func TestDisplayPath(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestResolveFSPath(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
root string
|
||||||
|
baseDir string
|
||||||
|
userPath string
|
||||||
|
wantPath string
|
||||||
|
wantDisplay string
|
||||||
|
wantErr string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "sibling inside root",
|
||||||
|
root: "prompts",
|
||||||
|
baseDir: "prompts/nested",
|
||||||
|
userPath: "./messages/user.tmpl",
|
||||||
|
wantPath: "prompts/nested/messages/user.tmpl",
|
||||||
|
wantDisplay: "nested/messages/user.tmpl",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "parent inside root",
|
||||||
|
root: "prompts",
|
||||||
|
baseDir: "prompts/nested",
|
||||||
|
userPath: "../shared/user.tmpl",
|
||||||
|
wantPath: "prompts/shared/user.tmpl",
|
||||||
|
wantDisplay: "shared/user.tmpl",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "escape rejected",
|
||||||
|
root: "prompts",
|
||||||
|
baseDir: "prompts/nested",
|
||||||
|
userPath: "../../outside.tmpl",
|
||||||
|
wantErr: "escapes source root",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "absolute path rejected",
|
||||||
|
root: "prompts",
|
||||||
|
baseDir: "prompts/nested",
|
||||||
|
userPath: "/outside.tmpl",
|
||||||
|
wantErr: "must be relative",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "empty path rejected",
|
||||||
|
root: "prompts",
|
||||||
|
baseDir: "prompts/nested",
|
||||||
|
userPath: " ",
|
||||||
|
wantErr: "path is required",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "dot root allows normal relative path",
|
||||||
|
root: ".",
|
||||||
|
baseDir: ".",
|
||||||
|
userPath: "schemas/events.schema.json",
|
||||||
|
wantPath: "schemas/events.schema.json",
|
||||||
|
wantDisplay: "schemas/events.schema.json",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "dot root rejects parent escape",
|
||||||
|
root: ".",
|
||||||
|
baseDir: ".",
|
||||||
|
userPath: "../outside.tmpl",
|
||||||
|
wantErr: "escapes source root",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
gotPath, gotDisplay, err := ResolveFSPath(tc.root, tc.baseDir, tc.userPath)
|
||||||
|
if tc.wantErr != "" {
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("expected error containing %q", tc.wantErr)
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), tc.wantErr) {
|
||||||
|
t.Fatalf("expected error to contain %q, got %v", tc.wantErr, err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected no error, got %v", err)
|
||||||
|
}
|
||||||
|
if gotPath != tc.wantPath || gotDisplay != tc.wantDisplay {
|
||||||
|
t.Fatalf("expected path/display %q/%q, got %q/%q", tc.wantPath, tc.wantDisplay, gotPath, gotDisplay)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestStemStripsYAMLExtensions(t *testing.T) {
|
func TestStemStripsYAMLExtensions(t *testing.T) {
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
|
|||||||
@@ -139,8 +139,8 @@ func (c *OpenAICompatibleClient) Generate(ctx context.Context, req domain.Genera
|
|||||||
defer httpResp.Body.Close()
|
defer httpResp.Body.Close()
|
||||||
|
|
||||||
if httpResp.StatusCode < 200 || httpResp.StatusCode >= 300 {
|
if httpResp.StatusCode < 200 || httpResp.StatusCode >= 300 {
|
||||||
body, _ := io.ReadAll(io.LimitReader(httpResp.Body, 4096))
|
_, _ = io.Copy(io.Discard, io.LimitReader(httpResp.Body, 4096))
|
||||||
return nil, fmt.Errorf("%w: status=%d body=%q", ErrUnexpectedStatus, httpResp.StatusCode, strings.TrimSpace(string(body)))
|
return nil, fmt.Errorf("%w: status=%d", ErrUnexpectedStatus, httpResp.StatusCode)
|
||||||
}
|
}
|
||||||
|
|
||||||
var wireResp openAIChatResponse
|
var wireResp openAIChatResponse
|
||||||
|
|||||||
@@ -903,9 +903,10 @@ func TestOpenAICompatibleClientEndpointOverride(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestOpenAICompatibleClientNon2xxError(t *testing.T) {
|
func TestOpenAICompatibleClientNon2xxError(t *testing.T) {
|
||||||
|
const sensitiveBody = `provider-secret-fragment request_payload_details`
|
||||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
w.WriteHeader(http.StatusBadRequest)
|
w.WriteHeader(http.StatusBadRequest)
|
||||||
_, _ = w.Write([]byte(`{"error":"bad request payload"}`))
|
_, _ = w.Write([]byte(`{"error":"` + sensitiveBody + `"}`))
|
||||||
}))
|
}))
|
||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
|
|
||||||
@@ -923,8 +924,11 @@ func TestOpenAICompatibleClientNon2xxError(t *testing.T) {
|
|||||||
if !errors.Is(err, ErrUnexpectedStatus) {
|
if !errors.Is(err, ErrUnexpectedStatus) {
|
||||||
t.Fatalf("expected ErrUnexpectedStatus, got %v", err)
|
t.Fatalf("expected ErrUnexpectedStatus, got %v", err)
|
||||||
}
|
}
|
||||||
if !strings.Contains(err.Error(), "400") || !strings.Contains(err.Error(), "bad request payload") {
|
if !strings.Contains(err.Error(), "status=400") {
|
||||||
t.Fatalf("expected status/body details, got %v", err)
|
t.Fatalf("expected status detail, got %v", err)
|
||||||
|
}
|
||||||
|
if strings.Contains(err.Error(), sensitiveBody) {
|
||||||
|
t.Fatalf("expected provider response body to be redacted, got %v", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,7 @@
|
|||||||
|
id: deepseek-4-flash
|
||||||
|
endpoint: https://openrouter.ai/api/v1
|
||||||
|
model: deepseek/deepseek-v4-flash
|
||||||
|
#reasoning_effort: medium
|
||||||
|
timeout_seconds: 180
|
||||||
|
api_key_env: OPENROUTER_API_KEY
|
||||||
|
service_tier: flex
|
||||||
@@ -193,6 +193,11 @@ func loadPromptDefinition(ctx context.Context, fsys fs.FS, root string, id strin
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("failed to read prompt definition directory: %w", err)
|
return nil, fmt.Errorf("failed to read prompt definition directory: %w", err)
|
||||||
}
|
}
|
||||||
|
cleanRoot := filecatalog.CleanFSRoot(root)
|
||||||
|
rootInfo, err := fs.Stat(fsys, cleanRoot)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("failed to read prompt definition directory: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
var matches []promptDefinitionMatch
|
var matches []promptDefinitionMatch
|
||||||
for _, fullPath := range files {
|
for _, fullPath := range files {
|
||||||
@@ -220,7 +225,7 @@ func loadPromptDefinition(ctx context.Context, fsys fs.FS, root string, id strin
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
def, err := normalizePromptDefinitionFromFS(raw, fsys, fullPath)
|
def, err := normalizePromptDefinitionFromFS(raw, fsys, root, fullPath, rootInfo.IsDir())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if fileMatch || strings.TrimSpace(raw.ID) == id {
|
if fileMatch || strings.TrimSpace(raw.ID) == id {
|
||||||
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidPromptDefinition, relPath, err)
|
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidPromptDefinition, relPath, err)
|
||||||
@@ -295,14 +300,23 @@ func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*d
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func normalizePromptDefinitionFromFS(raw *promptDefinitionFile, fsys fs.FS, sourcePath string) (*domain.PromptDefinition, error) {
|
func normalizePromptDefinitionFromFS(raw *promptDefinitionFile, fsys fs.FS, root string, sourcePath string, rootIsDir bool) (*domain.PromptDefinition, error) {
|
||||||
promptDir := path.Dir(sourcePath)
|
promptDir := path.Dir(sourcePath)
|
||||||
return normalizePromptDefinitionWithContent(raw, func(contentFile string) (string, string, error) {
|
return normalizePromptDefinitionWithContent(raw, func(contentFile string) (string, string, error) {
|
||||||
resolvedPath := strings.TrimSpace(contentFile)
|
var resolvedPath string
|
||||||
|
if rootIsDir {
|
||||||
|
var err error
|
||||||
|
resolvedPath, _, err = filecatalog.ResolveFSPath(root, promptDir, contentFile)
|
||||||
|
if err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
resolvedPath = strings.TrimSpace(contentFile)
|
||||||
if !path.IsAbs(resolvedPath) {
|
if !path.IsAbs(resolvedPath) {
|
||||||
resolvedPath = path.Join(promptDir, resolvedPath)
|
resolvedPath = path.Join(promptDir, resolvedPath)
|
||||||
}
|
}
|
||||||
resolvedPath = strings.TrimPrefix(path.Clean(resolvedPath), "/")
|
resolvedPath = strings.TrimPrefix(path.Clean(resolvedPath), "/")
|
||||||
|
}
|
||||||
|
|
||||||
body, err := fs.ReadFile(fsys, resolvedPath)
|
body, err := fs.ReadFile(fsys, resolvedPath)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -359,6 +359,69 @@ output:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestFSRepositoryContentFileContainment(t *testing.T) {
|
||||||
|
t.Run("nested prompt can reference file inside root", func(t *testing.T) {
|
||||||
|
repo := NewFSRepository(fstest.MapFS{
|
||||||
|
"prompts/nested/prompt.yaml": &fstest.MapFile{Data: []byte(`
|
||||||
|
id: fs-contained-prompt
|
||||||
|
version: "1.0.0"
|
||||||
|
messages:
|
||||||
|
- role: user
|
||||||
|
content_file: ../shared/user.tmpl
|
||||||
|
output:
|
||||||
|
format: markdown
|
||||||
|
validation_mode: basic
|
||||||
|
repair_attempts: 0
|
||||||
|
`)},
|
||||||
|
"prompts/shared/user.tmpl": &fstest.MapFile{Data: []byte(`Inside root.`)},
|
||||||
|
}, "prompts")
|
||||||
|
|
||||||
|
got, err := repo.GetPromptDefinition(context.Background(), "fs-contained-prompt", "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected no error, got %v", err)
|
||||||
|
}
|
||||||
|
if len(got.Templates) != 1 || got.Templates[0].Content != "Inside root." {
|
||||||
|
t.Fatalf("expected contained content file, got %+v", got.Templates)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
contentFile string
|
||||||
|
wantErr string
|
||||||
|
}{
|
||||||
|
{name: "parent escape rejected", contentFile: "../outside.tmpl", wantErr: "escapes source root"},
|
||||||
|
{name: "absolute path rejected", contentFile: "/outside.tmpl", wantErr: "must be relative"},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
repo := NewFSRepository(fstest.MapFS{
|
||||||
|
"prompts/prompt.yaml": &fstest.MapFile{Data: []byte(`
|
||||||
|
id: fs-escaped-prompt
|
||||||
|
version: "1.0.0"
|
||||||
|
messages:
|
||||||
|
- role: user
|
||||||
|
content_file: ` + tc.contentFile + `
|
||||||
|
output:
|
||||||
|
format: markdown
|
||||||
|
validation_mode: basic
|
||||||
|
repair_attempts: 0
|
||||||
|
`)},
|
||||||
|
"outside.tmpl": &fstest.MapFile{Data: []byte(`Outside root.`)},
|
||||||
|
}, "prompts")
|
||||||
|
|
||||||
|
_, err := repo.GetPromptDefinition(context.Background(), "fs-escaped-prompt", "")
|
||||||
|
if !errors.Is(err, ErrInvalidPromptDefinition) {
|
||||||
|
t.Fatalf("expected ErrInvalidPromptDefinition, got %v", err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), tc.wantErr) {
|
||||||
|
t.Fatalf("expected error to contain %q, got %v", tc.wantErr, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestFSRepositoryRejectsDuplicatePromptIDs(t *testing.T) {
|
func TestFSRepositoryRejectsDuplicatePromptIDs(t *testing.T) {
|
||||||
repo := NewFSRepository(fstest.MapFS{
|
repo := NewFSRepository(fstest.MapFS{
|
||||||
"one.yaml": &fstest.MapFile{Data: []byte(`
|
"one.yaml": &fstest.MapFile{Data: []byte(`
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||||
|
"gitea.maximumdirect.net/eric/scriptorium/internal/filecatalog"
|
||||||
"github.com/santhosh-tekuri/jsonschema/v6"
|
"github.com/santhosh-tekuri/jsonschema/v6"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -244,17 +245,24 @@ func (v *FSValidator) resolveSchemaPath(schemaPath string) (string, error) {
|
|||||||
return "", errors.New("schema filesystem is nil")
|
return "", errors.New("schema filesystem is nil")
|
||||||
}
|
}
|
||||||
|
|
||||||
cleanRoot := cleanFSRoot(v.root)
|
cleanRoot := filecatalog.CleanFSRoot(v.root)
|
||||||
rootInfo, err := fs.Stat(v.fsys, cleanRoot)
|
rootInfo, err := fs.Stat(v.fsys, cleanRoot)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", fmt.Errorf("failed to access schema source %q: %w", cleanRoot, err)
|
return "", fmt.Errorf("failed to access schema source %q: %w", cleanRoot, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
cleanSchemaPath := cleanSchemaFSPath(schemaPath)
|
|
||||||
var resolved string
|
var resolved string
|
||||||
if rootInfo.IsDir() {
|
if rootInfo.IsDir() {
|
||||||
resolved = path.Join(cleanRoot, cleanSchemaPath)
|
resolvedPath, _, err := filecatalog.ResolveFSPath(cleanRoot, cleanRoot, schemaPath)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
resolved = resolvedPath
|
||||||
} else {
|
} else {
|
||||||
|
cleanSchemaPath, err := cleanSchemaFSPath(schemaPath)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
if cleanSchemaPath != path.Base(cleanRoot) {
|
if cleanSchemaPath != path.Base(cleanRoot) {
|
||||||
return "", fmt.Errorf("schema path %q does not match schema file %q", cleanSchemaPath, path.Base(cleanRoot))
|
return "", fmt.Errorf("schema path %q does not match schema file %q", cleanSchemaPath, path.Base(cleanRoot))
|
||||||
}
|
}
|
||||||
@@ -267,18 +275,16 @@ func (v *FSValidator) resolveSchemaPath(schemaPath string) (string, error) {
|
|||||||
return resolved, nil
|
return resolved, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func cleanSchemaFSPath(schemaPath string) string {
|
func cleanSchemaFSPath(schemaPath string) (string, error) {
|
||||||
cleaned := strings.TrimSpace(schemaPath)
|
cleaned := strings.TrimSpace(schemaPath)
|
||||||
cleaned = strings.TrimPrefix(path.Clean(cleaned), "/")
|
if cleaned == "" {
|
||||||
return cleaned
|
return "", errors.New("schema path is required for json_schema validation")
|
||||||
}
|
}
|
||||||
|
cleaned = path.Clean(cleaned)
|
||||||
func cleanFSRoot(root string) string {
|
if path.IsAbs(cleaned) {
|
||||||
root = strings.TrimSpace(root)
|
return "", fmt.Errorf("schema path %q must be relative", schemaPath)
|
||||||
if root == "" || root == "." {
|
|
||||||
return "."
|
|
||||||
}
|
}
|
||||||
return strings.TrimPrefix(path.Clean(root), "/")
|
return cleaned, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func fsSchemaResourceURL(schemaName string) string {
|
func fsSchemaResourceURL(schemaName string) string {
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"testing/fstest"
|
"testing/fstest"
|
||||||
|
|
||||||
@@ -276,6 +277,61 @@ func TestFSValidatorJSONSchemaSuccess(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestFSValidatorJSONSchemaPathContainment(t *testing.T) {
|
||||||
|
t.Run("nested schema inside root succeeds", func(t *testing.T) {
|
||||||
|
v := NewFSValidator(fstest.MapFS{
|
||||||
|
"schemas/nested/events.schema.json": &fstest.MapFile{Data: []byte(`{
|
||||||
|
"type": "object",
|
||||||
|
"required": ["events"],
|
||||||
|
"properties": {
|
||||||
|
"events": {"type": "array"}
|
||||||
|
}
|
||||||
|
}`)},
|
||||||
|
}, "schemas")
|
||||||
|
|
||||||
|
res, err := v.Validate(context.Background(), &domain.Artifact{Body: []byte(`{"events":[]}`)}, domain.OutputContract{
|
||||||
|
ValidationMode: domain.ValidationJSONSchema,
|
||||||
|
SchemaPath: "nested/events.schema.json",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("expected no error, got %v", err)
|
||||||
|
}
|
||||||
|
if res.Status != domain.ValidationPassed || !res.IsValid {
|
||||||
|
t.Fatalf("expected passed/valid, got status=%q valid=%v", res.Status, res.IsValid)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
schemaPath string
|
||||||
|
wantErr string
|
||||||
|
}{
|
||||||
|
{name: "parent escape rejected", schemaPath: "../outside.schema.json", wantErr: "escapes source root"},
|
||||||
|
{name: "absolute path rejected", schemaPath: "/outside.schema.json", wantErr: "must be relative"},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tc := range tests {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
v := NewFSValidator(fstest.MapFS{
|
||||||
|
"schemas/events.schema.json": &fstest.MapFile{Data: []byte(`{"type":"object"}`)},
|
||||||
|
"outside.schema.json": &fstest.MapFile{Data: []byte(`{"type":"object"}`)},
|
||||||
|
"schemas/outside.schema.json": &fstest.MapFile{Data: []byte(`{"type":"object"}`)},
|
||||||
|
}, "schemas")
|
||||||
|
|
||||||
|
_, err := v.Validate(context.Background(), &domain.Artifact{Body: []byte(`{"events":[]}`)}, domain.OutputContract{
|
||||||
|
ValidationMode: domain.ValidationJSONSchema,
|
||||||
|
SchemaPath: tc.schemaPath,
|
||||||
|
})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected schema path error")
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), tc.wantErr) {
|
||||||
|
t.Fatalf("expected error to contain %q, got %v", tc.wantErr, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestFSValidatorSingleSchemaFileUsesBaseName(t *testing.T) {
|
func TestFSValidatorSingleSchemaFileUsesBaseName(t *testing.T) {
|
||||||
v := NewFSValidator(fstest.MapFS{
|
v := NewFSValidator(fstest.MapFS{
|
||||||
"events.schema.json": &fstest.MapFile{Data: []byte(`{
|
"events.schema.json": &fstest.MapFile{Data: []byte(`{
|
||||||
|
|||||||
13
profiles.go
13
profiles.go
@@ -28,10 +28,21 @@ func OpenAICompatibleProfile(cfg OpenAICompatibleProfileConfig) Profile {
|
|||||||
ServiceTier: cfg.ServiceTier,
|
ServiceTier: cfg.ServiceTier,
|
||||||
ReasoningEffort: cfg.ReasoningEffort,
|
ReasoningEffort: cfg.ReasoningEffort,
|
||||||
APIKeyRequired: cfg.APIKeyRequired,
|
APIKeyRequired: cfg.APIKeyRequired,
|
||||||
ExtraParams: copyAnyMap(cfg.ExtraParams),
|
ExtraParams: copyShallowAnyMap(cfg.ExtraParams),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func copyShallowAnyMap(src map[string]any) map[string]any {
|
||||||
|
if src == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
out := make(map[string]any, len(src))
|
||||||
|
for k, v := range src {
|
||||||
|
out[k] = v
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
type memoryProfileRepository struct {
|
type memoryProfileRepository struct {
|
||||||
profiles map[string]domain.ExecutionProfile
|
profiles map[string]domain.ExecutionProfile
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user