Compare commits
14 Commits
39485d87f6
...
v0.11.0
| Author | SHA1 | Date | |
|---|---|---|---|
| 90b76ddad3 | |||
| d5b3d1e061 | |||
| 41083de46a | |||
| 07ac7e54c5 | |||
| 879cb021b2 | |||
| 574f88bd6a | |||
| d5d7a222a4 | |||
| aabd89aea7 | |||
| 9189cbfc22 | |||
| 872c166ed7 | |||
| 6742def4d3 | |||
| 1b39f82117 | |||
| f7d821067f | |||
| a16f66cbc7 |
2
LICENSE
2
LICENSE
@@ -1,4 +1,4 @@
|
||||
Copyright (c) 2026 eric.
|
||||
Copyright (c) 2026 Eric Rakestraw.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
||||
|
||||
|
||||
15
README.md
15
README.md
@@ -1,8 +1,12 @@
|
||||
# scriptorium
|
||||
|
||||
Scriptorium is a config-driven prompt execution engine.
|
||||
Scriptorium is a narrow prompt-execution application for rendering prompt
|
||||
requests, running them against OpenAI-compatible chat-completions endpoints, and
|
||||
serving the same run workflow over HTTP.
|
||||
|
||||
It separates prompt definitions (what to generate) from execution profiles (how to call an OpenAI-compatible model endpoint), then runs or renders a prepared request from named input artifacts.
|
||||
It keeps prompt definitions, execution profiles, schemas, and input artifacts as
|
||||
separate files so prompts can be reviewed and reused without baking model
|
||||
runtime settings into application code.
|
||||
|
||||
## Quickstart
|
||||
|
||||
@@ -23,16 +27,19 @@ This command renders the prepared prompt and effective runtime settings without
|
||||
|
||||
- [CLI reference](docs/cli.md)
|
||||
- [Configuration reference](docs/config.md)
|
||||
- [HTTP API reference](docs/api.md)
|
||||
- [Operations guide](docs/operations.md)
|
||||
- [Troubleshooting](docs/troubleshooting.md)
|
||||
- [Consumer integration overview](docs/consumers/api.md)
|
||||
- [Go library package](docs/consumers/pkg-scriptorium.md)
|
||||
- [HTTP API integration](docs/integrations/http-api.md)
|
||||
- [Subprocess integration](docs/integrations/subprocess.md)
|
||||
- [OpenAI-compatible chat integration](docs/integrations/openai-compatible-chat.md)
|
||||
- [Narratio subprocess integration](docs/integrations/narratio.md)
|
||||
- [Architecture policy](docs/policy/architecture.md)
|
||||
|
||||
## Examples
|
||||
|
||||
- `examples/config.yml`
|
||||
- `examples/config.full.yml`
|
||||
- `examples/render-markdown-summary.sh`
|
||||
- `examples/http-run.json`
|
||||
- `examples/go-library/prepare`
|
||||
|
||||
297
docs/api.md
Normal file
297
docs/api.md
Normal file
@@ -0,0 +1,297 @@
|
||||
# HTTP API Reference
|
||||
|
||||
This is the canonical public HTTP contract for Scriptorium.
|
||||
|
||||
Implemented route:
|
||||
|
||||
- `POST /v1/runs`
|
||||
|
||||
For CLI behavior, see [CLI reference](cli.md). For config and prompt/profile
|
||||
file formats, see [Configuration reference](config.md).
|
||||
|
||||
The maintained request-shape example is `examples/http-run.json`. It requires a
|
||||
running `serve` process with an artifact root that can read the referenced
|
||||
files, plus a reachable model endpoint for full execution.
|
||||
|
||||
## Base URL And Deployment
|
||||
|
||||
`scriptorium serve` listens on `server.addr` or `serve --addr`. The default is
|
||||
`:8080`.
|
||||
|
||||
The route path is always:
|
||||
|
||||
```text
|
||||
/v1/runs
|
||||
```
|
||||
|
||||
The HTTP adapter has no built-in authentication or authorization. Deploy it
|
||||
behind trusted network and authentication controls.
|
||||
|
||||
## Media Types
|
||||
|
||||
- Request body: JSON object.
|
||||
- Response body: JSON object.
|
||||
- Response `Content-Type`: `application/json`.
|
||||
|
||||
Requests are decoded as JSON regardless of the request `Content-Type` header.
|
||||
There are no shared query parameters.
|
||||
|
||||
## Request Limits
|
||||
|
||||
HTTP limits are configured through `server.*` config fields or `serve` flags:
|
||||
|
||||
- `server.max_request_bytes`: encoded JSON request body limit, including inline input bodies.
|
||||
- `server.max_artifact_bytes`: file artifact limit for HTTP `file` input references.
|
||||
- `server.max_response_bytes`: encoded JSON response limit, including artifact body and optional raw output.
|
||||
|
||||
Each limit defaults to `16777216` bytes. `0` disables that limit.
|
||||
|
||||
## `POST /v1/runs`
|
||||
|
||||
Runs one prompt request and returns the generated artifact, validation result,
|
||||
and metadata.
|
||||
|
||||
### Request Body
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt_id": "generic.markdown_summary",
|
||||
"profile_id": "local-fast",
|
||||
"prompt_version": "1.0.0",
|
||||
"inputs": {
|
||||
"transcript": {
|
||||
"type": "file",
|
||||
"uri": "./examples/fixtures/transcript.md"
|
||||
},
|
||||
"glossary": {
|
||||
"type": "inline",
|
||||
"body": "party:\n - Rin"
|
||||
}
|
||||
},
|
||||
"vars": {
|
||||
"session_date": "2026-05-04"
|
||||
},
|
||||
"model": {
|
||||
"endpoint": "http://localhost:8000/v1",
|
||||
"model": "gpt-4o-mini",
|
||||
"temperature": 0,
|
||||
"max_tokens": 800,
|
||||
"top_p": 1,
|
||||
"timeout_seconds": 120,
|
||||
"service_tier": "priority",
|
||||
"reasoning_effort": "medium",
|
||||
"api_key_env": "SCRIPTORIUM_API_KEY",
|
||||
"extra_params": {
|
||||
"provider_option": "enabled"
|
||||
}
|
||||
},
|
||||
"include_raw_output": false
|
||||
}
|
||||
```
|
||||
|
||||
Request fields:
|
||||
|
||||
| Field | Required | Description |
|
||||
| --- | --- | --- |
|
||||
| `prompt_id` | yes | Prompt ID. Must not be blank. |
|
||||
| `prompt_version` | no | Prompt version filter. |
|
||||
| `profile_id` | no | Execution profile ID. If omitted, the prompt must define `default_profile`. |
|
||||
| `inputs` | yes | Object mapping prompt input names to input references. Must contain at least one entry. |
|
||||
| `vars` | no | Object mapping template variable names to string values. |
|
||||
| `model` | no | Runtime model override object. |
|
||||
| `include_raw_output` | no | When `true`, include `raw_model_output` in the response. |
|
||||
|
||||
Input reference fields:
|
||||
|
||||
| Field | Required | Description |
|
||||
| --- | --- | --- |
|
||||
| `type` | yes | `file` or `inline`. |
|
||||
| `uri` | for `file` | File URI/path. |
|
||||
| `body` | for `inline` | Inline artifact body. |
|
||||
|
||||
HTTP `file` references require `server.artifact_root` or `serve
|
||||
--artifact-root`. Relative file URIs resolve against that root. Absolute file
|
||||
URIs are accepted only when lexically inside the root. Relative traversal and
|
||||
absolute paths outside the root return `400 artifact_not_allowed`.
|
||||
|
||||
The containment check is lexical and does not resolve symlinks. Symlinks inside
|
||||
the artifact root are followed by the operating system, including symlinks that
|
||||
point outside the root. Keep the artifact root narrow and not writable by
|
||||
untrusted users.
|
||||
|
||||
Model override fields:
|
||||
|
||||
| Field | Description |
|
||||
| --- | --- |
|
||||
| `endpoint` | Runtime endpoint override. |
|
||||
| `model` | Runtime model override. |
|
||||
| `temperature` | Number in range `0..2`. Explicit `0` is an override. |
|
||||
| `max_tokens` | Integer greater than or equal to `0`. Explicit `0` is an override. |
|
||||
| `top_p` | Number in range `0..1`. Explicit `0` is an override. |
|
||||
| `timeout_seconds` | Integer greater than or equal to `0`. Explicit `0` disables the outbound client timeout. |
|
||||
| `service_tier` | Provider-specific request tier. |
|
||||
| `reasoning_effort` | Provider-specific reasoning setting. |
|
||||
| `api_key_env` | Name of an environment variable containing the API key. |
|
||||
| `extra_params` | JSON-compatible provider-specific top-level request fields. |
|
||||
|
||||
Raw API-key values are not accepted in HTTP payloads. A field such as
|
||||
`api_key` is rejected as unknown JSON.
|
||||
|
||||
`extra_params` keys must not be empty and must not collide with reserved
|
||||
outbound fields: `model`, `session_id`, `messages`, `temperature`,
|
||||
`max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or
|
||||
`response_format`.
|
||||
|
||||
### Strict JSON Rules
|
||||
|
||||
Request decoding is strict:
|
||||
|
||||
- malformed JSON returns `400 invalid_json`
|
||||
- unknown request fields return `400 invalid_json`
|
||||
- unknown `inputs` item fields return `400 invalid_json`
|
||||
- unknown `model` fields return `400 invalid_json`
|
||||
- trailing JSON tokens after the request object return `400 invalid_json`
|
||||
- request bodies above the configured limit return `413 request_too_large`
|
||||
|
||||
### Success Response
|
||||
|
||||
Status: `200 OK`
|
||||
|
||||
```json
|
||||
{
|
||||
"artifact": {
|
||||
"name": "output",
|
||||
"content_type": "text/markdown",
|
||||
"body": "Generated content",
|
||||
"size": 17,
|
||||
"hash": "..."
|
||||
},
|
||||
"validation": {
|
||||
"status": "passed",
|
||||
"mode": "basic",
|
||||
"repair_attempts": 0,
|
||||
"is_valid": true
|
||||
},
|
||||
"metadata": {
|
||||
"run_id": "...",
|
||||
"prompt_id": "generic.markdown_summary",
|
||||
"prompt_version": "1.0.0",
|
||||
"prompt_hash": "...",
|
||||
"rendered_prompt_hash": "...",
|
||||
"selected_profile_id": "local-fast",
|
||||
"model_name": "gpt-4o-mini",
|
||||
"endpoint": "http://localhost:8000/v1",
|
||||
"model_params": {
|
||||
"endpoint": "http://localhost:8000/v1",
|
||||
"model": "gpt-4o-mini",
|
||||
"temperature": 0.2,
|
||||
"max_tokens": 500,
|
||||
"top_p": 1,
|
||||
"timeout_seconds": 90
|
||||
},
|
||||
"input_hashes": {
|
||||
"transcript": "..."
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": 11,
|
||||
"completion_tokens": 22,
|
||||
"total_tokens": 33,
|
||||
"cached_tokens": 0,
|
||||
"cache_write_tokens": 0
|
||||
},
|
||||
"start_time": "2026-05-04T12:00:00Z",
|
||||
"end_time": "2026-05-04T12:00:01Z",
|
||||
"duration_ms": 1000,
|
||||
"validation_mode": "basic",
|
||||
"validation_status": "passed",
|
||||
"repair_attempts_used": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Response fields:
|
||||
|
||||
- `artifact`: generated output artifact.
|
||||
- `validation`: validation result for the generated artifact.
|
||||
- `metadata`: run and effective runtime metadata.
|
||||
- `raw_model_output`: omitted unless `include_raw_output` is `true`.
|
||||
|
||||
`artifact.uri` is omitted when empty. `validation.errors` and
|
||||
`validation.schema_path` are omitted when empty. `model_params.service_tier`,
|
||||
`model_params.reasoning_effort`, `model_params.api_key_env`, and
|
||||
`model_params.extra_params` are omitted when empty.
|
||||
|
||||
`metadata.usage.cached_tokens` and `metadata.usage.cache_write_tokens` are
|
||||
always present as numbers. They are `0` when the provider omits compatible cache
|
||||
usage fields or reports no cache activity.
|
||||
|
||||
### Validation Failure Response
|
||||
|
||||
Generated-content validation failures still return `200 OK`.
|
||||
|
||||
```json
|
||||
{
|
||||
"validation": {
|
||||
"status": "failed",
|
||||
"mode": "json",
|
||||
"errors": ["invalid JSON: ..."],
|
||||
"repair_attempts": 0,
|
||||
"is_valid": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The response still includes `artifact` and `metadata`.
|
||||
|
||||
## Error Responses
|
||||
|
||||
Error body shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"code": "invalid_request",
|
||||
"message": "prompt_id is required"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Current status/code mapping:
|
||||
|
||||
| Status | Code | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `400` | `invalid_json` | Malformed JSON, unknown JSON field, or trailing JSON token. |
|
||||
| `400` | `invalid_request` | Missing/invalid request fields or invalid runtime overrides. |
|
||||
| `400` | `profile_required` | No `profile_id` and prompt has no `default_profile`. |
|
||||
| `400` | `prompt_load_failed` | Prompt definition YAML/contract failed to load. |
|
||||
| `400` | `profile_load_failed` | Profile YAML/contract failed to load, including raw `api_key`. |
|
||||
| `400` | `artifact_not_allowed` | HTTP file refs are disabled or requested path is outside artifact root. |
|
||||
| `400` | `artifact_read_failed` | Input artifact could not be read or input ref was unsupported/invalid. |
|
||||
| `400` | `prompt_render_failed` | Prompt template rendering failed. |
|
||||
| `400` | `api_key_env_missing` | Selected `api_key_env` variable is unset or empty. |
|
||||
| `404` | `not_found` | Route path is unknown. |
|
||||
| `404` | `prompt_not_found` | Prompt ID/version was not found. |
|
||||
| `404` | `profile_not_found` | Profile ID was not found. |
|
||||
| `405` | `method_not_allowed` | Method is not `POST` on `/v1/runs`. |
|
||||
| `413` | `request_too_large` | Encoded JSON request body exceeds configured request limit. |
|
||||
| `413` | `artifact_too_large` | HTTP file input artifact exceeds configured artifact limit. |
|
||||
| `413` | `response_too_large` | Encoded JSON response exceeds configured response limit. |
|
||||
| `500` | `validation_runtime_failed` | Validator runtime/schema loading failed. |
|
||||
| `500` | `internal_error` | Unclassified server error. |
|
||||
| `502` | `llm_failed` | Outbound model request failed. |
|
||||
|
||||
HTTP error messages are intentionally concise and do not include sensitive
|
||||
internal causes.
|
||||
|
||||
## Retry And Idempotency
|
||||
|
||||
Scriptorium does not provide idempotency keys, pagination, caching headers, or
|
||||
rate limiting.
|
||||
|
||||
Clients may retry transport failures or `5xx` responses when their surrounding
|
||||
workflow can tolerate another model call. A retry can generate different output
|
||||
and incur another provider request.
|
||||
|
||||
## Example File
|
||||
|
||||
- `examples/http-run.json`
|
||||
181
docs/cli.md
181
docs/cli.md
@@ -10,124 +10,195 @@ go run ./cmd/scriptorium render \
|
||||
--input glossary=./examples/fixtures/glossary.yml
|
||||
```
|
||||
|
||||
`render` prepares and formats the prompt without calling an LLM.
|
||||
`render` prepares the prompt, loads input artifacts, resolves the execution
|
||||
profile, and prints the prepared request without calling an LLM.
|
||||
|
||||
## Command Overview
|
||||
|
||||
- `scriptorium run`: prepare prompt, call the configured LLM, write generated output, print a run summary.
|
||||
- `scriptorium render`: prepare prompt only; write prepared-run output as `text` or `json`.
|
||||
- `scriptorium serve`: start the HTTP server.
|
||||
- `scriptorium run`: prepare a prompt, call the configured LLM, write generated output, and print a run summary.
|
||||
- `scriptorium render`: prepare a prompt only; write prepared-run output as `text` or `json`.
|
||||
- `scriptorium serve`: start the HTTP server for `POST /v1/runs`.
|
||||
|
||||
Integration references:
|
||||
Canonical related references:
|
||||
|
||||
- [HTTP contract](integrations/http-api.md)
|
||||
- [Narratio subprocess contract](integrations/narratio.md)
|
||||
- [Configuration reference](config.md)
|
||||
- [HTTP API reference](api.md)
|
||||
- [Subprocess integration](integrations/subprocess.md)
|
||||
|
||||
## Common Argument Rules
|
||||
## Common Rules
|
||||
|
||||
- `--config` is supported by `run`, `render`, and `serve`.
|
||||
- `run` and `render` require:
|
||||
- `--prompt`
|
||||
- at least one `--input`
|
||||
- an effective `prompt_dir` from flags or config
|
||||
- `serve` requires an effective `prompt_dir` from flags or config.
|
||||
- `profile_dir` is optional. If omitted, only built-in profiles are available; if provided, custom profiles override built-ins with the same ID.
|
||||
- Built-in profile IDs are listed in the [configuration reference](config.md#profile-definition-files).
|
||||
- Positional arguments are rejected.
|
||||
- Prompt cache control is configured in prompt YAML (`messages[].cache_control`), not with CLI flags.
|
||||
- Provider-specific `reasoning_effort` and `extra_params` are configured in profile YAML or HTTP model overrides, not with CLI flags.
|
||||
- `run` and `render` require `--prompt`, at least one `--input`, and an effective `prompt_dir`.
|
||||
- `serve` requires an effective `prompt_dir`.
|
||||
- `profile_dir` is optional. Without it, only built-in profiles are available.
|
||||
- If `profile_dir` is set, custom profiles override built-in profiles with the same ID.
|
||||
- Prompt cache control, `session_id`, structured output, and provider-specific profile fields are configured in YAML, not with CLI flags.
|
||||
|
||||
Config precedence is:
|
||||
|
||||
1. built-in defaults
|
||||
2. config file values
|
||||
3. CLI flags
|
||||
|
||||
## Flag Reference
|
||||
|
||||
### `scriptorium run`
|
||||
|
||||
- `--config <path>`: app config file path.
|
||||
```bash
|
||||
scriptorium run [flags]
|
||||
```
|
||||
|
||||
Required through flags or config:
|
||||
|
||||
- `--prompt-dir <dir>`: prompt definition directory.
|
||||
|
||||
Required as flags:
|
||||
|
||||
- `--prompt <id>`: prompt ID to execute.
|
||||
- `--input name=path`: input file mapping. Repeat or use comma-separated mappings.
|
||||
|
||||
Optional flags:
|
||||
|
||||
- `--config <path>`: application config file.
|
||||
- `--profile-dir <dir>`: custom profile definition directory.
|
||||
- `--schema-dir <dir>`: schema base directory for `json_schema` validation.
|
||||
- `--prompt <id>`: prompt ID to execute. Required.
|
||||
- `--prompt-id <id>`: deprecated alias for `--prompt`.
|
||||
- `--profile <id>`: explicit profile override.
|
||||
- `--profile-id <id>`: deprecated alias for `--profile`.
|
||||
- `--input name=path`: input mapping (repeatable, comma-separated accepted).
|
||||
- `--var name=value`: template variable mapping (repeatable, comma-separated accepted).
|
||||
- `--out <path>`: write artifact body to file instead of stdout.
|
||||
- `--profile <id>`: execution profile override. If omitted, the prompt `default_profile` is used.
|
||||
- `--var name=value`: template variable mapping. Repeat or use comma-separated mappings.
|
||||
- `--out <path>`: write generated artifact body to a file instead of stdout.
|
||||
- `--llm-base-url <url>`: runtime endpoint override.
|
||||
- `--model <name>`: runtime model override.
|
||||
- `--api-key-env <name>`: runtime API key environment-variable name override.
|
||||
- `--api-key-env <name>`: runtime API-key environment variable name override.
|
||||
- `--temperature <float>`: runtime temperature override.
|
||||
- `--max-tokens <int>`: runtime max tokens override.
|
||||
- `--top-p <float>`: runtime top-p override.
|
||||
- `--timeout <duration>`: runtime timeout override (Go duration syntax, for example `30s`, `2m`).
|
||||
- `--timeout <duration>`: runtime timeout override using Go duration syntax, such as `30s` or `2m`.
|
||||
|
||||
Numeric runtime override flags are presence-aware:
|
||||
Deprecated aliases:
|
||||
|
||||
- omitted numeric flags preserve the selected profile/default value
|
||||
- explicit zero values override the selected profile/default value (`--temperature 0`, `--max-tokens 0`, `--top-p 0`, `--timeout 0s`)
|
||||
- `--prompt-id <id>`: alias for `--prompt`.
|
||||
- `--profile-id <id>`: alias for `--profile`.
|
||||
|
||||
Runtime override notes:
|
||||
|
||||
- Omitted numeric override flags preserve the selected profile/default value.
|
||||
- Explicit zero values override the selected profile/default value.
|
||||
- `--timeout 0s` disables the outbound HTTP client timeout for that request.
|
||||
- There is no raw API-key flag; use `--api-key-env`.
|
||||
|
||||
### `scriptorium render`
|
||||
|
||||
- Supports the same flags as `run`, except:
|
||||
- no `--schema-dir` flag.
|
||||
- Adds:
|
||||
- `--format text|json`: prepared-run output format.
|
||||
```bash
|
||||
scriptorium render [flags]
|
||||
```
|
||||
|
||||
Required through flags or config:
|
||||
|
||||
- `--prompt-dir <dir>`: prompt definition directory.
|
||||
|
||||
Required as flags:
|
||||
|
||||
- `--prompt <id>`: prompt ID to render.
|
||||
- `--input name=path`: input file mapping. Repeat or use comma-separated mappings.
|
||||
|
||||
Optional flags:
|
||||
|
||||
- `--config <path>`: application config file.
|
||||
- `--prompt-dir <dir>`: prompt definition directory.
|
||||
- `--profile-dir <dir>`: custom profile definition directory.
|
||||
- `--profile <id>`: execution profile override.
|
||||
- `--var name=value`: template variable mapping. Repeat or use comma-separated mappings.
|
||||
- `--out <path>`: write prepared-run output to a file instead of stdout.
|
||||
- `--llm-base-url <url>`: runtime endpoint override for the prepared request.
|
||||
- `--model <name>`: runtime model override for the prepared request.
|
||||
- `--api-key-env <name>`: runtime API-key environment variable name override.
|
||||
- `--temperature <float>`: runtime temperature override.
|
||||
- `--max-tokens <int>`: runtime max tokens override.
|
||||
- `--top-p <float>`: runtime top-p override.
|
||||
- `--timeout <duration>`: runtime timeout override using Go duration syntax.
|
||||
- `--format text|json`: prepared-run output format. Defaults to config `defaults.render_format`, then `text`.
|
||||
|
||||
Deprecated aliases:
|
||||
|
||||
- `--prompt-id <id>`: alias for `--prompt`.
|
||||
- `--profile-id <id>`: alias for `--profile`.
|
||||
|
||||
Notes:
|
||||
- `render` still resolves profile and runtime settings.
|
||||
- `render` still validates that `api_key_env` exists if the selected profile or overrides require it.
|
||||
|
||||
- `render` resolves profiles, loads schemas for `json_schema` prompts, and validates `api_key_env`.
|
||||
- `render` does not accept `--schema-dir`; use config `schema_dir` for render-time schema lookup.
|
||||
- `render` does not call the LLM.
|
||||
|
||||
### `scriptorium serve`
|
||||
|
||||
- `--config <path>`: app config file path.
|
||||
```bash
|
||||
scriptorium serve [flags]
|
||||
```
|
||||
|
||||
Required through flags or config:
|
||||
|
||||
- `--prompt-dir <dir>`: prompt definition directory.
|
||||
|
||||
Optional flags:
|
||||
|
||||
- `--config <path>`: application config file.
|
||||
- `--addr <listen-address>`: HTTP listen address.
|
||||
- `--prompt-dir <dir>`: prompt definition directory.
|
||||
- `--profile-dir <dir>`: custom profile definition directory.
|
||||
- `--schema-dir <dir>`: schema base directory for `json_schema` validation.
|
||||
- `--artifact-root <dir>`: base directory for HTTP `file` input references.
|
||||
- `--max-request-bytes <n>`: maximum HTTP request body bytes; `0` disables the limit.
|
||||
- `--max-artifact-bytes <n>`: maximum HTTP file artifact bytes; `0` disables the limit.
|
||||
- `--max-response-bytes <n>`: maximum encoded HTTP response body bytes; `0` disables the limit.
|
||||
|
||||
Notes:
|
||||
|
||||
- `serve` does not accept runtime model override flags such as `--model` or `--llm-base-url`.
|
||||
- HTTP `file` input references are rejected unless an artifact root is configured through `server.artifact_root` or `--artifact-root`.
|
||||
- `--artifact-root` affects only `serve`; `run` and `render` file input paths are unchanged.
|
||||
- HTTP request fields and error codes are documented in the [HTTP API reference](api.md).
|
||||
- HTTP `file` input references are rejected unless an artifact root is configured.
|
||||
- HTTP size-limit flags affect only `serve`.
|
||||
|
||||
## Input And Variable Syntax
|
||||
|
||||
- `--input name=path` maps prompt input names to local file paths.
|
||||
- `--var name=value` maps template variable names to values.
|
||||
- If a prompt defines `session_id: "{{ .session_id }}"`, pass the OpenRouter sticky-routing value with `--var session_id=<value>`.
|
||||
- `--var name=value` maps prompt template variables to string values.
|
||||
- Both flags can be repeated.
|
||||
- Both flags also support comma-separated batches, for example:
|
||||
- `--input transcript=./t.md,glossary=./g.yml`
|
||||
- `--var session_id=42,session_date=2026-05-04`
|
||||
- Both flags also accept comma-separated mappings, such as `--input transcript=./t.md,glossary=./g.yml`.
|
||||
- Values may contain `=` after the first separator, such as `--var note=a=b=c`.
|
||||
- Empty names and empty values are rejected.
|
||||
|
||||
CLI `run` and `render` convert every `--input` mapping to a `file` artifact
|
||||
reference. HTTP also supports `inline` input references; see [HTTP API
|
||||
reference](api.md).
|
||||
|
||||
## Output Behavior
|
||||
|
||||
`run`:
|
||||
|
||||
- Writes generated artifact content to stdout by default.
|
||||
- Writes generated artifact content to `--out` when provided.
|
||||
- Prints run summary metadata to stderr on success.
|
||||
- Appends `cached_tokens=<n> cache_write_tokens=<n>` to the summary only when the provider reports non-zero cache usage.
|
||||
- Prints a success summary to stderr.
|
||||
- Prints errors to stderr on failure.
|
||||
|
||||
`render`:
|
||||
|
||||
- Writes prepared-run output to stdout by default.
|
||||
- Writes prepared-run output to `--out` when provided.
|
||||
- Does not print a success summary line.
|
||||
- Does not print a success summary.
|
||||
|
||||
`serve`:
|
||||
|
||||
- Logs startup and server errors to stderr.
|
||||
|
||||
## Exit Codes
|
||||
|
||||
- `0`: success.
|
||||
- `1`: runtime/parse/config/load/render/generation/output-write error.
|
||||
- `2`: `run` completed, output was generated, but validation status is `failed`.
|
||||
|
||||
When `run` exits `2`, output may already be written to stdout or `--out`.
|
||||
- `1`: parse, config, load, render, generation, output-write, or runtime error.
|
||||
- `2`: `run` completed and wrote output, but validation status is `failed`.
|
||||
|
||||
## Common Workflows
|
||||
|
||||
Render prompt inputs and template variables as JSON:
|
||||
Render prompt inputs and variables as JSON:
|
||||
|
||||
```bash
|
||||
go run ./cmd/scriptorium render \
|
||||
@@ -139,7 +210,7 @@ go run ./cmd/scriptorium render \
|
||||
--format json
|
||||
```
|
||||
|
||||
Run a prompt with profile override and file output:
|
||||
Run a prompt with an explicit profile and file output:
|
||||
|
||||
```bash
|
||||
go run ./cmd/scriptorium run \
|
||||
@@ -151,12 +222,12 @@ go run ./cmd/scriptorium run \
|
||||
--out ./summary.md
|
||||
```
|
||||
|
||||
Start the HTTP server with explicit config:
|
||||
Start the HTTP server with example config:
|
||||
|
||||
```bash
|
||||
go run ./cmd/scriptorium serve --config ./examples/config.yml
|
||||
```
|
||||
|
||||
Copyable example script:
|
||||
Copyable maintained script:
|
||||
|
||||
- `examples/render-markdown-summary.sh`
|
||||
|
||||
277
docs/config.md
277
docs/config.md
@@ -2,30 +2,32 @@
|
||||
|
||||
## Config Discovery And Precedence
|
||||
|
||||
Application settings are loaded in this order:
|
||||
Application settings are resolved in this order:
|
||||
|
||||
1. Built-in defaults
|
||||
1. built-in defaults
|
||||
2. `config.yml` values
|
||||
3. CLI overrides
|
||||
|
||||
When `--config` is not provided, Scriptorium searches for config files in this order:
|
||||
When `--config` is omitted, Scriptorium searches:
|
||||
|
||||
1. `/usr/local/etc/scriptorium/config.yml`
|
||||
2. `/etc/scriptorium/config.yml`
|
||||
|
||||
If neither file exists, Scriptorium continues with built-in defaults.
|
||||
If neither file exists, Scriptorium uses built-in defaults. When
|
||||
`--config <path>` is provided, that file must exist and decode successfully.
|
||||
|
||||
When `--config <path>` is provided, that file is required.
|
||||
|
||||
## Minimal App Config
|
||||
## Minimal Working Config
|
||||
|
||||
```yaml
|
||||
prompt_dir: ./examples/prompts
|
||||
```
|
||||
|
||||
This is enough to use `run` and `render` when prompts select built-in profiles.
|
||||
This is enough for `run` and `render` when selected prompts use built-in
|
||||
profiles. Set `profile_dir` when prompts or requests use custom profiles.
|
||||
|
||||
## Production-Oriented App Config
|
||||
The maintained repository example is `examples/config.yml`.
|
||||
|
||||
## Production-Oriented Config
|
||||
|
||||
```yaml
|
||||
prompt_dir: /opt/scriptorium/prompts
|
||||
@@ -35,48 +37,56 @@ schema_dir: /opt/scriptorium/schemas
|
||||
server:
|
||||
addr: 127.0.0.1:8080
|
||||
artifact_root: /var/lib/scriptorium/artifacts
|
||||
max_request_bytes: 16777216
|
||||
max_artifact_bytes: 16777216
|
||||
max_response_bytes: 16777216
|
||||
|
||||
defaults:
|
||||
render_format: text
|
||||
```
|
||||
|
||||
## App Config File (`config.yml`)
|
||||
The maintained full example is `examples/config.full.yml`.
|
||||
|
||||
## App Config Reference
|
||||
|
||||
Top-level fields:
|
||||
|
||||
- `prompt_dir` (optional): default prompt definition directory.
|
||||
- `profile_dir` (optional): default custom profile definition directory.
|
||||
- `schema_dir` (optional): base directory for schema files used by `json_schema` validation.
|
||||
- `server.addr` (optional): default listen address for `serve`.
|
||||
- `server.artifact_root` (optional): base directory for HTTP `file` input references.
|
||||
- `defaults.render_format` (optional): default `render` output format (`text` or `json`).
|
||||
| Field | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `prompt_dir` | unset | Directory containing prompt definition YAML files. Required effectively by `run`, `render`, and `serve`. |
|
||||
| `profile_dir` | unset | Directory containing custom profile YAML files. Built-in profiles remain available when unset. |
|
||||
| `schema_dir` | `.` | Base directory for relative JSON Schema paths. |
|
||||
| `server` | `{}` | HTTP service settings used by `serve`. |
|
||||
| `defaults` | `{}` | Adapter defaults. |
|
||||
|
||||
Built-in defaults:
|
||||
`server` fields:
|
||||
|
||||
- `schema_dir`: `.`
|
||||
- `server.addr`: `:8080`
|
||||
- `server.artifact_root`: unset; HTTP `file` input references are rejected until configured.
|
||||
- `defaults.render_format`: `text`
|
||||
| Field | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `server.addr` | `:8080` | Listen address for `serve`. |
|
||||
| `server.artifact_root` | unset | Base directory for HTTP `file` input references. Without it, HTTP file refs are rejected. |
|
||||
| `server.max_request_bytes` | `16777216` | Maximum encoded HTTP request body bytes. `0` disables the limit. |
|
||||
| `server.max_artifact_bytes` | `16777216` | Maximum HTTP file artifact bytes. `0` disables the limit. |
|
||||
| `server.max_response_bytes` | `16777216` | Maximum encoded HTTP response bytes. `0` disables the limit. |
|
||||
|
||||
Validation behavior:
|
||||
`defaults` fields:
|
||||
|
||||
- Config decoding is strict; unknown YAML fields are rejected.
|
||||
- Raw API key fields are not supported in `config.yml`.
|
||||
| Field | Default | Description |
|
||||
| --- | --- | --- |
|
||||
| `defaults.render_format` | `text` | Default `render` output format: `text` or `json`. |
|
||||
|
||||
HTTP artifact root behavior:
|
||||
Config rules:
|
||||
|
||||
- `server.artifact_root` applies only to `serve`.
|
||||
- HTTP `inline` input references work without an artifact root.
|
||||
- HTTP `file` input references are resolved against `server.artifact_root` and must stay inside it.
|
||||
- Relative traversal and absolute paths outside the root are rejected.
|
||||
- Symlinks inside the root are followed by the operating system; do not make the artifact root writable by untrusted users.
|
||||
- CLI `run` and `render` file inputs keep their normal direct filesystem path behavior.
|
||||
- YAML decoding is strict; unknown fields are rejected.
|
||||
- HTTP size limits must be greater than or equal to `0`.
|
||||
- Empty string config values are ignored.
|
||||
- Raw API key fields are not supported in app config.
|
||||
|
||||
## Prompt Definition Files
|
||||
|
||||
Prompt definitions are YAML files anywhere under `prompt_dir`, including nested subdirectories.
|
||||
|
||||
Subdirectories are organizational only. Callers still select prompts by the YAML `id`, not by file path. For example, `prompts/dnd/recap.yaml` may still declare `id: dnd.recap`, and callers use `--prompt dnd.recap`.
|
||||
Prompt definitions are YAML files anywhere under `prompt_dir`. Nested
|
||||
directories are organizational; callers select prompts by YAML `id`, not file
|
||||
path.
|
||||
|
||||
Example:
|
||||
|
||||
@@ -109,84 +119,76 @@ output:
|
||||
repair_attempts: 0
|
||||
```
|
||||
|
||||
Field reference:
|
||||
Prompt fields:
|
||||
|
||||
- `id` (required): prompt identifier.
|
||||
- `version` (required): prompt version.
|
||||
- `default_profile` (optional): profile ID used when request does not provide `profile_id`.
|
||||
- `description` (optional): prompt description.
|
||||
- `session_id` (optional): Go-template string for OpenRouter sticky-routing `session_id`; rendered from request vars.
|
||||
- `inputs` (optional list): expected named inputs.
|
||||
- `messages` (required list): prompt message templates.
|
||||
- `output` (required object): output contract.
|
||||
| Field | Required | Description |
|
||||
| --- | --- | --- |
|
||||
| `id` | yes | Prompt identifier used by `--prompt` and HTTP `prompt_id`. |
|
||||
| `version` | yes | Prompt version. |
|
||||
| `default_profile` | no | Profile ID used when a request does not provide a profile. |
|
||||
| `description` | no | Human-readable description. |
|
||||
| `session_id` | no | Go-template string rendered from request vars and forwarded as provider `session_id` when non-empty. |
|
||||
| `inputs` | no | Named input declarations. |
|
||||
| `messages` | yes | Chat message templates. |
|
||||
| `output` | yes | Output format and validation contract. |
|
||||
|
||||
`inputs[]` fields:
|
||||
|
||||
- `name` (required)
|
||||
- `required` (optional, boolean)
|
||||
- `required` (optional boolean)
|
||||
- `content_type` (optional metadata)
|
||||
- `description` (optional)
|
||||
|
||||
`messages[]` fields:
|
||||
|
||||
- `role` (required)
|
||||
- `content` or `content_file` (exactly one is required)
|
||||
- `cache_control` (optional object): provider prompt-cache metadata for this message
|
||||
- exactly one of `content` or `content_file`
|
||||
- `cache_control` (optional)
|
||||
|
||||
Message rules:
|
||||
|
||||
- `content_file` resolves relative to the prompt YAML file location.
|
||||
- Repeated roles are allowed.
|
||||
- `content_file` is resolved relative to the prompt YAML file location.
|
||||
- Nested prompt files keep the same relative `content_file` behavior; `./recap.user.md` next to `dnd/recap.yaml` resolves from `dnd/`.
|
||||
- Prompt decoding is strict; unknown YAML fields are rejected.
|
||||
- Duplicate prompt IDs are invalid. If multiple files declare the requested prompt ID, Scriptorium fails instead of choosing one.
|
||||
- Prompt YAML decoding is strict.
|
||||
- Duplicate input names are invalid.
|
||||
- Duplicate prompt IDs are invalid for a requested ID/version.
|
||||
|
||||
`messages[].cache_control` fields:
|
||||
|
||||
- `type` (required when `cache_control` is present): currently only `ephemeral`.
|
||||
- `ttl` (optional): currently only `1h`; omitted from outbound requests when unset.
|
||||
| Field | Required | Supported values |
|
||||
| --- | --- | --- |
|
||||
| `type` | yes | `ephemeral` |
|
||||
| `ttl` | no | `1h` |
|
||||
|
||||
Example cache-controlled message:
|
||||
`session_id` behavior:
|
||||
|
||||
```yaml
|
||||
messages:
|
||||
- role: system
|
||||
content_file: ./stable_context.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
ttl: 1h
|
||||
- role: user
|
||||
content: |
|
||||
{{input "transcript"}}
|
||||
```
|
||||
|
||||
Use cache control on stable reusable prompt content. Dynamic per-run inputs before the cache-controlled message change the provider cache key.
|
||||
|
||||
Example prompt-level session ID:
|
||||
|
||||
```yaml
|
||||
session_id: "{{ .session_id }}"
|
||||
```
|
||||
|
||||
When configured, `session_id` is rendered with the same variable context as messages. The rendered value is trimmed, omitted when empty, and rejected if longer than 256 characters. CLI callers pass the value through `--var session_id=<value>`; HTTP callers pass it through `"vars": {"session_id": "<value>"}`.
|
||||
- Rendered with the same variable context as message templates.
|
||||
- Trimmed and omitted when empty.
|
||||
- Rejected when longer than 256 Unicode code points.
|
||||
- CLI callers pass variables with `--var`; HTTP callers use `vars`.
|
||||
|
||||
`output` fields:
|
||||
|
||||
- `format` (required): `text`, `markdown`, or `json`.
|
||||
- `validation_mode` (required): `none`, `basic`, `json`, or `json_schema`.
|
||||
- `schema_path` (required when `validation_mode: json_schema`).
|
||||
- `repair_attempts` (required): integer `>= 0`.
|
||||
| Field | Required | Supported values |
|
||||
| --- | --- | --- |
|
||||
| `format` | yes | `text`, `markdown`, `json` |
|
||||
| `validation_mode` | yes | `none`, `basic`, `json`, `json_schema` |
|
||||
| `schema_path` | only for `json_schema` | Relative to `schema_dir` unless absolute. |
|
||||
| `repair_attempts` | yes | Integer greater than or equal to `0`. |
|
||||
|
||||
Repair behavior boundary:
|
||||
Repair boundary:
|
||||
|
||||
- `repair_attempts` is part of the prompt contract.
|
||||
- CLI and HTTP currently construct the runner without a repairer, so normal `run`/`serve` execution does not perform output repair attempts.
|
||||
- The current CLI and HTTP wiring constructs the runner without a repairer, so normal `run` and `serve` execution does not perform repair attempts.
|
||||
|
||||
## Profile Definition Files
|
||||
|
||||
Scriptorium includes built-in execution profiles. Custom execution profiles are YAML files anywhere under `profile_dir`, including nested subdirectories.
|
||||
Execution profiles are YAML files anywhere under `profile_dir`. Nested
|
||||
directories are organizational; callers select profiles by YAML `id`, not file
|
||||
path.
|
||||
|
||||
Subdirectories are organizational only. Callers still select profiles by the YAML `id`, not by file path. For example, `profiles/local/local-quality.yaml` may still declare `id: local-quality`, and callers use `--profile local-quality`.
|
||||
Scriptorium also ships built-in profiles. Custom profiles override built-ins
|
||||
with the same ID.
|
||||
|
||||
Example:
|
||||
|
||||
@@ -203,36 +205,44 @@ service_tier: priority
|
||||
reasoning_effort: medium
|
||||
extra_params:
|
||||
provider_route: primary
|
||||
provider_options:
|
||||
retry_budget: 2
|
||||
```
|
||||
|
||||
Field reference:
|
||||
Profile fields:
|
||||
|
||||
- `id` (required)
|
||||
- `endpoint` (required)
|
||||
- `model` (required)
|
||||
- `temperature` (optional): range `0..2`
|
||||
- `max_tokens` (optional): `>= 0`
|
||||
- `top_p` (optional): range `0..1`
|
||||
- `timeout_seconds` (optional): `>= 0`
|
||||
- `service_tier` (optional): provider-specific request tier such as OpenRouter `flex` or `priority`
|
||||
- `reasoning_effort` (optional): serialized as top-level `reasoning_effort` in outbound chat-completions requests
|
||||
- `api_key_env` (optional)
|
||||
- `extra_params` (optional map): JSON-compatible provider-specific parameters. Values may be strings, numbers, booleans, objects, or arrays.
|
||||
| Field | Required | Description |
|
||||
| --- | --- | --- |
|
||||
| `id` | yes | Profile identifier. |
|
||||
| `endpoint` | yes | OpenAI-compatible base URL including `/v1`. |
|
||||
| `model` | yes | Provider model name. |
|
||||
| `temperature` | no | Range `0..2`. |
|
||||
| `max_tokens` | no | Integer greater than or equal to `0`. |
|
||||
| `top_p` | no | Range `0..1`. |
|
||||
| `timeout_seconds` | no | Integer greater than or equal to `0`. |
|
||||
| `service_tier` | no | Provider-specific request tier. |
|
||||
| `reasoning_effort` | no | Provider-specific reasoning setting. |
|
||||
| `api_key_env` | no | Environment variable name containing the API key. |
|
||||
| `extra_params` | no | JSON-compatible provider-specific top-level request fields. |
|
||||
|
||||
Execution defaults before profile/request overrides:
|
||||
|
||||
| Field | Default |
|
||||
| --- | --- |
|
||||
| `temperature` | `0.0` |
|
||||
| `max_tokens` | `0` |
|
||||
| `top_p` | `1.0` |
|
||||
| `timeout_seconds` | `600` |
|
||||
|
||||
Profile rules:
|
||||
|
||||
- `profile_dir` is optional. If omitted, only built-in profiles are available.
|
||||
- If `profile_dir` is set, custom profiles from that directory override built-in profiles with the same `id`.
|
||||
- Duplicate IDs within the custom profile directory are invalid. Matching IDs across custom and built-in profiles are valid override behavior.
|
||||
- Profile decoding is strict; unknown YAML fields are rejected.
|
||||
- Profile YAML decoding is strict.
|
||||
- Duplicate custom profile IDs are invalid.
|
||||
- Matching custom and built-in IDs are valid override behavior.
|
||||
- Raw `api_key` is rejected; use `api_key_env`.
|
||||
- If `api_key_env` is set, that environment variable must be set when preparing/running.
|
||||
- Duplicate profile IDs are invalid. If multiple files declare the requested profile ID, Scriptorium fails instead of choosing one.
|
||||
- `extra_params` keys must not be empty and must not collide with reserved outbound request fields: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or `response_format`.
|
||||
- If `api_key_env` is set, the named environment variable must be set before `run`, `render`, or HTTP execution can prepare the request.
|
||||
- Profile numeric fields merge by non-zero value. Request overrides are presence-aware, so explicit zero values are supported through CLI flags or HTTP model overrides.
|
||||
- `extra_params` keys must not be empty and must not collide with reserved outbound fields: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or `response_format`.
|
||||
|
||||
Built-in profile IDs:
|
||||
Built-in profile catalog:
|
||||
|
||||
| Provider | ID | Model | API key env |
|
||||
| --- | --- | --- | --- |
|
||||
@@ -260,56 +270,55 @@ Built-in profile IDs:
|
||||
| openai | `gpt-5-mini` | `openai/gpt-5.4-mini` | `OPENROUTER_API_KEY` |
|
||||
| openai | `gpt-5-nano` | `openai/gpt-5.4-nano` | `OPENROUTER_API_KEY` |
|
||||
|
||||
Current outbound request behavior:
|
||||
|
||||
- The OpenAI-compatible client currently serializes: `model`, optional `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, optional `response_format` for `json_schema` prompts, and `extra_params`.
|
||||
- `extra_params` are flattened into provider-specific top-level JSON request fields. They are not wrapped in an `extra_params` object on the outbound provider request.
|
||||
- Messages without `cache_control` serialize with string `content`.
|
||||
- Messages with `cache_control` serialize as a single text content-block array containing `cache_control`.
|
||||
|
||||
## Schema Behavior
|
||||
|
||||
Schemas are JSON files, typically in `schema_dir`.
|
||||
Schemas are JSON files, typically under `schema_dir`.
|
||||
|
||||
Rules:
|
||||
|
||||
- `output.validation_mode: json_schema` requires `output.schema_path`.
|
||||
- Relative `schema_path` values resolve from `schema_dir`, including explicit nested paths such as `dnd/structured_events.schema.json`.
|
||||
- Relative `schema_path` values resolve from `schema_dir`.
|
||||
- Absolute `schema_path` values are used directly.
|
||||
- Scriptorium does not recursively search schemas by basename; nested schemas must be referenced by their relative path.
|
||||
- Missing or invalid schema documents cause runtime validation errors.
|
||||
- Invalid generated JSON causes validation status `failed` (not a runtime error).
|
||||
- Nested schemas must be referenced by relative path; schemas are not searched recursively by basename.
|
||||
- Missing or invalid schema documents are runtime validation errors.
|
||||
- Invalid generated JSON produces validation status `failed`, not a runtime error.
|
||||
|
||||
Supported artifact reference types for request inputs are `file` and `inline`.
|
||||
For HTTP `serve`, `file` references require `server.artifact_root` and must stay
|
||||
inside that root. CLI `run` and `render` file inputs are not restricted by
|
||||
`server.artifact_root`.
|
||||
## Artifact References
|
||||
|
||||
Supported request input artifact reference types are:
|
||||
|
||||
- `file`
|
||||
- `inline`
|
||||
|
||||
CLI `run` and `render` create `file` references from `--input name=path`.
|
||||
|
||||
HTTP `file` references require `server.artifact_root` or `serve
|
||||
--artifact-root`. Relative file URIs resolve under that root. Absolute paths
|
||||
and relative traversal outside the root are rejected by lexical checks. Symlinks
|
||||
inside the root are followed by the operating system, including symlinks that
|
||||
point outside the root.
|
||||
|
||||
HTTP `inline` references do not require an artifact root.
|
||||
|
||||
## Secrets Handling
|
||||
|
||||
- Keep secret values in environment variables.
|
||||
- Store only environment-variable names in profile `api_key_env`.
|
||||
- Do not put raw API keys in config, prompts, profiles, CLI flags, or HTTP request bodies.
|
||||
- Store only environment-variable names in `api_key_env`.
|
||||
- Do not put raw API keys in config, prompts, profiles, CLI arguments, examples, or HTTP request bodies.
|
||||
|
||||
## Maintained Examples
|
||||
|
||||
- App config: `examples/config.yml`
|
||||
- Minimal app config: `examples/config.yml`
|
||||
- Full app config: `examples/config.full.yml`
|
||||
- Prompt examples: `examples/prompts/`
|
||||
- Custom profile examples: `examples/profiles/`
|
||||
- Schema examples: `examples/schemas/`
|
||||
- Input fixtures: `examples/fixtures/`
|
||||
- Render example script: `examples/render-markdown-summary.sh`
|
||||
- HTTP request example: `examples/http-run.json`
|
||||
|
||||
Example organizational layout:
|
||||
|
||||
```text
|
||||
examples/prompts/dnd/recap.yaml
|
||||
examples/profiles/local/local-quality.yaml
|
||||
examples/schemas/dnd/structured_events.schema.json
|
||||
```
|
||||
- Render script: `examples/render-markdown-summary.sh`
|
||||
- HTTP request-shape example: `examples/http-run.json`
|
||||
|
||||
## Integration References
|
||||
|
||||
- [Inbound HTTP contract](integrations/http-api.md)
|
||||
- [CLI reference](cli.md)
|
||||
- [HTTP API reference](api.md)
|
||||
- [Outbound OpenAI-compatible contract](integrations/openai-compatible-chat.md)
|
||||
|
||||
@@ -1,13 +1,122 @@
|
||||
# Consumer API Overview
|
||||
# Consumer Integration Overview
|
||||
|
||||
Scriptorium can be used by consumers through three implemented surfaces:
|
||||
This guide is for applications that call Scriptorium from another codebase.
|
||||
|
||||
- CLI commands, documented in [CLI reference](../cli.md).
|
||||
- HTTP `POST /v1/runs`, documented in [HTTP API integration](../integrations/http-api.md).
|
||||
- Go package `gitea.maximumdirect.net/eric/scriptorium`, documented in [pkg-scriptorium](pkg-scriptorium.md).
|
||||
Scriptorium exposes three integration surfaces:
|
||||
|
||||
The Go package is the typed in-process API. It prepares prompts, runs prompts, accepts file or inline artifacts, supports per-request execution overrides, and exposes stable public errors for `errors.Is`.
|
||||
| Surface | Use when |
|
||||
| --- | --- |
|
||||
| Go package | The consumer is Go, needs typed requests/results, or wants injected LLM clients for tests. |
|
||||
| CLI subprocess | The consumer wants process isolation or is not written in Go. |
|
||||
| HTTP API | The consumer needs a service boundary or remote access to `POST /v1/runs`. |
|
||||
|
||||
Use the Go package when the caller is a Go program that wants typed requests/results, context cancellation, repeated calls without subprocess overhead, or fake LLM injection for tests. Use the CLI or HTTP surfaces when process isolation, language neutrality, or an HTTP boundary is preferred.
|
||||
Canonical references:
|
||||
|
||||
Raw API key values are not accepted in public payloads and are not returned in prepared or run results. Execution profiles may reference an environment variable name through `api_key_env`.
|
||||
- Go package: [Package scriptorium](pkg-scriptorium.md)
|
||||
- CLI subprocess: [Subprocess integration](../integrations/subprocess.md)
|
||||
- HTTP: [HTTP API reference](../api.md)
|
||||
- File formats: [Configuration reference](../config.md)
|
||||
|
||||
## Required Deployment Inputs
|
||||
|
||||
Every integration needs operators to provide:
|
||||
|
||||
- prompt definitions;
|
||||
- profile definitions or built-in profile IDs;
|
||||
- schema files when prompts use `json_schema`;
|
||||
- input artifacts or inline input bodies;
|
||||
- API-key environment variables or direct per-request keys where supported.
|
||||
|
||||
Raw API keys do not belong in config, prompt files, profile YAML, CLI
|
||||
arguments, or HTTP request bodies.
|
||||
|
||||
## Recommended Workflow
|
||||
|
||||
Use the Go package when:
|
||||
|
||||
- the consumer is a Go application;
|
||||
- the application needs `context.Context` cancellation;
|
||||
- repeated calls should avoid subprocess startup;
|
||||
- tests need a fake LLM client;
|
||||
- direct per-request `RunRequest.APIKey` is required.
|
||||
|
||||
Use the CLI subprocess when:
|
||||
|
||||
- the consumer is not Go;
|
||||
- process isolation is useful;
|
||||
- stdout/stderr separation and exit codes are enough;
|
||||
- the consumer already manages local files and environment variables.
|
||||
|
||||
Use HTTP when:
|
||||
|
||||
- Scriptorium should run as a service;
|
||||
- multiple clients need a shared prompt/profile deployment;
|
||||
- clients can reach a trusted, protected HTTP boundary.
|
||||
|
||||
## Minimal Go Example
|
||||
|
||||
```go
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
PromptDir: "./examples/prompts",
|
||||
ProfileDir: "./examples/profiles",
|
||||
SchemaDir: "./examples/schemas",
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{
|
||||
PromptID: "generic.markdown_summary",
|
||||
Inputs: map[string]scriptorium.ArtifactRef{
|
||||
"transcript": scriptorium.File("./examples/fixtures/transcript.md"),
|
||||
"glossary": scriptorium.File("./examples/fixtures/glossary.yml"),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_ = prepared.Messages
|
||||
```
|
||||
|
||||
Run the maintained package example:
|
||||
|
||||
```bash
|
||||
go run ./examples/go-library/prepare
|
||||
```
|
||||
|
||||
## Subprocess Workflow
|
||||
|
||||
Invoke `scriptorium render` for preflight and `scriptorium run` for generation.
|
||||
Capture stdout and stderr separately. Treat exit code `2` from `run` as a
|
||||
completed generation with failed validation.
|
||||
|
||||
See [Subprocess integration](../integrations/subprocess.md) for the stable
|
||||
invocation contract.
|
||||
|
||||
## HTTP Workflow
|
||||
|
||||
Run `scriptorium serve` behind trusted controls and send JSON requests to
|
||||
`POST /v1/runs`.
|
||||
|
||||
Do not duplicate endpoint schemas in consumers. Use the [HTTP API
|
||||
reference](../api.md) as the authoritative contract.
|
||||
|
||||
## Consumer Responsibilities
|
||||
|
||||
Consumers are responsible for:
|
||||
|
||||
- selecting prompt/profile IDs as deployment configuration;
|
||||
- supplying all required inputs and vars;
|
||||
- protecting generated artifacts and rendered prompts as sensitive data;
|
||||
- deciding whether to keep output when validation fails;
|
||||
- implementing retries only when another model call is acceptable.
|
||||
|
||||
Scriptorium does not persist run state. Retrying a failed or timed-out request
|
||||
can produce different output and can incur another provider request.
|
||||
|
||||
## Status Behavior
|
||||
|
||||
- Go package methods return typed results or errors that support `errors.Is`.
|
||||
- CLI `run` exits `2` when generation succeeds but validation fails.
|
||||
- HTTP returns `200 OK` for generated-content validation failures and exposes the failed status in the response body.
|
||||
- Runtime validation failures are errors.
|
||||
|
||||
@@ -6,7 +6,22 @@ Import path:
|
||||
import "gitea.maximumdirect.net/eric/scriptorium"
|
||||
```
|
||||
|
||||
The root package is a public facade over Scriptorium's prompt execution use case. It keeps `internal/*` packages private while exposing typed construction, preparation, execution, inputs, results, and errors.
|
||||
The root package is the public Go facade for Scriptorium's prompt prepare/run
|
||||
workflow. It exposes typed requests, results, source options, injected LLM
|
||||
clients, and stable public errors while keeping `internal/*` packages private.
|
||||
|
||||
## Intended Use Cases
|
||||
|
||||
Use the package when a Go application needs:
|
||||
|
||||
- in-process prompt preparation or execution;
|
||||
- typed request/result structs;
|
||||
- direct `context.Context` cancellation;
|
||||
- injected/fake LLM clients for tests;
|
||||
- direct per-request `RunRequest.APIKey`.
|
||||
|
||||
Use [Subprocess integration](../integrations/subprocess.md) or the [HTTP API](../api.md)
|
||||
when a process or service boundary is preferred.
|
||||
|
||||
## Construct An Engine
|
||||
|
||||
@@ -21,21 +36,56 @@ if err != nil {
|
||||
}
|
||||
```
|
||||
|
||||
`PromptDir` is required unless an explicit prompt source option is supplied. `ProfileDir` is optional; omit it to use built-in profiles only, or set it to overlay custom profiles above built-ins. `SchemaDir` defaults to the built-in schema directory. `Timeout` and `HTTPClient` configure the default OpenAI-compatible client used by `Run` when no custom LLM client is supplied.
|
||||
`Config` fields:
|
||||
|
||||
## Asset Sources
|
||||
| Field | Description |
|
||||
| --- | --- |
|
||||
| `PromptDir` | Prompt definition directory. Required unless `WithPromptFS` or `WithPromptFile` is used. |
|
||||
| `ProfileDir` | Optional custom profile directory overlaid above built-in profiles. |
|
||||
| `SchemaDir` | Schema directory. Defaults to `.` when empty. |
|
||||
| `Timeout` | Default timeout for the built-in OpenAI-compatible client. |
|
||||
| `HTTPClient` | Optional HTTP client for the built-in OpenAI-compatible client. |
|
||||
|
||||
Directory fields on `Config` remain the compatibility path. Explicit source options override the matching directory field:
|
||||
`NewEngine` accepts `nil` options and ignores them. Invalid construction wraps
|
||||
`ErrInvalidConfig`.
|
||||
|
||||
- `WithPromptFS(fsys, root)` and `WithPromptFile(path)`
|
||||
- `WithProfileFS(fsys, root)` and `WithProfileFile(path)`
|
||||
- `WithSchemaFS(fsys, root)` and `WithSchemaFile(path)`
|
||||
## Source Options
|
||||
|
||||
Prompt and profile sources load standard Scriptorium YAML with the same strict validation as directory sources. Prompt `content_file` paths resolve relative to the prompt file in the same source. Profile options overlay custom profiles above built-ins. Schema `fs.FS` sources preserve prompt `schema_path` semantics; schema file options expose the file by its base name.
|
||||
Directory fields are the compatibility path. Explicit source options override
|
||||
the matching directory field.
|
||||
|
||||
Prompt sources:
|
||||
|
||||
- `WithPromptFS(fsys, root)`
|
||||
- `WithPromptFile(path)`
|
||||
|
||||
Profile sources:
|
||||
|
||||
- `WithProfileFS(fsys, root)`
|
||||
- `WithProfileFile(path)`
|
||||
- `WithProfiles(profiles...)`
|
||||
|
||||
Schema sources:
|
||||
|
||||
- `WithSchemaFS(fsys, root)`
|
||||
- `WithSchemaFile(path)`
|
||||
|
||||
LLM source:
|
||||
|
||||
- `WithLLMClient(client)`
|
||||
|
||||
Source behavior:
|
||||
|
||||
- Prompt and profile YAML use the same strict rules as directory loading.
|
||||
- Prompt `content_file` values resolve relative to the prompt file.
|
||||
- `fs.FS` roots are containment boundaries for prompt content files and schema paths.
|
||||
- File options expose the selected file by its base name.
|
||||
- Profile source precedence is in-memory profiles, then explicit profile file/FS/directory source, then built-ins.
|
||||
- `WithLLMClient(nil)` returns `ErrInvalidConfig`.
|
||||
|
||||
## In-Memory Profiles
|
||||
|
||||
Use `WithProfiles` when the consuming application already has profile settings in typed Go configuration:
|
||||
Use `WithProfiles` when the application already has typed model settings:
|
||||
|
||||
```go
|
||||
profile := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
|
||||
@@ -48,13 +98,33 @@ profile := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfi
|
||||
engine, err := scriptorium.NewEngine(cfg, scriptorium.WithProfiles(profile))
|
||||
```
|
||||
|
||||
In-memory profiles have highest precedence, followed by configured profile file/FS/directory sources, then built-in profiles. Duplicate IDs in one `WithProfiles` call return `ErrInvalidConfig`.
|
||||
`Profile` and `OpenAICompatibleProfileConfig` include:
|
||||
|
||||
`Profile` and `OpenAICompatibleProfileConfig` include endpoint, model, numeric defaults, service tier, reasoning effort, `APIKeyRequired`, and JSON-compatible `ExtraParams`. `WithProfiles` validates `ExtraParams` and returns `ErrInvalidConfig` for unsupported values such as functions, channels, non-string map keys, non-finite floats, or cyclic values. Raw API-key fields are not accepted. When `APIKeyRequired` is true, pass the secret with `RunRequest.APIKey`.
|
||||
- `ID`
|
||||
- `Endpoint`
|
||||
- `Model`
|
||||
- `Temperature`
|
||||
- `MaxTokens`
|
||||
- `TopP`
|
||||
- `TimeoutSeconds`
|
||||
- `ServiceTier`
|
||||
- `ReasoningEffort`
|
||||
- `APIKeyRequired`
|
||||
- `ExtraParams`
|
||||
|
||||
## Prepare A Prompt
|
||||
`WithProfiles` rejects duplicate IDs in one call. In-memory profiles do not
|
||||
store raw keys. When `APIKeyRequired` is true, pass the secret on each request
|
||||
with `RunRequest.APIKey`.
|
||||
|
||||
`Prepare` resolves the prompt definition, profile, inputs, variables, output contract, structured-output metadata, and rendered messages without calling an LLM.
|
||||
`ExtraParams` must be JSON-compatible: strings, booleans, finite numbers,
|
||||
objects with string keys, arrays/slices, and nil. Unsupported values, non-string
|
||||
map keys, non-finite floats, and cycles return `ErrInvalidConfig` for profiles
|
||||
or `ErrInvalidRequest` for request overrides.
|
||||
|
||||
## Prepare Workflow
|
||||
|
||||
`Prepare` resolves prompt/profile/input/schema state and renders messages
|
||||
without calling an LLM.
|
||||
|
||||
```go
|
||||
prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{
|
||||
@@ -67,18 +137,19 @@ prepared, err := engine.Prepare(ctx, scriptorium.RunRequest{
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_ = prepared.Messages
|
||||
_ = prepared.EffectiveModelParams
|
||||
```
|
||||
|
||||
Input helpers:
|
||||
`PreparedRun` includes prompt ID/version/hash, selected profile, effective
|
||||
model params, output contract, structured-output metadata, input hashes,
|
||||
rendered prompt hash, rendered messages, and timing fields. It does not include
|
||||
raw API-key values, model output, validation results, or internal target
|
||||
presence metadata.
|
||||
|
||||
- `scriptorium.File(path)` loads an input artifact from a file.
|
||||
- `scriptorium.Inline(body)` passes inline input content.
|
||||
- `scriptorium.InlineWithURI(uri, body)` passes inline content with URI metadata.
|
||||
## Run Workflow
|
||||
|
||||
## Run A Prompt
|
||||
|
||||
`Run` prepares the prompt, calls the configured LLM client, builds the output artifact, and validates the output.
|
||||
`Run` calls `Prepare`, invokes the configured LLM client, builds the output
|
||||
artifact, and validates the output.
|
||||
|
||||
```go
|
||||
result, err := engine.Run(ctx, scriptorium.RunRequest{
|
||||
@@ -95,13 +166,25 @@ if err != nil {
|
||||
_ = result.Artifact
|
||||
```
|
||||
|
||||
`RunResult` includes the run ID, output artifact, raw output, validation result, prompt/profile/model metadata, effective model parameters, input hashes, token/cache usage, and timing fields. Validation content failures return a successful `RunResult` with failed validation status. Runtime validation errors return `ErrValidation`.
|
||||
`RunResult` includes run ID, output artifact, raw output, validation result,
|
||||
prompt/profile/model metadata, effective model params, input hashes, usage, and
|
||||
timing fields.
|
||||
|
||||
For the public Go API, pass provider credentials with `RunRequest.APIKey`. The value is request-scoped, uses `json:"-"`, is preferred over profile `api_key_env` by the default OpenAI-compatible client, and is not included in `PreparedRun` or `RunResult` JSON. Normal Go string formatting of `RunRequest` reports only whether a direct key is set. Do not store raw keys in config, prompt files, or profile YAML.
|
||||
Generated-content validation failures return a successful `RunResult` with
|
||||
`Validation.Status == ValidationFailed`. Runtime/schema validation errors
|
||||
return an error that matches `ErrValidation`.
|
||||
|
||||
Avoid logging raw request structs with reflection-based debug dumpers; exported fields remain visible to tools that bypass `String` and `GoString` methods.
|
||||
## Inputs
|
||||
|
||||
## Inject An LLM Client
|
||||
Input helpers:
|
||||
|
||||
- `File(path)`: file-backed artifact reference.
|
||||
- `Inline(body)`: inline artifact body.
|
||||
- `InlineWithURI(uri, body)`: inline artifact body with URI metadata.
|
||||
|
||||
Input map keys must match the prompt's expected input names.
|
||||
|
||||
## Injected LLM Clients
|
||||
|
||||
Use `WithLLMClient` for tests or custom model integrations:
|
||||
|
||||
@@ -118,11 +201,34 @@ func (fakeLLM) Generate(ctx context.Context, req scriptorium.GenerateRequest) (*
|
||||
engine, err := scriptorium.NewEngine(cfg, scriptorium.WithLLMClient(fakeLLM{}))
|
||||
```
|
||||
|
||||
The injected client receives the rendered prompt, effective execution target, target presence metadata for explicit numeric overrides, structured-output spec, and request API key when provided. `GenerateRequest.APIKey` also uses `json:"-"`, and normal Go string formatting reports only whether a direct key is set. Custom and fake clients should avoid logging or serializing it. `WithLLMClient(nil)` returns `ErrInvalidConfig`.
|
||||
Injected clients receive:
|
||||
|
||||
## Request Overrides
|
||||
- rendered prompt;
|
||||
- effective execution target;
|
||||
- numeric target presence metadata;
|
||||
- structured-output spec when applicable;
|
||||
- direct request API key when provided.
|
||||
|
||||
`RunRequest.Execution` accepts per-request overrides. Numeric override fields are pointers so explicit zero values are preserved:
|
||||
Custom clients should not log raw prompts or API keys by default.
|
||||
|
||||
## Overrides And API Keys
|
||||
|
||||
`RunRequest` fields:
|
||||
|
||||
| Field | Description |
|
||||
| --- | --- |
|
||||
| `PromptID` | Prompt ID. |
|
||||
| `PromptVersion` | Optional prompt version filter. |
|
||||
| `ProfileID` | Optional profile override. |
|
||||
| `APIKey` | Direct per-request API key. |
|
||||
| `Inputs` | Input artifact references. |
|
||||
| `Vars` | Template variables. |
|
||||
| `Execution` | Per-request model overrides. |
|
||||
| `Validation` | Per-request output contract override. |
|
||||
| `Metadata` | Request metadata reserved for callers. |
|
||||
|
||||
`RunRequest.Execution` uses pointer fields for numeric values so explicit zero
|
||||
overrides are preserved:
|
||||
|
||||
```go
|
||||
zero := 0
|
||||
@@ -131,11 +237,19 @@ req.Execution = &scriptorium.ExecutionTargetOverride{
|
||||
}
|
||||
```
|
||||
|
||||
`ExecutionTargetOverride.ExtraParams` accepts JSON-compatible values and copies typed maps/slices so later caller mutation does not affect the run. Unsupported values, non-string map keys, non-finite floats, and cycles return `ErrInvalidRequest`.
|
||||
Direct `RunRequest.APIKey` takes precedence over profile `api_key_env` for the
|
||||
default OpenAI-compatible client. It is request-scoped, uses `json:"-"`, and is
|
||||
not included in `PreparedRun` or `RunResult` JSON. Normal Go string formatting
|
||||
of `RunRequest` and `GenerateRequest` reports only whether a direct key is set.
|
||||
|
||||
Raw API keys do not belong in profile YAML, in-memory profiles, or app config.
|
||||
Avoid reflection-based debug dumps of request structs because exported fields
|
||||
remain visible to tools that bypass `String` and `GoString`.
|
||||
|
||||
## Errors
|
||||
|
||||
Public methods wrap context while preserving stable sentinel checks with `errors.Is`:
|
||||
Public methods wrap context while preserving stable sentinel checks with
|
||||
`errors.Is`:
|
||||
|
||||
- `ErrInvalidConfig`
|
||||
- `ErrInvalidRequest`
|
||||
@@ -158,8 +272,13 @@ if errors.Is(err, scriptorium.ErrPromptNotFound) {
|
||||
|
||||
## Examples
|
||||
|
||||
Run the prepare-only example from the repository root:
|
||||
Run the maintained prepare-only example from the repository root:
|
||||
|
||||
```bash
|
||||
go run ./examples/go-library/prepare
|
||||
```
|
||||
|
||||
See also:
|
||||
|
||||
- [Configuration reference](../config.md)
|
||||
- [Consumer integration overview](api.md)
|
||||
|
||||
@@ -1,217 +0,0 @@
|
||||
# HTTP API Integration
|
||||
|
||||
## Scope
|
||||
|
||||
This document defines the implemented inbound HTTP contract for Scriptorium.
|
||||
|
||||
Current scope is only:
|
||||
|
||||
- `POST /v1/runs`
|
||||
|
||||
For CLI behavior, see the [CLI reference](../cli.md).
|
||||
|
||||
## Endpoint
|
||||
|
||||
- Method: `POST`
|
||||
- Path: `/v1/runs`
|
||||
- Content type: JSON request/response
|
||||
|
||||
Route behavior:
|
||||
|
||||
- unknown path: `404 not_found`
|
||||
- unsupported method on `/v1/runs`: `405 method_not_allowed`
|
||||
|
||||
Copyable request example file:
|
||||
|
||||
- `examples/http-run.json`
|
||||
|
||||
## Request Body
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt_id": "generic.structured_events",
|
||||
"profile_id": "local-quality",
|
||||
"prompt_version": "1.0.0",
|
||||
"inputs": {
|
||||
"transcript": {"type": "file", "uri": "./examples/fixtures/transcript.md"},
|
||||
"glossary": {"type": "inline", "body": "party:\n - Rin"}
|
||||
},
|
||||
"vars": {
|
||||
"session_date": "2026-05-04"
|
||||
},
|
||||
"model": {
|
||||
"endpoint": "http://localhost:8000/v1",
|
||||
"model": "gpt-4o-mini",
|
||||
"temperature": 0.0,
|
||||
"max_tokens": 800,
|
||||
"top_p": 1.0,
|
||||
"timeout_seconds": 120,
|
||||
"service_tier": "priority",
|
||||
"reasoning_effort": "medium",
|
||||
"api_key_env": "SCRIPTORIUM_API_KEY",
|
||||
"extra_params": {
|
||||
"route": "primary",
|
||||
"provider_options": {
|
||||
"retry_budget": 2
|
||||
}
|
||||
}
|
||||
},
|
||||
"include_raw_output": false
|
||||
}
|
||||
```
|
||||
|
||||
Required fields:
|
||||
|
||||
- `prompt_id`
|
||||
- `inputs` (must contain at least one named input)
|
||||
|
||||
Input reference types currently supported by runtime artifact loading:
|
||||
|
||||
- `file`
|
||||
- `inline`
|
||||
|
||||
HTTP `file` references require `server.artifact_root` or `serve --artifact-root`.
|
||||
Relative file URIs resolve inside that root. Absolute file URIs are accepted
|
||||
only when they remain inside the root. Requests that escape the root, including
|
||||
`..` traversal and absolute paths outside the root, return
|
||||
`400 artifact_not_allowed`. `inline` references do not require an artifact root.
|
||||
|
||||
Model override notes:
|
||||
|
||||
- Numeric model override fields distinguish omitted values from explicit zero values. For example, omitting `temperature` preserves the selected profile/default value, while `"temperature": 0` explicitly sets the effective temperature to zero.
|
||||
- `extra_params` accepts JSON-compatible values: strings, numbers, booleans, objects, and arrays.
|
||||
- `extra_params` are passed through effective model metadata and flattened into top-level provider request fields by the OpenAI-compatible client.
|
||||
- `extra_params` keys must not be empty and must not collide with reserved outbound fields: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or `response_format`.
|
||||
- Raw API-key values are not accepted. Use `api_key_env` to name an environment variable.
|
||||
|
||||
## Strict JSON Rules
|
||||
|
||||
Request decoding uses strict JSON field checks:
|
||||
|
||||
- unknown request fields are rejected with `400 invalid_json`
|
||||
- unknown `model` fields are rejected with `400 invalid_json`
|
||||
- raw API-key payload fields such as `api_key` are rejected as unknown fields
|
||||
|
||||
## Success Response
|
||||
|
||||
Status: `200 OK`
|
||||
|
||||
Response shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"artifact": {
|
||||
"name": "output",
|
||||
"content_type": "application/json",
|
||||
"body": "{\"summary\":\"...\"}",
|
||||
"uri": "",
|
||||
"size": 123,
|
||||
"hash": "..."
|
||||
},
|
||||
"validation": {
|
||||
"status": "passed",
|
||||
"mode": "json_schema",
|
||||
"errors": [],
|
||||
"schema_path": "structured_events.schema.json",
|
||||
"repair_attempts": 0,
|
||||
"is_valid": true
|
||||
},
|
||||
"metadata": {
|
||||
"run_id": "...",
|
||||
"prompt_id": "generic.structured_events",
|
||||
"prompt_version": "1.0.0",
|
||||
"prompt_hash": "...",
|
||||
"rendered_prompt_hash": "...",
|
||||
"selected_profile_id": "local-quality",
|
||||
"model_name": "gpt-4o-mini",
|
||||
"endpoint": "http://localhost:8000/v1",
|
||||
"model_params": {
|
||||
"endpoint": "http://localhost:8000/v1",
|
||||
"model": "gpt-4o-mini",
|
||||
"temperature": 0,
|
||||
"max_tokens": 800,
|
||||
"top_p": 1,
|
||||
"timeout_seconds": 120,
|
||||
"service_tier": "priority",
|
||||
"reasoning_effort": "medium",
|
||||
"api_key_env": "SCRIPTORIUM_API_KEY",
|
||||
"extra_params": {
|
||||
"route": "primary",
|
||||
"provider_options": {
|
||||
"retry_budget": 2
|
||||
}
|
||||
}
|
||||
},
|
||||
"input_hashes": {
|
||||
"transcript": "..."
|
||||
},
|
||||
"usage": {
|
||||
"prompt_tokens": 11,
|
||||
"completion_tokens": 22,
|
||||
"total_tokens": 33,
|
||||
"cached_tokens": 0,
|
||||
"cache_write_tokens": 0
|
||||
},
|
||||
"start_time": "2026-05-04T12:00:00Z",
|
||||
"end_time": "2026-05-04T12:00:01Z",
|
||||
"duration_ms": 1000,
|
||||
"validation_mode": "json_schema",
|
||||
"validation_status": "passed",
|
||||
"repair_attempts_used": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`raw_model_output` is omitted by default.
|
||||
|
||||
`metadata.usage.cached_tokens` and `metadata.usage.cache_write_tokens` are always present as numbers. They are `0` when the provider omits compatible cache usage fields or reports no cache activity.
|
||||
|
||||
To include it, send:
|
||||
|
||||
- `"include_raw_output": true`
|
||||
|
||||
## Validation Failure Behavior
|
||||
|
||||
Validation content failures do not map to HTTP error status.
|
||||
|
||||
Behavior:
|
||||
|
||||
- status remains `200 OK`
|
||||
- `validation.status` is `failed`
|
||||
- validation errors are returned in `validation.errors`
|
||||
|
||||
## Error Responses
|
||||
|
||||
Error body shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"code": "invalid_request",
|
||||
"message": "prompt_id is required"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Current error mapping (non-exhaustive):
|
||||
|
||||
- `400 invalid_json`: malformed JSON or unknown JSON fields
|
||||
- `400 invalid_request`: missing/invalid request fields
|
||||
- `400 profile_required`: no explicit `profile_id` and prompt has no `default_profile`
|
||||
- `400 prompt_load_failed`: prompt definition invalid/unloadable
|
||||
- `400 profile_load_failed`: profile invalid/unloadable
|
||||
- `400 artifact_not_allowed`: file input artifact is outside the configured artifact root or file refs are not enabled
|
||||
- `400 artifact_read_failed`: input artifact loading failed
|
||||
- `400 prompt_render_failed`: template render failed
|
||||
- `400 api_key_env_missing`: named API-key environment variable is missing
|
||||
- `404 prompt_not_found`
|
||||
- `404 profile_not_found`
|
||||
- `502 llm_failed`: outbound model request failed
|
||||
- `500 validation_runtime_failed`: validator runtime/schema-load failure
|
||||
- `500 internal_error`
|
||||
|
||||
## Security And Deployment Note
|
||||
|
||||
The HTTP adapter has no built-in authentication or authorization.
|
||||
|
||||
Deploy behind trusted controls (for example authenticated gateway/reverse proxy and network boundaries).
|
||||
@@ -1,114 +0,0 @@
|
||||
# Narratio Subprocess Integration
|
||||
|
||||
## Purpose
|
||||
|
||||
This document defines the supported subprocess contract for Narratio invoking Scriptorium through the public CLI.
|
||||
|
||||
This is a CLI contract, not an internal Go package integration.
|
||||
|
||||
## Supported Commands
|
||||
|
||||
Narratio should invoke:
|
||||
|
||||
- `scriptorium run`
|
||||
- `scriptorium render`
|
||||
|
||||
Use `run` for generation.
|
||||
|
||||
Use `render` for preflight/debug output without LLM execution.
|
||||
|
||||
## Recommended Invocation Shapes
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
scriptorium run \
|
||||
--prompt <prompt_id> \
|
||||
--input transcript=<path> \
|
||||
--out <artifact_path>
|
||||
```
|
||||
|
||||
Render:
|
||||
|
||||
```bash
|
||||
scriptorium render \
|
||||
--prompt <prompt_id> \
|
||||
--input transcript=<path> \
|
||||
--format json
|
||||
```
|
||||
|
||||
Narratio may add:
|
||||
|
||||
- `--config <path>`
|
||||
- `--profile <profile_id>`
|
||||
- repeatable `--input name=path`
|
||||
- repeatable `--var name=value`
|
||||
- runtime overrides when explicitly needed (`--model`, `--llm-base-url`, `--timeout`, etc.)
|
||||
|
||||
## Config And Directory Behavior
|
||||
|
||||
Narratio can rely on resolved app config or pass explicit paths.
|
||||
|
||||
- default config search order:
|
||||
1. `/usr/local/etc/scriptorium/config.yml`
|
||||
2. `/etc/scriptorium/config.yml`
|
||||
- explicit `--config` requires file existence and valid syntax
|
||||
- CLI flags override config values
|
||||
|
||||
## Profile Selection
|
||||
|
||||
Profile selection follows runner behavior:
|
||||
|
||||
1. explicit `--profile`
|
||||
2. prompt `default_profile`
|
||||
3. error if neither is available
|
||||
|
||||
Narratio should treat prompt/profile IDs as deployment configuration, not hardcoded logic.
|
||||
|
||||
## Input And Variable Contract
|
||||
|
||||
- Inputs use repeated `--input name=path`.
|
||||
- Input names must match prompt definition input names.
|
||||
- Variables use repeated `--var name=value` for small metadata values.
|
||||
- Prefer file inputs for large content.
|
||||
|
||||
## Environment Contract
|
||||
|
||||
- Pass through required API-key environment variables referenced by `api_key_env`.
|
||||
- Never pass raw API keys via CLI arguments.
|
||||
- Keep subprocess environment scoped to required variables.
|
||||
|
||||
## Output And Error Handling
|
||||
|
||||
`run`:
|
||||
|
||||
- stdout: artifact body unless `--out` is used
|
||||
- `--out`: writes artifact to file
|
||||
- stderr: success summary and errors
|
||||
|
||||
`render`:
|
||||
|
||||
- stdout: prepared-run output unless `--out` is used
|
||||
- stderr: errors
|
||||
|
||||
Narratio should capture stdout and stderr separately.
|
||||
|
||||
## Exit Status Contract
|
||||
|
||||
- `0`: success
|
||||
- `1`: parse/config/load/render/generation/IO/runtime error
|
||||
- `2`: run completed but validation failed
|
||||
|
||||
A `run` exit code `2` can still produce output (stdout or `--out`).
|
||||
|
||||
## Security Notes
|
||||
|
||||
- Treat generated artifacts and stderr logs as potentially sensitive.
|
||||
- Avoid logging full rendered prompts by default in production contexts.
|
||||
- Use controlled output paths and access controls for persisted artifacts.
|
||||
|
||||
## Canonical References
|
||||
|
||||
- CLI behavior: [CLI reference](../cli.md)
|
||||
- Config behavior: [Configuration reference](../config.md)
|
||||
- Operations and failure handling: [Operations guide](../operations.md), [Troubleshooting](../troubleshooting.md)
|
||||
@@ -177,7 +177,7 @@ Malformed responses return `ErrMalformedResponse`.
|
||||
## Error Handling
|
||||
|
||||
- network/request-construction failures: `ErrRequestFailed`
|
||||
- non-2xx HTTP status: `ErrUnexpectedStatus` (includes status code and trimmed response body snippet)
|
||||
- non-2xx HTTP status: `ErrUnexpectedStatus` (includes status code; provider response bodies are not included)
|
||||
- malformed response shape/content: `ErrMalformedResponse`
|
||||
|
||||
## Unsupported Or Non-Serialized Fields
|
||||
|
||||
130
docs/integrations/subprocess.md
Normal file
130
docs/integrations/subprocess.md
Normal file
@@ -0,0 +1,130 @@
|
||||
# Subprocess Integration
|
||||
|
||||
This document defines the supported subprocess contract for downstream
|
||||
applications invoking Scriptorium through the public CLI.
|
||||
|
||||
This is a CLI contract. Go callers that want an in-process typed API should use
|
||||
the [package guide](../consumers/pkg-scriptorium.md).
|
||||
|
||||
## Supported Commands
|
||||
|
||||
Downstream applications should invoke:
|
||||
|
||||
- `scriptorium render` for preflight/debug output without LLM execution.
|
||||
- `scriptorium run` for generation.
|
||||
|
||||
`scriptorium serve` is an HTTP service command, not the recommended subprocess
|
||||
contract for per-request execution.
|
||||
|
||||
## Recommended Invocation Shapes
|
||||
|
||||
Render:
|
||||
|
||||
```bash
|
||||
scriptorium render \
|
||||
--config <config_path> \
|
||||
--prompt <prompt_id> \
|
||||
--input transcript=<path> \
|
||||
--format json
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
scriptorium run \
|
||||
--config <config_path> \
|
||||
--prompt <prompt_id> \
|
||||
--input transcript=<path> \
|
||||
--out <artifact_path>
|
||||
```
|
||||
|
||||
Callers may add:
|
||||
|
||||
- `--profile <profile_id>`
|
||||
- repeatable `--input name=path`
|
||||
- repeatable `--var name=value`
|
||||
- runtime overrides when explicitly needed, such as `--model`, `--llm-base-url`, `--api-key-env`, and `--timeout`
|
||||
|
||||
Do not pass raw API keys as command arguments.
|
||||
|
||||
## Config And Directory Behavior
|
||||
|
||||
Callers can rely on resolved app config or pass explicit paths.
|
||||
|
||||
Default config search order:
|
||||
|
||||
1. `/usr/local/etc/scriptorium/config.yml`
|
||||
2. `/etc/scriptorium/config.yml`
|
||||
|
||||
Rules:
|
||||
|
||||
- Explicit `--config` requires file existence and valid syntax.
|
||||
- CLI flags override config values.
|
||||
- `run` and `render` require an effective `prompt_dir`.
|
||||
- `profile_dir` is optional because built-in profiles are available.
|
||||
|
||||
## Profile Selection
|
||||
|
||||
Profile selection follows runner behavior:
|
||||
|
||||
1. explicit `--profile`
|
||||
2. prompt `default_profile`
|
||||
3. error if neither is available
|
||||
|
||||
Treat prompt and profile IDs as deployment configuration, not hardcoded business
|
||||
logic.
|
||||
|
||||
## Input And Variable Contract
|
||||
|
||||
- Inputs use repeated `--input name=path`.
|
||||
- Input names must match prompt definition input names.
|
||||
- Variables use repeated `--var name=value`.
|
||||
- Both flags also accept comma-separated mappings.
|
||||
- Prefer file inputs for large content.
|
||||
|
||||
CLI inputs are file references. HTTP-only `inline` references are documented in
|
||||
the [HTTP API reference](../api.md).
|
||||
|
||||
## Environment Contract
|
||||
|
||||
- Pass through required API-key environment variables referenced by `api_key_env`.
|
||||
- Keep subprocess environments scoped to required variables.
|
||||
- Use `--api-key-env` only to name an environment variable.
|
||||
- Never pass raw API keys via argv.
|
||||
|
||||
## Stdout And Stderr
|
||||
|
||||
`run`:
|
||||
|
||||
- stdout: generated artifact body unless `--out` is used.
|
||||
- stderr: success summary and errors.
|
||||
|
||||
`render`:
|
||||
|
||||
- stdout: prepared-run output unless `--out` is used.
|
||||
- stderr: errors.
|
||||
|
||||
Capture stdout and stderr separately. Do not parse stderr as a stable data
|
||||
format beyond exit status handling.
|
||||
|
||||
## Exit Status Contract
|
||||
|
||||
- `0`: success.
|
||||
- `1`: parse, config, load, render, generation, IO, or runtime error.
|
||||
- `2`: `run` completed and output was written, but validation failed.
|
||||
|
||||
A `run` exit code `2` can still produce output on stdout or at `--out`.
|
||||
Consumers must decide whether to keep or discard that output.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- Treat generated artifacts, rendered prompts, stdout, and stderr as potentially sensitive.
|
||||
- Use controlled output paths and access controls for persisted artifacts.
|
||||
- Avoid logging full rendered prompts or generated artifacts by default.
|
||||
|
||||
## Canonical References
|
||||
|
||||
- CLI behavior: [CLI reference](../cli.md)
|
||||
- Config and file formats: [Configuration reference](../config.md)
|
||||
- Operations: [Operations guide](../operations.md)
|
||||
- Troubleshooting: [Troubleshooting](../troubleshooting.md)
|
||||
@@ -1,156 +1,81 @@
|
||||
# Adapter And Repository Internals
|
||||
# Adapter Internals
|
||||
|
||||
## Purpose
|
||||
|
||||
This document describes implemented adapter/repository boundaries and their current behavior.
|
||||
Adapters translate external interfaces into domain requests and translate domain results back out. They wire dependencies, apply app config, and own IO concerns, but they do not make runner decisions.
|
||||
|
||||
Source-loading behavior belongs in `docs/internal/sources.md`. User-facing CLI, HTTP, and package contracts belong in `docs/cli.md`, `docs/api.md`, and `docs/consumers/pkg-scriptorium.md`.
|
||||
|
||||
## Adapter Map
|
||||
|
||||
- `internal/adapter/cli`: CLI command parsing, app wiring, stdout/stderr handling, exit codes.
|
||||
- `internal/adapter/http`: HTTP request/response mapping for `POST /v1/runs`.
|
||||
- root package `scriptorium`: public Go library facade for preparing and running prompt requests.
|
||||
- `internal/promptdef`: filesystem and `fs.FS` prompt-definition repositories.
|
||||
- `internal/profile`: filesystem, `fs.FS`, and overlay execution-profile repositories.
|
||||
- `internal/filecatalog`: shared YAML discovery and display-path helpers for prompt/profile repositories.
|
||||
- `internal/profile/builtin`: embedded built-in execution-profile repository.
|
||||
- `internal/artifact`: input artifact reader.
|
||||
- `internal/prompt`: Go-template renderer.
|
||||
- `internal/llm`: OpenAI-compatible LLM client implementation.
|
||||
- `internal/validate`: filesystem and `fs.FS` output validators.
|
||||
- `internal/format`: prepared-run formatters for `render` output.
|
||||
- `cmd/scriptorium`: process entrypoint.
|
||||
- `internal/adapter/cli`: command parsing, config handoff, runner construction, stdout/stderr, exit codes.
|
||||
- `internal/adapter/http`: `POST /v1/runs` request/response mapping and HTTP error/status mapping.
|
||||
- root package `scriptorium`: public Go facade over internal runner types and dependencies.
|
||||
|
||||
Supporting implementation packages used during adapter wiring:
|
||||
|
||||
- `internal/config`
|
||||
- `internal/defaults`
|
||||
- `internal/format`
|
||||
- `internal/llm`
|
||||
- `internal/prompt`
|
||||
|
||||
## Inputs And Outputs
|
||||
|
||||
CLI adapter:
|
||||
|
||||
- Input: process args, filesystem config/assets, environment.
|
||||
- Output: exit code, stdout artifact/prepared output, stderr summaries/errors.
|
||||
- `run` summaries include cache usage counters only when either parsed cache counter is non-zero.
|
||||
- Input: process args, optional config file, filesystem sources, environment variables.
|
||||
- Output: process exit code, stdout artifact/prepared output, stderr summaries and errors.
|
||||
|
||||
HTTP adapter:
|
||||
|
||||
- Input: JSON request body (`runRequestDTO`).
|
||||
- Output: JSON success/error body with mapped status codes.
|
||||
- Success metadata includes token usage plus cache usage counters.
|
||||
- Input: HTTP request method/path/headers/body for `POST /v1/runs`.
|
||||
- Output: JSON success or error body with mapped status code.
|
||||
|
||||
Public library facade:
|
||||
Public Go facade:
|
||||
|
||||
- Input: typed `scriptorium.RunRequest` values.
|
||||
- Input: typed `scriptorium.Config`, `Option`, and `RunRequest` values.
|
||||
- Output: typed `PreparedRun` and `RunResult` values plus public sentinel errors.
|
||||
- Custom LLM behavior is injected with `WithLLMClient`; otherwise the default OpenAI-compatible client is used.
|
||||
- `RunRequest.APIKey` is a request-scoped Go value only; it is converted into internal execution state for LLM generation and stripped from public result types.
|
||||
- Prompt, profile, and schema source options can use directories, single files, or `fs.FS` roots. Explicit source options override the matching `Config` directory field.
|
||||
- Public types are facade types converted at the package boundary; internal domain types remain internal.
|
||||
|
||||
Prompt/profile repositories:
|
||||
|
||||
- Input: prompt/profile YAML files under configured directories or `fs.FS` roots.
|
||||
- Output: normalized domain definitions/profiles or typed errors.
|
||||
- Shared YAML catalog helpers provide recursive discovery, extension filtering, deterministic ordering, file stems, and `fs.FS` display paths.
|
||||
- Single-file public sources are represented as `fs.FS` roots containing one YAML file; lookup still uses YAML `id` values.
|
||||
|
||||
Profile repository composition:
|
||||
|
||||
- Built-in profiles are embedded and loaded through the same profile validation rules as filesystem profiles.
|
||||
- When no custom profile directory is configured, the runner receives the built-in profile repository.
|
||||
- When a custom profile directory/file/`fs.FS` source is configured, the runner receives an overlay repository with custom profiles as primary and built-ins as fallback.
|
||||
- Overlay lookup falls back only after custom profile-not-found errors; custom load/validation/raw-key errors are returned directly.
|
||||
|
||||
Artifact reader:
|
||||
|
||||
- Input: `domain.ArtifactRef`.
|
||||
- Output: loaded `domain.Artifact`.
|
||||
|
||||
LLM adapter:
|
||||
|
||||
- Input: `domain.GenerateRequest`.
|
||||
- Output: `domain.GenerateResponse`.
|
||||
- Direct API-key values are preferred when present; otherwise `api_key_env` is resolved from the process environment.
|
||||
|
||||
Validator:
|
||||
|
||||
- Input: artifact body + output contract.
|
||||
- Output: validation result or runtime validation error.
|
||||
- Schema documents may be loaded from a directory, single file, or `fs.FS` root in the public package. CLI and HTTP continue to use directory-backed schema loading.
|
||||
|
||||
## Boundaries
|
||||
|
||||
- Adapters convert external representations to domain requests and back.
|
||||
- Use-case decisions remain in `internal/usecase`.
|
||||
- External dependency details stay scoped to adapter packages.
|
||||
- Adapters convert external shapes to `domain.RunRequest` and back.
|
||||
- Runner orchestration remains in `internal/usecase`.
|
||||
- Prompt/profile/schema/artifact source rules remain in repository, validator, and artifact packages.
|
||||
- LLM provider request serialization remains in `internal/llm`.
|
||||
- Public package types are facade types; internal domain types do not leak across the package boundary.
|
||||
|
||||
## Config Fields Used
|
||||
|
||||
Primary app settings consumed by adapters:
|
||||
Adapter app settings:
|
||||
|
||||
- `prompt_dir`
|
||||
- `profile_dir` (optional custom profile source)
|
||||
- `profile_dir`
|
||||
- `schema_dir`
|
||||
- `server.addr`
|
||||
- `server.artifact_root` (HTTP `serve` file input root)
|
||||
- `server.artifact_root`
|
||||
- `server.max_request_bytes`
|
||||
- `server.max_artifact_bytes`
|
||||
- `server.max_response_bytes`
|
||||
- `defaults.render_format`
|
||||
|
||||
Execution profile/request settings used through runner:
|
||||
Execution request/profile settings passed through the runner:
|
||||
|
||||
- `endpoint`, `model`, `temperature`, `max_tokens`, `top_p`, `timeout_seconds`, `service_tier`, `api_key_env`, `reasoning_effort`, `extra_params`
|
||||
- CLI and HTTP request adapters preserve caller intent for numeric runtime overrides. Omitted values remain absent; explicit zero values are mapped as explicit overrides.
|
||||
- HTTP `extra_params` accepts JSON-compatible values and maps them to domain request overrides without provider-specific adapter logic.
|
||||
- `endpoint`
|
||||
- `model`
|
||||
- `temperature`
|
||||
- `max_tokens`
|
||||
- `top_p`
|
||||
- `timeout_seconds`
|
||||
- `service_tier`
|
||||
- `api_key_env`
|
||||
- `reasoning_effort`
|
||||
- `extra_params`
|
||||
|
||||
## External Dependencies
|
||||
CLI and HTTP preserve numeric override presence so omitted values and explicit zero values remain distinct.
|
||||
|
||||
- YAML decoding: `gopkg.in/yaml.v3` (strict known-fields mode in config/prompt/profile loaders).
|
||||
- JSON Schema validation: `github.com/santhosh-tekuri/jsonschema/v6`.
|
||||
- HTTP client/server: Go standard library.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
Strict decoding and input checks:
|
||||
|
||||
- config/prompt/profile loaders reject unknown YAML fields.
|
||||
- prompt/profile repositories scan nested subdirectories recursively.
|
||||
- prompt/profile lookup uses YAML `id` values; subdirectory paths are organizational only.
|
||||
- prompt `content_file` paths resolve relative to the prompt YAML file within the same source.
|
||||
- duplicate prompt/profile IDs are invalid and fail instead of using first-match behavior.
|
||||
- duplicate profile IDs across custom and built-in sources are allowed; the custom source overrides the built-in profile.
|
||||
- HTTP DTO decoder rejects unknown JSON fields.
|
||||
- raw API key payload fields are rejected by strict decoding in profile/http paths.
|
||||
|
||||
Artifact refs:
|
||||
|
||||
- Supported reference types: `inline`, `file`.
|
||||
- Unsupported types return `ErrUnsupportedRefType`.
|
||||
- CLI `run` and `render` use direct filesystem file reads for `file` references.
|
||||
- HTTP `serve` uses a restricted artifact reader: `inline` references work without a root, while `file` references require `server.artifact_root` or `--artifact-root` and must stay inside that root.
|
||||
- HTTP file paths are resolved with clean absolute paths and containment checks, not string-prefix checks.
|
||||
- Symlinks inside the root are followed by the operating system; the configured root must not be writable by untrusted users.
|
||||
|
||||
LLM adapter:
|
||||
|
||||
- endpoint appends `/chat/completions`.
|
||||
- rendered messages without cache control serialize with string `content`.
|
||||
- rendered messages with cache control serialize as one text content block with `cache_control`.
|
||||
- non-empty `reasoning_effort` serializes as a top-level provider request field.
|
||||
- `extra_params` flatten into provider-specific top-level JSON request fields.
|
||||
- reserved `extra_params` keys are rejected before the provider call: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, and `response_format`.
|
||||
- empty `extra_params` keys and values that cannot be JSON-encoded are rejected before the provider call.
|
||||
- compatible cache usage response fields are parsed into domain token usage.
|
||||
- non-2xx responses map to request failure errors.
|
||||
- malformed responses (including missing/empty first choice content) are errors.
|
||||
- direct API-key values are never serialized in provider request bodies.
|
||||
|
||||
Validator:
|
||||
|
||||
- `basic`, `json`, `json_schema` content failures return `ValidationFailed` results.
|
||||
- schema load/compile/path failures are runtime errors.
|
||||
- schema lookup uses explicit `schema_path` values relative to `schema_dir`; it does not recursively search by basename.
|
||||
|
||||
HTTP error mapping:
|
||||
|
||||
- maps domain/use-case errors to stable HTTP code + error code/message.
|
||||
- distinguishes missing profile selection and missing `api_key_env` variable using stable use-case sentinel errors.
|
||||
- avoids returning internal wrapped-cause details in response payload.
|
||||
|
||||
## CLI Adapter Semantics
|
||||
## CLI Adapter
|
||||
|
||||
Implemented commands:
|
||||
|
||||
@@ -158,30 +83,73 @@ Implemented commands:
|
||||
- `render`
|
||||
- `serve`
|
||||
|
||||
Behavior highlights:
|
||||
Behavior:
|
||||
|
||||
- `run` exit `2` indicates validation failed after generation.
|
||||
- `render` does not call the LLM.
|
||||
- `serve` exposes HTTP handler only; no built-in auth.
|
||||
- `render` supports `--format text|json`; `render` does not expose `--schema-dir`.
|
||||
- deprecated aliases `--prompt-id` and `--profile-id` are still accepted.
|
||||
- `run` constructs a runner with direct filesystem artifact reading and calls `Runner.Run`.
|
||||
- `render` constructs a runner and calls `Runner.Prepare`; it does not call the LLM.
|
||||
- `serve` constructs a restricted artifact reader and HTTP handler, then starts an unauthenticated HTTP server.
|
||||
- `run` exits `2` when generation succeeds but validation fails.
|
||||
- parse, runtime, and output-write errors exit `1`.
|
||||
- deprecated `--prompt-id` and `--profile-id` aliases are accepted.
|
||||
|
||||
## Tests To Inspect Before Changing
|
||||
## HTTP Adapter
|
||||
|
||||
Behavior:
|
||||
|
||||
- Accepts only `POST /v1/runs`.
|
||||
- Decodes JSON strictly and rejects unknown fields and trailing JSON tokens.
|
||||
- Rejects empty `prompt_id` and empty `inputs` before calling the runner.
|
||||
- Does not accept raw API key values in the request body.
|
||||
- Returns validation failures as `200` responses with failed validation details.
|
||||
- Maps request-body, artifact, and encoded-response size failures to `413`.
|
||||
- Maps domain and repository errors to stable error codes without returning wrapped internal cause text.
|
||||
|
||||
The HTTP adapter has no built-in authentication or authorization. Deployment controls must be provided outside the process.
|
||||
|
||||
## Public Go Facade
|
||||
|
||||
Behavior:
|
||||
|
||||
- `NewEngine` wires the same default runner components as CLI/HTTP unless options override them.
|
||||
- Prompt, profile, and schema sources may come from directories, single files, or `fs.FS` roots.
|
||||
- `WithProfiles` adds in-memory profiles ahead of file-backed and built-in profiles.
|
||||
- `WithLLMClient` injects custom model behavior.
|
||||
- `RunRequest.APIKey` is request-scoped and direct; it is used only for generation and is stripped from public results.
|
||||
- internal errors are mapped to public sentinels in `errors.go`.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
Adapters should:
|
||||
|
||||
- keep external error payloads concise and stable.
|
||||
- avoid leaking raw secret values.
|
||||
- use sentinels and typed errors for mapping.
|
||||
- preserve strict external input decoding.
|
||||
- keep validation content failures distinct from runtime errors.
|
||||
|
||||
CLI writes human-readable summaries to stderr. HTTP writes JSON error envelopes. The public Go facade returns typed errors.
|
||||
|
||||
## State And Manifests
|
||||
|
||||
Adapters do not add durable run state.
|
||||
|
||||
- No adapter writes run manifests.
|
||||
- No adapter implements checkpoint, skip, or resume behavior.
|
||||
- CLI output files are caller-selected artifacts, not internal state.
|
||||
|
||||
## Tests To Inspect
|
||||
|
||||
- `internal/adapter/cli/run_test.go`
|
||||
- `internal/adapter/http/handler_test.go`
|
||||
- `internal/promptdef/repository_test.go`
|
||||
- `internal/profile/repository_test.go`
|
||||
- `internal/artifact/reader_test.go`
|
||||
- `internal/prompt/renderer_test.go`
|
||||
- `internal/llm/openai_compatible_client_test.go`
|
||||
- `internal/validate/standard_validator_test.go`
|
||||
- `engine_test.go`
|
||||
- `internal/format/prepared_run_test.go`
|
||||
- `internal/llm/openai_compatible_client_test.go`
|
||||
|
||||
## Architectural Invariants
|
||||
|
||||
- Adapter packages do not own runner decision logic.
|
||||
- External request/response strictness is part of contract stability.
|
||||
- Prepared-render output never includes resolved API key values.
|
||||
- Outbound OpenAI-compatible request includes currently serialized first-class fields (`model`, optional `session_id`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `service_tier`, optional `reasoning_effort`, optional `response_format`) plus validated `extra_params` flattened as provider-specific top-level fields.
|
||||
- Outbound cache control is message-level only; no top-level cache-control field is serialized.
|
||||
- Adapter packages stay thin and translation-focused.
|
||||
- App config is resolved before dependency construction.
|
||||
- External input strictness is part of contract stability.
|
||||
- CLI and HTTP construct runners without a repairer.
|
||||
- HTTP endpoint details remain canonical in `docs/api.md`.
|
||||
- Public Go package details remain canonical in `docs/consumers/pkg-scriptorium.md`.
|
||||
|
||||
@@ -2,29 +2,32 @@
|
||||
|
||||
## Purpose
|
||||
|
||||
`internal/usecase.Runner` is the core use case orchestrator for prompt preparation and execution.
|
||||
`internal/usecase.Runner` is the core prompt-execution orchestrator. It prepares prompt requests, calls the configured LLM client for `Run`, validates generated output, and returns domain results.
|
||||
|
||||
It owns request validation, prompt/profile resolution, runtime-parameter merge, artifact loading, prompt rendering, structured-output setup, LLM invocation, output validation, and result metadata.
|
||||
Transport parsing, DTOs, CLI output, HTTP status mapping, and public package type conversion belong outside the runner.
|
||||
|
||||
## Inputs And Outputs
|
||||
|
||||
Primary input type:
|
||||
Primary inputs:
|
||||
|
||||
- `domain.RunRequest`
|
||||
- repositories/readers/renderers/validators injected at construction
|
||||
- `context.Context` for cancellation
|
||||
|
||||
Primary output types:
|
||||
Primary outputs:
|
||||
|
||||
- `domain.PreparedRun` from `Prepare`
|
||||
- `domain.RunResult` from `Run`
|
||||
- wrapped sentinel errors for adapter mapping
|
||||
|
||||
LLM boundary types:
|
||||
|
||||
- `domain.GenerateRequest`
|
||||
- `domain.GenerateResponse`
|
||||
|
||||
## Boundaries
|
||||
## Dependencies
|
||||
|
||||
`Runner` coordinates the following interfaces:
|
||||
`Runner` depends on package interfaces instead of concrete adapter types:
|
||||
|
||||
- `promptdef.Repository`
|
||||
- `profile.Repository`
|
||||
@@ -34,136 +37,110 @@ LLM boundary types:
|
||||
- `validate.Validator`
|
||||
- optional `usecase.OutputRepairer`
|
||||
|
||||
Transport concerns (CLI flags, HTTP DTO parsing, status-code mapping) stay outside runner.
|
||||
The CLI, HTTP adapter, and public Go package construct these dependencies and pass them in.
|
||||
|
||||
## Config Fields Used
|
||||
## Config Fields
|
||||
|
||||
`Runner` does not read app config files directly.
|
||||
`Runner` does not read app config files. Effective behavior is determined by injected dependencies and the `domain.RunRequest`.
|
||||
|
||||
It receives fully constructed repositories/readers/validators from adapters. Effective behavior depends on adapter wiring, including:
|
||||
Adapter wiring commonly reflects these app config fields:
|
||||
|
||||
- prompt/profile directories
|
||||
- schema base directory
|
||||
- selected profile/runtime overrides in request
|
||||
- `prompt_dir`
|
||||
- `profile_dir`
|
||||
- `schema_dir`
|
||||
- `server.artifact_root`
|
||||
- HTTP request/artifact/response size limits
|
||||
|
||||
## External Adapters Used
|
||||
|
||||
`Runner` works with adapter implementations via interfaces. Current wiring from CLI/HTTP uses:
|
||||
|
||||
- filesystem prompt/profile repositories
|
||||
- composite artifact reader
|
||||
- Go-template prompt renderer
|
||||
- OpenAI-compatible LLM client
|
||||
- standard validator
|
||||
|
||||
## State And Resume Behavior
|
||||
|
||||
`Runner` is stateless across requests.
|
||||
|
||||
- No durable run-state storage.
|
||||
- No built-in resume/skip checkpoints.
|
||||
- Each `Run`/`Prepare` executes from request inputs and current repositories.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
Primary runner error classes:
|
||||
|
||||
- `ErrInvalidRequest`: invalid run request envelope.
|
||||
- `ErrProfileRequired`: specific invalid-request reason when neither request `profile_id` nor prompt `default_profile` is available.
|
||||
- `ErrAPIKeyEnvMissing`: specific invalid-request reason when `api_key_env` is set but the named environment variable is unset/empty.
|
||||
- `ErrPromptLoad`: prompt-definition repository load failures.
|
||||
- `ErrProfileLoad`: execution-profile repository load failures.
|
||||
- `ErrArtifactLoad`: artifact read failures.
|
||||
- `ErrPromptRender`: template render failures.
|
||||
- `ErrLLMGenerate`: outbound model request failures.
|
||||
- `ErrValidation`: validation runtime failures (including structured-output schema load/compile failures).
|
||||
|
||||
Reason sentinel behavior:
|
||||
|
||||
- `ErrProfileRequired` and `ErrAPIKeyEnvMissing` are wrapped with `ErrInvalidRequest`.
|
||||
- Adapters can use `errors.Is` for stable reason mapping without matching runner prose.
|
||||
|
||||
Validation content failures are not run errors:
|
||||
|
||||
- `Run` can succeed with `Validation.Status == failed`.
|
||||
- CLI maps this to exit code `2`.
|
||||
- HTTP returns `200` with failed validation details.
|
||||
Runtime model settings are resolved from the selected profile plus request overrides.
|
||||
|
||||
## Prepare Flow
|
||||
|
||||
`Prepare` performs:
|
||||
`Prepare`:
|
||||
|
||||
1. validate request basics (prompt ID present).
|
||||
2. load prompt definition by ID/version.
|
||||
3. select profile ID:
|
||||
- explicit request profile ID
|
||||
- prompt `default_profile`
|
||||
- otherwise return an invalid request with `ErrProfileRequired`
|
||||
4. load execution profile.
|
||||
5. merge effective runtime target:
|
||||
- built-in execution defaults
|
||||
- selected profile values
|
||||
- request overrides
|
||||
- request numeric overrides are presence-aware, so omitted values preserve the current effective value and explicit zero values override it
|
||||
6. verify credentials when the effective target names `api_key_env`:
|
||||
- a request-scoped direct API key satisfies the credential requirement
|
||||
- otherwise a missing/empty env value returns an invalid request with `ErrAPIKeyEnvMissing`
|
||||
- only the environment-variable name is returned in public output; secret values are never returned
|
||||
7. resolve output contract and structured-output schema payload when `json_schema` mode is active.
|
||||
8. read input artifacts.
|
||||
9. render prompt messages, including any normalized message cache-control metadata.
|
||||
10. compute prompt/input/render hashes and return `PreparedRun`.
|
||||
1. requires a non-empty prompt ID.
|
||||
2. loads the prompt definition and computes its hash.
|
||||
3. selects the profile from request `profile_id`, then prompt `default_profile`.
|
||||
4. loads the selected execution profile.
|
||||
5. merges built-in execution defaults, profile values, and request overrides.
|
||||
6. applies request-scoped direct API key values for public Go callers.
|
||||
7. validates endpoint, model, and credential requirements.
|
||||
8. resolves the output contract and JSON Schema document when required.
|
||||
9. reads input artifacts.
|
||||
10. renders prompt messages and hashes the rendered prompt.
|
||||
11. returns a prepared run without calling the LLM.
|
||||
|
||||
`rendered_prompt_hash` includes cache-control metadata when present because it affects the outbound provider request. Prompts without cache control keep the role/content hash behavior.
|
||||
|
||||
`Prepare` does not call the LLM.
|
||||
|
||||
Runtime target notes:
|
||||
|
||||
- Profile `extra_params` and request `extra_params` carry JSON-compatible values through prepared output, run metadata, and `domain.GenerateRequest.Target`.
|
||||
- The OpenAI-compatible client serializes non-empty `reasoning_effort` as a top-level provider request field.
|
||||
- The OpenAI-compatible client flattens `extra_params` into provider-specific top-level JSON request fields.
|
||||
- Empty `extra_params` keys, reserved outbound field names, and values that cannot be JSON-encoded fail before the provider request.
|
||||
- Resolved API-key values are never serialized in prepared/run output, public results, logs, or HTTP responses.
|
||||
- Public direct API-key values are carried only far enough to call the configured LLM client and are excluded from JSON/YAML serialization.
|
||||
Numeric request overrides are presence-aware: omitted values preserve the current effective value, while explicit zero values are real overrides.
|
||||
|
||||
## Run Flow
|
||||
|
||||
`Run` performs:
|
||||
`Run`:
|
||||
|
||||
1. generate run ID.
|
||||
2. call `Prepare`.
|
||||
3. call LLM with prepared messages/effective target/structured-output spec.
|
||||
4. build output artifact content type from output format.
|
||||
5. validate output.
|
||||
6. optionally attempt bounded repair when repairer is injected and contract allows it.
|
||||
7. return `RunResult` with artifact, raw output, validation, hashes, profile/model metadata, token/cache usage, and timestamps.
|
||||
1. creates a run ID and start timestamp.
|
||||
2. calls `Prepare`.
|
||||
3. calls the injected LLM client with rendered messages, effective target, target presence, and structured-output settings.
|
||||
4. builds the output artifact.
|
||||
5. validates the output.
|
||||
6. optionally attempts bounded repair when a repairer is injected and the contract permits repair.
|
||||
7. returns the run result with artifact, raw output, validation, hashes, selected profile/model metadata, usage, and timing.
|
||||
|
||||
## Repair Hook Boundary
|
||||
`Run` must reuse `Prepare`; prepare logic should not be duplicated elsewhere.
|
||||
|
||||
Repair attempts occur only when all are true:
|
||||
## Validation And Repair
|
||||
|
||||
- repairer is injected
|
||||
- `repair_attempts > 0`
|
||||
Validation content failures are returned as successful run results with `Validation.Status == failed`. They are not runtime errors.
|
||||
|
||||
Validation runtime failures, such as schema load or compile errors, return `ErrValidation`.
|
||||
|
||||
Repair attempts occur only when all conditions are true:
|
||||
|
||||
- a repairer is injected
|
||||
- `repair_attempts` is greater than zero
|
||||
- validation status is `failed`
|
||||
- validation mode is `json` or `json_schema`
|
||||
|
||||
Current production wiring boundary:
|
||||
CLI and HTTP wiring call `usecase.NewRunner(...)`, which does not inject a repairer. Normal CLI and HTTP execution therefore does not repair invalid output.
|
||||
|
||||
- CLI and HTTP adapters call `usecase.NewRunner(...)` (no repairer argument).
|
||||
- Therefore normal CLI/HTTP execution does not perform repair attempts today.
|
||||
## Failure Behavior
|
||||
|
||||
## Tests To Inspect Before Changing
|
||||
Stable runner sentinels include:
|
||||
|
||||
- `ErrInvalidRequest`
|
||||
- `ErrProfileRequired`
|
||||
- `ErrAPIKeyEnvMissing`
|
||||
- `ErrAPIKeyRequired`
|
||||
- `ErrPromptLoad`
|
||||
- `ErrProfileLoad`
|
||||
- `ErrArtifactLoad`
|
||||
- `ErrPromptRender`
|
||||
- `ErrLLMGenerate`
|
||||
- `ErrValidation`
|
||||
|
||||
Adapters should use `errors.Is` against sentinels and lower-level repository errors instead of matching message text.
|
||||
|
||||
Secret values must not appear in prepared output, run results, logs, HTTP responses, or serialized public package results. The effective API-key environment-variable name may appear.
|
||||
|
||||
## State And Manifests
|
||||
|
||||
The runner is stateless across requests.
|
||||
|
||||
- No durable run store.
|
||||
- No manifest files.
|
||||
- No checkpoint, skip, or resume behavior.
|
||||
- Recovery is a new request after correcting inputs, config, or environment.
|
||||
|
||||
## Tests To Inspect
|
||||
|
||||
- `internal/usecase/runner_test.go`
|
||||
- `internal/usecase/integration_test.go`
|
||||
- `engine_test.go`
|
||||
- `internal/adapter/cli/run_test.go`
|
||||
- `internal/adapter/http/handler_test.go`
|
||||
|
||||
## Architectural Invariants
|
||||
|
||||
- `Run` reuses `Prepare`; prepare logic is not duplicated.
|
||||
- Effective API-key environment-variable name may appear; resolved secret value must not.
|
||||
- Structured-output schema document must load before LLM call for `json_schema` mode.
|
||||
- Use-case decisions stay in `internal/usecase`.
|
||||
- `Run` reuses `Prepare`.
|
||||
- Prompt/profile/artifact/schema loading remains behind injected boundaries.
|
||||
- Validation content failures are result state; validation runtime failures are errors.
|
||||
- Repair loops are bounded by `repair_attempts` and repairer presence.
|
||||
- Runner stays transport-agnostic.
|
||||
- Resolved secret values are never serialized or emitted.
|
||||
|
||||
157
docs/internal/sources.md
Normal file
157
docs/internal/sources.md
Normal file
@@ -0,0 +1,157 @@
|
||||
# Source Internals
|
||||
|
||||
## Purpose
|
||||
|
||||
This document covers implemented prompt, profile, schema, artifact, and catalog source behavior. It is for developers changing loaders or source wiring.
|
||||
|
||||
Full user-facing YAML and config reference material belongs in `docs/config.md`.
|
||||
|
||||
## Prompt Definition Sources
|
||||
|
||||
`internal/promptdef` provides directory-backed and `fs.FS` repositories.
|
||||
|
||||
Behavior:
|
||||
|
||||
- recursively scans `.yaml` and `.yml` files.
|
||||
- decodes YAML with known-fields checking.
|
||||
- looks up prompts by YAML `id`, not by path.
|
||||
- optionally filters by prompt `version`.
|
||||
- rejects duplicate matching prompt IDs.
|
||||
- requires `id`, `version`, and at least one message.
|
||||
- requires each message to set exactly one of `content` or `content_file`.
|
||||
- resolves filesystem `content_file` values relative to the prompt YAML file.
|
||||
- resolves `fs.FS` `content_file` values inside the configured source root.
|
||||
- permits prompt subdirectories only as organization; they are not part of prompt identity.
|
||||
|
||||
For `fs.FS` roots, absolute paths and relative traversal outside the source root are rejected by catalog path helpers.
|
||||
|
||||
## Profile Sources
|
||||
|
||||
`internal/profile` provides directory-backed, `fs.FS`, and overlay repositories. `internal/profile/builtin` embeds built-in profile YAML assets and exposes them through the same repository interface.
|
||||
|
||||
Behavior:
|
||||
|
||||
- recursively scans `.yaml` and `.yml` files.
|
||||
- decodes YAML with known-fields checking.
|
||||
- looks up profiles by YAML `id`, not by path.
|
||||
- rejects duplicate IDs inside the same source.
|
||||
- rejects raw `api_key` fields in YAML; file-backed profiles must use `api_key_env`.
|
||||
- validates required `endpoint` and `model` values.
|
||||
- validates numeric profile ranges.
|
||||
|
||||
Overlay behavior:
|
||||
|
||||
- custom profiles are primary.
|
||||
- built-in profiles are fallback.
|
||||
- fallback occurs only after a primary `ErrProfileNotFound`.
|
||||
- primary validation, YAML, duplicate, and raw-key errors are returned directly.
|
||||
- duplicate IDs across custom and built-in sources are allowed because the custom profile overrides the built-in one.
|
||||
|
||||
The public Go facade can add in-memory profiles ahead of file-backed and built-in profiles.
|
||||
|
||||
## Schema Sources
|
||||
|
||||
`internal/validate` provides:
|
||||
|
||||
- `StandardValidator` for filesystem paths.
|
||||
- `FSValidator` for `fs.FS` roots and single-file public schema sources.
|
||||
|
||||
Behavior:
|
||||
|
||||
- `json_schema` validation requires a non-empty `schema_path`.
|
||||
- filesystem schema paths resolve relative to `schema_dir` unless absolute.
|
||||
- directory-backed schema lookup uses the explicit `schema_path`; it does not search recursively by basename.
|
||||
- `fs.FS` schema paths must remain inside the configured source root.
|
||||
- single-file schema sources match by the configured file base name.
|
||||
- schema documents are loaded before the LLM call for structured output.
|
||||
- JSON parse failures are validation content failures.
|
||||
- schema access, decode, registration, and compile failures are runtime validation errors.
|
||||
|
||||
## Artifact Sources
|
||||
|
||||
`internal/artifact` supports two input artifact reference types:
|
||||
|
||||
- `inline`
|
||||
- `file`
|
||||
|
||||
Inline behavior:
|
||||
|
||||
- requires a non-empty body.
|
||||
- produces text/plain artifacts.
|
||||
- hashes the body bytes.
|
||||
|
||||
Direct file behavior:
|
||||
|
||||
- used by CLI `run`, CLI `render`, and the public Go facade.
|
||||
- requires a non-empty URI.
|
||||
- reads from the process filesystem without HTTP artifact-root restrictions.
|
||||
- infers content type from file extension, defaulting to text/plain.
|
||||
|
||||
Restricted file behavior:
|
||||
|
||||
- used by HTTP `serve`.
|
||||
- allows inline artifacts even when no artifact root is configured.
|
||||
- denies file artifacts when no artifact root is configured.
|
||||
- resolves relative file URIs against `server.artifact_root`.
|
||||
- accepts absolute file URIs only when they pass containment checks.
|
||||
- applies `server.max_artifact_bytes` when configured.
|
||||
|
||||
Restricted containment is lexical. It cleans paths and checks the relative path against the configured root; it does not resolve symlinks. Symlinks inside the root are followed by the operating system, including symlinks that target files outside the root.
|
||||
|
||||
## Catalog Helpers
|
||||
|
||||
`internal/filecatalog` centralizes shared source helpers:
|
||||
|
||||
- recursive YAML discovery for filesystem and `fs.FS` roots.
|
||||
- deterministic sorting.
|
||||
- `.yaml` and `.yml` filtering.
|
||||
- display paths for diagnostics.
|
||||
- YAML file stems.
|
||||
- `fs.FS` root cleaning and containment checks.
|
||||
|
||||
Repository code should use these helpers instead of reimplementing path traversal and containment rules.
|
||||
|
||||
## Failure Behavior
|
||||
|
||||
Common source failures:
|
||||
|
||||
- missing prompt/profile/schema/artifact files.
|
||||
- invalid YAML or JSON.
|
||||
- unknown YAML fields.
|
||||
- duplicate prompt or profile IDs.
|
||||
- prompt/profile validation errors.
|
||||
- raw API key fields in profile YAML.
|
||||
- unsupported artifact reference type.
|
||||
- missing inline body or file URI.
|
||||
- artifact outside HTTP root.
|
||||
- artifact exceeding HTTP size limit.
|
||||
- schema load or compile failure.
|
||||
|
||||
Prompt/profile repository lookup errors are mapped by adapters separately from runtime runner errors. Validation content failures remain result state; source and schema runtime failures return errors.
|
||||
|
||||
## State And Manifests
|
||||
|
||||
Source packages do not persist run state.
|
||||
|
||||
- No manifests are read or written.
|
||||
- No source package implements skip or resume behavior.
|
||||
- Source reads reflect the current filesystem or `fs.FS` state for each request.
|
||||
|
||||
## Tests To Inspect
|
||||
|
||||
- `internal/promptdef/repository_test.go`
|
||||
- `internal/profile/repository_test.go`
|
||||
- `internal/profile/builtin/repository_test.go`
|
||||
- `internal/artifact/reader_test.go`
|
||||
- `internal/validate/standard_validator_test.go`
|
||||
- `internal/usecase/integration_test.go`
|
||||
- `engine_test.go`
|
||||
|
||||
## Architectural Invariants
|
||||
|
||||
- Prompt/profile identity comes from YAML `id`.
|
||||
- External YAML decoding remains strict.
|
||||
- File-backed profile YAML never accepts raw API key values.
|
||||
- Built-in profiles are fallback, not a replacement for custom source validation.
|
||||
- HTTP file artifacts remain rooted by lexical containment.
|
||||
- Schema runtime failures remain errors, while JSON/schema content mismatches remain validation results.
|
||||
@@ -2,128 +2,163 @@
|
||||
|
||||
## Scope
|
||||
|
||||
This document covers day-to-day operation of the CLI and HTTP service for currently implemented behavior.
|
||||
|
||||
For command syntax, see [CLI reference](cli.md). For file formats and defaults, see [Configuration reference](config.md).
|
||||
This guide covers operating the implemented CLI commands and HTTP service. It
|
||||
does not replace the [CLI reference](cli.md), [Configuration reference](config.md),
|
||||
or [HTTP API reference](api.md).
|
||||
|
||||
## Operational Model
|
||||
|
||||
Scriptorium executes one request at a time per CLI invocation or HTTP request.
|
||||
Scriptorium executes one prompt request per CLI invocation or HTTP request.
|
||||
|
||||
Important boundaries:
|
||||
|
||||
- No durable run state is stored.
|
||||
- No built-in resume, checkpoint, archive, or backup workflow exists.
|
||||
- Recovery is rerun-based: fix inputs/config, then rerun.
|
||||
- No manifest, archive, checkpoint, or built-in backup workflow is written.
|
||||
- No built-in resume behavior exists.
|
||||
- Recovery is rerun-based: correct inputs, config, or environment, then run again.
|
||||
|
||||
## Filesystem Layout And Config
|
||||
## Filesystem Layout
|
||||
|
||||
Scriptorium depends on:
|
||||
Operational deployments usually provide:
|
||||
|
||||
- prompt definition files (`prompt_dir`)
|
||||
- execution profile files (`profile_dir`)
|
||||
- optional JSON schemas (`schema_dir`)
|
||||
- `prompt_dir`: prompt definition YAML files and adjacent `content_file` templates.
|
||||
- `profile_dir`: optional custom profile YAML files.
|
||||
- `schema_dir`: optional JSON Schema files.
|
||||
- `server.artifact_root`: optional HTTP file-input root for `serve`.
|
||||
|
||||
Config discovery order when `--config` is omitted:
|
||||
|
||||
1. `/usr/local/etc/scriptorium/config.yml`
|
||||
2. `/etc/scriptorium/config.yml`
|
||||
|
||||
If neither exists, built-in defaults are used. If `--config <path>` is provided, that file must exist and parse successfully.
|
||||
|
||||
Built-in defaults relevant to operations:
|
||||
|
||||
- `schema_dir: .`
|
||||
- `server.addr: :8080`
|
||||
- `server.artifact_root`: unset; HTTP `file` input references are rejected until configured
|
||||
- `defaults.render_format: text`
|
||||
Keep these directories readable by the Scriptorium process. Keep
|
||||
`server.artifact_root` narrow and not writable by untrusted users.
|
||||
|
||||
## Normal CLI Workflow
|
||||
|
||||
Use `render` first when you need to verify prompt resolution and runtime settings without calling a model.
|
||||
Use `render` before `run` when changing prompt/profile/input wiring:
|
||||
|
||||
Use `run` for generation.
|
||||
```bash
|
||||
go run ./cmd/scriptorium render \
|
||||
--config ./examples/config.yml \
|
||||
--prompt generic.markdown_summary \
|
||||
--input transcript=./examples/fixtures/transcript.md \
|
||||
--input glossary=./examples/fixtures/glossary.yml \
|
||||
--format json
|
||||
```
|
||||
|
||||
Typical sequence:
|
||||
Use `run` for generation after preflight:
|
||||
|
||||
1. Confirm prompt/profile directories resolve through config or flags.
|
||||
2. Confirm required input files exist and map to prompt input names.
|
||||
3. Confirm required API-key environment variables are set.
|
||||
4. Confirm the selected profile's model endpoint is reachable from the process environment.
|
||||
5. Run `render` for preflight when changing prompt/profile/input wiring.
|
||||
6. Run `run` for actual generation.
|
||||
```bash
|
||||
go run ./cmd/scriptorium run \
|
||||
--config ./examples/config.yml \
|
||||
--prompt generic.markdown_summary \
|
||||
--input transcript=./examples/fixtures/transcript.md \
|
||||
--input glossary=./examples/fixtures/glossary.yml \
|
||||
--out ./summary.md
|
||||
```
|
||||
|
||||
## Secrets Handling
|
||||
Before production runs, confirm:
|
||||
|
||||
Raw API keys are not accepted in config files, profile files as `api_key`, CLI flags, or HTTP request bodies.
|
||||
|
||||
Operational pattern:
|
||||
|
||||
- Set environment variables that hold secret values.
|
||||
- Set profile `api_key_env` (or runtime override `api_key_env`) to the environment variable name.
|
||||
- Keep process environments scoped to only required variables.
|
||||
- the effective config path is the intended one;
|
||||
- prompt/profile/schema directories are readable;
|
||||
- input file paths exist and match prompt input names;
|
||||
- required API-key environment variables are set;
|
||||
- the selected model endpoint is reachable from the process environment.
|
||||
|
||||
## HTTP Service Operation
|
||||
|
||||
Start service with:
|
||||
Start the service with:
|
||||
|
||||
```bash
|
||||
go run ./cmd/scriptorium serve --config ./examples/config.yml
|
||||
```
|
||||
|
||||
Current inbound API behavior:
|
||||
The implemented HTTP route is `POST /v1/runs`; request and response fields are
|
||||
defined in the [HTTP API reference](api.md).
|
||||
|
||||
- Route: `POST /v1/runs`
|
||||
- JSON request parsing rejects unknown fields.
|
||||
- Validation content failures still return `200 OK` with `validation.status: "failed"`.
|
||||
- `inline` input references work without filesystem configuration.
|
||||
- `file` input references require `server.artifact_root` or `serve --artifact-root`; relative paths resolve inside that root and paths outside it are rejected.
|
||||
The maintained HTTP request-shape example is `examples/http-run.json`.
|
||||
|
||||
Security caveat:
|
||||
HTTP service notes:
|
||||
|
||||
- Unknown JSON fields are rejected.
|
||||
- `inline` input references work without an artifact root.
|
||||
- `file` input references require `server.artifact_root` or `serve --artifact-root`.
|
||||
- Request bodies, HTTP file input artifacts, and encoded JSON responses are size-limited.
|
||||
- Validation content failures return `200 OK` with `validation.status: "failed"`.
|
||||
|
||||
Security boundary:
|
||||
|
||||
- `serve` has no built-in authentication or authorization.
|
||||
- Deploy only behind trusted controls (private network boundary, authenticated reverse proxy, API gateway, or equivalent).
|
||||
- Keep the HTTP artifact root as narrow as practical and do not make it writable by untrusted users.
|
||||
- Put it behind trusted controls such as a private network, authenticated reverse proxy, or API gateway.
|
||||
- Do not expose an artifact root containing unrelated sensitive files.
|
||||
- Symlinks inside the artifact root are followed by the operating system.
|
||||
|
||||
## Secrets Handling
|
||||
|
||||
Raw API keys are not accepted in app config, profiles, CLI flags, or HTTP
|
||||
request bodies.
|
||||
|
||||
Use this pattern:
|
||||
|
||||
1. Set an environment variable containing the secret value.
|
||||
2. Store only the variable name in profile `api_key_env` or request override `api_key_env`.
|
||||
3. Scope the process environment to the minimum required variables.
|
||||
|
||||
## Output, Logs, And Exit Codes
|
||||
|
||||
`run` command:
|
||||
`run`:
|
||||
|
||||
- Generated artifact body goes to stdout by default.
|
||||
- `--out` writes generated artifact to a file.
|
||||
- Summary metadata line is written to stderr on success.
|
||||
- Exit code `2` means generation completed but validation failed.
|
||||
- stdout: generated artifact body unless `--out` is used.
|
||||
- stderr: summary on success, errors on failure.
|
||||
- exit `2`: generation completed and output was written, but validation failed.
|
||||
|
||||
`render` command:
|
||||
`render`:
|
||||
|
||||
- Prepared-run output goes to stdout by default.
|
||||
- `--out` writes prepared-run output to a file.
|
||||
- Exit code is `0` on success and `1` on failure.
|
||||
- stdout: prepared-run output unless `--out` is used.
|
||||
- stderr: errors.
|
||||
- exit `0` on success, `1` on failure.
|
||||
|
||||
`serve` command:
|
||||
`serve`:
|
||||
|
||||
- Startup and server errors are written to stderr.
|
||||
- stderr: startup and server errors.
|
||||
- HTTP response body: JSON success or error envelope.
|
||||
|
||||
## Validation Behavior In Operations
|
||||
## Validation Behavior
|
||||
|
||||
Validation modes (`none`, `basic`, `json`, `json_schema`) are defined by prompt output contract.
|
||||
Prompt `output.validation_mode` controls validation:
|
||||
|
||||
Operational interpretation:
|
||||
- `none`: skipped.
|
||||
- `basic`: output body must not be empty.
|
||||
- `json`: output body must parse as JSON.
|
||||
- `json_schema`: output body must parse as JSON and satisfy the configured schema.
|
||||
|
||||
- Validation runtime errors are hard failures (`run` exit `1`; HTTP error response).
|
||||
- Validation content failures are soft failures (`run` exit `2`; HTTP `200` with failed status).
|
||||
Runtime/schema failures are hard failures (`run` exit `1`, HTTP error).
|
||||
Generated-content validation failures are soft failures (`run` exit `2`, HTTP
|
||||
`200 OK` with failed validation status).
|
||||
|
||||
A failed validation run can still produce output. Decide whether to keep or discard that output in your surrounding workflow.
|
||||
## Size Limits
|
||||
|
||||
## Safe Recovery Steps
|
||||
Defaults are documented in [Configuration reference](config.md). Operationally:
|
||||
|
||||
For failed runs or requests:
|
||||
- Keep default HTTP limits unless larger payloads are measured and expected.
|
||||
- Prefer `inline` HTTP inputs for small payloads.
|
||||
- Prefer `file` HTTP inputs for larger local artifacts under a controlled artifact root.
|
||||
- Increase `server.max_response_bytes` when generated artifacts or requested raw output are expected to be large.
|
||||
- Use `0` only when another trusted layer enforces size limits.
|
||||
|
||||
1. Capture stderr output or HTTP error code/message.
|
||||
2. Confirm config path and directory settings.
|
||||
3. Verify prompt/profile IDs and input mappings.
|
||||
4. Verify API-key environment-variable presence when required.
|
||||
5. Reproduce with `render --format json` when prompt/profile/input resolution is uncertain.
|
||||
## Maintained Examples
|
||||
|
||||
- `examples/config.yml`
|
||||
- `examples/config.full.yml`
|
||||
- `examples/render-markdown-summary.sh`
|
||||
- `examples/http-run.json`
|
||||
|
||||
## Safe Recovery
|
||||
|
||||
For failed CLI commands or HTTP requests:
|
||||
|
||||
1. Capture stderr or the HTTP error `code` and `message`.
|
||||
2. Confirm config path and effective directory settings.
|
||||
3. Verify prompt ID, profile ID, schema path, and input mappings.
|
||||
4. Verify required API-key environment variables.
|
||||
5. Reproduce with `render --format json` when pre-LLM resolution is uncertain.
|
||||
6. Rerun after correction.
|
||||
|
||||
Because Scriptorium does not persist run state, rerun is the canonical recovery path.
|
||||
Because Scriptorium does not persist run state, rerun is the supported recovery
|
||||
path.
|
||||
|
||||
@@ -11,6 +11,7 @@ Scriptorium is a narrow prompt-execution application with three entry paths:
|
||||
- CLI `run`
|
||||
- CLI `render`
|
||||
- HTTP `POST /v1/runs` through `serve`
|
||||
- public Go package `gitea.maximumdirect.net/eric/scriptorium`
|
||||
|
||||
Domain behavior is centralized in `internal/usecase` and `internal/domain`.
|
||||
|
||||
@@ -26,6 +27,7 @@ Domain behavior is centralized in `internal/usecase` and `internal/domain`.
|
||||
|
||||
Current package map:
|
||||
|
||||
- root package `scriptorium`: public Go facade over engine construction, source options, request/result types, and error mapping.
|
||||
- `cmd/scriptorium`: process entrypoint.
|
||||
- `internal/adapter/cli`: command parsing, app wiring for CLI commands, output behavior.
|
||||
- `internal/adapter/http`: HTTP DTO mapping and error/status mapping.
|
||||
@@ -34,7 +36,9 @@ Current package map:
|
||||
- `internal/domain`: core request/result and contract types.
|
||||
- `internal/usecase`: `Runner` prepare/run orchestration and repair-hook boundary.
|
||||
- `internal/promptdef`: filesystem prompt-definition repository.
|
||||
- `internal/profile`: filesystem execution-profile repository.
|
||||
- `internal/profile`: filesystem, `fs.FS`, and overlay execution-profile repositories.
|
||||
- `internal/profile/builtin`: embedded built-in execution profiles.
|
||||
- `internal/filecatalog`: shared YAML discovery and `fs.FS` source helpers.
|
||||
- `internal/artifact`: artifact reference readers.
|
||||
- `internal/prompt`: template renderer.
|
||||
- `internal/llm`: provider-neutral LLM client interface and OpenAI-compatible implementation.
|
||||
@@ -45,6 +49,7 @@ Detailed component behavior is documented in:
|
||||
|
||||
- `docs/internal/runner.md`
|
||||
- `docs/internal/adapters.md`
|
||||
- `docs/internal/sources.md`
|
||||
|
||||
## Configuration And Precedence
|
||||
|
||||
@@ -69,9 +74,10 @@ Scriptorium has no durable run-state store.
|
||||
|
||||
Current external contracts:
|
||||
|
||||
- inbound HTTP contract: `POST /v1/runs`
|
||||
- inbound HTTP contract: `POST /v1/runs`, documented canonically in `docs/api.md`
|
||||
- outbound model contract: OpenAI-compatible chat completions subset
|
||||
- subprocess contract for integrators: CLI `run`/`render`
|
||||
- public Go package contract: `docs/consumers/pkg-scriptorium.md`
|
||||
|
||||
Integration docs belong under `docs/integrations/`.
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ This document defines contributor workflow for Scriptorium.
|
||||
|
||||
## Repository Layout
|
||||
|
||||
- root package `scriptorium`: public Go facade, options, types, and error mapping.
|
||||
- `cmd/scriptorium`: application entrypoint.
|
||||
- `internal/domain`: core contracts.
|
||||
- `internal/usecase`: runner orchestration.
|
||||
@@ -13,6 +14,8 @@ This document defines contributor workflow for Scriptorium.
|
||||
- `internal/defaults`: default constants.
|
||||
- `internal/promptdef`: prompt-definition repository.
|
||||
- `internal/profile`: execution-profile repository.
|
||||
- `internal/profile/builtin`: embedded built-in execution profiles.
|
||||
- `internal/filecatalog`: shared source discovery and path helpers.
|
||||
- `internal/artifact`: artifact readers.
|
||||
- `internal/prompt`: prompt rendering.
|
||||
- `internal/llm`: LLM client interface and OpenAI-compatible implementation.
|
||||
@@ -38,7 +41,9 @@ go test ./...
|
||||
Targeted test runs commonly used during changes:
|
||||
|
||||
```bash
|
||||
go test .
|
||||
go test ./internal/adapter/cli ./internal/adapter/http ./internal/usecase
|
||||
go test ./internal/...
|
||||
```
|
||||
|
||||
## Coding Conventions
|
||||
@@ -82,7 +87,8 @@ go test ./internal/adapter/cli ./internal/adapter/http ./internal/usecase
|
||||
3. Keep business decisions in `internal/usecase`.
|
||||
4. Add focused adapter tests for mapping, parse, and error behavior.
|
||||
5. Document the new/changed boundary in `docs/internal/adapters.md`.
|
||||
6. If external contract changes, update `docs/integrations/` in the same change.
|
||||
6. If source-loading behavior changes, update `docs/internal/sources.md`.
|
||||
7. If an external contract changes, update the canonical public or integration doc in the same change.
|
||||
|
||||
## How To Update Prompt/Profile/Schema Assets
|
||||
|
||||
@@ -99,5 +105,6 @@ When behavior changes:
|
||||
2. Keep non-roadmap docs limited to implemented behavior.
|
||||
3. Update links after file moves/renames.
|
||||
4. Re-run relevant tests and smoke commands.
|
||||
5. For internal boundary docs, check references with `rg "docs/internal|internal/sources" docs/policy docs/internal`.
|
||||
|
||||
Docs work is complete only when code/tests/examples/docs agree.
|
||||
|
||||
@@ -1,324 +0,0 @@
|
||||
# Full Codebase Cleanup And Hardening Plan
|
||||
|
||||
This document is a decision-complete implementation plan for the current full-codebase audit findings. It is a planning document only. Implementation should proceed in stages and should not change unrelated behavior.
|
||||
|
||||
The goal is to polish and harden the public and internal code paths without expanding Scriptorium's scope. Keep the existing package boundaries unless a stage explicitly calls for a helper extraction.
|
||||
|
||||
## Guiding Decisions
|
||||
|
||||
- Treat configured `fs.FS` roots as real containment boundaries for public source options.
|
||||
- Keep local directory-backed CLI behavior compatible unless this plan explicitly names a change.
|
||||
- Keep HTTP `serve` minimal, but safe by default against accidental large request, file, and response bodies.
|
||||
- Do not add HTTP authentication in this cleanup pass.
|
||||
- Do not add new dependencies.
|
||||
- Document only implemented behavior outside `docs/roadmap/`.
|
||||
|
||||
## Stage 1: Enforce Source-Root Containment For `fs.FS` Prompt And Schema Sources
|
||||
|
||||
Problem:
|
||||
|
||||
Public source options such as `WithPromptFS(fsys, root)` and `WithSchemaFS(fsys, root)` describe `root` as the source boundary, but prompt `content_file` and schema path resolution can clean `..` paths above that root when the supplied `fs.FS` is broader than the configured root.
|
||||
|
||||
Decision:
|
||||
|
||||
For public `fs.FS` source options, the configured root is a containment boundary. Prompt `content_file` paths and schema paths must resolve inside that root. Absolute paths and relative traversal that escape the root are invalid.
|
||||
|
||||
Implementation:
|
||||
|
||||
1. Add a small shared internal helper for `fs.FS` path resolution.
|
||||
- Prefer `internal/filecatalog` if the helper naturally belongs with existing clean/display path utilities.
|
||||
- Inputs should include a source root and a user path.
|
||||
- It should trim whitespace, clean slash paths with `path.Clean`, reject empty paths where the caller requires a file, reject absolute paths, and reject any path whose clean form escapes the clean root.
|
||||
- Use path-component checks, not string-prefix checks alone.
|
||||
- Return both the resolved `fs.FS` path and a display path when useful for errors.
|
||||
2. Update `internal/promptdef` `fsRepository` content-file loading.
|
||||
- `content_file: ./local.tmpl` beside the prompt should continue to work.
|
||||
- Nested prompt files should keep the existing relative-to-prompt-file behavior.
|
||||
- `content_file` values that escape the configured `WithPromptFS` root should return a prompt-load error.
|
||||
3. Update `internal/validate` `FSValidator` schema resolution.
|
||||
- `WithSchemaFS(fsys, root)` should allow schema paths inside `root`.
|
||||
- `WithSchemaFile(path)` should keep the existing single-file behavior: prompt `schema_path` must match the selected file's base name.
|
||||
- Schema paths that escape the configured root should return validation/schema-load errors.
|
||||
4. Preserve directory-backed compatibility unless a failing test reveals an inconsistency that must be fixed.
|
||||
- `WithPromptFile(path)` should continue resolving `content_file` values relative to the selected prompt file's directory.
|
||||
- `Config.SchemaDir` / CLI `--schema-dir` should keep documented behavior, including absolute `schema_path` support, because these are operator-controlled local filesystem paths.
|
||||
|
||||
Tests:
|
||||
|
||||
- Add `internal/promptdef` tests for `fs.FS` `content_file` traversal:
|
||||
- sibling file inside root succeeds;
|
||||
- nested file inside root succeeds;
|
||||
- `../outside.tmpl` from a prompt under the root is rejected;
|
||||
- absolute-style `/outside.tmpl` is rejected.
|
||||
- Add public package tests through `WithPromptFS` proving escaped `content_file` returns `ErrPromptLoad`.
|
||||
- Add `internal/validate` tests for `WithSchemaFS` traversal:
|
||||
- schema inside root succeeds;
|
||||
- `../outside.schema.json` is rejected;
|
||||
- absolute-style paths are rejected.
|
||||
- Keep existing `WithSchemaFile` tests passing.
|
||||
|
||||
Documentation:
|
||||
|
||||
- Update `docs/consumers/pkg-scriptorium.md` to state that `WithPromptFS` and `WithSchemaFS` roots are containment boundaries.
|
||||
- Update `docs/config.md` only if directory-backed behavior changes. Otherwise leave its local-directory schema-path behavior intact.
|
||||
- Update `docs/internal/adapters.md` if shared source-resolution behavior is documented there.
|
||||
|
||||
## Stage 2: Add HTTP Request, Artifact, And Response Size Limits
|
||||
|
||||
Problem:
|
||||
|
||||
HTTP `serve` decodes request bodies directly from `r.Body`, reads file artifacts fully into memory, and serializes generated artifact bodies fully into the response. This is acceptable for trusted small local use, but it is not hardened against accidental or hostile large inputs.
|
||||
|
||||
Decision:
|
||||
|
||||
Add configurable HTTP size limits with conservative defaults. Limits apply only to HTTP `serve`; CLI `run` and `render` keep existing direct filesystem behavior.
|
||||
|
||||
Default limits:
|
||||
|
||||
- `server.max_request_bytes`: 16 MiB.
|
||||
- `server.max_artifact_bytes`: 16 MiB.
|
||||
- `server.max_response_bytes`: 16 MiB.
|
||||
|
||||
Use `0` to disable a specific limit only where this is consistent with existing config style. Negative values are invalid config.
|
||||
|
||||
Implementation:
|
||||
|
||||
1. Add default constants in `internal/defaults`.
|
||||
2. Extend app config in `internal/config`.
|
||||
- Add `server.max_request_bytes`.
|
||||
- Add `server.max_artifact_bytes`.
|
||||
- Add `server.max_response_bytes`.
|
||||
- Apply built-in defaults, config-file values, and CLI overrides according to existing precedence.
|
||||
- Reject negative values.
|
||||
3. Add `serve` CLI overrides.
|
||||
- `--max-request-bytes`
|
||||
- `--max-artifact-bytes`
|
||||
- `--max-response-bytes`
|
||||
- Keep these flags scoped to `serve`.
|
||||
4. Extend HTTP handler construction.
|
||||
- Add `httpadapter.HandlerOptions` with request and response limit fields.
|
||||
- Keep `httpadapter.NewHandler(runner)` as a default constructor for existing tests and callers.
|
||||
- Add `httpadapter.NewHandlerWithOptions(runner, options)` for `serve` wiring.
|
||||
- `NewHandler(runner)` should apply built-in defaults.
|
||||
- `NewHandlerWithOptions(runner, options)` should use the supplied values exactly, so `0` means disabled after config validation.
|
||||
5. Limit request decoding.
|
||||
- Wrap `r.Body` with `http.MaxBytesReader` when `max_request_bytes > 0`.
|
||||
- Return `413 request_too_large` when decoding fails due to size.
|
||||
- Continue returning `400 invalid_json` for malformed JSON.
|
||||
- Ensure the decoder rejects trailing JSON tokens if it does not already.
|
||||
6. Limit HTTP file artifact reads.
|
||||
- Add a max-bytes option to the restricted HTTP artifact reader.
|
||||
- Use `os.Open`, `Stat`, and `io.LimitReader` or equivalent instead of unbounded `os.ReadFile` for restricted HTTP file reads.
|
||||
- If a file exceeds the configured limit, return a specific artifact error that maps to `413 artifact_too_large`.
|
||||
- Keep inline artifact bodies covered by the request-body limit.
|
||||
7. Limit HTTP response artifact bodies.
|
||||
- Build the response DTO, marshal it to JSON bytes, and compare the final encoded response size against `max_response_bytes` when the limit is positive.
|
||||
- Return `413 response_too_large` when the encoded response exceeds the configured limit.
|
||||
- Do not truncate successful artifacts silently.
|
||||
- Apply the same encoded-response check when `include_raw_output` is true.
|
||||
|
||||
Tests:
|
||||
|
||||
- Config tests:
|
||||
- defaults are applied;
|
||||
- config file values load;
|
||||
- CLI overrides win;
|
||||
- negative values are rejected.
|
||||
- CLI tests:
|
||||
- `serve` parses the three flags;
|
||||
- `run` and `render` do not gain these flags.
|
||||
- HTTP tests:
|
||||
- oversized request body returns `413 request_too_large`;
|
||||
- malformed JSON below the limit still returns `400 invalid_json`;
|
||||
- successful request below the limit still works;
|
||||
- oversized generated artifact returns the configured too-large error;
|
||||
- `include_raw_output` does not bypass response limits.
|
||||
- Artifact tests:
|
||||
- restricted file reader accepts files at or below the limit;
|
||||
- restricted file reader rejects files above the limit;
|
||||
- unlimited mode with `0` keeps existing behavior.
|
||||
|
||||
Documentation:
|
||||
|
||||
- Update `docs/config.md` with the new server limit fields and defaults.
|
||||
- Update `docs/cli.md` with the new `serve` flags.
|
||||
- Update `docs/integrations/http-api.md` with `413` errors.
|
||||
- Update `docs/operations.md` with sizing guidance.
|
||||
- Update `docs/troubleshooting.md` with common size-limit failures.
|
||||
- Update `docs/internal/adapters.md` with the HTTP limit boundary.
|
||||
|
||||
## Stage 3: Harden `OpenAICompatibleProfile` ExtraParams Copying
|
||||
|
||||
Problem:
|
||||
|
||||
`OpenAICompatibleProfile` currently deep-copies `ExtraParams` through the general internal copy helper. Cyclic caller-provided maps can recurse indefinitely before `WithProfiles` can validate and return `ErrInvalidConfig`.
|
||||
|
||||
Decision:
|
||||
|
||||
`OpenAICompatibleProfile` is a convenience constructor, not a validator. It must not recursively walk caller-provided `ExtraParams`. Validation and safe deep copying belong in `WithProfiles` through the existing public JSON validation path.
|
||||
|
||||
Implementation:
|
||||
|
||||
1. Change `OpenAICompatibleProfile` to use a shallow map copy for `ExtraParams`.
|
||||
- Copy only the top-level `map[string]any`.
|
||||
- Do not recursively copy nested values.
|
||||
- Do not call `copyAnyMap` from this constructor.
|
||||
2. Keep `WithProfiles` validation and deep-copy behavior unchanged.
|
||||
- Unsupported values, non-finite numbers, non-string map keys, and cycles should still return `ErrInvalidConfig`.
|
||||
3. Review `copyAnyMap` call sites.
|
||||
- Keep it for trusted internal-to-public conversions where values come from already-decoded JSON-like data.
|
||||
- Do not use it for untrusted public caller input before validation.
|
||||
4. Add a short comment near the constructor if needed to clarify that recursive validation is intentionally deferred.
|
||||
|
||||
Tests:
|
||||
|
||||
- Add a public package test where `OpenAICompatibleProfile` receives cyclic `ExtraParams`.
|
||||
- The constructor must return promptly.
|
||||
- `NewEngine(..., WithProfiles(profile))` must return `ErrInvalidConfig`.
|
||||
- Add a test proving non-cyclic nested `ExtraParams` still work through `WithProfiles`.
|
||||
- Keep existing mutation-isolation tests passing.
|
||||
|
||||
Documentation:
|
||||
|
||||
- No user-facing behavior change is required if existing docs already state that `WithProfiles` validates `ExtraParams`.
|
||||
- Update docs only if constructor behavior is currently described as validating or deep-copying recursively.
|
||||
|
||||
## Stage 4: Redact Provider Non-2xx Response Bodies From Default Errors
|
||||
|
||||
Problem:
|
||||
|
||||
The OpenAI-compatible client includes a trimmed provider response body snippet in `ErrUnexpectedStatus`. CLI and library callers may log this error. Provider error bodies can include prompt fragments, schema details, request IDs, or other sensitive operational data.
|
||||
|
||||
Decision:
|
||||
|
||||
Default errors should include the provider status code but not the response body. Do not add a debug mode in this pass unless an existing debug/logging surface already supports it.
|
||||
|
||||
Implementation:
|
||||
|
||||
1. Change the non-2xx error returned by `internal/llm.OpenAICompatibleClient`.
|
||||
- Keep wrapping `ErrUnexpectedStatus`.
|
||||
- Include `status=<code>`.
|
||||
- Do not include response body text.
|
||||
2. Drain and close the response body safely enough for normal HTTP client reuse.
|
||||
- It is acceptable to read and discard a small bounded amount if needed.
|
||||
- Do not store or return the discarded content.
|
||||
3. Review tests that assert the old body-snippet behavior and update them.
|
||||
4. Review CLI, HTTP, and public package error mapping.
|
||||
- HTTP should remain generic and not leak provider details.
|
||||
- CLI/library errors should retain enough status context to diagnose provider failures.
|
||||
|
||||
Tests:
|
||||
|
||||
- Update `internal/llm` non-2xx tests:
|
||||
- `errors.Is(err, ErrUnexpectedStatus)` remains true;
|
||||
- the status code appears in the error string;
|
||||
- the provider response body does not appear in the error string.
|
||||
- Add a regression test with a body containing distinctive sensitive-looking text and assert it is absent.
|
||||
|
||||
Documentation:
|
||||
|
||||
- Update `docs/integrations/openai-compatible-chat.md` to remove the claim that `ErrUnexpectedStatus` includes a response body snippet.
|
||||
- Update troubleshooting docs only if they currently instruct users to inspect provider body snippets.
|
||||
|
||||
## Stage 5: Clarify Artifact-Root Symlink Semantics
|
||||
|
||||
Problem:
|
||||
|
||||
The restricted HTTP artifact reader uses lexical path containment before reading the file. Symlinks inside the artifact root are followed by the operating system. This is documented, but the phrase "must stay inside the root" can be overread as a strict realpath guarantee.
|
||||
|
||||
Decision:
|
||||
|
||||
Keep symlink-following behavior for this cleanup pass, but make the code and docs explicit that containment is lexical and relies on the artifact root not being writable by untrusted users. This avoids a potentially breaking change for deployments that intentionally use symlinks.
|
||||
|
||||
Implementation:
|
||||
|
||||
1. Rename or comment the restricted reader's path-resolution helper to make the lexical nature clear.
|
||||
2. Add tests documenting current symlink behavior where the platform supports symlinks.
|
||||
- A symlink inside the root to a file outside the root is followed.
|
||||
- The test should skip cleanly if symlink creation is unavailable.
|
||||
3. Keep traversal rejection tests for `..` and absolute paths outside the root.
|
||||
4. Do not introduce `filepath.EvalSymlinks` in this pass.
|
||||
|
||||
Documentation:
|
||||
|
||||
- Update `docs/config.md`, `docs/operations.md`, `docs/integrations/http-api.md`, and `docs/internal/adapters.md` to say:
|
||||
- lexical traversal outside the root is rejected;
|
||||
- symlinks inside the root are followed;
|
||||
- the artifact root must not be writable by untrusted users.
|
||||
- Avoid wording that implies strict realpath containment unless the implementation changes to enforce it.
|
||||
|
||||
Future option:
|
||||
|
||||
If strict filesystem containment becomes required, add an opt-in or replacement mode that resolves both the configured root and requested file with `filepath.EvalSymlinks` before reading, rejects symlink escapes, and documents any compatibility impact.
|
||||
|
||||
## Stage 6: Align CLI Help And Documentation
|
||||
|
||||
Problem:
|
||||
|
||||
The `serve` usage text omits `--artifact-root`, even though the flag exists. New limit flags from Stage 2 also need to appear consistently in CLI help and docs.
|
||||
|
||||
Decision:
|
||||
|
||||
Keep CLI help concise but complete for supported flags.
|
||||
|
||||
Implementation:
|
||||
|
||||
1. Update `printUsage` in `internal/adapter/cli`.
|
||||
- Include `--artifact-root DIR` in the `serve` usage line.
|
||||
- Include the new size-limit flags from Stage 2.
|
||||
- Keep the line readable; splitting long usage text into multiple lines is acceptable if tests are updated.
|
||||
2. Update CLI tests that assert usage output.
|
||||
3. Confirm `docs/cli.md` matches actual flags.
|
||||
|
||||
Tests:
|
||||
|
||||
- Add or update CLI usage tests to assert that `serve` help mentions:
|
||||
- `--artifact-root`;
|
||||
- `--max-request-bytes`;
|
||||
- `--max-artifact-bytes`;
|
||||
- `--max-response-bytes`.
|
||||
|
||||
Documentation:
|
||||
|
||||
- Update `docs/cli.md` and any command examples affected by line wrapping or flag additions.
|
||||
|
||||
## Stage 7: Final Verification
|
||||
|
||||
Run the full verification set after all stages:
|
||||
|
||||
```bash
|
||||
go test ./...
|
||||
go vet ./...
|
||||
go run ./examples/go-library/prepare
|
||||
go run ./cmd/scriptorium render \
|
||||
--config ./examples/config.yml \
|
||||
--prompt generic.markdown_summary \
|
||||
--input transcript=./examples/fixtures/transcript.md \
|
||||
--input glossary=./examples/fixtures/glossary.yml \
|
||||
--format json
|
||||
```
|
||||
|
||||
Also run targeted packages while implementing each stage:
|
||||
|
||||
```bash
|
||||
go test ./internal/promptdef ./internal/validate ./internal/filecatalog
|
||||
go test ./internal/artifact ./internal/adapter/http ./internal/adapter/cli ./internal/config
|
||||
go test .
|
||||
```
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not collapse internal packages merely to reduce package count.
|
||||
- Do not add HTTP authentication or authorization.
|
||||
- Do not remove HTTP file inputs.
|
||||
- Do not change public request/result type names or method signatures.
|
||||
- Do not change CLI `run` or `render` local file-input behavior.
|
||||
- Do not silently truncate request, artifact, provider, or response bodies.
|
||||
- Do not accept raw API keys through config files, profile files, CLI flags, or HTTP payloads.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- Public `fs.FS` roots are intended to be narrower than the supplied filesystem and should therefore be enforced.
|
||||
- The initial HTTP size-limit defaults are intentionally conservative and can be tuned by operators.
|
||||
- Provider response-body diagnostics are less important than safe default error handling.
|
||||
- Symlink compatibility is more important than strict realpath containment for the immediate cleanup pass, provided documentation is explicit.
|
||||
@@ -1,19 +1,25 @@
|
||||
# Troubleshooting
|
||||
|
||||
This guide lists recurring implemented failure modes and safe fixes.
|
||||
This guide lists common implemented failure modes and safe fixes.
|
||||
|
||||
For command syntax, see [CLI reference](cli.md). For configuration and file formats, see [Configuration reference](config.md). For operational behavior, see [Operations guide](operations.md).
|
||||
Canonical references:
|
||||
|
||||
## Missing Or Invalid Config File
|
||||
- [CLI reference](cli.md)
|
||||
- [Configuration reference](config.md)
|
||||
- [HTTP API reference](api.md)
|
||||
- [Operations guide](operations.md)
|
||||
|
||||
## Missing Or Invalid Config
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI errors such as `application config error: config file not found` or `invalid config YAML`.
|
||||
- CLI error includes `application config error`, `config file not found`, `invalid config YAML`, or `invalid config`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- `--config` points to a missing file.
|
||||
- Config YAML has syntax errors or unknown fields.
|
||||
- YAML syntax is invalid.
|
||||
- Config contains unknown fields or negative HTTP size limits.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
@@ -23,40 +29,34 @@ go run ./cmd/scriptorium render --config /path/to/config.yml --prompt generic.ma
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Correct file path.
|
||||
- Remove unknown fields.
|
||||
- Correct the config path.
|
||||
- Fix YAML syntax.
|
||||
- Keep secrets out of config.
|
||||
- Remove unknown fields.
|
||||
- Keep raw secrets out of config.
|
||||
|
||||
Relevant links:
|
||||
Relevant links: [Configuration reference](config.md), [CLI reference](cli.md)
|
||||
|
||||
- [Configuration reference](config.md)
|
||||
- [CLI reference](cli.md)
|
||||
|
||||
## Missing Prompt Directory Settings
|
||||
## Missing Prompt Directory
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI parse errors saying prompt directory is required.
|
||||
- CLI parse error says the prompt directory is required.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- Neither CLI flags nor config provide an effective `prompt_dir`.
|
||||
- Neither config nor CLI flags provide an effective `prompt_dir`.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Run the failing command with explicit `--prompt-dir` once to verify.
|
||||
- Re-run once with explicit `--prompt-dir`.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Set `prompt_dir` in config, or always pass `--prompt-dir`.
|
||||
- Set `prompt_dir` in config or pass `--prompt-dir`.
|
||||
|
||||
Relevant links:
|
||||
Relevant links: [Configuration reference](config.md), [CLI reference](cli.md)
|
||||
|
||||
- [Configuration reference](config.md)
|
||||
- [CLI reference](cli.md)
|
||||
|
||||
## Unknown Or Unsupported Flags
|
||||
## Unknown Flags
|
||||
|
||||
Symptom:
|
||||
|
||||
@@ -64,33 +64,33 @@ Symptom:
|
||||
|
||||
Likely cause:
|
||||
|
||||
- Typo or command mismatch (for example, `serve` with runtime model override flags).
|
||||
- Typo.
|
||||
- Flag is valid for another command.
|
||||
- `serve` was given runtime model override flags.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Compare command against the command-specific flag list.
|
||||
- Compare the command with the command-specific flag list.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Remove unsupported flags.
|
||||
- Use `run`/`render` for runtime model overrides.
|
||||
- Use `run` or `render` for runtime model overrides.
|
||||
|
||||
Relevant links:
|
||||
Relevant links: [CLI reference](cli.md)
|
||||
|
||||
- [CLI reference](cli.md)
|
||||
|
||||
## Prompt Definition Load Failures
|
||||
## Prompt Load Failures
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI run/render error from prompt loading.
|
||||
- HTTP `404 prompt_not_found` or `400 prompt_load_failed`.
|
||||
- CLI run/render fails during prompt loading.
|
||||
- HTTP returns `404 prompt_not_found` or `400 prompt_load_failed`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- Prompt ID not found.
|
||||
- Invalid prompt YAML.
|
||||
- Invalid prompt contract (for example bad validation mode, message content/content_file rule violation, missing schema path for `json_schema`).
|
||||
- Prompt ID/version does not exist.
|
||||
- Prompt YAML is invalid or has unknown fields.
|
||||
- Prompt contract is invalid, such as missing messages, invalid output mode, bad `content_file`, or missing `schema_path` for `json_schema`.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
@@ -100,28 +100,25 @@ go run ./cmd/scriptorium render --config ./examples/config.yml --prompt <prompt-
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Correct prompt ID.
|
||||
- Fix prompt YAML and contract fields.
|
||||
- Ensure referenced `content_file` paths exist.
|
||||
- Correct prompt ID/version.
|
||||
- Fix prompt YAML and referenced `content_file` paths.
|
||||
- Fix output contract fields.
|
||||
|
||||
Relevant links:
|
||||
Relevant links: [Configuration reference](config.md), [CLI reference](cli.md)
|
||||
|
||||
- [Configuration reference](config.md)
|
||||
- [CLI reference](cli.md)
|
||||
|
||||
## Profile Definition Load Failures
|
||||
## Profile Load Failures
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI run/render error from profile loading.
|
||||
- HTTP `404 profile_not_found` or `400 profile_load_failed`.
|
||||
- CLI run/render fails during profile loading.
|
||||
- HTTP returns `404 profile_not_found`, `400 profile_load_failed`, or `400 profile_required`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- Profile ID missing/not found.
|
||||
- Invalid profile YAML.
|
||||
- Invalid profile values.
|
||||
- Raw `api_key` field present (rejected).
|
||||
- Profile ID does not exist.
|
||||
- Request omitted profile and prompt has no `default_profile`.
|
||||
- Profile YAML is invalid or has unknown fields.
|
||||
- Profile contains raw `api_key`.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
@@ -131,85 +128,52 @@ go run ./cmd/scriptorium render --config ./examples/config.yml --prompt generic.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Correct profile ID.
|
||||
- Correct profile ID or prompt `default_profile`.
|
||||
- Fix profile YAML and value ranges.
|
||||
- Replace `api_key` with `api_key_env`.
|
||||
- Replace raw `api_key` with `api_key_env`.
|
||||
|
||||
Relevant links:
|
||||
Relevant links: [Configuration reference](config.md), [CLI reference](cli.md)
|
||||
|
||||
- [Configuration reference](config.md)
|
||||
- [CLI reference](cli.md)
|
||||
|
||||
## Input Artifact Read Failures
|
||||
## Input Artifact Failures
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI run/render error reading input artifacts.
|
||||
- HTTP `400 artifact_read_failed`.
|
||||
- HTTP `400 artifact_not_allowed`.
|
||||
- CLI run/render fails while reading inputs.
|
||||
- HTTP returns `400 artifact_read_failed`, `400 artifact_not_allowed`, or `413 artifact_too_large`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- File path in input mapping does not exist or is unreadable.
|
||||
- Unsupported artifact reference type in HTTP request.
|
||||
- HTTP `file` input references are disabled because no artifact root is configured.
|
||||
- HTTP `file` input path escapes the configured artifact root.
|
||||
- Input file path is missing or unreadable.
|
||||
- HTTP input type is unsupported or missing required fields.
|
||||
- HTTP file refs are disabled because no artifact root is configured.
|
||||
- HTTP file path is lexically outside the artifact root.
|
||||
- HTTP file input exceeds `server.max_artifact_bytes`.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Verify every mapped file path exists and is readable by the process.
|
||||
- For HTTP, verify each input uses supported `type` values.
|
||||
- For HTTP `file` inputs, verify `server.artifact_root` or `serve --artifact-root` is configured and the requested path stays inside that root.
|
||||
- Verify each input path exists and is readable by the process.
|
||||
- For HTTP, verify input refs use `file` or `inline`.
|
||||
- For HTTP file refs, verify the artifact root and compare file size to `server.max_artifact_bytes`.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Correct file paths and permissions.
|
||||
- Use supported input types (`file`, `inline`).
|
||||
- Configure a narrow HTTP artifact root when HTTP file inputs are required.
|
||||
- Use relative paths under the artifact root, or switch to `inline` inputs.
|
||||
- Correct paths and permissions.
|
||||
- Configure a narrow artifact root for HTTP file refs.
|
||||
- Use relative paths under the artifact root or switch to `inline`.
|
||||
- Increase `server.max_artifact_bytes` only for expected larger inputs.
|
||||
|
||||
Relevant links:
|
||||
|
||||
- [CLI reference](cli.md)
|
||||
- [Configuration reference](config.md)
|
||||
- [HTTP API integration](integrations/http-api.md)
|
||||
|
||||
## Prompt Template Render Failures
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI run/render error from prompt rendering.
|
||||
- HTTP `400 prompt_render_failed`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- Template references missing input names.
|
||||
- Template syntax or data reference issues.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Run `render --format json` with the same prompt, inputs, vars, and profile selection.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Align template `{{input "name"}}` references with actual input mappings.
|
||||
- Fix template syntax and variable names.
|
||||
|
||||
Relevant links:
|
||||
|
||||
- [CLI reference](cli.md)
|
||||
- [Configuration reference](config.md)
|
||||
Relevant links: [HTTP API reference](api.md), [Configuration reference](config.md)
|
||||
|
||||
## Missing API-Key Environment Variable
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI run/render invalid request error about missing API-key environment variable.
|
||||
- HTTP `400 api_key_env_missing`.
|
||||
- CLI render/run fails with an API-key environment error.
|
||||
- HTTP returns `400 api_key_env_missing`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- Selected profile or override sets `api_key_env`, but that environment variable is unset/empty.
|
||||
- Selected profile or runtime override sets `api_key_env`, but the environment variable is unset or empty.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
@@ -219,85 +183,68 @@ printenv SCRIPTORIUM_API_KEY
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Set the required environment variable before invoking CLI/service.
|
||||
- Or use a profile that does not require API key auth for the target endpoint.
|
||||
- Set the required environment variable before starting the CLI command or HTTP service.
|
||||
- Or use a profile that does not require provider API-key auth.
|
||||
|
||||
Relevant links:
|
||||
Relevant links: [Configuration reference](config.md), [Operations guide](operations.md)
|
||||
|
||||
- [Configuration reference](config.md)
|
||||
- [Operations guide](operations.md)
|
||||
## Prompt Template Render Failures
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI render/run fails during prompt rendering.
|
||||
- HTTP returns `400 prompt_render_failed`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- Template references an input that was not supplied.
|
||||
- Template syntax or variable reference is invalid.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Run `render --format json` with the same prompt, inputs, vars, and profile.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Align `{{input "name"}}` references with request input names.
|
||||
- Fix template syntax and variable names.
|
||||
|
||||
Relevant links: [Configuration reference](config.md), [CLI reference](cli.md)
|
||||
|
||||
## LLM Request Failures
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI `run` fails with LLM generation errors.
|
||||
- CLI `run` fails during generation.
|
||||
- HTTP returns `502 llm_failed`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- Endpoint unreachable.
|
||||
- Non-2xx response from provider.
|
||||
- Timeout.
|
||||
- Malformed provider response.
|
||||
- Endpoint is unreachable.
|
||||
- Provider returns non-2xx.
|
||||
- Request times out.
|
||||
- Provider response is malformed.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Confirm endpoint URL and model in selected profile/overrides.
|
||||
- Retry with `render` first to confirm pre-LLM preparation works.
|
||||
- Check provider/network logs for non-2xx responses and timeouts.
|
||||
- Run `render` first to confirm pre-LLM preparation works.
|
||||
- Check selected endpoint/model in prepared output.
|
||||
- Check network/provider logs for timeout or non-2xx details.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Correct endpoint/model settings.
|
||||
- Adjust timeout if needed.
|
||||
- Resolve provider-side or network issues.
|
||||
- Correct endpoint/model/profile settings.
|
||||
- Adjust timeout when appropriate.
|
||||
- Resolve provider or network issue.
|
||||
|
||||
Relevant links:
|
||||
Relevant links: [Operations guide](operations.md), [Configuration reference](config.md)
|
||||
|
||||
- [CLI reference](cli.md)
|
||||
- [Configuration reference](config.md)
|
||||
- [Operations guide](operations.md)
|
||||
|
||||
## Prompt Cache Misses Or No Cache Usage
|
||||
## Validation Failed
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI run summary omits `cached_tokens` / `cache_write_tokens`.
|
||||
- HTTP `metadata.usage.cached_tokens` and `metadata.usage.cache_write_tokens` are both `0`.
|
||||
- Provider cost or latency does not improve after repeated similar runs.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- The selected prompt has no `messages[].cache_control`.
|
||||
- Dynamic per-run input appears before the cache-controlled message and changes the provider cache key.
|
||||
- The provider does not support the serialized cache-control shape for the selected model.
|
||||
- The provider imposes minimum token thresholds or cache-breakpoint limits.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Run `render --format json` and verify the intended rendered message includes `cache_control`.
|
||||
- Confirm stable reusable context appears before the cache-controlled message, with dynamic input after it.
|
||||
- Check provider docs/logs for model support, minimum token thresholds, and breakpoint limits.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Move stable reusable context before the cache-controlled message.
|
||||
- Move highly dynamic input after the cache breakpoint.
|
||||
- Keep `cache_control.type: ephemeral` and, when using `ttl`, set `ttl: 1h`.
|
||||
- Use CLI cache counters or HTTP cache usage fields to verify cache reads/writes after rerunning.
|
||||
|
||||
Relevant links:
|
||||
|
||||
- [Configuration reference](config.md)
|
||||
- [OpenAI-compatible chat integration](integrations/openai-compatible-chat.md)
|
||||
|
||||
## Validation Status Failed (`run` Exit 2 Or HTTP 200 With Failed Status)
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI exits with code `2`.
|
||||
- HTTP returns `200`, but `validation.status` is `failed`.
|
||||
- CLI `run` exits `2`.
|
||||
- HTTP returns `200 OK` with `validation.status` set to `failed`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
@@ -305,18 +252,15 @@ Likely cause:
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Inspect validation mode and validation errors in CLI summary/HTTP response.
|
||||
- Inspect validation errors in CLI stderr or the HTTP response.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Refine prompt constraints.
|
||||
- Tighten schema or adjust model/profile settings.
|
||||
- Refine prompt instructions.
|
||||
- Adjust schema or model/profile settings.
|
||||
- Rerun after correction.
|
||||
|
||||
Relevant links:
|
||||
|
||||
- [Configuration reference](config.md)
|
||||
- [Operations guide](operations.md)
|
||||
Relevant links: [Operations guide](operations.md), [HTTP API reference](api.md)
|
||||
|
||||
## Validation Runtime Failure
|
||||
|
||||
@@ -327,48 +271,91 @@ Symptom:
|
||||
|
||||
Likely cause:
|
||||
|
||||
- `json_schema` schema file missing/inaccessible.
|
||||
- Invalid schema JSON document.
|
||||
- `json_schema` schema file is missing or unreadable.
|
||||
- Schema JSON is invalid.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Verify `schema_dir` and `output.schema_path` resolution.
|
||||
- Check schema file readability and valid JSON syntax.
|
||||
- Verify `schema_dir` and prompt `output.schema_path`.
|
||||
- Check schema file readability and JSON syntax.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Correct schema path.
|
||||
- Fix schema JSON content.
|
||||
- Correct schema path or permissions.
|
||||
- Fix schema JSON.
|
||||
- Rerun.
|
||||
|
||||
Relevant links:
|
||||
Relevant links: [Configuration reference](config.md), [Operations guide](operations.md)
|
||||
|
||||
- [Configuration reference](config.md)
|
||||
- [Operations guide](operations.md)
|
||||
|
||||
## HTTP Request Parsing/Contract Errors
|
||||
## HTTP JSON Or Request Contract Errors
|
||||
|
||||
Symptom:
|
||||
|
||||
- HTTP `400 invalid_json` or `400 invalid_request`.
|
||||
- HTTP returns `400 invalid_json` or `400 invalid_request`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- Malformed JSON body.
|
||||
- Unknown JSON fields.
|
||||
- Missing required `prompt_id` or `inputs`.
|
||||
- JSON body is malformed.
|
||||
- Request has unknown fields or trailing JSON tokens.
|
||||
- Required `prompt_id` or `inputs` is missing.
|
||||
- Runtime override values are out of range.
|
||||
- `extra_params` collides with reserved outbound fields.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Revalidate request JSON.
|
||||
- Confirm required request fields are present.
|
||||
- Revalidate request JSON and compare fields with the API reference.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Send valid JSON with only supported fields.
|
||||
- Ensure `prompt_id` and at least one input mapping are included.
|
||||
- Send one JSON object with only supported fields.
|
||||
- Include `prompt_id` and at least one input.
|
||||
- Use valid model override ranges.
|
||||
- Remove reserved `extra_params` keys.
|
||||
|
||||
Relevant links:
|
||||
Relevant links: [HTTP API reference](api.md)
|
||||
|
||||
- [Operations guide](operations.md)
|
||||
- [CLI reference](cli.md)
|
||||
## HTTP Size Limit Errors
|
||||
|
||||
Symptom:
|
||||
|
||||
- HTTP returns `413 request_too_large`, `413 artifact_too_large`, or `413 response_too_large`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- JSON request body exceeds `server.max_request_bytes`.
|
||||
- HTTP file input exceeds `server.max_artifact_bytes`.
|
||||
- Encoded JSON response exceeds `server.max_response_bytes`.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Compare request, file input, and expected response sizes with configured limits.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Use smaller inline inputs or switch to file inputs under the artifact root.
|
||||
- Reduce generated output size.
|
||||
- Omit `include_raw_output`.
|
||||
- Increase limits only when the deployment expects larger payloads.
|
||||
|
||||
Relevant links: [HTTP API reference](api.md), [Operations guide](operations.md)
|
||||
|
||||
## HTTP Route Or Method Errors
|
||||
|
||||
Symptom:
|
||||
|
||||
- HTTP returns `404 not_found` or `405 method_not_allowed`.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- Path is not `/v1/runs`.
|
||||
- Method on `/v1/runs` is not `POST`.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Check the request URL and method.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Send `POST /v1/runs`.
|
||||
|
||||
Relevant links: [HTTP API reference](api.md)
|
||||
|
||||
107
engine_test.go
107
engine_test.go
@@ -881,6 +881,38 @@ output:
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrepareWithPromptFSRejectsEscapedContentFile(t *testing.T) {
|
||||
promptFS := fstest.MapFS{
|
||||
"assets/prompts/fs-escape.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: fs.escape
|
||||
version: "1.0.0"
|
||||
default_profile: local-fast
|
||||
messages:
|
||||
- role: user
|
||||
content_file: ../outside.tmpl
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
"assets/outside.tmpl": &fstest.MapFile{Data: []byte(`Outside root.`)},
|
||||
}
|
||||
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
PromptDir: t.TempDir(),
|
||||
ProfileDir: "./examples/profiles",
|
||||
SchemaDir: "./examples/schemas",
|
||||
}, scriptorium.WithPromptFS(promptFS, "assets/prompts"))
|
||||
if err != nil {
|
||||
t.Fatalf("expected engine construction to succeed, got %v", err)
|
||||
}
|
||||
|
||||
_, err = engine.Prepare(context.Background(), scriptorium.RunRequest{PromptID: "fs.escape"})
|
||||
if !errors.Is(err, scriptorium.ErrPromptLoad) {
|
||||
t.Fatalf("expected ErrPromptLoad, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrepareWorksWithPromptFile(t *testing.T) {
|
||||
promptDir := t.TempDir()
|
||||
promptPath := filepath.Join(promptDir, "single.yaml")
|
||||
@@ -893,7 +925,7 @@ inputs:
|
||||
required: true
|
||||
messages:
|
||||
- role: user
|
||||
content: "Summarize {{input \"transcript\"}} from file."
|
||||
content_file: ./single.tmpl
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
@@ -901,6 +933,9 @@ output:
|
||||
`), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(promptDir, "single.tmpl"), []byte(`Summarize {{input "transcript"}} from file.`), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
ProfileDir: "./examples/profiles",
|
||||
@@ -922,6 +957,9 @@ output:
|
||||
if prepared.PromptID != "single.file.prompt" {
|
||||
t.Fatalf("unexpected prompt id: %q", prepared.PromptID)
|
||||
}
|
||||
if len(prepared.Messages) != 1 || !strings.Contains(prepared.Messages[0].Content, "from file") {
|
||||
t.Fatalf("expected content_file body from prompt file, got %+v", prepared.Messages)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrepareWorksWithProfileFSOverBuiltIns(t *testing.T) {
|
||||
@@ -1116,6 +1154,73 @@ func TestOpenAICompatibleProfileRunsThroughNormalProfilePath(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleProfileDefersExtraParamsValidation(t *testing.T) {
|
||||
cyclic := map[string]any{}
|
||||
cyclic["self"] = cyclic
|
||||
|
||||
prof := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
|
||||
ID: "cyclic-template-profile",
|
||||
Endpoint: "http://cyclic-template/v1",
|
||||
Model: "cyclic-template-model",
|
||||
ExtraParams: cyclic,
|
||||
})
|
||||
|
||||
_, err := scriptorium.NewEngine(scriptorium.Config{PromptDir: "./examples/prompts"},
|
||||
scriptorium.WithProfiles(prof),
|
||||
)
|
||||
if !errors.Is(err, scriptorium.ErrInvalidConfig) {
|
||||
t.Fatalf("expected ErrInvalidConfig, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleProfileNestedExtraParamsRunThroughWithProfiles(t *testing.T) {
|
||||
fake := &fakeLLMClient{response: &scriptorium.GenerateResponse{Content: "ok"}}
|
||||
nested := map[string]any{
|
||||
"labels": map[string]string{"route": "primary"},
|
||||
"ids": []int{1, 2, 3},
|
||||
}
|
||||
extraParams := map[string]any{
|
||||
"nested": nested,
|
||||
}
|
||||
prof := scriptorium.OpenAICompatibleProfile(scriptorium.OpenAICompatibleProfileConfig{
|
||||
ID: "nested-template-profile",
|
||||
Endpoint: "http://nested-template/v1",
|
||||
Model: "nested-template-model",
|
||||
ExtraParams: extraParams,
|
||||
})
|
||||
extraParams["added"] = "mutated-after-construction"
|
||||
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
PromptDir: "./examples/prompts",
|
||||
SchemaDir: "./examples/schemas",
|
||||
}, scriptorium.WithProfiles(prof), scriptorium.WithLLMClient(fake))
|
||||
if err != nil {
|
||||
t.Fatalf("expected engine construction to succeed, got %v", err)
|
||||
}
|
||||
nested["added"] = "mutated-after-construction"
|
||||
|
||||
_, err = engine.Run(context.Background(), scriptorium.RunRequest{
|
||||
PromptID: "generic.markdown_summary",
|
||||
ProfileID: "nested-template-profile",
|
||||
Inputs: map[string]scriptorium.ArtifactRef{
|
||||
"transcript": scriptorium.Inline("Rin opens the gate."),
|
||||
"glossary": scriptorium.Inline("gate: A guarded passage."),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected run to succeed, got %v", err)
|
||||
}
|
||||
want := map[string]any{
|
||||
"nested": map[string]any{
|
||||
"labels": map[string]string{"route": "primary"},
|
||||
"ids": []int{1, 2, 3},
|
||||
},
|
||||
}
|
||||
if !reflect.DeepEqual(fake.requests[0].Target.ExtraParams, want) {
|
||||
t.Fatalf("unexpected extra params:\ngot=%#v\nwant=%#v", fake.requests[0].Target.ExtraParams, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInMemoryProfileAPIKeyRequiredBehavior(t *testing.T) {
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
PromptDir: "./examples/prompts",
|
||||
|
||||
13
examples/config.full.yml
Normal file
13
examples/config.full.yml
Normal file
@@ -0,0 +1,13 @@
|
||||
prompt_dir: ./examples/prompts
|
||||
profile_dir: ./examples/profiles
|
||||
schema_dir: ./examples/schemas
|
||||
|
||||
server:
|
||||
addr: 127.0.0.1:8080
|
||||
artifact_root: .
|
||||
max_request_bytes: 16777216
|
||||
max_artifact_bytes: 16777216
|
||||
max_response_bytes: 16777216
|
||||
|
||||
defaults:
|
||||
render_format: text
|
||||
@@ -79,6 +79,9 @@ type serveConfig struct {
|
||||
profileDir string
|
||||
schemaDir string
|
||||
artifactRoot string
|
||||
maxRequestBytes int64
|
||||
maxArtifactBytes int64
|
||||
maxResponseBytes int64
|
||||
}
|
||||
|
||||
type commonCommandSettings struct {
|
||||
@@ -87,6 +90,9 @@ type commonCommandSettings struct {
|
||||
schemaDir string
|
||||
serverAddr string
|
||||
artifactRoot string
|
||||
maxRequestBytes int64
|
||||
maxArtifactBytes int64
|
||||
maxResponseBytes int64
|
||||
defaultRenderFormat renderformat.PreparedRunOutputFormat
|
||||
}
|
||||
|
||||
@@ -204,7 +210,7 @@ func serveCommand(args []string, stderr io.Writer) int {
|
||||
return ExitRuntimeError
|
||||
}
|
||||
|
||||
artifactReader, err := artifactadapter.NewRestrictedCompositeReader(cfg.artifactRoot)
|
||||
artifactReader, err := artifactadapter.NewRestrictedCompositeReaderWithLimit(cfg.artifactRoot, cfg.maxArtifactBytes)
|
||||
if err != nil {
|
||||
fmt.Fprintf(stderr, "artifact root error: %v\n", err)
|
||||
return ExitRuntimeError
|
||||
@@ -212,7 +218,10 @@ func serveCommand(args []string, stderr io.Writer) int {
|
||||
|
||||
runner := newRunnerWithArtifactReader(cfg.promptDir, cfg.profileDir, cfg.schemaDir, llmClient, artifactReader)
|
||||
|
||||
h := httpadapter.NewHandler(runner)
|
||||
h := httpadapter.NewHandlerWithOptions(runner, httpadapter.HandlerOptions{
|
||||
MaxRequestBytes: cfg.maxRequestBytes,
|
||||
MaxResponseBytes: cfg.maxResponseBytes,
|
||||
})
|
||||
srv := &http.Server{
|
||||
Addr: cfg.addr,
|
||||
Handler: h,
|
||||
@@ -290,6 +299,9 @@ func parseServeArgs(args []string) (*serveConfig, error) {
|
||||
fs.StringVar(&cfg.profileDir, "profile-dir", "", "directory containing execution profile YAML files")
|
||||
fs.StringVar(&cfg.schemaDir, "schema-dir", "", "base directory for validation schemas")
|
||||
fs.StringVar(&cfg.artifactRoot, "artifact-root", "", "base directory for HTTP file input artifacts")
|
||||
fs.Int64Var(&cfg.maxRequestBytes, "max-request-bytes", 0, "maximum HTTP request body bytes; 0 disables the limit")
|
||||
fs.Int64Var(&cfg.maxArtifactBytes, "max-artifact-bytes", 0, "maximum HTTP file artifact bytes; 0 disables the limit")
|
||||
fs.Int64Var(&cfg.maxResponseBytes, "max-response-bytes", 0, "maximum HTTP response body bytes; 0 disables the limit")
|
||||
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return nil, err
|
||||
@@ -304,6 +316,9 @@ func parseServeArgs(args []string) (*serveConfig, error) {
|
||||
SchemaDir: cfg.schemaDirIfSet(fs),
|
||||
ServerAddr: cfg.addrIfSet(fs),
|
||||
ArtifactRoot: cfg.artifactRootIfSet(fs),
|
||||
MaxRequestBytes: cfg.maxRequestBytesIfSet(fs),
|
||||
MaxArtifactBytes: cfg.maxArtifactBytesIfSet(fs),
|
||||
MaxResponseBytes: cfg.maxResponseBytesIfSet(fs),
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -314,6 +329,9 @@ func parseServeArgs(args []string) (*serveConfig, error) {
|
||||
cfg.schemaDir = settings.schemaDir
|
||||
cfg.addr = settings.serverAddr
|
||||
cfg.artifactRoot = settings.artifactRoot
|
||||
cfg.maxRequestBytes = settings.maxRequestBytes
|
||||
cfg.maxArtifactBytes = settings.maxArtifactBytes
|
||||
cfg.maxResponseBytes = settings.maxResponseBytes
|
||||
|
||||
if err := validateRequiredLibraryDirs(cfg.promptDir); err != nil {
|
||||
return nil, err
|
||||
@@ -450,6 +468,27 @@ func (c *serveConfig) artifactRootIfSet(fs *flag.FlagSet) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (c *serveConfig) maxRequestBytesIfSet(fs *flag.FlagSet) *int64 {
|
||||
if flagWasSet(fs, "max-request-bytes") {
|
||||
return &c.maxRequestBytes
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *serveConfig) maxArtifactBytesIfSet(fs *flag.FlagSet) *int64 {
|
||||
if flagWasSet(fs, "max-artifact-bytes") {
|
||||
return &c.maxArtifactBytes
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *serveConfig) maxResponseBytesIfSet(fs *flag.FlagSet) *int64 {
|
||||
if flagWasSet(fs, "max-response-bytes") {
|
||||
return &c.maxResponseBytes
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func registerConfigPathFlag(fs *flag.FlagSet, target *string) {
|
||||
fs.StringVar(
|
||||
target,
|
||||
@@ -488,6 +527,9 @@ func resolveCommonSettings(fs *flag.FlagSet, configPath string, overrides appcon
|
||||
schemaDir: settings.SchemaDir,
|
||||
serverAddr: settings.ServerAddr,
|
||||
artifactRoot: settings.ArtifactRoot,
|
||||
maxRequestBytes: settings.MaxRequestBytes,
|
||||
maxArtifactBytes: settings.MaxArtifactBytes,
|
||||
maxResponseBytes: settings.MaxResponseBytes,
|
||||
defaultRenderFormat: settings.DefaultRenderFormat,
|
||||
}, nil
|
||||
}
|
||||
@@ -666,5 +708,5 @@ func printUsage(w io.Writer) {
|
||||
fmt.Fprintln(w, "usage: scriptorium <run|render|serve> ...")
|
||||
fmt.Fprintln(w, " run: scriptorium run [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID --input name=path [--input ...] [--profile ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--var k=v] [--out path] [--timeout 10m]")
|
||||
fmt.Fprintln(w, " render: scriptorium render [--config PATH] [--prompt-dir DIR] [--profile-dir DIR] --prompt ID --input name=path [--input ...] [--profile ID] [--llm-base-url URL] [--model NAME] [--api-key-env ENV] [--temperature N] [--max-tokens N] [--top-p N] [--var k=v] [--format text|json] [--out path] [--timeout 10m]")
|
||||
fmt.Fprintf(w, " serve: scriptorium serve [--config PATH] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR]\n", defaults.HTTPAddrDefault)
|
||||
fmt.Fprintf(w, " serve: scriptorium serve [--config PATH] [--addr %s] [--prompt-dir DIR] [--profile-dir DIR] [--schema-dir DIR] [--artifact-root DIR] [--max-request-bytes N] [--max-artifact-bytes N] [--max-response-bytes N]\n", defaults.HTTPAddrDefault)
|
||||
}
|
||||
|
||||
@@ -192,6 +192,26 @@ func TestParseServeArgsRejectsRuntimeOverrideFlags(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestUsageIncludesServeFileAndSizeLimitFlags(t *testing.T) {
|
||||
var stderr bytes.Buffer
|
||||
code := Run(nil, io.Discard, &stderr)
|
||||
if code != ExitRuntimeError {
|
||||
t.Fatalf("expected usage path to return runtime error, got %d", code)
|
||||
}
|
||||
|
||||
usage := stderr.String()
|
||||
for _, want := range []string{
|
||||
"--artifact-root",
|
||||
"--max-request-bytes",
|
||||
"--max-artifact-bytes",
|
||||
"--max-response-bytes",
|
||||
} {
|
||||
if !strings.Contains(usage, want) {
|
||||
t.Fatalf("expected usage to include %q, got %q", want, usage)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseRunArgsTimeout(t *testing.T) {
|
||||
cfg, err := parseRunArgs([]string{
|
||||
"--prompt-dir", "./prompts",
|
||||
@@ -436,12 +456,18 @@ schema_dir: ./from-config/schemas
|
||||
server:
|
||||
addr: 127.0.0.1:9000
|
||||
artifact_root: ./from-config/artifacts
|
||||
max_request_bytes: 1024
|
||||
max_artifact_bytes: 2048
|
||||
max_response_bytes: 4096
|
||||
`)
|
||||
|
||||
cfg, err := parseServeArgs([]string{
|
||||
"--config", configPath,
|
||||
"--addr", ":7777",
|
||||
"--artifact-root", "./from-cli/artifacts",
|
||||
"--max-request-bytes", "0",
|
||||
"--max-artifact-bytes", "8192",
|
||||
"--max-response-bytes", "16384",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected valid args, got %v", err)
|
||||
@@ -462,6 +488,15 @@ server:
|
||||
if cfg.artifactRoot != filepath.Clean("./from-cli/artifacts") {
|
||||
t.Fatalf("expected CLI artifact root override, got %q", cfg.artifactRoot)
|
||||
}
|
||||
if cfg.maxRequestBytes != 0 {
|
||||
t.Fatalf("expected CLI max request bytes override, got %d", cfg.maxRequestBytes)
|
||||
}
|
||||
if cfg.maxArtifactBytes != 8192 {
|
||||
t.Fatalf("expected CLI max artifact bytes override, got %d", cfg.maxArtifactBytes)
|
||||
}
|
||||
if cfg.maxResponseBytes != 16384 {
|
||||
t.Fatalf("expected CLI max response bytes override, got %d", cfg.maxResponseBytes)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseServeArgsWithConfigProvidesRequiredDirectoriesAndAddr(t *testing.T) {
|
||||
@@ -472,6 +507,9 @@ schema_dir: ./from-config/schemas
|
||||
server:
|
||||
addr: 127.0.0.1:9000
|
||||
artifact_root: ./from-config/artifacts
|
||||
max_request_bytes: 1024
|
||||
max_artifact_bytes: 2048
|
||||
max_response_bytes: 4096
|
||||
`)
|
||||
|
||||
cfg, err := parseServeArgs([]string{
|
||||
@@ -496,6 +534,72 @@ server:
|
||||
if cfg.artifactRoot != filepath.Clean("./from-config/artifacts") {
|
||||
t.Fatalf("expected artifact root from config, got %q", cfg.artifactRoot)
|
||||
}
|
||||
if cfg.maxRequestBytes != 1024 {
|
||||
t.Fatalf("expected max request bytes from config, got %d", cfg.maxRequestBytes)
|
||||
}
|
||||
if cfg.maxArtifactBytes != 2048 {
|
||||
t.Fatalf("expected max artifact bytes from config, got %d", cfg.maxArtifactBytes)
|
||||
}
|
||||
if cfg.maxResponseBytes != 4096 {
|
||||
t.Fatalf("expected max response bytes from config, got %d", cfg.maxResponseBytes)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseServeArgsRejectsNegativeSizeLimits(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
flag string
|
||||
}{
|
||||
{name: "request", flag: "--max-request-bytes"},
|
||||
{name: "artifact", flag: "--max-artifact-bytes"},
|
||||
{name: "response", flag: "--max-response-bytes"},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
_, err := parseServeArgs([]string{
|
||||
"--prompt-dir", "./prompts",
|
||||
tc.flag, "-1",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("expected negative size limit error")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunAndRenderRejectServeSizeLimitFlags(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
parse func([]string) error
|
||||
}{
|
||||
{
|
||||
name: "run",
|
||||
parse: func(args []string) error {
|
||||
_, err := parseRunArgs(args)
|
||||
return err
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "render",
|
||||
parse: func(args []string) error {
|
||||
_, err := parseRenderArgs(args)
|
||||
return err
|
||||
},
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
err := tc.parse([]string{
|
||||
"--prompt-dir", "./prompts",
|
||||
"--prompt", "p",
|
||||
"--input", "a=b",
|
||||
"--max-request-bytes", "1024",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("expected unsupported flag error")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunAndRenderBuildEquivalentRuntimeOverrideRequestsForSharedFlags(t *testing.T) {
|
||||
|
||||
@@ -4,10 +4,12 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/artifact"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/profile"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/promptdef"
|
||||
@@ -20,10 +22,23 @@ type Runner interface {
|
||||
|
||||
type Handler struct {
|
||||
runner Runner
|
||||
options HandlerOptions
|
||||
}
|
||||
|
||||
type HandlerOptions struct {
|
||||
MaxRequestBytes int64
|
||||
MaxResponseBytes int64
|
||||
}
|
||||
|
||||
func NewHandler(runner Runner) *Handler {
|
||||
return &Handler{runner: runner}
|
||||
return NewHandlerWithOptions(runner, HandlerOptions{
|
||||
MaxRequestBytes: defaults.HTTPMaxRequestBytesDefault,
|
||||
MaxResponseBytes: defaults.HTTPMaxResponseBytesDefault,
|
||||
})
|
||||
}
|
||||
|
||||
func NewHandlerWithOptions(runner Runner, options HandlerOptions) *Handler {
|
||||
return &Handler{runner: runner, options: options}
|
||||
}
|
||||
|
||||
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -37,9 +52,26 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
|
||||
var req runRequestDTO
|
||||
dec := json.NewDecoder(r.Body)
|
||||
body := r.Body
|
||||
if h.options.MaxRequestBytes > 0 {
|
||||
body = http.MaxBytesReader(w, r.Body, h.options.MaxRequestBytes)
|
||||
}
|
||||
dec := json.NewDecoder(body)
|
||||
dec.DisallowUnknownFields()
|
||||
if err := dec.Decode(&req); err != nil {
|
||||
if isRequestTooLarge(err) {
|
||||
writeError(w, http.StatusRequestEntityTooLarge, "request_too_large", "request body is too large")
|
||||
return
|
||||
}
|
||||
writeError(w, http.StatusBadRequest, "invalid_json", "invalid JSON request body")
|
||||
return
|
||||
}
|
||||
var trailing any
|
||||
if err := dec.Decode(&trailing); err != io.EOF {
|
||||
if isRequestTooLarge(err) {
|
||||
writeError(w, http.StatusRequestEntityTooLarge, "request_too_large", "request body is too large")
|
||||
return
|
||||
}
|
||||
writeError(w, http.StatusBadRequest, "invalid_json", "invalid JSON request body")
|
||||
return
|
||||
}
|
||||
@@ -121,7 +153,7 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
raw := res.RawOutput
|
||||
resp.RawModelOutput = &raw
|
||||
}
|
||||
writeJSON(w, http.StatusOK, resp)
|
||||
writeLimitedJSON(w, http.StatusOK, resp, h.options.MaxResponseBytes)
|
||||
}
|
||||
|
||||
func executionTargetOverrideFromModelOverrideDTO(dto *modelOverrideRequestDTO) *domain.ExecutionTargetOverride {
|
||||
@@ -190,6 +222,8 @@ func mapRunError(err error) (int, string, string) {
|
||||
return http.StatusBadRequest, "profile_load_failed", "failed to load execution profile"
|
||||
case errors.Is(err, artifact.ErrFileNotAllowed), errors.Is(err, artifact.ErrFileOutsideRoot):
|
||||
return http.StatusBadRequest, "artifact_not_allowed", "file input artifact is not allowed"
|
||||
case errors.Is(err, artifact.ErrFileTooLarge):
|
||||
return http.StatusRequestEntityTooLarge, "artifact_too_large", "file input artifact is too large"
|
||||
case errors.Is(err, usecase.ErrArtifactLoad):
|
||||
return http.StatusBadRequest, "artifact_read_failed", "failed to read input artifact"
|
||||
case errors.Is(err, usecase.ErrPromptRender):
|
||||
@@ -204,9 +238,23 @@ func mapRunError(err error) (int, string, string) {
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, status int, v any) {
|
||||
writeLimitedJSON(w, status, v, 0)
|
||||
}
|
||||
|
||||
func writeLimitedJSON(w http.ResponseWriter, status int, v any, maxBytes int64) {
|
||||
data, err := json.Marshal(v)
|
||||
if err != nil {
|
||||
writeError(w, http.StatusInternalServerError, "internal_error", "internal server error")
|
||||
return
|
||||
}
|
||||
data = append(data, '\n')
|
||||
if maxBytes > 0 && int64(len(data)) > maxBytes {
|
||||
writeError(w, http.StatusRequestEntityTooLarge, "response_too_large", "response body is too large")
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(v)
|
||||
_, _ = w.Write(data)
|
||||
}
|
||||
|
||||
func writeError(w http.ResponseWriter, status int, code, message string) {
|
||||
@@ -217,3 +265,8 @@ func writeError(w http.ResponseWriter, status int, code, message string) {
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func isRequestTooLarge(err error) bool {
|
||||
var maxBytesErr *http.MaxBytesError
|
||||
return errors.As(err, &maxBytesErr)
|
||||
}
|
||||
|
||||
@@ -243,6 +243,24 @@ func TestHandlerFileRefsUnderArtifactRootWork(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerFileRefsAboveArtifactLimitAreRejected(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(root, "large.txt"), []byte("123456"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
h := newArtifactRootHandlerWithLimit(t, root, 5)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{
|
||||
"prompt_id":"p",
|
||||
"inputs":{"x":{"type":"file","uri":"large.txt"}}
|
||||
}`))
|
||||
w := httptest.NewRecorder()
|
||||
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
assertHTTPErrorCode(t, w, http.StatusRequestEntityTooLarge, "artifact_too_large")
|
||||
}
|
||||
|
||||
func TestHandlerFileRefsOutsideArtifactRootAreRejected(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
outside := t.TempDir()
|
||||
@@ -576,6 +594,69 @@ func TestHandlerInvalidJSON(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerRejectsTrailingJSON(t *testing.T) {
|
||||
h := NewHandler(&fakeRunner{})
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p","inputs":{"x":{"type":"file","uri":"a"}}} {}`))
|
||||
w := httptest.NewRecorder()
|
||||
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
assertHTTPErrorCode(t, w, http.StatusBadRequest, "invalid_json")
|
||||
}
|
||||
|
||||
func TestHandlerRequestTooLarge(t *testing.T) {
|
||||
h := NewHandlerWithOptions(&fakeRunner{}, HandlerOptions{MaxRequestBytes: 12})
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p","inputs":{"x":{"type":"file","uri":"a"}}}`))
|
||||
w := httptest.NewRecorder()
|
||||
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
assertHTTPErrorCode(t, w, http.StatusRequestEntityTooLarge, "request_too_large")
|
||||
}
|
||||
|
||||
func TestHandlerMalformedJSONBelowLimitStillBadRequest(t *testing.T) {
|
||||
h := NewHandlerWithOptions(&fakeRunner{}, HandlerOptions{MaxRequestBytes: 1024})
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString("{"))
|
||||
w := httptest.NewRecorder()
|
||||
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
assertHTTPErrorCode(t, w, http.StatusBadRequest, "invalid_json")
|
||||
}
|
||||
|
||||
func TestHandlerResponseTooLarge(t *testing.T) {
|
||||
h := NewHandlerWithOptions(&fakeRunner{result: &domain.RunResult{
|
||||
Artifact: domain.Artifact{Body: []byte(strings.Repeat("x", 128))},
|
||||
Validation: domain.ValidationResult{Status: domain.ValidationPassed, Mode: domain.ValidationBasic, IsValid: true},
|
||||
EffectiveModelParams: domain.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
|
||||
}}, HandlerOptions{MaxRequestBytes: 1024, MaxResponseBytes: 64})
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"prompt_id":"p","inputs":{"x":{"type":"file","uri":"a"}}}`))
|
||||
w := httptest.NewRecorder()
|
||||
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
assertHTTPErrorCode(t, w, http.StatusRequestEntityTooLarge, "response_too_large")
|
||||
}
|
||||
|
||||
func TestHandlerRawOutputDoesNotBypassResponseLimit(t *testing.T) {
|
||||
h := NewHandlerWithOptions(&fakeRunner{result: &domain.RunResult{
|
||||
Artifact: domain.Artifact{Body: []byte("ok")},
|
||||
RawOutput: strings.Repeat("raw", 80),
|
||||
Validation: domain.ValidationResult{Status: domain.ValidationPassed, Mode: domain.ValidationBasic, IsValid: true},
|
||||
EffectiveModelParams: domain.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
|
||||
}}, HandlerOptions{MaxRequestBytes: 1024, MaxResponseBytes: 128})
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{
|
||||
"prompt_id":"p",
|
||||
"inputs":{"x":{"type":"file","uri":"a"}},
|
||||
"include_raw_output":true
|
||||
}`))
|
||||
w := httptest.NewRecorder()
|
||||
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
assertHTTPErrorCode(t, w, http.StatusRequestEntityTooLarge, "response_too_large")
|
||||
}
|
||||
|
||||
func TestHandlerMissingPromptID(t *testing.T) {
|
||||
h := NewHandler(&fakeRunner{})
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{"inputs":{"x":{"type":"file","uri":"a"}}}`))
|
||||
@@ -770,7 +851,13 @@ func wrap(stage error, cause error) error {
|
||||
func newArtifactRootHandler(t *testing.T, root string) *Handler {
|
||||
t.Helper()
|
||||
|
||||
reader, err := artifact.NewRestrictedCompositeReader(root)
|
||||
return newArtifactRootHandlerWithLimit(t, root, 0)
|
||||
}
|
||||
|
||||
func newArtifactRootHandlerWithLimit(t *testing.T, root string, maxArtifactBytes int64) *Handler {
|
||||
t.Helper()
|
||||
|
||||
reader, err := artifact.NewRestrictedCompositeReaderWithLimit(root, maxArtifactBytes)
|
||||
if err != nil {
|
||||
t.Fatalf("expected restricted artifact reader: %v", err)
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"fmt"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||
"io"
|
||||
"mime"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@@ -19,6 +20,7 @@ var (
|
||||
ErrMissingFilePath = errors.New("missing file path for file artifact")
|
||||
ErrFileNotAllowed = errors.New("file artifact references are not allowed")
|
||||
ErrFileOutsideRoot = errors.New("file artifact path is outside artifact root")
|
||||
ErrFileTooLarge = errors.New("file artifact exceeds size limit")
|
||||
)
|
||||
|
||||
// Reader resolves artifact references into actual artifacts.
|
||||
@@ -40,7 +42,11 @@ func NewCompositeReader() Reader {
|
||||
}
|
||||
|
||||
func NewRestrictedCompositeReader(root string) (Reader, error) {
|
||||
fileReader, err := newRestrictedFileReader(root)
|
||||
return NewRestrictedCompositeReaderWithLimit(root, 0)
|
||||
}
|
||||
|
||||
func NewRestrictedCompositeReaderWithLimit(root string, maxBytes int64) (Reader, error) {
|
||||
fileReader, err := newRestrictedFileReader(root, maxBytes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -123,9 +129,13 @@ func (r deniedFileReader) Read(ctx context.Context, ref domain.ArtifactRef) (*do
|
||||
|
||||
type restrictedFileReader struct {
|
||||
root string
|
||||
maxBytes int64
|
||||
}
|
||||
|
||||
func newRestrictedFileReader(root string) (Reader, error) {
|
||||
func newRestrictedFileReader(root string, maxBytes int64) (Reader, error) {
|
||||
if maxBytes < 0 {
|
||||
return nil, fmt.Errorf("artifact size limit must be greater than or equal to 0")
|
||||
}
|
||||
cleanRoot := strings.TrimSpace(root)
|
||||
if cleanRoot == "" {
|
||||
return deniedFileReader{}, nil
|
||||
@@ -134,7 +144,7 @@ func newRestrictedFileReader(root string) (Reader, error) {
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("resolve artifact root: %w", err)
|
||||
}
|
||||
return &restrictedFileReader{root: absRoot}, nil
|
||||
return &restrictedFileReader{root: absRoot, maxBytes: maxBytes}, nil
|
||||
}
|
||||
|
||||
func (r *restrictedFileReader) Read(ctx context.Context, ref domain.ArtifactRef) (*domain.Artifact, error) {
|
||||
@@ -148,14 +158,15 @@ func (r *restrictedFileReader) Read(ctx context.Context, ref domain.ArtifactRef)
|
||||
return nil, ErrMissingFilePath
|
||||
}
|
||||
|
||||
path, err := r.resolve(ref.URI)
|
||||
path, err := r.resolveLexicalPath(ref.URI)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return readFileArtifact(path)
|
||||
return readFileArtifactWithLimit(path, r.maxBytes)
|
||||
}
|
||||
|
||||
func (r *restrictedFileReader) resolve(rawPath string) (string, error) {
|
||||
// resolveLexicalPath checks cleaned path containment without resolving symlinks.
|
||||
func (r *restrictedFileReader) resolveLexicalPath(rawPath string) (string, error) {
|
||||
cleanPath := filepath.Clean(strings.TrimSpace(rawPath))
|
||||
var candidate string
|
||||
if filepath.IsAbs(cleanPath) {
|
||||
@@ -181,10 +192,39 @@ func (r *restrictedFileReader) resolve(rawPath string) (string, error) {
|
||||
}
|
||||
|
||||
func readFileArtifact(path string) (*domain.Artifact, error) {
|
||||
data, err := os.ReadFile(path)
|
||||
return readFileArtifactWithLimit(path, 0)
|
||||
}
|
||||
|
||||
func readFileArtifactWithLimit(path string, maxBytes int64) (*domain.Artifact, error) {
|
||||
if maxBytes < 0 {
|
||||
return nil, fmt.Errorf("file size limit must be greater than or equal to 0")
|
||||
}
|
||||
|
||||
file, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read file %s: %w", path, err)
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
info, err := file.Stat()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to stat file %s: %w", path, err)
|
||||
}
|
||||
if maxBytes > 0 && info.Size() > maxBytes {
|
||||
return nil, ErrFileTooLarge
|
||||
}
|
||||
|
||||
var reader io.Reader = file
|
||||
if maxBytes > 0 {
|
||||
reader = io.LimitReader(file, maxBytes+1)
|
||||
}
|
||||
data, err := io.ReadAll(reader)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read file %s: %w", path, err)
|
||||
}
|
||||
if maxBytes > 0 && int64(len(data)) > maxBytes {
|
||||
return nil, ErrFileTooLarge
|
||||
}
|
||||
|
||||
contentType := mime.TypeByExtension(filepath.Ext(path))
|
||||
if contentType == "" {
|
||||
|
||||
@@ -112,6 +112,34 @@ func TestRestrictedCompositeReader(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
func TestRestrictedCompositeReaderFollowsSymlinkInsideRoot(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
root := t.TempDir()
|
||||
outside := t.TempDir()
|
||||
|
||||
target := filepath.Join(outside, "linked.txt")
|
||||
if err := os.WriteFile(target, []byte("linked outside root"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
link := filepath.Join(root, "linked.txt")
|
||||
if err := os.Symlink(target, link); err != nil {
|
||||
t.Skipf("symlink creation unavailable: %v", err)
|
||||
}
|
||||
|
||||
reader, err := NewRestrictedCompositeReader(root)
|
||||
if err != nil {
|
||||
t.Fatalf("expected restricted reader construction, got %v", err)
|
||||
}
|
||||
|
||||
art, err := reader.Read(ctx, domain.ArtifactRef{Type: domain.ArtifactRefFile, URI: "linked.txt"})
|
||||
if err != nil {
|
||||
t.Fatalf("expected symlink inside root to be followed, got %v", err)
|
||||
}
|
||||
if string(art.Body) != "linked outside root" {
|
||||
t.Fatalf("unexpected artifact body: %q", string(art.Body))
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestrictedCompositeReaderWithoutRootDeniesFileRefs(t *testing.T) {
|
||||
reader, err := NewRestrictedCompositeReader("")
|
||||
if err != nil {
|
||||
@@ -132,6 +160,54 @@ func TestRestrictedCompositeReaderWithoutRootDeniesFileRefs(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestrictedCompositeReaderFileSizeLimit(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
root := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(root, "exact.txt"), []byte("12345"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, "large.txt"), []byte("123456"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
reader, err := NewRestrictedCompositeReaderWithLimit(root, 5)
|
||||
if err != nil {
|
||||
t.Fatalf("expected restricted reader construction, got %v", err)
|
||||
}
|
||||
|
||||
art, err := reader.Read(ctx, domain.ArtifactRef{Type: domain.ArtifactRefFile, URI: "exact.txt"})
|
||||
if err != nil {
|
||||
t.Fatalf("expected file at limit to succeed, got %v", err)
|
||||
}
|
||||
if string(art.Body) != "12345" {
|
||||
t.Fatalf("unexpected artifact body: %q", string(art.Body))
|
||||
}
|
||||
|
||||
_, err = reader.Read(ctx, domain.ArtifactRef{Type: domain.ArtifactRefFile, URI: "large.txt"})
|
||||
if !errors.Is(err, ErrFileTooLarge) {
|
||||
t.Fatalf("expected ErrFileTooLarge, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRestrictedCompositeReaderFileSizeLimitZeroDisablesLimit(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(root, "large.txt"), []byte("123456"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
reader, err := NewRestrictedCompositeReaderWithLimit(root, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("expected restricted reader construction, got %v", err)
|
||||
}
|
||||
art, err := reader.Read(context.Background(), domain.ArtifactRef{Type: domain.ArtifactRefFile, URI: "large.txt"})
|
||||
if err != nil {
|
||||
t.Fatalf("expected unlimited reader to succeed, got %v", err)
|
||||
}
|
||||
if string(art.Body) != "123456" {
|
||||
t.Fatalf("unexpected artifact body: %q", string(art.Body))
|
||||
}
|
||||
}
|
||||
|
||||
func TestFileReader_Read(t *testing.T) {
|
||||
content := []byte("test file content")
|
||||
tmpFile, err := os.CreateTemp("", "artifact_test_*.txt")
|
||||
|
||||
@@ -42,6 +42,9 @@ type Config struct {
|
||||
type ServerConfig struct {
|
||||
Addr string `yaml:"addr"`
|
||||
ArtifactRoot string `yaml:"artifact_root"`
|
||||
MaxRequestBytes *int64 `yaml:"max_request_bytes"`
|
||||
MaxArtifactBytes *int64 `yaml:"max_artifact_bytes"`
|
||||
MaxResponseBytes *int64 `yaml:"max_response_bytes"`
|
||||
}
|
||||
|
||||
type DefaultsConfig struct {
|
||||
@@ -55,6 +58,9 @@ type AppSettings struct {
|
||||
SchemaDir string
|
||||
ServerAddr string
|
||||
ArtifactRoot string
|
||||
MaxRequestBytes int64
|
||||
MaxArtifactBytes int64
|
||||
MaxResponseBytes int64
|
||||
DefaultRenderFormat renderformat.PreparedRunOutputFormat
|
||||
}
|
||||
|
||||
@@ -65,6 +71,9 @@ type CLIOverrides struct {
|
||||
SchemaDir string
|
||||
ServerAddr string
|
||||
ArtifactRoot string
|
||||
MaxRequestBytes *int64
|
||||
MaxArtifactBytes *int64
|
||||
MaxResponseBytes *int64
|
||||
RenderFormat string
|
||||
}
|
||||
|
||||
@@ -73,6 +82,9 @@ func BuiltInDefaults() AppSettings {
|
||||
return AppSettings{
|
||||
SchemaDir: defaults.SchemaDirDefault,
|
||||
ServerAddr: defaults.HTTPAddrDefault,
|
||||
MaxRequestBytes: defaults.HTTPMaxRequestBytesDefault,
|
||||
MaxArtifactBytes: defaults.HTTPMaxArtifactBytesDefault,
|
||||
MaxResponseBytes: defaults.HTTPMaxResponseBytesDefault,
|
||||
DefaultRenderFormat: renderformat.DefaultPreparedRunOutputFormat,
|
||||
}
|
||||
}
|
||||
@@ -148,6 +160,24 @@ func ApplyCLIOverrides(base AppSettings, overrides CLIOverrides) (AppSettings, e
|
||||
if v := strings.TrimSpace(overrides.ArtifactRoot); v != "" {
|
||||
out.ArtifactRoot = filepath.Clean(v)
|
||||
}
|
||||
if overrides.MaxRequestBytes != nil {
|
||||
if *overrides.MaxRequestBytes < 0 {
|
||||
return AppSettings{}, fmt.Errorf("%w: server.max_request_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||
}
|
||||
out.MaxRequestBytes = *overrides.MaxRequestBytes
|
||||
}
|
||||
if overrides.MaxArtifactBytes != nil {
|
||||
if *overrides.MaxArtifactBytes < 0 {
|
||||
return AppSettings{}, fmt.Errorf("%w: server.max_artifact_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||
}
|
||||
out.MaxArtifactBytes = *overrides.MaxArtifactBytes
|
||||
}
|
||||
if overrides.MaxResponseBytes != nil {
|
||||
if *overrides.MaxResponseBytes < 0 {
|
||||
return AppSettings{}, fmt.Errorf("%w: server.max_response_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||
}
|
||||
out.MaxResponseBytes = *overrides.MaxResponseBytes
|
||||
}
|
||||
if rawFormat := strings.TrimSpace(overrides.RenderFormat); rawFormat != "" {
|
||||
parsed, err := renderformat.ParsePreparedRunOutputFormat(rawFormat)
|
||||
if err != nil {
|
||||
@@ -190,6 +220,24 @@ func applyConfig(base AppSettings, cfg Config) (AppSettings, error) {
|
||||
if v := strings.TrimSpace(cfg.Server.ArtifactRoot); v != "" {
|
||||
out.ArtifactRoot = filepath.Clean(v)
|
||||
}
|
||||
if cfg.Server.MaxRequestBytes != nil {
|
||||
if *cfg.Server.MaxRequestBytes < 0 {
|
||||
return AppSettings{}, fmt.Errorf("%w: server.max_request_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||
}
|
||||
out.MaxRequestBytes = *cfg.Server.MaxRequestBytes
|
||||
}
|
||||
if cfg.Server.MaxArtifactBytes != nil {
|
||||
if *cfg.Server.MaxArtifactBytes < 0 {
|
||||
return AppSettings{}, fmt.Errorf("%w: server.max_artifact_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||
}
|
||||
out.MaxArtifactBytes = *cfg.Server.MaxArtifactBytes
|
||||
}
|
||||
if cfg.Server.MaxResponseBytes != nil {
|
||||
if *cfg.Server.MaxResponseBytes < 0 {
|
||||
return AppSettings{}, fmt.Errorf("%w: server.max_response_bytes must be greater than or equal to 0", ErrInvalidConfig)
|
||||
}
|
||||
out.MaxResponseBytes = *cfg.Server.MaxResponseBytes
|
||||
}
|
||||
if rawFormat := strings.TrimSpace(cfg.Defaults.RenderFormat); rawFormat != "" {
|
||||
parsed, err := renderformat.ParsePreparedRunOutputFormat(rawFormat)
|
||||
if err != nil {
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
|
||||
renderformat "gitea.maximumdirect.net/eric/scriptorium/internal/format"
|
||||
)
|
||||
|
||||
@@ -24,6 +25,20 @@ func TestLoadConfigMissingImplicitPathUsesBuiltInDefaults(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuiltInDefaultsIncludeHTTPSizeLimits(t *testing.T) {
|
||||
got := BuiltInDefaults()
|
||||
|
||||
if got.MaxRequestBytes != defaults.HTTPMaxRequestBytesDefault {
|
||||
t.Fatalf("unexpected max request bytes: %d", got.MaxRequestBytes)
|
||||
}
|
||||
if got.MaxArtifactBytes != defaults.HTTPMaxArtifactBytesDefault {
|
||||
t.Fatalf("unexpected max artifact bytes: %d", got.MaxArtifactBytes)
|
||||
}
|
||||
if got.MaxResponseBytes != defaults.HTTPMaxResponseBytesDefault {
|
||||
t.Fatalf("unexpected max response bytes: %d", got.MaxResponseBytes)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadConfigMissingExplicitPathReturnsError(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
missing := filepath.Join(tmp, "missing.yml")
|
||||
@@ -93,6 +108,9 @@ schema_dir: ./schemas
|
||||
server:
|
||||
addr: 127.0.0.1:9090
|
||||
artifact_root: ./artifacts
|
||||
max_request_bytes: 1024
|
||||
max_artifact_bytes: 2048
|
||||
max_response_bytes: 4096
|
||||
defaults:
|
||||
render_format: json
|
||||
`)
|
||||
@@ -117,11 +135,58 @@ defaults:
|
||||
if got.ArtifactRoot != filepath.Clean("./artifacts") {
|
||||
t.Fatalf("unexpected server.artifact_root: %q", got.ArtifactRoot)
|
||||
}
|
||||
if got.MaxRequestBytes != 1024 {
|
||||
t.Fatalf("unexpected server.max_request_bytes: %d", got.MaxRequestBytes)
|
||||
}
|
||||
if got.MaxArtifactBytes != 2048 {
|
||||
t.Fatalf("unexpected server.max_artifact_bytes: %d", got.MaxArtifactBytes)
|
||||
}
|
||||
if got.MaxResponseBytes != 4096 {
|
||||
t.Fatalf("unexpected server.max_response_bytes: %d", got.MaxResponseBytes)
|
||||
}
|
||||
if got.DefaultRenderFormat != renderformat.PreparedRunFormatJSON {
|
||||
t.Fatalf("unexpected defaults.render_format: %q", got.DefaultRenderFormat)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadConfigAcceptsZeroHTTPSizeLimits(t *testing.T) {
|
||||
path := writeConfigFile(t, "config.yml", `
|
||||
server:
|
||||
max_request_bytes: 0
|
||||
max_artifact_bytes: 0
|
||||
max_response_bytes: 0
|
||||
`)
|
||||
|
||||
got, err := LoadConfig(path, true)
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if got.MaxRequestBytes != 0 || got.MaxArtifactBytes != 0 || got.MaxResponseBytes != 0 {
|
||||
t.Fatalf("expected zero limits to be preserved, got request=%d artifact=%d response=%d", got.MaxRequestBytes, got.MaxArtifactBytes, got.MaxResponseBytes)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadConfigRejectsNegativeHTTPSizeLimits(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
body string
|
||||
}{
|
||||
{name: "request", body: "server:\n max_request_bytes: -1\n"},
|
||||
{name: "artifact", body: "server:\n max_artifact_bytes: -1\n"},
|
||||
{name: "response", body: "server:\n max_response_bytes: -1\n"},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
path := writeConfigFile(t, "config.yml", tc.body)
|
||||
_, err := LoadConfig(path, true)
|
||||
if !errors.Is(err, ErrInvalidConfig) {
|
||||
t.Fatalf("expected ErrInvalidConfig, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadConfigEmptyFileResolvesToBuiltInDefaults(t *testing.T) {
|
||||
path := writeConfigFile(t, "config.yml", "")
|
||||
|
||||
@@ -190,8 +255,14 @@ func TestApplyCLIOverridesAppliesPrecedence(t *testing.T) {
|
||||
SchemaDir: "/from/config/schemas",
|
||||
ServerAddr: ":1234",
|
||||
ArtifactRoot: "/from/config/artifacts",
|
||||
MaxRequestBytes: 111,
|
||||
MaxArtifactBytes: 222,
|
||||
MaxResponseBytes: 333,
|
||||
DefaultRenderFormat: renderformat.PreparedRunFormatJSON,
|
||||
}
|
||||
maxRequestBytes := int64(0)
|
||||
maxArtifactBytes := int64(444)
|
||||
maxResponseBytes := int64(555)
|
||||
|
||||
got, err := ApplyCLIOverrides(base, CLIOverrides{
|
||||
PromptDir: "./prompts-cli",
|
||||
@@ -199,6 +270,9 @@ func TestApplyCLIOverridesAppliesPrecedence(t *testing.T) {
|
||||
SchemaDir: "./schemas-cli",
|
||||
ServerAddr: ":8081",
|
||||
ArtifactRoot: "./artifacts-cli",
|
||||
MaxRequestBytes: &maxRequestBytes,
|
||||
MaxArtifactBytes: &maxArtifactBytes,
|
||||
MaxResponseBytes: &maxResponseBytes,
|
||||
RenderFormat: "text",
|
||||
})
|
||||
if err != nil {
|
||||
@@ -220,11 +294,42 @@ func TestApplyCLIOverridesAppliesPrecedence(t *testing.T) {
|
||||
if got.ArtifactRoot != filepath.Clean("./artifacts-cli") {
|
||||
t.Fatalf("unexpected artifact root: %q", got.ArtifactRoot)
|
||||
}
|
||||
if got.MaxRequestBytes != 0 {
|
||||
t.Fatalf("unexpected max request bytes: %d", got.MaxRequestBytes)
|
||||
}
|
||||
if got.MaxArtifactBytes != 444 {
|
||||
t.Fatalf("unexpected max artifact bytes: %d", got.MaxArtifactBytes)
|
||||
}
|
||||
if got.MaxResponseBytes != 555 {
|
||||
t.Fatalf("unexpected max response bytes: %d", got.MaxResponseBytes)
|
||||
}
|
||||
if got.DefaultRenderFormat != renderformat.PreparedRunFormatText {
|
||||
t.Fatalf("unexpected render format: %q", got.DefaultRenderFormat)
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyCLIOverridesRejectsNegativeHTTPSizeLimits(t *testing.T) {
|
||||
negative := int64(-1)
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
overrides CLIOverrides
|
||||
}{
|
||||
{name: "request", overrides: CLIOverrides{MaxRequestBytes: &negative}},
|
||||
{name: "artifact", overrides: CLIOverrides{MaxArtifactBytes: &negative}},
|
||||
{name: "response", overrides: CLIOverrides{MaxResponseBytes: &negative}},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
_, err := ApplyCLIOverrides(BuiltInDefaults(), tc.overrides)
|
||||
if !errors.Is(err, ErrInvalidConfig) {
|
||||
t.Fatalf("expected ErrInvalidConfig, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyCLIOverridesInvalidRenderFormatReturnsError(t *testing.T) {
|
||||
_, err := ApplyCLIOverrides(BuiltInDefaults(), CLIOverrides{RenderFormat: "yaml"})
|
||||
if err == nil {
|
||||
|
||||
@@ -14,6 +14,9 @@ const (
|
||||
ContentTypeTextMarkdown = "text/markdown"
|
||||
ContentTypeApplicationJSON = "application/json"
|
||||
OpenAIChatCompletionsPath = "/chat/completions"
|
||||
HTTPMaxRequestBytesDefault = 16 * 1024 * 1024
|
||||
HTTPMaxArtifactBytesDefault = 16 * 1024 * 1024
|
||||
HTTPMaxResponseBytesDefault = 16 * 1024 * 1024
|
||||
|
||||
ExecutionDefaultTemperature = 0.0
|
||||
ExecutionDefaultMaxTokens = 0
|
||||
|
||||
@@ -2,6 +2,7 @@ package filecatalog
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
@@ -93,6 +94,42 @@ func DisplayPath(root string, name string) string {
|
||||
return cleanName
|
||||
}
|
||||
|
||||
// ResolveFSPath resolves userPath from baseDir and keeps it inside root.
|
||||
func ResolveFSPath(root string, baseDir string, userPath string) (string, string, error) {
|
||||
cleanRoot := CleanFSRoot(root)
|
||||
cleanBase := path.Clean(strings.TrimSpace(baseDir))
|
||||
if cleanBase == "" {
|
||||
cleanBase = cleanRoot
|
||||
}
|
||||
if !containsFSPath(cleanRoot, cleanBase) {
|
||||
return "", "", fmt.Errorf("base path %q is outside source root %q", cleanBase, cleanRoot)
|
||||
}
|
||||
|
||||
cleanUserPath := strings.TrimSpace(userPath)
|
||||
if cleanUserPath == "" {
|
||||
return "", "", fmt.Errorf("path is required")
|
||||
}
|
||||
cleanUserPath = path.Clean(cleanUserPath)
|
||||
if path.IsAbs(cleanUserPath) {
|
||||
return "", "", fmt.Errorf("path %q must be relative", userPath)
|
||||
}
|
||||
|
||||
resolved := path.Clean(path.Join(cleanBase, cleanUserPath))
|
||||
if !containsFSPath(cleanRoot, resolved) {
|
||||
return "", "", fmt.Errorf("path %q escapes source root %q", userPath, cleanRoot)
|
||||
}
|
||||
return resolved, DisplayPath(cleanRoot, resolved), nil
|
||||
}
|
||||
|
||||
func containsFSPath(root string, name string) bool {
|
||||
root = CleanFSRoot(root)
|
||||
name = path.Clean(name)
|
||||
if root == "." {
|
||||
return name == "." || (name != ".." && !strings.HasPrefix(name, "../"))
|
||||
}
|
||||
return name == root || strings.HasPrefix(name, strings.TrimSuffix(root, "/")+"/")
|
||||
}
|
||||
|
||||
// Stem strips .yaml or .yml from a file name.
|
||||
func Stem(name string) string {
|
||||
name = strings.TrimSuffix(name, ".yaml")
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
)
|
||||
@@ -131,6 +132,92 @@ func TestDisplayPath(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveFSPath(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
root string
|
||||
baseDir string
|
||||
userPath string
|
||||
wantPath string
|
||||
wantDisplay string
|
||||
wantErr string
|
||||
}{
|
||||
{
|
||||
name: "sibling inside root",
|
||||
root: "prompts",
|
||||
baseDir: "prompts/nested",
|
||||
userPath: "./messages/user.tmpl",
|
||||
wantPath: "prompts/nested/messages/user.tmpl",
|
||||
wantDisplay: "nested/messages/user.tmpl",
|
||||
},
|
||||
{
|
||||
name: "parent inside root",
|
||||
root: "prompts",
|
||||
baseDir: "prompts/nested",
|
||||
userPath: "../shared/user.tmpl",
|
||||
wantPath: "prompts/shared/user.tmpl",
|
||||
wantDisplay: "shared/user.tmpl",
|
||||
},
|
||||
{
|
||||
name: "escape rejected",
|
||||
root: "prompts",
|
||||
baseDir: "prompts/nested",
|
||||
userPath: "../../outside.tmpl",
|
||||
wantErr: "escapes source root",
|
||||
},
|
||||
{
|
||||
name: "absolute path rejected",
|
||||
root: "prompts",
|
||||
baseDir: "prompts/nested",
|
||||
userPath: "/outside.tmpl",
|
||||
wantErr: "must be relative",
|
||||
},
|
||||
{
|
||||
name: "empty path rejected",
|
||||
root: "prompts",
|
||||
baseDir: "prompts/nested",
|
||||
userPath: " ",
|
||||
wantErr: "path is required",
|
||||
},
|
||||
{
|
||||
name: "dot root allows normal relative path",
|
||||
root: ".",
|
||||
baseDir: ".",
|
||||
userPath: "schemas/events.schema.json",
|
||||
wantPath: "schemas/events.schema.json",
|
||||
wantDisplay: "schemas/events.schema.json",
|
||||
},
|
||||
{
|
||||
name: "dot root rejects parent escape",
|
||||
root: ".",
|
||||
baseDir: ".",
|
||||
userPath: "../outside.tmpl",
|
||||
wantErr: "escapes source root",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
gotPath, gotDisplay, err := ResolveFSPath(tc.root, tc.baseDir, tc.userPath)
|
||||
if tc.wantErr != "" {
|
||||
if err == nil {
|
||||
t.Fatalf("expected error containing %q", tc.wantErr)
|
||||
}
|
||||
if !strings.Contains(err.Error(), tc.wantErr) {
|
||||
t.Fatalf("expected error to contain %q, got %v", tc.wantErr, err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if gotPath != tc.wantPath || gotDisplay != tc.wantDisplay {
|
||||
t.Fatalf("expected path/display %q/%q, got %q/%q", tc.wantPath, tc.wantDisplay, gotPath, gotDisplay)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestStemStripsYAMLExtensions(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
|
||||
@@ -139,8 +139,8 @@ func (c *OpenAICompatibleClient) Generate(ctx context.Context, req domain.Genera
|
||||
defer httpResp.Body.Close()
|
||||
|
||||
if httpResp.StatusCode < 200 || httpResp.StatusCode >= 300 {
|
||||
body, _ := io.ReadAll(io.LimitReader(httpResp.Body, 4096))
|
||||
return nil, fmt.Errorf("%w: status=%d body=%q", ErrUnexpectedStatus, httpResp.StatusCode, strings.TrimSpace(string(body)))
|
||||
_, _ = io.Copy(io.Discard, io.LimitReader(httpResp.Body, 4096))
|
||||
return nil, fmt.Errorf("%w: status=%d", ErrUnexpectedStatus, httpResp.StatusCode)
|
||||
}
|
||||
|
||||
var wireResp openAIChatResponse
|
||||
|
||||
@@ -903,9 +903,10 @@ func TestOpenAICompatibleClientEndpointOverride(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientNon2xxError(t *testing.T) {
|
||||
const sensitiveBody = `provider-secret-fragment request_payload_details`
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte(`{"error":"bad request payload"}`))
|
||||
_, _ = w.Write([]byte(`{"error":"` + sensitiveBody + `"}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
@@ -923,8 +924,11 @@ func TestOpenAICompatibleClientNon2xxError(t *testing.T) {
|
||||
if !errors.Is(err, ErrUnexpectedStatus) {
|
||||
t.Fatalf("expected ErrUnexpectedStatus, got %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "400") || !strings.Contains(err.Error(), "bad request payload") {
|
||||
t.Fatalf("expected status/body details, got %v", err)
|
||||
if !strings.Contains(err.Error(), "status=400") {
|
||||
t.Fatalf("expected status detail, got %v", err)
|
||||
}
|
||||
if strings.Contains(err.Error(), sensitiveBody) {
|
||||
t.Fatalf("expected provider response body to be redacted, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -193,6 +193,11 @@ func loadPromptDefinition(ctx context.Context, fsys fs.FS, root string, id strin
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read prompt definition directory: %w", err)
|
||||
}
|
||||
cleanRoot := filecatalog.CleanFSRoot(root)
|
||||
rootInfo, err := fs.Stat(fsys, cleanRoot)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read prompt definition directory: %w", err)
|
||||
}
|
||||
|
||||
var matches []promptDefinitionMatch
|
||||
for _, fullPath := range files {
|
||||
@@ -220,7 +225,7 @@ func loadPromptDefinition(ctx context.Context, fsys fs.FS, root string, id strin
|
||||
continue
|
||||
}
|
||||
|
||||
def, err := normalizePromptDefinitionFromFS(raw, fsys, fullPath)
|
||||
def, err := normalizePromptDefinitionFromFS(raw, fsys, root, fullPath, rootInfo.IsDir())
|
||||
if err != nil {
|
||||
if fileMatch || strings.TrimSpace(raw.ID) == id {
|
||||
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidPromptDefinition, relPath, err)
|
||||
@@ -295,14 +300,23 @@ func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*d
|
||||
})
|
||||
}
|
||||
|
||||
func normalizePromptDefinitionFromFS(raw *promptDefinitionFile, fsys fs.FS, sourcePath string) (*domain.PromptDefinition, error) {
|
||||
func normalizePromptDefinitionFromFS(raw *promptDefinitionFile, fsys fs.FS, root string, sourcePath string, rootIsDir bool) (*domain.PromptDefinition, error) {
|
||||
promptDir := path.Dir(sourcePath)
|
||||
return normalizePromptDefinitionWithContent(raw, func(contentFile string) (string, string, error) {
|
||||
resolvedPath := strings.TrimSpace(contentFile)
|
||||
var resolvedPath string
|
||||
if rootIsDir {
|
||||
var err error
|
||||
resolvedPath, _, err = filecatalog.ResolveFSPath(root, promptDir, contentFile)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
} else {
|
||||
resolvedPath = strings.TrimSpace(contentFile)
|
||||
if !path.IsAbs(resolvedPath) {
|
||||
resolvedPath = path.Join(promptDir, resolvedPath)
|
||||
}
|
||||
resolvedPath = strings.TrimPrefix(path.Clean(resolvedPath), "/")
|
||||
}
|
||||
|
||||
body, err := fs.ReadFile(fsys, resolvedPath)
|
||||
if err != nil {
|
||||
|
||||
@@ -359,6 +359,69 @@ output:
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSRepositoryContentFileContainment(t *testing.T) {
|
||||
t.Run("nested prompt can reference file inside root", func(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"prompts/nested/prompt.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: fs-contained-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content_file: ../shared/user.tmpl
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
"prompts/shared/user.tmpl": &fstest.MapFile{Data: []byte(`Inside root.`)},
|
||||
}, "prompts")
|
||||
|
||||
got, err := repo.GetPromptDefinition(context.Background(), "fs-contained-prompt", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if len(got.Templates) != 1 || got.Templates[0].Content != "Inside root." {
|
||||
t.Fatalf("expected contained content file, got %+v", got.Templates)
|
||||
}
|
||||
})
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
contentFile string
|
||||
wantErr string
|
||||
}{
|
||||
{name: "parent escape rejected", contentFile: "../outside.tmpl", wantErr: "escapes source root"},
|
||||
{name: "absolute path rejected", contentFile: "/outside.tmpl", wantErr: "must be relative"},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"prompts/prompt.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: fs-escaped-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content_file: ` + tc.contentFile + `
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
"outside.tmpl": &fstest.MapFile{Data: []byte(`Outside root.`)},
|
||||
}, "prompts")
|
||||
|
||||
_, err := repo.GetPromptDefinition(context.Background(), "fs-escaped-prompt", "")
|
||||
if !errors.Is(err, ErrInvalidPromptDefinition) {
|
||||
t.Fatalf("expected ErrInvalidPromptDefinition, got %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), tc.wantErr) {
|
||||
t.Fatalf("expected error to contain %q, got %v", tc.wantErr, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSRepositoryRejectsDuplicatePromptIDs(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"one.yaml": &fstest.MapFile{Data: []byte(`
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/filecatalog"
|
||||
"github.com/santhosh-tekuri/jsonschema/v6"
|
||||
)
|
||||
|
||||
@@ -244,17 +245,24 @@ func (v *FSValidator) resolveSchemaPath(schemaPath string) (string, error) {
|
||||
return "", errors.New("schema filesystem is nil")
|
||||
}
|
||||
|
||||
cleanRoot := cleanFSRoot(v.root)
|
||||
cleanRoot := filecatalog.CleanFSRoot(v.root)
|
||||
rootInfo, err := fs.Stat(v.fsys, cleanRoot)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to access schema source %q: %w", cleanRoot, err)
|
||||
}
|
||||
|
||||
cleanSchemaPath := cleanSchemaFSPath(schemaPath)
|
||||
var resolved string
|
||||
if rootInfo.IsDir() {
|
||||
resolved = path.Join(cleanRoot, cleanSchemaPath)
|
||||
resolvedPath, _, err := filecatalog.ResolveFSPath(cleanRoot, cleanRoot, schemaPath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
resolved = resolvedPath
|
||||
} else {
|
||||
cleanSchemaPath, err := cleanSchemaFSPath(schemaPath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if cleanSchemaPath != path.Base(cleanRoot) {
|
||||
return "", fmt.Errorf("schema path %q does not match schema file %q", cleanSchemaPath, path.Base(cleanRoot))
|
||||
}
|
||||
@@ -267,18 +275,16 @@ func (v *FSValidator) resolveSchemaPath(schemaPath string) (string, error) {
|
||||
return resolved, nil
|
||||
}
|
||||
|
||||
func cleanSchemaFSPath(schemaPath string) string {
|
||||
func cleanSchemaFSPath(schemaPath string) (string, error) {
|
||||
cleaned := strings.TrimSpace(schemaPath)
|
||||
cleaned = strings.TrimPrefix(path.Clean(cleaned), "/")
|
||||
return cleaned
|
||||
}
|
||||
|
||||
func cleanFSRoot(root string) string {
|
||||
root = strings.TrimSpace(root)
|
||||
if root == "" || root == "." {
|
||||
return "."
|
||||
if cleaned == "" {
|
||||
return "", errors.New("schema path is required for json_schema validation")
|
||||
}
|
||||
return strings.TrimPrefix(path.Clean(root), "/")
|
||||
cleaned = path.Clean(cleaned)
|
||||
if path.IsAbs(cleaned) {
|
||||
return "", fmt.Errorf("schema path %q must be relative", schemaPath)
|
||||
}
|
||||
return cleaned, nil
|
||||
}
|
||||
|
||||
func fsSchemaResourceURL(schemaName string) string {
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
@@ -276,6 +277,61 @@ func TestFSValidatorJSONSchemaSuccess(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSValidatorJSONSchemaPathContainment(t *testing.T) {
|
||||
t.Run("nested schema inside root succeeds", func(t *testing.T) {
|
||||
v := NewFSValidator(fstest.MapFS{
|
||||
"schemas/nested/events.schema.json": &fstest.MapFile{Data: []byte(`{
|
||||
"type": "object",
|
||||
"required": ["events"],
|
||||
"properties": {
|
||||
"events": {"type": "array"}
|
||||
}
|
||||
}`)},
|
||||
}, "schemas")
|
||||
|
||||
res, err := v.Validate(context.Background(), &domain.Artifact{Body: []byte(`{"events":[]}`)}, domain.OutputContract{
|
||||
ValidationMode: domain.ValidationJSONSchema,
|
||||
SchemaPath: "nested/events.schema.json",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if res.Status != domain.ValidationPassed || !res.IsValid {
|
||||
t.Fatalf("expected passed/valid, got status=%q valid=%v", res.Status, res.IsValid)
|
||||
}
|
||||
})
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
schemaPath string
|
||||
wantErr string
|
||||
}{
|
||||
{name: "parent escape rejected", schemaPath: "../outside.schema.json", wantErr: "escapes source root"},
|
||||
{name: "absolute path rejected", schemaPath: "/outside.schema.json", wantErr: "must be relative"},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
v := NewFSValidator(fstest.MapFS{
|
||||
"schemas/events.schema.json": &fstest.MapFile{Data: []byte(`{"type":"object"}`)},
|
||||
"outside.schema.json": &fstest.MapFile{Data: []byte(`{"type":"object"}`)},
|
||||
"schemas/outside.schema.json": &fstest.MapFile{Data: []byte(`{"type":"object"}`)},
|
||||
}, "schemas")
|
||||
|
||||
_, err := v.Validate(context.Background(), &domain.Artifact{Body: []byte(`{"events":[]}`)}, domain.OutputContract{
|
||||
ValidationMode: domain.ValidationJSONSchema,
|
||||
SchemaPath: tc.schemaPath,
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("expected schema path error")
|
||||
}
|
||||
if !strings.Contains(err.Error(), tc.wantErr) {
|
||||
t.Fatalf("expected error to contain %q, got %v", tc.wantErr, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSValidatorSingleSchemaFileUsesBaseName(t *testing.T) {
|
||||
v := NewFSValidator(fstest.MapFS{
|
||||
"events.schema.json": &fstest.MapFile{Data: []byte(`{
|
||||
|
||||
13
profiles.go
13
profiles.go
@@ -28,10 +28,21 @@ func OpenAICompatibleProfile(cfg OpenAICompatibleProfileConfig) Profile {
|
||||
ServiceTier: cfg.ServiceTier,
|
||||
ReasoningEffort: cfg.ReasoningEffort,
|
||||
APIKeyRequired: cfg.APIKeyRequired,
|
||||
ExtraParams: copyAnyMap(cfg.ExtraParams),
|
||||
ExtraParams: copyShallowAnyMap(cfg.ExtraParams),
|
||||
}
|
||||
}
|
||||
|
||||
func copyShallowAnyMap(src map[string]any) map[string]any {
|
||||
if src == nil {
|
||||
return nil
|
||||
}
|
||||
out := make(map[string]any, len(src))
|
||||
for k, v := range src {
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
type memoryProfileRepository struct {
|
||||
profiles map[string]domain.ExecutionProfile
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user