Document runtime provider parameters
This commit is contained in:
@@ -33,6 +33,7 @@ Integration references:
|
|||||||
- `serve` requires an effective `prompt_dir` and `profile_dir` (from flags or config).
|
- `serve` requires an effective `prompt_dir` and `profile_dir` (from flags or config).
|
||||||
- Positional arguments are rejected.
|
- Positional arguments are rejected.
|
||||||
- Prompt cache control is configured in prompt YAML (`messages[].cache_control`), not with CLI flags.
|
- Prompt cache control is configured in prompt YAML (`messages[].cache_control`), not with CLI flags.
|
||||||
|
- Provider-specific `reasoning_effort` and `extra_params` are configured in profile YAML or HTTP model overrides, not with CLI flags.
|
||||||
|
|
||||||
## Flag Reference
|
## Flag Reference
|
||||||
|
|
||||||
@@ -57,6 +58,11 @@ Integration references:
|
|||||||
- `--top-p <float>`: runtime top-p override.
|
- `--top-p <float>`: runtime top-p override.
|
||||||
- `--timeout <duration>`: runtime timeout override (Go duration syntax, for example `30s`, `2m`).
|
- `--timeout <duration>`: runtime timeout override (Go duration syntax, for example `30s`, `2m`).
|
||||||
|
|
||||||
|
Numeric runtime override flags are presence-aware:
|
||||||
|
|
||||||
|
- omitted numeric flags preserve the selected profile/default value
|
||||||
|
- explicit zero values override the selected profile/default value (`--temperature 0`, `--max-tokens 0`, `--top-p 0`, `--timeout 0s`)
|
||||||
|
|
||||||
### `scriptorium render`
|
### `scriptorium render`
|
||||||
|
|
||||||
- Supports the same flags as `run`, except:
|
- Supports the same flags as `run`, except:
|
||||||
|
|||||||
@@ -189,6 +189,11 @@ top_p: 1.0
|
|||||||
timeout_seconds: 90
|
timeout_seconds: 90
|
||||||
api_key_env: SCRIPTORIUM_API_KEY
|
api_key_env: SCRIPTORIUM_API_KEY
|
||||||
service_tier: priority
|
service_tier: priority
|
||||||
|
reasoning_effort: medium
|
||||||
|
extra_params:
|
||||||
|
provider_route: primary
|
||||||
|
provider_options:
|
||||||
|
retry_budget: 2
|
||||||
```
|
```
|
||||||
|
|
||||||
Field reference:
|
Field reference:
|
||||||
@@ -201,9 +206,9 @@ Field reference:
|
|||||||
- `top_p` (optional): range `0..1`
|
- `top_p` (optional): range `0..1`
|
||||||
- `timeout_seconds` (optional): `>= 0`
|
- `timeout_seconds` (optional): `>= 0`
|
||||||
- `service_tier` (optional): provider-specific request tier such as OpenRouter `flex` or `priority`
|
- `service_tier` (optional): provider-specific request tier such as OpenRouter `flex` or `priority`
|
||||||
- `reasoning_effort` (optional)
|
- `reasoning_effort` (optional): serialized as top-level `reasoning_effort` in outbound chat-completions requests
|
||||||
- `api_key_env` (optional)
|
- `api_key_env` (optional)
|
||||||
- `extra_params` (optional map of strings)
|
- `extra_params` (optional map): JSON-compatible provider-specific parameters. Values may be strings, numbers, booleans, objects, or arrays.
|
||||||
|
|
||||||
Profile rules:
|
Profile rules:
|
||||||
|
|
||||||
@@ -211,13 +216,14 @@ Profile rules:
|
|||||||
- Raw `api_key` is rejected; use `api_key_env`.
|
- Raw `api_key` is rejected; use `api_key_env`.
|
||||||
- If `api_key_env` is set, that environment variable must be set when preparing/running.
|
- If `api_key_env` is set, that environment variable must be set when preparing/running.
|
||||||
- Duplicate profile IDs are invalid. If multiple files declare the requested profile ID, Scriptorium fails instead of choosing one.
|
- Duplicate profile IDs are invalid. If multiple files declare the requested profile ID, Scriptorium fails instead of choosing one.
|
||||||
|
- `extra_params` keys must not be empty and must not collide with reserved outbound request fields: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or `response_format`.
|
||||||
|
|
||||||
Current outbound request behavior:
|
Current outbound request behavior:
|
||||||
|
|
||||||
- The OpenAI-compatible client currently serializes: `model`, optional `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, and optional `response_format` for `json_schema` prompts.
|
- The OpenAI-compatible client currently serializes: `model`, optional `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, optional `response_format` for `json_schema` prompts, and `extra_params`.
|
||||||
|
- `extra_params` are flattened into provider-specific top-level JSON request fields. They are not wrapped in an `extra_params` object on the outbound provider request.
|
||||||
- Messages without `cache_control` serialize with string `content`.
|
- Messages without `cache_control` serialize with string `content`.
|
||||||
- Messages with `cache_control` serialize as a single text content-block array containing `cache_control`.
|
- Messages with `cache_control` serialize as a single text content-block array containing `cache_control`.
|
||||||
- `reasoning_effort` and `extra_params` are parsed and carried in effective settings, but are not currently serialized into outbound chat-completions requests.
|
|
||||||
|
|
||||||
## Schema Behavior
|
## Schema Behavior
|
||||||
|
|
||||||
|
|||||||
@@ -50,7 +50,10 @@ Copyable request example file:
|
|||||||
"reasoning_effort": "medium",
|
"reasoning_effort": "medium",
|
||||||
"api_key_env": "SCRIPTORIUM_API_KEY",
|
"api_key_env": "SCRIPTORIUM_API_KEY",
|
||||||
"extra_params": {
|
"extra_params": {
|
||||||
"route": "primary"
|
"route": "primary",
|
||||||
|
"provider_options": {
|
||||||
|
"retry_budget": 2
|
||||||
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"include_raw_output": false
|
"include_raw_output": false
|
||||||
@@ -67,6 +70,14 @@ Input reference types currently supported by runtime artifact loading:
|
|||||||
- `file`
|
- `file`
|
||||||
- `inline`
|
- `inline`
|
||||||
|
|
||||||
|
Model override notes:
|
||||||
|
|
||||||
|
- Numeric model override fields distinguish omitted values from explicit zero values. For example, omitting `temperature` preserves the selected profile/default value, while `"temperature": 0` explicitly sets the effective temperature to zero.
|
||||||
|
- `extra_params` accepts JSON-compatible values: strings, numbers, booleans, objects, and arrays.
|
||||||
|
- `extra_params` are passed through effective model metadata and flattened into top-level provider request fields by the OpenAI-compatible client.
|
||||||
|
- `extra_params` keys must not be empty and must not collide with reserved outbound fields: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or `response_format`.
|
||||||
|
- Raw API-key values are not accepted. Use `api_key_env` to name an environment variable.
|
||||||
|
|
||||||
## Strict JSON Rules
|
## Strict JSON Rules
|
||||||
|
|
||||||
Request decoding uses strict JSON field checks:
|
Request decoding uses strict JSON field checks:
|
||||||
@@ -119,7 +130,10 @@ Response shape:
|
|||||||
"reasoning_effort": "medium",
|
"reasoning_effort": "medium",
|
||||||
"api_key_env": "SCRIPTORIUM_API_KEY",
|
"api_key_env": "SCRIPTORIUM_API_KEY",
|
||||||
"extra_params": {
|
"extra_params": {
|
||||||
"route": "primary"
|
"route": "primary",
|
||||||
|
"provider_options": {
|
||||||
|
"retry_budget": 2
|
||||||
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"input_hashes": {
|
"input_hashes": {
|
||||||
|
|||||||
@@ -32,10 +32,48 @@ Serialized JSON fields:
|
|||||||
- `max_tokens` (only when non-zero)
|
- `max_tokens` (only when non-zero)
|
||||||
- `top_p` (only when non-zero)
|
- `top_p` (only when non-zero)
|
||||||
- `service_tier` (only when non-empty)
|
- `service_tier` (only when non-empty)
|
||||||
|
- `reasoning_effort` (only when non-empty)
|
||||||
- `response_format` (only when structured output is provided)
|
- `response_format` (only when structured output is provided)
|
||||||
|
- profile/request `extra_params` as additional provider-specific top-level fields
|
||||||
|
|
||||||
`service_tier` is provider-specific. OpenRouter currently documents request values such as `flex` and `priority`; Scriptorium forwards any non-empty configured value and lets the backend validate support.
|
`service_tier` is provider-specific. OpenRouter currently documents request values such as `flex` and `priority`; Scriptorium forwards any non-empty configured value and lets the backend validate support.
|
||||||
|
|
||||||
|
`reasoning_effort` is provider-specific. Scriptorium forwards any non-empty configured value as top-level `reasoning_effort` and lets the backend validate support.
|
||||||
|
|
||||||
|
`extra_params` are flattened into the outbound JSON object. They are not wrapped in an `extra_params` object:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"model": "gpt-4o-mini",
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": "rendered text"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"provider_route": "primary",
|
||||||
|
"provider_options": {
|
||||||
|
"retry_budget": 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`extra_params` values must be JSON-compatible. Supported value shapes include strings, numbers, booleans, objects, and arrays.
|
||||||
|
|
||||||
|
Reserved `extra_params` keys are rejected before the HTTP request is made:
|
||||||
|
|
||||||
|
- `model`
|
||||||
|
- `session_id`
|
||||||
|
- `messages`
|
||||||
|
- `temperature`
|
||||||
|
- `max_tokens`
|
||||||
|
- `top_p`
|
||||||
|
- `service_tier`
|
||||||
|
- `reasoning_effort`
|
||||||
|
- `response_format`
|
||||||
|
|
||||||
|
Empty `extra_params` keys and values that cannot be encoded as JSON are also rejected before the HTTP request is made.
|
||||||
|
|
||||||
`session_id` is rendered from prompt YAML using request variables and serialized as a top-level JSON request field. Scriptorium does not send an `x-session-id` header. Empty rendered session IDs are omitted, and values longer than 256 characters are rejected before the HTTP request.
|
`session_id` is rendered from prompt YAML using request variables and serialized as a top-level JSON request field. Scriptorium does not send an `x-session-id` header. Empty rendered session IDs are omitted, and values longer than 256 characters are rejected before the HTTP request.
|
||||||
|
|
||||||
Messages without prompt cache control serialize with string `content`:
|
Messages without prompt cache control serialize with string `content`:
|
||||||
@@ -138,12 +176,7 @@ Malformed responses return `ErrMalformedResponse`.
|
|||||||
|
|
||||||
## Unsupported Or Non-Serialized Fields
|
## Unsupported Or Non-Serialized Fields
|
||||||
|
|
||||||
The following fields may exist in profile/effective settings but are not currently serialized into outbound chat-completions payloads:
|
The client does not serialize top-level `cache_control`.
|
||||||
|
|
||||||
- `reasoning_effort`
|
|
||||||
- `extra_params`
|
|
||||||
|
|
||||||
The client also does not serialize top-level `cache_control`.
|
|
||||||
|
|
||||||
No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.
|
No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.
|
||||||
|
|
||||||
|
|||||||
@@ -69,6 +69,8 @@ Primary app settings consumed by adapters:
|
|||||||
Execution profile/request settings used through runner:
|
Execution profile/request settings used through runner:
|
||||||
|
|
||||||
- `endpoint`, `model`, `temperature`, `max_tokens`, `top_p`, `timeout_seconds`, `service_tier`, `api_key_env`, `reasoning_effort`, `extra_params`
|
- `endpoint`, `model`, `temperature`, `max_tokens`, `top_p`, `timeout_seconds`, `service_tier`, `api_key_env`, `reasoning_effort`, `extra_params`
|
||||||
|
- CLI and HTTP request adapters preserve caller intent for numeric runtime overrides. Omitted values remain absent; explicit zero values are mapped as explicit overrides.
|
||||||
|
- HTTP `extra_params` accepts JSON-compatible values and maps them to domain request overrides without provider-specific adapter logic.
|
||||||
|
|
||||||
## External Dependencies
|
## External Dependencies
|
||||||
|
|
||||||
@@ -97,6 +99,10 @@ LLM adapter:
|
|||||||
- endpoint appends `/chat/completions`.
|
- endpoint appends `/chat/completions`.
|
||||||
- rendered messages without cache control serialize with string `content`.
|
- rendered messages without cache control serialize with string `content`.
|
||||||
- rendered messages with cache control serialize as one text content block with `cache_control`.
|
- rendered messages with cache control serialize as one text content block with `cache_control`.
|
||||||
|
- non-empty `reasoning_effort` serializes as a top-level provider request field.
|
||||||
|
- `extra_params` flatten into provider-specific top-level JSON request fields.
|
||||||
|
- reserved `extra_params` keys are rejected before the provider call: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, and `response_format`.
|
||||||
|
- empty `extra_params` keys and values that cannot be JSON-encoded are rejected before the provider call.
|
||||||
- compatible cache usage response fields are parsed into domain token usage.
|
- compatible cache usage response fields are parsed into domain token usage.
|
||||||
- non-2xx responses map to request failure errors.
|
- non-2xx responses map to request failure errors.
|
||||||
- malformed responses (including missing/empty first choice content) are errors.
|
- malformed responses (including missing/empty first choice content) are errors.
|
||||||
@@ -146,5 +152,5 @@ Behavior highlights:
|
|||||||
- Adapter packages do not own runner decision logic.
|
- Adapter packages do not own runner decision logic.
|
||||||
- External request/response strictness is part of contract stability.
|
- External request/response strictness is part of contract stability.
|
||||||
- Prepared-render output never includes resolved API key values.
|
- Prepared-render output never includes resolved API key values.
|
||||||
- Outbound OpenAI-compatible request includes only currently serialized fields (`model`, optional `session_id`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `service_tier`, optional `response_format`).
|
- Outbound OpenAI-compatible request includes currently serialized first-class fields (`model`, optional `session_id`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `service_tier`, optional `reasoning_effort`, optional `response_format`) plus validated `extra_params` flattened as provider-specific top-level fields.
|
||||||
- Outbound cache control is message-level only; no top-level cache-control field is serialized.
|
- Outbound cache control is message-level only; no top-level cache-control field is serialized.
|
||||||
|
|||||||
@@ -103,6 +103,7 @@ Validation content failures are not run errors:
|
|||||||
- built-in execution defaults
|
- built-in execution defaults
|
||||||
- selected profile values
|
- selected profile values
|
||||||
- request overrides
|
- request overrides
|
||||||
|
- request numeric overrides are presence-aware, so omitted values preserve the current effective value and explicit zero values override it
|
||||||
6. verify required `api_key_env` environment variable:
|
6. verify required `api_key_env` environment variable:
|
||||||
- missing/empty env value returns an invalid request with `ErrAPIKeyEnvMissing`
|
- missing/empty env value returns an invalid request with `ErrAPIKeyEnvMissing`
|
||||||
- only the environment-variable name is retained; secret value is never returned
|
- only the environment-variable name is retained; secret value is never returned
|
||||||
@@ -115,6 +116,14 @@ Validation content failures are not run errors:
|
|||||||
|
|
||||||
`Prepare` does not call the LLM.
|
`Prepare` does not call the LLM.
|
||||||
|
|
||||||
|
Runtime target notes:
|
||||||
|
|
||||||
|
- Profile `extra_params` and request `extra_params` carry JSON-compatible values through prepared output, run metadata, and `domain.GenerateRequest.Target`.
|
||||||
|
- The OpenAI-compatible client serializes non-empty `reasoning_effort` as a top-level provider request field.
|
||||||
|
- The OpenAI-compatible client flattens `extra_params` into provider-specific top-level JSON request fields.
|
||||||
|
- Empty `extra_params` keys, reserved outbound field names, and values that cannot be JSON-encoded fail before the provider request.
|
||||||
|
- Resolved API-key values are never stored in `PreparedRun`, `RunResult`, logs, or HTTP responses.
|
||||||
|
|
||||||
## Run Flow
|
## Run Flow
|
||||||
|
|
||||||
`Run` performs:
|
`Run` performs:
|
||||||
|
|||||||
Reference in New Issue
Block a user