Compare commits
9 Commits
4d4bb7a121
...
v0.10.0
| Author | SHA1 | Date | |
|---|---|---|---|
| 23872dd742 | |||
| 7ffbf5f6ca | |||
| d0dc30fcc9 | |||
| b38f7b4dc3 | |||
| 0512995931 | |||
| 049a5feadb | |||
| 1798e9c575 | |||
| 5d4bc8c2b9 | |||
| 63fb8fc132 |
@@ -28,10 +28,6 @@ steps:
|
||||
|
||||
build_binary linux amd64 ""
|
||||
build_binary linux arm64 ""
|
||||
build_binary darwin amd64 ""
|
||||
build_binary darwin arm64 ""
|
||||
build_binary windows amd64 ".exe"
|
||||
build_binary windows arm64 ".exe"
|
||||
|
||||
- name: publish-release
|
||||
image: woodpeckerci/plugin-release
|
||||
|
||||
@@ -33,6 +33,7 @@ Integration references:
|
||||
- `serve` requires an effective `prompt_dir` and `profile_dir` (from flags or config).
|
||||
- Positional arguments are rejected.
|
||||
- Prompt cache control is configured in prompt YAML (`messages[].cache_control`), not with CLI flags.
|
||||
- Provider-specific `reasoning_effort` and `extra_params` are configured in profile YAML or HTTP model overrides, not with CLI flags.
|
||||
|
||||
## Flag Reference
|
||||
|
||||
@@ -57,6 +58,11 @@ Integration references:
|
||||
- `--top-p <float>`: runtime top-p override.
|
||||
- `--timeout <duration>`: runtime timeout override (Go duration syntax, for example `30s`, `2m`).
|
||||
|
||||
Numeric runtime override flags are presence-aware:
|
||||
|
||||
- omitted numeric flags preserve the selected profile/default value
|
||||
- explicit zero values override the selected profile/default value (`--temperature 0`, `--max-tokens 0`, `--top-p 0`, `--timeout 0s`)
|
||||
|
||||
### `scriptorium render`
|
||||
|
||||
- Supports the same flags as `run`, except:
|
||||
@@ -83,6 +89,7 @@ Notes:
|
||||
|
||||
- `--input name=path` maps prompt input names to local file paths.
|
||||
- `--var name=value` maps template variable names to values.
|
||||
- If a prompt defines `session_id: "{{ .session_id }}"`, pass the OpenRouter sticky-routing value with `--var session_id=<value>`.
|
||||
- Both flags can be repeated.
|
||||
- Both flags also support comma-separated batches, for example:
|
||||
- `--input transcript=./t.md,glossary=./g.yml`
|
||||
|
||||
@@ -104,6 +104,7 @@ Field reference:
|
||||
- `version` (required): prompt version.
|
||||
- `default_profile` (optional): profile ID used when request does not provide `profile_id`.
|
||||
- `description` (optional): prompt description.
|
||||
- `session_id` (optional): Go-template string for OpenRouter sticky-routing `session_id`; rendered from request vars.
|
||||
- `inputs` (optional list): expected named inputs.
|
||||
- `messages` (required list): prompt message templates.
|
||||
- `output` (required object): output contract.
|
||||
@@ -150,6 +151,14 @@ messages:
|
||||
|
||||
Use cache control on stable reusable prompt content. Dynamic per-run inputs before the cache-controlled message change the provider cache key.
|
||||
|
||||
Example prompt-level session ID:
|
||||
|
||||
```yaml
|
||||
session_id: "{{ .session_id }}"
|
||||
```
|
||||
|
||||
When configured, `session_id` is rendered with the same variable context as messages. The rendered value is trimmed, omitted when empty, and rejected if longer than 256 characters. CLI callers pass the value through `--var session_id=<value>`; HTTP callers pass it through `"vars": {"session_id": "<value>"}`.
|
||||
|
||||
`output` fields:
|
||||
|
||||
- `format` (required): `text`, `markdown`, or `json`.
|
||||
@@ -180,6 +189,11 @@ top_p: 1.0
|
||||
timeout_seconds: 90
|
||||
api_key_env: SCRIPTORIUM_API_KEY
|
||||
service_tier: priority
|
||||
reasoning_effort: medium
|
||||
extra_params:
|
||||
provider_route: primary
|
||||
provider_options:
|
||||
retry_budget: 2
|
||||
```
|
||||
|
||||
Field reference:
|
||||
@@ -192,9 +206,9 @@ Field reference:
|
||||
- `top_p` (optional): range `0..1`
|
||||
- `timeout_seconds` (optional): `>= 0`
|
||||
- `service_tier` (optional): provider-specific request tier such as OpenRouter `flex` or `priority`
|
||||
- `reasoning_effort` (optional)
|
||||
- `reasoning_effort` (optional): serialized as top-level `reasoning_effort` in outbound chat-completions requests
|
||||
- `api_key_env` (optional)
|
||||
- `extra_params` (optional map of strings)
|
||||
- `extra_params` (optional map): JSON-compatible provider-specific parameters. Values may be strings, numbers, booleans, objects, or arrays.
|
||||
|
||||
Profile rules:
|
||||
|
||||
@@ -202,13 +216,14 @@ Profile rules:
|
||||
- Raw `api_key` is rejected; use `api_key_env`.
|
||||
- If `api_key_env` is set, that environment variable must be set when preparing/running.
|
||||
- Duplicate profile IDs are invalid. If multiple files declare the requested profile ID, Scriptorium fails instead of choosing one.
|
||||
- `extra_params` keys must not be empty and must not collide with reserved outbound request fields: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or `response_format`.
|
||||
|
||||
Current outbound request behavior:
|
||||
|
||||
- The OpenAI-compatible client currently serializes: `model`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, and optional `response_format` for `json_schema` prompts.
|
||||
- The OpenAI-compatible client currently serializes: `model`, optional `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, optional `response_format` for `json_schema` prompts, and `extra_params`.
|
||||
- `extra_params` are flattened into provider-specific top-level JSON request fields. They are not wrapped in an `extra_params` object on the outbound provider request.
|
||||
- Messages without `cache_control` serialize with string `content`.
|
||||
- Messages with `cache_control` serialize as a single text content-block array containing `cache_control`.
|
||||
- `reasoning_effort` and `extra_params` are parsed and carried in effective settings, but are not currently serialized into outbound chat-completions requests.
|
||||
|
||||
## Schema Behavior
|
||||
|
||||
|
||||
@@ -50,7 +50,10 @@ Copyable request example file:
|
||||
"reasoning_effort": "medium",
|
||||
"api_key_env": "SCRIPTORIUM_API_KEY",
|
||||
"extra_params": {
|
||||
"route": "primary"
|
||||
"route": "primary",
|
||||
"provider_options": {
|
||||
"retry_budget": 2
|
||||
}
|
||||
}
|
||||
},
|
||||
"include_raw_output": false
|
||||
@@ -67,6 +70,14 @@ Input reference types currently supported by runtime artifact loading:
|
||||
- `file`
|
||||
- `inline`
|
||||
|
||||
Model override notes:
|
||||
|
||||
- Numeric model override fields distinguish omitted values from explicit zero values. For example, omitting `temperature` preserves the selected profile/default value, while `"temperature": 0` explicitly sets the effective temperature to zero.
|
||||
- `extra_params` accepts JSON-compatible values: strings, numbers, booleans, objects, and arrays.
|
||||
- `extra_params` are passed through effective model metadata and flattened into top-level provider request fields by the OpenAI-compatible client.
|
||||
- `extra_params` keys must not be empty and must not collide with reserved outbound fields: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or `response_format`.
|
||||
- Raw API-key values are not accepted. Use `api_key_env` to name an environment variable.
|
||||
|
||||
## Strict JSON Rules
|
||||
|
||||
Request decoding uses strict JSON field checks:
|
||||
@@ -119,7 +130,10 @@ Response shape:
|
||||
"reasoning_effort": "medium",
|
||||
"api_key_env": "SCRIPTORIUM_API_KEY",
|
||||
"extra_params": {
|
||||
"route": "primary"
|
||||
"route": "primary",
|
||||
"provider_options": {
|
||||
"retry_budget": 2
|
||||
}
|
||||
}
|
||||
},
|
||||
"input_hashes": {
|
||||
|
||||
@@ -26,15 +26,56 @@ Example:
|
||||
Serialized JSON fields:
|
||||
|
||||
- `model` (required after fallback resolution)
|
||||
- `session_id` (only when the rendered prompt includes a non-empty session ID)
|
||||
- `messages` (rendered prompt messages)
|
||||
- `temperature` (only when non-zero)
|
||||
- `max_tokens` (only when non-zero)
|
||||
- `top_p` (only when non-zero)
|
||||
- `temperature` (when non-zero, or when explicitly overridden to zero)
|
||||
- `max_tokens` (when non-zero, or when explicitly overridden to zero)
|
||||
- `top_p` (when non-zero, or when explicitly overridden to zero)
|
||||
- `service_tier` (only when non-empty)
|
||||
- `reasoning_effort` (only when non-empty)
|
||||
- `response_format` (only when structured output is provided)
|
||||
- profile/request `extra_params` as additional provider-specific top-level fields
|
||||
|
||||
`service_tier` is provider-specific. OpenRouter currently documents request values such as `flex` and `priority`; Scriptorium forwards any non-empty configured value and lets the backend validate support.
|
||||
|
||||
`reasoning_effort` is provider-specific. Scriptorium forwards any non-empty configured value as top-level `reasoning_effort` and lets the backend validate support.
|
||||
|
||||
`extra_params` are flattened into the outbound JSON object. They are not wrapped in an `extra_params` object:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "gpt-4o-mini",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "rendered text"
|
||||
}
|
||||
],
|
||||
"provider_route": "primary",
|
||||
"provider_options": {
|
||||
"retry_budget": 2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`extra_params` values must be JSON-compatible. Supported value shapes include strings, numbers, booleans, objects, and arrays.
|
||||
|
||||
Reserved `extra_params` keys are rejected before the HTTP request is made:
|
||||
|
||||
- `model`
|
||||
- `session_id`
|
||||
- `messages`
|
||||
- `temperature`
|
||||
- `max_tokens`
|
||||
- `top_p`
|
||||
- `service_tier`
|
||||
- `reasoning_effort`
|
||||
- `response_format`
|
||||
|
||||
Empty `extra_params` keys and values that cannot be encoded as JSON are also rejected before the HTTP request is made.
|
||||
|
||||
`session_id` is rendered from prompt YAML using request variables and serialized as a top-level JSON request field. Scriptorium does not send an `x-session-id` header. Empty rendered session IDs are omitted, and values longer than 256 characters are rejected before the HTTP request.
|
||||
|
||||
Messages without prompt cache control serialize with string `content`:
|
||||
|
||||
```json
|
||||
@@ -101,6 +142,7 @@ Base timeout comes from client configuration.
|
||||
Per-request override:
|
||||
|
||||
- if `Target.TimeoutSeconds > 0`, use that value for request timeout
|
||||
- if `Target.TimeoutSeconds == 0` and the value came from an explicit request override, disable the HTTP client timeout
|
||||
- if `Target.TimeoutSeconds < 0`, request is rejected (`ErrInvalidRequest`)
|
||||
|
||||
## Response Expectations
|
||||
@@ -135,12 +177,7 @@ Malformed responses return `ErrMalformedResponse`.
|
||||
|
||||
## Unsupported Or Non-Serialized Fields
|
||||
|
||||
The following fields may exist in profile/effective settings but are not currently serialized into outbound chat-completions payloads:
|
||||
|
||||
- `reasoning_effort`
|
||||
- `extra_params`
|
||||
|
||||
The client also does not serialize top-level `cache_control` or `session_id`.
|
||||
The client does not serialize top-level `cache_control`.
|
||||
|
||||
No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.
|
||||
|
||||
|
||||
@@ -69,6 +69,8 @@ Primary app settings consumed by adapters:
|
||||
Execution profile/request settings used through runner:
|
||||
|
||||
- `endpoint`, `model`, `temperature`, `max_tokens`, `top_p`, `timeout_seconds`, `service_tier`, `api_key_env`, `reasoning_effort`, `extra_params`
|
||||
- CLI and HTTP request adapters preserve caller intent for numeric runtime overrides. Omitted values remain absent; explicit zero values are mapped as explicit overrides.
|
||||
- HTTP `extra_params` accepts JSON-compatible values and maps them to domain request overrides without provider-specific adapter logic.
|
||||
|
||||
## External Dependencies
|
||||
|
||||
@@ -97,6 +99,10 @@ LLM adapter:
|
||||
- endpoint appends `/chat/completions`.
|
||||
- rendered messages without cache control serialize with string `content`.
|
||||
- rendered messages with cache control serialize as one text content block with `cache_control`.
|
||||
- non-empty `reasoning_effort` serializes as a top-level provider request field.
|
||||
- `extra_params` flatten into provider-specific top-level JSON request fields.
|
||||
- reserved `extra_params` keys are rejected before the provider call: `model`, `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, and `response_format`.
|
||||
- empty `extra_params` keys and values that cannot be JSON-encoded are rejected before the provider call.
|
||||
- compatible cache usage response fields are parsed into domain token usage.
|
||||
- non-2xx responses map to request failure errors.
|
||||
- malformed responses (including missing/empty first choice content) are errors.
|
||||
@@ -146,5 +152,5 @@ Behavior highlights:
|
||||
- Adapter packages do not own runner decision logic.
|
||||
- External request/response strictness is part of contract stability.
|
||||
- Prepared-render output never includes resolved API key values.
|
||||
- Outbound OpenAI-compatible request includes only currently serialized fields (`model`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `service_tier`, optional `response_format`).
|
||||
- Outbound cache control is message-level only; no top-level cache-control/session fields are serialized.
|
||||
- Outbound OpenAI-compatible request includes currently serialized first-class fields (`model`, optional `session_id`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `service_tier`, optional `reasoning_effort`, optional `response_format`) plus validated `extra_params` flattened as provider-specific top-level fields.
|
||||
- Outbound cache control is message-level only; no top-level cache-control field is serialized.
|
||||
|
||||
@@ -103,6 +103,7 @@ Validation content failures are not run errors:
|
||||
- built-in execution defaults
|
||||
- selected profile values
|
||||
- request overrides
|
||||
- request numeric overrides are presence-aware, so omitted values preserve the current effective value and explicit zero values override it
|
||||
6. verify required `api_key_env` environment variable:
|
||||
- missing/empty env value returns an invalid request with `ErrAPIKeyEnvMissing`
|
||||
- only the environment-variable name is retained; secret value is never returned
|
||||
@@ -115,6 +116,14 @@ Validation content failures are not run errors:
|
||||
|
||||
`Prepare` does not call the LLM.
|
||||
|
||||
Runtime target notes:
|
||||
|
||||
- Profile `extra_params` and request `extra_params` carry JSON-compatible values through prepared output, run metadata, and `domain.GenerateRequest.Target`.
|
||||
- The OpenAI-compatible client serializes non-empty `reasoning_effort` as a top-level provider request field.
|
||||
- The OpenAI-compatible client flattens `extra_params` into provider-specific top-level JSON request fields.
|
||||
- Empty `extra_params` keys, reserved outbound field names, and values that cannot be JSON-encoded fail before the provider request.
|
||||
- Resolved API-key values are never stored in `PreparedRun`, `RunResult`, logs, or HTTP responses.
|
||||
|
||||
## Run Flow
|
||||
|
||||
`Run` performs:
|
||||
|
||||
@@ -1,136 +0,0 @@
|
||||
# Prompt Cache Control Roadmap
|
||||
|
||||
## Purpose
|
||||
|
||||
Scriptorium should support provider prompt-cache controls for OpenAI-compatible gateways that expose Anthropic-style cache breakpoints, especially OpenRouter.
|
||||
|
||||
The feature should preserve existing prompt definitions. Prompt authors should opt in with optional message-level cache metadata, and Scriptorium should report cache usage when compatible providers return it.
|
||||
|
||||
Implementation steps belong in `docs/roadmap/implementation.md`.
|
||||
|
||||
## Target State
|
||||
|
||||
Prompt authors can mark a rendered message as an explicit cache breakpoint:
|
||||
|
||||
```yaml
|
||||
messages:
|
||||
- role: system
|
||||
content_file: ../common/transcript.system.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
ttl: 1h
|
||||
```
|
||||
|
||||
Existing messages without `cache_control` continue to render and serialize as string content:
|
||||
|
||||
```json
|
||||
{
|
||||
"role": "system",
|
||||
"content": "rendered text"
|
||||
}
|
||||
```
|
||||
|
||||
Messages with `cache_control` serialize as a single text content block:
|
||||
|
||||
```json
|
||||
{
|
||||
"role": "system",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "rendered text",
|
||||
"cache_control": {
|
||||
"type": "ephemeral",
|
||||
"ttl": "1h"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Scriptorium should parse and expose provider cache usage when compatible response fields are present, including cached prompt tokens and cache-write tokens.
|
||||
|
||||
## Prompt Authoring Policy
|
||||
|
||||
Cache breakpoints should be used for stable reusable prompt prefixes.
|
||||
|
||||
Recommended ordering:
|
||||
|
||||
1. Put stable, reusable context first.
|
||||
2. Put `cache_control` on the last stable message that should be part of the reusable prefix.
|
||||
3. Put per-run dynamic inputs after that breakpoint.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
messages:
|
||||
- role: system
|
||||
content_file: ../common/transcript.system.md
|
||||
- role: user
|
||||
content_file: ./character_meta_analysis.task.md
|
||||
- role: user
|
||||
content_file: ./character_meta_analysis.instructions.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
ttl: 1h
|
||||
- role: user
|
||||
content: |
|
||||
<<<PREVIOUS_SESSION_RECAP
|
||||
{{input "recap"}}
|
||||
PREVIOUS_SESSION_RECAP>>>
|
||||
- role: user
|
||||
content: |
|
||||
<<<CURRENT_SESSION_TRANSCRIPT
|
||||
{{input "transcript"}}
|
||||
CURRENT_SESSION_TRANSCRIPT>>>
|
||||
```
|
||||
|
||||
## Supported Cache-Control Shape
|
||||
|
||||
Initial support is message-level only:
|
||||
|
||||
```yaml
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
ttl: 1h
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `cache_control` is optional on each message.
|
||||
- `cache_control.type` is required when `cache_control` is present.
|
||||
- The only supported `type` value is `ephemeral`.
|
||||
- `ttl` is optional.
|
||||
- When set, the only supported `ttl` value is `1h`.
|
||||
- Empty `ttl` is omitted from the outbound payload.
|
||||
- Prompt decoding remains strict; unknown cache-control fields are rejected.
|
||||
- Existing `content` and `content_file` rules remain unchanged.
|
||||
|
||||
## Compatibility
|
||||
|
||||
This feature should be backward compatible for existing prompt definitions.
|
||||
|
||||
Compatibility requirements:
|
||||
|
||||
- Existing prompt YAML without `cache_control` loads unchanged.
|
||||
- Existing `render` output remains valid.
|
||||
- Existing outbound request payloads remain string-content messages unless `cache_control` is configured.
|
||||
- Existing integrations do not need to send new request fields.
|
||||
- CLI and HTTP callers do not need new request options for the initial feature.
|
||||
|
||||
The only intentional prompt-definition contract change is the new optional `messages[].cache_control` object.
|
||||
|
||||
## Deferred Work
|
||||
|
||||
These are intentionally out of scope for the initial feature:
|
||||
|
||||
- `content_blocks` prompt syntax.
|
||||
- Multiple text blocks inside a single message.
|
||||
- Image, tool, or non-text content blocks.
|
||||
- Provider-specific automatic prompt caching toggles.
|
||||
- Top-level OpenRouter `cache_control`.
|
||||
- Top-level OpenRouter `session_id`.
|
||||
- General-purpose serialization of `extra_params`.
|
||||
- Provider-specific validation profiles for cache-control limits.
|
||||
|
||||
These can be added later without changing the message-level cache-control contract.
|
||||
@@ -1,358 +1,248 @@
|
||||
# Prompt Cache Control Implementation Plan
|
||||
# Runtime Parameter Implementation Plan
|
||||
|
||||
## Scope
|
||||
This plan implements the target state in `docs/roadmap/params.md`.
|
||||
|
||||
This plan implements the feature defined in `docs/roadmap/cache.md`.
|
||||
Audience: LLM coding agents implementing the feature in order. Follow `docs/policy/architecture.md`, `docs/policy/development.md`, and `docs/policy/documentation.md` before changing code.
|
||||
|
||||
The implementation target is message-level Anthropic/OpenRouter-style cache control for rendered prompt messages, plus cache usage observability. Do not implement deferred roadmap items such as `content_blocks`, top-level `cache_control`, `session_id`, or general `extra_params` serialization.
|
||||
## Constraints
|
||||
|
||||
Follow the policy documents under `docs/policy/` while implementing:
|
||||
- Keep adapters thin. CLI and HTTP should capture caller intent and map it into domain request types; merge decisions belong in `internal/usecase`.
|
||||
- Keep external decoding strict. Unknown YAML/JSON fields must continue to fail.
|
||||
- Do not accept or emit raw API key values.
|
||||
- Do not add dependencies unless there is a clear need. This feature should use the standard library plus existing dependencies.
|
||||
- Do not expand the HTTP API surface beyond `POST /v1/runs`.
|
||||
- Do not add provider-specific adapter packages.
|
||||
- Keep each stage passing `go test ./...` before moving to the next stage.
|
||||
|
||||
- Keep prompt/profile/HTTP/config decoding strict.
|
||||
- Keep adapters thin.
|
||||
- Keep provider request serialization in `internal/llm`.
|
||||
- Keep prompt loading in `internal/promptdef`.
|
||||
- Update canonical non-roadmap docs only when the behavior is implemented.
|
||||
- Do not accept or emit raw secret values.
|
||||
## Stage 1: Presence-Aware Request Overrides
|
||||
|
||||
## Stage 1: Domain Types And Prompt Loader
|
||||
Goal: make per-request numeric execution overrides presence-aware while keeping resolved execution settings concrete.
|
||||
|
||||
Objective: make prompt YAML accept and validate optional `messages[].cache_control`, and carry normalized cache-control metadata in domain prompt definitions.
|
||||
### Domain Changes
|
||||
|
||||
Files to update:
|
||||
|
||||
- `internal/domain/domain.go`
|
||||
- `internal/promptdef/filesystem_repository.go`
|
||||
- `internal/promptdef/repository_test.go`
|
||||
- `internal/promptdef/testdata/`
|
||||
|
||||
Domain decisions:
|
||||
|
||||
- Add a typed cache-control value in `internal/domain`.
|
||||
- Use a pointer on message structs so absence is distinguishable from an empty object.
|
||||
- Use `omitempty` on JSON tags for the new field so existing no-cache prompt and render hashes do not gain `null` fields.
|
||||
|
||||
Use this domain shape:
|
||||
1. In `internal/domain/domain.go`, add a request-only type:
|
||||
|
||||
```go
|
||||
type CacheControlType string
|
||||
|
||||
const (
|
||||
CacheControlEphemeral CacheControlType = "ephemeral"
|
||||
)
|
||||
|
||||
type CacheControl struct {
|
||||
Type CacheControlType `yaml:"type" json:"type"`
|
||||
TTL string `yaml:"ttl,omitempty" json:"ttl,omitempty"`
|
||||
type ExecutionTargetOverride struct {
|
||||
Endpoint string `json:"endpoint,omitempty"`
|
||||
Model string `json:"model,omitempty"`
|
||||
Temperature *float64 `json:"temperature,omitempty"`
|
||||
MaxTokens *int `json:"max_tokens,omitempty"`
|
||||
TopP *float64 `json:"top_p,omitempty"`
|
||||
TimeoutSeconds *int `json:"timeout_seconds,omitempty"`
|
||||
ServiceTier string `json:"service_tier,omitempty"`
|
||||
ReasoningEffort string `json:"reasoning_effort,omitempty"`
|
||||
APIKeyEnv string `json:"api_key_env,omitempty"`
|
||||
ExtraParams map[string]any `json:"extra_params,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Add cache control to:
|
||||
2. Change `domain.RunRequest.Execution` from `*ExecutionTarget` to `*ExecutionTargetOverride`.
|
||||
3. Change `ExecutionProfile.ExtraParams` and `ExecutionTarget.ExtraParams` from `map[string]string` to `map[string]any`.
|
||||
4. Keep `ExecutionTarget` concrete. It represents the resolved effective runtime target after defaults, profile, and request overrides are merged.
|
||||
|
||||
```go
|
||||
type PromptMessageTemplate struct {
|
||||
Role string `yaml:"role"`
|
||||
Content string `yaml:"content"`
|
||||
ContentFile string `yaml:"content_file"`
|
||||
CacheControl *CacheControl `yaml:"cache_control,omitempty" json:"cache_control,omitempty"`
|
||||
}
|
||||
```
|
||||
### Runner Changes
|
||||
|
||||
Prompt loader changes:
|
||||
1. Update `internal/usecase/runner.go` so profile values still merge over built-in defaults and request overrides merge over that result.
|
||||
2. Keep the existing concrete profile merge semantics for profile numeric fields.
|
||||
3. Add a separate request override merge path that uses pointer presence:
|
||||
- `nil` numeric pointer means omitted; preserve the current value.
|
||||
- non-nil numeric pointer means explicit override, even when the value is `0`.
|
||||
4. Validate request override numeric values before or during merge:
|
||||
- `temperature`: `0 <= value <= 2`
|
||||
- `max_tokens`: `value >= 0`
|
||||
- `top_p`: `0 <= value <= 1`
|
||||
- `timeout_seconds`: `value >= 0`
|
||||
5. Preserve existing validation after merge:
|
||||
- effective endpoint required
|
||||
- effective model required
|
||||
- `api_key_env`, when set, must name a non-empty environment variable
|
||||
6. Preserve secret handling. The resolved API key value must never be stored in `PreparedRun`, `RunResult`, logs, or HTTP responses.
|
||||
|
||||
- Add this field to `promptMessageFile`:
|
||||
### CLI Changes
|
||||
|
||||
```go
|
||||
CacheControl *cacheControlFile `yaml:"cache_control"`
|
||||
```
|
||||
1. Update `internal/adapter/cli/run.go` request construction to build `domain.ExecutionTargetOverride`.
|
||||
2. Use the existing `flagWasSet` booleans to populate numeric pointers only when the user provided the flag.
|
||||
3. Required behavior:
|
||||
- omitted `--temperature` preserves profile/default temperature;
|
||||
- `--temperature 0` explicitly sets temperature to zero;
|
||||
- omitted `--top-p` preserves profile/default top-p;
|
||||
- `--top-p 0` explicitly sets top-p to zero;
|
||||
- omitted `--max-tokens` preserves profile/default max tokens;
|
||||
- `--max-tokens 0` explicitly sets max tokens to zero;
|
||||
- omitted `--timeout` preserves profile/default timeout;
|
||||
- `--timeout 0s` explicitly sets timeout seconds to zero.
|
||||
4. Do not add new CLI flags in this stage.
|
||||
|
||||
- Add a private `cacheControlFile` struct with `type` and `ttl`.
|
||||
- In `normalizePromptDefinition`, validate `cache_control` per message.
|
||||
- Trim whitespace from `type` and `ttl`.
|
||||
- Reject present `cache_control` when `type` is empty.
|
||||
- Reject any `type` other than `ephemeral`.
|
||||
- Accept omitted or empty `ttl`.
|
||||
- Reject any non-empty `ttl` other than `1h`.
|
||||
- Copy normalized values into `domain.PromptMessageTemplate.CacheControl`.
|
||||
- Preserve the existing rule that each message must set exactly one of `content` or `content_file`.
|
||||
### HTTP Changes
|
||||
|
||||
Testing requirements:
|
||||
1. Update `internal/adapter/http/dto.go` so numeric model override fields are pointers:
|
||||
- `Temperature *float64`
|
||||
- `MaxTokens *int`
|
||||
- `TopP *float64`
|
||||
- `TimeoutSeconds *int`
|
||||
2. Update DTO mapping in `internal/adapter/http/handler.go` to build `domain.ExecutionTargetOverride`.
|
||||
3. Preserve strict JSON decoding and existing error mapping.
|
||||
4. Required behavior:
|
||||
- omitted numeric JSON fields preserve profile/default values;
|
||||
- explicit numeric zero JSON fields override profile/default values.
|
||||
|
||||
- Add a valid fixture with `cache_control: {type: ephemeral, ttl: 1h}`.
|
||||
- Add a valid fixture with `cache_control: {type: ephemeral}`.
|
||||
- Add invalid fixtures for:
|
||||
- empty `cache_control.type`
|
||||
- unsupported `cache_control.type`
|
||||
- unsupported `cache_control.ttl`
|
||||
- unknown field under `cache_control`
|
||||
- Assert valid fixtures load with the expected `CacheControl` values.
|
||||
- Assert invalid fixtures fail with `promptdef.ErrInvalidPromptDefinition` or `promptdef.ErrInvalidYAML` as appropriate.
|
||||
- Re-run:
|
||||
### Tests
|
||||
|
||||
```bash
|
||||
go test ./internal/promptdef
|
||||
```
|
||||
Add or update tests in:
|
||||
|
||||
Do not update renderer, formatter, LLM serialization, adapters, or user-facing docs in this stage.
|
||||
|
||||
## Stage 2: Rendering, Prepared Output, And Render Hashing
|
||||
|
||||
Objective: preserve cache-control metadata through rendering, expose it in prepared-run output, and make `rendered_prompt_hash` reflect outbound-affecting cache metadata.
|
||||
|
||||
Files to update:
|
||||
|
||||
- `internal/domain/domain.go`
|
||||
- `internal/prompt/go_renderer.go`
|
||||
- `internal/prompt/renderer_test.go`
|
||||
- `internal/usecase/runner.go`
|
||||
- `internal/usecase/runner_test.go`
|
||||
- `internal/domain/prepared_run_test.go`
|
||||
- `internal/format/prepared_run.go`
|
||||
- `internal/format/prepared_run_test.go`
|
||||
|
||||
Domain changes:
|
||||
|
||||
- Add cache control to rendered messages:
|
||||
|
||||
```go
|
||||
type RenderedMessage struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
CacheControl *CacheControl `json:"cache_control,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
Renderer changes:
|
||||
|
||||
- When a `PromptMessageTemplate` has `CacheControl`, copy it to the returned `RenderedMessage`.
|
||||
- Do not mutate the template's pointer in place.
|
||||
- Add and use a small helper such as `cloneCacheControl` to avoid aliasing.
|
||||
- Do not change template execution behavior for `{{input "name"}}` or `{{.var}}`.
|
||||
|
||||
Rendered hash decision:
|
||||
|
||||
- `rendered_prompt_hash` must change when cache-control metadata changes, because the outbound provider request changes.
|
||||
- Preserve existing hashes for no-cache prompts by keeping the current role/content hash flow.
|
||||
- Append normalized cache-control metadata only when present, for example:
|
||||
- `cache_control.type=<type>`
|
||||
- `cache_control.ttl=<ttl>` only when non-empty
|
||||
- Include cache-control fields in a stable order.
|
||||
|
||||
Prepared-run formatting:
|
||||
|
||||
- JSON prepared-run output should include `cache_control` on messages only when present.
|
||||
- Text prepared-run output should show cache-control metadata under the specific message, before `content`, for example:
|
||||
|
||||
```text
|
||||
cache_control: ephemeral ttl=1h
|
||||
content: |
|
||||
```
|
||||
|
||||
- If `ttl` is empty, omit the `ttl=...` suffix.
|
||||
- Do not print cache-control lines for messages without cache control.
|
||||
|
||||
Testing requirements:
|
||||
|
||||
- Renderer test: cache control is copied to rendered messages.
|
||||
- Renderer test: changing returned rendered cache control does not mutate the source template.
|
||||
- Runner test: `rendered_prompt_hash` changes when cache control differs and remains stable for identical rendered messages.
|
||||
- Format JSON test: `cache_control` appears only when present.
|
||||
- Format text test: cache control appears under the correct message.
|
||||
- Existing render/format tests still pass.
|
||||
- Re-run:
|
||||
|
||||
```bash
|
||||
go test ./internal/prompt ./internal/usecase ./internal/domain ./internal/format
|
||||
```
|
||||
|
||||
Do not update outbound LLM serialization or adapter response surfaces in this stage.
|
||||
|
||||
## Stage 3: Outbound Chat Serialization And Usage Parsing
|
||||
|
||||
Objective: serialize cache-controlled messages as one-item text content-block arrays and parse cache usage fields from compatible provider responses.
|
||||
|
||||
Files to update:
|
||||
|
||||
- `internal/llm/openai_compatible_client.go`
|
||||
- `internal/llm/openai_compatible_client_test.go`
|
||||
- `internal/domain/domain.go`
|
||||
|
||||
Request serialization decisions:
|
||||
|
||||
- Keep string content for messages without cache control.
|
||||
- Use content blocks only when `RenderedMessage.CacheControl` is non-nil.
|
||||
- Do not introduce `content_blocks` prompt syntax.
|
||||
- Do not serialize top-level `cache_control`, top-level `session_id`, or `extra_params`.
|
||||
- Split request and response wire message structs. The current `openAIChatMessage` is used in both request and response paths; changing request content to `any` should not make response decoding less strict than necessary.
|
||||
|
||||
Use request wire types equivalent to:
|
||||
|
||||
```go
|
||||
type openAIChatRequestMessage struct {
|
||||
Role string `json:"role"`
|
||||
Content any `json:"content"`
|
||||
}
|
||||
|
||||
type openAIChatTextContentBlock struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
CacheControl *openAICacheControl `json:"cache_control,omitempty"`
|
||||
}
|
||||
|
||||
type openAICacheControl struct {
|
||||
Type string `json:"type"`
|
||||
TTL string `json:"ttl,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
When cache control is present, content should serialize as:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"type": "text",
|
||||
"text": "rendered text",
|
||||
"cache_control": {
|
||||
"type": "ephemeral",
|
||||
"ttl": "1h"
|
||||
}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
When `ttl` is empty, omit `ttl`.
|
||||
|
||||
Usage parsing decisions:
|
||||
|
||||
- Extend `domain.TokenUsage`:
|
||||
|
||||
```go
|
||||
type TokenUsage struct {
|
||||
PromptTokens int
|
||||
CompletionTokens int
|
||||
TotalTokens int
|
||||
CachedTokens int
|
||||
CacheWriteTokens int
|
||||
}
|
||||
```
|
||||
|
||||
- Parse `usage.prompt_tokens_details.cached_tokens` into `CachedTokens`.
|
||||
- Parse `usage.cache_write_tokens` into `CacheWriteTokens`.
|
||||
- Treat absent cache fields as zero.
|
||||
- Do not fail response decoding if cache fields are absent.
|
||||
- If a provider later returns a different cache-write field, add it in a later change after observing a real response.
|
||||
|
||||
Testing requirements:
|
||||
|
||||
- Existing request serialization test still sees string message content for no-cache messages.
|
||||
- New request serialization test sees content-block array for cache-controlled messages.
|
||||
- New request serialization test verifies empty `ttl` is omitted.
|
||||
- Response parsing test verifies cached and cache-write tokens are mapped.
|
||||
- Response parsing test verifies absent cache usage fields remain zero.
|
||||
- Existing structured-output and service-tier behavior remains unchanged.
|
||||
- Re-run:
|
||||
|
||||
```bash
|
||||
go test ./internal/llm
|
||||
```
|
||||
|
||||
Do not update CLI or HTTP observability in this stage unless required by compilation after extending `TokenUsage`.
|
||||
|
||||
## Stage 4: CLI And HTTP Cache Usage Observability
|
||||
|
||||
Objective: surface parsed cache usage through existing CLI and HTTP result paths without adding request knobs.
|
||||
|
||||
Files to update:
|
||||
|
||||
- `internal/adapter/cli/run.go`
|
||||
- `internal/adapter/cli/run_test.go`
|
||||
- `internal/adapter/http/dto.go`
|
||||
- `internal/adapter/http/handler.go`
|
||||
- `internal/adapter/http/handler_test.go`
|
||||
|
||||
CLI decisions:
|
||||
Required test coverage:
|
||||
|
||||
- Do not add CLI flags.
|
||||
- Keep success summaries concise.
|
||||
- Preserve the existing summary output when cache usage is zero.
|
||||
- When either cache field is non-zero, append:
|
||||
- Runner preserves profile value when request numeric override is omitted.
|
||||
- Runner applies explicit zero request override for `temperature`.
|
||||
- Runner applies explicit zero request override for `top_p`.
|
||||
- Runner applies explicit zero request override for `max_tokens`.
|
||||
- Runner applies explicit zero request override for `timeout_seconds`.
|
||||
- Invalid request override ranges fail as invalid request errors.
|
||||
- CLI `--temperature 0` reaches effective settings as zero.
|
||||
- HTTP `"temperature": 0` reaches effective settings as zero.
|
||||
- HTTP omitted `temperature` preserves profile/default value.
|
||||
|
||||
```text
|
||||
cached_tokens=<n> cache_write_tokens=<n>
|
||||
```
|
||||
### Verification
|
||||
|
||||
HTTP decisions:
|
||||
|
||||
- Do not add request fields.
|
||||
- Add numeric fields to `metadata.usage`:
|
||||
- `cached_tokens`
|
||||
- `cache_write_tokens`
|
||||
- Always include these fields as numbers, matching the current non-omitempty numeric usage fields.
|
||||
- Continue omitting raw model output unless `include_raw_output` is true.
|
||||
|
||||
Testing requirements:
|
||||
|
||||
- CLI summary omits cache fields when both are zero.
|
||||
- CLI summary includes cache fields when either is non-zero.
|
||||
- HTTP success response includes `metadata.usage.cached_tokens`.
|
||||
- HTTP success response includes `metadata.usage.cache_write_tokens`.
|
||||
- Existing HTTP request mapping is unchanged.
|
||||
- Re-run:
|
||||
Run:
|
||||
|
||||
```bash
|
||||
go test ./internal/adapter/cli ./internal/adapter/http
|
||||
go test ./...
|
||||
```
|
||||
|
||||
## Stage 5: Canonical Documentation
|
||||
## Stage 2: JSON-Compatible `extra_params`
|
||||
|
||||
Objective: update implemented-behavior docs after the feature works and tests pass.
|
||||
Goal: allow provider-specific parameters to carry JSON-compatible values throughout profile, HTTP, prepared output, metadata, and LLM request construction.
|
||||
|
||||
Files to update:
|
||||
### Domain And Loader Changes
|
||||
|
||||
1. Complete all compile fixes from changing `ExtraParams` to `map[string]any`.
|
||||
2. Ensure `internal/profile/filesystem_repository.go` continues to decode profiles strictly while allowing nested JSON-compatible values under `extra_params`.
|
||||
3. Add profile repository tests for `extra_params` containing:
|
||||
- string
|
||||
- number
|
||||
- boolean
|
||||
- nested object or array
|
||||
4. Ensure formatter output remains deterministic:
|
||||
- keep sorting `extra_params` keys in `internal/format/prepared_run.go`;
|
||||
- render non-string values with stable JSON encoding in text output.
|
||||
5. Preserve JSON formatter behavior through normal `encoding/json` output.
|
||||
|
||||
### HTTP Changes
|
||||
|
||||
1. Change HTTP model override `ExtraParams` to `map[string]any`.
|
||||
2. Add handler tests proving HTTP accepts JSON-compatible `extra_params` values.
|
||||
3. Preserve strict rejection of unknown fields and raw API-key payload fields.
|
||||
|
||||
### Verification
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
go test ./...
|
||||
```
|
||||
|
||||
## Stage 3: Outbound Serialization
|
||||
|
||||
Goal: serialize `reasoning_effort` and `extra_params` to the OpenAI-compatible chat-completions request.
|
||||
|
||||
### LLM Adapter Changes
|
||||
|
||||
1. In `internal/llm/openai_compatible_client.go`, add first-class outbound support for `reasoning_effort`.
|
||||
2. Add `extra_params` support by flattening `domain.ExecutionTarget.ExtraParams` into additional top-level JSON request fields.
|
||||
3. Implement reserved-field collision checks before the HTTP request is made.
|
||||
4. Reserved keys must include:
|
||||
- `model`
|
||||
- `session_id`
|
||||
- `messages`
|
||||
- `temperature`
|
||||
- `max_tokens`
|
||||
- `top_p`
|
||||
- `service_tier`
|
||||
- `reasoning_effort`
|
||||
- `response_format`
|
||||
5. Reject empty `extra_params` keys.
|
||||
6. Ensure each `extra_params` value can be marshaled as JSON. If marshaling fails, return `ErrInvalidRequest` with context.
|
||||
7. Keep existing request behavior unchanged when `reasoning_effort` and `extra_params` are unset.
|
||||
|
||||
### Recommended Implementation Shape
|
||||
|
||||
Use a custom marshal path for the outbound chat request rather than string manipulation.
|
||||
|
||||
One acceptable shape:
|
||||
|
||||
- Add `ReasoningEffort string` and `ExtraParams map[string]any` to the internal `openAIChatRequest`.
|
||||
- Add a helper that converts `openAIChatRequest` into `map[string]any`, inserts first-class fields when set, then inserts `ExtraParams` after collision validation.
|
||||
- Marshal that map with `encoding/json`.
|
||||
|
||||
Do not construct outbound JSON with manual string concatenation.
|
||||
|
||||
### Tests
|
||||
|
||||
Update `internal/llm/openai_compatible_client_test.go`.
|
||||
|
||||
Required test coverage:
|
||||
|
||||
- outbound JSON includes `reasoning_effort` when set;
|
||||
- outbound JSON omits `reasoning_effort` when unset;
|
||||
- outbound JSON includes string, number, boolean, object, and array `extra_params`;
|
||||
- reserved `extra_params` keys fail before provider call;
|
||||
- empty `extra_params` keys fail before provider call;
|
||||
- existing message, cache-control, service-tier, response-format, and usage parsing tests continue to pass.
|
||||
|
||||
### Verification
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
go test ./...
|
||||
```
|
||||
|
||||
## Stage 4: Documentation And Examples
|
||||
|
||||
Goal: move implemented behavior from roadmap to canonical docs after code is complete.
|
||||
|
||||
Update only after Stages 1 through 3 are implemented.
|
||||
|
||||
### Required Docs
|
||||
|
||||
Update:
|
||||
|
||||
- `docs/config.md`
|
||||
- `docs/integrations/openai-compatible-chat.md`
|
||||
- `docs/cli.md`
|
||||
- `docs/troubleshooting.md`
|
||||
- `docs/integrations/http-api.md`
|
||||
- `docs/integrations/openai-compatible-chat.md`
|
||||
- `docs/internal/runner.md`
|
||||
- `docs/internal/adapters.md`, if adapter response behavior needs an explicit note
|
||||
- `docs/roadmap/cache.md`, only if the accepted target changes during implementation
|
||||
- `docs/internal/adapters.md`
|
||||
|
||||
Documentation requirements:
|
||||
Required documentation content:
|
||||
|
||||
- Keep all unimplemented ideas under `docs/roadmap/`.
|
||||
- Document `messages[].cache_control` in the prompt definition reference in `docs/config.md`.
|
||||
- Document the outbound message serialization rule in `docs/integrations/openai-compatible-chat.md`.
|
||||
- Document parsed cache usage fields in `docs/integrations/openai-compatible-chat.md`.
|
||||
- In `docs/cli.md`, note that prompt cache control is configured in prompt YAML, not with CLI flags.
|
||||
- In `docs/troubleshooting.md`, add cache miss guidance:
|
||||
- stable context should appear before the cache breakpoint
|
||||
- dynamic input before the breakpoint changes the cache key
|
||||
- providers may impose minimum token thresholds and breakpoint limits
|
||||
- use cached/cache-write token counts to verify behavior
|
||||
- Do not claim support for deferred roadmap items.
|
||||
- `reasoning_effort` is serialized outbound when set.
|
||||
- `extra_params` serializes as provider-specific top-level outbound JSON fields.
|
||||
- `extra_params` supports JSON-compatible values.
|
||||
- reserved `extra_params` fields are rejected.
|
||||
- per-request numeric overrides distinguish omitted values from explicit zero values.
|
||||
- CLI explicit zero behavior for existing numeric flags.
|
||||
- HTTP explicit zero behavior for model override numeric fields.
|
||||
- no raw API-key values are accepted or emitted.
|
||||
|
||||
Testing and verification:
|
||||
### Examples
|
||||
|
||||
Update examples only if needed to keep them accurate and runnable.
|
||||
|
||||
If adding an `extra_params` example, keep it secret-free and simple. Prefer a harmless provider-routing example over a vendor-specific feature that requires special credentials.
|
||||
|
||||
### Verification
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
go test ./...
|
||||
```
|
||||
|
||||
If documentation examples include new YAML snippets, ensure they match the strict prompt loader schema.
|
||||
|
||||
## Stage 6: Final Verification
|
||||
|
||||
Objective: verify the whole change set end to end.
|
||||
|
||||
Required commands:
|
||||
|
||||
```bash
|
||||
go test ./...
|
||||
go build ./cmd/scriptorium
|
||||
```
|
||||
|
||||
Recommended smoke checks:
|
||||
|
||||
```bash
|
||||
go run ./cmd/scriptorium render \
|
||||
--config ./examples/config.yml \
|
||||
--prompt generic.markdown_summary \
|
||||
@@ -361,30 +251,12 @@ go run ./cmd/scriptorium render \
|
||||
--format json
|
||||
```
|
||||
|
||||
If adding a new example prompt with `cache_control`, also smoke-render that prompt with `--format json` and inspect that the rendered message includes `cache_control`.
|
||||
## Final Checks
|
||||
|
||||
Final review checklist:
|
||||
Before considering the feature complete:
|
||||
|
||||
- Existing prompt definitions without `cache_control` still load.
|
||||
- Prompt definitions with `cache_control` render successfully.
|
||||
- Unknown cache-control YAML fields are rejected.
|
||||
- No-cache outbound messages still serialize with string `content`.
|
||||
- Cache-controlled outbound messages serialize with text content-block arrays.
|
||||
- `rendered_prompt_hash` changes when cache-control metadata changes.
|
||||
- Cache usage fields are zero when providers omit them.
|
||||
- CLI and HTTP request surfaces have no new knobs.
|
||||
- CLI and HTTP response surfaces expose cache usage.
|
||||
- Non-roadmap docs describe only implemented behavior.
|
||||
- `docs/roadmap/cache.md` still defines target state and deferred work, not task steps.
|
||||
|
||||
## Stage Boundaries
|
||||
|
||||
Stages 1 through 4 can be implemented in separate prompts if needed. Stage 5 should happen only after behavior is implemented. Stage 6 should be performed after all implementation and documentation changes are complete.
|
||||
|
||||
If a stage fails because prior stage changes were not implemented, stop and complete the earlier stage first rather than papering over missing domain fields or DTO mappings.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None required for the initial implementation.
|
||||
|
||||
The plan chooses message-level `cache_control`, string content for legacy messages, text-block array content only when cache control is present, and no new CLI/HTTP request knobs. These choices match `docs/roadmap/cache.md` and preserve existing prompt-definition compatibility.
|
||||
1. Confirm `git diff` contains only intended code, test, doc, and example changes.
|
||||
2. Confirm all non-roadmap docs describe implemented behavior only.
|
||||
3. Confirm no output path exposes raw API key values.
|
||||
4. Confirm `go test ./...` passes.
|
||||
5. Confirm the render smoke command passes.
|
||||
|
||||
96
docs/roadmap/params.md
Normal file
96
docs/roadmap/params.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Runtime Parameter Feature Roadmap
|
||||
|
||||
This roadmap defines the target behavior for runtime model parameters.
|
||||
|
||||
Current behavior has two limitations:
|
||||
|
||||
- `reasoning_effort` and `extra_params` are parsed into effective execution settings but are not serialized into outbound OpenAI-compatible chat-completions requests.
|
||||
- Per-request numeric execution overrides use zero-value merge semantics, so callers cannot reliably override a profile value with an explicit zero such as `temperature: 0`.
|
||||
|
||||
The implementation plan for this feature lives in `docs/roadmap/implementation.md`.
|
||||
|
||||
## Target State
|
||||
|
||||
Scriptorium should preserve the existing separation between prompt definitions, execution profiles, and per-request execution overrides while making runtime parameter behavior explicit and predictable.
|
||||
|
||||
Expected end state:
|
||||
|
||||
- Effective execution settings remain visible in prepared-run output, run metadata, and HTTP metadata without exposing raw secret values.
|
||||
- `reasoning_effort` is treated as a first-class effective execution setting and is serialized to the outbound OpenAI-compatible request when set.
|
||||
- `extra_params` supports provider-specific OpenAI-compatible request fields.
|
||||
- `extra_params` is serialized as additional top-level outbound JSON fields.
|
||||
- `extra_params` values support JSON-compatible scalar, object, and array values.
|
||||
- `extra_params` cannot override first-class outbound request fields.
|
||||
- Per-request numeric overrides preserve caller intent, including explicit zero values.
|
||||
- Omitted per-request numeric overrides continue to inherit the selected profile and built-in defaults.
|
||||
- External decoding remains strict for config, prompt, profile, and HTTP request payloads.
|
||||
|
||||
## Policy Decisions
|
||||
|
||||
### `extra_params`
|
||||
|
||||
`extra_params` should serialize as additional top-level outbound JSON fields in the OpenAI-compatible chat-completions request.
|
||||
|
||||
Reasoning:
|
||||
|
||||
Most OpenAI-compatible providers expose vendor-specific chat-completions parameters as top-level fields. This keeps Scriptorium's adapter compatible with that ecosystem without adding first-class fields for every provider option.
|
||||
|
||||
`extra_params` must not silently override Scriptorium-owned fields. Reserved outbound fields include at least:
|
||||
|
||||
- `model`
|
||||
- `session_id`
|
||||
- `messages`
|
||||
- `temperature`
|
||||
- `max_tokens`
|
||||
- `top_p`
|
||||
- `service_tier`
|
||||
- `reasoning_effort`
|
||||
- `response_format`
|
||||
|
||||
If a caller supplies a reserved key through `extra_params`, Scriptorium should fail before making the outbound HTTP request.
|
||||
|
||||
`extra_params` should use JSON-compatible values rather than only strings.
|
||||
|
||||
Reasoning:
|
||||
|
||||
Provider-specific parameters commonly need booleans, numbers, objects, or arrays. String-only values would force awkward encoding and would likely require a later compatibility break.
|
||||
|
||||
### Presence-Aware Overrides
|
||||
|
||||
Per-request execution overrides should use a presence-aware type with pointer fields for optional numeric values.
|
||||
|
||||
Reasoning:
|
||||
|
||||
The resolved execution target should remain a concrete value used by prepared runs, generated requests, and metadata. Optionality matters at the request boundary, not after the runner has resolved the effective target.
|
||||
|
||||
This keeps adapter and merge logic precise while avoiding nil checks in formatter, metadata, and LLM serialization paths.
|
||||
|
||||
## Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- Runtime merge behavior for per-request execution overrides.
|
||||
- HTTP model override decoding for explicit zero numeric values.
|
||||
- CLI execution override handling for explicit zero numeric flags.
|
||||
- Outbound serialization of `reasoning_effort`.
|
||||
- Outbound serialization of JSON-compatible `extra_params`.
|
||||
- Tests and documentation for the changed implemented behavior.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- Expanding the HTTP API beyond `POST /v1/runs`.
|
||||
- Adding built-in HTTP authentication or authorization.
|
||||
- Adding durable run state, run history, or multi-step orchestration.
|
||||
- Adding broad provider-specific adapter packages.
|
||||
- Adding new CLI flags for every provider-specific parameter.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- A profile containing `reasoning_effort: medium` produces an outbound request with `reasoning_effort`.
|
||||
- HTTP callers can pass `reasoning_effort` through the existing `model` override object and have it appear outbound.
|
||||
- A profile or HTTP request containing JSON-compatible `extra_params` produces outbound top-level JSON fields according to the reserved-field policy.
|
||||
- Reserved `extra_params` collisions fail before the outbound provider call.
|
||||
- CLI callers can pass `--temperature 0` and observe `temperature: 0` in rendered/effective settings and outbound requests.
|
||||
- HTTP callers can send `"temperature": 0` and observe the same behavior.
|
||||
- Omitting `temperature` continues to preserve the selected profile/default value.
|
||||
- Raw API key values remain unsupported in config, profiles, CLI flags, HTTP payloads, logs, and rendered output.
|
||||
@@ -513,18 +513,25 @@ func buildRunRequestFromConfig(cfg *runConfig) (domain.RunRequest, error) {
|
||||
inputs[name] = domain.ArtifactRef{Type: domain.ArtifactRefFile, URI: path}
|
||||
}
|
||||
|
||||
var modelOverride *domain.ExecutionTarget
|
||||
var modelOverride *domain.ExecutionTargetOverride
|
||||
if cfg.llmBaseURLSet || cfg.modelSet || cfg.temperatureSet || cfg.maxTokensSet || cfg.topPSet || cfg.apiKeyEnvSet || cfg.timeoutSet {
|
||||
modelOverride = &domain.ExecutionTarget{
|
||||
Endpoint: cfg.llmBaseURL,
|
||||
Model: cfg.model,
|
||||
Temperature: cfg.temperature,
|
||||
MaxTokens: cfg.maxTokens,
|
||||
TopP: cfg.topP,
|
||||
APIKeyEnv: cfg.apiKeyEnv,
|
||||
modelOverride = &domain.ExecutionTargetOverride{
|
||||
Endpoint: cfg.llmBaseURL,
|
||||
Model: cfg.model,
|
||||
APIKeyEnv: cfg.apiKeyEnv,
|
||||
}
|
||||
if cfg.temperatureSet {
|
||||
modelOverride.Temperature = &cfg.temperature
|
||||
}
|
||||
if cfg.maxTokensSet {
|
||||
modelOverride.MaxTokens = &cfg.maxTokens
|
||||
}
|
||||
if cfg.topPSet {
|
||||
modelOverride.TopP = &cfg.topP
|
||||
}
|
||||
if cfg.timeoutSet {
|
||||
modelOverride.TimeoutSeconds = int(cfg.timeout.Seconds())
|
||||
timeoutSeconds := int(cfg.timeout.Seconds())
|
||||
modelOverride.TimeoutSeconds = &timeoutSeconds
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -747,6 +747,35 @@ func TestRenderCommandDefaultFormatTextIncludesPreparedDetailsAndNoSecrets(t *te
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderCommandExplicitZeroTemperatureReachesEffectiveSettings(t *testing.T) {
|
||||
lib := newCLITestLibrary(t)
|
||||
inputPath := lib.writeInputFile(t, "transcript.md", "hello transcript")
|
||||
|
||||
writePromptFile(t, lib.promptDir, "prompt.render", "local-default")
|
||||
profile := `id: local-default
|
||||
endpoint: http://127.0.0.1:1/v1
|
||||
model: profile-model
|
||||
temperature: 0.7
|
||||
`
|
||||
if err := os.WriteFile(filepath.Join(lib.profileDir, "local-default.yaml"), []byte(profile), 0o644); err != nil {
|
||||
t.Fatalf("failed to write profile fixture: %v", err)
|
||||
}
|
||||
|
||||
code, stdout, stderr := runCLICommand(t, renderCommand, []string{
|
||||
"--prompt-dir", lib.promptDir,
|
||||
"--profile-dir", lib.profileDir,
|
||||
"--prompt", "prompt.render",
|
||||
"--input", "transcript=" + inputPath,
|
||||
"--temperature", "0",
|
||||
})
|
||||
if code != ExitOK {
|
||||
t.Fatalf("expected ExitOK, got %d stderr=%q", code, stderr)
|
||||
}
|
||||
if !strings.Contains(stdout, "\n temperature: 0\n") {
|
||||
t.Fatalf("expected explicit zero temperature in effective settings, got:\n%s", stdout)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRenderCommandSucceedsWithPromptAndProfileDirsFromConfig(t *testing.T) {
|
||||
lib := newCLITestLibrary(t)
|
||||
inputPath := lib.writeInputFile(t, "transcript.md", "hello transcript")
|
||||
|
||||
@@ -21,16 +21,16 @@ type inputRefDTO struct {
|
||||
}
|
||||
|
||||
type modelOverrideRequestDTO struct {
|
||||
Endpoint string `json:"endpoint,omitempty"`
|
||||
Model string `json:"model,omitempty"`
|
||||
Temperature float64 `json:"temperature,omitempty"`
|
||||
MaxTokens int `json:"max_tokens,omitempty"`
|
||||
TopP float64 `json:"top_p,omitempty"`
|
||||
TimeoutSeconds int `json:"timeout_seconds,omitempty"`
|
||||
ServiceTier string `json:"service_tier,omitempty"`
|
||||
ReasoningEffort string `json:"reasoning_effort,omitempty"`
|
||||
APIKeyEnv string `json:"api_key_env,omitempty"`
|
||||
ExtraParams map[string]string `json:"extra_params,omitempty"`
|
||||
Endpoint string `json:"endpoint,omitempty"`
|
||||
Model string `json:"model,omitempty"`
|
||||
Temperature *float64 `json:"temperature,omitempty"`
|
||||
MaxTokens *int `json:"max_tokens,omitempty"`
|
||||
TopP *float64 `json:"top_p,omitempty"`
|
||||
TimeoutSeconds *int `json:"timeout_seconds,omitempty"`
|
||||
ServiceTier string `json:"service_tier,omitempty"`
|
||||
ReasoningEffort string `json:"reasoning_effort,omitempty"`
|
||||
APIKeyEnv string `json:"api_key_env,omitempty"`
|
||||
ExtraParams map[string]any `json:"extra_params,omitempty"`
|
||||
}
|
||||
|
||||
type runResponseDTO struct {
|
||||
@@ -70,16 +70,16 @@ type metadataDTO struct {
|
||||
}
|
||||
|
||||
type modelParamsDTO struct {
|
||||
Endpoint string `json:"endpoint"`
|
||||
Model string `json:"model"`
|
||||
Temperature float64 `json:"temperature"`
|
||||
MaxTokens int `json:"max_tokens"`
|
||||
TopP float64 `json:"top_p"`
|
||||
TimeoutSeconds int `json:"timeout_seconds"`
|
||||
ServiceTier string `json:"service_tier,omitempty"`
|
||||
ReasoningEffort string `json:"reasoning_effort,omitempty"`
|
||||
APIKeyEnv string `json:"api_key_env,omitempty"`
|
||||
ExtraParams map[string]string `json:"extra_params,omitempty"`
|
||||
Endpoint string `json:"endpoint"`
|
||||
Model string `json:"model"`
|
||||
Temperature float64 `json:"temperature"`
|
||||
MaxTokens int `json:"max_tokens"`
|
||||
TopP float64 `json:"top_p"`
|
||||
TimeoutSeconds int `json:"timeout_seconds"`
|
||||
ServiceTier string `json:"service_tier,omitempty"`
|
||||
ReasoningEffort string `json:"reasoning_effort,omitempty"`
|
||||
APIKeyEnv string `json:"api_key_env,omitempty"`
|
||||
ExtraParams map[string]any `json:"extra_params,omitempty"`
|
||||
}
|
||||
|
||||
type tokenUsageDTO struct {
|
||||
|
||||
@@ -61,9 +61,9 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
}
|
||||
|
||||
var model *domain.ExecutionTarget
|
||||
var model *domain.ExecutionTargetOverride
|
||||
if req.Model != nil {
|
||||
model = executionTargetFromModelOverrideDTO(req.Model)
|
||||
model = executionTargetOverrideFromModelOverrideDTO(req.Model)
|
||||
}
|
||||
|
||||
res, err := h.runner.Run(r.Context(), domain.RunRequest{
|
||||
@@ -123,11 +123,11 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, http.StatusOK, resp)
|
||||
}
|
||||
|
||||
func executionTargetFromModelOverrideDTO(dto *modelOverrideRequestDTO) *domain.ExecutionTarget {
|
||||
func executionTargetOverrideFromModelOverrideDTO(dto *modelOverrideRequestDTO) *domain.ExecutionTargetOverride {
|
||||
if dto == nil {
|
||||
return nil
|
||||
}
|
||||
return &domain.ExecutionTarget{
|
||||
return &domain.ExecutionTargetOverride{
|
||||
Endpoint: dto.Endpoint,
|
||||
Model: dto.Model,
|
||||
Temperature: dto.Temperature,
|
||||
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
"time"
|
||||
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/llm"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/profile"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/promptdef"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/usecase"
|
||||
@@ -32,6 +33,34 @@ func (f *fakeRunner) Run(ctx context.Context, req domain.RunRequest) (*domain.Ru
|
||||
return f.result, nil
|
||||
}
|
||||
|
||||
type handlerPromptRepo struct {
|
||||
def *domain.PromptDefinition
|
||||
}
|
||||
|
||||
func (r handlerPromptRepo) GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error) {
|
||||
return r.def, nil
|
||||
}
|
||||
|
||||
type handlerProfileRepo struct {
|
||||
profile *domain.ExecutionProfile
|
||||
}
|
||||
|
||||
func (r handlerProfileRepo) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
|
||||
return r.profile, nil
|
||||
}
|
||||
|
||||
type handlerArtifactReader struct{}
|
||||
|
||||
func (handlerArtifactReader) Read(ctx context.Context, ref domain.ArtifactRef) (*domain.Artifact, error) {
|
||||
return &domain.Artifact{Name: "input", Body: []byte("input"), Hash: "hash"}, nil
|
||||
}
|
||||
|
||||
type handlerRenderer struct{}
|
||||
|
||||
func (handlerRenderer) Render(ctx context.Context, definition *domain.PromptDefinition, inputs map[string]*domain.Artifact, vars map[string]string) (*domain.RenderedPrompt, error) {
|
||||
return &domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}}, nil
|
||||
}
|
||||
|
||||
func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
|
||||
start := time.Now().UTC()
|
||||
end := start.Add(2 * time.Second)
|
||||
@@ -147,7 +176,7 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
|
||||
if r.last.Execution == nil || r.last.Execution.Model != "gpt-x" {
|
||||
t.Fatalf("expected model override, got %#v", r.last.Execution)
|
||||
}
|
||||
if r.last.Execution.TimeoutSeconds != 120 {
|
||||
if r.last.Execution.TimeoutSeconds == nil || *r.last.Execution.TimeoutSeconds != 120 {
|
||||
t.Fatalf("expected timeout_seconds override 120, got %#v", r.last.Execution)
|
||||
}
|
||||
if r.last.Execution.ServiceTier != "flex" {
|
||||
@@ -228,20 +257,136 @@ func TestHandlerModelOverrideMapsAllSupportedExecutionFields(t *testing.T) {
|
||||
got := r.last.Execution
|
||||
if got.Endpoint != "http://override/v1" ||
|
||||
got.Model != "override-model" ||
|
||||
got.Temperature != 0.6 ||
|
||||
got.MaxTokens != 250 ||
|
||||
got.TopP != 0.85 ||
|
||||
got.TimeoutSeconds != 33 ||
|
||||
got.ServiceTier != "flex" ||
|
||||
got.ReasoningEffort != "medium" ||
|
||||
got.APIKeyEnv != "SCRIPTORIUM_API_KEY" {
|
||||
t.Fatalf("unexpected mapped execution target: %+v", got)
|
||||
}
|
||||
if !reflect.DeepEqual(got.ExtraParams, map[string]string{"provider_option": "on"}) {
|
||||
if got.Temperature == nil || *got.Temperature != 0.6 {
|
||||
t.Fatalf("unexpected mapped temperature: %#v", got.Temperature)
|
||||
}
|
||||
if got.MaxTokens == nil || *got.MaxTokens != 250 {
|
||||
t.Fatalf("unexpected mapped max_tokens: %#v", got.MaxTokens)
|
||||
}
|
||||
if got.TopP == nil || *got.TopP != 0.85 {
|
||||
t.Fatalf("unexpected mapped top_p: %#v", got.TopP)
|
||||
}
|
||||
if got.TimeoutSeconds == nil || *got.TimeoutSeconds != 33 {
|
||||
t.Fatalf("unexpected mapped timeout_seconds: %#v", got.TimeoutSeconds)
|
||||
}
|
||||
if !reflect.DeepEqual(got.ExtraParams, map[string]any{"provider_option": "on"}) {
|
||||
t.Fatalf("unexpected mapped extra_params: %#v", got.ExtraParams)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerModelOverrideAcceptsJSONCompatibleExtraParams(t *testing.T) {
|
||||
r := &fakeRunner{result: &domain.RunResult{
|
||||
Artifact: domain.Artifact{Body: []byte("ok")},
|
||||
Validation: domain.ValidationResult{Status: domain.ValidationPassed, Mode: domain.ValidationBasic, IsValid: true},
|
||||
EffectiveModelParams: domain.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1"},
|
||||
}}
|
||||
h := NewHandler(r)
|
||||
|
||||
reqBody := `{
|
||||
"prompt_id": "prompt-1",
|
||||
"inputs": {"transcript": {"type": "file", "uri": "./t.md"}},
|
||||
"model": {
|
||||
"extra_params": {
|
||||
"string_value": "enabled",
|
||||
"number_value": 42,
|
||||
"boolean_value": true,
|
||||
"object_value": {"nested": "value", "count": 2},
|
||||
"array_value": ["first", 3, false]
|
||||
}
|
||||
}
|
||||
}`
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(reqBody))
|
||||
w := httptest.NewRecorder()
|
||||
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
if r.last.Execution == nil {
|
||||
t.Fatal("expected execution override in run request")
|
||||
}
|
||||
want := map[string]any{
|
||||
"string_value": "enabled",
|
||||
"number_value": float64(42),
|
||||
"boolean_value": true,
|
||||
"object_value": map[string]any{"nested": "value", "count": float64(2)},
|
||||
"array_value": []any{"first", float64(3), false},
|
||||
}
|
||||
if !reflect.DeepEqual(r.last.Execution.ExtraParams, want) {
|
||||
t.Fatalf("unexpected mapped extra_params:\ngot=%#v\nwant=%#v", r.last.Execution.ExtraParams, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerModelOverrideExplicitZeroTemperatureMapsAsPresent(t *testing.T) {
|
||||
r := &fakeRunner{result: &domain.RunResult{
|
||||
Artifact: domain.Artifact{Body: []byte("ok")},
|
||||
Validation: domain.ValidationResult{Status: domain.ValidationPassed, Mode: domain.ValidationBasic, IsValid: true},
|
||||
EffectiveModelParams: domain.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1", Temperature: 0},
|
||||
}}
|
||||
h := NewHandler(r)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{
|
||||
"prompt_id": "prompt-1",
|
||||
"inputs": {"transcript": {"type": "file", "uri": "./t.md"}},
|
||||
"model": {"temperature": 0}
|
||||
}`))
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
if r.last.Execution == nil || r.last.Execution.Temperature == nil {
|
||||
t.Fatalf("expected temperature override to be present, got %#v", r.last.Execution)
|
||||
}
|
||||
if *r.last.Execution.Temperature != 0 {
|
||||
t.Fatalf("expected zero temperature override, got %v", *r.last.Execution.Temperature)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerModelOverrideOmittedTemperatureMapsAsAbsent(t *testing.T) {
|
||||
r := &fakeRunner{result: &domain.RunResult{
|
||||
Artifact: domain.Artifact{Body: []byte("ok")},
|
||||
Validation: domain.ValidationResult{Status: domain.ValidationPassed, Mode: domain.ValidationBasic, IsValid: true},
|
||||
EffectiveModelParams: domain.ExecutionTarget{Endpoint: "http://llm/v1", Model: "m1", Temperature: 0.7},
|
||||
}}
|
||||
h := NewHandler(r)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{
|
||||
"prompt_id": "prompt-1",
|
||||
"inputs": {"transcript": {"type": "file", "uri": "./t.md"}},
|
||||
"model": {"model": "override-model"}
|
||||
}`))
|
||||
w := httptest.NewRecorder()
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("expected 200, got %d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
if r.last.Execution == nil {
|
||||
t.Fatal("expected model override")
|
||||
}
|
||||
if r.last.Execution.Temperature != nil {
|
||||
t.Fatalf("expected omitted temperature to remain absent, got %#v", r.last.Execution.Temperature)
|
||||
}
|
||||
|
||||
var resp map[string]any
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("invalid JSON response: %v", err)
|
||||
}
|
||||
metadata := resp["metadata"].(map[string]any)
|
||||
params := metadata["model_params"].(map[string]any)
|
||||
if params["temperature"] != 0.7 {
|
||||
t.Fatalf("expected effective profile/default temperature in response, got %#v", params["temperature"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerResponseMetadataModelParamsIncludesAllSupportedFields(t *testing.T) {
|
||||
r := &fakeRunner{result: &domain.RunResult{
|
||||
Artifact: domain.Artifact{
|
||||
@@ -262,8 +407,10 @@ func TestHandlerResponseMetadataModelParamsIncludesAllSupportedFields(t *testing
|
||||
ServiceTier: "priority",
|
||||
ReasoningEffort: "high",
|
||||
APIKeyEnv: "SCRIPTORIUM_API_KEY",
|
||||
ExtraParams: map[string]string{
|
||||
ExtraParams: map[string]any{
|
||||
"provider_option": "on",
|
||||
"number_value": 42,
|
||||
"object_value": map[string]any{"nested": "value"},
|
||||
},
|
||||
},
|
||||
}}
|
||||
@@ -318,6 +465,13 @@ func TestHandlerResponseMetadataModelParamsIncludesAllSupportedFields(t *testing
|
||||
if extraParams["provider_option"] != "on" {
|
||||
t.Fatalf("unexpected extra_params.provider_option: %#v", extraParams["provider_option"])
|
||||
}
|
||||
if extraParams["number_value"] != float64(42) {
|
||||
t.Fatalf("unexpected extra_params.number_value: %#v", extraParams["number_value"])
|
||||
}
|
||||
objectValue, ok := extraParams["object_value"].(map[string]any)
|
||||
if !ok || objectValue["nested"] != "value" {
|
||||
t.Fatalf("unexpected extra_params.object_value: %#v", extraParams["object_value"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerInvalidJSON(t *testing.T) {
|
||||
@@ -352,6 +506,54 @@ func TestHandlerMissingPromptID(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerReservedExtraParamsThroughRunnerMapsToInvalidRequest(t *testing.T) {
|
||||
llmClient, err := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
runner := usecase.NewRunner(
|
||||
handlerPromptRepo{def: &domain.PromptDefinition{
|
||||
ID: "p",
|
||||
Version: "1",
|
||||
DefaultProfile: "exec",
|
||||
Templates: []domain.PromptMessageTemplate{{Role: "user", Content: "hi"}},
|
||||
OutputFormat: domain.FormatText,
|
||||
Validation: domain.OutputContract{Format: domain.FormatText, ValidationMode: domain.ValidationNone},
|
||||
}},
|
||||
handlerProfileRepo{profile: &domain.ExecutionProfile{
|
||||
ID: "exec",
|
||||
Endpoint: "http://example.invalid/v1",
|
||||
Model: "model",
|
||||
}},
|
||||
handlerArtifactReader{},
|
||||
handlerRenderer{},
|
||||
llmClient,
|
||||
nil,
|
||||
)
|
||||
h := NewHandler(runner)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/v1/runs", bytes.NewBufferString(`{
|
||||
"prompt_id":"p",
|
||||
"inputs":{"x":{"type":"file","uri":"a"}},
|
||||
"model":{"extra_params":{"model":"collision"}}
|
||||
}`))
|
||||
w := httptest.NewRecorder()
|
||||
|
||||
h.ServeHTTP(w, req)
|
||||
|
||||
if w.Code != http.StatusBadRequest {
|
||||
t.Fatalf("expected 400, got %d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
var resp map[string]any
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
|
||||
t.Fatalf("invalid JSON response: %v", err)
|
||||
}
|
||||
errBody := resp["error"].(map[string]any)
|
||||
if errBody["code"] != "invalid_request" {
|
||||
t.Fatalf("expected invalid_request code, got %#v", errBody["code"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerUsecaseErrorMapping(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
|
||||
@@ -47,6 +47,11 @@ const (
|
||||
CacheControlEphemeral CacheControlType = "ephemeral"
|
||||
)
|
||||
|
||||
const (
|
||||
// SessionIDMaxLength is OpenRouter's documented maximum session_id length.
|
||||
SessionIDMaxLength = 256
|
||||
)
|
||||
|
||||
// CacheControl describes provider cache metadata attached to prompt content.
|
||||
type CacheControl struct {
|
||||
Type CacheControlType `yaml:"type" json:"type"`
|
||||
@@ -60,7 +65,7 @@ type RunRequest struct {
|
||||
ProfileID string
|
||||
Inputs map[string]ArtifactRef
|
||||
Vars map[string]string
|
||||
Execution *ExecutionTarget
|
||||
Execution *ExecutionTargetOverride
|
||||
Validation *OutputContract
|
||||
Metadata map[string]string
|
||||
}
|
||||
@@ -90,19 +95,21 @@ type RunResult struct {
|
||||
// PreparedRun contains pre-LLM execution state from the prepare/render phase.
|
||||
// It must never include resolved API key values, model output, or validation data.
|
||||
type PreparedRun struct {
|
||||
PromptID string `json:"prompt_id"`
|
||||
PromptVersion string `json:"prompt_version,omitempty"`
|
||||
PromptHash string `json:"prompt_hash,omitempty"`
|
||||
SelectedProfileID string `json:"selected_profile_id"`
|
||||
EffectiveModelParams ExecutionTarget `json:"effective_model_params"`
|
||||
OutputContract OutputContract `json:"output_contract"`
|
||||
StructuredOutput *StructuredOutputSpec `json:"structured_output,omitempty"`
|
||||
InputHashes map[string]string `json:"input_hashes,omitempty"`
|
||||
RenderedPromptHash string `json:"rendered_prompt_hash"`
|
||||
Messages []RenderedMessage `json:"messages"`
|
||||
StartTime time.Time `json:"start_time,omitempty"`
|
||||
EndTime time.Time `json:"end_time,omitempty"`
|
||||
DurationMS int64 `json:"duration_ms,omitempty"`
|
||||
PromptID string `json:"prompt_id"`
|
||||
PromptVersion string `json:"prompt_version,omitempty"`
|
||||
PromptHash string `json:"prompt_hash,omitempty"`
|
||||
SelectedProfileID string `json:"selected_profile_id"`
|
||||
EffectiveModelParams ExecutionTarget `json:"effective_model_params"`
|
||||
TargetPresence ExecutionTargetPresence `json:"-"`
|
||||
OutputContract OutputContract `json:"output_contract"`
|
||||
StructuredOutput *StructuredOutputSpec `json:"structured_output,omitempty"`
|
||||
InputHashes map[string]string `json:"input_hashes,omitempty"`
|
||||
SessionID string `json:"session_id,omitempty"`
|
||||
RenderedPromptHash string `json:"rendered_prompt_hash"`
|
||||
Messages []RenderedMessage `json:"messages"`
|
||||
StartTime time.Time `json:"start_time,omitempty"`
|
||||
EndTime time.Time `json:"end_time,omitempty"`
|
||||
DurationMS int64 `json:"duration_ms,omitempty"`
|
||||
}
|
||||
|
||||
// ArtifactRef represents a reference to an input artifact.
|
||||
@@ -128,6 +135,7 @@ type PromptDefinition struct {
|
||||
Version string `yaml:"version"`
|
||||
DefaultProfile string `yaml:"default_profile"`
|
||||
Description string `yaml:"description"`
|
||||
SessionID string `yaml:"session_id" json:"session_id,omitempty"`
|
||||
Inputs []PromptInput `yaml:"inputs"`
|
||||
Templates []PromptMessageTemplate `yaml:"templates"`
|
||||
OutputFormat OutputFormat `yaml:"output_format"`
|
||||
@@ -152,31 +160,54 @@ type PromptMessageTemplate struct {
|
||||
|
||||
// ExecutionProfile describes how and where to execute a model.
|
||||
type ExecutionProfile struct {
|
||||
ID string `yaml:"id"`
|
||||
Endpoint string `yaml:"endpoint"`
|
||||
Model string `yaml:"model"`
|
||||
Temperature float64 `yaml:"temperature"`
|
||||
MaxTokens int `yaml:"max_tokens"`
|
||||
TopP float64 `yaml:"top_p"`
|
||||
TimeoutSeconds int `yaml:"timeout_seconds"`
|
||||
ServiceTier string `yaml:"service_tier"`
|
||||
ReasoningEffort string `yaml:"reasoning_effort"`
|
||||
APIKeyEnv string `yaml:"api_key_env"`
|
||||
ExtraParams map[string]string `yaml:"extra_params"`
|
||||
ID string `yaml:"id"`
|
||||
Endpoint string `yaml:"endpoint"`
|
||||
Model string `yaml:"model"`
|
||||
Temperature float64 `yaml:"temperature"`
|
||||
MaxTokens int `yaml:"max_tokens"`
|
||||
TopP float64 `yaml:"top_p"`
|
||||
TimeoutSeconds int `yaml:"timeout_seconds"`
|
||||
ServiceTier string `yaml:"service_tier"`
|
||||
ReasoningEffort string `yaml:"reasoning_effort"`
|
||||
APIKeyEnv string `yaml:"api_key_env"`
|
||||
ExtraParams map[string]any `yaml:"extra_params"`
|
||||
}
|
||||
|
||||
// ExecutionTargetOverride represents per-request runtime setting overrides.
|
||||
type ExecutionTargetOverride struct {
|
||||
Endpoint string `json:"endpoint,omitempty"`
|
||||
Model string `json:"model,omitempty"`
|
||||
Temperature *float64 `json:"temperature,omitempty"`
|
||||
MaxTokens *int `json:"max_tokens,omitempty"`
|
||||
TopP *float64 `json:"top_p,omitempty"`
|
||||
TimeoutSeconds *int `json:"timeout_seconds,omitempty"`
|
||||
ServiceTier string `json:"service_tier,omitempty"`
|
||||
ReasoningEffort string `json:"reasoning_effort,omitempty"`
|
||||
APIKeyEnv string `json:"api_key_env,omitempty"`
|
||||
ExtraParams map[string]any `json:"extra_params,omitempty"`
|
||||
}
|
||||
|
||||
// ExecutionTargetPresence tracks which effective runtime fields came from an
|
||||
// explicit request override even when the resolved value is a zero value.
|
||||
type ExecutionTargetPresence struct {
|
||||
Temperature bool
|
||||
MaxTokens bool
|
||||
TopP bool
|
||||
TimeoutSeconds bool
|
||||
}
|
||||
|
||||
// ExecutionTarget represents effective model runtime settings for a run.
|
||||
type ExecutionTarget struct {
|
||||
Endpoint string `yaml:"endpoint" json:"endpoint"`
|
||||
Model string `yaml:"model" json:"model"`
|
||||
Temperature float64 `yaml:"temperature" json:"temperature"`
|
||||
MaxTokens int `yaml:"max_tokens" json:"max_tokens"`
|
||||
TopP float64 `yaml:"top_p" json:"top_p"`
|
||||
TimeoutSeconds int `yaml:"timeout_seconds" json:"timeout_seconds"`
|
||||
ServiceTier string `yaml:"service_tier" json:"service_tier"`
|
||||
ReasoningEffort string `yaml:"reasoning_effort" json:"reasoning_effort"`
|
||||
APIKeyEnv string `yaml:"api_key_env" json:"api_key_env"`
|
||||
ExtraParams map[string]string `yaml:"extra_params" json:"extra_params"`
|
||||
Endpoint string `yaml:"endpoint" json:"endpoint"`
|
||||
Model string `yaml:"model" json:"model"`
|
||||
Temperature float64 `yaml:"temperature" json:"temperature"`
|
||||
MaxTokens int `yaml:"max_tokens" json:"max_tokens"`
|
||||
TopP float64 `yaml:"top_p" json:"top_p"`
|
||||
TimeoutSeconds int `yaml:"timeout_seconds" json:"timeout_seconds"`
|
||||
ServiceTier string `yaml:"service_tier" json:"service_tier"`
|
||||
ReasoningEffort string `yaml:"reasoning_effort" json:"reasoning_effort"`
|
||||
APIKeyEnv string `yaml:"api_key_env" json:"api_key_env"`
|
||||
ExtraParams map[string]any `yaml:"extra_params" json:"extra_params"`
|
||||
}
|
||||
|
||||
// OutputContract defines the requirements for the output artifact.
|
||||
@@ -189,7 +220,8 @@ type OutputContract struct {
|
||||
|
||||
// RenderedPrompt represents the prompt after template application.
|
||||
type RenderedPrompt struct {
|
||||
Messages []RenderedMessage `json:"messages"`
|
||||
SessionID string `json:"session_id,omitempty"`
|
||||
Messages []RenderedMessage `json:"messages"`
|
||||
}
|
||||
|
||||
// RenderedMessage is a single message in a rendered prompt.
|
||||
@@ -203,6 +235,7 @@ type RenderedMessage struct {
|
||||
type GenerateRequest struct {
|
||||
Prompt RenderedPrompt
|
||||
Target ExecutionTarget
|
||||
TargetPresence ExecutionTargetPresence
|
||||
StructuredOutput *StructuredOutputSpec
|
||||
}
|
||||
|
||||
|
||||
@@ -102,3 +102,39 @@ func TestPreparedRunJSONIncludesMessageCacheControlOnlyWhenPresent(t *testing.T)
|
||||
t.Fatalf("expected second message to omit cache_control, got %#v", decoded.Messages[1])
|
||||
}
|
||||
}
|
||||
|
||||
func TestPreparedRunJSONIncludesSessionIDOnlyWhenPresent(t *testing.T) {
|
||||
prepared := PreparedRun{
|
||||
PromptID: "prompt.id",
|
||||
SelectedProfileID: "local-fast",
|
||||
EffectiveModelParams: ExecutionTarget{
|
||||
Endpoint: "http://llm/v1",
|
||||
Model: "gpt-test",
|
||||
},
|
||||
SessionID: "session-123",
|
||||
RenderedPromptHash: "rendered-hash",
|
||||
Messages: []RenderedMessage{{Role: "user", Content: "Summarize this."}},
|
||||
}
|
||||
|
||||
b, err := json.Marshal(prepared)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal failed: %v", err)
|
||||
}
|
||||
|
||||
var decoded map[string]any
|
||||
if err := json.Unmarshal(b, &decoded); err != nil {
|
||||
t.Fatalf("unmarshal failed: %v", err)
|
||||
}
|
||||
if decoded["session_id"] != "session-123" {
|
||||
t.Fatalf("expected session_id in prepared run JSON, got %#v", decoded["session_id"])
|
||||
}
|
||||
|
||||
prepared.SessionID = ""
|
||||
b, err = json.Marshal(prepared)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal failed: %v", err)
|
||||
}
|
||||
if strings.Contains(string(b), "session_id") {
|
||||
t.Fatalf("expected empty session_id to be omitted, got %s", b)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -96,6 +96,9 @@ func (textPreparedRunFormatter) Format(prepared *domain.PreparedRun) ([]byte, er
|
||||
if prepared.PromptHash != "" {
|
||||
fmt.Fprintf(&b, "prompt_hash: %s\n", prepared.PromptHash)
|
||||
}
|
||||
if prepared.SessionID != "" {
|
||||
fmt.Fprintf(&b, "session_id: %s\n", prepared.SessionID)
|
||||
}
|
||||
fmt.Fprintf(&b, "rendered_prompt_hash: %s\n", prepared.RenderedPromptHash)
|
||||
|
||||
target := prepared.EffectiveModelParams
|
||||
@@ -123,7 +126,11 @@ func (textPreparedRunFormatter) Format(prepared *domain.PreparedRun) ([]byte, er
|
||||
}
|
||||
sort.Strings(keys)
|
||||
for _, k := range keys {
|
||||
fmt.Fprintf(&b, " %s: %s\n", k, target.ExtraParams[k])
|
||||
renderedValue, err := formatExtraParamTextValue(target.ExtraParams[k])
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to format extra_params.%s: %w", k, err)
|
||||
}
|
||||
fmt.Fprintf(&b, " %s: %s\n", k, renderedValue)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -172,3 +179,15 @@ func (textPreparedRunFormatter) Format(prepared *domain.PreparedRun) ([]byte, er
|
||||
|
||||
return b.Bytes(), nil
|
||||
}
|
||||
|
||||
func formatExtraParamTextValue(value any) (string, error) {
|
||||
if s, ok := value.(string); ok {
|
||||
return s, nil
|
||||
}
|
||||
|
||||
b, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return string(b), nil
|
||||
}
|
||||
|
||||
@@ -49,6 +49,36 @@ func TestTextFormatterIncludesPreparedRunDetails(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTextFormatterRendersExtraParamsDeterministically(t *testing.T) {
|
||||
prepared := samplePreparedRun()
|
||||
prepared.EffectiveModelParams.ExtraParams = map[string]any{
|
||||
"z_string": "enabled",
|
||||
"b_number": 42,
|
||||
"a_object": map[string]any{
|
||||
"nested": "value",
|
||||
"count": 2,
|
||||
},
|
||||
"c_array": []any{"first", 3, false},
|
||||
}
|
||||
|
||||
out, err := FormatPreparedRun(prepared, PreparedRunFormatText)
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
s := string(out)
|
||||
|
||||
want := strings.Join([]string{
|
||||
" extra_params:",
|
||||
" a_object: {\"count\":2,\"nested\":\"value\"}",
|
||||
" b_number: 42",
|
||||
" c_array: [\"first\",3,false]",
|
||||
" z_string: enabled",
|
||||
}, "\n")
|
||||
if !strings.Contains(s, want) {
|
||||
t.Fatalf("expected deterministic extra_params block %q, got:\n%s", want, s)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTextFormatterDoesNotIncludeResolvedAPIKeyValue(t *testing.T) {
|
||||
const secret = "super-secret-api-key"
|
||||
t.Setenv("SCRIPTORIUM_API_KEY", secret)
|
||||
@@ -89,6 +119,19 @@ func TestTextFormatterIncludesMessageCacheControlBeforeContent(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTextFormatterIncludesSessionIDWhenPresent(t *testing.T) {
|
||||
prepared := samplePreparedRun()
|
||||
prepared.SessionID = "session-123"
|
||||
|
||||
out, err := FormatPreparedRun(prepared, PreparedRunFormatText)
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if !strings.Contains(string(out), "session_id: session-123\n") {
|
||||
t.Fatalf("expected session_id in text output, got:\n%s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTextFormatterOmitsEmptyCacheControlTTL(t *testing.T) {
|
||||
prepared := samplePreparedRun()
|
||||
prepared.Messages = []domain.RenderedMessage{
|
||||
@@ -116,6 +159,13 @@ func TestTextFormatterOmitsEmptyCacheControlTTL(t *testing.T) {
|
||||
|
||||
func TestJSONFormatterEmitsValidJSONAndIncludesPreparedRunFields(t *testing.T) {
|
||||
prepared := samplePreparedRun()
|
||||
prepared.SessionID = "session-123"
|
||||
prepared.EffectiveModelParams.ExtraParams = map[string]any{
|
||||
"number": 42,
|
||||
"nested": map[string]any{
|
||||
"enabled": true,
|
||||
},
|
||||
}
|
||||
|
||||
out, err := FormatPreparedRun(prepared, PreparedRunFormatJSON)
|
||||
if err != nil {
|
||||
@@ -139,9 +189,24 @@ func TestJSONFormatterEmitsValidJSONAndIncludesPreparedRunFields(t *testing.T) {
|
||||
if decoded["rendered_prompt_hash"] != "rendered-hash" {
|
||||
t.Fatalf("expected rendered_prompt_hash in json output, got %#v", decoded["rendered_prompt_hash"])
|
||||
}
|
||||
if _, ok := decoded["effective_model_params"]; !ok {
|
||||
if decoded["session_id"] != "session-123" {
|
||||
t.Fatalf("expected session_id in json output, got %#v", decoded["session_id"])
|
||||
}
|
||||
modelParams, ok := decoded["effective_model_params"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected effective_model_params in json output, got %#v", decoded)
|
||||
}
|
||||
extraParams, ok := modelParams["extra_params"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected extra_params in json output, got %#v", modelParams["extra_params"])
|
||||
}
|
||||
if extraParams["number"] != float64(42) {
|
||||
t.Fatalf("unexpected numeric extra param in json output: %#v", extraParams["number"])
|
||||
}
|
||||
nested, ok := extraParams["nested"].(map[string]any)
|
||||
if !ok || nested["enabled"] != true {
|
||||
t.Fatalf("unexpected nested extra param in json output: %#v", extraParams["nested"])
|
||||
}
|
||||
if _, ok := decoded["input_hashes"]; !ok {
|
||||
t.Fatalf("expected input_hashes in json output, got %#v", decoded)
|
||||
}
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||
@@ -89,7 +90,12 @@ func (c *OpenAICompatibleClient) Generate(ctx context.Context, req domain.Genera
|
||||
return nil, fmt.Errorf("%w: %v", ErrInvalidRequest, err)
|
||||
}
|
||||
|
||||
payload, err := json.Marshal(wireReq)
|
||||
wirePayload, err := openAIChatRequestPayload(wireReq)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %v", ErrInvalidRequest, err)
|
||||
}
|
||||
|
||||
payload, err := json.Marshal(wirePayload)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: failed to encode request: %v", ErrRequestFailed, err)
|
||||
}
|
||||
@@ -110,6 +116,8 @@ func (c *OpenAICompatibleClient) Generate(ctx context.Context, req domain.Genera
|
||||
effectiveTimeout := c.timeout
|
||||
if req.Target.TimeoutSeconds > 0 {
|
||||
effectiveTimeout = time.Duration(req.Target.TimeoutSeconds) * time.Second
|
||||
} else if req.TargetPresence.TimeoutSeconds {
|
||||
effectiveTimeout = 0
|
||||
}
|
||||
|
||||
httpClient := c.httpClient
|
||||
@@ -169,24 +177,36 @@ func openAIChatRequestFromGenerateRequest(req domain.GenerateRequest, defaultMod
|
||||
wireReq := openAIChatRequest{
|
||||
Model: model,
|
||||
}
|
||||
if sessionID := strings.TrimSpace(req.Prompt.SessionID); sessionID != "" {
|
||||
if n := utf8.RuneCountInString(sessionID); n > domain.SessionIDMaxLength {
|
||||
return openAIChatRequest{}, fmt.Errorf("session_id length %d exceeds maximum %d", n, domain.SessionIDMaxLength)
|
||||
}
|
||||
wireReq.SessionID = sessionID
|
||||
}
|
||||
|
||||
wireReq.Messages = make([]openAIChatRequestMessage, 0, len(req.Prompt.Messages))
|
||||
for _, msg := range req.Prompt.Messages {
|
||||
wireReq.Messages = append(wireReq.Messages, openAIChatRequestMessageFromRenderedMessage(msg))
|
||||
}
|
||||
|
||||
if req.Target.Temperature != 0 {
|
||||
if req.Target.Temperature != 0 || req.TargetPresence.Temperature {
|
||||
wireReq.Temperature = &req.Target.Temperature
|
||||
}
|
||||
if req.Target.MaxTokens != 0 {
|
||||
if req.Target.MaxTokens != 0 || req.TargetPresence.MaxTokens {
|
||||
wireReq.MaxTokens = &req.Target.MaxTokens
|
||||
}
|
||||
if req.Target.TopP != 0 {
|
||||
if req.Target.TopP != 0 || req.TargetPresence.TopP {
|
||||
wireReq.TopP = &req.Target.TopP
|
||||
}
|
||||
if strings.TrimSpace(req.Target.ServiceTier) != "" {
|
||||
wireReq.ServiceTier = req.Target.ServiceTier
|
||||
}
|
||||
if strings.TrimSpace(req.Target.ReasoningEffort) != "" {
|
||||
wireReq.ReasoningEffort = req.Target.ReasoningEffort
|
||||
}
|
||||
if len(req.Target.ExtraParams) > 0 {
|
||||
wireReq.ExtraParams = req.Target.ExtraParams
|
||||
}
|
||||
if req.StructuredOutput != nil {
|
||||
responseFormat, err := toOpenAIResponseFormat(req.StructuredOutput)
|
||||
if err != nil {
|
||||
@@ -199,13 +219,71 @@ func openAIChatRequestFromGenerateRequest(req domain.GenerateRequest, defaultMod
|
||||
}
|
||||
|
||||
type openAIChatRequest struct {
|
||||
Model string `json:"model"`
|
||||
Messages []openAIChatRequestMessage `json:"messages"`
|
||||
Temperature *float64 `json:"temperature,omitempty"`
|
||||
MaxTokens *int `json:"max_tokens,omitempty"`
|
||||
TopP *float64 `json:"top_p,omitempty"`
|
||||
ServiceTier string `json:"service_tier,omitempty"`
|
||||
ResponseFormat *openAIResponseFormat `json:"response_format,omitempty"`
|
||||
Model string `json:"model"`
|
||||
SessionID string `json:"session_id,omitempty"`
|
||||
Messages []openAIChatRequestMessage `json:"messages"`
|
||||
Temperature *float64 `json:"temperature,omitempty"`
|
||||
MaxTokens *int `json:"max_tokens,omitempty"`
|
||||
TopP *float64 `json:"top_p,omitempty"`
|
||||
ServiceTier string `json:"service_tier,omitempty"`
|
||||
ReasoningEffort string `json:"reasoning_effort,omitempty"`
|
||||
ResponseFormat *openAIResponseFormat `json:"response_format,omitempty"`
|
||||
ExtraParams map[string]any `json:"-"`
|
||||
}
|
||||
|
||||
func openAIChatRequestPayload(req openAIChatRequest) (map[string]any, error) {
|
||||
out := map[string]any{
|
||||
"model": req.Model,
|
||||
"messages": req.Messages,
|
||||
}
|
||||
if req.SessionID != "" {
|
||||
out["session_id"] = req.SessionID
|
||||
}
|
||||
if req.Temperature != nil {
|
||||
out["temperature"] = *req.Temperature
|
||||
}
|
||||
if req.MaxTokens != nil {
|
||||
out["max_tokens"] = *req.MaxTokens
|
||||
}
|
||||
if req.TopP != nil {
|
||||
out["top_p"] = *req.TopP
|
||||
}
|
||||
if req.ServiceTier != "" {
|
||||
out["service_tier"] = req.ServiceTier
|
||||
}
|
||||
if req.ReasoningEffort != "" {
|
||||
out["reasoning_effort"] = req.ReasoningEffort
|
||||
}
|
||||
if req.ResponseFormat != nil {
|
||||
out["response_format"] = req.ResponseFormat
|
||||
}
|
||||
|
||||
for key, value := range req.ExtraParams {
|
||||
if key == "" {
|
||||
return nil, errors.New("extra_params key must not be empty")
|
||||
}
|
||||
if _, reserved := reservedOpenAIChatRequestFields[key]; reserved {
|
||||
return nil, fmt.Errorf("extra_params key %q collides with reserved request field", key)
|
||||
}
|
||||
if _, err := json.Marshal(value); err != nil {
|
||||
return nil, fmt.Errorf("extra_params.%s must be JSON-serializable: %w", key, err)
|
||||
}
|
||||
out[key] = value
|
||||
}
|
||||
|
||||
return out, nil
|
||||
}
|
||||
|
||||
var reservedOpenAIChatRequestFields = map[string]struct{}{
|
||||
"model": {},
|
||||
"session_id": {},
|
||||
"messages": {},
|
||||
"temperature": {},
|
||||
"max_tokens": {},
|
||||
"top_p": {},
|
||||
"service_tier": {},
|
||||
"reasoning_effort": {},
|
||||
"response_format": {},
|
||||
}
|
||||
|
||||
type openAIChatRequestMessage struct {
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"math"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
@@ -181,7 +182,7 @@ func TestOpenAICompatibleClientSerializesCacheControlledMessageAsContentBlock(t
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
|
||||
for _, forbidden := range []string{"cache_control", "session_id", "extra_params"} {
|
||||
for _, forbidden := range []string{"cache_control", "extra_params"} {
|
||||
if _, exists := observedBody[forbidden]; exists {
|
||||
t.Fatalf("expected top-level %s to be omitted, got %#v", forbidden, observedBody[forbidden])
|
||||
}
|
||||
@@ -262,6 +263,96 @@ func TestOpenAICompatibleClientOmitsEmptyCacheControlTTL(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientSerializesSessionID(t *testing.T) {
|
||||
var observedBody map[string]any
|
||||
var observedSessionHeader string
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
observedSessionHeader = r.Header.Get("x-session-id")
|
||||
defer r.Body.Close()
|
||||
if err := json.NewDecoder(r.Body).Decode(&observedBody); err != nil {
|
||||
t.Fatalf("failed to decode request body: %v", err)
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: ts.URL + "/v1"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{
|
||||
SessionID: " session-123 ",
|
||||
Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}},
|
||||
},
|
||||
Target: domain.ExecutionTarget{Model: "model"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if observedBody["session_id"] != "session-123" {
|
||||
t.Fatalf("expected top-level session_id, got %#v", observedBody["session_id"])
|
||||
}
|
||||
if observedSessionHeader != "" {
|
||||
t.Fatalf("did not expect x-session-id header, got %q", observedSessionHeader)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientOmitsEmptySessionID(t *testing.T) {
|
||||
var observedBody map[string]any
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
defer r.Body.Close()
|
||||
if err := json.NewDecoder(r.Body).Decode(&observedBody); err != nil {
|
||||
t.Fatalf("failed to decode request body: %v", err)
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: ts.URL + "/v1"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{
|
||||
SessionID: " ",
|
||||
Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}},
|
||||
},
|
||||
Target: domain.ExecutionTarget{Model: "model"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if _, exists := observedBody["session_id"]; exists {
|
||||
t.Fatalf("expected empty session_id to be omitted, got %#v", observedBody["session_id"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientRejectsTooLongSessionID(t *testing.T) {
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||
BaseURL: "http://example.com/v1",
|
||||
Model: "model",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{
|
||||
SessionID: strings.Repeat("x", domain.SessionIDMaxLength+1),
|
||||
Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}},
|
||||
},
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("expected invalid request error")
|
||||
}
|
||||
if !errors.Is(err, ErrInvalidRequest) {
|
||||
t.Fatalf("expected ErrInvalidRequest, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientParsesCacheUsage(t *testing.T) {
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = w.Write([]byte(`{
|
||||
@@ -327,7 +418,7 @@ func TestOpenAICompatibleClientOmitsResponseFormatWhenNoStructuredOutput(t *test
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientOmitsReasoningEffortAndExtraParams(t *testing.T) {
|
||||
func TestOpenAICompatibleClientSerializesReasoningEffortAndExtraParams(t *testing.T) {
|
||||
var observedBody map[string]any
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
defer r.Body.Close()
|
||||
@@ -348,19 +439,257 @@ func TestOpenAICompatibleClientOmitsReasoningEffortAndExtraParams(t *testing.T)
|
||||
Target: domain.ExecutionTarget{
|
||||
Model: "model",
|
||||
ReasoningEffort: "high",
|
||||
ExtraParams: map[string]string{
|
||||
"provider_option": "on",
|
||||
ExtraParams: map[string]any{
|
||||
"string_value": "on",
|
||||
"number_value": 42,
|
||||
"boolean_value": true,
|
||||
"object_value": map[string]any{"nested": "value", "count": 2},
|
||||
"array_value": []any{"first", 3, false},
|
||||
},
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if observedBody["reasoning_effort"] != "high" {
|
||||
t.Fatalf("expected reasoning_effort high, got %#v", observedBody["reasoning_effort"])
|
||||
}
|
||||
if observedBody["string_value"] != "on" {
|
||||
t.Fatalf("unexpected string extra param: %#v", observedBody["string_value"])
|
||||
}
|
||||
if observedBody["number_value"] != float64(42) {
|
||||
t.Fatalf("unexpected number extra param: %#v", observedBody["number_value"])
|
||||
}
|
||||
if observedBody["boolean_value"] != true {
|
||||
t.Fatalf("unexpected boolean extra param: %#v", observedBody["boolean_value"])
|
||||
}
|
||||
objectValue, ok := observedBody["object_value"].(map[string]any)
|
||||
if !ok || objectValue["nested"] != "value" || objectValue["count"] != float64(2) {
|
||||
t.Fatalf("unexpected object extra param: %#v", observedBody["object_value"])
|
||||
}
|
||||
if _, exists := observedBody["extra_params"]; exists {
|
||||
t.Fatalf("expected extra_params wrapper omitted, got %#v", observedBody["extra_params"])
|
||||
}
|
||||
arrayValue, ok := observedBody["array_value"].([]any)
|
||||
if !ok || len(arrayValue) != 3 || arrayValue[0] != "first" || arrayValue[1] != float64(3) || arrayValue[2] != false {
|
||||
t.Fatalf("unexpected array extra param: %#v", observedBody["array_value"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientOmitsReasoningEffortWhenUnset(t *testing.T) {
|
||||
var observedBody map[string]any
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
defer r.Body.Close()
|
||||
if err := json.NewDecoder(r.Body).Decode(&observedBody); err != nil {
|
||||
t.Fatalf("failed to decode request body: %v", err)
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: ts.URL + "/v1"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||
Target: domain.ExecutionTarget{Model: "model"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if _, exists := observedBody["reasoning_effort"]; exists {
|
||||
t.Fatalf("expected reasoning_effort omitted, got %#v", observedBody["reasoning_effort"])
|
||||
}
|
||||
if _, exists := observedBody["extra_params"]; exists {
|
||||
t.Fatalf("expected extra_params omitted, got %#v", observedBody["extra_params"])
|
||||
t.Fatalf("expected extra_params wrapper omitted, got %#v", observedBody["extra_params"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientSerializesExplicitZeroNumericOverrides(t *testing.T) {
|
||||
var observedBody map[string]any
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
defer r.Body.Close()
|
||||
if err := json.NewDecoder(r.Body).Decode(&observedBody); err != nil {
|
||||
t.Fatalf("failed to decode request body: %v", err)
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: ts.URL + "/v1"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||
Target: domain.ExecutionTarget{Model: "model"},
|
||||
TargetPresence: domain.ExecutionTargetPresence{
|
||||
Temperature: true,
|
||||
MaxTokens: true,
|
||||
TopP: true,
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if observedBody["temperature"] != float64(0) {
|
||||
t.Fatalf("expected explicit zero temperature, got %#v", observedBody["temperature"])
|
||||
}
|
||||
if observedBody["max_tokens"] != float64(0) {
|
||||
t.Fatalf("expected explicit zero max_tokens, got %#v", observedBody["max_tokens"])
|
||||
}
|
||||
if observedBody["top_p"] != float64(0) {
|
||||
t.Fatalf("expected explicit zero top_p, got %#v", observedBody["top_p"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientOmitsImplicitZeroNumericFields(t *testing.T) {
|
||||
var observedBody map[string]any
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
defer r.Body.Close()
|
||||
if err := json.NewDecoder(r.Body).Decode(&observedBody); err != nil {
|
||||
t.Fatalf("failed to decode request body: %v", err)
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: ts.URL + "/v1"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||
Target: domain.ExecutionTarget{Model: "model"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
for _, field := range []string{"temperature", "max_tokens", "top_p"} {
|
||||
if _, exists := observedBody[field]; exists {
|
||||
t.Fatalf("expected implicit zero field %q to be omitted, got body %#v", field, observedBody)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientExplicitZeroTimeoutDisablesClientTimeout(t *testing.T) {
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||
BaseURL: ts.URL + "/v1",
|
||||
Timeout: time.Nanosecond,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||
Target: domain.ExecutionTarget{Model: "model", TimeoutSeconds: 0},
|
||||
TargetPresence: domain.ExecutionTargetPresence{TimeoutSeconds: true},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected explicit zero timeout to disable client timeout, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientOmittedTimeoutUsesClientTimeout(t *testing.T) {
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||
BaseURL: ts.URL + "/v1",
|
||||
Timeout: time.Nanosecond,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||
Target: domain.ExecutionTarget{Model: "model", TimeoutSeconds: 0},
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("expected omitted timeout to use client timeout")
|
||||
}
|
||||
if !errors.Is(err, ErrRequestFailed) {
|
||||
t.Fatalf("expected ErrRequestFailed, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientRejectsInvalidExtraParamsBeforeProviderCall(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
extraParams map[string]any
|
||||
want string
|
||||
}{
|
||||
{name: "empty key", extraParams: map[string]any{"": "empty"}, want: "key must not be empty"},
|
||||
{name: "unserializable value", extraParams: map[string]any{"bad": math.Inf(1)}, want: "JSON-serializable"},
|
||||
}
|
||||
for _, key := range []string{
|
||||
"model",
|
||||
"session_id",
|
||||
"messages",
|
||||
"temperature",
|
||||
"max_tokens",
|
||||
"top_p",
|
||||
"service_tier",
|
||||
"reasoning_effort",
|
||||
"response_format",
|
||||
} {
|
||||
tests = append(tests, struct {
|
||||
name string
|
||||
extraParams map[string]any
|
||||
want string
|
||||
}{
|
||||
name: "reserved key " + key,
|
||||
extraParams: map[string]any{key: "collision"},
|
||||
want: "reserved request field",
|
||||
})
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
called := false
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
called = true
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: ts.URL + "/v1"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||
Target: domain.ExecutionTarget{Model: "model", ExtraParams: tc.extraParams},
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("expected invalid request error")
|
||||
}
|
||||
if !errors.Is(err, ErrInvalidRequest) {
|
||||
t.Fatalf("expected ErrInvalidRequest, got %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), tc.want) {
|
||||
t.Fatalf("expected error to contain %q, got %v", tc.want, err)
|
||||
}
|
||||
if called {
|
||||
t.Fatal("provider should not be called for invalid extra_params")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ package profile
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@@ -85,6 +86,63 @@ temperature: 0.1
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid profile with JSON-compatible extra params", func(t *testing.T) {
|
||||
writeProfileTestFile(t, filepath.Join(tmpDir, "json-extra-params.yaml"), `
|
||||
id: json-extra-params
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: nested-model
|
||||
extra_params:
|
||||
string_value: enabled
|
||||
number_value: 42
|
||||
boolean_value: true
|
||||
object_value:
|
||||
nested: value
|
||||
count: 2
|
||||
array_value:
|
||||
- first
|
||||
- 3
|
||||
- false
|
||||
`)
|
||||
|
||||
p, err := repo.GetProfile(ctx, "json-extra-params")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
|
||||
var got map[string]any
|
||||
encoded, err := json.Marshal(p.ExtraParams)
|
||||
if err != nil {
|
||||
t.Fatalf("expected extra_params to marshal as JSON, got %v", err)
|
||||
}
|
||||
if err := json.Unmarshal(encoded, &got); err != nil {
|
||||
t.Fatalf("expected extra_params JSON to decode, got %v", err)
|
||||
}
|
||||
|
||||
if got["string_value"] != "enabled" {
|
||||
t.Fatalf("unexpected string extra param: %#v", got["string_value"])
|
||||
}
|
||||
if got["number_value"] != float64(42) {
|
||||
t.Fatalf("unexpected number extra param: %#v", got["number_value"])
|
||||
}
|
||||
if got["boolean_value"] != true {
|
||||
t.Fatalf("unexpected boolean extra param: %#v", got["boolean_value"])
|
||||
}
|
||||
objectValue, ok := got["object_value"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected object extra param, got %#v", got["object_value"])
|
||||
}
|
||||
if objectValue["nested"] != "value" || objectValue["count"] != float64(2) {
|
||||
t.Fatalf("unexpected object extra param: %#v", objectValue)
|
||||
}
|
||||
arrayValue, ok := got["array_value"].([]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected array extra param, got %#v", got["array_value"])
|
||||
}
|
||||
if len(arrayValue) != 3 || arrayValue[0] != "first" || arrayValue[1] != float64(3) || arrayValue[2] != false {
|
||||
t.Fatalf("unexpected array extra param: %#v", arrayValue)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("duplicate profile IDs fail as ambiguous", func(t *testing.T) {
|
||||
writeProfileTestFile(t, filepath.Join(tmpDir, "duplicate-profile-a.yaml"), `
|
||||
id: duplicate-profile
|
||||
|
||||
@@ -6,7 +6,9 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||
"strings"
|
||||
"text/template"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
var (
|
||||
@@ -50,6 +52,11 @@ func (r *goRenderer) Render(ctx context.Context, definition *domain.PromptDefini
|
||||
},
|
||||
}
|
||||
|
||||
sessionID, err := renderSessionID(definition.SessionID, funcs, vars)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var renderedMessages []domain.RenderedMessage
|
||||
|
||||
for i, tmplMsg := range definition.Templates {
|
||||
@@ -82,10 +89,33 @@ func (r *goRenderer) Render(ctx context.Context, definition *domain.PromptDefini
|
||||
}
|
||||
|
||||
return &domain.RenderedPrompt{
|
||||
Messages: renderedMessages,
|
||||
SessionID: sessionID,
|
||||
Messages: renderedMessages,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func renderSessionID(raw string, funcs template.FuncMap, vars map[string]string) (string, error) {
|
||||
if strings.TrimSpace(raw) == "" {
|
||||
return "", nil
|
||||
}
|
||||
|
||||
tmpl, err := template.New("session_id").Funcs(funcs).Option("missingkey=error").Parse(raw)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("%w: session_id: %v", ErrInvalidTemplate, err)
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
if err := tmpl.Execute(&buf, vars); err != nil {
|
||||
return "", fmt.Errorf("%w: session_id: %w", ErrRenderFailure, err)
|
||||
}
|
||||
|
||||
sessionID := strings.TrimSpace(buf.String())
|
||||
if n := utf8.RuneCountInString(sessionID); n > domain.SessionIDMaxLength {
|
||||
return "", fmt.Errorf("%w: session_id length %d exceeds maximum %d", ErrRenderFailure, n, domain.SessionIDMaxLength)
|
||||
}
|
||||
return sessionID, nil
|
||||
}
|
||||
|
||||
func cloneCacheControl(in *domain.CacheControl) *domain.CacheControl {
|
||||
if in == nil {
|
||||
return nil
|
||||
|
||||
@@ -3,6 +3,7 @@ package prompt
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||
@@ -155,6 +156,78 @@ func TestGoRenderer_Render(t *testing.T) {
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rendering session id from vars", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
SessionID: " {{ .session_id }} ",
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "Speak in a {{.tone}} tone."},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, map[string]string{
|
||||
"tone": "concise",
|
||||
"session_id": "agent-session-123",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if res.SessionID != "agent-session-123" {
|
||||
t.Fatalf("unexpected session id: %q", res.SessionID)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("empty rendered session id is omitted", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
SessionID: " ",
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "Speak in a {{.tone}} tone."},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if res.SessionID != "" {
|
||||
t.Fatalf("expected empty session id, got %q", res.SessionID)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("missing session id var fails rendering", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
SessionID: "{{ .session_id }}",
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "Speak in a {{.tone}} tone."},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if !errors.Is(err, ErrRenderFailure) {
|
||||
t.Fatalf("expected ErrRenderFailure, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("too long rendered session id fails rendering", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
SessionID: "{{ .session_id }}",
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "Speak in a {{.tone}} tone."},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := renderer.Render(ctx, def, inputs, map[string]string{
|
||||
"tone": "concise",
|
||||
"session_id": strings.Repeat("x", domain.SessionIDMaxLength+1),
|
||||
})
|
||||
if !errors.Is(err, ErrRenderFailure) {
|
||||
t.Fatalf("expected ErrRenderFailure, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("inserting required input artifact", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
|
||||
@@ -29,6 +29,7 @@ type promptDefinitionFile struct {
|
||||
Version string `yaml:"version"`
|
||||
DefaultProfile *string `yaml:"default_profile"`
|
||||
Description string `yaml:"description"`
|
||||
SessionID string `yaml:"session_id"`
|
||||
Inputs []promptInputFile `yaml:"inputs"`
|
||||
Messages []promptMessageFile `yaml:"messages"`
|
||||
Output promptOutputContractFile `yaml:"output"`
|
||||
@@ -274,6 +275,7 @@ func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*d
|
||||
Version: version,
|
||||
DefaultProfile: defaultProfile,
|
||||
Description: strings.TrimSpace(raw.Description),
|
||||
SessionID: strings.TrimSpace(raw.SessionID),
|
||||
Inputs: inputs,
|
||||
Templates: templates,
|
||||
OutputFormat: raw.Output.Format,
|
||||
|
||||
@@ -96,6 +96,16 @@ func TestFilesystemRepository_GetPromptDefinition(t *testing.T) {
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid session id template", func(t *testing.T) {
|
||||
p, err := repo.GetPromptDefinition(ctx, "valid-session-id", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.SessionID != "{{ .session_id }}" {
|
||||
t.Fatalf("expected trimmed session_id template, got %q", p.SessionID)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid nested file-backed prompt resolves content file relative to nested YAML", func(t *testing.T) {
|
||||
nestedDir := filepath.Join(tmpDir, "dnd", "recap")
|
||||
if err := os.MkdirAll(nestedDir, 0o755); err != nil {
|
||||
|
||||
10
internal/promptdef/testdata/valid_session_id.yaml
vendored
Normal file
10
internal/promptdef/testdata/valid_session_id.yaml
vendored
Normal file
@@ -0,0 +1,10 @@
|
||||
id: valid-session-id
|
||||
version: "1.0.0"
|
||||
session_id: " {{ .session_id }} "
|
||||
messages:
|
||||
- role: user
|
||||
content: Hello.
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
@@ -90,11 +90,15 @@ func (r *Runner) Run(ctx context.Context, req domain.RunRequest) (*domain.RunRes
|
||||
}
|
||||
|
||||
genResp, err := r.llm.Generate(ctx, domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{Messages: prepared.Messages},
|
||||
Prompt: domain.RenderedPrompt{SessionID: prepared.SessionID, Messages: prepared.Messages},
|
||||
Target: prepared.EffectiveModelParams,
|
||||
TargetPresence: prepared.TargetPresence,
|
||||
StructuredOutput: prepared.StructuredOutput,
|
||||
})
|
||||
if err != nil {
|
||||
if errors.Is(err, llm.ErrInvalidRequest) {
|
||||
return nil, fmt.Errorf("%w: %w", ErrInvalidRequest, err)
|
||||
}
|
||||
return nil, fmt.Errorf("%w: %w", ErrLLMGenerate, err)
|
||||
}
|
||||
|
||||
@@ -187,7 +191,10 @@ func (r *Runner) Prepare(ctx context.Context, req domain.RunRequest) (*domain.Pr
|
||||
return nil, fmt.Errorf("%w: %w", ErrProfileLoad, err)
|
||||
}
|
||||
|
||||
effectiveModel := resolveExecutionTarget(execProfile, req.Execution)
|
||||
effectiveModel, targetPresence, err := resolveExecutionTarget(execProfile, req.Execution)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: %w", ErrInvalidRequest, err)
|
||||
}
|
||||
if strings.TrimSpace(effectiveModel.Endpoint) == "" {
|
||||
return nil, fmt.Errorf("%w: execution endpoint is required", ErrInvalidRequest)
|
||||
}
|
||||
@@ -230,9 +237,11 @@ func (r *Runner) Prepare(ctx context.Context, req domain.RunRequest) (*domain.Pr
|
||||
PromptHash: promptDefinitionHash,
|
||||
SelectedProfileID: selectedProfileID,
|
||||
EffectiveModelParams: effectiveModel,
|
||||
TargetPresence: targetPresence,
|
||||
OutputContract: effectiveContract,
|
||||
StructuredOutput: structuredOutput,
|
||||
InputHashes: inputHashes,
|
||||
SessionID: renderedPrompt.SessionID,
|
||||
RenderedPromptHash: hashRenderedPrompt(*renderedPrompt),
|
||||
Messages: renderedPrompt.Messages,
|
||||
StartTime: start,
|
||||
@@ -354,22 +363,75 @@ func mergeExecutionTarget(base domain.ExecutionTarget, override domain.Execution
|
||||
out.APIKeyEnv = override.APIKeyEnv
|
||||
}
|
||||
if len(override.ExtraParams) > 0 {
|
||||
cp := make(map[string]string, len(override.ExtraParams))
|
||||
for k, v := range override.ExtraParams {
|
||||
cp[k] = v
|
||||
}
|
||||
out.ExtraParams = cp
|
||||
out.ExtraParams = copyExtraParams(override.ExtraParams)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func resolveExecutionTarget(profileValue *domain.ExecutionProfile, override *domain.ExecutionTarget) domain.ExecutionTarget {
|
||||
func mergeExecutionTargetOverride(base domain.ExecutionTarget, override domain.ExecutionTargetOverride) (domain.ExecutionTarget, domain.ExecutionTargetPresence, error) {
|
||||
out := base
|
||||
var presence domain.ExecutionTargetPresence
|
||||
if override.Endpoint != "" {
|
||||
out.Endpoint = override.Endpoint
|
||||
}
|
||||
if override.Model != "" {
|
||||
out.Model = override.Model
|
||||
}
|
||||
if override.Temperature != nil {
|
||||
if *override.Temperature < 0 || *override.Temperature > 2 {
|
||||
return domain.ExecutionTarget{}, domain.ExecutionTargetPresence{}, errors.New("temperature must be between 0 and 2")
|
||||
}
|
||||
out.Temperature = *override.Temperature
|
||||
presence.Temperature = true
|
||||
}
|
||||
if override.MaxTokens != nil {
|
||||
if *override.MaxTokens < 0 {
|
||||
return domain.ExecutionTarget{}, domain.ExecutionTargetPresence{}, errors.New("max_tokens must be greater than or equal to 0")
|
||||
}
|
||||
out.MaxTokens = *override.MaxTokens
|
||||
presence.MaxTokens = true
|
||||
}
|
||||
if override.TopP != nil {
|
||||
if *override.TopP < 0 || *override.TopP > 1 {
|
||||
return domain.ExecutionTarget{}, domain.ExecutionTargetPresence{}, errors.New("top_p must be between 0 and 1")
|
||||
}
|
||||
out.TopP = *override.TopP
|
||||
presence.TopP = true
|
||||
}
|
||||
if override.TimeoutSeconds != nil {
|
||||
if *override.TimeoutSeconds < 0 {
|
||||
return domain.ExecutionTarget{}, domain.ExecutionTargetPresence{}, errors.New("timeout_seconds must be greater than or equal to 0")
|
||||
}
|
||||
out.TimeoutSeconds = *override.TimeoutSeconds
|
||||
presence.TimeoutSeconds = true
|
||||
}
|
||||
if strings.TrimSpace(override.ServiceTier) != "" {
|
||||
out.ServiceTier = override.ServiceTier
|
||||
}
|
||||
if strings.TrimSpace(override.ReasoningEffort) != "" {
|
||||
out.ReasoningEffort = override.ReasoningEffort
|
||||
}
|
||||
if strings.TrimSpace(override.APIKeyEnv) != "" {
|
||||
out.APIKeyEnv = override.APIKeyEnv
|
||||
}
|
||||
if len(override.ExtraParams) > 0 {
|
||||
out.ExtraParams = copyExtraParams(override.ExtraParams)
|
||||
}
|
||||
return out, presence, nil
|
||||
}
|
||||
|
||||
func resolveExecutionTarget(profileValue *domain.ExecutionProfile, override *domain.ExecutionTargetOverride) (domain.ExecutionTarget, domain.ExecutionTargetPresence, error) {
|
||||
out := defaults.ExecutionTargetDefault()
|
||||
out = mergeExecutionTarget(out, executionProfileToTarget(profileValue))
|
||||
var presence domain.ExecutionTargetPresence
|
||||
if override != nil {
|
||||
out = mergeExecutionTarget(out, *override)
|
||||
var err error
|
||||
out, presence, err = mergeExecutionTargetOverride(out, *override)
|
||||
if err != nil {
|
||||
return domain.ExecutionTarget{}, domain.ExecutionTargetPresence{}, err
|
||||
}
|
||||
}
|
||||
return out
|
||||
return out, presence, nil
|
||||
}
|
||||
|
||||
func validateAPIKeyEnv(apiKeyEnv string) error {
|
||||
@@ -387,13 +449,6 @@ func executionProfileToTarget(p *domain.ExecutionProfile) domain.ExecutionTarget
|
||||
if p == nil {
|
||||
return domain.ExecutionTarget{}
|
||||
}
|
||||
cp := map[string]string(nil)
|
||||
if len(p.ExtraParams) > 0 {
|
||||
cp = make(map[string]string, len(p.ExtraParams))
|
||||
for k, v := range p.ExtraParams {
|
||||
cp[k] = v
|
||||
}
|
||||
}
|
||||
return domain.ExecutionTarget{
|
||||
Endpoint: p.Endpoint,
|
||||
Model: p.Model,
|
||||
@@ -404,10 +459,21 @@ func executionProfileToTarget(p *domain.ExecutionProfile) domain.ExecutionTarget
|
||||
ServiceTier: p.ServiceTier,
|
||||
ReasoningEffort: p.ReasoningEffort,
|
||||
APIKeyEnv: p.APIKeyEnv,
|
||||
ExtraParams: cp,
|
||||
ExtraParams: copyExtraParams(p.ExtraParams),
|
||||
}
|
||||
}
|
||||
|
||||
func copyExtraParams(src map[string]any) map[string]any {
|
||||
if len(src) == 0 {
|
||||
return nil
|
||||
}
|
||||
cp := make(map[string]any, len(src))
|
||||
for k, v := range src {
|
||||
cp[k] = v
|
||||
}
|
||||
return cp
|
||||
}
|
||||
|
||||
func resolveOutputContract(def *domain.PromptDefinition, override *domain.OutputContract) domain.OutputContract {
|
||||
contract := def.Validation
|
||||
if contract.Format == "" {
|
||||
@@ -424,6 +490,11 @@ func resolveOutputContract(def *domain.PromptDefinition, override *domain.Output
|
||||
|
||||
func hashRenderedPrompt(p domain.RenderedPrompt) string {
|
||||
var b strings.Builder
|
||||
if p.SessionID != "" {
|
||||
b.WriteString("session_id=")
|
||||
b.WriteString(p.SessionID)
|
||||
b.WriteString("\n---\n")
|
||||
}
|
||||
for _, msg := range p.Messages {
|
||||
b.WriteString(msg.Role)
|
||||
b.WriteByte('\n')
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/llm"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/profile"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/prompt"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/promptdef"
|
||||
@@ -160,7 +161,7 @@ func TestRunnerPrepareWithExplicitProfileSelection(t *testing.T) {
|
||||
"a://t": {Body: []byte("transcript"), Hash: hashString("transcript")},
|
||||
"a://g": {Body: []byte("glossary"), Hash: hashString("glossary")},
|
||||
}}
|
||||
renderer := &fakeRenderer{rendered: &domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "system", Content: "sys"}, {Role: "user", Content: "usr"}}}}
|
||||
renderer := &fakeRenderer{rendered: &domain.RenderedPrompt{SessionID: "session-123", Messages: []domain.RenderedMessage{{Role: "system", Content: "sys"}, {Role: "user", Content: "usr"}}}}
|
||||
llmClient := &fakeLLM{forbid: true}
|
||||
|
||||
runner := NewRunner(promptRepo, execRepo, reader, renderer, llmClient, nil)
|
||||
@@ -172,7 +173,7 @@ func TestRunnerPrepareWithExplicitProfileSelection(t *testing.T) {
|
||||
"transcript": {Type: domain.ArtifactRefFile, URI: "a://t"},
|
||||
"glossary": {Type: domain.ArtifactRefFile, URI: "a://g"},
|
||||
},
|
||||
Execution: &domain.ExecutionTarget{Endpoint: "http://override/v1", Model: "m", Temperature: 0.3, TimeoutSeconds: 90},
|
||||
Execution: &domain.ExecutionTargetOverride{Endpoint: "http://override/v1", Model: "m", Temperature: float64Ptr(0.3), TimeoutSeconds: intPtr(90)},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
@@ -198,6 +199,9 @@ func TestRunnerPrepareWithExplicitProfileSelection(t *testing.T) {
|
||||
if len(prepared.Messages) != 2 {
|
||||
t.Fatalf("expected two messages, got %d", len(prepared.Messages))
|
||||
}
|
||||
if prepared.SessionID != "session-123" {
|
||||
t.Fatalf("expected prepared session id, got %q", prepared.SessionID)
|
||||
}
|
||||
if llmClient.calls != 0 {
|
||||
t.Fatalf("prepare should not call llm, calls=%d", llmClient.calls)
|
||||
}
|
||||
@@ -269,11 +273,11 @@ func TestRunnerPrepareRuntimeOverrideBeatsSelectedProfileValue(t *testing.T) {
|
||||
PromptID: "p",
|
||||
ProfileID: "exec",
|
||||
Inputs: singleInputRef(),
|
||||
Execution: &domain.ExecutionTarget{
|
||||
Execution: &domain.ExecutionTargetOverride{
|
||||
Endpoint: "http://override/v1",
|
||||
Model: "override-model",
|
||||
Temperature: 0.7,
|
||||
TimeoutSeconds: 30,
|
||||
Temperature: float64Ptr(0.7),
|
||||
TimeoutSeconds: intPtr(30),
|
||||
ServiceTier: "flex",
|
||||
},
|
||||
})
|
||||
@@ -291,6 +295,143 @@ func TestRunnerPrepareRuntimeOverrideBeatsSelectedProfileValue(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunnerPrepareRequestNumericOverridePresence(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
override *domain.ExecutionTargetOverride
|
||||
wantTemperature float64
|
||||
wantMaxTokens int
|
||||
wantTopP float64
|
||||
wantTimeoutSecs int
|
||||
wantPresence domain.ExecutionTargetPresence
|
||||
}{
|
||||
{
|
||||
name: "omitted preserves profile values",
|
||||
override: &domain.ExecutionTargetOverride{},
|
||||
wantTemperature: 0.7,
|
||||
wantMaxTokens: 321,
|
||||
wantTopP: 0.8,
|
||||
wantTimeoutSecs: 45,
|
||||
},
|
||||
{
|
||||
name: "explicit zero temperature",
|
||||
override: &domain.ExecutionTargetOverride{Temperature: float64Ptr(0)},
|
||||
wantTemperature: 0,
|
||||
wantMaxTokens: 321,
|
||||
wantTopP: 0.8,
|
||||
wantTimeoutSecs: 45,
|
||||
wantPresence: domain.ExecutionTargetPresence{Temperature: true},
|
||||
},
|
||||
{
|
||||
name: "explicit zero max tokens",
|
||||
override: &domain.ExecutionTargetOverride{MaxTokens: intPtr(0)},
|
||||
wantTemperature: 0.7,
|
||||
wantMaxTokens: 0,
|
||||
wantTopP: 0.8,
|
||||
wantTimeoutSecs: 45,
|
||||
wantPresence: domain.ExecutionTargetPresence{MaxTokens: true},
|
||||
},
|
||||
{
|
||||
name: "explicit zero top p",
|
||||
override: &domain.ExecutionTargetOverride{TopP: float64Ptr(0)},
|
||||
wantTemperature: 0.7,
|
||||
wantMaxTokens: 321,
|
||||
wantTopP: 0,
|
||||
wantTimeoutSecs: 45,
|
||||
wantPresence: domain.ExecutionTargetPresence{TopP: true},
|
||||
},
|
||||
{
|
||||
name: "explicit zero timeout",
|
||||
override: &domain.ExecutionTargetOverride{TimeoutSeconds: intPtr(0)},
|
||||
wantTemperature: 0.7,
|
||||
wantMaxTokens: 321,
|
||||
wantTopP: 0.8,
|
||||
wantTimeoutSecs: 0,
|
||||
wantPresence: domain.ExecutionTargetPresence{TimeoutSeconds: true},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
runner := NewRunner(
|
||||
&fakePromptRepo{def: promptDef(domain.FormatText, domain.ValidationNone, 0)},
|
||||
&fakeExecutionProfileRepo{profiles: map[string]*domain.ExecutionProfile{
|
||||
"exec": {
|
||||
ID: "exec",
|
||||
Endpoint: "http://profile/v1",
|
||||
Model: "profile-model",
|
||||
Temperature: 0.7,
|
||||
MaxTokens: 321,
|
||||
TopP: 0.8,
|
||||
TimeoutSeconds: 45,
|
||||
},
|
||||
}},
|
||||
defaultArtifactReader(),
|
||||
defaultRenderer(),
|
||||
&fakeLLM{forbid: true},
|
||||
nil,
|
||||
)
|
||||
|
||||
prepared, err := runner.Prepare(context.Background(), domain.RunRequest{
|
||||
PromptID: "p",
|
||||
ProfileID: "exec",
|
||||
Inputs: singleInputRef(),
|
||||
Execution: tc.override,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
got := prepared.EffectiveModelParams
|
||||
if got.Temperature != tc.wantTemperature ||
|
||||
got.MaxTokens != tc.wantMaxTokens ||
|
||||
got.TopP != tc.wantTopP ||
|
||||
got.TimeoutSeconds != tc.wantTimeoutSecs {
|
||||
t.Fatalf("unexpected effective numeric settings: %+v", got)
|
||||
}
|
||||
if prepared.TargetPresence != tc.wantPresence {
|
||||
t.Fatalf("unexpected target presence: got %+v want %+v", prepared.TargetPresence, tc.wantPresence)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunnerPrepareInvalidRequestNumericOverridesFail(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
override *domain.ExecutionTargetOverride
|
||||
}{
|
||||
{name: "temperature below range", override: &domain.ExecutionTargetOverride{Temperature: float64Ptr(-0.1)}},
|
||||
{name: "temperature above range", override: &domain.ExecutionTargetOverride{Temperature: float64Ptr(2.1)}},
|
||||
{name: "max tokens below range", override: &domain.ExecutionTargetOverride{MaxTokens: intPtr(-1)}},
|
||||
{name: "top p below range", override: &domain.ExecutionTargetOverride{TopP: float64Ptr(-0.1)}},
|
||||
{name: "top p above range", override: &domain.ExecutionTargetOverride{TopP: float64Ptr(1.1)}},
|
||||
{name: "timeout below range", override: &domain.ExecutionTargetOverride{TimeoutSeconds: intPtr(-1)}},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
runner := NewRunner(
|
||||
&fakePromptRepo{def: promptDef(domain.FormatText, domain.ValidationNone, 0)},
|
||||
&fakeExecutionProfileRepo{profiles: map[string]*domain.ExecutionProfile{"exec": defaultExecutionProfile()}},
|
||||
defaultArtifactReader(),
|
||||
defaultRenderer(),
|
||||
&fakeLLM{forbid: true},
|
||||
nil,
|
||||
)
|
||||
|
||||
_, err := runner.Prepare(context.Background(), domain.RunRequest{
|
||||
PromptID: "p",
|
||||
ProfileID: "exec",
|
||||
Inputs: singleInputRef(),
|
||||
Execution: tc.override,
|
||||
})
|
||||
if !errors.Is(err, ErrInvalidRequest) {
|
||||
t.Fatalf("expected ErrInvalidRequest, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunnerPrepareSelectedProfileBeatsBuiltInDefault(t *testing.T) {
|
||||
promptRepo := &fakePromptRepo{def: promptDef(domain.FormatText, domain.ValidationNone, 0)}
|
||||
execRepo := &fakeExecutionProfileRepo{profiles: map[string]*domain.ExecutionProfile{
|
||||
@@ -632,6 +773,45 @@ func TestHashRenderedPromptIncludesCacheControlWhenPresent(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestHashRenderedPromptIncludesSessionIDWhenPresent(t *testing.T) {
|
||||
withoutSession := domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
||||
{Role: "system", Content: "sys"},
|
||||
{Role: "user", Content: "usr"},
|
||||
}}
|
||||
withSession := domain.RenderedPrompt{
|
||||
SessionID: "session-123",
|
||||
Messages: []domain.RenderedMessage{
|
||||
{Role: "system", Content: "sys"},
|
||||
{Role: "user", Content: "usr"},
|
||||
},
|
||||
}
|
||||
alsoWithSession := domain.RenderedPrompt{
|
||||
SessionID: "session-123",
|
||||
Messages: []domain.RenderedMessage{
|
||||
{Role: "system", Content: "sys"},
|
||||
{Role: "user", Content: "usr"},
|
||||
},
|
||||
}
|
||||
otherSession := domain.RenderedPrompt{
|
||||
SessionID: "session-456",
|
||||
Messages: []domain.RenderedMessage{
|
||||
{Role: "system", Content: "sys"},
|
||||
{Role: "user", Content: "usr"},
|
||||
},
|
||||
}
|
||||
|
||||
sessionHash := hashRenderedPrompt(withSession)
|
||||
if sessionHash == hashRenderedPrompt(withoutSession) {
|
||||
t.Fatal("expected session_id to change rendered prompt hash")
|
||||
}
|
||||
if sessionHash != hashRenderedPrompt(alsoWithSession) {
|
||||
t.Fatal("expected identical session_id to produce stable hash")
|
||||
}
|
||||
if sessionHash == hashRenderedPrompt(otherSession) {
|
||||
t.Fatal("expected session_id value changes to affect rendered prompt hash")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunnerRunSuccessful(t *testing.T) {
|
||||
promptRepo := &fakePromptRepo{def: promptDef(domain.FormatMarkdown, domain.ValidationBasic, 0)}
|
||||
execRepo := &fakeExecutionProfileRepo{profiles: map[string]*domain.ExecutionProfile{"exec": defaultExecutionProfile()}}
|
||||
@@ -639,7 +819,7 @@ func TestRunnerRunSuccessful(t *testing.T) {
|
||||
"a://t": {Body: []byte("transcript"), Hash: hashString("transcript")},
|
||||
"a://g": {Body: []byte("glossary"), Hash: hashString("glossary")},
|
||||
}}
|
||||
renderer := &fakeRenderer{rendered: &domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "system", Content: "sys"}, {Role: "user", Content: "usr"}}}}
|
||||
renderer := &fakeRenderer{rendered: &domain.RenderedPrompt{SessionID: "session-123", Messages: []domain.RenderedMessage{{Role: "system", Content: "sys"}, {Role: "user", Content: "usr"}}}}
|
||||
llmClient := &fakeLLM{resp: &domain.GenerateResponse{Content: "# recap", Usage: domain.TokenUsage{TotalTokens: 7}}}
|
||||
|
||||
runner := NewRunner(promptRepo, execRepo, reader, renderer, llmClient, nil)
|
||||
@@ -651,7 +831,7 @@ func TestRunnerRunSuccessful(t *testing.T) {
|
||||
"transcript": {Type: domain.ArtifactRefFile, URI: "a://t"},
|
||||
"glossary": {Type: domain.ArtifactRefFile, URI: "a://g"},
|
||||
},
|
||||
Execution: &domain.ExecutionTarget{Endpoint: "http://override/v1", Model: "m", Temperature: 0.3, TimeoutSeconds: 90},
|
||||
Execution: &domain.ExecutionTargetOverride{Endpoint: "http://override/v1", Model: "m", Temperature: float64Ptr(0.3), TimeoutSeconds: intPtr(90)},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
@@ -686,6 +866,48 @@ func TestRunnerRunSuccessful(t *testing.T) {
|
||||
if llmClient.lastReq.Target.TimeoutSeconds != 90 {
|
||||
t.Fatalf("expected timeout propagation, got %d", llmClient.lastReq.Target.TimeoutSeconds)
|
||||
}
|
||||
if !llmClient.lastReq.TargetPresence.Temperature || !llmClient.lastReq.TargetPresence.TimeoutSeconds {
|
||||
t.Fatalf("expected numeric override presence to be sent to llm, got %+v", llmClient.lastReq.TargetPresence)
|
||||
}
|
||||
if llmClient.lastReq.Prompt.SessionID != "session-123" {
|
||||
t.Fatalf("expected session id to be sent to llm, got %q", llmClient.lastReq.Prompt.SessionID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunnerRunPassesExtraParamsToGenerateRequestTarget(t *testing.T) {
|
||||
extraParams := map[string]any{
|
||||
"string_value": "enabled",
|
||||
"number_value": 42,
|
||||
"boolean_value": true,
|
||||
"object_value": map[string]any{"nested": "value"},
|
||||
"array_value": []any{"first", 3, false},
|
||||
}
|
||||
promptRepo := &fakePromptRepo{def: promptDef(domain.FormatText, domain.ValidationNone, 0)}
|
||||
execRepo := &fakeExecutionProfileRepo{profiles: map[string]*domain.ExecutionProfile{
|
||||
"exec": {
|
||||
ID: "exec",
|
||||
Endpoint: "http://profile/v1",
|
||||
Model: "profile-model",
|
||||
ExtraParams: extraParams,
|
||||
},
|
||||
}}
|
||||
llmClient := &fakeLLM{resp: &domain.GenerateResponse{Content: "ok"}}
|
||||
runner := NewRunner(promptRepo, execRepo, defaultArtifactReader(), defaultRenderer(), llmClient, nil)
|
||||
|
||||
res, err := runner.Run(context.Background(), domain.RunRequest{
|
||||
PromptID: "p",
|
||||
ProfileID: "exec",
|
||||
Inputs: singleInputRef(),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(res.EffectiveModelParams.ExtraParams, extraParams) {
|
||||
t.Fatalf("expected run result extra_params to match profile values, got %#v", res.EffectiveModelParams.ExtraParams)
|
||||
}
|
||||
if !reflect.DeepEqual(llmClient.lastReq.Target.ExtraParams, extraParams) {
|
||||
t.Fatalf("expected generate request extra_params to match profile values, got %#v", llmClient.lastReq.Target.ExtraParams)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunnerRunAndPrepareResolveSameProfileAndEffectiveSettings(t *testing.T) {
|
||||
@@ -704,7 +926,7 @@ func TestRunnerRunAndPrepareResolveSameProfileAndEffectiveSettings(t *testing.T)
|
||||
Inputs: map[string]domain.ArtifactRef{
|
||||
"transcript": {Type: domain.ArtifactRefFile, URI: "a://t"},
|
||||
},
|
||||
Execution: &domain.ExecutionTarget{Endpoint: "http://override/v1", Model: "m", Temperature: 0.3, TimeoutSeconds: 90},
|
||||
Execution: &domain.ExecutionTargetOverride{Endpoint: "http://override/v1", Model: "m", Temperature: float64Ptr(0.3), TimeoutSeconds: intPtr(90)},
|
||||
}
|
||||
|
||||
prepared, err := runner.Prepare(context.Background(), req)
|
||||
@@ -832,11 +1054,11 @@ func TestRunnerRunExplicitRuntimeOverrideBeatsSelectedProfileValue(t *testing.T)
|
||||
PromptID: "p",
|
||||
ProfileID: "exec",
|
||||
Inputs: singleInputRef(),
|
||||
Execution: &domain.ExecutionTarget{
|
||||
Execution: &domain.ExecutionTargetOverride{
|
||||
Endpoint: "http://override/v1",
|
||||
Model: "override-model",
|
||||
Temperature: 0.7,
|
||||
TimeoutSeconds: 30,
|
||||
Temperature: float64Ptr(0.7),
|
||||
TimeoutSeconds: intPtr(30),
|
||||
ServiceTier: "flex",
|
||||
},
|
||||
})
|
||||
@@ -971,7 +1193,7 @@ func TestRunnerRunRuntimeAPIKeyEnvOverrideWorks(t *testing.T) {
|
||||
PromptID: "p",
|
||||
ProfileID: "exec",
|
||||
Inputs: singleInputRef(),
|
||||
Execution: &domain.ExecutionTarget{APIKeyEnv: envName},
|
||||
Execution: &domain.ExecutionTargetOverride{APIKeyEnv: envName},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
@@ -996,7 +1218,7 @@ func TestRunnerRunRuntimeAPIKeyEnvOverrideBeatsProfile(t *testing.T) {
|
||||
PromptID: "p",
|
||||
ProfileID: "exec",
|
||||
Inputs: singleInputRef(),
|
||||
Execution: &domain.ExecutionTarget{APIKeyEnv: runtimeEnv},
|
||||
Execution: &domain.ExecutionTargetOverride{APIKeyEnv: runtimeEnv},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
@@ -1095,6 +1317,28 @@ func TestRunnerRunLLMFailure(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunnerRunLLMInvalidRequestMapsToUsecaseInvalidRequest(t *testing.T) {
|
||||
runner := NewRunner(
|
||||
&fakePromptRepo{def: promptDef(domain.FormatText, domain.ValidationNone, 0)},
|
||||
&fakeExecutionProfileRepo{profiles: map[string]*domain.ExecutionProfile{"exec": defaultExecutionProfile()}},
|
||||
defaultArtifactReader(),
|
||||
defaultRenderer(),
|
||||
&fakeLLM{err: llm.ErrInvalidRequest},
|
||||
nil,
|
||||
)
|
||||
_, err := runner.Run(context.Background(), domain.RunRequest{
|
||||
PromptID: "p",
|
||||
ProfileID: "exec",
|
||||
Inputs: singleInputRef(),
|
||||
})
|
||||
if !errors.Is(err, ErrInvalidRequest) {
|
||||
t.Fatalf("expected ErrInvalidRequest, got %v", err)
|
||||
}
|
||||
if errors.Is(err, ErrLLMGenerate) {
|
||||
t.Fatalf("did not expect ErrLLMGenerate, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunnerRunValidationStillWorks(t *testing.T) {
|
||||
validator := &fakeValidator{result: domain.ValidationResult{Status: domain.ValidationFailed, Mode: domain.ValidationBasic, Errors: []string{"bad"}, IsValid: false}}
|
||||
runner := NewRunner(
|
||||
@@ -1137,7 +1381,7 @@ func TestRunnerRunStructuredRepairRemainsBoundedAndUsesEffectiveModelSettings(t
|
||||
PromptID: "p",
|
||||
ProfileID: "exec",
|
||||
Inputs: singleInputRef(),
|
||||
Execution: &domain.ExecutionTarget{Endpoint: "http://override/v1", Model: "override-model", TimeoutSeconds: 22},
|
||||
Execution: &domain.ExecutionTargetOverride{Endpoint: "http://override/v1", Model: "override-model", TimeoutSeconds: intPtr(22)},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
@@ -1224,7 +1468,7 @@ func TestExecutionProfileToTargetPopulatesAllFieldsAndCopiesExtraParams(t *testi
|
||||
ServiceTier: "priority",
|
||||
ReasoningEffort: "medium",
|
||||
APIKeyEnv: "SCRIPTORIUM_API_KEY",
|
||||
ExtraParams: map[string]string{
|
||||
ExtraParams: map[string]any{
|
||||
"provider_option": "on",
|
||||
},
|
||||
}
|
||||
@@ -1263,12 +1507,18 @@ func TestResolveExecutionTargetProfileValuesPopulateAllSupportedFields(t *testin
|
||||
ServiceTier: "priority",
|
||||
ReasoningEffort: "low",
|
||||
APIKeyEnv: "PROFILE_KEY",
|
||||
ExtraParams: map[string]string{
|
||||
ExtraParams: map[string]any{
|
||||
"profile_option": "enabled",
|
||||
},
|
||||
}
|
||||
|
||||
target := resolveExecutionTarget(profileValue, nil)
|
||||
target, presence, err := resolveExecutionTarget(profileValue, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if presence != (domain.ExecutionTargetPresence{}) {
|
||||
t.Fatalf("expected no request override presence, got %+v", presence)
|
||||
}
|
||||
if target.Endpoint != profileValue.Endpoint ||
|
||||
target.Model != profileValue.Model ||
|
||||
target.Temperature != profileValue.Temperature ||
|
||||
@@ -1297,32 +1547,38 @@ func TestResolveExecutionTargetRuntimeOverridesBeatProfileForAllOverrideableFiel
|
||||
ServiceTier: "priority",
|
||||
ReasoningEffort: "medium",
|
||||
APIKeyEnv: "PROFILE_KEY",
|
||||
ExtraParams: map[string]string{
|
||||
ExtraParams: map[string]any{
|
||||
"profile_only": "yes",
|
||||
},
|
||||
}
|
||||
override := &domain.ExecutionTarget{
|
||||
override := &domain.ExecutionTargetOverride{
|
||||
Endpoint: "http://override/v1",
|
||||
Model: "override-model",
|
||||
Temperature: 0.9,
|
||||
MaxTokens: 111,
|
||||
TopP: 0.5,
|
||||
TimeoutSeconds: 30,
|
||||
Temperature: float64Ptr(0.9),
|
||||
MaxTokens: intPtr(111),
|
||||
TopP: float64Ptr(0.5),
|
||||
TimeoutSeconds: intPtr(30),
|
||||
ServiceTier: "flex",
|
||||
ReasoningEffort: "high",
|
||||
APIKeyEnv: "RUNTIME_KEY",
|
||||
ExtraParams: map[string]string{
|
||||
ExtraParams: map[string]any{
|
||||
"runtime_only": "yes",
|
||||
},
|
||||
}
|
||||
|
||||
target := resolveExecutionTarget(profileValue, override)
|
||||
target, presence, err := resolveExecutionTarget(profileValue, override)
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if presence != (domain.ExecutionTargetPresence{Temperature: true, MaxTokens: true, TopP: true, TimeoutSeconds: true}) {
|
||||
t.Fatalf("unexpected override presence: %+v", presence)
|
||||
}
|
||||
if target.Endpoint != override.Endpoint ||
|
||||
target.Model != override.Model ||
|
||||
target.Temperature != override.Temperature ||
|
||||
target.MaxTokens != override.MaxTokens ||
|
||||
target.TopP != override.TopP ||
|
||||
target.TimeoutSeconds != override.TimeoutSeconds ||
|
||||
target.Temperature != *override.Temperature ||
|
||||
target.MaxTokens != *override.MaxTokens ||
|
||||
target.TopP != *override.TopP ||
|
||||
target.TimeoutSeconds != *override.TimeoutSeconds ||
|
||||
target.ServiceTier != override.ServiceTier ||
|
||||
target.ReasoningEffort != override.ReasoningEffort ||
|
||||
target.APIKeyEnv != override.APIKeyEnv {
|
||||
@@ -1366,12 +1622,12 @@ func TestMergeExecutionTargetEmptyStringOverridesDoNotErase(t *testing.T) {
|
||||
|
||||
func TestMergeExecutionTargetEmptyExtraParamsDoesNotErase(t *testing.T) {
|
||||
base := domain.ExecutionTarget{
|
||||
ExtraParams: map[string]string{
|
||||
ExtraParams: map[string]any{
|
||||
"keep": "value",
|
||||
},
|
||||
}
|
||||
override := domain.ExecutionTarget{
|
||||
ExtraParams: map[string]string{},
|
||||
ExtraParams: map[string]any{},
|
||||
}
|
||||
|
||||
merged := mergeExecutionTarget(base, override)
|
||||
@@ -1447,6 +1703,14 @@ func singleInputRef() map[string]domain.ArtifactRef {
|
||||
return map[string]domain.ArtifactRef{"transcript": {Type: domain.ArtifactRefFile, URI: "a://ok"}}
|
||||
}
|
||||
|
||||
func float64Ptr(v float64) *float64 {
|
||||
return &v
|
||||
}
|
||||
|
||||
func intPtr(v int) *int {
|
||||
return &v
|
||||
}
|
||||
|
||||
func newMinimalRunner(promptRepo *fakePromptRepo, execRepo *fakeExecutionProfileRepo) *Runner {
|
||||
return NewRunner(
|
||||
promptRepo,
|
||||
|
||||
Reference in New Issue
Block a user