Implement support for OpenRouter sticky routing via a session_id variable
This commit is contained in:
@@ -83,6 +83,7 @@ Notes:
|
||||
|
||||
- `--input name=path` maps prompt input names to local file paths.
|
||||
- `--var name=value` maps template variable names to values.
|
||||
- If a prompt defines `session_id: "{{ .session_id }}"`, pass the OpenRouter sticky-routing value with `--var session_id=<value>`.
|
||||
- Both flags can be repeated.
|
||||
- Both flags also support comma-separated batches, for example:
|
||||
- `--input transcript=./t.md,glossary=./g.yml`
|
||||
|
||||
@@ -104,6 +104,7 @@ Field reference:
|
||||
- `version` (required): prompt version.
|
||||
- `default_profile` (optional): profile ID used when request does not provide `profile_id`.
|
||||
- `description` (optional): prompt description.
|
||||
- `session_id` (optional): Go-template string for OpenRouter sticky-routing `session_id`; rendered from request vars.
|
||||
- `inputs` (optional list): expected named inputs.
|
||||
- `messages` (required list): prompt message templates.
|
||||
- `output` (required object): output contract.
|
||||
@@ -150,6 +151,14 @@ messages:
|
||||
|
||||
Use cache control on stable reusable prompt content. Dynamic per-run inputs before the cache-controlled message change the provider cache key.
|
||||
|
||||
Example prompt-level session ID:
|
||||
|
||||
```yaml
|
||||
session_id: "{{ .session_id }}"
|
||||
```
|
||||
|
||||
When configured, `session_id` is rendered with the same variable context as messages. The rendered value is trimmed, omitted when empty, and rejected if longer than 256 characters. CLI callers pass the value through `--var session_id=<value>`; HTTP callers pass it through `"vars": {"session_id": "<value>"}`.
|
||||
|
||||
`output` fields:
|
||||
|
||||
- `format` (required): `text`, `markdown`, or `json`.
|
||||
@@ -205,7 +214,7 @@ Profile rules:
|
||||
|
||||
Current outbound request behavior:
|
||||
|
||||
- The OpenAI-compatible client currently serializes: `model`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, and optional `response_format` for `json_schema` prompts.
|
||||
- The OpenAI-compatible client currently serializes: `model`, optional `session_id`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, and optional `response_format` for `json_schema` prompts.
|
||||
- Messages without `cache_control` serialize with string `content`.
|
||||
- Messages with `cache_control` serialize as a single text content-block array containing `cache_control`.
|
||||
- `reasoning_effort` and `extra_params` are parsed and carried in effective settings, but are not currently serialized into outbound chat-completions requests.
|
||||
|
||||
@@ -26,6 +26,7 @@ Example:
|
||||
Serialized JSON fields:
|
||||
|
||||
- `model` (required after fallback resolution)
|
||||
- `session_id` (only when the rendered prompt includes a non-empty session ID)
|
||||
- `messages` (rendered prompt messages)
|
||||
- `temperature` (only when non-zero)
|
||||
- `max_tokens` (only when non-zero)
|
||||
@@ -35,6 +36,8 @@ Serialized JSON fields:
|
||||
|
||||
`service_tier` is provider-specific. OpenRouter currently documents request values such as `flex` and `priority`; Scriptorium forwards any non-empty configured value and lets the backend validate support.
|
||||
|
||||
`session_id` is rendered from prompt YAML using request variables and serialized as a top-level JSON request field. Scriptorium does not send an `x-session-id` header. Empty rendered session IDs are omitted, and values longer than 256 characters are rejected before the HTTP request.
|
||||
|
||||
Messages without prompt cache control serialize with string `content`:
|
||||
|
||||
```json
|
||||
@@ -140,7 +143,7 @@ The following fields may exist in profile/effective settings but are not current
|
||||
- `reasoning_effort`
|
||||
- `extra_params`
|
||||
|
||||
The client also does not serialize top-level `cache_control` or `session_id`.
|
||||
The client also does not serialize top-level `cache_control`.
|
||||
|
||||
No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.
|
||||
|
||||
|
||||
@@ -146,5 +146,5 @@ Behavior highlights:
|
||||
- Adapter packages do not own runner decision logic.
|
||||
- External request/response strictness is part of contract stability.
|
||||
- Prepared-render output never includes resolved API key values.
|
||||
- Outbound OpenAI-compatible request includes only currently serialized fields (`model`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `service_tier`, optional `response_format`).
|
||||
- Outbound cache control is message-level only; no top-level cache-control/session fields are serialized.
|
||||
- Outbound OpenAI-compatible request includes only currently serialized fields (`model`, optional `session_id`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `service_tier`, optional `response_format`).
|
||||
- Outbound cache control is message-level only; no top-level cache-control field is serialized.
|
||||
|
||||
@@ -129,7 +129,6 @@ These are intentionally out of scope for the initial feature:
|
||||
- Image, tool, or non-text content blocks.
|
||||
- Provider-specific automatic prompt caching toggles.
|
||||
- Top-level OpenRouter `cache_control`.
|
||||
- Top-level OpenRouter `session_id`.
|
||||
- General-purpose serialization of `extra_params`.
|
||||
- Provider-specific validation profiles for cache-control limits.
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
This plan implements the feature defined in `docs/roadmap/cache.md`.
|
||||
|
||||
The implementation target is message-level Anthropic/OpenRouter-style cache control for rendered prompt messages, plus cache usage observability. Do not implement deferred roadmap items such as `content_blocks`, top-level `cache_control`, `session_id`, or general `extra_params` serialization.
|
||||
The implementation target is message-level Anthropic/OpenRouter-style cache control for rendered prompt messages, plus cache usage observability. Do not implement deferred roadmap items such as `content_blocks`, top-level `cache_control`, or general `extra_params` serialization.
|
||||
|
||||
Follow the policy documents under `docs/policy/` while implementing:
|
||||
|
||||
@@ -182,7 +182,7 @@ Request serialization decisions:
|
||||
- Keep string content for messages without cache control.
|
||||
- Use content blocks only when `RenderedMessage.CacheControl` is non-nil.
|
||||
- Do not introduce `content_blocks` prompt syntax.
|
||||
- Do not serialize top-level `cache_control`, top-level `session_id`, or `extra_params`.
|
||||
- Do not serialize top-level `cache_control` or `extra_params`.
|
||||
- Split request and response wire message structs. The current `openAIChatMessage` is used in both request and response paths; changing request content to `any` should not make response decoding less strict than necessary.
|
||||
|
||||
Use request wire types equivalent to:
|
||||
|
||||
Reference in New Issue
Block a user