Document prompt cache control behavior

This commit is contained in:
2026-07-02 23:10:24 +00:00
parent 5dcb3cd4fc
commit 4d4bb7a121
7 changed files with 113 additions and 4 deletions

View File

@@ -128,7 +128,9 @@ Response shape:
"usage": {
"prompt_tokens": 11,
"completion_tokens": 22,
"total_tokens": 33
"total_tokens": 33,
"cached_tokens": 0,
"cache_write_tokens": 0
},
"start_time": "2026-05-04T12:00:00Z",
"end_time": "2026-05-04T12:00:01Z",
@@ -142,6 +144,8 @@ Response shape:
`raw_model_output` is omitted by default.
`metadata.usage.cached_tokens` and `metadata.usage.cache_write_tokens` are always present as numbers. They are `0` when the provider omits compatible cache usage fields or reports no cache activity.
To include it, send:
- `"include_raw_output": true`

View File

@@ -26,7 +26,7 @@ Example:
Serialized JSON fields:
- `model` (required after fallback resolution)
- `messages` (role/content pairs from rendered prompt)
- `messages` (rendered prompt messages)
- `temperature` (only when non-zero)
- `max_tokens` (only when non-zero)
- `top_p` (only when non-zero)
@@ -35,6 +35,35 @@ Serialized JSON fields:
`service_tier` is provider-specific. OpenRouter currently documents request values such as `flex` and `priority`; Scriptorium forwards any non-empty configured value and lets the backend validate support.
Messages without prompt cache control serialize with string `content`:
```json
{
"role": "system",
"content": "rendered text"
}
```
Messages with prompt cache control serialize as a single text content-block array:
```json
{
"role": "system",
"content": [
{
"type": "text",
"text": "rendered text",
"cache_control": {
"type": "ephemeral",
"ttl": "1h"
}
}
]
}
```
When cache-control `ttl` is unset in the prompt definition, `ttl` is omitted from the outbound payload.
Structured output is currently `json_schema` only, serialized as:
```json
@@ -82,6 +111,13 @@ Expected successful response shape (subset used):
- `usage.prompt_tokens`
- `usage.completion_tokens`
- `usage.total_tokens`
- `usage.prompt_tokens_details.cached_tokens` (optional)
- `usage.cache_write_tokens` (optional)
Absent cache usage fields are treated as zero. Parsed cache usage is exposed through run results and adapter response surfaces as:
- `cached_tokens`
- `cache_write_tokens`
Malformed response conditions include:
@@ -104,6 +140,8 @@ The following fields may exist in profile/effective settings but are not current
- `reasoning_effort`
- `extra_params`
The client also does not serialize top-level `cache_control` or `session_id`.
No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.
## Relationship To Runner