Add a feature roadmap and implementation plan for cache_control values

2026-07-02 17:55:56 -05:00
parent bc099a31ad
commit 1f63f8afbb
2 changed files with 526 additions and 0 deletions
--- a/docs/roadmap/cache.md
+++ b/docs/roadmap/cache.md
@@ -0,0 +1,136 @@
+# Prompt Cache Control Roadmap
+
+## Purpose
+
+Scriptorium should support provider prompt-cache controls for OpenAI-compatible gateways that expose Anthropic-style cache breakpoints, especially OpenRouter.
+
+The feature should preserve existing prompt definitions. Prompt authors should opt in with optional message-level cache metadata, and Scriptorium should report cache usage when compatible providers return it.
+
+Implementation steps belong in `docs/roadmap/implementation.md`.
+
+## Target State
+
+Prompt authors can mark a rendered message as an explicit cache breakpoint:
+
+```yaml
+messages:
+  - role: system
+    content_file: ../common/transcript.system.md
+    cache_control:
+      type: ephemeral
+      ttl: 1h
+```
+
+Existing messages without `cache_control` continue to render and serialize as string content:
+
+```json
+{
+  "role": "system",
+  "content": "rendered text"
+}
+```
+
+Messages with `cache_control` serialize as a single text content block:
+
+```json
+{
+  "role": "system",
+  "content": [
+    {
+      "type": "text",
+      "text": "rendered text",
+      "cache_control": {
+        "type": "ephemeral",
+        "ttl": "1h"
+      }
+    }
+  ]
+}
+```
+
+Scriptorium should parse and expose provider cache usage when compatible response fields are present, including cached prompt tokens and cache-write tokens.
+
+## Prompt Authoring Policy
+
+Cache breakpoints should be used for stable reusable prompt prefixes.
+
+Recommended ordering:
+
+1. Put stable, reusable context first.
+2. Put `cache_control` on the last stable message that should be part of the reusable prefix.
+3. Put per-run dynamic inputs after that breakpoint.
+
+Example:
+
+```yaml
+messages:
+  - role: system
+    content_file: ../common/transcript.system.md
+  - role: user
+    content_file: ./character_meta_analysis.task.md
+  - role: user
+    content_file: ./character_meta_analysis.instructions.md
+    cache_control:
+      type: ephemeral
+      ttl: 1h
+  - role: user
+    content: |
+      <<<PREVIOUS_SESSION_RECAP
+      {{input "recap"}}
+      PREVIOUS_SESSION_RECAP>>>
+  - role: user
+    content: |
+      <<<CURRENT_SESSION_TRANSCRIPT
+      {{input "transcript"}}
+      CURRENT_SESSION_TRANSCRIPT>>>
+```
+
+## Supported Cache-Control Shape
+
+Initial support is message-level only:
+
+```yaml
+cache_control:
+  type: ephemeral
+  ttl: 1h
+```
+
+Rules:
+
+- `cache_control` is optional on each message.
+- `cache_control.type` is required when `cache_control` is present.
+- The only supported `type` value is `ephemeral`.
+- `ttl` is optional.
+- When set, the only supported `ttl` value is `1h`.
+- Empty `ttl` is omitted from the outbound payload.
+- Prompt decoding remains strict; unknown cache-control fields are rejected.
+- Existing `content` and `content_file` rules remain unchanged.
+
+## Compatibility
+
+This feature should be backward compatible for existing prompt definitions.
+
+Compatibility requirements:
+
+- Existing prompt YAML without `cache_control` loads unchanged.
+- Existing `render` output remains valid.
+- Existing outbound request payloads remain string-content messages unless `cache_control` is configured.
+- Existing integrations do not need to send new request fields.
+- CLI and HTTP callers do not need new request options for the initial feature.
+
+The only intentional prompt-definition contract change is the new optional `messages[].cache_control` object.
+
+## Deferred Work
+
+These are intentionally out of scope for the initial feature:
+
+- `content_blocks` prompt syntax.
+- Multiple text blocks inside a single message.
+- Image, tool, or non-text content blocks.
+- Provider-specific automatic prompt caching toggles.
+- Top-level OpenRouter `cache_control`.
+- Top-level OpenRouter `session_id`.
+- General-purpose serialization of `extra_params`.
+- Provider-specific validation profiles for cache-control limits.
+
+These can be added later without changing the message-level cache-control contract.