Prompt Cache Control Roadmap

Purpose

Scriptorium should support provider prompt-cache controls for OpenAI-compatible gateways that expose Anthropic-style cache breakpoints, especially OpenRouter.

The feature should preserve existing prompt definitions. Prompt authors should opt in with optional message-level cache metadata, and Scriptorium should report cache usage when compatible providers return it.

Implementation steps belong in docs/roadmap/implementation.md.

Target State

Prompt authors can mark a rendered message as an explicit cache breakpoint:

messages:
  - role: system
    content_file: ../common/transcript.system.md
    cache_control:
      type: ephemeral
      ttl: 1h

Existing messages without cache_control continue to render and serialize as string content:

{
  "role": "system",
  "content": "rendered text"
}

Messages with cache_control serialize as a single text content block:

{
  "role": "system",
  "content": [
    {
      "type": "text",
      "text": "rendered text",
      "cache_control": {
        "type": "ephemeral",
        "ttl": "1h"
      }
    }
  ]
}

Scriptorium should parse and expose provider cache usage when compatible response fields are present, including cached prompt tokens and cache-write tokens.

Prompt Authoring Policy

Cache breakpoints should be used for stable reusable prompt prefixes.

Recommended ordering:

Put stable, reusable context first.
Put cache_control on the last stable message that should be part of the reusable prefix.
Put per-run dynamic inputs after that breakpoint.

Example:

messages:
  - role: system
    content_file: ../common/transcript.system.md
  - role: user
    content_file: ./character_meta_analysis.task.md
  - role: user
    content_file: ./character_meta_analysis.instructions.md
    cache_control:
      type: ephemeral
      ttl: 1h
  - role: user
    content: |
      <<<PREVIOUS_SESSION_RECAP
      {{input "recap"}}
      PREVIOUS_SESSION_RECAP>>>
  - role: user
    content: |
      <<<CURRENT_SESSION_TRANSCRIPT
      {{input "transcript"}}
      CURRENT_SESSION_TRANSCRIPT>>>

Supported Cache-Control Shape

Initial support is message-level only:

cache_control:
  type: ephemeral
  ttl: 1h

Rules:

cache_control is optional on each message.
cache_control.type is required when cache_control is present.
The only supported type value is ephemeral.
ttl is optional.
When set, the only supported ttl value is 1h.
Empty ttl is omitted from the outbound payload.
Prompt decoding remains strict; unknown cache-control fields are rejected.
Existing content and content_file rules remain unchanged.

Compatibility

This feature should be backward compatible for existing prompt definitions.

Compatibility requirements:

Existing prompt YAML without cache_control loads unchanged.
Existing render output remains valid.
Existing outbound request payloads remain string-content messages unless cache_control is configured.
Existing integrations do not need to send new request fields.
CLI and HTTP callers do not need new request options for the initial feature.

The only intentional prompt-definition contract change is the new optional messages[].cache_control object.

Deferred Work

These are intentionally out of scope for the initial feature:

content_blocks prompt syntax.
Multiple text blocks inside a single message.
Image, tool, or non-text content blocks.
Provider-specific automatic prompt caching toggles.
Top-level OpenRouter cache_control.
General-purpose serialization of extra_params.
Provider-specific validation profiles for cache-control limits.

These can be added later without changing the message-level cache-control contract.

3.7 KiB Raw Blame History