scriptorium/docs/roadmap/implementation.md

Prompt Cache Control Implementation Plan

Scope

This plan implements the feature defined in docs/roadmap/cache.md.

The implementation target is message-level Anthropic/OpenRouter-style cache control for rendered prompt messages, plus cache usage observability. Do not implement deferred roadmap items such as content_blocks, top-level cache_control, session_id, or general extra_params serialization.

Follow the policy documents under docs/policy/ while implementing:

• Keep prompt/profile/HTTP/config decoding strict.
• Keep adapters thin.
• Keep provider request serialization in internal/llm.
• Keep prompt loading in internal/promptdef.
• Update canonical non-roadmap docs only when the behavior is implemented.
• Do not accept or emit raw secret values.

Stage 1: Domain Types And Prompt Loader

Objective: make prompt YAML accept and validate optional messages[].cache_control, and carry normalized cache-control metadata in domain prompt definitions.

Files to update:

• internal/domain/domain.go
• internal/promptdef/filesystem_repository.go
• internal/promptdef/repository_test.go
• internal/promptdef/testdata/

Domain decisions:

• Add a typed cache-control value in internal/domain.
• Use a pointer on message structs so absence is distinguishable from an empty object.
• Use omitempty on JSON tags for the new field so existing no-cache prompt and render hashes do not gain null fields.

Use this domain shape:

type CacheControlType string

const (
    CacheControlEphemeral CacheControlType = "ephemeral"
)

type CacheControl struct {
    Type CacheControlType `yaml:"type" json:"type"`
    TTL  string           `yaml:"ttl,omitempty" json:"ttl,omitempty"`
}

Add cache control to:

type PromptMessageTemplate struct {
    Role         string        `yaml:"role"`
    Content      string        `yaml:"content"`
    ContentFile  string        `yaml:"content_file"`
    CacheControl *CacheControl `yaml:"cache_control,omitempty" json:"cache_control,omitempty"`
}

Prompt loader changes:

• Add this field to promptMessageFile:

CacheControl *cacheControlFile `yaml:"cache_control"`

• Add a private cacheControlFile struct with type and ttl.
• In normalizePromptDefinition, validate cache_control per message.
• Trim whitespace from type and ttl.
• Reject present cache_control when type is empty.
• Reject any type other than ephemeral.
• Accept omitted or empty ttl.
• Reject any non-empty ttl other than 1h.
• Copy normalized values into domain.PromptMessageTemplate.CacheControl.
• Preserve the existing rule that each message must set exactly one of content or content_file.

Testing requirements:

• Add a valid fixture with cache_control: {type: ephemeral, ttl: 1h}.
• Add a valid fixture with cache_control: {type: ephemeral}.
• Add invalid fixtures for:
  • empty cache_control.type
  • unsupported cache_control.type
  • unsupported cache_control.ttl
  • unknown field under cache_control
• Assert valid fixtures load with the expected CacheControl values.
• Assert invalid fixtures fail with promptdef.ErrInvalidPromptDefinition or promptdef.ErrInvalidYAML as appropriate.
• Re-run:

go test ./internal/promptdef

Do not update renderer, formatter, LLM serialization, adapters, or user-facing docs in this stage.

Stage 2: Rendering, Prepared Output, And Render Hashing

Objective: preserve cache-control metadata through rendering, expose it in prepared-run output, and make rendered_prompt_hash reflect outbound-affecting cache metadata.

Files to update:

• internal/domain/domain.go
• internal/prompt/go_renderer.go
• internal/prompt/renderer_test.go
• internal/usecase/runner.go
• internal/usecase/runner_test.go
• internal/domain/prepared_run_test.go
• internal/format/prepared_run.go
• internal/format/prepared_run_test.go

Domain changes:

• Add cache control to rendered messages:

type RenderedMessage struct {
    Role         string        `json:"role"`
    Content      string        `json:"content"`
    CacheControl *CacheControl `json:"cache_control,omitempty"`
}

Renderer changes:

• When a PromptMessageTemplate has CacheControl, copy it to the returned RenderedMessage.
• Do not mutate the template's pointer in place.
• Add and use a small helper such as cloneCacheControl to avoid aliasing.
• Do not change template execution behavior for {{input "name"}} or {{.var}}.

Rendered hash decision:

• rendered_prompt_hash must change when cache-control metadata changes, because the outbound provider request changes.
• Preserve existing hashes for no-cache prompts by keeping the current role/content hash flow.
• Append normalized cache-control metadata only when present, for example:
  • cache_control.type=<type>
  • cache_control.ttl=<ttl> only when non-empty
• Include cache-control fields in a stable order.

Prepared-run formatting:

• JSON prepared-run output should include cache_control on messages only when present.
• Text prepared-run output should show cache-control metadata under the specific message, before content, for example:

      cache_control: ephemeral ttl=1h
      content: |

• If ttl is empty, omit the ttl=... suffix.
• Do not print cache-control lines for messages without cache control.

Testing requirements:

• Renderer test: cache control is copied to rendered messages.
• Renderer test: changing returned rendered cache control does not mutate the source template.
• Runner test: rendered_prompt_hash changes when cache control differs and remains stable for identical rendered messages.
• Format JSON test: cache_control appears only when present.
• Format text test: cache control appears under the correct message.
• Existing render/format tests still pass.
• Re-run:

go test ./internal/prompt ./internal/usecase ./internal/domain ./internal/format

Do not update outbound LLM serialization or adapter response surfaces in this stage.

Stage 3: Outbound Chat Serialization And Usage Parsing

Objective: serialize cache-controlled messages as one-item text content-block arrays and parse cache usage fields from compatible provider responses.

Files to update:

• internal/llm/openai_compatible_client.go
• internal/llm/openai_compatible_client_test.go
• internal/domain/domain.go

Request serialization decisions:

• Keep string content for messages without cache control.
• Use content blocks only when RenderedMessage.CacheControl is non-nil.
• Do not introduce content_blocks prompt syntax.
• Do not serialize top-level cache_control, top-level session_id, or extra_params.
• Split request and response wire message structs. The current openAIChatMessage is used in both request and response paths; changing request content to any should not make response decoding less strict than necessary.

Use request wire types equivalent to:

type openAIChatRequestMessage struct {
    Role    string `json:"role"`
    Content any    `json:"content"`
}

type openAIChatTextContentBlock struct {
    Type         string               `json:"type"`
    Text         string               `json:"text"`
    CacheControl *openAICacheControl  `json:"cache_control,omitempty"`
}

type openAICacheControl struct {
    Type string `json:"type"`
    TTL  string `json:"ttl,omitempty"`
}

When cache control is present, content should serialize as:

[
  {
    "type": "text",
    "text": "rendered text",
    "cache_control": {
      "type": "ephemeral",
      "ttl": "1h"
    }
  }
]

When ttl is empty, omit ttl.

Usage parsing decisions:

• Extend domain.TokenUsage:

type TokenUsage struct {
    PromptTokens     int
    CompletionTokens int
    TotalTokens      int
    CachedTokens     int
    CacheWriteTokens int
}

• Parse usage.prompt_tokens_details.cached_tokens into CachedTokens.
• Parse usage.cache_write_tokens into CacheWriteTokens.
• Treat absent cache fields as zero.
• Do not fail response decoding if cache fields are absent.
• If a provider later returns a different cache-write field, add it in a later change after observing a real response.

Testing requirements:

• Existing request serialization test still sees string message content for no-cache messages.
• New request serialization test sees content-block array for cache-controlled messages.
• New request serialization test verifies empty ttl is omitted.
• Response parsing test verifies cached and cache-write tokens are mapped.
• Response parsing test verifies absent cache usage fields remain zero.
• Existing structured-output and service-tier behavior remains unchanged.
• Re-run:

go test ./internal/llm

Do not update CLI or HTTP observability in this stage unless required by compilation after extending TokenUsage.

Stage 4: CLI And HTTP Cache Usage Observability

Objective: surface parsed cache usage through existing CLI and HTTP result paths without adding request knobs.

Files to update:

• internal/adapter/cli/run.go
• internal/adapter/cli/run_test.go
• internal/adapter/http/dto.go
• internal/adapter/http/handler.go
• internal/adapter/http/handler_test.go

CLI decisions:

• Do not add CLI flags.
• Keep success summaries concise.
• Preserve the existing summary output when cache usage is zero.
• When either cache field is non-zero, append:

cached_tokens=<n> cache_write_tokens=<n>

HTTP decisions:

• Do not add request fields.
• Add numeric fields to metadata.usage:
  • cached_tokens
  • cache_write_tokens
• Always include these fields as numbers, matching the current non-omitempty numeric usage fields.
• Continue omitting raw model output unless include_raw_output is true.

Testing requirements:

• CLI summary omits cache fields when both are zero.
• CLI summary includes cache fields when either is non-zero.
• HTTP success response includes metadata.usage.cached_tokens.
• HTTP success response includes metadata.usage.cache_write_tokens.
• Existing HTTP request mapping is unchanged.
• Re-run:

go test ./internal/adapter/cli ./internal/adapter/http

Stage 5: Canonical Documentation

Objective: update implemented-behavior docs after the feature works and tests pass.

Files to update:

• docs/config.md
• docs/integrations/openai-compatible-chat.md
• docs/cli.md
• docs/troubleshooting.md
• docs/internal/runner.md
• docs/internal/adapters.md, if adapter response behavior needs an explicit note
• docs/roadmap/cache.md, only if the accepted target changes during implementation

Documentation requirements:

• Keep all unimplemented ideas under docs/roadmap/.
• Document messages[].cache_control in the prompt definition reference in docs/config.md.
• Document the outbound message serialization rule in docs/integrations/openai-compatible-chat.md.
• Document parsed cache usage fields in docs/integrations/openai-compatible-chat.md.
• In docs/cli.md, note that prompt cache control is configured in prompt YAML, not with CLI flags.
• In docs/troubleshooting.md, add cache miss guidance:
  • stable context should appear before the cache breakpoint
  • dynamic input before the breakpoint changes the cache key
  • providers may impose minimum token thresholds and breakpoint limits
  • use cached/cache-write token counts to verify behavior
• Do not claim support for deferred roadmap items.

Testing and verification:

go test ./...

If documentation examples include new YAML snippets, ensure they match the strict prompt loader schema.

Stage 6: Final Verification

Objective: verify the whole change set end to end.

Required commands:

go test ./...
go build ./cmd/scriptorium

Recommended smoke checks:

go run ./cmd/scriptorium render \
  --config ./examples/config.yml \
  --prompt generic.markdown_summary \
  --input transcript=./examples/fixtures/transcript.md \
  --input glossary=./examples/fixtures/glossary.yml \
  --format json

If adding a new example prompt with cache_control, also smoke-render that prompt with --format json and inspect that the rendered message includes cache_control.

Final review checklist:

• Existing prompt definitions without cache_control still load.
• Prompt definitions with cache_control render successfully.
• Unknown cache-control YAML fields are rejected.
• No-cache outbound messages still serialize with string content.
• Cache-controlled outbound messages serialize with text content-block arrays.
• rendered_prompt_hash changes when cache-control metadata changes.
• Cache usage fields are zero when providers omit them.
• CLI and HTTP request surfaces have no new knobs.
• CLI and HTTP response surfaces expose cache usage.
• Non-roadmap docs describe only implemented behavior.
• docs/roadmap/cache.md still defines target state and deferred work, not task steps.

Stage Boundaries

Stages 1 through 4 can be implemented in separate prompts if needed. Stage 5 should happen only after behavior is implemented. Stage 6 should be performed after all implementation and documentation changes are complete.

If a stage fails because prior stage changes were not implemented, stop and complete the earlier stage first rather than papering over missing domain fields or DTO mappings.

Open Questions

None required for the initial implementation.

The plan chooses message-level cache_control, string content for legacy messages, text-block array content only when cache control is present, and no new CLI/HTTP request knobs. These choices match docs/roadmap/cache.md and preserve existing prompt-definition compatibility.