Compare commits
5 Commits
1f63f8afbb
...
4d4bb7a121
| Author | SHA1 | Date | |
|---|---|---|---|
| 4d4bb7a121 | |||
| 5dcb3cd4fc | |||
| efe346893c | |||
| c95d6fcfec | |||
| 0badb4364d |
@@ -32,6 +32,7 @@ Integration references:
|
||||
- an effective `prompt_dir` and `profile_dir` (from flags or config)
|
||||
- `serve` requires an effective `prompt_dir` and `profile_dir` (from flags or config).
|
||||
- Positional arguments are rejected.
|
||||
- Prompt cache control is configured in prompt YAML (`messages[].cache_control`), not with CLI flags.
|
||||
|
||||
## Flag Reference
|
||||
|
||||
@@ -93,6 +94,7 @@ Notes:
|
||||
- Writes generated artifact content to stdout by default.
|
||||
- Writes generated artifact content to `--out` when provided.
|
||||
- Prints run summary metadata to stderr on success.
|
||||
- Appends `cached_tokens=<n> cache_write_tokens=<n>` to the summary only when the provider reports non-zero cache usage.
|
||||
- Prints errors to stderr on failure.
|
||||
|
||||
`render`:
|
||||
|
||||
@@ -119,6 +119,7 @@ Field reference:
|
||||
|
||||
- `role` (required)
|
||||
- `content` or `content_file` (exactly one is required)
|
||||
- `cache_control` (optional object): provider prompt-cache metadata for this message
|
||||
|
||||
Message rules:
|
||||
|
||||
@@ -128,6 +129,27 @@ Message rules:
|
||||
- Prompt decoding is strict; unknown YAML fields are rejected.
|
||||
- Duplicate prompt IDs are invalid. If multiple files declare the requested prompt ID, Scriptorium fails instead of choosing one.
|
||||
|
||||
`messages[].cache_control` fields:
|
||||
|
||||
- `type` (required when `cache_control` is present): currently only `ephemeral`.
|
||||
- `ttl` (optional): currently only `1h`; omitted from outbound requests when unset.
|
||||
|
||||
Example cache-controlled message:
|
||||
|
||||
```yaml
|
||||
messages:
|
||||
- role: system
|
||||
content_file: ./stable_context.md
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
ttl: 1h
|
||||
- role: user
|
||||
content: |
|
||||
{{input "transcript"}}
|
||||
```
|
||||
|
||||
Use cache control on stable reusable prompt content. Dynamic per-run inputs before the cache-controlled message change the provider cache key.
|
||||
|
||||
`output` fields:
|
||||
|
||||
- `format` (required): `text`, `markdown`, or `json`.
|
||||
@@ -184,6 +206,8 @@ Profile rules:
|
||||
Current outbound request behavior:
|
||||
|
||||
- The OpenAI-compatible client currently serializes: `model`, `messages`, `temperature`, `max_tokens`, `top_p`, `service_tier`, and optional `response_format` for `json_schema` prompts.
|
||||
- Messages without `cache_control` serialize with string `content`.
|
||||
- Messages with `cache_control` serialize as a single text content-block array containing `cache_control`.
|
||||
- `reasoning_effort` and `extra_params` are parsed and carried in effective settings, but are not currently serialized into outbound chat-completions requests.
|
||||
|
||||
## Schema Behavior
|
||||
|
||||
@@ -128,7 +128,9 @@ Response shape:
|
||||
"usage": {
|
||||
"prompt_tokens": 11,
|
||||
"completion_tokens": 22,
|
||||
"total_tokens": 33
|
||||
"total_tokens": 33,
|
||||
"cached_tokens": 0,
|
||||
"cache_write_tokens": 0
|
||||
},
|
||||
"start_time": "2026-05-04T12:00:00Z",
|
||||
"end_time": "2026-05-04T12:00:01Z",
|
||||
@@ -142,6 +144,8 @@ Response shape:
|
||||
|
||||
`raw_model_output` is omitted by default.
|
||||
|
||||
`metadata.usage.cached_tokens` and `metadata.usage.cache_write_tokens` are always present as numbers. They are `0` when the provider omits compatible cache usage fields or reports no cache activity.
|
||||
|
||||
To include it, send:
|
||||
|
||||
- `"include_raw_output": true`
|
||||
|
||||
@@ -26,7 +26,7 @@ Example:
|
||||
Serialized JSON fields:
|
||||
|
||||
- `model` (required after fallback resolution)
|
||||
- `messages` (role/content pairs from rendered prompt)
|
||||
- `messages` (rendered prompt messages)
|
||||
- `temperature` (only when non-zero)
|
||||
- `max_tokens` (only when non-zero)
|
||||
- `top_p` (only when non-zero)
|
||||
@@ -35,6 +35,35 @@ Serialized JSON fields:
|
||||
|
||||
`service_tier` is provider-specific. OpenRouter currently documents request values such as `flex` and `priority`; Scriptorium forwards any non-empty configured value and lets the backend validate support.
|
||||
|
||||
Messages without prompt cache control serialize with string `content`:
|
||||
|
||||
```json
|
||||
{
|
||||
"role": "system",
|
||||
"content": "rendered text"
|
||||
}
|
||||
```
|
||||
|
||||
Messages with prompt cache control serialize as a single text content-block array:
|
||||
|
||||
```json
|
||||
{
|
||||
"role": "system",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "rendered text",
|
||||
"cache_control": {
|
||||
"type": "ephemeral",
|
||||
"ttl": "1h"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
When cache-control `ttl` is unset in the prompt definition, `ttl` is omitted from the outbound payload.
|
||||
|
||||
Structured output is currently `json_schema` only, serialized as:
|
||||
|
||||
```json
|
||||
@@ -82,6 +111,13 @@ Expected successful response shape (subset used):
|
||||
- `usage.prompt_tokens`
|
||||
- `usage.completion_tokens`
|
||||
- `usage.total_tokens`
|
||||
- `usage.prompt_tokens_details.cached_tokens` (optional)
|
||||
- `usage.cache_write_tokens` (optional)
|
||||
|
||||
Absent cache usage fields are treated as zero. Parsed cache usage is exposed through run results and adapter response surfaces as:
|
||||
|
||||
- `cached_tokens`
|
||||
- `cache_write_tokens`
|
||||
|
||||
Malformed response conditions include:
|
||||
|
||||
@@ -104,6 +140,8 @@ The following fields may exist in profile/effective settings but are not current
|
||||
- `reasoning_effort`
|
||||
- `extra_params`
|
||||
|
||||
The client also does not serialize top-level `cache_control` or `session_id`.
|
||||
|
||||
No built-in retries, tool-calls, or multi-request payload modes are implemented in this client.
|
||||
|
||||
## Relationship To Runner
|
||||
|
||||
@@ -22,11 +22,13 @@ CLI adapter:
|
||||
|
||||
- Input: process args, filesystem config/assets, environment.
|
||||
- Output: exit code, stdout artifact/prepared output, stderr summaries/errors.
|
||||
- `run` summaries include cache usage counters only when either parsed cache counter is non-zero.
|
||||
|
||||
HTTP adapter:
|
||||
|
||||
- Input: JSON request body (`runRequestDTO`).
|
||||
- Output: JSON success/error body with mapped status codes.
|
||||
- Success metadata includes token usage plus cache usage counters.
|
||||
|
||||
Filesystem repositories:
|
||||
|
||||
@@ -93,6 +95,9 @@ Artifact refs:
|
||||
LLM adapter:
|
||||
|
||||
- endpoint appends `/chat/completions`.
|
||||
- rendered messages without cache control serialize with string `content`.
|
||||
- rendered messages with cache control serialize as one text content block with `cache_control`.
|
||||
- compatible cache usage response fields are parsed into domain token usage.
|
||||
- non-2xx responses map to request failure errors.
|
||||
- malformed responses (including missing/empty first choice content) are errors.
|
||||
|
||||
@@ -142,3 +147,4 @@ Behavior highlights:
|
||||
- External request/response strictness is part of contract stability.
|
||||
- Prepared-render output never includes resolved API key values.
|
||||
- Outbound OpenAI-compatible request includes only currently serialized fields (`model`, `messages`, optional `temperature`, `max_tokens`, `top_p`, optional `service_tier`, optional `response_format`).
|
||||
- Outbound cache control is message-level only; no top-level cache-control/session fields are serialized.
|
||||
|
||||
@@ -108,9 +108,11 @@ Validation content failures are not run errors:
|
||||
- only the environment-variable name is retained; secret value is never returned
|
||||
7. resolve output contract and structured-output schema payload when `json_schema` mode is active.
|
||||
8. read input artifacts.
|
||||
9. render prompt messages.
|
||||
9. render prompt messages, including any normalized message cache-control metadata.
|
||||
10. compute prompt/input/render hashes and return `PreparedRun`.
|
||||
|
||||
`rendered_prompt_hash` includes cache-control metadata when present because it affects the outbound provider request. Prompts without cache control keep the role/content hash behavior.
|
||||
|
||||
`Prepare` does not call the LLM.
|
||||
|
||||
## Run Flow
|
||||
@@ -123,7 +125,7 @@ Validation content failures are not run errors:
|
||||
4. build output artifact content type from output format.
|
||||
5. validate output.
|
||||
6. optionally attempt bounded repair when repairer is injected and contract allows it.
|
||||
7. return `RunResult` with artifact, raw output, validation, hashes, profile/model metadata, usage, and timestamps.
|
||||
7. return `RunResult` with artifact, raw output, validation, hashes, profile/model metadata, token/cache usage, and timestamps.
|
||||
|
||||
## Repair Hook Boundary
|
||||
|
||||
|
||||
@@ -252,6 +252,39 @@ Relevant links:
|
||||
- [Configuration reference](config.md)
|
||||
- [Operations guide](operations.md)
|
||||
|
||||
## Prompt Cache Misses Or No Cache Usage
|
||||
|
||||
Symptom:
|
||||
|
||||
- CLI run summary omits `cached_tokens` / `cache_write_tokens`.
|
||||
- HTTP `metadata.usage.cached_tokens` and `metadata.usage.cache_write_tokens` are both `0`.
|
||||
- Provider cost or latency does not improve after repeated similar runs.
|
||||
|
||||
Likely cause:
|
||||
|
||||
- The selected prompt has no `messages[].cache_control`.
|
||||
- Dynamic per-run input appears before the cache-controlled message and changes the provider cache key.
|
||||
- The provider does not support the serialized cache-control shape for the selected model.
|
||||
- The provider imposes minimum token thresholds or cache-breakpoint limits.
|
||||
|
||||
Diagnostic step:
|
||||
|
||||
- Run `render --format json` and verify the intended rendered message includes `cache_control`.
|
||||
- Confirm stable reusable context appears before the cache-controlled message, with dynamic input after it.
|
||||
- Check provider docs/logs for model support, minimum token thresholds, and breakpoint limits.
|
||||
|
||||
Safe fix:
|
||||
|
||||
- Move stable reusable context before the cache-controlled message.
|
||||
- Move highly dynamic input after the cache breakpoint.
|
||||
- Keep `cache_control.type: ephemeral` and, when using `ttl`, set `ttl: 1h`.
|
||||
- Use CLI cache counters or HTTP cache usage fields to verify cache reads/writes after rerunning.
|
||||
|
||||
Relevant links:
|
||||
|
||||
- [Configuration reference](config.md)
|
||||
- [OpenAI-compatible chat integration](integrations/openai-compatible-chat.md)
|
||||
|
||||
## Validation Status Failed (`run` Exit 2 Or HTTP 200 With Failed Status)
|
||||
|
||||
Symptom:
|
||||
|
||||
@@ -606,7 +606,7 @@ func printSummary(stderr io.Writer, res *domain.RunResult) {
|
||||
if res == nil {
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(stderr, "prompt=%s@%s selected_profile=%s model=%s validation=%s mode=%s validation_errors=%d prompt_hash=%s inputs=%d usage=%d/%d/%d\n",
|
||||
fmt.Fprintf(stderr, "prompt=%s@%s selected_profile=%s model=%s validation=%s mode=%s validation_errors=%d prompt_hash=%s inputs=%d usage=%d/%d/%d",
|
||||
res.PromptID,
|
||||
res.PromptVersion,
|
||||
res.SelectedProfileID,
|
||||
@@ -620,6 +620,10 @@ func printSummary(stderr io.Writer, res *domain.RunResult) {
|
||||
res.Usage.CompletionTokens,
|
||||
res.Usage.TotalTokens,
|
||||
)
|
||||
if res.Usage.CachedTokens != 0 || res.Usage.CacheWriteTokens != 0 {
|
||||
fmt.Fprintf(stderr, " cached_tokens=%d cache_write_tokens=%d", res.Usage.CachedTokens, res.Usage.CacheWriteTokens)
|
||||
}
|
||||
fmt.Fprintln(stderr)
|
||||
}
|
||||
|
||||
func printUsage(w io.Writer) {
|
||||
|
||||
@@ -1064,6 +1064,38 @@ func TestWriteOutputAndSummaryUseSeparateWriters(t *testing.T) {
|
||||
if !strings.Contains(stderr.String(), "prompt=p@1") {
|
||||
t.Fatalf("expected summary on stderr, got %q", stderr.String())
|
||||
}
|
||||
if strings.Contains(stderr.String(), "cached_tokens=") || strings.Contains(stderr.String(), "cache_write_tokens=") {
|
||||
t.Fatalf("expected zero cache usage to be omitted from summary, got %q", stderr.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintSummaryIncludesCacheUsageWhenPresent(t *testing.T) {
|
||||
var stderr bytes.Buffer
|
||||
|
||||
printSummary(&stderr, &domain.RunResult{
|
||||
PromptID: "p",
|
||||
PromptVersion: "1",
|
||||
SelectedProfileID: "exec",
|
||||
ModelName: "m",
|
||||
Validation: domain.ValidationResult{Status: domain.ValidationPassed, Mode: domain.ValidationBasic},
|
||||
RenderedPromptHash: "h",
|
||||
InputHashes: map[string]string{"in": "x"},
|
||||
Usage: domain.TokenUsage{
|
||||
PromptTokens: 10,
|
||||
CompletionTokens: 5,
|
||||
TotalTokens: 15,
|
||||
CachedTokens: 0,
|
||||
CacheWriteTokens: 3,
|
||||
},
|
||||
})
|
||||
|
||||
summary := stderr.String()
|
||||
if !strings.Contains(summary, "usage=10/5/15") {
|
||||
t.Fatalf("expected base usage summary, got %q", summary)
|
||||
}
|
||||
if !strings.Contains(summary, "cached_tokens=0 cache_write_tokens=3") {
|
||||
t.Fatalf("expected cache usage in summary, got %q", summary)
|
||||
}
|
||||
}
|
||||
|
||||
type cliTestLibrary struct {
|
||||
|
||||
@@ -86,6 +86,8 @@ type tokenUsageDTO struct {
|
||||
PromptTokens int `json:"prompt_tokens"`
|
||||
CompletionTokens int `json:"completion_tokens"`
|
||||
TotalTokens int `json:"total_tokens"`
|
||||
CachedTokens int `json:"cached_tokens"`
|
||||
CacheWriteTokens int `json:"cache_write_tokens"`
|
||||
}
|
||||
|
||||
type validationDTO struct {
|
||||
|
||||
@@ -105,6 +105,8 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
PromptTokens: res.Usage.PromptTokens,
|
||||
CompletionTokens: res.Usage.CompletionTokens,
|
||||
TotalTokens: res.Usage.TotalTokens,
|
||||
CachedTokens: res.Usage.CachedTokens,
|
||||
CacheWriteTokens: res.Usage.CacheWriteTokens,
|
||||
},
|
||||
StartTime: res.StartTime,
|
||||
EndTime: res.EndTime,
|
||||
|
||||
@@ -66,11 +66,17 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
|
||||
APIKeyEnv: envName,
|
||||
},
|
||||
InputHashes: map[string]string{"transcript": "h1"},
|
||||
Usage: domain.TokenUsage{PromptTokens: 1, CompletionTokens: 2, TotalTokens: 3},
|
||||
StartTime: start,
|
||||
EndTime: end,
|
||||
Duration: 2 * time.Second,
|
||||
RawOutput: "hello",
|
||||
Usage: domain.TokenUsage{
|
||||
PromptTokens: 1,
|
||||
CompletionTokens: 2,
|
||||
TotalTokens: 3,
|
||||
CachedTokens: 4,
|
||||
CacheWriteTokens: 5,
|
||||
},
|
||||
StartTime: start,
|
||||
EndTime: end,
|
||||
Duration: 2 * time.Second,
|
||||
RawOutput: "hello",
|
||||
}}
|
||||
|
||||
h := NewHandler(r)
|
||||
@@ -111,6 +117,13 @@ func TestHandlerPostRunsSuccessWithExplicitProfileID(t *testing.T) {
|
||||
if metadata["model_name"] != "m1" || metadata["endpoint"] != "http://llm/v1" {
|
||||
t.Fatalf("unexpected model metadata: name=%#v endpoint=%#v", metadata["model_name"], metadata["endpoint"])
|
||||
}
|
||||
usage := metadata["usage"].(map[string]any)
|
||||
if usage["prompt_tokens"] != float64(1) || usage["completion_tokens"] != float64(2) || usage["total_tokens"] != float64(3) {
|
||||
t.Fatalf("unexpected base usage metadata: %#v", usage)
|
||||
}
|
||||
if usage["cached_tokens"] != float64(4) || usage["cache_write_tokens"] != float64(5) {
|
||||
t.Fatalf("unexpected cache usage metadata: %#v", usage)
|
||||
}
|
||||
modelParams := metadata["model_params"].(map[string]any)
|
||||
if modelParams["api_key_env"] != envName {
|
||||
t.Fatalf("expected model_params.api_key_env=%q, got %#v", envName, modelParams["api_key_env"])
|
||||
@@ -171,6 +184,10 @@ func TestHandlerPostRunsSuccessUsingPromptDefaultProfile(t *testing.T) {
|
||||
if metadata["selected_profile_id"] != "prompt-default" {
|
||||
t.Fatalf("expected selected_profile_id from result, got %#v", metadata["selected_profile_id"])
|
||||
}
|
||||
usage := metadata["usage"].(map[string]any)
|
||||
if usage["cached_tokens"] != float64(0) || usage["cache_write_tokens"] != float64(0) {
|
||||
t.Fatalf("expected zero cache usage fields to be included, got %#v", usage)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerModelOverrideMapsAllSupportedExecutionFields(t *testing.T) {
|
||||
|
||||
@@ -40,6 +40,19 @@ const (
|
||||
ValidationSkipped ValidationStatus = "skipped"
|
||||
)
|
||||
|
||||
// CacheControlType defines provider cache behavior for prompt content.
|
||||
type CacheControlType string
|
||||
|
||||
const (
|
||||
CacheControlEphemeral CacheControlType = "ephemeral"
|
||||
)
|
||||
|
||||
// CacheControl describes provider cache metadata attached to prompt content.
|
||||
type CacheControl struct {
|
||||
Type CacheControlType `yaml:"type" json:"type"`
|
||||
TTL string `yaml:"ttl,omitempty" json:"ttl,omitempty"`
|
||||
}
|
||||
|
||||
// RunRequest represents a request to generate a single artifact.
|
||||
type RunRequest struct {
|
||||
PromptID string
|
||||
@@ -131,9 +144,10 @@ type PromptInput struct {
|
||||
|
||||
// PromptMessageTemplate defines a template for a chat message.
|
||||
type PromptMessageTemplate struct {
|
||||
Role string `yaml:"role"`
|
||||
Content string `yaml:"content"`
|
||||
ContentFile string `yaml:"content_file"`
|
||||
Role string `yaml:"role"`
|
||||
Content string `yaml:"content"`
|
||||
ContentFile string `yaml:"content_file"`
|
||||
CacheControl *CacheControl `yaml:"cache_control,omitempty" json:"cache_control,omitempty"`
|
||||
}
|
||||
|
||||
// ExecutionProfile describes how and where to execute a model.
|
||||
@@ -180,8 +194,9 @@ type RenderedPrompt struct {
|
||||
|
||||
// RenderedMessage is a single message in a rendered prompt.
|
||||
type RenderedMessage struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
CacheControl *CacheControl `json:"cache_control,omitempty"`
|
||||
}
|
||||
|
||||
// GenerateRequest is the internal request passed to the LLM client.
|
||||
@@ -222,6 +237,8 @@ type TokenUsage struct {
|
||||
PromptTokens int
|
||||
CompletionTokens int
|
||||
TotalTokens int
|
||||
CachedTokens int
|
||||
CacheWriteTokens int
|
||||
}
|
||||
|
||||
// ValidationResult represents the outcome of an output validation.
|
||||
|
||||
@@ -53,3 +53,52 @@ func TestPreparedRunJSONDoesNotIncludeSecretValues(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPreparedRunJSONIncludesMessageCacheControlOnlyWhenPresent(t *testing.T) {
|
||||
prepared := PreparedRun{
|
||||
PromptID: "prompt.id",
|
||||
SelectedProfileID: "local-fast",
|
||||
EffectiveModelParams: ExecutionTarget{
|
||||
Endpoint: "http://llm/v1",
|
||||
Model: "gpt-test",
|
||||
},
|
||||
RenderedPromptHash: "rendered-hash",
|
||||
Messages: []RenderedMessage{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "You are helpful.",
|
||||
CacheControl: &CacheControl{
|
||||
Type: CacheControlEphemeral,
|
||||
TTL: "1h",
|
||||
},
|
||||
},
|
||||
{Role: "user", Content: "Summarize this."},
|
||||
},
|
||||
}
|
||||
|
||||
b, err := json.Marshal(prepared)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal failed: %v", err)
|
||||
}
|
||||
|
||||
var decoded struct {
|
||||
Messages []map[string]any `json:"messages"`
|
||||
}
|
||||
if err := json.Unmarshal(b, &decoded); err != nil {
|
||||
t.Fatalf("unmarshal failed: %v", err)
|
||||
}
|
||||
if len(decoded.Messages) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(decoded.Messages))
|
||||
}
|
||||
|
||||
cacheControl, ok := decoded.Messages[0]["cache_control"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected cache_control on first message, got %#v", decoded.Messages[0])
|
||||
}
|
||||
if cacheControl["type"] != string(CacheControlEphemeral) || cacheControl["ttl"] != "1h" {
|
||||
t.Fatalf("unexpected cache_control payload: %#v", cacheControl)
|
||||
}
|
||||
if _, ok := decoded.Messages[1]["cache_control"]; ok {
|
||||
t.Fatalf("expected second message to omit cache_control, got %#v", decoded.Messages[1])
|
||||
}
|
||||
}
|
||||
|
||||
@@ -151,6 +151,13 @@ func (textPreparedRunFormatter) Format(prepared *domain.PreparedRun) ([]byte, er
|
||||
messages := byRole[role]
|
||||
for i, msg := range messages {
|
||||
fmt.Fprintf(&b, " - message: %d\n", i+1)
|
||||
if msg.CacheControl != nil {
|
||||
fmt.Fprintf(&b, " cache_control: %s", msg.CacheControl.Type)
|
||||
if msg.CacheControl.TTL != "" {
|
||||
fmt.Fprintf(&b, " ttl=%s", msg.CacheControl.TTL)
|
||||
}
|
||||
fmt.Fprintln(&b)
|
||||
}
|
||||
fmt.Fprintln(&b, " content: |")
|
||||
content := msg.Content
|
||||
if content == "" {
|
||||
|
||||
@@ -62,6 +62,58 @@ func TestTextFormatterDoesNotIncludeResolvedAPIKeyValue(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTextFormatterIncludesMessageCacheControlBeforeContent(t *testing.T) {
|
||||
prepared := samplePreparedRun()
|
||||
prepared.Messages = []domain.RenderedMessage{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "System guidance.",
|
||||
CacheControl: &domain.CacheControl{
|
||||
Type: domain.CacheControlEphemeral,
|
||||
TTL: "1h",
|
||||
},
|
||||
},
|
||||
{Role: "user", Content: "Summarize the transcript."},
|
||||
}
|
||||
|
||||
out, err := FormatPreparedRun(prepared, PreparedRunFormatText)
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
s := string(out)
|
||||
if !strings.Contains(s, " system:\n - message: 1\n cache_control: ephemeral ttl=1h\n content: |") {
|
||||
t.Fatalf("expected system message cache control before content, got:\n%s", s)
|
||||
}
|
||||
if strings.Count(s, "cache_control:") != 1 {
|
||||
t.Fatalf("expected exactly one cache_control line, got:\n%s", s)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTextFormatterOmitsEmptyCacheControlTTL(t *testing.T) {
|
||||
prepared := samplePreparedRun()
|
||||
prepared.Messages = []domain.RenderedMessage{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "System guidance.",
|
||||
CacheControl: &domain.CacheControl{
|
||||
Type: domain.CacheControlEphemeral,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
out, err := FormatPreparedRun(prepared, PreparedRunFormatText)
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
s := string(out)
|
||||
if !strings.Contains(s, " cache_control: ephemeral\n") {
|
||||
t.Fatalf("expected cache_control line without ttl, got:\n%s", s)
|
||||
}
|
||||
if strings.Contains(s, "ttl=") {
|
||||
t.Fatalf("expected empty ttl to be omitted, got:\n%s", s)
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONFormatterEmitsValidJSONAndIncludesPreparedRunFields(t *testing.T) {
|
||||
prepared := samplePreparedRun()
|
||||
|
||||
@@ -98,6 +150,47 @@ func TestJSONFormatterEmitsValidJSONAndIncludesPreparedRunFields(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONFormatterIncludesMessageCacheControlOnlyWhenPresent(t *testing.T) {
|
||||
prepared := samplePreparedRun()
|
||||
prepared.Messages = []domain.RenderedMessage{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "System guidance.",
|
||||
CacheControl: &domain.CacheControl{
|
||||
Type: domain.CacheControlEphemeral,
|
||||
TTL: "1h",
|
||||
},
|
||||
},
|
||||
{Role: "user", Content: "Summarize the transcript."},
|
||||
}
|
||||
|
||||
out, err := FormatPreparedRun(prepared, PreparedRunFormatJSON)
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
|
||||
var decoded struct {
|
||||
Messages []map[string]any `json:"messages"`
|
||||
}
|
||||
if err := json.Unmarshal(out, &decoded); err != nil {
|
||||
t.Fatalf("expected valid json output, got %v", err)
|
||||
}
|
||||
if len(decoded.Messages) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(decoded.Messages))
|
||||
}
|
||||
|
||||
cacheControl, ok := decoded.Messages[0]["cache_control"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected first message cache_control, got %#v", decoded.Messages[0])
|
||||
}
|
||||
if cacheControl["type"] != string(domain.CacheControlEphemeral) || cacheControl["ttl"] != "1h" {
|
||||
t.Fatalf("unexpected cache_control payload: %#v", cacheControl)
|
||||
}
|
||||
if _, ok := decoded.Messages[1]["cache_control"]; ok {
|
||||
t.Fatalf("expected second message to omit cache_control, got %#v", decoded.Messages[1])
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONFormatterDoesNotIncludeResolvedAPIKeyValue(t *testing.T) {
|
||||
const secret = "super-secret-api-key"
|
||||
t.Setenv("SCRIPTORIUM_API_KEY", secret)
|
||||
|
||||
@@ -151,6 +151,8 @@ func (c *OpenAICompatibleClient) Generate(ctx context.Context, req domain.Genera
|
||||
PromptTokens: wireResp.Usage.PromptTokens,
|
||||
CompletionTokens: wireResp.Usage.CompletionTokens,
|
||||
TotalTokens: wireResp.Usage.TotalTokens,
|
||||
CachedTokens: wireResp.Usage.PromptTokensDetails.CachedTokens,
|
||||
CacheWriteTokens: wireResp.Usage.CacheWriteTokens,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
@@ -168,12 +170,9 @@ func openAIChatRequestFromGenerateRequest(req domain.GenerateRequest, defaultMod
|
||||
Model: model,
|
||||
}
|
||||
|
||||
wireReq.Messages = make([]openAIChatMessage, 0, len(req.Prompt.Messages))
|
||||
wireReq.Messages = make([]openAIChatRequestMessage, 0, len(req.Prompt.Messages))
|
||||
for _, msg := range req.Prompt.Messages {
|
||||
wireReq.Messages = append(wireReq.Messages, openAIChatMessage{
|
||||
Role: msg.Role,
|
||||
Content: msg.Content,
|
||||
})
|
||||
wireReq.Messages = append(wireReq.Messages, openAIChatRequestMessageFromRenderedMessage(msg))
|
||||
}
|
||||
|
||||
if req.Target.Temperature != 0 {
|
||||
@@ -200,28 +199,48 @@ func openAIChatRequestFromGenerateRequest(req domain.GenerateRequest, defaultMod
|
||||
}
|
||||
|
||||
type openAIChatRequest struct {
|
||||
Model string `json:"model"`
|
||||
Messages []openAIChatMessage `json:"messages"`
|
||||
Temperature *float64 `json:"temperature,omitempty"`
|
||||
MaxTokens *int `json:"max_tokens,omitempty"`
|
||||
TopP *float64 `json:"top_p,omitempty"`
|
||||
ServiceTier string `json:"service_tier,omitempty"`
|
||||
ResponseFormat *openAIResponseFormat `json:"response_format,omitempty"`
|
||||
Model string `json:"model"`
|
||||
Messages []openAIChatRequestMessage `json:"messages"`
|
||||
Temperature *float64 `json:"temperature,omitempty"`
|
||||
MaxTokens *int `json:"max_tokens,omitempty"`
|
||||
TopP *float64 `json:"top_p,omitempty"`
|
||||
ServiceTier string `json:"service_tier,omitempty"`
|
||||
ResponseFormat *openAIResponseFormat `json:"response_format,omitempty"`
|
||||
}
|
||||
|
||||
type openAIChatMessage struct {
|
||||
type openAIChatRequestMessage struct {
|
||||
Role string `json:"role"`
|
||||
Content any `json:"content"`
|
||||
}
|
||||
|
||||
type openAIChatTextContentBlock struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
CacheControl *openAICacheControl `json:"cache_control,omitempty"`
|
||||
}
|
||||
|
||||
type openAICacheControl struct {
|
||||
Type string `json:"type"`
|
||||
TTL string `json:"ttl,omitempty"`
|
||||
}
|
||||
|
||||
type openAIChatResponseMessage struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
}
|
||||
|
||||
type openAIChatResponse struct {
|
||||
Choices []struct {
|
||||
Message openAIChatMessage `json:"message"`
|
||||
Message openAIChatResponseMessage `json:"message"`
|
||||
} `json:"choices"`
|
||||
Usage struct {
|
||||
PromptTokens int `json:"prompt_tokens"`
|
||||
CompletionTokens int `json:"completion_tokens"`
|
||||
TotalTokens int `json:"total_tokens"`
|
||||
PromptTokens int `json:"prompt_tokens"`
|
||||
CompletionTokens int `json:"completion_tokens"`
|
||||
TotalTokens int `json:"total_tokens"`
|
||||
PromptTokensDetails struct {
|
||||
CachedTokens int `json:"cached_tokens"`
|
||||
} `json:"prompt_tokens_details"`
|
||||
CacheWriteTokens int `json:"cache_write_tokens"`
|
||||
} `json:"usage"`
|
||||
}
|
||||
|
||||
@@ -236,6 +255,28 @@ type openAIJSONSchemaEnvelope struct {
|
||||
Schema any `json:"schema"`
|
||||
}
|
||||
|
||||
func openAIChatRequestMessageFromRenderedMessage(msg domain.RenderedMessage) openAIChatRequestMessage {
|
||||
wireMsg := openAIChatRequestMessage{
|
||||
Role: msg.Role,
|
||||
Content: msg.Content,
|
||||
}
|
||||
if msg.CacheControl == nil {
|
||||
return wireMsg
|
||||
}
|
||||
|
||||
wireMsg.Content = []openAIChatTextContentBlock{
|
||||
{
|
||||
Type: "text",
|
||||
Text: msg.Content,
|
||||
CacheControl: &openAICacheControl{
|
||||
Type: string(msg.CacheControl.Type),
|
||||
TTL: msg.CacheControl.TTL,
|
||||
},
|
||||
},
|
||||
}
|
||||
return wireMsg
|
||||
}
|
||||
|
||||
func toOpenAIResponseFormat(spec *domain.StructuredOutputSpec) (*openAIResponseFormat, error) {
|
||||
if spec == nil {
|
||||
return nil, nil
|
||||
|
||||
@@ -89,6 +89,9 @@ func TestOpenAICompatibleClientGenerateSuccess(t *testing.T) {
|
||||
if resp.Usage.PromptTokens != 11 || resp.Usage.CompletionTokens != 22 || resp.Usage.TotalTokens != 33 {
|
||||
t.Fatalf("unexpected usage: %+v", resp.Usage)
|
||||
}
|
||||
if resp.Usage.CachedTokens != 0 || resp.Usage.CacheWriteTokens != 0 {
|
||||
t.Fatalf("expected absent cache usage fields to remain zero, got %+v", resp.Usage)
|
||||
}
|
||||
|
||||
if obs.Authorization != "Bearer secret-key" {
|
||||
t.Fatalf("unexpected Authorization header: %q", obs.Authorization)
|
||||
@@ -144,6 +147,155 @@ func TestOpenAICompatibleClientGenerateSuccess(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientSerializesCacheControlledMessageAsContentBlock(t *testing.T) {
|
||||
var observedBody map[string]any
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
defer r.Body.Close()
|
||||
if err := json.NewDecoder(r.Body).Decode(&observedBody); err != nil {
|
||||
t.Fatalf("failed to decode request body: %v", err)
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: ts.URL + "/v1"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "Stable instructions.",
|
||||
CacheControl: &domain.CacheControl{
|
||||
Type: domain.CacheControlEphemeral,
|
||||
TTL: "1h",
|
||||
},
|
||||
},
|
||||
{Role: "user", Content: "Dynamic request."},
|
||||
}},
|
||||
Target: domain.ExecutionTarget{Model: "model"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
|
||||
for _, forbidden := range []string{"cache_control", "session_id", "extra_params"} {
|
||||
if _, exists := observedBody[forbidden]; exists {
|
||||
t.Fatalf("expected top-level %s to be omitted, got %#v", forbidden, observedBody[forbidden])
|
||||
}
|
||||
}
|
||||
|
||||
msgs, ok := observedBody["messages"].([]any)
|
||||
if !ok || len(msgs) != 2 {
|
||||
t.Fatalf("unexpected messages payload: %#v", observedBody["messages"])
|
||||
}
|
||||
msg0 := msgs[0].(map[string]any)
|
||||
if msg0["role"] != "system" {
|
||||
t.Fatalf("unexpected first message role: %#v", msg0["role"])
|
||||
}
|
||||
contentBlocks, ok := msg0["content"].([]any)
|
||||
if !ok || len(contentBlocks) != 1 {
|
||||
t.Fatalf("expected first message content block array, got %#v", msg0["content"])
|
||||
}
|
||||
block := contentBlocks[0].(map[string]any)
|
||||
if block["type"] != "text" || block["text"] != "Stable instructions." {
|
||||
t.Fatalf("unexpected text content block: %#v", block)
|
||||
}
|
||||
cacheControl, ok := block["cache_control"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected cache_control on content block, got %#v", block)
|
||||
}
|
||||
if cacheControl["type"] != string(domain.CacheControlEphemeral) || cacheControl["ttl"] != "1h" {
|
||||
t.Fatalf("unexpected cache_control payload: %#v", cacheControl)
|
||||
}
|
||||
|
||||
msg1 := msgs[1].(map[string]any)
|
||||
if msg1["role"] != "user" || msg1["content"] != "Dynamic request." {
|
||||
t.Fatalf("expected uncached message to keep string content, got %#v", msg1)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientOmitsEmptyCacheControlTTL(t *testing.T) {
|
||||
var observedBody map[string]any
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
defer r.Body.Close()
|
||||
if err := json.NewDecoder(r.Body).Decode(&observedBody); err != nil {
|
||||
t.Fatalf("failed to decode request body: %v", err)
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: ts.URL + "/v1"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "Stable instructions.",
|
||||
CacheControl: &domain.CacheControl{
|
||||
Type: domain.CacheControlEphemeral,
|
||||
},
|
||||
},
|
||||
}},
|
||||
Target: domain.ExecutionTarget{Model: "model"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
|
||||
msgs := observedBody["messages"].([]any)
|
||||
msg0 := msgs[0].(map[string]any)
|
||||
contentBlocks := msg0["content"].([]any)
|
||||
block := contentBlocks[0].(map[string]any)
|
||||
cacheControl := block["cache_control"].(map[string]any)
|
||||
if cacheControl["type"] != string(domain.CacheControlEphemeral) {
|
||||
t.Fatalf("unexpected cache_control type: %#v", cacheControl)
|
||||
}
|
||||
if _, exists := cacheControl["ttl"]; exists {
|
||||
t.Fatalf("expected empty ttl to be omitted, got %#v", cacheControl)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientParsesCacheUsage(t *testing.T) {
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = w.Write([]byte(`{
|
||||
"choices": [{"message": {"role": "assistant", "content": "ok"}}],
|
||||
"usage": {
|
||||
"prompt_tokens": 100,
|
||||
"completion_tokens": 20,
|
||||
"total_tokens": 120,
|
||||
"prompt_tokens_details": {"cached_tokens": 80},
|
||||
"cache_write_tokens": 60
|
||||
}
|
||||
}`))
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: ts.URL + "/v1", Model: "model"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
resp, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if resp.Usage.PromptTokens != 100 || resp.Usage.CompletionTokens != 20 || resp.Usage.TotalTokens != 120 {
|
||||
t.Fatalf("unexpected base usage fields: %+v", resp.Usage)
|
||||
}
|
||||
if resp.Usage.CachedTokens != 80 || resp.Usage.CacheWriteTokens != 60 {
|
||||
t.Fatalf("unexpected cache usage fields: %+v", resp.Usage)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenAICompatibleClientOmitsResponseFormatWhenNoStructuredOutput(t *testing.T) {
|
||||
var observedBody map[string]any
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
@@ -75,8 +75,9 @@ func (r *goRenderer) Render(ctx context.Context, definition *domain.PromptDefini
|
||||
}
|
||||
|
||||
renderedMessages = append(renderedMessages, domain.RenderedMessage{
|
||||
Role: tmplMsg.Role,
|
||||
Content: buf.String(),
|
||||
Role: tmplMsg.Role,
|
||||
Content: buf.String(),
|
||||
CacheControl: cloneCacheControl(tmplMsg.CacheControl),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -84,3 +85,11 @@ func (r *goRenderer) Render(ctx context.Context, definition *domain.PromptDefini
|
||||
Messages: renderedMessages,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func cloneCacheControl(in *domain.CacheControl) *domain.CacheControl {
|
||||
if in == nil {
|
||||
return nil
|
||||
}
|
||||
out := *in
|
||||
return &out
|
||||
}
|
||||
|
||||
@@ -78,6 +78,66 @@ func TestGoRenderer_Render(t *testing.T) {
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("copying cache control to rendered messages", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "You are concise.",
|
||||
CacheControl: &domain.CacheControl{
|
||||
Type: domain.CacheControlEphemeral,
|
||||
TTL: "1h",
|
||||
},
|
||||
},
|
||||
{Role: "user", Content: "Analyze this: {{input \"transcript\"}}"},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(res.Messages) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(res.Messages))
|
||||
}
|
||||
if res.Messages[0].CacheControl == nil {
|
||||
t.Fatal("expected rendered cache control")
|
||||
}
|
||||
if res.Messages[0].CacheControl.Type != domain.CacheControlEphemeral {
|
||||
t.Fatalf("unexpected cache control type: %q", res.Messages[0].CacheControl.Type)
|
||||
}
|
||||
if res.Messages[0].CacheControl.TTL != "1h" {
|
||||
t.Fatalf("unexpected cache control ttl: %q", res.Messages[0].CacheControl.TTL)
|
||||
}
|
||||
if res.Messages[1].CacheControl != nil {
|
||||
t.Fatalf("expected no cache control on second message, got %#v", res.Messages[1].CacheControl)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rendered cache control does not alias source template", func(t *testing.T) {
|
||||
source := &domain.CacheControl{Type: domain.CacheControlEphemeral, TTL: "1h"}
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "You are concise.", CacheControl: source},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if res.Messages[0].CacheControl == source {
|
||||
t.Fatal("expected rendered cache control to be cloned")
|
||||
}
|
||||
|
||||
res.Messages[0].CacheControl.TTL = ""
|
||||
if source.TTL != "1h" {
|
||||
t.Fatalf("source cache control was mutated, ttl=%q", source.TTL)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("accessing vars", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
|
||||
@@ -42,9 +42,15 @@ type promptInputFile struct {
|
||||
}
|
||||
|
||||
type promptMessageFile struct {
|
||||
Role string `yaml:"role"`
|
||||
Content string `yaml:"content"`
|
||||
ContentFile string `yaml:"content_file"`
|
||||
Role string `yaml:"role"`
|
||||
Content string `yaml:"content"`
|
||||
ContentFile string `yaml:"content_file"`
|
||||
CacheControl *cacheControlFile `yaml:"cache_control"`
|
||||
}
|
||||
|
||||
type cacheControlFile struct {
|
||||
Type string `yaml:"type"`
|
||||
TTL string `yaml:"ttl"`
|
||||
}
|
||||
|
||||
type promptOutputContractFile struct {
|
||||
@@ -212,6 +218,11 @@ func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*d
|
||||
return nil, fmt.Errorf("message %d (%s) must set exactly one of content or content_file", i, role)
|
||||
}
|
||||
|
||||
cacheControl, err := normalizeCacheControl(msg.CacheControl)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("message %d (%s) cache_control: %w", i, role, err)
|
||||
}
|
||||
|
||||
templateContent := msg.Content
|
||||
resolvedContentFile := ""
|
||||
if hasContentFile {
|
||||
@@ -230,9 +241,10 @@ func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*d
|
||||
}
|
||||
|
||||
templates = append(templates, domain.PromptMessageTemplate{
|
||||
Role: role,
|
||||
Content: templateContent,
|
||||
ContentFile: resolvedContentFile,
|
||||
Role: role,
|
||||
Content: templateContent,
|
||||
ContentFile: resolvedContentFile,
|
||||
CacheControl: cacheControl,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -274,6 +286,30 @@ func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*d
|
||||
}, nil
|
||||
}
|
||||
|
||||
func normalizeCacheControl(raw *cacheControlFile) (*domain.CacheControl, error) {
|
||||
if raw == nil {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
cacheType := strings.TrimSpace(raw.Type)
|
||||
if cacheType == "" {
|
||||
return nil, errors.New("type is required")
|
||||
}
|
||||
if domain.CacheControlType(cacheType) != domain.CacheControlEphemeral {
|
||||
return nil, fmt.Errorf("unsupported type %q", cacheType)
|
||||
}
|
||||
|
||||
ttl := strings.TrimSpace(raw.TTL)
|
||||
if ttl != "" && ttl != "1h" {
|
||||
return nil, fmt.Errorf("unsupported ttl %q", ttl)
|
||||
}
|
||||
|
||||
return &domain.CacheControl{
|
||||
Type: domain.CacheControlType(cacheType),
|
||||
TTL: ttl,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func isValidOutputFormat(f domain.OutputFormat) bool {
|
||||
switch f {
|
||||
case domain.FormatText, domain.FormatMarkdown, domain.FormatJSON:
|
||||
|
||||
@@ -68,6 +68,34 @@ func TestFilesystemRepository_GetPromptDefinition(t *testing.T) {
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid cache control with ttl", func(t *testing.T) {
|
||||
p, err := repo.GetPromptDefinition(ctx, "valid-cache-control-ttl", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if len(p.Templates) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(p.Templates))
|
||||
}
|
||||
assertCacheControl(t, p.Templates[0].CacheControl, domain.CacheControlEphemeral, "1h")
|
||||
if p.Templates[1].CacheControl != nil {
|
||||
t.Fatalf("expected second message cache control to be nil, got %#v", p.Templates[1].CacheControl)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid cache control without ttl", func(t *testing.T) {
|
||||
p, err := repo.GetPromptDefinition(ctx, "valid-cache-control-without-ttl", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if len(p.Templates) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(p.Templates))
|
||||
}
|
||||
assertCacheControl(t, p.Templates[0].CacheControl, domain.CacheControlEphemeral, "")
|
||||
if p.Templates[1].CacheControl != nil {
|
||||
t.Fatalf("expected second message cache control to be nil, got %#v", p.Templates[1].CacheControl)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid nested file-backed prompt resolves content file relative to nested YAML", func(t *testing.T) {
|
||||
nestedDir := filepath.Join(tmpDir, "dnd", "recap")
|
||||
if err := os.MkdirAll(nestedDir, 0o755); err != nil {
|
||||
@@ -258,6 +286,10 @@ output:
|
||||
{name: "invalid validation mode", id: "invalid_validation_mode", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"invalid validation mode"}},
|
||||
{name: "json_schema without schema_path", id: "json_schema_without_schema_path", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"schema_path"}},
|
||||
{name: "unknown input field", id: "unknown_input_field", targetErr: ErrInvalidYAML, errSubstrs: []string{"field unknown_input_setting not found"}},
|
||||
{name: "empty cache control type", id: "empty_cache_control_type", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"cache_control", "type is required"}},
|
||||
{name: "unsupported cache control type", id: "unsupported_cache_control_type", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"cache_control", "unsupported type"}},
|
||||
{name: "unsupported cache control ttl", id: "unsupported_cache_control_ttl", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"cache_control", "unsupported ttl"}},
|
||||
{name: "unknown cache control field", id: "unknown_cache_control_field", targetErr: ErrInvalidYAML, errSubstrs: []string{"field unexpected not found"}},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
@@ -282,6 +314,19 @@ output:
|
||||
})
|
||||
}
|
||||
|
||||
func assertCacheControl(t *testing.T, got *domain.CacheControl, wantType domain.CacheControlType, wantTTL string) {
|
||||
t.Helper()
|
||||
if got == nil {
|
||||
t.Fatal("expected cache control, got nil")
|
||||
}
|
||||
if got.Type != wantType {
|
||||
t.Fatalf("unexpected cache control type: got %q want %q", got.Type, wantType)
|
||||
}
|
||||
if got.TTL != wantTTL {
|
||||
t.Fatalf("unexpected cache control ttl: got %q want %q", got.TTL, wantTTL)
|
||||
}
|
||||
}
|
||||
|
||||
func writePromptTestFile(t *testing.T, path string, content string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(path, []byte(strings.TrimLeft(content, "\n")), 0o644); err != nil {
|
||||
|
||||
10
internal/promptdef/testdata/empty_cache_control_type.yaml
vendored
Normal file
10
internal/promptdef/testdata/empty_cache_control_type.yaml
vendored
Normal file
@@ -0,0 +1,10 @@
|
||||
id: empty-cache-control-type
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control: {}
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
12
internal/promptdef/testdata/unknown_cache_control_field.yaml
vendored
Normal file
12
internal/promptdef/testdata/unknown_cache_control_field.yaml
vendored
Normal file
@@ -0,0 +1,12 @@
|
||||
id: unknown-cache-control-field
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
unexpected: true
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
12
internal/promptdef/testdata/unsupported_cache_control_ttl.yaml
vendored
Normal file
12
internal/promptdef/testdata/unsupported_cache_control_ttl.yaml
vendored
Normal file
@@ -0,0 +1,12 @@
|
||||
id: unsupported-cache-control-ttl
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
ttl: 5m
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
11
internal/promptdef/testdata/unsupported_cache_control_type.yaml
vendored
Normal file
11
internal/promptdef/testdata/unsupported_cache_control_type.yaml
vendored
Normal file
@@ -0,0 +1,11 @@
|
||||
id: unsupported-cache-control-type
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control:
|
||||
type: persistent
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
14
internal/promptdef/testdata/valid_cache_control_ttl.yaml
vendored
Normal file
14
internal/promptdef/testdata/valid_cache_control_ttl.yaml
vendored
Normal file
@@ -0,0 +1,14 @@
|
||||
id: valid-cache-control-ttl
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
ttl: 1h
|
||||
- role: user
|
||||
content: "Summarize the input."
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
13
internal/promptdef/testdata/valid_cache_control_without_ttl.yaml
vendored
Normal file
13
internal/promptdef/testdata/valid_cache_control_without_ttl.yaml
vendored
Normal file
@@ -0,0 +1,13 @@
|
||||
id: valid-cache-control-without-ttl
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content: "Summarize the input."
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
@@ -428,6 +428,14 @@ func hashRenderedPrompt(p domain.RenderedPrompt) string {
|
||||
b.WriteString(msg.Role)
|
||||
b.WriteByte('\n')
|
||||
b.WriteString(msg.Content)
|
||||
if msg.CacheControl != nil {
|
||||
b.WriteString("\ncache_control.type=")
|
||||
b.WriteString(string(msg.CacheControl.Type))
|
||||
if msg.CacheControl.TTL != "" {
|
||||
b.WriteString("\ncache_control.ttl=")
|
||||
b.WriteString(msg.CacheControl.TTL)
|
||||
}
|
||||
}
|
||||
b.WriteString("\n---\n")
|
||||
}
|
||||
h := sha256.Sum256([]byte(b.String()))
|
||||
|
||||
@@ -577,6 +577,61 @@ func TestDeriveStructuredSchemaName(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestHashRenderedPromptIncludesCacheControlWhenPresent(t *testing.T) {
|
||||
uncached := domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
||||
{Role: "system", Content: "sys"},
|
||||
{Role: "user", Content: "usr"},
|
||||
}}
|
||||
wantLegacyHash := hashString("system\nsys\n---\nuser\nusr\n---\n")
|
||||
if got := hashRenderedPrompt(uncached); got != wantLegacyHash {
|
||||
t.Fatalf("expected no-cache hash to preserve legacy input, got %q want %q", got, wantLegacyHash)
|
||||
}
|
||||
|
||||
withCache := domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "sys",
|
||||
CacheControl: &domain.CacheControl{
|
||||
Type: domain.CacheControlEphemeral,
|
||||
TTL: "1h",
|
||||
},
|
||||
},
|
||||
{Role: "user", Content: "usr"},
|
||||
}}
|
||||
alsoWithCache := domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "sys",
|
||||
CacheControl: &domain.CacheControl{
|
||||
Type: domain.CacheControlEphemeral,
|
||||
TTL: "1h",
|
||||
},
|
||||
},
|
||||
{Role: "user", Content: "usr"},
|
||||
}}
|
||||
withoutTTL := domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "sys",
|
||||
CacheControl: &domain.CacheControl{
|
||||
Type: domain.CacheControlEphemeral,
|
||||
},
|
||||
},
|
||||
{Role: "user", Content: "usr"},
|
||||
}}
|
||||
|
||||
cachedHash := hashRenderedPrompt(withCache)
|
||||
if cachedHash == hashRenderedPrompt(uncached) {
|
||||
t.Fatal("expected cache control to change rendered prompt hash")
|
||||
}
|
||||
if cachedHash != hashRenderedPrompt(alsoWithCache) {
|
||||
t.Fatal("expected identical cache control metadata to produce stable hash")
|
||||
}
|
||||
if cachedHash == hashRenderedPrompt(withoutTTL) {
|
||||
t.Fatal("expected ttl changes to affect rendered prompt hash")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunnerRunSuccessful(t *testing.T) {
|
||||
promptRepo := &fakePromptRepo{def: promptDef(domain.FormatMarkdown, domain.ValidationBasic, 0)}
|
||||
execRepo := &fakeExecutionProfileRepo{profiles: map[string]*domain.ExecutionProfile{"exec": defaultExecutionProfile()}}
|
||||
|
||||
Reference in New Issue
Block a user