Document per-run session and reasoning overrides

This commit is contained in:
2026-07-29 19:47:39 +00:00
parent f6ee18f6b3
commit 0a839aa16d
7 changed files with 73 additions and 24 deletions

View File

@@ -97,6 +97,33 @@ For programmatic profiles,
[`OpenAICompatibleProfile`](../../profiles.go) converts ordinary [`OpenAICompatibleProfile`](../../profiles.go) converts ordinary
OpenAI-compatible settings into a value accepted by `WithProfiles`. OpenAI-compatible settings into a value accepted by `WithProfiles`.
### Set A Per-Run Session And Reasoning
Supply a direct session ID when one prompt should be correlated with a
consumer-managed conversation or workflow without changing prompt variables:
```go
reasoning := "high"
result, err := engine.Run(ctx, promptkit.RunRequest{
PromptID: "meeting.summary",
SessionID: "conversation-42",
Inputs: map[string]promptkit.ArtifactRef{
"note": promptkit.Inline("Synthetic meeting notes"),
},
Execution: &promptkit.ExecutionTargetOverride{
ReasoningEffort: &reasoning,
},
})
```
A nil reasoning pointer inherits the selected profile, a pointer to a
nonblank string replaces it, and a pointer to a blank string disables
reasoning for that run. Session IDs are correlation metadata, not credentials;
use stable, non-secret values that are safe to expose to collaborators and
providers. The
[`RunRequest` and `ExecutionTargetOverride` GoDoc](../../types.go) owns the
exact normalization, precedence, error, copying, and exposure contract.
### Register A Custom Backend ### Register A Custom Backend
Register a reusable OpenAI-compatible connection once, then select it from a Register a reusable OpenAI-compatible connection once, then select it from a

View File

@@ -91,7 +91,8 @@ of a named input. Missing variables and input references are errors.
The optional `session_id` uses the same template data and input helper. Its The optional `session_id` uses the same template data and input helper. Its
rendered value is trimmed, omitted when empty, and limited to 256 Unicode code rendered value is trimmed, omitted when empty, and limited to 256 Unicode code
points. points. A nonblank direct request session ID bypasses this template completely;
a blank direct value leaves the template behavior unchanged.
### Cache Control ### Cache Control
@@ -203,9 +204,12 @@ explicit zero is preserved. In particular, an explicit request
the caller context and transport timeout intact. the caller context and transport timeout intact.
Non-empty profile strings replace backend defaults, and non-empty request Non-empty profile strings replace backend defaults, and non-empty request
strings replace both. Backend identity is retained when either layer overrides strings replace both. Request reasoning is the exception: a nil
the endpoint. A non-empty `extra_params` map at each layer replaces the entire `ReasoningEffort` pointer inherits the profile, a pointer to a nonblank string
lower-precedence map rather than merging keys. trims and replaces it, and a pointer to a blank string clears it. Backend
identity is retained when either layer overrides the endpoint. A non-empty
`extra_params` map at each layer replaces the entire lower-precedence map
rather than merging keys.
The [outbound integration contract](integrations/openai-compatible-chat.md) The [outbound integration contract](integrations/openai-compatible-chat.md)
defines how the effective settings are serialized. defines how the effective settings are serialized.

View File

@@ -47,14 +47,16 @@ Each ordinary message contains its `role` and string `content`. A
cache-controlled message instead uses a text content block containing `type`, cache-controlled message instead uses a text content block containing `type`,
`text`, and `cache_control`; an empty cache-control TTL is omitted. `text`, and `cache_control`; an empty cache-control TTL is omitted.
A non-empty session ID is trimmed, checked against the internal domain limit, The effective direct or prompt-rendered session ID is trimmed, limited to 256
and sent as top-level `session_id`. It is not sent as a session header. Unicode code points, and sent when nonempty as top-level `session_id`. It is
never also sent as a session header.
The client conditionally includes: The client conditionally includes:
- `temperature`, `max_tokens`, and `top_p` when non-zero or explicitly - `temperature`, `max_tokens`, and `top_p` when non-zero or explicitly
present; present;
- non-empty `service_tier` and `reasoning_effort`; and - non-empty `service_tier` and effective `reasoning_effort`; an explicitly
disabled reasoning setting is empty and therefore omitted; and
- `response_format` for JSON Schema structured output, including its name, - `response_format` for JSON Schema structured output, including its name,
strict flag, and schema document. strict flag, and schema document.

View File

@@ -31,8 +31,8 @@ The runner has no durable run or session store.
`Prepare` performs the reusable pre-generation workflow: `Prepare` performs the reusable pre-generation workflow:
1. validate the prompt selection and load the prompt definition; 1. validate the required prompt selection and normalize any direct session ID;
2. hash the loaded definition; 2. load the prompt definition and hash the original definition;
3. select the request profile or the prompt's default profile; 3. select the request profile or the prompt's default profile;
4. resolve the profile's backend ID, when present; 4. resolve the profile's backend ID, when present;
5. resolve application-neutral defaults, backend defaults, profile values, 5. resolve application-neutral defaults, backend defaults, profile values,
@@ -41,7 +41,8 @@ The runner has no durable run or session store.
7. resolve the output contract and load a structured-output schema when 7. resolve the output contract and load a structured-output schema when
required; required;
8. load and hash input artifacts; 8. load and hash input artifacts;
9. render and hash the prompt; and 9. render messages, resolve the effective session ID, and hash the effective
rendered prompt; and
10. return the effective settings, source identities, messages, hashes, and 10. return the effective settings, source identities, messages, hashes, and
preparation timing. preparation timing.
@@ -54,6 +55,15 @@ apply in that order. A profile requiring a direct key clears an inherited
backend environment name unless the request supplies its own name. Secret backend environment name unless the request supplies its own name. Secret
values remain excluded from serialized metadata. values remain excluded from serialized metadata.
Reasoning overrides are tri-state: nil inherits the profile, a pointer to a
nonblank string trims and replaces it, and a pointer to a blank string clears
it. A nonblank direct session is normalized before source loading, bypasses
the prompt session template, and is applied after ordinary message rendering.
A blank direct value retains prompt-template behavior. The runner clears the
template only on a value copy of the definition, so the definition hash always
describes the original source while the rendered-prompt hash includes the
effective direct or rendered session.
The registry is read-only after engine construction. Concurrent `Prepare` and The registry is read-only after engine construction. Concurrent `Prepare` and
`Run` calls resolve independent defensive backend values and keep all `Run` calls resolve independent defensive backend values and keep all
invocation state local. invocation state local.
@@ -71,10 +81,11 @@ target, validation errors, prior output, and structured-output specification.
This capability remains internal and is not a public option. This capability remains internal and is not a public option.
A successful result includes the output artifact and raw output, validation A successful result includes the output artifact and raw output, validation
state, prompt and rendered-prompt hashes, selected profile and backend, state, effective session ID, prompt and rendered-prompt hashes, selected
effective settings, input hashes, token usage, a generated run identifier, and profile and backend, effective settings, input hashes, token usage, a generated
UTC timing. The same effective target, including backend identity, reaches run identifier, and UTC timing. The same effective session reaches generation
generation and any repair attempt. through the rendered prompt. The same effective target, including backend
identity, reaches generation and any repair attempt.
## Failure Categories ## Failure Categories
@@ -84,16 +95,18 @@ validation failures. Wrapping preserves the package identities mapped by the
public facade and retains collaborator identities where they are part of the public facade and retains collaborator identities where they are part of the
internal contract. Context cancellation propagates through the invoked internal contract. Context cancellation propagates through the invoked
collaborator and is classified by the owning operation. collaborator and is classified by the owning operation.
An overlong direct session is an invalid request before source loading, while
an invalid or overlong prompt session template remains a prompt-render failure.
An unknown selected backend, or a selected backend with no configured resolver, An unknown selected backend, or a selected backend with no configured resolver,
is classified as a profile-load failure. is classified as a profile-load failure.
## Test Ownership And Changes ## Test Ownership And Changes
The [runner tests](../../internal/usecase/runner_test.go) own preparation order, The [runner tests](../../internal/usecase/runner_test.go) own preparation order,
selection and override precedence, schema-before-generation behavior, hashing, selection and override precedence, direct-session resolution,
generation and validation outcomes, backend propagation, bounded repair, schema-before-generation behavior, hashing, generation and validation
credentials and redaction, error categories, artifact metadata, usage, and outcomes, backend propagation, bounded repair, credentials and redaction,
timing. error categories, artifact metadata, usage, and timing.
Changes to orchestration should continue to use the existing package Changes to orchestration should continue to use the existing package
interfaces, keep request state local to an invocation, and preserve `Run`'s use interfaces, keep request state local to an invocation, and preserve `Run`'s use

View File

@@ -1,6 +1,6 @@
# Explicit Per-Run Profile Overrides Implementation Plan # Explicit Per-Run Profile Overrides Implementation Plan
**Status:** Ready for implementation. **Status:** Complete.
## Purpose ## Purpose
@@ -456,7 +456,7 @@ existing body-only transport contract remains intact.
## Stage 3 — Durable Documentation And Final Validation ## Stage 3 — Durable Documentation And Final Validation
**Status:** Pending. **Status:** Complete.
### Goal ### Goal

View File

@@ -1,6 +1,6 @@
# Explicit Per-Run Profile Overrides # Explicit Per-Run Profile Overrides
**Status:** Selected. **Status:** Complete.
## Purpose ## Purpose

View File

@@ -98,7 +98,8 @@ type RunRequest struct {
// opaque consumer metadata, not a credential, and may be exposed in // opaque consumer metadata, not a credential, and may be exposed in
// prepared values, results, collaborator requests, provider requests, and // prepared values, results, collaborator requests, provider requests, and
// provider observability. Callers should use stable, non-sensitive // provider observability. Callers should use stable, non-sensitive
// identifiers. // identifiers. An overlong direct value makes Prepare or Run return an
// error matching ErrInvalidRequest.
SessionID string SessionID string
// APIKey is a request-scoped direct credential. It takes precedence over // APIKey is a request-scoped direct credential. It takes precedence over
// APIKeyEnv, is passed to the selected LLMClient, and is never included in // APIKeyEnv, is passed to the selected LLMClient, and is never included in
@@ -300,7 +301,8 @@ type ExecutionTarget struct {
TimeoutSeconds int `json:"timeout_seconds"` TimeoutSeconds int `json:"timeout_seconds"`
// ServiceTier is an optional provider-specific request tier. // ServiceTier is an optional provider-specific request tier.
ServiceTier string `json:"service_tier"` ServiceTier string `json:"service_tier"`
// ReasoningEffort is an optional provider-specific reasoning setting. // ReasoningEffort is the effective opaque provider-specific reasoning
// setting. An empty value instructs model clients to omit reasoning.
ReasoningEffort string `json:"reasoning_effort"` ReasoningEffort string `json:"reasoning_effort"`
// APIKeyEnv is an environment-variable name, not its credential value. // APIKeyEnv is an environment-variable name, not its credential value.
APIKeyEnv string `json:"api_key_env"` APIKeyEnv string `json:"api_key_env"`
@@ -337,7 +339,8 @@ type ExecutionTargetOverride struct {
// ReasoningEffort controls the per-run reasoning setting. Nil inherits the // ReasoningEffort controls the per-run reasoning setting. Nil inherits the
// profile value. A pointer to a non-blank string trims and replaces the // profile value. A pointer to a non-blank string trims and replaces the
// profile value. A pointer to an empty or whitespace-only string clears the // profile value. A pointer to an empty or whitespace-only string clears the
// inherited value and disables reasoning for this run. // inherited value and disables reasoning for this run. Non-blank values
// are opaque and are not validated against a fixed vocabulary.
ReasoningEffort *string ReasoningEffort *string
// APIKeyEnv replaces the profile or backend environment-variable name when // APIKeyEnv replaces the profile or backend environment-variable name when
// non-blank. A direct RunRequest.APIKey still takes precedence over // non-blank. A direct RunRequest.APIKey still takes precedence over