Document per-run session and reasoning overrides
This commit is contained in:
@@ -97,6 +97,33 @@ For programmatic profiles,
|
||||
[`OpenAICompatibleProfile`](../../profiles.go) converts ordinary
|
||||
OpenAI-compatible settings into a value accepted by `WithProfiles`.
|
||||
|
||||
### Set A Per-Run Session And Reasoning
|
||||
|
||||
Supply a direct session ID when one prompt should be correlated with a
|
||||
consumer-managed conversation or workflow without changing prompt variables:
|
||||
|
||||
```go
|
||||
reasoning := "high"
|
||||
result, err := engine.Run(ctx, promptkit.RunRequest{
|
||||
PromptID: "meeting.summary",
|
||||
SessionID: "conversation-42",
|
||||
Inputs: map[string]promptkit.ArtifactRef{
|
||||
"note": promptkit.Inline("Synthetic meeting notes"),
|
||||
},
|
||||
Execution: &promptkit.ExecutionTargetOverride{
|
||||
ReasoningEffort: &reasoning,
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
A nil reasoning pointer inherits the selected profile, a pointer to a
|
||||
nonblank string replaces it, and a pointer to a blank string disables
|
||||
reasoning for that run. Session IDs are correlation metadata, not credentials;
|
||||
use stable, non-secret values that are safe to expose to collaborators and
|
||||
providers. The
|
||||
[`RunRequest` and `ExecutionTargetOverride` GoDoc](../../types.go) owns the
|
||||
exact normalization, precedence, error, copying, and exposure contract.
|
||||
|
||||
### Register A Custom Backend
|
||||
|
||||
Register a reusable OpenAI-compatible connection once, then select it from a
|
||||
|
||||
@@ -91,7 +91,8 @@ of a named input. Missing variables and input references are errors.
|
||||
|
||||
The optional `session_id` uses the same template data and input helper. Its
|
||||
rendered value is trimmed, omitted when empty, and limited to 256 Unicode code
|
||||
points.
|
||||
points. A nonblank direct request session ID bypasses this template completely;
|
||||
a blank direct value leaves the template behavior unchanged.
|
||||
|
||||
### Cache Control
|
||||
|
||||
@@ -203,9 +204,12 @@ explicit zero is preserved. In particular, an explicit request
|
||||
the caller context and transport timeout intact.
|
||||
|
||||
Non-empty profile strings replace backend defaults, and non-empty request
|
||||
strings replace both. Backend identity is retained when either layer overrides
|
||||
the endpoint. A non-empty `extra_params` map at each layer replaces the entire
|
||||
lower-precedence map rather than merging keys.
|
||||
strings replace both. Request reasoning is the exception: a nil
|
||||
`ReasoningEffort` pointer inherits the profile, a pointer to a nonblank string
|
||||
trims and replaces it, and a pointer to a blank string clears it. Backend
|
||||
identity is retained when either layer overrides the endpoint. A non-empty
|
||||
`extra_params` map at each layer replaces the entire lower-precedence map
|
||||
rather than merging keys.
|
||||
The [outbound integration contract](integrations/openai-compatible-chat.md)
|
||||
defines how the effective settings are serialized.
|
||||
|
||||
|
||||
@@ -47,14 +47,16 @@ Each ordinary message contains its `role` and string `content`. A
|
||||
cache-controlled message instead uses a text content block containing `type`,
|
||||
`text`, and `cache_control`; an empty cache-control TTL is omitted.
|
||||
|
||||
A non-empty session ID is trimmed, checked against the internal domain limit,
|
||||
and sent as top-level `session_id`. It is not sent as a session header.
|
||||
The effective direct or prompt-rendered session ID is trimmed, limited to 256
|
||||
Unicode code points, and sent when nonempty as top-level `session_id`. It is
|
||||
never also sent as a session header.
|
||||
|
||||
The client conditionally includes:
|
||||
|
||||
- `temperature`, `max_tokens`, and `top_p` when non-zero or explicitly
|
||||
present;
|
||||
- non-empty `service_tier` and `reasoning_effort`; and
|
||||
- non-empty `service_tier` and effective `reasoning_effort`; an explicitly
|
||||
disabled reasoning setting is empty and therefore omitted; and
|
||||
- `response_format` for JSON Schema structured output, including its name,
|
||||
strict flag, and schema document.
|
||||
|
||||
|
||||
@@ -31,8 +31,8 @@ The runner has no durable run or session store.
|
||||
|
||||
`Prepare` performs the reusable pre-generation workflow:
|
||||
|
||||
1. validate the prompt selection and load the prompt definition;
|
||||
2. hash the loaded definition;
|
||||
1. validate the required prompt selection and normalize any direct session ID;
|
||||
2. load the prompt definition and hash the original definition;
|
||||
3. select the request profile or the prompt's default profile;
|
||||
4. resolve the profile's backend ID, when present;
|
||||
5. resolve application-neutral defaults, backend defaults, profile values,
|
||||
@@ -41,7 +41,8 @@ The runner has no durable run or session store.
|
||||
7. resolve the output contract and load a structured-output schema when
|
||||
required;
|
||||
8. load and hash input artifacts;
|
||||
9. render and hash the prompt; and
|
||||
9. render messages, resolve the effective session ID, and hash the effective
|
||||
rendered prompt; and
|
||||
10. return the effective settings, source identities, messages, hashes, and
|
||||
preparation timing.
|
||||
|
||||
@@ -54,6 +55,15 @@ apply in that order. A profile requiring a direct key clears an inherited
|
||||
backend environment name unless the request supplies its own name. Secret
|
||||
values remain excluded from serialized metadata.
|
||||
|
||||
Reasoning overrides are tri-state: nil inherits the profile, a pointer to a
|
||||
nonblank string trims and replaces it, and a pointer to a blank string clears
|
||||
it. A nonblank direct session is normalized before source loading, bypasses
|
||||
the prompt session template, and is applied after ordinary message rendering.
|
||||
A blank direct value retains prompt-template behavior. The runner clears the
|
||||
template only on a value copy of the definition, so the definition hash always
|
||||
describes the original source while the rendered-prompt hash includes the
|
||||
effective direct or rendered session.
|
||||
|
||||
The registry is read-only after engine construction. Concurrent `Prepare` and
|
||||
`Run` calls resolve independent defensive backend values and keep all
|
||||
invocation state local.
|
||||
@@ -71,10 +81,11 @@ target, validation errors, prior output, and structured-output specification.
|
||||
This capability remains internal and is not a public option.
|
||||
|
||||
A successful result includes the output artifact and raw output, validation
|
||||
state, prompt and rendered-prompt hashes, selected profile and backend,
|
||||
effective settings, input hashes, token usage, a generated run identifier, and
|
||||
UTC timing. The same effective target, including backend identity, reaches
|
||||
generation and any repair attempt.
|
||||
state, effective session ID, prompt and rendered-prompt hashes, selected
|
||||
profile and backend, effective settings, input hashes, token usage, a generated
|
||||
run identifier, and UTC timing. The same effective session reaches generation
|
||||
through the rendered prompt. The same effective target, including backend
|
||||
identity, reaches generation and any repair attempt.
|
||||
|
||||
## Failure Categories
|
||||
|
||||
@@ -84,16 +95,18 @@ validation failures. Wrapping preserves the package identities mapped by the
|
||||
public facade and retains collaborator identities where they are part of the
|
||||
internal contract. Context cancellation propagates through the invoked
|
||||
collaborator and is classified by the owning operation.
|
||||
An overlong direct session is an invalid request before source loading, while
|
||||
an invalid or overlong prompt session template remains a prompt-render failure.
|
||||
An unknown selected backend, or a selected backend with no configured resolver,
|
||||
is classified as a profile-load failure.
|
||||
|
||||
## Test Ownership And Changes
|
||||
|
||||
The [runner tests](../../internal/usecase/runner_test.go) own preparation order,
|
||||
selection and override precedence, schema-before-generation behavior, hashing,
|
||||
generation and validation outcomes, backend propagation, bounded repair,
|
||||
credentials and redaction, error categories, artifact metadata, usage, and
|
||||
timing.
|
||||
selection and override precedence, direct-session resolution,
|
||||
schema-before-generation behavior, hashing, generation and validation
|
||||
outcomes, backend propagation, bounded repair, credentials and redaction,
|
||||
error categories, artifact metadata, usage, and timing.
|
||||
|
||||
Changes to orchestration should continue to use the existing package
|
||||
interfaces, keep request state local to an invocation, and preserve `Run`'s use
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Explicit Per-Run Profile Overrides Implementation Plan
|
||||
|
||||
**Status:** Ready for implementation.
|
||||
**Status:** Complete.
|
||||
|
||||
## Purpose
|
||||
|
||||
@@ -456,7 +456,7 @@ existing body-only transport contract remains intact.
|
||||
|
||||
## Stage 3 — Durable Documentation And Final Validation
|
||||
|
||||
**Status:** Pending.
|
||||
**Status:** Complete.
|
||||
|
||||
### Goal
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Explicit Per-Run Profile Overrides
|
||||
|
||||
**Status:** Selected.
|
||||
**Status:** Complete.
|
||||
|
||||
## Purpose
|
||||
|
||||
|
||||
9
types.go
9
types.go
@@ -98,7 +98,8 @@ type RunRequest struct {
|
||||
// opaque consumer metadata, not a credential, and may be exposed in
|
||||
// prepared values, results, collaborator requests, provider requests, and
|
||||
// provider observability. Callers should use stable, non-sensitive
|
||||
// identifiers.
|
||||
// identifiers. An overlong direct value makes Prepare or Run return an
|
||||
// error matching ErrInvalidRequest.
|
||||
SessionID string
|
||||
// APIKey is a request-scoped direct credential. It takes precedence over
|
||||
// APIKeyEnv, is passed to the selected LLMClient, and is never included in
|
||||
@@ -300,7 +301,8 @@ type ExecutionTarget struct {
|
||||
TimeoutSeconds int `json:"timeout_seconds"`
|
||||
// ServiceTier is an optional provider-specific request tier.
|
||||
ServiceTier string `json:"service_tier"`
|
||||
// ReasoningEffort is an optional provider-specific reasoning setting.
|
||||
// ReasoningEffort is the effective opaque provider-specific reasoning
|
||||
// setting. An empty value instructs model clients to omit reasoning.
|
||||
ReasoningEffort string `json:"reasoning_effort"`
|
||||
// APIKeyEnv is an environment-variable name, not its credential value.
|
||||
APIKeyEnv string `json:"api_key_env"`
|
||||
@@ -337,7 +339,8 @@ type ExecutionTargetOverride struct {
|
||||
// ReasoningEffort controls the per-run reasoning setting. Nil inherits the
|
||||
// profile value. A pointer to a non-blank string trims and replaces the
|
||||
// profile value. A pointer to an empty or whitespace-only string clears the
|
||||
// inherited value and disables reasoning for this run.
|
||||
// inherited value and disables reasoning for this run. Non-blank values
|
||||
// are opaque and are not validated against a fixed vocabulary.
|
||||
ReasoningEffort *string
|
||||
// APIKeyEnv replaces the profile or backend environment-variable name when
|
||||
// non-blank. A direct RunRequest.APIKey still takes precedence over
|
||||
|
||||
Reference in New Issue
Block a user