Document per-run session and reasoning overrides
This commit is contained in:
@@ -97,6 +97,33 @@ For programmatic profiles,
|
|||||||
[`OpenAICompatibleProfile`](../../profiles.go) converts ordinary
|
[`OpenAICompatibleProfile`](../../profiles.go) converts ordinary
|
||||||
OpenAI-compatible settings into a value accepted by `WithProfiles`.
|
OpenAI-compatible settings into a value accepted by `WithProfiles`.
|
||||||
|
|
||||||
|
### Set A Per-Run Session And Reasoning
|
||||||
|
|
||||||
|
Supply a direct session ID when one prompt should be correlated with a
|
||||||
|
consumer-managed conversation or workflow without changing prompt variables:
|
||||||
|
|
||||||
|
```go
|
||||||
|
reasoning := "high"
|
||||||
|
result, err := engine.Run(ctx, promptkit.RunRequest{
|
||||||
|
PromptID: "meeting.summary",
|
||||||
|
SessionID: "conversation-42",
|
||||||
|
Inputs: map[string]promptkit.ArtifactRef{
|
||||||
|
"note": promptkit.Inline("Synthetic meeting notes"),
|
||||||
|
},
|
||||||
|
Execution: &promptkit.ExecutionTargetOverride{
|
||||||
|
ReasoningEffort: &reasoning,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
A nil reasoning pointer inherits the selected profile, a pointer to a
|
||||||
|
nonblank string replaces it, and a pointer to a blank string disables
|
||||||
|
reasoning for that run. Session IDs are correlation metadata, not credentials;
|
||||||
|
use stable, non-secret values that are safe to expose to collaborators and
|
||||||
|
providers. The
|
||||||
|
[`RunRequest` and `ExecutionTargetOverride` GoDoc](../../types.go) owns the
|
||||||
|
exact normalization, precedence, error, copying, and exposure contract.
|
||||||
|
|
||||||
### Register A Custom Backend
|
### Register A Custom Backend
|
||||||
|
|
||||||
Register a reusable OpenAI-compatible connection once, then select it from a
|
Register a reusable OpenAI-compatible connection once, then select it from a
|
||||||
|
|||||||
@@ -91,7 +91,8 @@ of a named input. Missing variables and input references are errors.
|
|||||||
|
|
||||||
The optional `session_id` uses the same template data and input helper. Its
|
The optional `session_id` uses the same template data and input helper. Its
|
||||||
rendered value is trimmed, omitted when empty, and limited to 256 Unicode code
|
rendered value is trimmed, omitted when empty, and limited to 256 Unicode code
|
||||||
points.
|
points. A nonblank direct request session ID bypasses this template completely;
|
||||||
|
a blank direct value leaves the template behavior unchanged.
|
||||||
|
|
||||||
### Cache Control
|
### Cache Control
|
||||||
|
|
||||||
@@ -203,9 +204,12 @@ explicit zero is preserved. In particular, an explicit request
|
|||||||
the caller context and transport timeout intact.
|
the caller context and transport timeout intact.
|
||||||
|
|
||||||
Non-empty profile strings replace backend defaults, and non-empty request
|
Non-empty profile strings replace backend defaults, and non-empty request
|
||||||
strings replace both. Backend identity is retained when either layer overrides
|
strings replace both. Request reasoning is the exception: a nil
|
||||||
the endpoint. A non-empty `extra_params` map at each layer replaces the entire
|
`ReasoningEffort` pointer inherits the profile, a pointer to a nonblank string
|
||||||
lower-precedence map rather than merging keys.
|
trims and replaces it, and a pointer to a blank string clears it. Backend
|
||||||
|
identity is retained when either layer overrides the endpoint. A non-empty
|
||||||
|
`extra_params` map at each layer replaces the entire lower-precedence map
|
||||||
|
rather than merging keys.
|
||||||
The [outbound integration contract](integrations/openai-compatible-chat.md)
|
The [outbound integration contract](integrations/openai-compatible-chat.md)
|
||||||
defines how the effective settings are serialized.
|
defines how the effective settings are serialized.
|
||||||
|
|
||||||
|
|||||||
@@ -47,14 +47,16 @@ Each ordinary message contains its `role` and string `content`. A
|
|||||||
cache-controlled message instead uses a text content block containing `type`,
|
cache-controlled message instead uses a text content block containing `type`,
|
||||||
`text`, and `cache_control`; an empty cache-control TTL is omitted.
|
`text`, and `cache_control`; an empty cache-control TTL is omitted.
|
||||||
|
|
||||||
A non-empty session ID is trimmed, checked against the internal domain limit,
|
The effective direct or prompt-rendered session ID is trimmed, limited to 256
|
||||||
and sent as top-level `session_id`. It is not sent as a session header.
|
Unicode code points, and sent when nonempty as top-level `session_id`. It is
|
||||||
|
never also sent as a session header.
|
||||||
|
|
||||||
The client conditionally includes:
|
The client conditionally includes:
|
||||||
|
|
||||||
- `temperature`, `max_tokens`, and `top_p` when non-zero or explicitly
|
- `temperature`, `max_tokens`, and `top_p` when non-zero or explicitly
|
||||||
present;
|
present;
|
||||||
- non-empty `service_tier` and `reasoning_effort`; and
|
- non-empty `service_tier` and effective `reasoning_effort`; an explicitly
|
||||||
|
disabled reasoning setting is empty and therefore omitted; and
|
||||||
- `response_format` for JSON Schema structured output, including its name,
|
- `response_format` for JSON Schema structured output, including its name,
|
||||||
strict flag, and schema document.
|
strict flag, and schema document.
|
||||||
|
|
||||||
|
|||||||
@@ -31,8 +31,8 @@ The runner has no durable run or session store.
|
|||||||
|
|
||||||
`Prepare` performs the reusable pre-generation workflow:
|
`Prepare` performs the reusable pre-generation workflow:
|
||||||
|
|
||||||
1. validate the prompt selection and load the prompt definition;
|
1. validate the required prompt selection and normalize any direct session ID;
|
||||||
2. hash the loaded definition;
|
2. load the prompt definition and hash the original definition;
|
||||||
3. select the request profile or the prompt's default profile;
|
3. select the request profile or the prompt's default profile;
|
||||||
4. resolve the profile's backend ID, when present;
|
4. resolve the profile's backend ID, when present;
|
||||||
5. resolve application-neutral defaults, backend defaults, profile values,
|
5. resolve application-neutral defaults, backend defaults, profile values,
|
||||||
@@ -41,7 +41,8 @@ The runner has no durable run or session store.
|
|||||||
7. resolve the output contract and load a structured-output schema when
|
7. resolve the output contract and load a structured-output schema when
|
||||||
required;
|
required;
|
||||||
8. load and hash input artifacts;
|
8. load and hash input artifacts;
|
||||||
9. render and hash the prompt; and
|
9. render messages, resolve the effective session ID, and hash the effective
|
||||||
|
rendered prompt; and
|
||||||
10. return the effective settings, source identities, messages, hashes, and
|
10. return the effective settings, source identities, messages, hashes, and
|
||||||
preparation timing.
|
preparation timing.
|
||||||
|
|
||||||
@@ -54,6 +55,15 @@ apply in that order. A profile requiring a direct key clears an inherited
|
|||||||
backend environment name unless the request supplies its own name. Secret
|
backend environment name unless the request supplies its own name. Secret
|
||||||
values remain excluded from serialized metadata.
|
values remain excluded from serialized metadata.
|
||||||
|
|
||||||
|
Reasoning overrides are tri-state: nil inherits the profile, a pointer to a
|
||||||
|
nonblank string trims and replaces it, and a pointer to a blank string clears
|
||||||
|
it. A nonblank direct session is normalized before source loading, bypasses
|
||||||
|
the prompt session template, and is applied after ordinary message rendering.
|
||||||
|
A blank direct value retains prompt-template behavior. The runner clears the
|
||||||
|
template only on a value copy of the definition, so the definition hash always
|
||||||
|
describes the original source while the rendered-prompt hash includes the
|
||||||
|
effective direct or rendered session.
|
||||||
|
|
||||||
The registry is read-only after engine construction. Concurrent `Prepare` and
|
The registry is read-only after engine construction. Concurrent `Prepare` and
|
||||||
`Run` calls resolve independent defensive backend values and keep all
|
`Run` calls resolve independent defensive backend values and keep all
|
||||||
invocation state local.
|
invocation state local.
|
||||||
@@ -71,10 +81,11 @@ target, validation errors, prior output, and structured-output specification.
|
|||||||
This capability remains internal and is not a public option.
|
This capability remains internal and is not a public option.
|
||||||
|
|
||||||
A successful result includes the output artifact and raw output, validation
|
A successful result includes the output artifact and raw output, validation
|
||||||
state, prompt and rendered-prompt hashes, selected profile and backend,
|
state, effective session ID, prompt and rendered-prompt hashes, selected
|
||||||
effective settings, input hashes, token usage, a generated run identifier, and
|
profile and backend, effective settings, input hashes, token usage, a generated
|
||||||
UTC timing. The same effective target, including backend identity, reaches
|
run identifier, and UTC timing. The same effective session reaches generation
|
||||||
generation and any repair attempt.
|
through the rendered prompt. The same effective target, including backend
|
||||||
|
identity, reaches generation and any repair attempt.
|
||||||
|
|
||||||
## Failure Categories
|
## Failure Categories
|
||||||
|
|
||||||
@@ -84,16 +95,18 @@ validation failures. Wrapping preserves the package identities mapped by the
|
|||||||
public facade and retains collaborator identities where they are part of the
|
public facade and retains collaborator identities where they are part of the
|
||||||
internal contract. Context cancellation propagates through the invoked
|
internal contract. Context cancellation propagates through the invoked
|
||||||
collaborator and is classified by the owning operation.
|
collaborator and is classified by the owning operation.
|
||||||
|
An overlong direct session is an invalid request before source loading, while
|
||||||
|
an invalid or overlong prompt session template remains a prompt-render failure.
|
||||||
An unknown selected backend, or a selected backend with no configured resolver,
|
An unknown selected backend, or a selected backend with no configured resolver,
|
||||||
is classified as a profile-load failure.
|
is classified as a profile-load failure.
|
||||||
|
|
||||||
## Test Ownership And Changes
|
## Test Ownership And Changes
|
||||||
|
|
||||||
The [runner tests](../../internal/usecase/runner_test.go) own preparation order,
|
The [runner tests](../../internal/usecase/runner_test.go) own preparation order,
|
||||||
selection and override precedence, schema-before-generation behavior, hashing,
|
selection and override precedence, direct-session resolution,
|
||||||
generation and validation outcomes, backend propagation, bounded repair,
|
schema-before-generation behavior, hashing, generation and validation
|
||||||
credentials and redaction, error categories, artifact metadata, usage, and
|
outcomes, backend propagation, bounded repair, credentials and redaction,
|
||||||
timing.
|
error categories, artifact metadata, usage, and timing.
|
||||||
|
|
||||||
Changes to orchestration should continue to use the existing package
|
Changes to orchestration should continue to use the existing package
|
||||||
interfaces, keep request state local to an invocation, and preserve `Run`'s use
|
interfaces, keep request state local to an invocation, and preserve `Run`'s use
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Explicit Per-Run Profile Overrides Implementation Plan
|
# Explicit Per-Run Profile Overrides Implementation Plan
|
||||||
|
|
||||||
**Status:** Ready for implementation.
|
**Status:** Complete.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
@@ -456,7 +456,7 @@ existing body-only transport contract remains intact.
|
|||||||
|
|
||||||
## Stage 3 — Durable Documentation And Final Validation
|
## Stage 3 — Durable Documentation And Final Validation
|
||||||
|
|
||||||
**Status:** Pending.
|
**Status:** Complete.
|
||||||
|
|
||||||
### Goal
|
### Goal
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Explicit Per-Run Profile Overrides
|
# Explicit Per-Run Profile Overrides
|
||||||
|
|
||||||
**Status:** Selected.
|
**Status:** Complete.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
|
|||||||
9
types.go
9
types.go
@@ -98,7 +98,8 @@ type RunRequest struct {
|
|||||||
// opaque consumer metadata, not a credential, and may be exposed in
|
// opaque consumer metadata, not a credential, and may be exposed in
|
||||||
// prepared values, results, collaborator requests, provider requests, and
|
// prepared values, results, collaborator requests, provider requests, and
|
||||||
// provider observability. Callers should use stable, non-sensitive
|
// provider observability. Callers should use stable, non-sensitive
|
||||||
// identifiers.
|
// identifiers. An overlong direct value makes Prepare or Run return an
|
||||||
|
// error matching ErrInvalidRequest.
|
||||||
SessionID string
|
SessionID string
|
||||||
// APIKey is a request-scoped direct credential. It takes precedence over
|
// APIKey is a request-scoped direct credential. It takes precedence over
|
||||||
// APIKeyEnv, is passed to the selected LLMClient, and is never included in
|
// APIKeyEnv, is passed to the selected LLMClient, and is never included in
|
||||||
@@ -300,7 +301,8 @@ type ExecutionTarget struct {
|
|||||||
TimeoutSeconds int `json:"timeout_seconds"`
|
TimeoutSeconds int `json:"timeout_seconds"`
|
||||||
// ServiceTier is an optional provider-specific request tier.
|
// ServiceTier is an optional provider-specific request tier.
|
||||||
ServiceTier string `json:"service_tier"`
|
ServiceTier string `json:"service_tier"`
|
||||||
// ReasoningEffort is an optional provider-specific reasoning setting.
|
// ReasoningEffort is the effective opaque provider-specific reasoning
|
||||||
|
// setting. An empty value instructs model clients to omit reasoning.
|
||||||
ReasoningEffort string `json:"reasoning_effort"`
|
ReasoningEffort string `json:"reasoning_effort"`
|
||||||
// APIKeyEnv is an environment-variable name, not its credential value.
|
// APIKeyEnv is an environment-variable name, not its credential value.
|
||||||
APIKeyEnv string `json:"api_key_env"`
|
APIKeyEnv string `json:"api_key_env"`
|
||||||
@@ -337,7 +339,8 @@ type ExecutionTargetOverride struct {
|
|||||||
// ReasoningEffort controls the per-run reasoning setting. Nil inherits the
|
// ReasoningEffort controls the per-run reasoning setting. Nil inherits the
|
||||||
// profile value. A pointer to a non-blank string trims and replaces the
|
// profile value. A pointer to a non-blank string trims and replaces the
|
||||||
// profile value. A pointer to an empty or whitespace-only string clears the
|
// profile value. A pointer to an empty or whitespace-only string clears the
|
||||||
// inherited value and disables reasoning for this run.
|
// inherited value and disables reasoning for this run. Non-blank values
|
||||||
|
// are opaque and are not validated against a fixed vocabulary.
|
||||||
ReasoningEffort *string
|
ReasoningEffort *string
|
||||||
// APIKeyEnv replaces the profile or backend environment-variable name when
|
// APIKeyEnv replaces the profile or backend environment-variable name when
|
||||||
// non-blank. A direct RunRequest.APIKey still takes precedence over
|
// non-blank. A direct RunRequest.APIKey still takes precedence over
|
||||||
|
|||||||
Reference in New Issue
Block a user