Add tri-state reasoning overrides
This commit is contained in:
176
docs/roadmap/profile-overrides.md
Normal file
176
docs/roadmap/profile-overrides.md
Normal file
@@ -0,0 +1,176 @@
|
||||
# Explicit Per-Run Profile Overrides
|
||||
|
||||
**Status:** Selected.
|
||||
|
||||
## Purpose
|
||||
|
||||
This roadmap defines the scope and target end state for explicit per-run
|
||||
session and reasoning controls. It establishes the behavioral boundary and
|
||||
policy choices for the work.
|
||||
|
||||
This document is planning material, not a description of current behavior.
|
||||
Current exported contracts remain owned by Go declarations and GoDoc, prompt
|
||||
and profile files by the [format reference](../formats.md), and outbound HTTP
|
||||
behavior by the
|
||||
[OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md).
|
||||
|
||||
## Motivation
|
||||
|
||||
Prompt definitions can currently render a session ID from request variables,
|
||||
and execution profiles can provide a default reasoning effort. A nonblank
|
||||
per-run reasoning value can replace that profile default, but consumers cannot
|
||||
directly supply a session ID or explicitly clear inherited reasoning.
|
||||
|
||||
Consumers should be able to reuse one prompt and profile across conversations,
|
||||
agent workflows, and reasoning levels. Choosing a session or reasoning level
|
||||
for one run should not require duplicate prompt definitions, duplicate
|
||||
profiles, or provider-specific values that mean "disabled."
|
||||
|
||||
## Scope
|
||||
|
||||
The work will add two independent per-run controls:
|
||||
|
||||
- a direct session ID associated with the run; and
|
||||
- a tri-state reasoning override that can inherit, replace, or disable the
|
||||
selected profile's reasoning effort.
|
||||
|
||||
These controls belong to the existing request and execution-target surfaces.
|
||||
They will not introduce another prompt or profile selection mechanism.
|
||||
|
||||
## Session ID Behavior
|
||||
|
||||
`RunRequest` will accept an optional direct session ID. Promptkit will treat
|
||||
session IDs as opaque, consumer-supplied correlation values rather than
|
||||
credentials or conversation storage keys managed by the library.
|
||||
|
||||
The effective session ID will resolve as follows:
|
||||
|
||||
1. a nonblank direct request session ID;
|
||||
2. the rendered prompt-definition session ID template; or
|
||||
3. no session ID.
|
||||
|
||||
A direct value will be trimmed and will take complete precedence over the
|
||||
prompt template. When a direct value is present, Promptkit will not parse or
|
||||
render the prompt's session ID template. An unused template therefore cannot
|
||||
fail the run because of a missing variable or another template error. A blank
|
||||
direct value means that no direct override was supplied and retains the prompt
|
||||
template behavior.
|
||||
|
||||
The direct value will use the existing session validation rule: at most 256
|
||||
Unicode code points after trimming. An invalid direct value will fail at the
|
||||
per-run request boundary. Prompt-rendered session IDs will retain their
|
||||
existing prompt-render failure boundary.
|
||||
|
||||
For the built-in OpenAI-compatible client, a nonempty effective session ID
|
||||
will continue to be sent as the top-level `session_id` request-body field. It
|
||||
will not also be sent through the `x-session-id` header. This matches
|
||||
OpenRouter's documented request schema and avoids two competing wire values.
|
||||
Promptkit will not claim that every OpenAI-compatible backend implements
|
||||
OpenRouter's sticky-routing or observability semantics.
|
||||
|
||||
Consumers are responsible for choosing stable identifiers for related calls.
|
||||
Session IDs may appear in prepared values, results, provider requests, and
|
||||
provider observability systems, so they must not contain credentials or
|
||||
unnecessarily sensitive data.
|
||||
|
||||
## Reasoning Override Behavior
|
||||
|
||||
The profile's reasoning effort will remain a reusable default. The per-run
|
||||
execution override will distinguish three states:
|
||||
|
||||
- omitted: inherit the selected profile's reasoning effort;
|
||||
- nonblank: replace the profile value with the trimmed request value; and
|
||||
- explicitly blank: disable inherited reasoning for the run.
|
||||
|
||||
The public `ExecutionTargetOverride.ReasoningEffort` field will become a string
|
||||
pointer so `nil`, a pointer to a nonblank string, and a pointer to a blank
|
||||
string represent those three states directly. `Profile.ReasoningEffort` and
|
||||
the effective `ExecutionTarget.ReasoningEffort` will remain strings.
|
||||
|
||||
Promptkit will not define a closed set of reasoning effort names because
|
||||
supported values may vary across OpenAI-compatible backends. Explicit disable
|
||||
will produce an empty effective reasoning setting, and the built-in client
|
||||
will omit `reasoning_effort` from the provider request. Promptkit will not
|
||||
translate disable into a provider-specific magic value such as `none`.
|
||||
|
||||
## Metadata And Generation Boundary
|
||||
|
||||
The effective direct or rendered session ID will:
|
||||
|
||||
- remain visible in `PreparedRun`;
|
||||
- be added to `RunResult`;
|
||||
- be included in the rendered-prompt hash; and
|
||||
- be supplied to injected model clients through the rendered prompt.
|
||||
|
||||
The prompt-definition hash will continue to describe the selected definition,
|
||||
including its configured session template, even when a direct value bypasses
|
||||
that template. The rendered-prompt hash will describe the effective session ID
|
||||
and rendered messages used for the run.
|
||||
|
||||
The effective reasoning effort will remain visible through
|
||||
`EffectiveModelParams` in prepared and completed run metadata and through the
|
||||
execution target supplied to injected model clients. An empty effective value
|
||||
means that the client should omit a reasoning setting.
|
||||
|
||||
## Compatibility
|
||||
|
||||
Adding a direct session field and completed-run session metadata is additive.
|
||||
Changing `ExecutionTargetOverride.ReasoningEffort` from `string` to `*string`
|
||||
is a source compatibility change for consumers that initialize that field.
|
||||
|
||||
Promptkit's [pre-`v1` release policy](../release.md#release-model) permits
|
||||
public API changes in a minor release. This work will use that release boundary
|
||||
in favor of a single idiomatic tri-state field rather than permanently adding
|
||||
a second disable flag or parallel override field. Release notes for the
|
||||
version that publishes the change will identify the required consumer update.
|
||||
|
||||
Existing behavior will otherwise remain compatible:
|
||||
|
||||
- prompts without a direct session ID continue rendering their session
|
||||
template;
|
||||
- prompts without either session source continue without a session ID;
|
||||
- profiles continue supplying reasoning defaults;
|
||||
- omitted per-run reasoning continues inheriting the profile;
|
||||
- endpoint-only and backend-selected profiles behave identically;
|
||||
- injected model clients continue receiving the effective rendered prompt and
|
||||
execution target; and
|
||||
- synchronous `Prepare` and `Run` behavior remains unchanged.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This scope does not include:
|
||||
|
||||
- conversation history, memory, or message persistence;
|
||||
- durable session storage or session lifecycle management;
|
||||
- automatic session ID generation;
|
||||
- user identity, authorization, or tenancy policy;
|
||||
- provider-specific reasoning vocabularies or capability discovery;
|
||||
- translating reasoning settings between provider protocols;
|
||||
- backend-specific header configuration or other transport capabilities;
|
||||
- changes to prompt or profile file formats;
|
||||
- concurrency limits, admission queues, asynchronous jobs, or retries; or
|
||||
- changes to model, backend, prompt, or profile selection.
|
||||
|
||||
## Target End State
|
||||
|
||||
This roadmap reaches its target end state when:
|
||||
|
||||
- consumers can provide a direct per-run session ID without modifying prompt
|
||||
variables or definitions;
|
||||
- a direct session ID bypasses and takes precedence over the prompt session
|
||||
template;
|
||||
- direct and rendered session IDs share the established normalization and
|
||||
maximum-length contract;
|
||||
- the effective session ID is carried consistently through preparation,
|
||||
hashing, execution, completed-run metadata, and injected clients;
|
||||
- the built-in client sends the effective session ID only as the documented
|
||||
top-level request-body field;
|
||||
- consumers can inherit, replace, or explicitly disable a profile's reasoning
|
||||
effort through one tri-state override;
|
||||
- explicit reasoning disable results in omission from the provider request;
|
||||
- effective reasoning remains visible in preparation, results, and the model
|
||||
client boundary;
|
||||
- the compatibility impact is identified for the next release according to
|
||||
the pre-`v1` minor-release policy; and
|
||||
- current-state GoDoc, consumer, integration, and internal documentation
|
||||
describe the implemented behavior without relying on this roadmap.
|
||||
Reference in New Issue
Block a user