97 lines
4.7 KiB
Markdown
97 lines
4.7 KiB
Markdown
# Runtime Parameter Feature Roadmap
|
|
|
|
This roadmap defines the target behavior for runtime model parameters.
|
|
|
|
Current behavior has two limitations:
|
|
|
|
- `reasoning_effort` and `extra_params` are parsed into effective execution settings but are not serialized into outbound OpenAI-compatible chat-completions requests.
|
|
- Per-request numeric execution overrides use zero-value merge semantics, so callers cannot reliably override a profile value with an explicit zero such as `temperature: 0`.
|
|
|
|
The implementation plan for this feature lives in `docs/roadmap/implementation.md`.
|
|
|
|
## Target State
|
|
|
|
Scriptorium should preserve the existing separation between prompt definitions, execution profiles, and per-request execution overrides while making runtime parameter behavior explicit and predictable.
|
|
|
|
Expected end state:
|
|
|
|
- Effective execution settings remain visible in prepared-run output, run metadata, and HTTP metadata without exposing raw secret values.
|
|
- `reasoning_effort` is treated as a first-class effective execution setting and is serialized to the outbound OpenAI-compatible request when set.
|
|
- `extra_params` supports provider-specific OpenAI-compatible request fields.
|
|
- `extra_params` is serialized as additional top-level outbound JSON fields.
|
|
- `extra_params` values support JSON-compatible scalar, object, and array values.
|
|
- `extra_params` cannot override first-class outbound request fields.
|
|
- Per-request numeric overrides preserve caller intent, including explicit zero values.
|
|
- Omitted per-request numeric overrides continue to inherit the selected profile and built-in defaults.
|
|
- External decoding remains strict for config, prompt, profile, and HTTP request payloads.
|
|
|
|
## Policy Decisions
|
|
|
|
### `extra_params`
|
|
|
|
`extra_params` should serialize as additional top-level outbound JSON fields in the OpenAI-compatible chat-completions request.
|
|
|
|
Reasoning:
|
|
|
|
Most OpenAI-compatible providers expose vendor-specific chat-completions parameters as top-level fields. This keeps Scriptorium's adapter compatible with that ecosystem without adding first-class fields for every provider option.
|
|
|
|
`extra_params` must not silently override Scriptorium-owned fields. Reserved outbound fields include at least:
|
|
|
|
- `model`
|
|
- `session_id`
|
|
- `messages`
|
|
- `temperature`
|
|
- `max_tokens`
|
|
- `top_p`
|
|
- `service_tier`
|
|
- `reasoning_effort`
|
|
- `response_format`
|
|
|
|
If a caller supplies a reserved key through `extra_params`, Scriptorium should fail before making the outbound HTTP request.
|
|
|
|
`extra_params` should use JSON-compatible values rather than only strings.
|
|
|
|
Reasoning:
|
|
|
|
Provider-specific parameters commonly need booleans, numbers, objects, or arrays. String-only values would force awkward encoding and would likely require a later compatibility break.
|
|
|
|
### Presence-Aware Overrides
|
|
|
|
Per-request execution overrides should use a presence-aware type with pointer fields for optional numeric values.
|
|
|
|
Reasoning:
|
|
|
|
The resolved execution target should remain a concrete value used by prepared runs, generated requests, and metadata. Optionality matters at the request boundary, not after the runner has resolved the effective target.
|
|
|
|
This keeps adapter and merge logic precise while avoiding nil checks in formatter, metadata, and LLM serialization paths.
|
|
|
|
## Scope
|
|
|
|
In scope:
|
|
|
|
- Runtime merge behavior for per-request execution overrides.
|
|
- HTTP model override decoding for explicit zero numeric values.
|
|
- CLI execution override handling for explicit zero numeric flags.
|
|
- Outbound serialization of `reasoning_effort`.
|
|
- Outbound serialization of JSON-compatible `extra_params`.
|
|
- Tests and documentation for the changed implemented behavior.
|
|
|
|
Out of scope:
|
|
|
|
- Expanding the HTTP API beyond `POST /v1/runs`.
|
|
- Adding built-in HTTP authentication or authorization.
|
|
- Adding durable run state, run history, or multi-step orchestration.
|
|
- Adding broad provider-specific adapter packages.
|
|
- Adding new CLI flags for every provider-specific parameter.
|
|
|
|
## Acceptance Criteria
|
|
|
|
- A profile containing `reasoning_effort: medium` produces an outbound request with `reasoning_effort`.
|
|
- HTTP callers can pass `reasoning_effort` through the existing `model` override object and have it appear outbound.
|
|
- A profile or HTTP request containing JSON-compatible `extra_params` produces outbound top-level JSON fields according to the reserved-field policy.
|
|
- Reserved `extra_params` collisions fail before the outbound provider call.
|
|
- CLI callers can pass `--temperature 0` and observe `temperature: 0` in rendered/effective settings and outbound requests.
|
|
- HTTP callers can send `"temperature": 0` and observe the same behavior.
|
|
- Omitting `temperature` continues to preserve the selected profile/default value.
|
|
- Raw API key values remain unsupported in config, profiles, CLI flags, HTTP payloads, logs, and rendered output.
|