Build presence-aware execution targets from effective configuration
This commit is contained in:
@@ -17,7 +17,7 @@ contributor workflow and validation.
|
|||||||
| `examples/go-library/observe` | Demonstrates offline built-in HTTP retry and synchronous metadata observation across structured proposal and assessment runs using embedded sources, two profiles on one finite backend, and an application decision. It is not a public library package. | [Example program](../../examples/go-library/observe/main.go) |
|
| `examples/go-library/observe` | Demonstrates offline built-in HTTP retry and synchronous metadata observation across structured proposal and assessment runs using embedded sources, two profiles on one finite backend, and an application decision. It is not a public library package. | [Example program](../../examples/go-library/observe/main.go) |
|
||||||
| `internal/backend` | Constructs each engine's immutable registry from maintained definitions and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |
|
| `internal/backend` | Constructs each engine's immutable registry from maintained definitions and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |
|
||||||
| `internal/catalog` | Strictly validates imported immutable maintained backend and profile catalog assets before engine assembly uses them. | [Catalog adapter](../../internal/catalog/catalog.go), [internal sources](sources.md#profiles-and-built-ins) |
|
| `internal/catalog` | Strictly validates imported immutable maintained backend and profile catalog assets before engine assembly uses them. | [Catalog adapter](../../internal/catalog/catalog.go), [internal sources](sources.md#profiles-and-built-ins) |
|
||||||
| `internal/configuration` | Loads keyed backend, profile, and prompt configuration through ConfigKit; records effective field presence, validates final values, and captures indexed prompt definitions and referenced message content. It is not yet connected to engine assembly. | [Typed loader](../../internal/configuration/load.go), [internal sources](sources.md#typed-configuration-loading) |
|
| `internal/configuration` | Loads keyed backend, profile, and prompt configuration through ConfigKit; records effective field presence, validates final values, captures indexed prompt definitions and referenced message content, and constructs immutable backend and presence-aware profile snapshots. It is not yet connected to engine assembly. | [Typed loader](../../internal/configuration/load.go), [internal sources](sources.md#typed-configuration-loading) |
|
||||||
| `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) |
|
| `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) |
|
||||||
| `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, profiles, execution targets, rendering, generation, and validation, and owns source-neutral invariants for shared execution settings, OpenAI-compatible base endpoints, session identifiers, and output contracts. Source parsing, required fields, other source-specific normalization, defaulting, and boundary-specific error classification remain with their callers. | [Domain declarations](../../internal/domain/domain.go), [endpoint invariant](../../internal/domain/endpoint.go) |
|
| `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, profiles, execution targets, rendering, generation, and validation, and owns source-neutral invariants for shared execution settings, OpenAI-compatible base endpoints, session identifiers, and output contracts. Source parsing, required fields, other source-specific normalization, defaulting, and boundary-specific error classification remain with their callers. | [Domain declarations](../../internal/domain/domain.go), [endpoint invariant](../../internal/domain/endpoint.go) |
|
||||||
| `internal/engine` | Assembles external catalogs, Promptkit-owned profiles, repositories, registry, capacity, collaborators, repairer, and the one use-case runner for each engine instance. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#profiles-and-built-ins), [internal runner](runner.md) |
|
| `internal/engine` | Assembles external catalogs, Promptkit-owned profiles, repositories, registry, capacity, collaborators, repairer, and the one use-case runner for each engine instance. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#profiles-and-built-ins), [internal runner](runner.md) |
|
||||||
|
|||||||
@@ -53,8 +53,16 @@ explicit profile ID. Both then apply the ordinary source precedence, resolve a
|
|||||||
named backend, and construct the effective target from framework, backend, and
|
named backend, and construct the effective target from framework, backend, and
|
||||||
profile values.
|
profile values.
|
||||||
|
|
||||||
Inspection stops after the resulting endpoint and model are structurally
|
The private ConfigKit loader can instead supply an already resolved,
|
||||||
validated. It does not check credential availability or perform prompt,
|
presence-aware target to this boundary. Request overrides still apply afterward,
|
||||||
|
and preparation checks endpoint and model completeness only after those
|
||||||
|
overrides. This path remains internal until the engine assembly switches to the
|
||||||
|
loader; the existing public engine still uses the repository path above.
|
||||||
|
|
||||||
|
For the existing repository path, inspection validates the resulting endpoint
|
||||||
|
and model. For the precomputed target path it can report an incomplete profile;
|
||||||
|
preparation remains responsible for checking completeness after request
|
||||||
|
overrides. Neither path checks credential availability or performs prompt,
|
||||||
artifact, schema, rendering, admission, or model-client work. The root
|
artifact, schema, rendering, admission, or model-client work. The root
|
||||||
[`Engine.InspectProfile`](../../engine.go) GoDoc owns the public operation's
|
[`Engine.InspectProfile`](../../engine.go) GoDoc owns the public operation's
|
||||||
exact contract.
|
exact contract.
|
||||||
|
|||||||
@@ -20,7 +20,11 @@ profile domain values. A lower-layer domain value replaced before the handoff
|
|||||||
is not separately validated. It also validates effective prompt fields, reads
|
is not separately validated. It also validates effective prompt fields, reads
|
||||||
referenced message content using the contributing field's passive source
|
referenced message content using the contributing field's passive source
|
||||||
context, checks static template syntax, and indexes immutable definitions by
|
context, checks static template syntax, and indexes immutable definitions by
|
||||||
exact ID and version. Its focused tests exercise real ConfigKit loading.
|
exact ID and version. The loader also builds an immutable backend registry from
|
||||||
|
the final definitions and indexes profile execution targets after inheritance,
|
||||||
|
applying the selected backend's defaults only then. These targets retain
|
||||||
|
explicit numeric-zero presence and null suppression for runtime execution.
|
||||||
|
Its focused tests exercise real ConfigKit loading.
|
||||||
|
|
||||||
This loader is not yet wired into engine assembly. The public engine continues
|
This loader is not yet wired into engine assembly. The public engine continues
|
||||||
to use the source and repository behavior described below. The public
|
to use the source and repository behavior described below. The public
|
||||||
|
|||||||
@@ -359,6 +359,8 @@ existing execution tests still work through the old assembly until cutover.
|
|||||||
|
|
||||||
## Stage 3 — Make Execution Settings Presence-Aware
|
## Stage 3 — Make Execution Settings Presence-Aware
|
||||||
|
|
||||||
|
**Completed:** 2026-09-30.
|
||||||
|
|
||||||
Scope: domain settings, target resolution, wire-presence propagation, profile
|
Scope: domain settings, target resolution, wire-presence propagation, profile
|
||||||
inspection. No new source discovery in execution packages.
|
inspection. No new source discovery in execution packages.
|
||||||
|
|
||||||
|
|||||||
@@ -35,6 +35,23 @@ type Registry struct {
|
|||||||
backends map[string]domain.Backend
|
backends map[string]domain.Backend
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// NewEffectiveRegistry stores already-validated effective backend definitions
|
||||||
|
// under their exact configuration keys. Configuration validation and
|
||||||
|
// normalization precede this ownership boundary.
|
||||||
|
func NewEffectiveRegistry(definitions map[string]domain.Backend) (*Registry, error) {
|
||||||
|
registry := &Registry{backends: make(map[string]domain.Backend, len(definitions))}
|
||||||
|
for id, definition := range definitions {
|
||||||
|
params, err := jsonvalue.CopyMap(definition.ExtraParams)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("backend %q extra parameters: %w", id, err)
|
||||||
|
}
|
||||||
|
definition.ID = id
|
||||||
|
definition.ExtraParams = params
|
||||||
|
registry.backends[id] = definition
|
||||||
|
}
|
||||||
|
return registry, nil
|
||||||
|
}
|
||||||
|
|
||||||
// NewRegistry constructs a registry containing maintained definitions followed
|
// NewRegistry constructs a registry containing maintained definitions followed
|
||||||
// by consumer additions. Every ID must be unique across both groups.
|
// by consumer additions. Every ID must be unique across both groups.
|
||||||
func NewRegistry(maintained, additions []domain.Backend) (*Registry, error) {
|
func NewRegistry(maintained, additions []domain.Backend) (*Registry, error) {
|
||||||
|
|||||||
@@ -48,6 +48,29 @@ func TestRegistryIncludesMaintainedDefinitions(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestEffectiveRegistryKeepsLiteralIDsAndOwnsValues(t *testing.T) {
|
||||||
|
params := map[string]any{"nested": map[string]any{"enabled": true}}
|
||||||
|
registry, err := backend.NewEffectiveRegistry(map[string]domain.Backend{
|
||||||
|
" local.v1 ": {Endpoint: validEndpoint, ExtraParams: params, ConcurrencyLimit: 2, QueueCapacitySet: true},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
params["nested"].(map[string]any)["enabled"] = false
|
||||||
|
selected, err := registry.GetBackend(" local.v1 ")
|
||||||
|
if err != nil || selected.ID != " local.v1 " || selected.ExtraParams["nested"].(map[string]any)["enabled"] != true {
|
||||||
|
t.Fatalf("effective registry changed identity or retained caller values: %+v, %v", selected, err)
|
||||||
|
}
|
||||||
|
if _, err := registry.GetBackend("local.v1"); !errors.Is(err, backend.ErrBackendNotFound) {
|
||||||
|
t.Fatalf("trimmed identity unexpectedly matched: %v", err)
|
||||||
|
}
|
||||||
|
selected.ExtraParams["nested"].(map[string]any)["enabled"] = false
|
||||||
|
again, _ := registry.GetBackend(" local.v1 ")
|
||||||
|
if again.ExtraParams["nested"].(map[string]any)["enabled"] != true {
|
||||||
|
t.Fatal("registry lookup exposed its stored values")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestRegistryNormalizesUniqueAdditionsAndIsolatesMutations(t *testing.T) {
|
func TestRegistryNormalizesUniqueAdditionsAndIsolatesMutations(t *testing.T) {
|
||||||
nested := map[string]int{"limit": 2}
|
nested := map[string]int{"limit": 2}
|
||||||
extraParams := map[string]any{
|
extraParams := map[string]any{
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"gitea.maximumdirect.net/eric/ConfigKit"
|
"gitea.maximumdirect.net/eric/ConfigKit"
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit/internal/backend"
|
||||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||||
"gitea.maximumdirect.net/eric/promptkit/internal/jsonvalue"
|
"gitea.maximumdirect.net/eric/promptkit/internal/jsonvalue"
|
||||||
"gitea.maximumdirect.net/eric/promptkit/internal/llm"
|
"gitea.maximumdirect.net/eric/promptkit/internal/llm"
|
||||||
@@ -89,11 +90,13 @@ const (
|
|||||||
)
|
)
|
||||||
|
|
||||||
type loaded struct {
|
type loaded struct {
|
||||||
result *configkit.Result[document]
|
result *configkit.Result[document]
|
||||||
backends map[string]map[string]fieldState
|
backends map[string]map[string]fieldState
|
||||||
profiles map[string]map[string]fieldState
|
profiles map[string]map[string]fieldState
|
||||||
prompts map[string]map[string]*promptEntry
|
prompts map[string]map[string]*promptEntry
|
||||||
ambiguous map[string]bool
|
ambiguous map[string]bool
|
||||||
|
registry *backend.Registry
|
||||||
|
indexedProfiles map[string]domain.ExecutionProfile
|
||||||
}
|
}
|
||||||
|
|
||||||
var envName = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
|
var envName = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
|
||||||
@@ -129,6 +132,9 @@ func load(main configkit.Input, underlays []configkit.Underlay) (*loaded, error)
|
|||||||
if err := state.materializePrompts(); err != nil {
|
if err := state.materializePrompts(); err != nil {
|
||||||
return nil, fmt.Errorf("load configuration: %w", err)
|
return nil, fmt.Errorf("load configuration: %w", err)
|
||||||
}
|
}
|
||||||
|
if err := state.buildExecutionProfiles(); err != nil {
|
||||||
|
return nil, fmt.Errorf("load configuration: %w", err)
|
||||||
|
}
|
||||||
return state, nil
|
return state, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
171
internal/configuration/targets.go
Normal file
171
internal/configuration/targets.go
Normal file
@@ -0,0 +1,171 @@
|
|||||||
|
package configuration
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit/internal/backend"
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit/internal/defaults"
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit/internal/jsonvalue"
|
||||||
|
profilepkg "gitea.maximumdirect.net/eric/promptkit/internal/profile"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (state *loaded) buildExecutionProfiles() error {
|
||||||
|
definitions := make(map[string]domain.Backend, len(state.result.Config.Backends))
|
||||||
|
for _, id := range sortedKeys(state.result.Config.Backends) {
|
||||||
|
config := state.result.Config.Backends[id]
|
||||||
|
params, err := jsonvalue.CopyMap(config.ExtraParams)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("backend %q extra parameters: %w", id, err)
|
||||||
|
}
|
||||||
|
definition := domain.Backend{ID: id, ExtraParams: params}
|
||||||
|
if config.Endpoint != nil {
|
||||||
|
definition.Endpoint = *config.Endpoint
|
||||||
|
}
|
||||||
|
if config.APIKeyEnv != nil {
|
||||||
|
definition.APIKeyEnv = *config.APIKeyEnv
|
||||||
|
}
|
||||||
|
if config.ConcurrencyLimit != nil {
|
||||||
|
definition.ConcurrencyLimit = *config.ConcurrencyLimit
|
||||||
|
}
|
||||||
|
if config.QueueCapacity != nil {
|
||||||
|
definition.QueueCapacity = *config.QueueCapacity
|
||||||
|
definition.QueueCapacitySet = definition.ConcurrencyLimit > 0
|
||||||
|
}
|
||||||
|
definitions[id] = definition
|
||||||
|
}
|
||||||
|
registry, err := backend.NewEffectiveRegistry(definitions)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
state.registry = registry
|
||||||
|
state.indexedProfiles = make(map[string]domain.ExecutionProfile, len(state.result.Config.Profiles))
|
||||||
|
for _, id := range sortedKeys(state.result.Config.Profiles) {
|
||||||
|
config := state.result.Config.Profiles[id]
|
||||||
|
fields := state.profiles[id]
|
||||||
|
target := defaults.ExecutionTargetDefault()
|
||||||
|
if config.Backend != nil && *config.Backend != "" {
|
||||||
|
selected, err := registry.GetBackend(*config.Backend)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("profile %q backend: %w", id, err)
|
||||||
|
}
|
||||||
|
target.BackendID = selected.ID
|
||||||
|
target.Endpoint = selected.Endpoint
|
||||||
|
target.APIKeyEnv = selected.APIKeyEnv
|
||||||
|
target.ExtraParams = selected.ExtraParams
|
||||||
|
}
|
||||||
|
presence := domain.ExecutionTargetPresence{}
|
||||||
|
if fields["endpoint"] != fieldAbsent {
|
||||||
|
target.Endpoint = valueOrZero(config.Endpoint)
|
||||||
|
}
|
||||||
|
if fields["model"] != fieldAbsent {
|
||||||
|
target.Model = valueOrZero(config.Model)
|
||||||
|
}
|
||||||
|
if fields["temperature"] != fieldAbsent {
|
||||||
|
target.Temperature = valueOrZero(config.Temperature)
|
||||||
|
presence.Temperature = fields["temperature"] == fieldValue
|
||||||
|
}
|
||||||
|
if fields["top_p"] != fieldAbsent {
|
||||||
|
target.TopP = valueOrZero(config.TopP)
|
||||||
|
presence.TopP = fields["top_p"] == fieldValue
|
||||||
|
}
|
||||||
|
if fields["max_tokens"] != fieldAbsent {
|
||||||
|
target.MaxTokens = valueOrZero(config.MaxTokens)
|
||||||
|
presence.MaxTokens = fields["max_tokens"] == fieldValue
|
||||||
|
}
|
||||||
|
if fields["timeout_seconds"] == fieldValue {
|
||||||
|
target.TimeoutSeconds = valueOrZero(config.TimeoutSeconds)
|
||||||
|
presence.TimeoutSeconds = true
|
||||||
|
}
|
||||||
|
if fields["service_tier"] != fieldAbsent {
|
||||||
|
target.ServiceTier = valueOrZero(config.ServiceTier)
|
||||||
|
}
|
||||||
|
if fields["reasoning_effort"] != fieldAbsent {
|
||||||
|
target.ReasoningEffort = valueOrZero(config.ReasoningEffort)
|
||||||
|
}
|
||||||
|
if fields["api_key_env"] != fieldAbsent {
|
||||||
|
target.APIKeyEnv = valueOrZero(config.APIKeyEnv)
|
||||||
|
}
|
||||||
|
if fields["api_key_required"] != fieldAbsent {
|
||||||
|
target.APIKeyRequired = valueOrZero(config.APIKeyRequired)
|
||||||
|
}
|
||||||
|
if target.APIKeyRequired && fields["api_key_env"] == fieldAbsent {
|
||||||
|
target.APIKeyEnv = ""
|
||||||
|
}
|
||||||
|
switch fields["extra_params"] {
|
||||||
|
case fieldNull:
|
||||||
|
target.ExtraParams = nil
|
||||||
|
case fieldValue:
|
||||||
|
merged, err := overlayProviderMap(target.ExtraParams, config.ExtraParams)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("profile %q extra parameters: %w", id, err)
|
||||||
|
}
|
||||||
|
target.ExtraParams = merged
|
||||||
|
}
|
||||||
|
profile := domain.ExecutionProfile{ID: id, BackendID: target.BackendID, ResolvedTarget: &target, TargetPresence: presence}
|
||||||
|
state.indexedProfiles[id] = profile
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func valueOrZero[T any](value *T) T {
|
||||||
|
if value == nil {
|
||||||
|
var zero T
|
||||||
|
return zero
|
||||||
|
}
|
||||||
|
return *value
|
||||||
|
}
|
||||||
|
|
||||||
|
// overlayProviderMap applies only the backend-to-profile JSON-object default
|
||||||
|
// merge. ConfigKit has already resolved source layering and inheritance.
|
||||||
|
func overlayProviderMap(lower, upper map[string]any) (map[string]any, error) {
|
||||||
|
merged, err := jsonvalue.CopyMap(lower)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if merged == nil {
|
||||||
|
merged = make(map[string]any)
|
||||||
|
}
|
||||||
|
for key, value := range upper {
|
||||||
|
if child, ok := value.(map[string]any); ok {
|
||||||
|
if existing, ok := merged[key].(map[string]any); ok {
|
||||||
|
combined, err := overlayProviderMap(existing, child)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
merged[key] = combined
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
copied, err := jsonvalue.Copy(value)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
merged[key] = copied
|
||||||
|
}
|
||||||
|
return merged, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetProfile returns an independently owned indexed profile for the internal
|
||||||
|
// runner's lookup boundary.
|
||||||
|
func (state *loaded) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if state == nil {
|
||||||
|
return nil, fmt.Errorf("%w: %q", profilepkg.ErrProfileNotFound, id)
|
||||||
|
}
|
||||||
|
profile, found := state.indexedProfiles[id]
|
||||||
|
if !found {
|
||||||
|
return nil, fmt.Errorf("%w: %q", profilepkg.ErrProfileNotFound, id)
|
||||||
|
}
|
||||||
|
target := *profile.ResolvedTarget
|
||||||
|
params, err := jsonvalue.CopyMap(target.ExtraParams)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
target.ExtraParams = params
|
||||||
|
profile.ResolvedTarget = &target
|
||||||
|
return &profile, nil
|
||||||
|
}
|
||||||
152
internal/configuration/targets_test.go
Normal file
152
internal/configuration/targets_test.go
Normal file
@@ -0,0 +1,152 @@
|
|||||||
|
package configuration
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit/internal/defaults"
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit/internal/llm"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestLoadBuildsEffectiveTargetsAfterInheritance(t *testing.T) {
|
||||||
|
state, err := load(mapping("effective-targets", map[string]any{
|
||||||
|
"backends": map[string]any{
|
||||||
|
"first": map[string]any{"endpoint": "https://first.example/v1", "api_key_env": "FIRST_KEY",
|
||||||
|
"extra_params": map[string]any{"source": "first", "nested": map[string]any{"old": true}}},
|
||||||
|
"second": map[string]any{"endpoint": "https://second.example/v1", "api_key_env": "SECOND_KEY",
|
||||||
|
"concurrency_limit": 2, "queue_capacity": 0,
|
||||||
|
"extra_params": map[string]any{"source": "second", "nested": map[string]any{"backend": true}}},
|
||||||
|
},
|
||||||
|
"profiles": map[string]any{
|
||||||
|
"base": map[string]any{"backend": "first", "model": "model", "temperature": 0.8,
|
||||||
|
"top_p": 0.9, "max_tokens": 100, "timeout_seconds": 30, "api_key_required": true,
|
||||||
|
"extra_params": map[string]any{"nested": map[string]any{"profile": true}}},
|
||||||
|
"child": map[string]any{"extends": "base", "backend": "second", "temperature": 0.0,
|
||||||
|
"top_p": nil, "max_tokens": 0, "timeout_seconds": nil, "api_key_env": nil,
|
||||||
|
"api_key_required": false,
|
||||||
|
"extra_params": map[string]any{"nested": map[string]any{"child": nil}}},
|
||||||
|
"direct-key": map[string]any{"backend": "second", "api_key_required": true},
|
||||||
|
"named-key": map[string]any{"backend": "second", "api_key_required": true, "api_key_env": "CUSTOM_KEY"},
|
||||||
|
"no-provider-map": map[string]any{"backend": "second", "extra_params": nil},
|
||||||
|
"no-timeout": map[string]any{"backend": "second", "timeout_seconds": 0},
|
||||||
|
"partial": map[string]any{},
|
||||||
|
"endpoint-only": map[string]any{"endpoint": "https://standalone.example/v1"},
|
||||||
|
"cleared-endpoint": map[string]any{"extends": "base", "endpoint": nil},
|
||||||
|
},
|
||||||
|
}), nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
selected, err := state.registry.GetBackend("second")
|
||||||
|
if err != nil || selected.QueueCapacity != 0 || !selected.QueueCapacitySet {
|
||||||
|
t.Fatalf("backend capacity was not preserved: %+v, %v", selected, err)
|
||||||
|
}
|
||||||
|
child, err := state.GetProfile(context.Background(), "child")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := state.GetProfile(context.Background(), " child "); err == nil {
|
||||||
|
t.Fatal("profile lookup silently changed a literal ID")
|
||||||
|
}
|
||||||
|
target := child.ResolvedTarget
|
||||||
|
if target.BackendID != "second" || target.Endpoint != "https://second.example/v1" || target.Model != "model" ||
|
||||||
|
target.Temperature != 0 || target.TopP != 0 || target.MaxTokens != 0 ||
|
||||||
|
target.TimeoutSeconds != defaults.ExecutionDefaultTimeoutSeconds || target.APIKeyEnv != "" || target.APIKeyRequired {
|
||||||
|
t.Fatalf("effective child target is wrong: %+v", target)
|
||||||
|
}
|
||||||
|
if !child.TargetPresence.Temperature || !child.TargetPresence.MaxTokens || child.TargetPresence.TopP || child.TargetPresence.TimeoutSeconds {
|
||||||
|
t.Fatalf("numeric presence was not retained: %+v", child.TargetPresence)
|
||||||
|
}
|
||||||
|
wantParams := map[string]any{"source": "second", "nested": map[string]any{
|
||||||
|
"backend": true, "profile": true, "child": nil,
|
||||||
|
}}
|
||||||
|
if !reflect.DeepEqual(target.ExtraParams, wantParams) {
|
||||||
|
t.Fatalf("backend/profile provider map overlay failed: %+v", target.ExtraParams)
|
||||||
|
}
|
||||||
|
if _, exists := target.ExtraParams["nested"].(map[string]any)["old"]; exists {
|
||||||
|
t.Fatal("previous backend derived parameter leaked into child")
|
||||||
|
}
|
||||||
|
for id, wantEnv := range map[string]string{"direct-key": "", "named-key": "CUSTOM_KEY"} {
|
||||||
|
profile, err := state.GetProfile(context.Background(), id)
|
||||||
|
if err != nil || profile.ResolvedTarget.APIKeyEnv != wantEnv {
|
||||||
|
t.Fatalf("credential source for %s: %+v, %v", id, profile, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
noMap, _ := state.GetProfile(context.Background(), "no-provider-map")
|
||||||
|
if noMap.ResolvedTarget.ExtraParams != nil {
|
||||||
|
t.Fatalf("explicit null provider map did not suppress backend defaults: %+v", noMap.ResolvedTarget.ExtraParams)
|
||||||
|
}
|
||||||
|
noTimeout, _ := state.GetProfile(context.Background(), "no-timeout")
|
||||||
|
if noTimeout.ResolvedTarget.TimeoutSeconds != 0 || !noTimeout.TargetPresence.TimeoutSeconds {
|
||||||
|
t.Fatalf("explicit zero timeout was lost: %+v", noTimeout)
|
||||||
|
}
|
||||||
|
partial, _ := state.GetProfile(context.Background(), "partial")
|
||||||
|
if partial.ResolvedTarget.Endpoint != "" || partial.ResolvedTarget.Model != "" {
|
||||||
|
t.Fatalf("partial profile gained a runnable target: %+v", partial.ResolvedTarget)
|
||||||
|
}
|
||||||
|
endpointOnly, _ := state.GetProfile(context.Background(), "endpoint-only")
|
||||||
|
if endpointOnly.ResolvedTarget.Endpoint != "https://standalone.example/v1" ||
|
||||||
|
endpointOnly.ResolvedTarget.BackendID != "" || endpointOnly.ResolvedTarget.Model != "" {
|
||||||
|
t.Fatalf("endpoint-only profile was not retained: %+v", endpointOnly.ResolvedTarget)
|
||||||
|
}
|
||||||
|
clearedEndpoint, _ := state.GetProfile(context.Background(), "cleared-endpoint")
|
||||||
|
if clearedEndpoint.ResolvedTarget.Endpoint != "" || clearedEndpoint.ResolvedTarget.BackendID != "first" {
|
||||||
|
t.Fatalf("explicit null recovered backend endpoint: %+v", clearedEndpoint.ResolvedTarget)
|
||||||
|
}
|
||||||
|
child.ResolvedTarget.ExtraParams["source"] = "mutated"
|
||||||
|
again, _ := state.GetProfile(context.Background(), "child")
|
||||||
|
if again.ResolvedTarget.ExtraParams["source"] != "second" {
|
||||||
|
t.Fatal("profile lookup exposed snapshot ownership")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadedZeroControlsReachBuiltInRequest(t *testing.T) {
|
||||||
|
payloads := make(chan map[string]any, 1)
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||||
|
var payload map[string]any
|
||||||
|
if err := json.NewDecoder(request.Body).Decode(&payload); err != nil {
|
||||||
|
writer.WriteHeader(http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
payloads <- payload
|
||||||
|
writer.Header().Set("Content-Type", "application/json")
|
||||||
|
_, _ = writer.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
state, err := load(mapping("wire-controls", map[string]any{
|
||||||
|
"backends": map[string]any{"local": map[string]any{"endpoint": server.URL + "/v1"}},
|
||||||
|
"profiles": map[string]any{"direct": map[string]any{
|
||||||
|
"backend": "local", "model": "example", "temperature": 0.0, "max_tokens": 0, "top_p": nil,
|
||||||
|
}},
|
||||||
|
}), nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
profile, err := state.GetProfile(context.Background(), "direct")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
client, err := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hello"}}},
|
||||||
|
Target: *profile.ResolvedTarget, TargetPresence: profile.TargetPresence,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
payload := <-payloads
|
||||||
|
if payload["temperature"] != float64(0) || payload["max_tokens"] != float64(0) {
|
||||||
|
t.Fatalf("explicit zeros were omitted from outbound request: %+v", payload)
|
||||||
|
}
|
||||||
|
if _, exists := payload["top_p"]; exists {
|
||||||
|
t.Fatalf("explicit null top_p was sent: %+v", payload)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -208,6 +208,10 @@ type ExecutionProfile struct {
|
|||||||
APIKeyEnv string `yaml:"api_key_env"`
|
APIKeyEnv string `yaml:"api_key_env"`
|
||||||
APIKeyRequired bool `yaml:"-" json:"-"`
|
APIKeyRequired bool `yaml:"-" json:"-"`
|
||||||
ExtraParams map[string]any `yaml:"extra_params"`
|
ExtraParams map[string]any `yaml:"extra_params"`
|
||||||
|
// ResolvedTarget is an execution-owned snapshot assembled from effective
|
||||||
|
// configuration and backend defaults. Legacy source repositories leave it nil.
|
||||||
|
ResolvedTarget *ExecutionTarget `yaml:"-" json:"-"`
|
||||||
|
TargetPresence ExecutionTargetPresence `yaml:"-" json:"-"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// ExecutionTargetOverride represents per-request runtime setting overrides.
|
// ExecutionTargetOverride represents per-request runtime setting overrides.
|
||||||
@@ -224,8 +228,9 @@ type ExecutionTargetOverride struct {
|
|||||||
ExtraParams map[string]any `json:"extra_params,omitempty"`
|
ExtraParams map[string]any `json:"extra_params,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// ExecutionTargetPresence tracks which effective runtime fields came from an
|
// ExecutionTargetPresence tracks effective numeric values that must be sent
|
||||||
// explicit request override even when the resolved value is a zero value.
|
// even when they are zero. Values may come from configuration or a request
|
||||||
|
// override; explicit configuration null suppresses the corresponding bit.
|
||||||
type ExecutionTargetPresence struct {
|
type ExecutionTargetPresence struct {
|
||||||
Temperature bool
|
Temperature bool
|
||||||
MaxTokens bool
|
MaxTokens bool
|
||||||
|
|||||||
@@ -36,6 +36,9 @@ func (r *Runner) resolveProfileSelection(
|
|||||||
}
|
}
|
||||||
|
|
||||||
profileValue := *selectedProfile
|
profileValue := *selectedProfile
|
||||||
|
if profileValue.ResolvedTarget != nil {
|
||||||
|
return &resolvedProfileSelection{id: normalizedID, profile: &profileValue}, nil
|
||||||
|
}
|
||||||
profileValue.BackendID = strings.TrimSpace(profileValue.BackendID)
|
profileValue.BackendID = strings.TrimSpace(profileValue.BackendID)
|
||||||
|
|
||||||
var selectedBackend *domain.Backend
|
var selectedBackend *domain.Backend
|
||||||
@@ -92,9 +95,11 @@ func (r *Runner) InspectProfile(
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
target, _ := resolveExecutionTarget(selection.backend, selection.profile, nil)
|
target, _ := resolveExecutionTarget(selection.backend, selection.profile, nil)
|
||||||
target, err = normalizeResolvedExecutionTarget(target)
|
if selection.profile.ResolvedTarget == nil {
|
||||||
if err != nil {
|
target, err = normalizeResolvedExecutionTarget(target)
|
||||||
return nil, fmt.Errorf("%w: %w", ErrProfileLoad, err)
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%w: %w", ErrProfileLoad, err)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
target.APIKey = ""
|
target.APIKey = ""
|
||||||
|
|
||||||
|
|||||||
85
internal/usecase/resolved_targets_test.go
Normal file
85
internal/usecase/resolved_targets_test.go
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
package usecase
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit/internal/defaults"
|
||||||
|
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestInspectProfileAllowsIncompleteResolvedTarget(t *testing.T) {
|
||||||
|
profile := &domain.ExecutionProfile{ID: "partial", ResolvedTarget: &domain.ExecutionTarget{
|
||||||
|
TimeoutSeconds: defaults.ExecutionDefaultTimeoutSeconds,
|
||||||
|
}}
|
||||||
|
runner := &Runner{profiles: staticExecutionProfileRepo{profile: profile}}
|
||||||
|
inspection, err := runner.InspectProfile(context.Background(), "partial")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("partial profile inspection failed: %v", err)
|
||||||
|
}
|
||||||
|
if inspection.EffectiveModelParams.Endpoint != "" || inspection.EffectiveModelParams.Model != "" {
|
||||||
|
t.Fatalf("inspection invented a runnable target: %+v", inspection)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPreparationCompletesResolvedTargetThroughRequestOverrides(t *testing.T) {
|
||||||
|
profile := &domain.ExecutionProfile{ID: "partial", ResolvedTarget: &domain.ExecutionTarget{
|
||||||
|
TimeoutSeconds: defaults.ExecutionDefaultTimeoutSeconds,
|
||||||
|
Temperature: 0, ExtraParams: map[string]any{"configured": true},
|
||||||
|
}, TargetPresence: domain.ExecutionTargetPresence{Temperature: true}}
|
||||||
|
runner := &Runner{
|
||||||
|
promptDefs: staticPromptRepo{def: &domain.PromptDefinition{ID: "prompt", Version: "v1", Validation: domain.OutputContract{
|
||||||
|
Format: domain.FormatText, ValidationMode: domain.ValidationNone,
|
||||||
|
}}},
|
||||||
|
profiles: staticExecutionProfileRepo{profile: profile}, renderer: staticRenderer{},
|
||||||
|
}
|
||||||
|
if _, err := runner.Prepare(context.Background(), domain.RunRequest{PromptID: "prompt", ProfileID: "partial"}); err == nil {
|
||||||
|
t.Fatal("incomplete target was accepted without overrides")
|
||||||
|
}
|
||||||
|
zero := 0.0
|
||||||
|
prepared, err := runner.Prepare(context.Background(), domain.RunRequest{
|
||||||
|
PromptID: "prompt", ProfileID: "partial",
|
||||||
|
Execution: &domain.ExecutionTargetOverride{
|
||||||
|
Endpoint: "https://runtime.example/v1", Model: "runtime-model", Temperature: &zero,
|
||||||
|
ExtraParams: map[string]any{"request": true},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("request did not complete partial target: %v", err)
|
||||||
|
}
|
||||||
|
if prepared.EffectiveModelParams.Endpoint != "https://runtime.example/v1" ||
|
||||||
|
prepared.EffectiveModelParams.Model != "runtime-model" || !prepared.TargetPresence.Temperature ||
|
||||||
|
prepared.EffectiveModelParams.ExtraParams["request"] != true {
|
||||||
|
t.Fatalf("request overrides were not applied: %+v", prepared)
|
||||||
|
}
|
||||||
|
if _, exists := prepared.EffectiveModelParams.ExtraParams["configured"]; exists {
|
||||||
|
t.Fatal("nonempty request provider map did not replace configured map")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolvedTargetCredentialAvailability(t *testing.T) {
|
||||||
|
profile := &domain.ExecutionProfile{ID: "direct", ResolvedTarget: &domain.ExecutionTarget{
|
||||||
|
Endpoint: "https://provider.example/v1", Model: "model", APIKeyRequired: true,
|
||||||
|
}}
|
||||||
|
runner := &Runner{
|
||||||
|
promptDefs: staticPromptRepo{def: &domain.PromptDefinition{ID: "prompt", Version: "v1", Validation: domain.OutputContract{
|
||||||
|
Format: domain.FormatText, ValidationMode: domain.ValidationNone,
|
||||||
|
}}},
|
||||||
|
profiles: staticExecutionProfileRepo{profile: profile}, renderer: staticRenderer{},
|
||||||
|
}
|
||||||
|
request := domain.RunRequest{PromptID: "prompt", ProfileID: "direct"}
|
||||||
|
if _, err := runner.Prepare(context.Background(), request); !errors.Is(err, ErrAPIKeyRequired) {
|
||||||
|
t.Fatalf("missing required direct key: %v", err)
|
||||||
|
}
|
||||||
|
request.APIKey = "direct-secret"
|
||||||
|
if _, err := runner.Prepare(context.Background(), request); err != nil {
|
||||||
|
t.Fatalf("direct key should satisfy requirement: %v", err)
|
||||||
|
}
|
||||||
|
request.APIKey = ""
|
||||||
|
t.Setenv("REQUEST_KEY", "runtime-secret")
|
||||||
|
request.Execution = &domain.ExecutionTargetOverride{APIKeyEnv: "REQUEST_KEY"}
|
||||||
|
if _, err := runner.Prepare(context.Background(), request); err != nil {
|
||||||
|
t.Fatalf("request environment override should satisfy requirement: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -620,9 +620,8 @@ func mergeExecutionTarget(base domain.ExecutionTarget, override domain.Execution
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
func mergeExecutionTargetOverride(base domain.ExecutionTarget, override domain.ExecutionTargetOverride) (domain.ExecutionTarget, domain.ExecutionTargetPresence) {
|
func mergeExecutionTargetOverride(base domain.ExecutionTarget, presence domain.ExecutionTargetPresence, override domain.ExecutionTargetOverride) (domain.ExecutionTarget, domain.ExecutionTargetPresence) {
|
||||||
out := base
|
out := base
|
||||||
var presence domain.ExecutionTargetPresence
|
|
||||||
if override.Endpoint != "" {
|
if override.Endpoint != "" {
|
||||||
out.Endpoint = override.Endpoint
|
out.Endpoint = override.Endpoint
|
||||||
}
|
}
|
||||||
@@ -661,12 +660,21 @@ func mergeExecutionTargetOverride(base domain.ExecutionTarget, override domain.E
|
|||||||
}
|
}
|
||||||
|
|
||||||
func resolveExecutionTarget(backendValue *domain.Backend, profileValue *domain.ExecutionProfile, override *domain.ExecutionTargetOverride) (domain.ExecutionTarget, domain.ExecutionTargetPresence) {
|
func resolveExecutionTarget(backendValue *domain.Backend, profileValue *domain.ExecutionProfile, override *domain.ExecutionTargetOverride) (domain.ExecutionTarget, domain.ExecutionTargetPresence) {
|
||||||
|
if profileValue != nil && profileValue.ResolvedTarget != nil {
|
||||||
|
out := *profileValue.ResolvedTarget
|
||||||
|
out.ExtraParams = copyExtraParams(out.ExtraParams)
|
||||||
|
presence := profileValue.TargetPresence
|
||||||
|
if override != nil {
|
||||||
|
return mergeExecutionTargetOverride(out, presence, *override)
|
||||||
|
}
|
||||||
|
return out, presence
|
||||||
|
}
|
||||||
out := defaults.ExecutionTargetDefault()
|
out := defaults.ExecutionTargetDefault()
|
||||||
out = mergeExecutionTarget(out, backendToTarget(backendValue))
|
out = mergeExecutionTarget(out, backendToTarget(backendValue))
|
||||||
out = mergeExecutionTarget(out, executionProfileToTarget(profileValue))
|
out = mergeExecutionTarget(out, executionProfileToTarget(profileValue))
|
||||||
var presence domain.ExecutionTargetPresence
|
var presence domain.ExecutionTargetPresence
|
||||||
if override != nil {
|
if override != nil {
|
||||||
out, presence = mergeExecutionTargetOverride(out, *override)
|
out, presence = mergeExecutionTargetOverride(out, presence, *override)
|
||||||
}
|
}
|
||||||
return out, presence
|
return out, presence
|
||||||
}
|
}
|
||||||
|
|||||||
19
types.go
19
types.go
@@ -573,18 +573,17 @@ type OpenAICompatibleProfileConfig struct {
|
|||||||
ExtraParams map[string]any
|
ExtraParams map[string]any
|
||||||
}
|
}
|
||||||
|
|
||||||
// ExecutionTargetPresence tracks which numeric runtime settings were explicit
|
// ExecutionTargetPresence reports which effective numeric settings should be
|
||||||
// request overrides, including explicit zero values. It has a stable JSON
|
// sent even when zero. It has a stable JSON representation and lets injected
|
||||||
// representation and is supplied to injected LLM clients so they can preserve
|
// clients preserve omission semantics.
|
||||||
// omission semantics.
|
|
||||||
type ExecutionTargetPresence struct {
|
type ExecutionTargetPresence struct {
|
||||||
// Temperature reports a non-nil ExecutionTargetOverride.Temperature.
|
// Temperature keeps a zero temperature explicit.
|
||||||
Temperature bool `json:"temperature"`
|
Temperature bool `json:"temperature"`
|
||||||
// MaxTokens reports a non-nil ExecutionTargetOverride.MaxTokens.
|
// MaxTokens keeps a zero token maximum explicit.
|
||||||
MaxTokens bool `json:"max_tokens"`
|
MaxTokens bool `json:"max_tokens"`
|
||||||
// TopP reports a non-nil ExecutionTargetOverride.TopP.
|
// TopP keeps a zero nucleus value explicit.
|
||||||
TopP bool `json:"top_p"`
|
TopP bool `json:"top_p"`
|
||||||
// TimeoutSeconds reports a non-nil ExecutionTargetOverride.TimeoutSeconds.
|
// TimeoutSeconds keeps a zero timeout explicit.
|
||||||
TimeoutSeconds bool `json:"timeout_seconds"`
|
TimeoutSeconds bool `json:"timeout_seconds"`
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -739,8 +738,8 @@ type GenerateRequest struct {
|
|||||||
Prompt RenderedPrompt `json:"prompt"`
|
Prompt RenderedPrompt `json:"prompt"`
|
||||||
// Target contains effective model settings without the direct API key.
|
// Target contains effective model settings without the direct API key.
|
||||||
Target ExecutionTarget `json:"target"`
|
Target ExecutionTarget `json:"target"`
|
||||||
// TargetPresence distinguishes inherited numeric zeros from explicit
|
// TargetPresence distinguishes omitted or suppressed controls from explicit
|
||||||
// request overrides.
|
// numeric values, including zero.
|
||||||
TargetPresence ExecutionTargetPresence `json:"target_presence"`
|
TargetPresence ExecutionTargetPresence `json:"target_presence"`
|
||||||
// StructuredOutput contains provider response constraints when requested.
|
// StructuredOutput contains provider response constraints when requested.
|
||||||
StructuredOutput *StructuredOutputSpec `json:"structured_output,omitempty"`
|
StructuredOutput *StructuredOutputSpec `json:"structured_output,omitempty"`
|
||||||
|
|||||||
Reference in New Issue
Block a user