Add local backend convenience constructor

This commit is contained in:
2026-07-30 03:38:23 +00:00
parent e361c97bb5
commit 147f5e5ff5
4 changed files with 509 additions and 739 deletions

View File

@@ -9,6 +9,11 @@ import (
// backend.
const BackendOpenRouter = backend.OpenRouterID
// BackendLocal is the case-sensitive conventional ID used by [LocalBackend].
// It is not a built-in or reserved backend and must be registered with
// [WithBackend].
const BackendLocal = "local"
// Backend configures one engine-scoped OpenAI-compatible backend.
//
// Backend has no stable JSON representation. Use keyed literals so additions
@@ -44,6 +49,26 @@ type Backend struct {
QueueCapacity *int
}
// LocalBackend returns a caller-owned Backend for a conventional local
// OpenAI-compatible endpoint. It sets ID to BackendLocal and copies endpoint
// and concurrencyLimit into Endpoint and ConcurrencyLimit without
// normalization or validation. APIKeyEnv, ExtraParams, and QueueCapacity keep
// their zero values.
//
// LocalBackend does not read environment variables, register the value, or
// mutate engine or package state. Supply the returned value through
// [WithBackend]; [NewEngine] then applies the ordinary backend validation and
// concurrency semantics, including default queue capacity for a positive
// limit, unlimited behavior for zero, and ErrInvalidConfig for a negative
// limit.
func LocalBackend(endpoint string, concurrencyLimit int) Backend {
return Backend{
ID: BackendLocal,
Endpoint: endpoint,
ConcurrencyLimit: concurrencyLimit,
}
}
// WithBackend adds one Backend registration to the constructed Engine.
//
// Registrations accumulate in option order. Every normalized ID must be unique

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,163 @@
# Local Backend Convenience
**Status:** Accepted.
## Purpose
Make the common case of using a local OpenAI-compatible endpoint concise and
easy to discover without introducing implicit configuration or a separate
backend abstraction.
The existing `Backend` type and registry remain the canonical, fully
configurable interface. A small convenience constructor will cover the usual
local-network case, while improved consumer documentation will make it clear
when an endpoint-only profile, the convenience constructor, or a complete
`Backend` value is appropriate.
## Motivation
Consumers can already use a local endpoint by setting `Profile.Endpoint`, or
register one as a backend with `WithBackend`. The first option is concise but
does not provide shared backend-level concurrency control. The second supports
the complete backend feature set but requires consumers to understand and
populate several fields for a common configuration.
Most consumers adding a local backend need only:
- a stable backend ID;
- an OpenAI-compatible endpoint; and
- a concurrency limit appropriate for the local server.
Promptkit should provide a direct path for that case while keeping all
configuration explicit and preserving the full registry interface for
advanced needs.
## Consumer Paths
Documentation should present three progressively more configurable paths:
1. Set `Profile.Endpoint` when a profile only needs to target a local endpoint
and does not need shared backend policy.
2. Use the local-backend convenience constructor when profiles should share a
named local endpoint and its concurrency limit.
3. Construct a complete `Backend` value when the consumer needs a custom
backend ID, authentication, extra request parameters, an explicit queue
capacity, or multiple local backends.
These are complementary interfaces. The convenience constructor must return an
ordinary `Backend`, so it does not create a second configuration model.
## Public Convenience API
The public package should expose:
```go
const BackendLocal = "local"
func LocalBackend(endpoint string, concurrencyLimit int) Backend
```
`LocalBackend` should return a `Backend` with:
- `ID` set to `BackendLocal`;
- `Endpoint` set to the supplied endpoint;
- `ConcurrencyLimit` set to the supplied limit; and
- all other fields left at their zero values.
The returned value is passed to `WithBackend` and follows the same copying,
normalization, validation, and registration rules as any consumer-constructed
`Backend`.
The constructor should be a transparent value constructor. It should not read
environment variables, mutate global state, register the backend, validate
arguments independently, or create profiles. Consumers may inspect or modify
the returned value before registration, although documentation should direct
substantially customized configurations to the full `Backend` form.
## Identity and Registration
`BackendLocal` is a conventional ID used by the convenience constructor. It is
not pre-registered and should not become a specially reserved registry ID.
Consumers remain responsible for registering the returned backend with
`WithBackend` and naming it from profiles through `BackendID`.
This distinction preserves compatibility with consumers that may already
register their own backend using the ID `"local"`. Normal duplicate-ID rules
still apply if a consumer attempts to register more than one backend with that
ID.
Consumers that need multiple local endpoints should choose distinct IDs and
use complete `Backend` values rather than the single conventional helper ID.
## Concurrency and Queue Semantics
The constructor must preserve the existing backend concurrency contract:
- a positive concurrency limit bounds simultaneous requests and uses the
existing default queue capacity because `QueueCapacity` remains `nil`;
- a zero concurrency limit leaves the backend unconstrained; and
- a negative concurrency limit is rejected through the existing engine
configuration validation path.
The constructor should not select a hidden default concurrency limit. Local
servers vary substantially in capacity, so the consumer should make this
choice explicitly.
## Documentation
The final documentation state has two canonical surfaces:
- Public Go documentation describes the exact contract of
`BackendLocal` and `LocalBackend`, including their conventional,
non-pre-registered nature.
- The [promptkit consumer guide](../consumers/pkg-promptkit.md) includes
a task-oriented local-endpoint section that shows the three consumer paths,
explains the decision between them, and provides concise examples of the
endpoint-only and convenience-constructor forms.
The consumer guide continues to document the full `Backend` interface as
the advanced path rather than attempting to reproduce every configuration
variation through convenience APIs.
## Compatibility
This feature is additive:
- existing endpoint-only profiles continue to work unchanged;
- existing `Backend` values and `WithBackend` registrations remain the
canonical general-purpose interface;
- existing registrations using the literal ID `"local"` remain valid; and
- OpenRouter defaults and all other backend behavior remain unchanged.
No consumer is required to adopt the convenience constructor.
## Non-Goals
This work does not include:
- pre-registering or implicitly enabling a local backend;
- discovering a local endpoint, API key, or concurrency limit from environment
variables;
- adding local-backend fields to `Config`;
- selecting a default local model or creating a profile automatically;
- adding a combined backend-and-profile constructor;
- adding convenience parameters for API keys, extra request parameters, or
queue capacity;
- replacing or redesigning the backend registry;
- adding support for non-OpenAI-compatible local APIs; or
- changing backend routing, scheduling, or queue behavior.
## Target End State
After this work:
- consumers with a simple one-profile local endpoint can continue to configure
it directly on the profile;
- consumers needing a shared local endpoint and concurrency policy can express
it with one `LocalBackend` call and register the returned value normally;
- consumers with advanced or multiple-local-backend requirements have a clear
path to the complete `Backend` interface;
- all local configuration remains explicit, inspectable, and compatible with
dependency injection; and
- canonical documentation makes the simplest suitable interface easy to find
without obscuring the underlying registry model.

View File

@@ -5,6 +5,7 @@ import (
"encoding/json"
"errors"
"fmt"
"reflect"
"strings"
"sync"
"sync/atomic"
@@ -138,6 +139,42 @@ func TestUnknownProfileBackendHasProfileLoadIdentity(t *testing.T) {
}
}
func TestLocalBackendConstructsAndRegistersConventionalBackend(t *testing.T) {
const limit = 2
endpoint := "http://local.example/v1"
backend := promptkit.LocalBackend(endpoint, limit)
want := promptkit.Backend{
ID: promptkit.BackendLocal,
Endpoint: endpoint,
ConcurrencyLimit: limit,
}
if !reflect.DeepEqual(backend, want) {
t.Fatalf("LocalBackend()=%+v, want %+v", backend, want)
}
engine, err := promptkit.NewEngine(promptkit.Config{},
promptkit.WithPromptFS(contractPromptFS("prompt", "local-profile", "message"), "."),
promptkit.WithBackend(backend),
promptkit.WithProfiles(promptkit.Profile{
ID: "local-profile",
BackendID: promptkit.BackendLocal,
Model: "model",
}),
)
if err != nil {
t.Fatalf("construct engine with local backend: %v", err)
}
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "prompt"})
if err != nil {
t.Fatalf("prepare with local backend: %v", err)
}
if prepared.SelectedBackendID != promptkit.BackendLocal ||
prepared.EffectiveModelParams.Endpoint != endpoint {
t.Fatalf("unexpected local backend preparation: %+v", prepared)
}
}
func TestCustomBackendFlowsThroughProfilesOverridesAndInjectedClient(t *testing.T) {
t.Setenv("CUSTOM_LLM_KEY", "test-key")
client := &fakeLLMClient{response: &promptkit.GenerateResponse{Content: "ok"}}