164 lines
6.4 KiB
Markdown
164 lines
6.4 KiB
Markdown
# Local Backend Convenience
|
|
|
|
**Status:** Accepted.
|
|
|
|
## Purpose
|
|
|
|
Make the common case of using a local OpenAI-compatible endpoint concise and
|
|
easy to discover without introducing implicit configuration or a separate
|
|
backend abstraction.
|
|
|
|
The existing `Backend` type and registry remain the canonical, fully
|
|
configurable interface. A small convenience constructor will cover the usual
|
|
local-network case, while improved consumer documentation will make it clear
|
|
when an endpoint-only profile, the convenience constructor, or a complete
|
|
`Backend` value is appropriate.
|
|
|
|
## Motivation
|
|
|
|
Consumers can already use a local endpoint by setting `Profile.Endpoint`, or
|
|
register one as a backend with `WithBackend`. The first option is concise but
|
|
does not provide shared backend-level concurrency control. The second supports
|
|
the complete backend feature set but requires consumers to understand and
|
|
populate several fields for a common configuration.
|
|
|
|
Most consumers adding a local backend need only:
|
|
|
|
- a stable backend ID;
|
|
- an OpenAI-compatible endpoint; and
|
|
- a concurrency limit appropriate for the local server.
|
|
|
|
Promptkit should provide a direct path for that case while keeping all
|
|
configuration explicit and preserving the full registry interface for
|
|
advanced needs.
|
|
|
|
## Consumer Paths
|
|
|
|
Documentation should present three progressively more configurable paths:
|
|
|
|
1. Set `Profile.Endpoint` when a profile only needs to target a local endpoint
|
|
and does not need shared backend policy.
|
|
2. Use the local-backend convenience constructor when profiles should share a
|
|
named local endpoint and its concurrency limit.
|
|
3. Construct a complete `Backend` value when the consumer needs a custom
|
|
backend ID, authentication, extra request parameters, an explicit queue
|
|
capacity, or multiple local backends.
|
|
|
|
These are complementary interfaces. The convenience constructor must return an
|
|
ordinary `Backend`, so it does not create a second configuration model.
|
|
|
|
## Public Convenience API
|
|
|
|
The public package should expose:
|
|
|
|
```go
|
|
const BackendLocal = "local"
|
|
|
|
func LocalBackend(endpoint string, concurrencyLimit int) Backend
|
|
```
|
|
|
|
`LocalBackend` should return a `Backend` with:
|
|
|
|
- `ID` set to `BackendLocal`;
|
|
- `Endpoint` set to the supplied endpoint;
|
|
- `ConcurrencyLimit` set to the supplied limit; and
|
|
- all other fields left at their zero values.
|
|
|
|
The returned value is passed to `WithBackend` and follows the same copying,
|
|
normalization, validation, and registration rules as any consumer-constructed
|
|
`Backend`.
|
|
|
|
The constructor should be a transparent value constructor. It should not read
|
|
environment variables, mutate global state, register the backend, validate
|
|
arguments independently, or create profiles. Consumers may inspect or modify
|
|
the returned value before registration, although documentation should direct
|
|
substantially customized configurations to the full `Backend` form.
|
|
|
|
## Identity and Registration
|
|
|
|
`BackendLocal` is a conventional ID used by the convenience constructor. It is
|
|
not pre-registered and should not become a specially reserved registry ID.
|
|
Consumers remain responsible for registering the returned backend with
|
|
`WithBackend` and naming it from profiles through `BackendID`.
|
|
|
|
This distinction preserves compatibility with consumers that may already
|
|
register their own backend using the ID `"local"`. Normal duplicate-ID rules
|
|
still apply if a consumer attempts to register more than one backend with that
|
|
ID.
|
|
|
|
Consumers that need multiple local endpoints should choose distinct IDs and
|
|
use complete `Backend` values rather than the single conventional helper ID.
|
|
|
|
## Concurrency and Queue Semantics
|
|
|
|
The constructor must preserve the existing backend concurrency contract:
|
|
|
|
- a positive concurrency limit bounds simultaneous requests and uses the
|
|
existing default queue capacity because `QueueCapacity` remains `nil`;
|
|
- a zero concurrency limit leaves the backend unconstrained; and
|
|
- a negative concurrency limit is rejected through the existing engine
|
|
configuration validation path.
|
|
|
|
The constructor should not select a hidden default concurrency limit. Local
|
|
servers vary substantially in capacity, so the consumer should make this
|
|
choice explicitly.
|
|
|
|
## Documentation
|
|
|
|
The final documentation state has two canonical surfaces:
|
|
|
|
- Public Go documentation describes the exact contract of
|
|
`BackendLocal` and `LocalBackend`, including their conventional,
|
|
non-pre-registered nature.
|
|
- The [promptkit consumer guide](../consumers/pkg-promptkit.md) includes
|
|
a task-oriented local-endpoint section that shows the three consumer paths,
|
|
explains the decision between them, and provides concise examples of the
|
|
endpoint-only and convenience-constructor forms.
|
|
|
|
The consumer guide continues to document the full `Backend` interface as
|
|
the advanced path rather than attempting to reproduce every configuration
|
|
variation through convenience APIs.
|
|
|
|
## Compatibility
|
|
|
|
This feature is additive:
|
|
|
|
- existing endpoint-only profiles continue to work unchanged;
|
|
- existing `Backend` values and `WithBackend` registrations remain the
|
|
canonical general-purpose interface;
|
|
- existing registrations using the literal ID `"local"` remain valid; and
|
|
- OpenRouter defaults and all other backend behavior remain unchanged.
|
|
|
|
No consumer is required to adopt the convenience constructor.
|
|
|
|
## Non-Goals
|
|
|
|
This work does not include:
|
|
|
|
- pre-registering or implicitly enabling a local backend;
|
|
- discovering a local endpoint, API key, or concurrency limit from environment
|
|
variables;
|
|
- adding local-backend fields to `Config`;
|
|
- selecting a default local model or creating a profile automatically;
|
|
- adding a combined backend-and-profile constructor;
|
|
- adding convenience parameters for API keys, extra request parameters, or
|
|
queue capacity;
|
|
- replacing or redesigning the backend registry;
|
|
- adding support for non-OpenAI-compatible local APIs; or
|
|
- changing backend routing, scheduling, or queue behavior.
|
|
|
|
## Target End State
|
|
|
|
After this work:
|
|
|
|
- consumers with a simple one-profile local endpoint can continue to configure
|
|
it directly on the profile;
|
|
- consumers needing a shared local endpoint and concurrency policy can express
|
|
it with one `LocalBackend` call and register the returned value normally;
|
|
- consumers with advanced or multiple-local-backend requirements have a clear
|
|
path to the complete `Backend` interface;
|
|
- all local configuration remains explicit, inspectable, and compatible with
|
|
dependency injection; and
|
|
- canonical documentation makes the simplest suitable interface easy to find
|
|
without obscuring the underlying registry model.
|