Add local backend convenience constructor
This commit is contained in:
File diff suppressed because it is too large
Load Diff
163
docs/roadmap/local-backend.md
Normal file
163
docs/roadmap/local-backend.md
Normal file
@@ -0,0 +1,163 @@
|
||||
# Local Backend Convenience
|
||||
|
||||
**Status:** Accepted.
|
||||
|
||||
## Purpose
|
||||
|
||||
Make the common case of using a local OpenAI-compatible endpoint concise and
|
||||
easy to discover without introducing implicit configuration or a separate
|
||||
backend abstraction.
|
||||
|
||||
The existing `Backend` type and registry remain the canonical, fully
|
||||
configurable interface. A small convenience constructor will cover the usual
|
||||
local-network case, while improved consumer documentation will make it clear
|
||||
when an endpoint-only profile, the convenience constructor, or a complete
|
||||
`Backend` value is appropriate.
|
||||
|
||||
## Motivation
|
||||
|
||||
Consumers can already use a local endpoint by setting `Profile.Endpoint`, or
|
||||
register one as a backend with `WithBackend`. The first option is concise but
|
||||
does not provide shared backend-level concurrency control. The second supports
|
||||
the complete backend feature set but requires consumers to understand and
|
||||
populate several fields for a common configuration.
|
||||
|
||||
Most consumers adding a local backend need only:
|
||||
|
||||
- a stable backend ID;
|
||||
- an OpenAI-compatible endpoint; and
|
||||
- a concurrency limit appropriate for the local server.
|
||||
|
||||
Promptkit should provide a direct path for that case while keeping all
|
||||
configuration explicit and preserving the full registry interface for
|
||||
advanced needs.
|
||||
|
||||
## Consumer Paths
|
||||
|
||||
Documentation should present three progressively more configurable paths:
|
||||
|
||||
1. Set `Profile.Endpoint` when a profile only needs to target a local endpoint
|
||||
and does not need shared backend policy.
|
||||
2. Use the local-backend convenience constructor when profiles should share a
|
||||
named local endpoint and its concurrency limit.
|
||||
3. Construct a complete `Backend` value when the consumer needs a custom
|
||||
backend ID, authentication, extra request parameters, an explicit queue
|
||||
capacity, or multiple local backends.
|
||||
|
||||
These are complementary interfaces. The convenience constructor must return an
|
||||
ordinary `Backend`, so it does not create a second configuration model.
|
||||
|
||||
## Public Convenience API
|
||||
|
||||
The public package should expose:
|
||||
|
||||
```go
|
||||
const BackendLocal = "local"
|
||||
|
||||
func LocalBackend(endpoint string, concurrencyLimit int) Backend
|
||||
```
|
||||
|
||||
`LocalBackend` should return a `Backend` with:
|
||||
|
||||
- `ID` set to `BackendLocal`;
|
||||
- `Endpoint` set to the supplied endpoint;
|
||||
- `ConcurrencyLimit` set to the supplied limit; and
|
||||
- all other fields left at their zero values.
|
||||
|
||||
The returned value is passed to `WithBackend` and follows the same copying,
|
||||
normalization, validation, and registration rules as any consumer-constructed
|
||||
`Backend`.
|
||||
|
||||
The constructor should be a transparent value constructor. It should not read
|
||||
environment variables, mutate global state, register the backend, validate
|
||||
arguments independently, or create profiles. Consumers may inspect or modify
|
||||
the returned value before registration, although documentation should direct
|
||||
substantially customized configurations to the full `Backend` form.
|
||||
|
||||
## Identity and Registration
|
||||
|
||||
`BackendLocal` is a conventional ID used by the convenience constructor. It is
|
||||
not pre-registered and should not become a specially reserved registry ID.
|
||||
Consumers remain responsible for registering the returned backend with
|
||||
`WithBackend` and naming it from profiles through `BackendID`.
|
||||
|
||||
This distinction preserves compatibility with consumers that may already
|
||||
register their own backend using the ID `"local"`. Normal duplicate-ID rules
|
||||
still apply if a consumer attempts to register more than one backend with that
|
||||
ID.
|
||||
|
||||
Consumers that need multiple local endpoints should choose distinct IDs and
|
||||
use complete `Backend` values rather than the single conventional helper ID.
|
||||
|
||||
## Concurrency and Queue Semantics
|
||||
|
||||
The constructor must preserve the existing backend concurrency contract:
|
||||
|
||||
- a positive concurrency limit bounds simultaneous requests and uses the
|
||||
existing default queue capacity because `QueueCapacity` remains `nil`;
|
||||
- a zero concurrency limit leaves the backend unconstrained; and
|
||||
- a negative concurrency limit is rejected through the existing engine
|
||||
configuration validation path.
|
||||
|
||||
The constructor should not select a hidden default concurrency limit. Local
|
||||
servers vary substantially in capacity, so the consumer should make this
|
||||
choice explicitly.
|
||||
|
||||
## Documentation
|
||||
|
||||
The final documentation state has two canonical surfaces:
|
||||
|
||||
- Public Go documentation describes the exact contract of
|
||||
`BackendLocal` and `LocalBackend`, including their conventional,
|
||||
non-pre-registered nature.
|
||||
- The [promptkit consumer guide](../consumers/pkg-promptkit.md) includes
|
||||
a task-oriented local-endpoint section that shows the three consumer paths,
|
||||
explains the decision between them, and provides concise examples of the
|
||||
endpoint-only and convenience-constructor forms.
|
||||
|
||||
The consumer guide continues to document the full `Backend` interface as
|
||||
the advanced path rather than attempting to reproduce every configuration
|
||||
variation through convenience APIs.
|
||||
|
||||
## Compatibility
|
||||
|
||||
This feature is additive:
|
||||
|
||||
- existing endpoint-only profiles continue to work unchanged;
|
||||
- existing `Backend` values and `WithBackend` registrations remain the
|
||||
canonical general-purpose interface;
|
||||
- existing registrations using the literal ID `"local"` remain valid; and
|
||||
- OpenRouter defaults and all other backend behavior remain unchanged.
|
||||
|
||||
No consumer is required to adopt the convenience constructor.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This work does not include:
|
||||
|
||||
- pre-registering or implicitly enabling a local backend;
|
||||
- discovering a local endpoint, API key, or concurrency limit from environment
|
||||
variables;
|
||||
- adding local-backend fields to `Config`;
|
||||
- selecting a default local model or creating a profile automatically;
|
||||
- adding a combined backend-and-profile constructor;
|
||||
- adding convenience parameters for API keys, extra request parameters, or
|
||||
queue capacity;
|
||||
- replacing or redesigning the backend registry;
|
||||
- adding support for non-OpenAI-compatible local APIs; or
|
||||
- changing backend routing, scheduling, or queue behavior.
|
||||
|
||||
## Target End State
|
||||
|
||||
After this work:
|
||||
|
||||
- consumers with a simple one-profile local endpoint can continue to configure
|
||||
it directly on the profile;
|
||||
- consumers needing a shared local endpoint and concurrency policy can express
|
||||
it with one `LocalBackend` call and register the returned value normally;
|
||||
- consumers with advanced or multiple-local-backend requirements have a clear
|
||||
path to the complete `Backend` interface;
|
||||
- all local configuration remains explicit, inspectable, and compatible with
|
||||
dependency injection; and
|
||||
- canonical documentation makes the simplest suitable interface easy to find
|
||||
without obscuring the underlying registry model.
|
||||
Reference in New Issue
Block a user