promptkit/docs/roadmap/local-backend.md

Local Backend Convenience

Status: Accepted.

Purpose

Make the common case of using a local OpenAI-compatible endpoint concise and easy to discover without introducing implicit configuration or a separate backend abstraction.

The existing Backend type and registry remain the canonical, fully configurable interface. A small convenience constructor will cover the usual local-network case, while improved consumer documentation will make it clear when an endpoint-only profile, the convenience constructor, or a complete Backend value is appropriate.

Motivation

Consumers can already use a local endpoint by setting Profile.Endpoint, or register one as a backend with WithBackend. The first option is concise but does not provide shared backend-level concurrency control. The second supports the complete backend feature set but requires consumers to understand and populate several fields for a common configuration.

Most consumers adding a local backend need only:

• a stable backend ID;
• an OpenAI-compatible endpoint; and
• a concurrency limit appropriate for the local server.

Promptkit should provide a direct path for that case while keeping all configuration explicit and preserving the full registry interface for advanced needs.

Consumer Paths

Documentation should present three progressively more configurable paths:

1. Set Profile.Endpoint when a profile only needs to target a local endpoint and does not need shared backend policy.
2. Use the local-backend convenience constructor when profiles should share a named local endpoint and its concurrency limit.
3. Construct a complete Backend value when the consumer needs a custom backend ID, authentication, extra request parameters, an explicit queue capacity, or multiple local backends.

These are complementary interfaces. The convenience constructor must return an ordinary Backend, so it does not create a second configuration model.

Public Convenience API

The public package should expose:

const BackendLocal = "local"

func LocalBackend(endpoint string, concurrencyLimit int) Backend

LocalBackend should return a Backend with:

• ID set to BackendLocal;
• Endpoint set to the supplied endpoint;
• ConcurrencyLimit set to the supplied limit; and
• all other fields left at their zero values.

The returned value is passed to WithBackend and follows the same copying, normalization, validation, and registration rules as any consumer-constructed Backend.

The constructor should be a transparent value constructor. It should not read environment variables, mutate global state, register the backend, validate arguments independently, or create profiles. Consumers may inspect or modify the returned value before registration, although documentation should direct substantially customized configurations to the full Backend form.

Identity and Registration

BackendLocal is a conventional ID used by the convenience constructor. It is not pre-registered and should not become a specially reserved registry ID. Consumers remain responsible for registering the returned backend with WithBackend and naming it from profiles through BackendID.

This distinction preserves compatibility with consumers that may already register their own backend using the ID "local". Normal duplicate-ID rules still apply if a consumer attempts to register more than one backend with that ID.

Consumers that need multiple local endpoints should choose distinct IDs and use complete Backend values rather than the single conventional helper ID.

Concurrency and Queue Semantics

The constructor must preserve the existing backend concurrency contract:

• a positive concurrency limit bounds simultaneous requests and uses the existing default queue capacity because QueueCapacity remains nil;
• a zero concurrency limit leaves the backend unconstrained; and
• a negative concurrency limit is rejected through the existing engine configuration validation path.

The constructor should not select a hidden default concurrency limit. Local servers vary substantially in capacity, so the consumer should make this choice explicitly.

Documentation

The final documentation state has two canonical surfaces:

• Public Go documentation describes the exact contract of BackendLocal and LocalBackend, including their conventional, non-pre-registered nature.
• The promptkit consumer guide includes a task-oriented local-endpoint section that shows the three consumer paths, explains the decision between them, and provides concise examples of the endpoint-only and convenience-constructor forms.

The consumer guide continues to document the full Backend interface as the advanced path rather than attempting to reproduce every configuration variation through convenience APIs.

Compatibility

This feature is additive:

• existing endpoint-only profiles continue to work unchanged;
• existing Backend values and WithBackend registrations remain the canonical general-purpose interface;
• existing registrations using the literal ID "local" remain valid; and
• OpenRouter defaults and all other backend behavior remain unchanged.

No consumer is required to adopt the convenience constructor.

Non-Goals

This work does not include:

• pre-registering or implicitly enabling a local backend;
• discovering a local endpoint, API key, or concurrency limit from environment variables;
• adding local-backend fields to Config;
• selecting a default local model or creating a profile automatically;
• adding a combined backend-and-profile constructor;
• adding convenience parameters for API keys, extra request parameters, or queue capacity;
• replacing or redesigning the backend registry;
• adding support for non-OpenAI-compatible local APIs; or
• changing backend routing, scheduling, or queue behavior.

Target End State

After this work:

• consumers with a simple one-profile local endpoint can continue to configure it directly on the profile;
• consumers needing a shared local endpoint and concurrency policy can express it with one LocalBackend call and register the returned value normally;
• consumers with advanced or multiple-local-backend requirements have a clear path to the complete Backend interface;
• all local configuration remains explicit, inspectable, and compatible with dependency injection; and
• canonical documentation makes the simplest suitable interface easy to find without obscuring the underlying registry model.