# Extensible LLM Backend Registry ## Purpose This roadmap defines the scope and target end state for formal OpenAI-compatible backend support in Promptkit. It establishes the behavioral boundary and policy choices for the work. This document is planning material, not a description of current behavior. Current public contracts remain owned by Go declarations and GoDoc, framework file formats by the [format reference](../formats.md), and outbound HTTP behavior by the [OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md). ## Motivation Execution profiles currently carry backend connection and authentication details alongside model and generation settings. This repeats values such as the OpenRouter endpoint and `OPENROUTER_API_KEY` across profiles, leaves the built-in catalog implicitly tied to one backend, and gives consumers no explicit extension point for naming other OpenAI-compatible services. Promptkit should distinguish: - a **backend**, which identifies reusable connection, authentication, and limited backend-wide request defaults; and - a **profile**, which selects a model and its reusable generation settings. That distinction should let consumers configure OpenRouter, OpenAI, local-network services, or other OpenAI-compatible deployments without duplicating backend defaults in every profile or replacing Promptkit's model client. ## Scope The work introduces an engine-scoped registry of OpenAI-compatible backend definitions. Each backend definition will provide: - a stable, non-empty backend ID; - a default OpenAI-compatible base endpoint; - an optional environment-variable name for its API key; and - limited JSON-compatible request defaults when they are genuinely backend-wide rather than model-specific. Promptkit will provide a small built-in backend catalog, initially containing OpenRouter. Downstream consumers will be able to register additional backend IDs during engine construction. Registration will be explicit and local to an engine; it will not mutate package-global state. Profiles will be able to name a backend. A profile that selects a backend may override its default endpoint and other supported defaults while continuing to own its model and generation settings. Existing endpoint-based custom profiles will remain supported without requiring a backend ID. The built-in model profile catalog will use the built-in OpenRouter backend instead of repeating OpenRouter connection and credential defaults in every profile. ## Resolution And Precedence Effective execution settings will resolve in this order: 1. application-neutral framework defaults; 2. selected backend defaults, when a backend is named; 3. selected profile values; and 4. explicit per-run overrides. A profile endpoint will override its selected backend's endpoint. A per-run endpoint override will continue to take precedence over both. Credential selection will preserve direct request credentials as the highest precedence. An explicit per-run environment-variable override will take precedence over profile credential configuration, which will take precedence over the backend's default environment-variable name. Backend definitions and profiles will contain credential-source metadata only, never resolved secret values. Backend request defaults, profile request values, and per-run request overrides will follow one deterministic replacement rule. Resolution must not introduce implicit deep merging whose result depends on map iteration or incidental representation. ## Registration And Validation Backend registration will be deterministic and validated during engine construction. - Built-in backend IDs are reserved and cannot be replaced by consumers. - Consumer registrations may add only new IDs. - Duplicate consumer IDs are invalid, including duplicates introduced through repeated configuration. - Backend IDs, endpoints, credential-source metadata, and request defaults must be validated before an engine is returned. - Selecting an unknown backend is an error associated with the profile or request boundary that selected it. - A profile without a backend must continue to provide the connection details required by the current endpoint-based path. - A backend may omit credential requirements so unauthenticated local-network services remain supported. Failures will retain error identities appropriate to the existing engine configuration, profile-loading, and per-run validation boundaries. ## Public And Extension Boundaries The effective backend ID will be observable in prepared and completed run metadata and will be available to injected model clients. This gives consumers and extensions a stable routing identity without requiring them to infer a backend from an endpoint URL. The existing public model-client injection boundary will remain supported. Because the initial registry supports only OpenAI-compatible backends, Promptkit does not need backend-specific transport factories or multiple protocol implementations in this scope. The built-in client will continue to send the resolved execution target to the selected OpenAI-compatible endpoint. Backend definitions are configuration values, not live service objects. They will not own mutable connections, credentials, health state, or process lifecycle. ## Compatibility The target end state preserves the existing ways consumers configure and run Promptkit: - Profiles with explicit endpoints and no backend ID continue to work. - Profile endpoint overrides remain supported. - Direct request API keys and environment-variable overrides retain their precedence. - Consumers may continue injecting a custom model client. - Model IDs and generation settings remain profile concerns. - Prompt selection, rendering, validation, artifacts, and synchronous `Prepare` and `Run` behavior remain unchanged except for exposing the resolved backend identity. ## Non-Goals This scope does not include: - non-OpenAI-compatible protocols or provider-specific SDKs; - backend-specific concurrency limits or admission queues; - asynchronous task submission or durable jobs; - model discovery, live model catalogs, or provider capability probing; - retries, rate limiting, failover, load balancing, or health checks; - automatic backend selection based on model names or endpoint inspection; - global mutable registration; - consumer replacement of built-in backend IDs; - credential discovery beyond configured direct values and environment variables; or - provider-specific application, deployment, or security policy. The dependent concurrency and bounded-queue work remains a separate future unit after backend identity and resolution are stable. ## Target End State This roadmap reaches its target end state when: - Promptkit has an immutable, engine-scoped registry of validated OpenAI-compatible backend definitions; - OpenRouter is available as a built-in backend with its endpoint and credential environment-variable default; - consumers can register additional unique backend IDs without modifying Promptkit; - profiles can select a backend and optionally override its endpoint; - existing endpoint-only profiles remain valid; - backend, profile, and per-run values resolve through documented, deterministic precedence; - secret values remain outside backend and profile definitions; - the effective backend identity is available in preparation, execution, and injected-client values; - the built-in model profiles use the OpenRouter backend rather than duplicate its connection defaults; - the built-in OpenAI-compatible client and injected clients continue to work through the existing generation boundary; and - current-state GoDoc, format, consumer, integration, architecture, and internal documentation describe the implemented behavior without relying on this roadmap.