Rakestrawhome Built-In Inference Backend

Purpose

Promptkit should provide the maintainer's Rakestrawhome inference service as a second built-in OpenAI-compatible backend, alongside OpenRouter. This removes the repeated backend and profile registration currently required in every downstream application that uses the service.

The feature is intentionally a built-in registration rather than another convenience constructor. Engines should receive the same immutable backend definition, credential convention, concurrency policy, and initial model profile without consumer assembly code.

Target End State

Every newly constructed engine contains two reserved built-in backend IDs:

openrouter, with its existing definition and behavior unchanged; and
rakestrawhome, configured with the OpenAI-compatible base endpoint https://inference.ai.rakestrawhome.com/v1, API-key environment variable RAKESTRAWHOME_INFERENCE_API_KEY, and a concurrency limit of four active model-generation calls per engine.

The Rakestrawhome backend uses Promptkit's existing default bounded waiting capacity of 1024. The capacity manager therefore permits no more than four simultaneous generation calls for this backend while retaining the existing high, bounded backlog policy. Ordinary backend, profile, runtime endpoint, credential, and request-setting precedence remains unchanged.

Promptkit also embeds this profile:

id: rakestrawhome-gemma-4-31b
backend: rakestrawhome
model: google/gemma-4-31b-it

The profile deliberately omits optional generation controls so the inference service and model apply their native defaults. It does not repeat the backend endpoint or credential environment-variable name. The existing gemma-4-31b OpenRouter profile and its current model spelling and settings remain unchanged.

Consumers can select rakestrawhome-gemma-4-31b anywhere an ordinary profile ID is accepted. Higher-precedence in-memory, configured, or application fallback profile sources may continue to replace the embedded profile by ID. Consumers can refer to the built-in backend through a new exported constant, BackendRakestrawHome, while the exact backend definition remains internal.

Backend Registry End State

Built-in registry construction owns a small maintained collection rather than an inline OpenRouter special case. Both built-ins enter the same normalization, validation, immutable storage, defensive-copy, and capacity-policy paths.

The internal backend package owns stable IDs and connection and capacity policy. The root facade exposes only the stable Rakestrawhome ID through BackendRakestrawHome. Public uniqueness and reservation rules apply equally to every built-in.

Consumer additions must remain unable to replace either reserved built-in ID. Registrations under all other valid IDs retain their current behavior. Engine construction must not contact the Rakestrawhome endpoint, read the API-key environment variable, or require credentials merely because the backend is registered. Credential resolution occurs only when a selected execution target uses it, under the existing precedence and error contracts.

The built-in OpenAI-compatible client will append chat/completions to the registered base path, producing https://inference.ai.rakestrawhome.com/v1/chat/completions. No provider- specific transport implementation is required.

Built-In Profile Catalog End State

The Rakestrawhome Gemma profile is an embedded YAML asset in the existing built-in profile repository. No additional repository layer or provider- specific profile loader exists.

Catalog validation permits the maintained built-in backend IDs and establishes that:

every embedded profile has a unique, nonblank ID;
every embedded profile selects an approved built-in backend;
no embedded profile contains a raw API key or repeats an endpoint or API-key environment-variable name owned by its backend; and
the new profile resolves to the exact intended backend and model through the ordinary repository and engine assembly.

Retain the existing source precedence: consumer profile sources may override the new built-in profile, but a malformed authoritative override remains an error rather than silently falling through.

Public Compatibility And Migration

This feature adds one public constant and one embedded profile ID. It does not remove or change an existing public declaration or profile.

The backend ID rakestrawhome becomes reserved. A downstream application that currently registers that ID manually must remove its WithBackend registration when adopting this feature; otherwise construction will correctly report a duplicate built-in ID. It may also remove any equivalent manually supplied profile and select rakestrawhome-gemma-4-31b directly. A consumer profile with the same profile ID remains a supported higher-precedence override rather than a registry collision.

Direct API-key and runtime endpoint overrides continue to work. Promptkit must never embed, inspect, log, or document an actual credential value; only the environment-variable name belongs in the built-in definition.

Documentation End State

Canonical current-state documentation:

describes the backend registry as containing multiple built-ins in the architecture policy and internal component overview;
describes both built-in backend definitions and the mixed-backend profile catalog in the framework format reference and internal source overview;
lists rakestrawhome-gemma-4-31b in the built-in profile catalog;
documents BackendRakestrawHome, the removal of redundant downstream registration, the credential variable, and the four-call concurrency policy in public GoDoc and consumer guidance; and
retains the OpenAI-compatible integration document as the canonical owner of endpoint composition and outbound wire behavior.

Acceptance Criteria

The completed feature is protected at the narrowest stable owners:

registry tests should cover both exact built-in definitions, default queue normalization, capacity-policy publication, lookup ownership, and duplicate rejection for both reserved IDs;
built-in profile tests should validate the generalized catalog invariants and exact Rakestrawhome profile;
a representative root integration test proves that selecting the new profile resolves the expected backend ID, endpoint, model, and credential environment-variable name without making a live network request or requiring a real credential;
the registry's exact capacity-policy tests and the existing generic engine capacity tests together protect the engine-local four-call limit without a duplicate provider-specific concurrency matrix; and
existing OpenRouter, custom-backend, profile-precedence, endpoint-override, credential, capacity, and injected-client tests should remain green without duplicating their full behavioral matrices for the new built-in.

Non-Goals

This feature does not add:

provider discovery, health checking, model enumeration, or live integration tests;
automatic fallback, retry, or routing between OpenRouter and Rakestrawhome;
global mutable registry state or consumer replacement of built-in IDs;
a second model-client protocol or non-OpenAI-compatible transport;
provider-specific generation defaults that have not been deliberately selected; or
any embedded API-key value.

Open Questions

None. The backend ID, public constant direction, endpoint base path, credential environment-variable name, concurrency and queue policy, profile ID, model name, optional-setting policy, compatibility behavior, and scope boundaries are fixed by this roadmap.

7.5 KiB Raw Blame History