165 lines
7.5 KiB
Markdown
165 lines
7.5 KiB
Markdown
# Rakestrawhome Built-In Inference Backend
|
|
|
|
## Purpose
|
|
|
|
Promptkit should provide the maintainer's Rakestrawhome inference service as a
|
|
second built-in OpenAI-compatible backend, alongside OpenRouter. This removes
|
|
the repeated backend and profile registration currently required in every
|
|
downstream application that uses the service.
|
|
|
|
The feature is intentionally a built-in registration rather than another
|
|
convenience constructor. Engines should receive the same immutable backend
|
|
definition, credential convention, concurrency policy, and initial model
|
|
profile without consumer assembly code.
|
|
|
|
## Target End State
|
|
|
|
Every newly constructed engine contains two reserved built-in backend IDs:
|
|
|
|
- `openrouter`, with its existing definition and behavior unchanged; and
|
|
- `rakestrawhome`, configured with the OpenAI-compatible base endpoint
|
|
`https://inference.ai.rakestrawhome.com/v1`, API-key environment variable
|
|
`RAKESTRAWHOME_INFERENCE_API_KEY`, and a concurrency limit of four active
|
|
model-generation calls per engine.
|
|
|
|
The Rakestrawhome backend uses Promptkit's existing default bounded waiting
|
|
capacity of 1024. The capacity manager therefore permits no more than four
|
|
simultaneous generation calls for this backend while retaining the existing
|
|
high, bounded backlog policy. Ordinary backend, profile, runtime endpoint,
|
|
credential, and request-setting precedence remains unchanged.
|
|
|
|
Promptkit also embeds this profile:
|
|
|
|
```yaml
|
|
id: rakestrawhome-gemma-4-31b
|
|
backend: rakestrawhome
|
|
model: google/gemma-4-31b-it
|
|
```
|
|
|
|
The profile deliberately omits optional generation controls so the inference
|
|
service and model apply their native defaults. It does not repeat the backend
|
|
endpoint or credential environment-variable name. The existing
|
|
`gemma-4-31b` OpenRouter profile and its current model spelling and settings
|
|
remain unchanged.
|
|
|
|
Consumers can select `rakestrawhome-gemma-4-31b` anywhere an ordinary profile
|
|
ID is accepted. Higher-precedence in-memory, configured, or application
|
|
fallback profile sources may continue to replace the embedded profile by ID.
|
|
Consumers can refer to the built-in backend through a new exported constant,
|
|
`BackendRakestrawHome`, while the exact backend definition remains internal.
|
|
|
|
## Backend Registry End State
|
|
|
|
Built-in registry construction owns a small maintained collection rather than
|
|
an inline OpenRouter special case. Both built-ins enter the same normalization,
|
|
validation, immutable storage, defensive-copy, and capacity-policy paths.
|
|
|
|
The internal backend package owns stable IDs and connection and capacity
|
|
policy. The root facade exposes only the stable Rakestrawhome ID through
|
|
`BackendRakestrawHome`. Public uniqueness and reservation rules apply equally
|
|
to every built-in.
|
|
|
|
Consumer additions must remain unable to replace either reserved built-in ID.
|
|
Registrations under all other valid IDs retain their current behavior. Engine
|
|
construction must not contact the Rakestrawhome endpoint, read the API-key
|
|
environment variable, or require credentials merely because the backend is
|
|
registered. Credential resolution occurs only when a selected execution target
|
|
uses it, under the existing precedence and error contracts.
|
|
|
|
The built-in OpenAI-compatible client will append `chat/completions` to the
|
|
registered base path, producing
|
|
`https://inference.ai.rakestrawhome.com/v1/chat/completions`. No provider-
|
|
specific transport implementation is required.
|
|
|
|
## Built-In Profile Catalog End State
|
|
|
|
The Rakestrawhome Gemma profile is an embedded YAML asset in the existing
|
|
built-in profile repository. No additional repository layer or provider-
|
|
specific profile loader exists.
|
|
|
|
Catalog validation permits the maintained built-in backend IDs and establishes
|
|
that:
|
|
|
|
- every embedded profile has a unique, nonblank ID;
|
|
- every embedded profile selects an approved built-in backend;
|
|
- no embedded profile contains a raw API key or repeats an endpoint or API-key
|
|
environment-variable name owned by its backend; and
|
|
- the new profile resolves to the exact intended backend and model through the
|
|
ordinary repository and engine assembly.
|
|
|
|
Retain the existing source precedence: consumer profile sources may override
|
|
the new built-in profile, but a malformed authoritative override remains an
|
|
error rather than silently falling through.
|
|
|
|
## Public Compatibility And Migration
|
|
|
|
This feature adds one public constant and one embedded profile ID. It does not
|
|
remove or change an existing public declaration or profile.
|
|
|
|
The backend ID `rakestrawhome` becomes reserved. A downstream application that
|
|
currently registers that ID manually must remove its `WithBackend`
|
|
registration when adopting this feature; otherwise construction will correctly
|
|
report a duplicate built-in ID. It may also remove any equivalent manually
|
|
supplied profile and select `rakestrawhome-gemma-4-31b` directly. A consumer
|
|
profile with the same profile ID remains a supported higher-precedence
|
|
override rather than a registry collision.
|
|
|
|
Direct API-key and runtime endpoint overrides continue to work. Promptkit must
|
|
never embed, inspect, log, or document an actual credential value; only the
|
|
environment-variable name belongs in the built-in definition.
|
|
|
|
## Documentation End State
|
|
|
|
Canonical current-state documentation:
|
|
|
|
- describes the backend registry as containing multiple built-ins in the
|
|
architecture policy and internal component overview;
|
|
- describes both built-in backend definitions and the mixed-backend profile
|
|
catalog in the framework format reference and internal source overview;
|
|
- lists `rakestrawhome-gemma-4-31b` in the built-in profile catalog;
|
|
- documents `BackendRakestrawHome`, the removal of redundant downstream
|
|
registration, the credential variable, and the four-call concurrency policy
|
|
in public GoDoc and consumer guidance; and
|
|
- retains the OpenAI-compatible integration document as the canonical owner of
|
|
endpoint composition and outbound wire behavior.
|
|
|
|
## Acceptance Criteria
|
|
|
|
The completed feature is protected at the narrowest stable owners:
|
|
|
|
- registry tests should cover both exact built-in definitions, default queue
|
|
normalization, capacity-policy publication, lookup ownership, and duplicate
|
|
rejection for both reserved IDs;
|
|
- built-in profile tests should validate the generalized catalog invariants and
|
|
exact Rakestrawhome profile;
|
|
- a representative root integration test proves that selecting the new
|
|
profile resolves the expected backend ID, endpoint, model, and credential
|
|
environment-variable name without making a live network request or
|
|
requiring a real credential;
|
|
- the registry's exact capacity-policy tests and the existing generic engine
|
|
capacity tests together protect the engine-local four-call limit without a
|
|
duplicate provider-specific concurrency matrix; and
|
|
- existing OpenRouter, custom-backend, profile-precedence, endpoint-override,
|
|
credential, capacity, and injected-client tests should remain green without
|
|
duplicating their full behavioral matrices for the new built-in.
|
|
|
|
## Non-Goals
|
|
|
|
This feature does not add:
|
|
|
|
- provider discovery, health checking, model enumeration, or live integration
|
|
tests;
|
|
- automatic fallback, retry, or routing between OpenRouter and Rakestrawhome;
|
|
- global mutable registry state or consumer replacement of built-in IDs;
|
|
- a second model-client protocol or non-OpenAI-compatible transport;
|
|
- provider-specific generation defaults that have not been deliberately
|
|
selected; or
|
|
- any embedded API-key value.
|
|
|
|
## Open Questions
|
|
|
|
None. The backend ID, public constant direction, endpoint base path,
|
|
credential environment-variable name, concurrency and queue policy, profile
|
|
ID, model name, optional-setting policy, compatibility behavior, and scope
|
|
boundaries are fixed by this roadmap.
|