Plan the built-in Rakestrawhome backend
This commit is contained in:
164
docs/roadmap/rakestrawhome-backend.md
Normal file
164
docs/roadmap/rakestrawhome-backend.md
Normal file
@@ -0,0 +1,164 @@
|
||||
# Rakestrawhome Built-In Inference Backend
|
||||
|
||||
## Purpose
|
||||
|
||||
Promptkit should provide the maintainer's Rakestrawhome inference service as a
|
||||
second built-in OpenAI-compatible backend, alongside OpenRouter. This removes
|
||||
the repeated backend and profile registration currently required in every
|
||||
downstream application that uses the service.
|
||||
|
||||
The feature is intentionally a built-in registration rather than another
|
||||
convenience constructor. Engines should receive the same immutable backend
|
||||
definition, credential convention, concurrency policy, and initial model
|
||||
profile without consumer assembly code.
|
||||
|
||||
## Target End State
|
||||
|
||||
Every newly constructed engine contains two reserved built-in backend IDs:
|
||||
|
||||
- `openrouter`, with its existing definition and behavior unchanged; and
|
||||
- `rakestrawhome`, configured with the OpenAI-compatible base endpoint
|
||||
`https://inference.ai.rakestrawhome.com/v1`, API-key environment variable
|
||||
`RAKESTRAWHOME_INFERENCE_API_KEY`, and a concurrency limit of four active
|
||||
model-generation calls per engine.
|
||||
|
||||
The Rakestrawhome backend uses Promptkit's existing default bounded waiting
|
||||
capacity of 1024. The capacity manager therefore permits no more than four
|
||||
simultaneous generation calls for this backend while retaining the existing
|
||||
high, bounded backlog policy. Ordinary backend, profile, runtime endpoint,
|
||||
credential, and request-setting precedence remains unchanged.
|
||||
|
||||
Promptkit also embeds this profile:
|
||||
|
||||
```yaml
|
||||
id: rakestrawhome-gemma-4-31b
|
||||
backend: rakestrawhome
|
||||
model: google/gemma-4-31b-it
|
||||
```
|
||||
|
||||
The profile deliberately omits optional generation controls so the inference
|
||||
service and model apply their native defaults. It does not repeat the backend
|
||||
endpoint or credential environment-variable name. The existing
|
||||
`gemma-4-31b` OpenRouter profile and its current model spelling and settings
|
||||
remain unchanged.
|
||||
|
||||
Consumers can select `rakestrawhome-gemma-4-31b` anywhere an ordinary profile
|
||||
ID is accepted. Higher-precedence in-memory, configured, or application
|
||||
fallback profile sources may continue to replace the embedded profile by ID.
|
||||
Consumers can refer to the built-in backend through a new exported constant,
|
||||
`BackendRakestrawHome`, while the exact backend definition remains internal.
|
||||
|
||||
## Backend Registry End State
|
||||
|
||||
Built-in registry construction owns a small maintained collection rather than
|
||||
an inline OpenRouter special case. Both built-ins enter the same normalization,
|
||||
validation, immutable storage, defensive-copy, and capacity-policy paths.
|
||||
|
||||
The internal backend package owns stable IDs and connection and capacity
|
||||
policy. The root facade exposes only the stable Rakestrawhome ID through
|
||||
`BackendRakestrawHome`. Public uniqueness and reservation rules apply equally
|
||||
to every built-in.
|
||||
|
||||
Consumer additions must remain unable to replace either reserved built-in ID.
|
||||
Registrations under all other valid IDs retain their current behavior. Engine
|
||||
construction must not contact the Rakestrawhome endpoint, read the API-key
|
||||
environment variable, or require credentials merely because the backend is
|
||||
registered. Credential resolution occurs only when a selected execution target
|
||||
uses it, under the existing precedence and error contracts.
|
||||
|
||||
The built-in OpenAI-compatible client will append `chat/completions` to the
|
||||
registered base path, producing
|
||||
`https://inference.ai.rakestrawhome.com/v1/chat/completions`. No provider-
|
||||
specific transport implementation is required.
|
||||
|
||||
## Built-In Profile Catalog End State
|
||||
|
||||
The Rakestrawhome Gemma profile is an embedded YAML asset in the existing
|
||||
built-in profile repository. No additional repository layer or provider-
|
||||
specific profile loader exists.
|
||||
|
||||
Catalog validation permits the maintained built-in backend IDs and establishes
|
||||
that:
|
||||
|
||||
- every embedded profile has a unique, nonblank ID;
|
||||
- every embedded profile selects an approved built-in backend;
|
||||
- no embedded profile contains a raw API key or repeats an endpoint or API-key
|
||||
environment-variable name owned by its backend; and
|
||||
- the new profile resolves to the exact intended backend and model through the
|
||||
ordinary repository and engine assembly.
|
||||
|
||||
Retain the existing source precedence: consumer profile sources may override
|
||||
the new built-in profile, but a malformed authoritative override remains an
|
||||
error rather than silently falling through.
|
||||
|
||||
## Public Compatibility And Migration
|
||||
|
||||
This feature adds one public constant and one embedded profile ID. It does not
|
||||
remove or change an existing public declaration or profile.
|
||||
|
||||
The backend ID `rakestrawhome` becomes reserved. A downstream application that
|
||||
currently registers that ID manually must remove its `WithBackend`
|
||||
registration when adopting this feature; otherwise construction will correctly
|
||||
report a duplicate built-in ID. It may also remove any equivalent manually
|
||||
supplied profile and select `rakestrawhome-gemma-4-31b` directly. A consumer
|
||||
profile with the same profile ID remains a supported higher-precedence
|
||||
override rather than a registry collision.
|
||||
|
||||
Direct API-key and runtime endpoint overrides continue to work. Promptkit must
|
||||
never embed, inspect, log, or document an actual credential value; only the
|
||||
environment-variable name belongs in the built-in definition.
|
||||
|
||||
## Documentation End State
|
||||
|
||||
Canonical current-state documentation:
|
||||
|
||||
- describes the backend registry as containing multiple built-ins in the
|
||||
architecture policy and internal component overview;
|
||||
- describes both built-in backend definitions and the mixed-backend profile
|
||||
catalog in the framework format reference and internal source overview;
|
||||
- lists `rakestrawhome-gemma-4-31b` in the built-in profile catalog;
|
||||
- documents `BackendRakestrawHome`, the removal of redundant downstream
|
||||
registration, the credential variable, and the four-call concurrency policy
|
||||
in public GoDoc and consumer guidance; and
|
||||
- retains the OpenAI-compatible integration document as the canonical owner of
|
||||
endpoint composition and outbound wire behavior.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
The completed feature is protected at the narrowest stable owners:
|
||||
|
||||
- registry tests should cover both exact built-in definitions, default queue
|
||||
normalization, capacity-policy publication, lookup ownership, and duplicate
|
||||
rejection for both reserved IDs;
|
||||
- built-in profile tests should validate the generalized catalog invariants and
|
||||
exact Rakestrawhome profile;
|
||||
- a representative root integration test proves that selecting the new
|
||||
profile resolves the expected backend ID, endpoint, model, and credential
|
||||
environment-variable name without making a live network request or
|
||||
requiring a real credential;
|
||||
- the registry's exact capacity-policy tests and the existing generic engine
|
||||
capacity tests together protect the engine-local four-call limit without a
|
||||
duplicate provider-specific concurrency matrix; and
|
||||
- existing OpenRouter, custom-backend, profile-precedence, endpoint-override,
|
||||
credential, capacity, and injected-client tests should remain green without
|
||||
duplicating their full behavioral matrices for the new built-in.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This feature does not add:
|
||||
|
||||
- provider discovery, health checking, model enumeration, or live integration
|
||||
tests;
|
||||
- automatic fallback, retry, or routing between OpenRouter and Rakestrawhome;
|
||||
- global mutable registry state or consumer replacement of built-in IDs;
|
||||
- a second model-client protocol or non-OpenAI-compatible transport;
|
||||
- provider-specific generation defaults that have not been deliberately
|
||||
selected; or
|
||||
- any embedded API-key value.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The backend ID, public constant direction, endpoint base path,
|
||||
credential environment-variable name, concurrency and queue policy, profile
|
||||
ID, model name, optional-setting policy, compatibility behavior, and scope
|
||||
boundaries are fixed by this roadmap.
|
||||
Reference in New Issue
Block a user