Document local backend configuration paths
This commit is contained in:
@@ -126,33 +126,84 @@ exact normalization, precedence, error, copying, and exposure contract.
|
||||
|
||||
### Register A Custom Backend
|
||||
|
||||
Register a reusable OpenAI-compatible connection once, then select it from a
|
||||
profile. This local backend limits model generation to two simultaneous calls;
|
||||
because `QueueCapacity` is omitted, the engine admits up to 1024 additional
|
||||
calls waiting behind them:
|
||||
Choose the smallest configuration that fits how the endpoint will be reused.
|
||||
|
||||
#### Use An Endpoint-Only Profile
|
||||
|
||||
Put the endpoint directly on an in-memory profile when only that profile needs
|
||||
it and shared backend identity or capacity policy is unnecessary:
|
||||
|
||||
```go
|
||||
engine, err := promptkit.NewEngine(promptkit.Config{
|
||||
PromptDir: "prompts",
|
||||
},
|
||||
promptkit.WithBackend(promptkit.Backend{
|
||||
ID: "local",
|
||||
Endpoint: "http://localhost:8000/v1",
|
||||
APIKeyEnv: "LOCAL_LLM_API_KEY",
|
||||
ConcurrencyLimit: 2,
|
||||
promptkit.WithProfiles(promptkit.Profile{
|
||||
ID: "local-summary",
|
||||
Endpoint: "http://localhost:8000/v1",
|
||||
Model: "example-model",
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
Endpoint-only profiles have an empty backend ID and remain unrestricted by
|
||||
backend capacity policy.
|
||||
|
||||
#### Use The Conventional Local Backend
|
||||
|
||||
Use `LocalBackend` when profiles should share the conventional `local`
|
||||
identity, endpoint, and concurrency limit:
|
||||
|
||||
```go
|
||||
engine, err := promptkit.NewEngine(promptkit.Config{
|
||||
PromptDir: "prompts",
|
||||
},
|
||||
promptkit.WithBackend(
|
||||
promptkit.LocalBackend("http://localhost:8000/v1", 2),
|
||||
),
|
||||
promptkit.WithProfiles(promptkit.Profile{
|
||||
ID: "local-summary",
|
||||
BackendID: "local",
|
||||
BackendID: promptkit.BackendLocal,
|
||||
Model: "example-model",
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
The helper is explicit: it does not pre-register a backend or read environment
|
||||
variables. Supplying a positive limit leaves queue capacity omitted, so normal
|
||||
backend registration selects the existing default waiting capacity of 1024.
|
||||
The returned value still enters the engine through `WithBackend`.
|
||||
|
||||
#### Configure A Complete Backend
|
||||
|
||||
Use a keyed `Backend` value for authentication, extra request parameters, an
|
||||
explicit queue capacity, a custom ID, or multiple local endpoints:
|
||||
|
||||
```go
|
||||
noWaiting := 0
|
||||
engine, err := promptkit.NewEngine(promptkit.Config{
|
||||
PromptDir: "prompts",
|
||||
},
|
||||
promptkit.WithBackend(promptkit.Backend{
|
||||
ID: "local-gpu",
|
||||
Endpoint: "http://gpu-host:8000/v1",
|
||||
APIKeyEnv: "LOCAL_GPU_API_KEY",
|
||||
ExtraParams: map[string]any{"provider_option": "enabled"},
|
||||
ConcurrencyLimit: 2,
|
||||
QueueCapacity: &noWaiting,
|
||||
}),
|
||||
promptkit.WithProfiles(promptkit.Profile{
|
||||
ID: "gpu-summary",
|
||||
BackendID: "local-gpu",
|
||||
Model: "example-model",
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
Use distinct custom IDs when registering multiple local endpoints.
|
||||
Registrations belong to one engine and custom IDs cannot replace built-ins.
|
||||
The [`Backend` and `WithBackend` GoDoc](../../backends.go) defines validation,
|
||||
copying, uniqueness, exact concurrency-field semantics, and request-default
|
||||
behavior.
|
||||
The [`Backend`, `LocalBackend`, and `WithBackend` GoDoc](../../backends.go)
|
||||
defines exact construction, validation, copying, uniqueness, concurrency, and
|
||||
request-default behavior.
|
||||
|
||||
Both file-backed and in-memory profiles select a registration through
|
||||
`backend` or `Profile.BackendID`. Profile and request endpoint overrides retain
|
||||
@@ -173,8 +224,8 @@ zero:
|
||||
```go
|
||||
noWaiting := 0
|
||||
backend := promptkit.Backend{
|
||||
ID: "local",
|
||||
Endpoint: "http://localhost:8000/v1",
|
||||
ID: "local-gpu",
|
||||
Endpoint: "http://gpu-host:8000/v1",
|
||||
ConcurrencyLimit: 2,
|
||||
QueueCapacity: &noWaiting,
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Local Backend Convenience Implementation Plan
|
||||
|
||||
**Status:** Ready for implementation.
|
||||
**Status:** Complete.
|
||||
|
||||
## Purpose
|
||||
|
||||
@@ -221,7 +221,7 @@ none of those other documents owns the affected task or contract.
|
||||
|
||||
## Stage 1: Add The Public Constructor And Contract Test
|
||||
|
||||
**Status:** Not started.
|
||||
**Status:** Complete.
|
||||
|
||||
### Objective
|
||||
|
||||
@@ -283,7 +283,7 @@ Stage 1 is complete when:
|
||||
|
||||
## Stage 2: Publish Consumer Guidance And Validate The Repository
|
||||
|
||||
**Status:** Not started.
|
||||
**Status:** Complete.
|
||||
|
||||
### Objective
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Local Backend Convenience
|
||||
|
||||
**Status:** Accepted.
|
||||
**Status:** Complete.
|
||||
|
||||
## Purpose
|
||||
|
||||
|
||||
Reference in New Issue
Block a user