Document local backend configuration paths
This commit is contained in:
@@ -126,33 +126,84 @@ exact normalization, precedence, error, copying, and exposure contract.
|
|||||||
|
|
||||||
### Register A Custom Backend
|
### Register A Custom Backend
|
||||||
|
|
||||||
Register a reusable OpenAI-compatible connection once, then select it from a
|
Choose the smallest configuration that fits how the endpoint will be reused.
|
||||||
profile. This local backend limits model generation to two simultaneous calls;
|
|
||||||
because `QueueCapacity` is omitted, the engine admits up to 1024 additional
|
#### Use An Endpoint-Only Profile
|
||||||
calls waiting behind them:
|
|
||||||
|
Put the endpoint directly on an in-memory profile when only that profile needs
|
||||||
|
it and shared backend identity or capacity policy is unnecessary:
|
||||||
|
|
||||||
```go
|
```go
|
||||||
engine, err := promptkit.NewEngine(promptkit.Config{
|
engine, err := promptkit.NewEngine(promptkit.Config{
|
||||||
PromptDir: "prompts",
|
PromptDir: "prompts",
|
||||||
},
|
},
|
||||||
promptkit.WithBackend(promptkit.Backend{
|
promptkit.WithProfiles(promptkit.Profile{
|
||||||
ID: "local",
|
ID: "local-summary",
|
||||||
Endpoint: "http://localhost:8000/v1",
|
Endpoint: "http://localhost:8000/v1",
|
||||||
APIKeyEnv: "LOCAL_LLM_API_KEY",
|
Model: "example-model",
|
||||||
ConcurrencyLimit: 2,
|
|
||||||
}),
|
}),
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Endpoint-only profiles have an empty backend ID and remain unrestricted by
|
||||||
|
backend capacity policy.
|
||||||
|
|
||||||
|
#### Use The Conventional Local Backend
|
||||||
|
|
||||||
|
Use `LocalBackend` when profiles should share the conventional `local`
|
||||||
|
identity, endpoint, and concurrency limit:
|
||||||
|
|
||||||
|
```go
|
||||||
|
engine, err := promptkit.NewEngine(promptkit.Config{
|
||||||
|
PromptDir: "prompts",
|
||||||
|
},
|
||||||
|
promptkit.WithBackend(
|
||||||
|
promptkit.LocalBackend("http://localhost:8000/v1", 2),
|
||||||
|
),
|
||||||
promptkit.WithProfiles(promptkit.Profile{
|
promptkit.WithProfiles(promptkit.Profile{
|
||||||
ID: "local-summary",
|
ID: "local-summary",
|
||||||
BackendID: "local",
|
BackendID: promptkit.BackendLocal,
|
||||||
Model: "example-model",
|
Model: "example-model",
|
||||||
}),
|
}),
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The helper is explicit: it does not pre-register a backend or read environment
|
||||||
|
variables. Supplying a positive limit leaves queue capacity omitted, so normal
|
||||||
|
backend registration selects the existing default waiting capacity of 1024.
|
||||||
|
The returned value still enters the engine through `WithBackend`.
|
||||||
|
|
||||||
|
#### Configure A Complete Backend
|
||||||
|
|
||||||
|
Use a keyed `Backend` value for authentication, extra request parameters, an
|
||||||
|
explicit queue capacity, a custom ID, or multiple local endpoints:
|
||||||
|
|
||||||
|
```go
|
||||||
|
noWaiting := 0
|
||||||
|
engine, err := promptkit.NewEngine(promptkit.Config{
|
||||||
|
PromptDir: "prompts",
|
||||||
|
},
|
||||||
|
promptkit.WithBackend(promptkit.Backend{
|
||||||
|
ID: "local-gpu",
|
||||||
|
Endpoint: "http://gpu-host:8000/v1",
|
||||||
|
APIKeyEnv: "LOCAL_GPU_API_KEY",
|
||||||
|
ExtraParams: map[string]any{"provider_option": "enabled"},
|
||||||
|
ConcurrencyLimit: 2,
|
||||||
|
QueueCapacity: &noWaiting,
|
||||||
|
}),
|
||||||
|
promptkit.WithProfiles(promptkit.Profile{
|
||||||
|
ID: "gpu-summary",
|
||||||
|
BackendID: "local-gpu",
|
||||||
|
Model: "example-model",
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Use distinct custom IDs when registering multiple local endpoints.
|
||||||
Registrations belong to one engine and custom IDs cannot replace built-ins.
|
Registrations belong to one engine and custom IDs cannot replace built-ins.
|
||||||
The [`Backend` and `WithBackend` GoDoc](../../backends.go) defines validation,
|
The [`Backend`, `LocalBackend`, and `WithBackend` GoDoc](../../backends.go)
|
||||||
copying, uniqueness, exact concurrency-field semantics, and request-default
|
defines exact construction, validation, copying, uniqueness, concurrency, and
|
||||||
behavior.
|
request-default behavior.
|
||||||
|
|
||||||
Both file-backed and in-memory profiles select a registration through
|
Both file-backed and in-memory profiles select a registration through
|
||||||
`backend` or `Profile.BackendID`. Profile and request endpoint overrides retain
|
`backend` or `Profile.BackendID`. Profile and request endpoint overrides retain
|
||||||
@@ -173,8 +224,8 @@ zero:
|
|||||||
```go
|
```go
|
||||||
noWaiting := 0
|
noWaiting := 0
|
||||||
backend := promptkit.Backend{
|
backend := promptkit.Backend{
|
||||||
ID: "local",
|
ID: "local-gpu",
|
||||||
Endpoint: "http://localhost:8000/v1",
|
Endpoint: "http://gpu-host:8000/v1",
|
||||||
ConcurrencyLimit: 2,
|
ConcurrencyLimit: 2,
|
||||||
QueueCapacity: &noWaiting,
|
QueueCapacity: &noWaiting,
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Local Backend Convenience Implementation Plan
|
# Local Backend Convenience Implementation Plan
|
||||||
|
|
||||||
**Status:** Ready for implementation.
|
**Status:** Complete.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
@@ -221,7 +221,7 @@ none of those other documents owns the affected task or contract.
|
|||||||
|
|
||||||
## Stage 1: Add The Public Constructor And Contract Test
|
## Stage 1: Add The Public Constructor And Contract Test
|
||||||
|
|
||||||
**Status:** Not started.
|
**Status:** Complete.
|
||||||
|
|
||||||
### Objective
|
### Objective
|
||||||
|
|
||||||
@@ -283,7 +283,7 @@ Stage 1 is complete when:
|
|||||||
|
|
||||||
## Stage 2: Publish Consumer Guidance And Validate The Repository
|
## Stage 2: Publish Consumer Guidance And Validate The Repository
|
||||||
|
|
||||||
**Status:** Not started.
|
**Status:** Complete.
|
||||||
|
|
||||||
### Objective
|
### Objective
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Local Backend Convenience
|
# Local Backend Convenience
|
||||||
|
|
||||||
**Status:** Accepted.
|
**Status:** Complete.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user