diff --git a/docs/consumers/pkg-promptkit.md b/docs/consumers/pkg-promptkit.md index 40f94ab..4ae3045 100644 --- a/docs/consumers/pkg-promptkit.md +++ b/docs/consumers/pkg-promptkit.md @@ -126,33 +126,84 @@ exact normalization, precedence, error, copying, and exposure contract. ### Register A Custom Backend -Register a reusable OpenAI-compatible connection once, then select it from a -profile. This local backend limits model generation to two simultaneous calls; -because `QueueCapacity` is omitted, the engine admits up to 1024 additional -calls waiting behind them: +Choose the smallest configuration that fits how the endpoint will be reused. + +#### Use An Endpoint-Only Profile + +Put the endpoint directly on an in-memory profile when only that profile needs +it and shared backend identity or capacity policy is unnecessary: ```go engine, err := promptkit.NewEngine(promptkit.Config{ PromptDir: "prompts", }, - promptkit.WithBackend(promptkit.Backend{ - ID: "local", - Endpoint: "http://localhost:8000/v1", - APIKeyEnv: "LOCAL_LLM_API_KEY", - ConcurrencyLimit: 2, + promptkit.WithProfiles(promptkit.Profile{ + ID: "local-summary", + Endpoint: "http://localhost:8000/v1", + Model: "example-model", }), +) +``` + +Endpoint-only profiles have an empty backend ID and remain unrestricted by +backend capacity policy. + +#### Use The Conventional Local Backend + +Use `LocalBackend` when profiles should share the conventional `local` +identity, endpoint, and concurrency limit: + +```go +engine, err := promptkit.NewEngine(promptkit.Config{ + PromptDir: "prompts", +}, + promptkit.WithBackend( + promptkit.LocalBackend("http://localhost:8000/v1", 2), + ), promptkit.WithProfiles(promptkit.Profile{ ID: "local-summary", - BackendID: "local", + BackendID: promptkit.BackendLocal, Model: "example-model", }), ) ``` +The helper is explicit: it does not pre-register a backend or read environment +variables. Supplying a positive limit leaves queue capacity omitted, so normal +backend registration selects the existing default waiting capacity of 1024. +The returned value still enters the engine through `WithBackend`. + +#### Configure A Complete Backend + +Use a keyed `Backend` value for authentication, extra request parameters, an +explicit queue capacity, a custom ID, or multiple local endpoints: + +```go +noWaiting := 0 +engine, err := promptkit.NewEngine(promptkit.Config{ + PromptDir: "prompts", +}, + promptkit.WithBackend(promptkit.Backend{ + ID: "local-gpu", + Endpoint: "http://gpu-host:8000/v1", + APIKeyEnv: "LOCAL_GPU_API_KEY", + ExtraParams: map[string]any{"provider_option": "enabled"}, + ConcurrencyLimit: 2, + QueueCapacity: &noWaiting, + }), + promptkit.WithProfiles(promptkit.Profile{ + ID: "gpu-summary", + BackendID: "local-gpu", + Model: "example-model", + }), +) +``` + +Use distinct custom IDs when registering multiple local endpoints. Registrations belong to one engine and custom IDs cannot replace built-ins. -The [`Backend` and `WithBackend` GoDoc](../../backends.go) defines validation, -copying, uniqueness, exact concurrency-field semantics, and request-default -behavior. +The [`Backend`, `LocalBackend`, and `WithBackend` GoDoc](../../backends.go) +defines exact construction, validation, copying, uniqueness, concurrency, and +request-default behavior. Both file-backed and in-memory profiles select a registration through `backend` or `Profile.BackendID`. Profile and request endpoint overrides retain @@ -173,8 +224,8 @@ zero: ```go noWaiting := 0 backend := promptkit.Backend{ - ID: "local", - Endpoint: "http://localhost:8000/v1", + ID: "local-gpu", + Endpoint: "http://gpu-host:8000/v1", ConcurrencyLimit: 2, QueueCapacity: &noWaiting, } diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index dad477e..2b8a8c6 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,6 +1,6 @@ # Local Backend Convenience Implementation Plan -**Status:** Ready for implementation. +**Status:** Complete. ## Purpose @@ -221,7 +221,7 @@ none of those other documents owns the affected task or contract. ## Stage 1: Add The Public Constructor And Contract Test -**Status:** Not started. +**Status:** Complete. ### Objective @@ -283,7 +283,7 @@ Stage 1 is complete when: ## Stage 2: Publish Consumer Guidance And Validate The Repository -**Status:** Not started. +**Status:** Complete. ### Objective diff --git a/docs/roadmap/local-backend.md b/docs/roadmap/local-backend.md index e3013ea..8d466e8 100644 --- a/docs/roadmap/local-backend.md +++ b/docs/roadmap/local-backend.md @@ -1,6 +1,6 @@ # Local Backend Convenience -**Status:** Accepted. +**Status:** Complete. ## Purpose