eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document backend capacity management

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-29 21:16:49 +00:00
parent d2c4051dd0
commit e61ab700c7
10 changed files with 269 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
doc.go
View File

@@ -9,9 +9,11 @@
//
// # Concurrency and ownership
//
// An Engine supports concurrent Prepare and Run calls. An injected [LLMClient]
// or [ArtifactReader] can therefore receive concurrent calls and must be safe
// for that use.
// An Engine supports concurrent Prepare and Run calls. Engine-local backend
// policies bound admitted Run calls and model generations where configured,
// while different backend pools and unlimited backends continue independently.
// An injected [LLMClient] or [ArtifactReader] can therefore still receive
// concurrent calls and must be safe for that use.
//
// NewEngine copies in-memory profiles and backend definitions. Prepare and Run
// copy request maps, slices, pointer values, and JSON-compatible extra