eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document local PromptKit backend support

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-30 05:22:41 +00:00
parent 241e9d2a89
commit b4363b3b73
5 changed files with 105 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
docs/internal/llm.md
View File

@@ -56,6 +56,13 @@ distinct and deterministically ordered. Endpoint-only profiles retain an empty
backend ID, which the published JSON omits. Successful completion responses and
recorded profile manifests identify the adapter provider as `promptkit`.
The CLI's preparation-only engine and the production adapter use the same
conversion helper to register the optional conventional `local` backend.
Preflight therefore resolves the same backend membership as runtime without
performing generation. When the registration is absent, a profile selecting
`backend: local` fails preparation instead of falling back to a built-in or
endpoint-only target.
Before execution, the adapter also contributes a non-secret checkpoint
fingerprint for the effective PromptKit profile source. It combines the
identity of PromptKit's compiled-in profile catalog with a deterministic digest
@@ -65,7 +72,11 @@ paths. It covers both explicit binding profiles and prompt-selected defaults,
so changing a model or other profile setting cannot reuse checkpoints created
under the prior profile source. This cache identity is independent of durable
profile provenance: run manifests continue to list only profiles actually
observed during LLM calls.
observed during LLM calls. When the local backend is registered, a second
fingerprint hashes its trimmed endpoint behind a stable marker. Changing that
semantic execution target invalidates checkpoint reuse. The raw endpoint is not
stored in checkpoint identity, and the local concurrency limit is excluded
because it changes scheduling rather than execution semantics.
## Shared Provider-Call Limit
@@ -85,8 +96,10 @@ PromptKit applies a second, independent admission limit when the selected
profile names a limited backend. It sits beneath the Notarius scheduled client,
so it may narrow but cannot expand the application-wide limit. Built-in
OpenRouter profiles select PromptKit's reserved backend and its upstream
capacity policy. Endpoint-only profiles do not select a PromptKit backend and
remain limited only by the Notarius scheduler.
capacity policy. A positive configured local-backend limit bounds active local
generations inside PromptKit; zero leaves that backend unlimited there.
Endpoint-only profiles do not select a PromptKit backend and remain limited
only by the Notarius scheduler.
## Prompt And Schema Assets