eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document prompt sessions and concurrency defaults

Browse Source
This commit is contained in:
Eric Rakestraw
2026-08-03 19:16:18 +00:00
parent 8a15b083a0
commit fc3c128171
10 changed files with 74 additions and 95 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
docs/operations.md
View File

@@ -266,16 +266,18 @@ transport-wide cap. Notarius does not add another timeout around PromptKit.
The pinned upstream boundary and profile-format links are in
[PromptKit Integration](integrations/pkg-promptkit.md).
Concurrency has two independent layers. Notarius **total_llm** is the
application-wide provider-call limit shared by all backends, modules, retries,
and validators. PromptKit may impose a narrower admission limit for the
selected backend. The effective active-generation bound is the intersection of
both limits and can therefore be lower than **total_llm**. Built-in OpenRouter
profiles use PromptKit's upstream backend limit; endpoint-only profiles have no
PromptKit backend limit and remain bounded by Notarius. For the configured
local backend, a zero **concurrency_limit** leaves only the Notarius scheduler
as a call limit. A positive value makes the effective active local-generation
bound the smaller of **total_llm** and that local limit.
Concurrency has two independent layers. Notarius **total_llm** defaults to 16
and is the application-wide provider-call limit shared by all backends,
modules, retries, and validators. PromptKit may impose a narrower admission
limit for the selected backend. The effective active-generation bound is the
intersection of the Notarius limit, any PromptKit backend limit, and work made
available by the pipeline. Built-in OpenRouter profiles use PromptKit's
upstream backend limit; endpoint-only profiles have no PromptKit backend limit
and remain bounded by Notarius. For the configured local backend, a zero
**concurrency_limit** leaves only the Notarius scheduler as a call limit. A
positive value makes the effective active local-generation bound the smaller
of **total_llm** and that local limit, so a local limit of four permits no more
than four active local generations.
For a positive local limit, PromptKit owns its default waiting capacity and
admission behavior. When a PromptKit backend has admitted all active and queued
@@ -289,3 +291,11 @@ under [PromptKit profiles](config.md#promptkit-profiles) and
limits and actual provider-call limits are independent. Notarius writes local
filesystem state only; remote storage, archival, and retention automation are
outside the implemented CLI.
Every run has an effective prompt session used for provider routing and run
provenance. The generated default is stable for the same input module and raw
input bytes; use [**--session-id**](cli.md#run) only when intentionally grouping
different invocations. Both generated and explicit values can be visible to
providers, manifests, checkpoints, and requested debug bundles. Do not put
credentials or other secrets in an explicit session identifier; command-line
values are not a credential mechanism.