Document local PromptKit backend support
This commit is contained in:
@@ -46,7 +46,7 @@ other than **version** is optional.
|
|||||||
| Field | Type | Default | Rules |
|
| Field | Type | Default | Rules |
|
||||||
| --- | --- | --- | --- |
|
| --- | --- | --- | --- |
|
||||||
| **version** | integer | none | Required; must be 4. |
|
| **version** | integer | none | Required; must be 4. |
|
||||||
| **promptkit** | object | none | Profile source configuration. |
|
| **promptkit** | object | none | Profile source and optional local-backend configuration. |
|
||||||
| **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. |
|
| **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. |
|
||||||
| **concurrency** | object | see below | Global LLM and extraction limits. |
|
| **concurrency** | object | see below | Global LLM and extraction limits. |
|
||||||
| **output** | object | see below | Published output settings. |
|
| **output** | object | see below | Published output settings. |
|
||||||
@@ -71,27 +71,58 @@ per-user root. An explicit empty output or debug directory is invalid.
|
|||||||
|
|
||||||
## PromptKit Profiles
|
## PromptKit Profiles
|
||||||
|
|
||||||
The optional **promptkit** object selects one source of profile definitions:
|
The optional **promptkit** object selects one source of profile definitions and
|
||||||
|
may register one conventional local OpenAI-compatible backend:
|
||||||
|
|
||||||
~~~yaml
|
~~~yaml
|
||||||
version: 4
|
version: 4
|
||||||
|
|
||||||
promptkit:
|
promptkit:
|
||||||
profile_dir: /path/to/profiles
|
profile_dir: ./profiles
|
||||||
# profile_file: /path/to/profiles.yml
|
# profile_file: ./profiles.yml
|
||||||
|
local_backend:
|
||||||
|
endpoint: http://localhost:8000/v1
|
||||||
|
concurrency_limit: 2
|
||||||
~~~
|
~~~
|
||||||
|
|
||||||
| Field | Type | Rules |
|
| Field | Type | Rules |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| **profile_dir** | string | Non-empty directory containing profile files. |
|
| **profile_dir** | string | Non-empty directory containing profile files. |
|
||||||
| **profile_file** | string | Non-empty profile file. |
|
| **profile_file** | string | Non-empty profile file. |
|
||||||
|
| **local_backend** | object | Optional registration for the conventional PromptKit backend ID **local**. |
|
||||||
|
| **local_backend.endpoint** | string | Required when **local_backend** is present; absolute HTTP or HTTPS URL with a host. |
|
||||||
|
| **local_backend.concurrency_limit** | integer | Optional non-negative limit; defaults to 0. |
|
||||||
|
|
||||||
Set at most one of these fields. Profile IDs used by a binding must be available
|
Set at most one of these fields. Profile IDs used by a binding must be available
|
||||||
from the selected PromptKit profile source when the pipeline is resolved.
|
from the selected PromptKit profile source when the pipeline is resolved.
|
||||||
Keep credentials out of this file: configure a profile to read its credential
|
The optional local registration may coexist with either profile source or with
|
||||||
from an environment variable, then set that environment variable only in the
|
PromptKit's built-in profiles.
|
||||||
run environment. PromptKit owns the profile-file format; see the
|
|
||||||
[PromptKit upstream boundary](integrations/pkg-promptkit.md) for the pinned
|
When **local_backend** is present, its endpoint is trimmed and must use HTTP or
|
||||||
package and canonical format reference.
|
HTTPS case-insensitively, be absolute, and have a non-empty host. URL paths are
|
||||||
|
allowed. User information, queries, and fragments are rejected. A zero
|
||||||
|
**concurrency_limit** leaves the local backend unrestricted inside PromptKit;
|
||||||
|
a positive value limits simultaneous local generations. The application-wide
|
||||||
|
**concurrency.total_llm** limit still applies in both cases. Neither local
|
||||||
|
backend field has an environment override. Omitting **local_backend** registers
|
||||||
|
nothing and preserves existing built-in and endpoint-only profile behavior.
|
||||||
|
|
||||||
|
A file-backed PromptKit profile selects the registration by its case-sensitive
|
||||||
|
backend ID:
|
||||||
|
|
||||||
|
~~~yaml
|
||||||
|
id: local-summary
|
||||||
|
backend: local
|
||||||
|
model: example-model
|
||||||
|
~~~
|
||||||
|
|
||||||
|
Keep credentials out of the local-backend object. A PromptKit profile may name
|
||||||
|
its credential environment variable through `api_key_env`; set that variable
|
||||||
|
only in the run environment. PromptKit owns the
|
||||||
|
[pinned profile-file format](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.3.0/docs/formats.md).
|
||||||
|
The [PromptKit upstream boundary](integrations/pkg-promptkit.md) identifies the
|
||||||
|
supported package API, and [Operations](operations.md#operational-limits)
|
||||||
|
describes the effective concurrency layers.
|
||||||
|
|
||||||
## Migrating Version 3 Configuration
|
## Migrating Version 3 Configuration
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
# PromptKit Integration
|
# PromptKit Integration
|
||||||
|
|
||||||
Notarius pins
|
Notarius pins
|
||||||
[`gitea.maximumdirect.net/eric/promptkit` v0.2.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.2.0)
|
[`gitea.maximumdirect.net/eric/promptkit` v0.3.0](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.3.0)
|
||||||
as its in-process prompt engine. The upstream
|
as its in-process prompt engine. The upstream
|
||||||
[Go package consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.2.0/docs/consumers/pkg-promptkit.md)
|
[Go package consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.3.0/docs/consumers/pkg-promptkit.md)
|
||||||
owns the public engine API, and the upstream
|
owns the public engine API, and the upstream
|
||||||
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.2.0/docs/formats.md)
|
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.3.0/docs/formats.md)
|
||||||
owns prompt, profile, and schema file contracts.
|
owns prompt, profile, and schema file contracts.
|
||||||
|
|
||||||
## Supported Boundary
|
## Supported Boundary
|
||||||
@@ -18,10 +18,16 @@ Notarius relies on the root `promptkit` package to:
|
|||||||
a direct session ID, prompt identity, and profile selection;
|
a direct session ID, prompt identity, and profile selection;
|
||||||
- return rendered debug material, validated structured output, selected
|
- return rendered debug material, validated structured output, selected
|
||||||
profile, backend, effective model metadata, and token usage;
|
profile, backend, effective model metadata, and token usage;
|
||||||
|
- register the optional conventional `local` backend through `BackendLocal`,
|
||||||
|
`LocalBackend`, and `WithBackend`;
|
||||||
- distinguish structured-output validation failure from execution failure; and
|
- distinguish structured-output validation failure from execution failure; and
|
||||||
- identify a missing explicit profile through `ErrProfileNotFound` and backend
|
- identify a missing explicit profile through `ErrProfileNotFound` and backend
|
||||||
admission exhaustion through `ErrCapacityExceeded`.
|
admission exhaustion through `ErrCapacityExceeded`.
|
||||||
|
|
||||||
|
The pinned
|
||||||
|
[`BackendLocal`, `LocalBackend`, and `WithBackend` API](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.3.0/backends.go)
|
||||||
|
owns the registration and backend-capacity contract.
|
||||||
|
|
||||||
Notarius does not use PromptKit's optional `ArtifactReader`. It materializes
|
Notarius does not use PromptKit's optional `ArtifactReader`. It materializes
|
||||||
source and reference content itself and supplies owned inline artifacts at the
|
source and reference content itself and supplies owned inline artifacts at the
|
||||||
adapter boundary. It also retains responsibility for pipeline retries,
|
adapter boundary. It also retains responsibility for pipeline retries,
|
||||||
@@ -38,8 +44,9 @@ Notarius records PromptKit's selected backend ID and effective reasoning
|
|||||||
setting as optional run-manifest provenance. Endpoint-only profiles have no
|
setting as optional run-manifest provenance. Endpoint-only profiles have no
|
||||||
backend ID. Debug prompt material also retains the selected backend ID and
|
backend ID. Debug prompt material also retains the selected backend ID and
|
||||||
PromptKit's stable lower-case `effective_model_params` JSON, which may include
|
PromptKit's stable lower-case `effective_model_params` JSON, which may include
|
||||||
`backend_id`. Notarius production configuration does not expose user-defined
|
`backend_id`. Notarius production configuration exposes one optional
|
||||||
PromptKit backend registration.
|
conventional `local` registration. It does not expose a general user-defined
|
||||||
|
PromptKit backend registry. Endpoint-only profiles remain supported unchanged.
|
||||||
|
|
||||||
Notarius retains its application-wide scheduled client around the PromptKit
|
Notarius retains its application-wide scheduled client around the PromptKit
|
||||||
adapter. PromptKit may apply a narrower limit for the selected backend;
|
adapter. PromptKit may apply a narrower limit for the selected backend;
|
||||||
@@ -55,7 +62,8 @@ module assets, maps its transport-neutral completion contract, prepares and
|
|||||||
executes requests, validates output, records provenance, captures debug
|
executes requests, validates output, records provenance, captures debug
|
||||||
material, redacts errors, and preserves timeout ownership.
|
material, redacts errors, and preserves timeout ownership.
|
||||||
[Configuration](../config.md#promptkit-profiles) defines how a Notarius
|
[Configuration](../config.md#promptkit-profiles) defines how a Notarius
|
||||||
configuration selects one PromptKit profile source.
|
configuration selects one PromptKit profile source and optionally registers
|
||||||
|
the conventional local backend.
|
||||||
|
|
||||||
PromptKit API or format changes outside this boundary are not implicitly
|
PromptKit API or format changes outside this boundary are not implicitly
|
||||||
supported. Updating the pinned version requires reviewing the adapter and
|
supported. Updating the pinned version requires reviewing the adapter and
|
||||||
|
|||||||
@@ -39,14 +39,20 @@ This establishes the public precedence order without giving environment input a
|
|||||||
second file schema. Loading and application reject malformed YAML, unsupported
|
second file schema. Loading and application reject malformed YAML, unsupported
|
||||||
file versions, unknown fields, invalid values, and identifiers that are empty
|
file versions, unknown fields, invalid values, and identifiers that are empty
|
||||||
or collide after whitespace normalization. The file application also makes the
|
or collide after whitespace normalization. The file application also makes the
|
||||||
effective extraction-worker default follow the effective LLM limit.
|
effective extraction-worker default follow the effective LLM limit. A present
|
||||||
|
PromptKit local-backend object requires and trims its endpoint, defaults its
|
||||||
|
omitted concurrency limit to zero, and is copied so the parsed file model
|
||||||
|
cannot alias the populated **Config**.
|
||||||
|
|
||||||
**Config.Validate** checks configuration-only invariants before resolution. It
|
**Config.Validate** checks configuration-only invariants before resolution. It
|
||||||
rejects incompatible profile sources, invalid state-surface values, unsupported
|
rejects incompatible profile sources, invalid state-surface values, unsupported
|
||||||
concurrency settings, malformed bindings and references, invalid retries, and
|
concurrency settings, malformed bindings and references, invalid retries, and
|
||||||
invalid pipeline, step, or lane structure. Its errors retain the closest known
|
invalid pipeline, step, or lane structure. PromptKit local-backend validation
|
||||||
pipeline, lane, and binding context. It deliberately does not require modules
|
accepts only an absolute HTTP or HTTPS endpoint with a host and no user
|
||||||
to be registered: that requires a catalog and belongs to resolution.
|
information, query, or fragment, and rejects a negative local concurrency
|
||||||
|
limit. Its errors retain the closest known pipeline, lane, and binding context.
|
||||||
|
It deliberately does not require modules to be registered: that requires a
|
||||||
|
catalog and belongs to resolution.
|
||||||
|
|
||||||
The exact user-selectable values and validation rules are defined in
|
The exact user-selectable values and validation rules are defined in
|
||||||
[Configuration](../config.md). Keep additions to the file model, an
|
[Configuration](../config.md). Keep additions to the file model, an
|
||||||
@@ -71,7 +77,8 @@ options, and returns the fixed ordered pipeline shape. The resulting
|
|||||||
changes, a clone of the input configuration, and the resolved pipeline.
|
changes, a clone of the input configuration, and the resolved pipeline.
|
||||||
Callers may therefore retain or modify their input slices and maps without
|
Callers may therefore retain or modify their input slices and maps without
|
||||||
changing the resolved result, and later consumers cannot mutate the original
|
changing the resolved result, and later consumers cannot mutate the original
|
||||||
configuration through the effective value.
|
configuration through the effective value. This ownership includes the nested
|
||||||
|
PromptKit local-backend value.
|
||||||
|
|
||||||
Resolution failures stop before module construction and source parsing. They
|
Resolution failures stop before module construction and source parsing. They
|
||||||
include an error path for an unconfigured pipeline, missing module, missing
|
include an error path for an unconfigured pipeline, missing module, missing
|
||||||
@@ -94,9 +101,12 @@ Configuration summaries must use **Redacted**, **RedactedSummaryPayload**, or
|
|||||||
Those methods copy every binding and nested option container, replace values
|
Those methods copy every binding and nested option container, replace values
|
||||||
whose key is credential-shaped with **[REDACTED]**, and omit materialized
|
whose key is credential-shaped with **[REDACTED]**, and omit materialized
|
||||||
reference content while retaining safe binding and reference provenance. The
|
reference content while retaining safe binding and reference provenance. The
|
||||||
payload must not alias the source configuration or resolved pipeline. This
|
payload must not alias the source configuration or resolved pipeline.
|
||||||
redaction is deliberately narrow: it protects configuration summaries and does
|
PromptKit's local endpoint and concurrency limit are preserved as non-secret
|
||||||
not authorize recording arbitrary environment values or provider requests.
|
configuration metadata in the independently owned summary; the object contains
|
||||||
|
no credential value. This redaction is deliberately narrow: it protects
|
||||||
|
configuration summaries and does not authorize recording arbitrary environment
|
||||||
|
values or provider requests.
|
||||||
|
|
||||||
## Invariants To Preserve
|
## Invariants To Preserve
|
||||||
|
|
||||||
|
|||||||
@@ -56,6 +56,13 @@ distinct and deterministically ordered. Endpoint-only profiles retain an empty
|
|||||||
backend ID, which the published JSON omits. Successful completion responses and
|
backend ID, which the published JSON omits. Successful completion responses and
|
||||||
recorded profile manifests identify the adapter provider as `promptkit`.
|
recorded profile manifests identify the adapter provider as `promptkit`.
|
||||||
|
|
||||||
|
The CLI's preparation-only engine and the production adapter use the same
|
||||||
|
conversion helper to register the optional conventional `local` backend.
|
||||||
|
Preflight therefore resolves the same backend membership as runtime without
|
||||||
|
performing generation. When the registration is absent, a profile selecting
|
||||||
|
`backend: local` fails preparation instead of falling back to a built-in or
|
||||||
|
endpoint-only target.
|
||||||
|
|
||||||
Before execution, the adapter also contributes a non-secret checkpoint
|
Before execution, the adapter also contributes a non-secret checkpoint
|
||||||
fingerprint for the effective PromptKit profile source. It combines the
|
fingerprint for the effective PromptKit profile source. It combines the
|
||||||
identity of PromptKit's compiled-in profile catalog with a deterministic digest
|
identity of PromptKit's compiled-in profile catalog with a deterministic digest
|
||||||
@@ -65,7 +72,11 @@ paths. It covers both explicit binding profiles and prompt-selected defaults,
|
|||||||
so changing a model or other profile setting cannot reuse checkpoints created
|
so changing a model or other profile setting cannot reuse checkpoints created
|
||||||
under the prior profile source. This cache identity is independent of durable
|
under the prior profile source. This cache identity is independent of durable
|
||||||
profile provenance: run manifests continue to list only profiles actually
|
profile provenance: run manifests continue to list only profiles actually
|
||||||
observed during LLM calls.
|
observed during LLM calls. When the local backend is registered, a second
|
||||||
|
fingerprint hashes its trimmed endpoint behind a stable marker. Changing that
|
||||||
|
semantic execution target invalidates checkpoint reuse. The raw endpoint is not
|
||||||
|
stored in checkpoint identity, and the local concurrency limit is excluded
|
||||||
|
because it changes scheduling rather than execution semantics.
|
||||||
|
|
||||||
## Shared Provider-Call Limit
|
## Shared Provider-Call Limit
|
||||||
|
|
||||||
@@ -85,8 +96,10 @@ PromptKit applies a second, independent admission limit when the selected
|
|||||||
profile names a limited backend. It sits beneath the Notarius scheduled client,
|
profile names a limited backend. It sits beneath the Notarius scheduled client,
|
||||||
so it may narrow but cannot expand the application-wide limit. Built-in
|
so it may narrow but cannot expand the application-wide limit. Built-in
|
||||||
OpenRouter profiles select PromptKit's reserved backend and its upstream
|
OpenRouter profiles select PromptKit's reserved backend and its upstream
|
||||||
capacity policy. Endpoint-only profiles do not select a PromptKit backend and
|
capacity policy. A positive configured local-backend limit bounds active local
|
||||||
remain limited only by the Notarius scheduler.
|
generations inside PromptKit; zero leaves that backend unlimited there.
|
||||||
|
Endpoint-only profiles do not select a PromptKit backend and remain limited
|
||||||
|
only by the Notarius scheduler.
|
||||||
|
|
||||||
## Prompt And Schema Assets
|
## Prompt And Schema Assets
|
||||||
|
|
||||||
|
|||||||
@@ -118,8 +118,10 @@ disabled. Without **--resume**, a recording-enabled run executes normally and
|
|||||||
does not load checkpoint state. Compatibility includes the resolved pipeline,
|
does not load checkpoint state. Compatibility includes the resolved pipeline,
|
||||||
input, selected lanes, runtime overrides, reference provenance, LLM-profile
|
input, selected lanes, runtime overrides, reference provenance, LLM-profile
|
||||||
provenance, the effective PromptKit profile-source fingerprint, and
|
provenance, the effective PromptKit profile-source fingerprint, and
|
||||||
prepared-component fingerprints. Changing profile content causes a cold miss
|
prepared-component fingerprints. When a local PromptKit backend is configured,
|
||||||
even when its profile ID is unchanged. A changed identity produces a cold miss;
|
compatibility also includes a non-secret fingerprint of its endpoint. Changing
|
||||||
|
profile content or the local endpoint causes a cold miss; changing only the
|
||||||
|
local concurrency limit does not. A changed identity produces a cold miss;
|
||||||
Notarius does not migrate, rewrite, or delete older checkpoint directories.
|
Notarius does not migrate, rewrite, or delete older checkpoint directories.
|
||||||
Reasoning-effort inheritance, replacement, and explicit clearing are distinct
|
Reasoning-effort inheritance, replacement, and explicit clearing are distinct
|
||||||
runtime identities, so checkpoints created under one state are not reused by
|
runtime identities, so checkpoints created under one state are not reused by
|
||||||
@@ -217,7 +219,7 @@ Provider execution settings and the generation timeout come from the selected
|
|||||||
PromptKit profile. The invocation-only **--reasoning-effort** and
|
PromptKit profile. The invocation-only **--reasoning-effort** and
|
||||||
**--clear-reasoning-effort** controls may replace or clear that profile setting
|
**--clear-reasoning-effort** controls may replace or clear that profile setting
|
||||||
for all LLM-backed calls in one run without changing the profile. PromptKit
|
for all LLM-backed calls in one run without changing the profile. PromptKit
|
||||||
v0.2.0 does not add a provider retry loop. Notarius binding retries rerun the
|
v0.3.0 does not add a provider retry loop. Notarius binding retries rerun the
|
||||||
complete module operation and validation chain as defined by
|
complete module operation and validation chain as defined by
|
||||||
[module bindings](config.md#module-bindings-and-validators).
|
[module bindings](config.md#module-bindings-and-validators).
|
||||||
|
|
||||||
@@ -234,13 +236,19 @@ and validators. PromptKit may impose a narrower admission limit for the
|
|||||||
selected backend. The effective active-generation bound is the intersection of
|
selected backend. The effective active-generation bound is the intersection of
|
||||||
both limits and can therefore be lower than **total_llm**. Built-in OpenRouter
|
both limits and can therefore be lower than **total_llm**. Built-in OpenRouter
|
||||||
profiles use PromptKit's upstream backend limit; endpoint-only profiles have no
|
profiles use PromptKit's upstream backend limit; endpoint-only profiles have no
|
||||||
PromptKit backend limit and remain bounded by Notarius.
|
PromptKit backend limit and remain bounded by Notarius. For the configured
|
||||||
|
local backend, a zero **concurrency_limit** leaves only the Notarius scheduler
|
||||||
|
as a call limit. A positive value makes the effective active local-generation
|
||||||
|
bound the smaller of **total_llm** and that local limit.
|
||||||
|
|
||||||
When a PromptKit backend has admitted all active and queued work, a new call
|
For a positive local limit, PromptKit owns its default waiting capacity and
|
||||||
fails as capacity exhaustion before generation. The adapter does not retry it.
|
admission behavior. When a PromptKit backend has admitted all active and queued
|
||||||
The calling stage's configured retry policy applies normally, and the run fails
|
work, a new call fails as capacity exhaustion before generation. The adapter
|
||||||
if those attempts are exhausted. Caller cancellation remains authoritative.
|
maps that failure to Notarius's existing provider-neutral capacity error and
|
||||||
Configuration contracts are documented under
|
does not retry it. The calling stage's configured retry policy applies
|
||||||
|
normally, and the run fails if those attempts are exhausted. Caller
|
||||||
|
cancellation remains authoritative. Configuration contracts are documented
|
||||||
|
under [PromptKit profiles](config.md#promptkit-profiles) and
|
||||||
[concurrency](config.md#concurrency-output-cache-and-debug). Extract-worker
|
[concurrency](config.md#concurrency-output-cache-and-debug). Extract-worker
|
||||||
limits and actual provider-call limits are independent. Notarius writes local
|
limits and actual provider-call limits are independent. Notarius writes local
|
||||||
filesystem state only; remote storage, archival, and retention automation are
|
filesystem state only; remote storage, archival, and retention automation are
|
||||||
|
|||||||
Reference in New Issue
Block a user