eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document smoke-test diagnostics

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-13 18:32:41 +00:00
parent b7a951685b
commit f4c721d934
13 changed files with 256 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
docs/formats.md
View File

@@ -195,7 +195,8 @@ backends:
An omitted or null `smoke_tests` block disables these metadata defaults.
`default_profile` is preserved as an exact, nonblank profile ID; its existence
is checked only if a diagnostic selects that default. Omitted/null
is checked only if an inference diagnostic selects that default. Omitted/null
`default_profile` means there is no inference default. Omitted/null
`reachability` or its endpoint uses the effective backend endpoint. An explicit
reachability endpoint is trimmed and must be an absolute HTTP(S) URL with a
host; unlike the backend base endpoint, its path and query are allowed. Null
@@ -204,9 +205,30 @@ means fallback, while an explicit blank value is invalid.
Omitted/null `authentication` means no authentication adapter is declared. A
non-null object requires `kind`, which is trimmed and case-normalized; the
recognized values are `none` and `openrouter_key`. Its omitted/null endpoint
uses the adapter's backend-relative destination; an explicit endpoint follows
the same full-URL rules as reachability. These values are configuration only:
engine construction does not contact a backend.
uses the adapter's backend-relative destination (`openrouter_key` appends
`/key` to the effective backend base); an explicit endpoint follows the same
full-URL rules as reachability. `none` declares authentication inapplicable.
These values are configuration only: engine construction does not contact a
backend.
These fields are resolved through the ordinary ConfigKit source layers before
Promptkit validates the final effective values. A higher-priority mapping can
override individual nested fields; an explicit null can clear the whole
`smoke_tests` block or one of its nested objects. An endpoint set to null uses
its documented fallback, while an explicit blank endpoint is invalid. The
metadata does not change ordinary generation credential requirements or
provider settings.
The selected v3 catalog modules ship diagnostic defaults in a separate,
opt-in `smoke-tests` root rather than changing the existing `catalog` root.
This Promptkit build selects [OpenRouter v3 `v3.20261013.0`](oss/attribution.md#promptkit-openrouter-catalog-v3)
with default profile `promptkit-openrouter-standard` and adapter
`openrouter_key`, and [Rakestrawhome v3 `v3.20261013.0`](oss/attribution.md#promptkit-rakestrawhome-catalog-v3)
with default profile `rakestrawhome-gemma-4-31b` and adapter `none`. The
attribution entries identify the selected module commits; these catalog
candidates are local and may not be publicly available. A consumer selecting
the extra root needs a compatible loader. These defaults do not promise
provider availability, account access, or free test requests.
## Profile Definitions