eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Publish effective LLM backend provenance

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-30 02:18:14 +00:00
parent f8333f2c15
commit 71a004bfc8
11 changed files with 194 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
docs/internal/llm.md
View File

@@ -49,10 +49,12 @@ to module calls, retries, and LLM-backed validators for the whole run.
An empty request profile lets the prompt select its configured default. The CLI
prepares every explicitly selected binding profile before a run begins, so a
missing explicit profile fails before stage execution. Calls record the profile
actually selected by PromptKit; the recorder deduplicates non-secret profile
identity, provider, and model values for manifest use. Successful completion
responses and recorded profile manifests identify the adapter provider as
`promptkit`.
actually selected by PromptKit. The recorder trims and deduplicates non-secret
profile identity, provider, model, selected backend ID, and effective reasoning
values for manifest use. Entries that differ in backend or reasoning remain
distinct and deterministically ordered. Endpoint-only profiles retain an empty
backend ID, which the published JSON omits. Successful completion responses and
recorded profile manifests identify the adapter provider as `promptkit`.
Before execution, the adapter also contributes a non-secret checkpoint
fingerprint for the effective PromptKit profile source. It combines the
@@ -159,18 +161,22 @@ contract is identified in
When debug recording is enabled, the pipeline decorates the shared client. The
wrapper records prepared prompt and response material, timing, selected profile
and model, and call identifiers in the run’s debug bundle, including material
available from a failed structured completion. For a successful completion, a
debug-write failure is surfaced; when the completion already failed, its call
error remains the result. Debug-bundle location, retention, and handling are
operational concerns documented in [Operations](../operations.md#debug-bundles).
and backend, effective model parameters, and call identifiers in the run’s
debug bundle, including material available from a failed structured completion.
Effective parameters use PromptKit's stable lower-case JSON field names and may
include `backend_id`. For a successful completion, a debug-write failure is
surfaced; when the completion already failed, its call error remains the
result. Debug-bundle location, retention, and handling are operational concerns
documented in [Operations](../operations.md#debug-bundles).
Run manifests receive selected profile summaries and component identities, not
prompt, schema, source, reference, or response content. Provider error text is
wrapped with prompt context and bearer credentials are redacted before it
crosses the runtime boundary. Known-secret redaction is available to other
runtime collaborators; it does not make prompt or response contents safe for
general logging.
Run manifests receive selected profile summaries, including optional effective
backend and reasoning provenance, and component identities—not prompt, schema,
source, reference, or response content. The published field semantics belong
to the [JSON output contract](../integrations/json-output.md#manifestjson).
Provider error text is wrapped with prompt context and bearer credentials are
redacted before it crosses the runtime boundary. Known-secret redaction is
available to other runtime collaborators; it does not make prompt or response
contents safe for general logging.
## Failure Boundaries