Document smoke-test diagnostics
This commit is contained in:
@@ -216,6 +216,69 @@ request-specific overrides. An empty diff is not proof of equivalent execution.
|
||||
Records can contain ordinary application data, so use the application's
|
||||
disclosure policy when presenting them.
|
||||
|
||||
## Run Explicit Smoke Diagnostics
|
||||
|
||||
[`Engine.SmokeTest`](../../smoke_diagnostics.go) runs one explicit diagnostic
|
||||
using a target resolved from the engine's captured configuration. It does not
|
||||
execute the selected application prompt or its inputs, session, artifacts, or
|
||||
output schema. Unlike the offline examples and default test suite, calling it
|
||||
can contact the selected provider and use configured credentials. It is a
|
||||
consumer operation, not an automatic readiness verdict or a release check.
|
||||
|
||||
For example, a caller can request the built-in text inference check and treat
|
||||
its optional fact as an observation rather than assuming that a nil Go error
|
||||
means success:
|
||||
|
||||
```go
|
||||
report, err := engine.SmokeTest(ctx, promptkit.SmokeTestRequest{
|
||||
Kind: promptkit.SmokeTestTextInference,
|
||||
ProfileID: profileID,
|
||||
APIKey: apiKey,
|
||||
})
|
||||
if err != nil {
|
||||
if report != nil {
|
||||
retainDisclosureSafeSummary(report)
|
||||
}
|
||||
return err
|
||||
}
|
||||
if report != nil {
|
||||
matched := report.Facts.ExpectedTextMatched
|
||||
switch {
|
||||
case matched == nil:
|
||||
// The comparison was not evaluated or is unknown/inapplicable.
|
||||
case *matched:
|
||||
// The private expected-text comparison matched.
|
||||
default:
|
||||
// It did not match; the operation may still have concluded normally.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
A nil Go error means the diagnostic concluded, not that its finding was
|
||||
positive. Fact pointers are tri-state: nil means unknown, not evaluated, or
|
||||
inapplicable. Reports contain factual observations and effective settings, not
|
||||
generated text or provider bodies; identifiers and settings can still be
|
||||
sensitive. The
|
||||
[`SmokeTest` GoDoc](../../smoke_diagnostics.go) owns selection precedence,
|
||||
exact fields, timeout and retry behavior, supported transports, and disclosure
|
||||
details.
|
||||
|
||||
Caller cancellation can return both an error and a partial report. Preserve the
|
||||
context error identity and inspect any available observations before returning:
|
||||
|
||||
```go
|
||||
report, err := engine.SmokeTest(ctx, request)
|
||||
if report != nil {
|
||||
retainDisclosureSafeSummary(report)
|
||||
}
|
||||
if err != nil {
|
||||
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
|
||||
// The report may describe work completed before interruption.
|
||||
}
|
||||
return err
|
||||
}
|
||||
```
|
||||
|
||||
## Inspect A Prompt Before Preparation
|
||||
|
||||
Use [`Engine.InspectPrompt`](../../engine.go) to check one configured prompt's
|
||||
|
||||
Reference in New Issue
Block a user