eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Report actual inference smoke test destinations

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-13 20:27:46 +00:00
parent 208ff42915
commit 116bd0e39b
9 changed files with 87 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/internal/generation.md
View File

@@ -44,8 +44,10 @@ clients.
The diagnostic path uses the same private attempt and retry implementation as
ordinary generation. It retains invocation-local transport, response,
generation, usage, capacity-wait, and retry-delay evidence before normal
result-discard and cancellation checks. This evidence is returned to diagnostic
orchestration; it is not added to public `ExecutionEvent` values or ordinary
result-discard and cancellation checks. Built-in preparation also supplies its
frozen request URL to the evidence path before attempts begin; injected
generation has no equivalent HTTP destination. This evidence is returned to
diagnostic orchestration; it is not added to public `ExecutionEvent` values or ordinary
observer accounting. The [model-client document](llm.md) describes the
transport-observation boundary, and the root
[`SmokeTestReport` GoDoc](../../smoke_diagnostics.go) owns report semantics.

12
docs/roadmap/remediation.md
View File

@@ -1,6 +1,6 @@
# Smoke Test Implementation Review Findings
Status: one finding completed; two open. Review completed 2026-10-13.
Status: two findings completed; one open. Review completed 2026-10-13.
Reviewed implementation commits after `8fbd058` through
`2f735aedf9b5dcc98164292db46ae1eab3e15cde`, against the
@@ -56,7 +56,7 @@ fix; `GOWORK=off go test -race ./internal/usecase .`, focused use-case vet, and
## Finding 2: Report The Actual Initial Inference Request Destination
Priority: P2. Status: open.
Priority: P2. Status: completed 2026-10-13.
Locations: [`recordGenerationEvidence`](../../internal/usecase/smoke_tester.go),
lines 240–248, and
@@ -91,6 +91,14 @@ that distinguishes the actual completion URL from the last destination. Reuse
existing HTTP redirect tests for credential/response attribution mechanics and
keep injected-client HTTP destination evidence unknown.
Completion: built-in generation now passes the frozen prepared request URL
through internal generation evidence. Diagnostic target/initial-attempt URLs
use that value, while the configured base stays in effective settings and
injected inference leaves HTTP destinations unknown. Public GoDoc and the
internal generation document describe the corrected behavior. No-redirect,
redirect, and injected regressions failed before the fix; race-enabled tests
passed for the facade, model client, generation, use case, and engine packages.
## Finding 3: Initialize Zero-Dispatch Usage Consistently
Priority: P3. Status: open.