Record request preparation baseline

This commit is contained in:
2026-10-09 00:25:39 +00:00
parent 3f157150e4
commit c63ca5352f

View File

@@ -2,7 +2,7 @@
Implement the remaining local work in the [performance roadmap](performance.md) Implement the remaining local work in the [performance roadmap](performance.md)
in order. Each sequential stage is scoped for one Luna coding-agent prompt. in order. Each sequential stage is scoped for one Luna coding-agent prompt.
Stages 1–4 are complete; Stages 5–14 remain pending. Update status and evidence Stages 1–5 are complete; Stages 6–14 remain pending. Update status and evidence
as each work unit lands. as each work unit lands.
The completed inspection plan is preserved at The completed inspection plan is preserved at
`481e2d4:docs/roadmap/implementation.md` in Git history. `481e2d4:docs/roadmap/implementation.md` in Git history.
@@ -216,6 +216,12 @@ Existing progress-cancellation and nonregular-file tests remain authoritative.
## Stage 5 — Establish Encoding Benchmarks And Error-Boundary Coverage ## Stage 5 — Establish Encoding Benchmarks And Error-Boundary Coverage
**Status: Complete (2026-10-09).** `PrepareGeneration` now has an offline
benchmark baseline for no extras, small extras, a 1 MiB string extra, nested
JSON extras, and structured output. Tests protect nested invalid-extra context
and request-error classification when the schema cannot be encoded. No
production behavior changed.
**Outcome:** A repeatable request-preparation baseline and focused protection **Outcome:** A repeatable request-preparation baseline and focused protection
for the classifications that Stage 6 must preserve. No production change. for the classifications that Stage 6 must preserve. No production change.
@@ -744,6 +750,42 @@ through EOF. Focused verification passed: `go test -mod=readonly
./internal/artifact`; the existing cancellation and nonregular-file checks ./internal/artifact`; the existing cancellation and nonregular-file checks
remain in the passing suite. remain in the passing suite.
### Request-Preparation Baseline
The benchmark and boundary-test fixture is committed at `3f15715` (`Add
request preparation baseline and boundary tests`). The benchmark measures
`PrepareGeneration` without dispatch, with client construction, request
fixtures, and the structured schema created before timing. It uses a synthetic
endpoint and no credential or provider calls. Environment: Linux/amd64, Intel
Core i5-7260U @ 2.20 GHz, benchmark parallelism 4, Go 1.26.8, ConfigKit
`v0.5.0`, JSON Schema `v6.0.2`, and backend modules `v3.20260930.0`.
Command (five 500 ms samples per case, without the race detector):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' -bench '^BenchmarkPrepareGeneration$' \
-benchmem -benchtime=500ms -count=5 ./internal/llm
```
Times list all five `ns/op` samples in command order, followed by median and
range. Allocation figures are `B/op; allocs/op`.
| Preparation case | `ns/op` samples (median; range) | Allocations |
| --- | --- | --- |
| No extras | 2771, 2873, 2872, 2907, 2885 (2873; 2771–2907) | 1632; 23 |
| Small extras | 4216, 4255, 4440, 4090, 4245 (4245; 4090–4440) | 1928; 32 |
| 1 MiB string extra | 3108655, 3046197, 3165484, 3115831, 3074278 (3108655; 3046197–3165484) | 2147898–2213942; 26–27 |
| Nested JSON extras | 5933, 5803, 5675, 5490, 5611 (5675; 5490–5933) | 2304; 40 |
| Structured output | 6421, 6018, 6479, 6153, 6712 (6421; 6018–6712) | 2632–2633; 47 |
The baseline covers payload construction and complete JSON encoding for both
ordinary and representative structured requests. The 1 MiB case's higher
allocation volume is expected to include the large string and encoded body;
these measurements are the comparison reference for the following request
encoding change, not a provider-latency estimate. Focused verification passed:
`go test -mod=readonly ./internal/llm`.
## Open Questions ## Open Questions
No consumer-policy decisions remain open. File reservation and encoding error No consumer-policy decisions remain open. File reservation and encoding error