Record requirement inspection benchmarks
This commit is contained in:
@@ -286,6 +286,10 @@ exercise one frozen body per generation.
|
||||
|
||||
## Stage 7 — Establish Inspection Benchmarks
|
||||
|
||||
**Status: Complete (2026-10-09).** Captured configuration lookup and public
|
||||
requirement-inspection benchmarks are committed. Repeated benchmark-only
|
||||
baseline measurements are recorded below; no production behavior changed.
|
||||
|
||||
**Outcome:** Owned lookup and assembled requirement-inspection baselines, with
|
||||
no production change. This separates fixture work from the selection refactor.
|
||||
|
||||
@@ -542,6 +546,55 @@ rather than rerunning merely to create the final commit.
|
||||
|
||||
## Execution Evidence
|
||||
|
||||
### Inspection Baseline
|
||||
|
||||
The benchmark-only fixture is committed at `04d5d695e0201a97c1070e1ff49ef4f0c9a7f5ba`
|
||||
(`Add requirement inspection benchmarks`). The configuration benchmarks build
|
||||
one captured snapshot per case before timing and measure the existing prompt
|
||||
snapshot and owned-profile lookups. The public benchmark builds one real Engine
|
||||
and both requests before timing, then calls only `InspectRequirements` with
|
||||
the caller variable supplied. None of the paths reads artifacts or schemas or
|
||||
contacts a provider. The small fixture has two provider-option entries; the
|
||||
larger fixture has 64 nested provider maps, each with mutable label/route slices
|
||||
and nested limit/route maps. Prompt lookup allocation is independent of profile
|
||||
extra size because it does not copy the selected profile. The profile lookup
|
||||
and assembled requirement inspection expose the mutable-map copy cost being
|
||||
measured. Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark
|
||||
parallelism 4, Go 1.26.8, ConfigKit `v0.5.0`, and JSON Schema `v6.0.2`.
|
||||
|
||||
Command (five 500 ms samples per case, without the race detector):
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
|
||||
go test -mod=readonly -run '^$' \
|
||||
-bench 'BenchmarkInspectionSelection|BenchmarkRequirementInspection' \
|
||||
-benchmem -benchtime=500ms -count=5 ./internal/configuration .
|
||||
```
|
||||
|
||||
Times list all five `ns/op` samples in command order, followed by median and
|
||||
range. Allocation figures are `B/op; allocs/op`.
|
||||
|
||||
| Configuration lookup | `ns/op` samples (median; range) | Allocations |
|
||||
| --- | --- | --- |
|
||||
| Prompt, small extras | 356.8, 354.2, 370.1, 352.8, 364.5 (356.8; 352.8–370.1) | 400; 4 |
|
||||
| Prompt, nested extras | 423.2, 478.8, 456.2, 436.6, 460.4 (456.2; 423.2–478.8) | 400; 4 |
|
||||
| Profile, small extras | 18785, 21077, 18952, 19701, 18682 (18952; 18682–21077) | 9538; 211 |
|
||||
| Profile, nested extras | 624104, 635513, 600281, 600667, 620228 (620228; 600281–635513) | 261024–261028; 5979 |
|
||||
|
||||
| Public requirement inspection | `ns/op` samples (median; range) | Allocations |
|
||||
| --- | --- | --- |
|
||||
| Small extras | 23022, 25193, 24687, 24974, 23157 (24687; 23022–25193) | 11922; 240 |
|
||||
| Nested extras | 635726, 642297, 591992, 605001, 590312 (605001; 590312–642297) | 263370–263372; 6008 |
|
||||
|
||||
The prompt snapshot lookup remained at 400 B and four allocations in both
|
||||
fixtures, while owned profile lookup rose from about 9.5 KB/211 allocations to
|
||||
about 261 KB/5979 allocations. Public requirement inspection rose from about
|
||||
11.9 KB/240 allocations to about 263 KB/6008 allocations. These results
|
||||
establish a baseline in which unrelated nested provider extras dominate the
|
||||
large profile-selection and requirement-inspection cases; they are local
|
||||
allocation measurements, not end-to-end provider-latency claims. Fixture
|
||||
identity and prompt/requirement inventories were checked outside timing.
|
||||
|
||||
### Validation And Compiled-Rendering Baseline
|
||||
|
||||
The benchmark-only baseline is commit
|
||||
|
||||
Reference in New Issue
Block a user