Record request preparation baseline
This commit is contained in:
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
Implement the remaining local work in the [performance roadmap](performance.md)
|
Implement the remaining local work in the [performance roadmap](performance.md)
|
||||||
in order. Each sequential stage is scoped for one Luna coding-agent prompt.
|
in order. Each sequential stage is scoped for one Luna coding-agent prompt.
|
||||||
Stages 1–4 are complete; Stages 5–14 remain pending. Update status and evidence
|
Stages 1–5 are complete; Stages 6–14 remain pending. Update status and evidence
|
||||||
as each work unit lands.
|
as each work unit lands.
|
||||||
The completed inspection plan is preserved at
|
The completed inspection plan is preserved at
|
||||||
`481e2d4:docs/roadmap/implementation.md` in Git history.
|
`481e2d4:docs/roadmap/implementation.md` in Git history.
|
||||||
@@ -216,6 +216,12 @@ Existing progress-cancellation and nonregular-file tests remain authoritative.
|
|||||||
|
|
||||||
## Stage 5 — Establish Encoding Benchmarks And Error-Boundary Coverage
|
## Stage 5 — Establish Encoding Benchmarks And Error-Boundary Coverage
|
||||||
|
|
||||||
|
**Status: Complete (2026-10-09).** `PrepareGeneration` now has an offline
|
||||||
|
benchmark baseline for no extras, small extras, a 1 MiB string extra, nested
|
||||||
|
JSON extras, and structured output. Tests protect nested invalid-extra context
|
||||||
|
and request-error classification when the schema cannot be encoded. No
|
||||||
|
production behavior changed.
|
||||||
|
|
||||||
**Outcome:** A repeatable request-preparation baseline and focused protection
|
**Outcome:** A repeatable request-preparation baseline and focused protection
|
||||||
for the classifications that Stage 6 must preserve. No production change.
|
for the classifications that Stage 6 must preserve. No production change.
|
||||||
|
|
||||||
@@ -744,6 +750,42 @@ through EOF. Focused verification passed: `go test -mod=readonly
|
|||||||
./internal/artifact`; the existing cancellation and nonregular-file checks
|
./internal/artifact`; the existing cancellation and nonregular-file checks
|
||||||
remain in the passing suite.
|
remain in the passing suite.
|
||||||
|
|
||||||
|
### Request-Preparation Baseline
|
||||||
|
|
||||||
|
The benchmark and boundary-test fixture is committed at `3f15715` (`Add
|
||||||
|
request preparation baseline and boundary tests`). The benchmark measures
|
||||||
|
`PrepareGeneration` without dispatch, with client construction, request
|
||||||
|
fixtures, and the structured schema created before timing. It uses a synthetic
|
||||||
|
endpoint and no credential or provider calls. Environment: Linux/amd64, Intel
|
||||||
|
Core i5-7260U @ 2.20 GHz, benchmark parallelism 4, Go 1.26.8, ConfigKit
|
||||||
|
`v0.5.0`, JSON Schema `v6.0.2`, and backend modules `v3.20260930.0`.
|
||||||
|
|
||||||
|
Command (five 500 ms samples per case, without the race detector):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
|
||||||
|
go test -mod=readonly -run '^$' -bench '^BenchmarkPrepareGeneration$' \
|
||||||
|
-benchmem -benchtime=500ms -count=5 ./internal/llm
|
||||||
|
```
|
||||||
|
|
||||||
|
Times list all five `ns/op` samples in command order, followed by median and
|
||||||
|
range. Allocation figures are `B/op; allocs/op`.
|
||||||
|
|
||||||
|
| Preparation case | `ns/op` samples (median; range) | Allocations |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| No extras | 2771, 2873, 2872, 2907, 2885 (2873; 2771–2907) | 1632; 23 |
|
||||||
|
| Small extras | 4216, 4255, 4440, 4090, 4245 (4245; 4090–4440) | 1928; 32 |
|
||||||
|
| 1 MiB string extra | 3108655, 3046197, 3165484, 3115831, 3074278 (3108655; 3046197–3165484) | 2147898–2213942; 26–27 |
|
||||||
|
| Nested JSON extras | 5933, 5803, 5675, 5490, 5611 (5675; 5490–5933) | 2304; 40 |
|
||||||
|
| Structured output | 6421, 6018, 6479, 6153, 6712 (6421; 6018–6712) | 2632–2633; 47 |
|
||||||
|
|
||||||
|
The baseline covers payload construction and complete JSON encoding for both
|
||||||
|
ordinary and representative structured requests. The 1 MiB case's higher
|
||||||
|
allocation volume is expected to include the large string and encoded body;
|
||||||
|
these measurements are the comparison reference for the following request
|
||||||
|
encoding change, not a provider-latency estimate. Focused verification passed:
|
||||||
|
`go test -mod=readonly ./internal/llm`.
|
||||||
|
|
||||||
## Open Questions
|
## Open Questions
|
||||||
|
|
||||||
No consumer-policy decisions remain open. File reservation and encoding error
|
No consumer-policy decisions remain open. File reservation and encoding error
|
||||||
|
|||||||
Reference in New Issue
Block a user