diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 0788461..d06ec62 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -2,7 +2,7 @@ Implement the remaining local work in the [performance roadmap](performance.md) in order. Each sequential stage is scoped for one Luna coding-agent prompt. -Stages 1–4 are complete; Stages 5–14 remain pending. Update status and evidence +Stages 1–5 are complete; Stages 6–14 remain pending. Update status and evidence as each work unit lands. The completed inspection plan is preserved at `481e2d4:docs/roadmap/implementation.md` in Git history. @@ -216,6 +216,12 @@ Existing progress-cancellation and nonregular-file tests remain authoritative. ## Stage 5 — Establish Encoding Benchmarks And Error-Boundary Coverage +**Status: Complete (2026-10-09).** `PrepareGeneration` now has an offline +benchmark baseline for no extras, small extras, a 1 MiB string extra, nested +JSON extras, and structured output. Tests protect nested invalid-extra context +and request-error classification when the schema cannot be encoded. No +production behavior changed. + **Outcome:** A repeatable request-preparation baseline and focused protection for the classifications that Stage 6 must preserve. No production change. @@ -744,6 +750,42 @@ through EOF. Focused verification passed: `go test -mod=readonly ./internal/artifact`; the existing cancellation and nonregular-file checks remain in the passing suite. +### Request-Preparation Baseline + +The benchmark and boundary-test fixture is committed at `3f15715` (`Add +request preparation baseline and boundary tests`). The benchmark measures +`PrepareGeneration` without dispatch, with client construction, request +fixtures, and the structured schema created before timing. It uses a synthetic +endpoint and no credential or provider calls. Environment: Linux/amd64, Intel +Core i5-7260U @ 2.20 GHz, benchmark parallelism 4, Go 1.26.8, ConfigKit +`v0.5.0`, JSON Schema `v6.0.2`, and backend modules `v3.20260930.0`. + +Command (five 500 ms samples per case, without the race detector): + +```sh +GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ + go test -mod=readonly -run '^$' -bench '^BenchmarkPrepareGeneration$' \ + -benchmem -benchtime=500ms -count=5 ./internal/llm +``` + +Times list all five `ns/op` samples in command order, followed by median and +range. Allocation figures are `B/op; allocs/op`. + +| Preparation case | `ns/op` samples (median; range) | Allocations | +| --- | --- | --- | +| No extras | 2771, 2873, 2872, 2907, 2885 (2873; 2771–2907) | 1632; 23 | +| Small extras | 4216, 4255, 4440, 4090, 4245 (4245; 4090–4440) | 1928; 32 | +| 1 MiB string extra | 3108655, 3046197, 3165484, 3115831, 3074278 (3108655; 3046197–3165484) | 2147898–2213942; 26–27 | +| Nested JSON extras | 5933, 5803, 5675, 5490, 5611 (5675; 5490–5933) | 2304; 40 | +| Structured output | 6421, 6018, 6479, 6153, 6712 (6421; 6018–6712) | 2632–2633; 47 | + +The baseline covers payload construction and complete JSON encoding for both +ordinary and representative structured requests. The 1 MiB case's higher +allocation volume is expected to include the large string and encoded body; +these measurements are the comparison reference for the following request +encoding change, not a provider-latency estimate. Focused verification passed: +`go test -mod=readonly ./internal/llm`. + ## Open Questions No consumer-policy decisions remain open. File reservation and encoding error