diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 4372c08..0436cbb 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -3,7 +3,7 @@ Implement the remaining local work in the [performance roadmap](performance.md) in order. Each sequential stage is scoped for one Luna coding-agent prompt. Update status and evidence as each work unit lands. -Stages 1–10 are complete; Stages 11–14 remain pending. The completed +Stages 1–11 are complete; Stages 12–14 remain pending. The completed inspection plan is preserved at `481e2d4:docs/roadmap/implementation.md` in Git history. @@ -421,6 +421,11 @@ prompt metadata remain identical for the same captured context. ## Stage 11 — Establish Assembled Workflow Benchmarks +**Status: Complete (2026-10-09).** The benchmark-only baseline is committed at +`743e78d`; it covers public preparation, prepared-execution creation, `Run`, +and repeated `Details()` with small, large-message, and mutable-payload +fixtures. Fixture checks and the repeated measurements are recorded below. + **Outcome:** Reproducible end-to-end local benchmarks, including the baseline for conditional snapshot optimization. No production change. @@ -713,6 +718,73 @@ GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ Verification passed on `c5f733a`: `go test -race -mod=readonly ./internal/configuration ./internal/usecase .`. +### Assembled Workflow Benchmark Baseline + +The benchmark-only fixture is committed at `743e78d` (`Add assembled workflow +benchmark baseline`). The same committed source produced the measurements +below. Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark +parallelism 4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema `v6.0.2`, and backend +modules `v3.20260930.0`. + +Command (five 500 ms samples per case, without the race detector): + +```sh +GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ + go test -mod=readonly -run '^$' \ + -bench '^(BenchmarkEnginePreparation|BenchmarkEngineRun|BenchmarkPreparedExecutionDetails)$' \ + -benchmem -benchtime=500ms -count=5 . +``` + +All cases use one assembled engine constructed before timing and a deterministic +injected client; there is no live HTTP or credential lookup. The small prompt +renders `Hello Ada`. The large-message prompt has a 256 KiB literal prefix and +renders that prefix plus a short interpolated suffix. The mutable-payload +prompt uses 64 nested provider-option entries, each with labels, limits, and +route maps/slices, plus a local JSON Schema with 256 optional string +properties and `additionalProperties: false`. Preflight confirms that the +injected `{}` response passes the real schema validator; the `Run` benchmark +also measures that validation. The schema, configuration, and request fixtures +are created before each timer starts. + +`BenchmarkEnginePreparation` measures ordinary `Prepare` separately from +`PrepareExecution` creation. The prepared-execution case discards each unused +handle inside its timed iteration and retains only the most recent discarded +handle pointer. `BenchmarkPreparedExecutionDetails` creates and validates one +ready handle outside timing, measures repeated public `Details()` calls on it, +retains only the latest caller-owned result, then discards the handle. Ordinary +prepare and run sinks are overwritten per iteration and cleared after timing. +The large-message result shares immutable string content, so its allocation +figures are not treated as deep-copy bytes. No benchmark retains every +iteration or includes engine construction/provider latency. + +Times list the five `ns/op` samples in command order, followed by median and +range. Allocation figures are `B/op; allocs/op` (ranges where they varied). + +| Benchmark case | `ns/op` samples (median; range) | Allocations | +| --- | --- | --- | +| Prepare, small | 10935, 10121, 13018, 10288, 8485 (10288; 8485–13018) | 4513; 50 | +| Prepare, large message | 2064223, 1627980, 1588515, 1008441, 911123 (1588515; 911123–2064223) | 872964–872988; 50 | +| Prepare, mutable payload | 6430306, 6519806, 5970344, 5854621, 6456738 (6430306; 5854621–6519806) | 2914750–2914824; 41134–41135 | +| Prepared execution creation + discard, small | 8194, 8016, 8095, 8073, 7939 (8073; 7939–8194) | 5210; 56 | +| Prepared execution creation + discard, large message | 937857, 955672, 919810, 1058503, 1094114 (955672; 919810–1094114) | 873679–873681; 56 | +| Prepared execution creation + discard, mutable payload | 8114785, 8394320, 10387083, 8418604, 8528766 (8418604; 8114785–10387083) | 3711282–3711329; 60727 | +| Run, small | 12157, 11921, 12128, 13762, 12491 (12157; 11921–13762) | 6091; 66 | +| Run, large message | 1082064, 1058363, 1040185, 996377, 951818 (1040185; 951818–1082064) | 874562–874577; 66–67 | +| Run, mutable payload | 6072294, 6338179, 6386632, 7167500, 6805715 (6386632; 6072294–7167500) | 2993599–2993661; 41808–41809 | +| Details, small | 1104, 1039, 1010, 1045, 1058 (1045; 1010–1104) | 1056; 6 | +| Details, large message | 975.1, 2480, 4734, 3402, 1008 (2480; 975.1–4734) | 1056; 6 | +| Details, mutable payload | 2275096, 3281027, 1504724, 1365349, 1329800 (1504724; 1329800–3281027) | 670315–670339; 11548 | + +The 256 KiB message does not make `Details()` allocate 256 KiB per call; +current public conversion shares immutable message strings while copying the +message slice and other metadata. By contrast, `Details()` over the mutable +payload costs about 670 KB and 11.5 thousand allocations per call, providing a +specific snapshot-copy baseline for later ownership analysis. Large-message +timings, especially `Details()`, are noisy and are retained as ranges rather +than point claims. Focused verification passed: `go test -mod=readonly .`; +representative rendered messages, prepared details, generated output, and +structured validation were checked outside benchmark timing. + ### Validation And Compiled-Rendering Baseline The benchmark-only baseline is commit