From df064bd1d3b2cd1de361649d31dadf0d6cc0c8f7 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 9 Oct 2026 01:55:56 +0000 Subject: [PATCH] Record final performance implementation results --- docs/roadmap/implementation.md | 103 +++++++++++++++++++++++++++------ 1 file changed, 85 insertions(+), 18 deletions(-) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index e9172ce..c564aa0 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,6 +1,6 @@ # Computational Efficiency Implementation Plan -**Status:** Stages 1–10 complete; Stage 11 pending. +**Status:** Stages 1–11 complete. ## Purpose And Baseline @@ -78,15 +78,15 @@ is disabled; the build cache is warmed before timing. | Measurement | Baseline observations | Final comparison | | --- | --- | --- | -| Tree metadata, 1,000 entries | Loose: 27.5–28.2 ms; packed: 25.0–28.1 ms. About 1.28 MB allocated per operation. | Pending Stage 11. | -| Unique-blob reads, 49,000 bytes | Loose: 1.86–1.96 s; packed: 2.57–3.08 s. About 34.57 MB allocated per operation. | Pending Stage 11. | -| Repeated-ID reads, 15,000 bytes across 1,000 paths | Loose: 2.12–3.00 ms; packed: 3.42–7.75 ms. About 50.7 KB allocated per operation. | Pending Stage 11. | -| Formatting, 1,000 unique files | Loose: 1.87–2.94 s; packed: 2.12–4.55 s. About 36.9 MB allocated per operation. | Pending Stage 11. | -| Formatting, 1,000 paths sharing one object | Loose: 4.52–12.66 ms; packed: 6.11–8.62 ms. About 1.49 MB allocated per operation. | Pending Stage 11. | -| Markdown links and parsing | 19.4–21.5 ms for two identical 1 MiB documents and 1,000 links; about 7.67 MB allocated per operation. | Pending Stage 11. | -| Facade check, standard Go suite only | 385–690 ms; 906 module bytes; 336–340 KB allocated in the benchmark process. | Pending Stage 11. | -| Facade check with selected file checks | 528–634 ms; 906 module bytes; 632–646 KB allocated in the benchmark process. | Pending Stage 11. | -| Retained content after formatting and Markdown parsing | 1,385,040–1,385,616 bytes above the post-GC metadata baseline, with the candidate and Markdown cache kept alive. | Pending Stage 11. | +| Tree metadata, 1,000 entries | Loose: 27.5–28.2 ms; packed: 25.0–28.1 ms. About 1.28 MB allocated per operation. | Loose: 33.0–35.1 ms; packed: 7.43–15.10 ms. About 1.28 MB allocated. | +| Unique-blob reads, 49,000 bytes | Loose: 1.86–1.96 s; packed: 2.57–3.08 s. About 34.57 MB allocated per operation. | Loose: 89.8–126.3 ms; packed: 41.5–61.0 ms. About 530 KB allocated. | +| Repeated-ID reads, 15,000 bytes across 1,000 paths | Loose: 2.12–3.00 ms; packed: 3.42–7.75 ms. About 50.7 KB allocated per operation. | Loose: 84.8–99.5 ms; packed: 22.3–22.9 ms. About 480–483 KB allocated; slower than baseline because raw file bytes are deliberately not retained between reads. | +| Formatting, 1,000 unique files | Loose: 1.87–2.94 s; packed: 2.12–4.55 s. About 36.9 MB allocated per operation. | Loose: 111.9–114.6 ms; packed: 56.2–59.1 ms. About 2.89 MB allocated. | +| Formatting, 1,000 paths sharing one object | Loose: 4.52–12.66 ms; packed: 6.11–8.62 ms. About 1.49 MB allocated per operation. | Loose: 2.58–2.81 ms; packed: 2.07–2.13 ms. About 121 KB allocated. | +| Markdown links and parsing | 19.4–21.5 ms for two identical 1 MiB documents and 1,000 links; about 7.67 MB allocated per operation. | 10.90–11.14 ms for the same fixture; about 2.29 MB allocated. | +| Facade check, standard Go suite only | 385–690 ms; 906 module bytes; 336–340 KB allocated in the benchmark process. | 374.0–386.9 ms; 906 module bytes; 336–337 KB allocated in the benchmark process. | +| Facade check with selected file checks | 528–634 ms; 906 module bytes; 632–646 KB allocated in the benchmark process. | 379.7–391.8 ms; 906 module bytes; 503–513 KB allocated in the benchmark process. | +| Retained content after formatting and Markdown parsing | 1,385,040–1,385,616 bytes above the post-GC metadata baseline, with the candidate and Markdown cache kept alive. | 180,384–198,000 bytes above the post-GC metadata baseline, with the same caches kept alive. | Raw output is in temporary storage at `/tmp/releasekit-performance-baseline.txt`. The shared machine produced substantial timing variation, especially for packed @@ -159,20 +159,78 @@ the fixture server. | Measurement | Baseline observations | Final comparison | | --- | --- | --- | -| Artifact declarations, two 16 MiB files | 94.3–95.4 ms; 33,554,432 bytes processed; about 68.9 KB allocated. | Pending Stage 11. | -| Declarations plus local verification | 186.7–188.4 ms; 33,554,432 binary bytes per each of two passes; about 136.2 KB allocated. | Pending Stage 11. | -| Facade delivery, fresh uploads | 429–447 ms; 32 MiB binary payload, two files plus manifest; 1.43–1.46 MB allocated. | Pending Stage 11. | -| Facade delivery, matching retry | 309.1–310.0 ms; 32 MiB of local verification plus streamed remote comparisons; 1.21–1.22 MB allocated. | Pending Stage 11. | -| Combined output, 4 KiB and `io.Discard` | 1.82–1.87 μs; 8,256 B and 3 allocations per operation. | Pending Stage 11. | -| Combined output, 8 MiB and `io.Discard` | 2.02–2.36 ms; 16.78 MB and 3 allocations per operation, including the completed capture copy. | Pending Stage 11. | -| Injected version selection | 0.745–0.754 ns; 0 B and 0 allocations. | Pending Stage 11. | -| Repeated real metadata fallback | 1.438–1.474 μs; 1,184 B and 7 allocations per operation. | Pending Stage 11. | +| Artifact declarations, two 16 MiB files | 94.3–95.4 ms; 33,554,432 bytes processed; about 68.9 KB allocated. | 11.4–19.4 μs; 33,554,432 declared bytes; 2,296 B and 33 allocations. Declaration no longer reads or hashes file contents. | +| Declarations plus local verification | 186.7–188.4 ms; 33,554,432 binary bytes per each of two passes; about 136.2 KB allocated. | 94.3–95.3 ms; one 33,554,432-byte verification pass; about 69.7 KB and 58–59 allocations. | +| Facade delivery, fresh uploads | 429–447 ms; 32 MiB binary payload, two files plus manifest; 1.43–1.46 MB allocated. | 331.9–415.3 ms; same payload; 1.38–1.39 MB allocated. | +| Facade delivery, matching retry | 309.1–310.0 ms; 32 MiB of local verification plus streamed remote comparisons; 1.21–1.22 MB allocated. | 210.7–217.5 ms; same local verification and streamed comparisons; 1.15 MB allocated. | +| Combined output, 4 KiB and `io.Discard` | 1.82–1.87 μs; 8,256 B and 3 allocations per operation. | 0.94–1.06 μs; 4,176 B and 2 allocations. | +| Combined output, 8 MiB and `io.Discard` | 2.02–2.36 ms; 16.78 MB and 3 allocations per operation, including the completed capture copy. | 1.73–2.22 ms; about 8.39 MB and 2 allocations. | +| Injected version selection | 0.745–0.754 ns; 0 B and 0 allocations. | 0.754–0.758 ns; 0 B and 0 allocations. | +| Repeated real metadata fallback | 1.438–1.474 μs; 1,184 B and 7 allocations per operation. | 4.59–4.65 ns after cache initialization; 0 B and 0 allocations. Separate initialization: 1.54–2.14 μs; 1,264 B and 9 allocations. | The delivery observations include local HTTP and filesystem costs specific to these fixtures. The output component benchmark does not include child startup; the real delivery benchmark does include HTTP request handling. These results describe measured workloads and do not establish runtime guarantees. +### Final Comparison Record + +The final benchmark run measured commit `42569b4`, before this Stage 11 record +was committed, using Go `go1.26.8 linux/amd64`, Git `2.53.0`, and Linux x86_64 +on an Intel Core i5-7260U at 2.20 GHz (`devbench-02`). It used the same +`-run '^$' -bench '^BenchmarkPerformance' -benchmem -count=3` boundaries and +the same temporary fixtures and cache setup as the baseline. Raw final output +is in `/tmp/releasekit-performance-final-stage1.txt`, +`/tmp/releasekit-performance-final-delivery.txt`, +`/tmp/releasekit-performance-final-output.txt`, and +`/tmp/releasekit-performance-final-version.txt`. + +The unique-blob batch reader reduced measured reads and formatting from seconds +to tens or hundreds of milliseconds and cut their allocations by roughly an +order of magnitude or more. Formatting repeated object IDs also improved in +both time and allocations because parsed outcomes are reused. Raw repeated-ID +reads regressed: 1,000 loose reads measured 84.8–99.5 ms and 480–483 KB, versus +2.12–3.00 ms and 50.7 KB at baseline; packed reads measured 22.3–22.9 ms versus +3.42–7.75 ms. This workload rereads small object bodies through the batch +session, while regular-file bytes intentionally remain caller-owned and are +not cached by the tree. That is the measured time cost of limiting retained +source bytes; scoped read reuse could be reconsidered if repeated raw reads +become a common workload. + +Retained processed content fell from about 1.39 MB to 180–198 KB with the +candidate and parsed Markdown cache kept alive. The Markdown benchmark now +allocates about 2.29 MB instead of 7.67 MB and measured 10.90–11.14 ms versus +19.4–21.5 ms at baseline. The complete offline facade check with selected file +checks measured 379.7–391.8 ms and 503–513 KB in the benchmark process, versus +528–634 ms and 632–646 KB at baseline. The Go-suite-only check remains +374.0–386.9 ms; its real test/compile subprocess is the main complete-check +cost, and benchmark-process allocations exclude subprocess allocations. + +Artifact declarations measured 11.0–11.5 µs and 2,296 B for two 16 MiB files. +Declarations plus current local verification measured 92.4–93.3 ms and about +69.7 KB, against 186.7–188.4 ms and 136.2 KB at baseline. The current flow +therefore removes one 32 MiB local read/hash pass while retaining one local +verification before hosted requests. In the prebuilt local-HTTP facade fixture, +fresh uploads measured 331.9–415.3 ms and matching retries 210.7–217.5 ms. A +retry still reads and verifies local assets and compares streamed remote bytes; +those fixture and remote-comparison costs remain in the complete workflow. + +Output capture allocations fell from 8,256 B to 4,176 B for 4 KiB, and from +about 16.78 MB to 8.39 MB for 8 MiB. The 8 MiB timing ranges overlap, while one +allocation and the completed-buffer copy are removed. The returned capture +remains caller-owned in memory; a failed output command can also retain its +diagnostic string. Warm runtime fallback access measured 4.59–4.65 ns and zero +allocations after a separate 1.54–2.14 µs, 1,264 B initialization. The +linker-injected selection remains allocation-free. + +Some microbenchmark timings varied substantially between repeated runs on this +shared machine; allocations were steadier. The local delivery server and +filesystem are workload fixtures, not a hosted service. These figures +describe only the recorded toolchain and workloads and do not imply numerical +runtime guarantees. Raw repeated-ID read retention and disk-backed large +diagnostics remain follow-up options if their measured workload frequency +justifies their memory and lifecycle tradeoffs. + 1. Extend the same benchmark prefix to declaration/preflight processing of two deterministic 16 MiB artifact files. Measure prebuilt facade `Deliver` against a local HTTP fixture for fresh uploads and matching retries. @@ -583,6 +641,15 @@ are recorded, and the worktree is clean. Completion depends on observed behavior and reviewed diffs; benchmarks establish improvements within their measured scope without adding numerical runtime guarantees. +**Completion:** Complete. The full offline suite (`go test ./...`) and compiled +facade/dependency check (`go run ./internal/apicheck`) passed. Focused race +validation passed for the runtime fallback, Go output adapter, and process +boundary. Targeted fixtures also cover cancellation and writer-failure causes. +The canonical internal docs describe Git reader lifetime and source ownership, +validation reuse and delivery verification, Go output ownership, and runtime +fallback reuse. Local links and Markdown whitespace were checked. All stages +are marked complete; this results document is retained for review. + ## Open Questions None required for this implementation. The plan selects scoped batch reading,