eric/ReleaseKit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Record final performance implementation results

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-09 01:55:56 +00:00
parent 42569b4af2
commit df064bd1d3
1 changed files with 85 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

103
docs/roadmap/implementation.md
View File

@@ -1,6 +1,6 @@
# Computational Efficiency Implementation Plan
**Status:** Stages 1–10 complete; Stage 11 pending.
**Status:** Stages 1–11 complete.
## Purpose And Baseline
@@ -78,15 +78,15 @@ is disabled; the build cache is warmed before timing.
| Measurement | Baseline observations | Final comparison |
| --- | --- | --- |
| Tree metadata, 1,000 entries | Loose: 27.5–28.2 ms; packed: 25.0–28.1 ms. About 1.28 MB allocated per operation. | Pending Stage 11. |
| Unique-blob reads, 49,000 bytes | Loose: 1.86–1.96 s; packed: 2.57–3.08 s. About 34.57 MB allocated per operation. | Pending Stage 11. |
| Repeated-ID reads, 15,000 bytes across 1,000 paths | Loose: 2.12–3.00 ms; packed: 3.42–7.75 ms. About 50.7 KB allocated per operation. | Pending Stage 11. |
| Formatting, 1,000 unique files | Loose: 1.87–2.94 s; packed: 2.12–4.55 s. About 36.9 MB allocated per operation. | Pending Stage 11. |
| Formatting, 1,000 paths sharing one object | Loose: 4.52–12.66 ms; packed: 6.11–8.62 ms. About 1.49 MB allocated per operation. | Pending Stage 11. |
| Markdown links and parsing | 19.4–21.5 ms for two identical 1 MiB documents and 1,000 links; about 7.67 MB allocated per operation. | Pending Stage 11. |
| Facade check, standard Go suite only | 385–690 ms; 906 module bytes; 336–340 KB allocated in the benchmark process. | Pending Stage 11. |
| Facade check with selected file checks | 528–634 ms; 906 module bytes; 632–646 KB allocated in the benchmark process. | Pending Stage 11. |
| Retained content after formatting and Markdown parsing | 1,385,040–1,385,616 bytes above the post-GC metadata baseline, with the candidate and Markdown cache kept alive. | Pending Stage 11. |
| Tree metadata, 1,000 entries | Loose: 27.5–28.2 ms; packed: 25.0–28.1 ms. About 1.28 MB allocated per operation. | Loose: 33.0–35.1 ms; packed: 7.43–15.10 ms. About 1.28 MB allocated. |
| Unique-blob reads, 49,000 bytes | Loose: 1.86–1.96 s; packed: 2.57–3.08 s. About 34.57 MB allocated per operation. | Loose: 89.8–126.3 ms; packed: 41.5–61.0 ms. About 530 KB allocated. |
| Repeated-ID reads, 15,000 bytes across 1,000 paths | Loose: 2.12–3.00 ms; packed: 3.42–7.75 ms. About 50.7 KB allocated per operation. | Loose: 84.8–99.5 ms; packed: 22.3–22.9 ms. About 480–483 KB allocated; slower than baseline because raw file bytes are deliberately not retained between reads. |
| Formatting, 1,000 unique files | Loose: 1.87–2.94 s; packed: 2.12–4.55 s. About 36.9 MB allocated per operation. | Loose: 111.9–114.6 ms; packed: 56.2–59.1 ms. About 2.89 MB allocated. |
| Formatting, 1,000 paths sharing one object | Loose: 4.52–12.66 ms; packed: 6.11–8.62 ms. About 1.49 MB allocated per operation. | Loose: 2.58–2.81 ms; packed: 2.07–2.13 ms. About 121 KB allocated. |
| Markdown links and parsing | 19.4–21.5 ms for two identical 1 MiB documents and 1,000 links; about 7.67 MB allocated per operation. | 10.90–11.14 ms for the same fixture; about 2.29 MB allocated. |
| Facade check, standard Go suite only | 385–690 ms; 906 module bytes; 336–340 KB allocated in the benchmark process. | 374.0–386.9 ms; 906 module bytes; 336–337 KB allocated in the benchmark process. |
| Facade check with selected file checks | 528–634 ms; 906 module bytes; 632–646 KB allocated in the benchmark process. | 379.7–391.8 ms; 906 module bytes; 503–513 KB allocated in the benchmark process. |
| Retained content after formatting and Markdown parsing | 1,385,040–1,385,616 bytes above the post-GC metadata baseline, with the candidate and Markdown cache kept alive. | 180,384–198,000 bytes above the post-GC metadata baseline, with the same caches kept alive. |
Raw output is in temporary storage at `/tmp/releasekit-performance-baseline.txt`.
The shared machine produced substantial timing variation, especially for packed
@@ -159,20 +159,78 @@ the fixture server.
| Measurement | Baseline observations | Final comparison |
| --- | --- | --- |
| Artifact declarations, two 16 MiB files | 94.3–95.4 ms; 33,554,432 bytes processed; about 68.9 KB allocated. | Pending Stage 11. |
| Declarations plus local verification | 186.7–188.4 ms; 33,554,432 binary bytes per each of two passes; about 136.2 KB allocated. | Pending Stage 11. |
| Facade delivery, fresh uploads | 429–447 ms; 32 MiB binary payload, two files plus manifest; 1.43–1.46 MB allocated. | Pending Stage 11. |
| Facade delivery, matching retry | 309.1–310.0 ms; 32 MiB of local verification plus streamed remote comparisons; 1.21–1.22 MB allocated. | Pending Stage 11. |
| Combined output, 4 KiB and `io.Discard` | 1.82–1.87 μs; 8,256 B and 3 allocations per operation. | Pending Stage 11. |
| Combined output, 8 MiB and `io.Discard` | 2.02–2.36 ms; 16.78 MB and 3 allocations per operation, including the completed capture copy. | Pending Stage 11. |
| Injected version selection | 0.745–0.754 ns; 0 B and 0 allocations. | Pending Stage 11. |
| Repeated real metadata fallback | 1.438–1.474 μs; 1,184 B and 7 allocations per operation. | Pending Stage 11. |
| Artifact declarations, two 16 MiB files | 94.3–95.4 ms; 33,554,432 bytes processed; about 68.9 KB allocated. | 11.4–19.4 μs; 33,554,432 declared bytes; 2,296 B and 33 allocations. Declaration no longer reads or hashes file contents. |
| Declarations plus local verification | 186.7–188.4 ms; 33,554,432 binary bytes per each of two passes; about 136.2 KB allocated. | 94.3–95.3 ms; one 33,554,432-byte verification pass; about 69.7 KB and 58–59 allocations. |
| Facade delivery, fresh uploads | 429–447 ms; 32 MiB binary payload, two files plus manifest; 1.43–1.46 MB allocated. | 331.9–415.3 ms; same payload; 1.38–1.39 MB allocated. |
| Facade delivery, matching retry | 309.1–310.0 ms; 32 MiB of local verification plus streamed remote comparisons; 1.21–1.22 MB allocated. | 210.7–217.5 ms; same local verification and streamed comparisons; 1.15 MB allocated. |
| Combined output, 4 KiB and `io.Discard` | 1.82–1.87 μs; 8,256 B and 3 allocations per operation. | 0.94–1.06 μs; 4,176 B and 2 allocations. |
| Combined output, 8 MiB and `io.Discard` | 2.02–2.36 ms; 16.78 MB and 3 allocations per operation, including the completed capture copy. | 1.73–2.22 ms; about 8.39 MB and 2 allocations. |
| Injected version selection | 0.745–0.754 ns; 0 B and 0 allocations. | 0.754–0.758 ns; 0 B and 0 allocations. |
| Repeated real metadata fallback | 1.438–1.474 μs; 1,184 B and 7 allocations per operation. | 4.59–4.65 ns after cache initialization; 0 B and 0 allocations. Separate initialization: 1.54–2.14 μs; 1,264 B and 9 allocations. |
The delivery observations include local HTTP and filesystem costs specific to
these fixtures. The output component benchmark does not include child startup;
the real delivery benchmark does include HTTP request handling. These results
describe measured workloads and do not establish runtime guarantees.
### Final Comparison Record
The final benchmark run measured commit `42569b4`, before this Stage 11 record
was committed, using Go `go1.26.8 linux/amd64`, Git `2.53.0`, and Linux x86_64
on an Intel Core i5-7260U at 2.20 GHz (`devbench-02`). It used the same
`-run '^$' -bench '^BenchmarkPerformance' -benchmem -count=3` boundaries and
the same temporary fixtures and cache setup as the baseline. Raw final output
is in `/tmp/releasekit-performance-final-stage1.txt`,
`/tmp/releasekit-performance-final-delivery.txt`,
`/tmp/releasekit-performance-final-output.txt`, and
`/tmp/releasekit-performance-final-version.txt`.
The unique-blob batch reader reduced measured reads and formatting from seconds
to tens or hundreds of milliseconds and cut their allocations by roughly an
order of magnitude or more. Formatting repeated object IDs also improved in
both time and allocations because parsed outcomes are reused. Raw repeated-ID
reads regressed: 1,000 loose reads measured 84.8–99.5 ms and 480–483 KB, versus
2.12–3.00 ms and 50.7 KB at baseline; packed reads measured 22.3–22.9 ms versus
3.42–7.75 ms. This workload rereads small object bodies through the batch
session, while regular-file bytes intentionally remain caller-owned and are
not cached by the tree. That is the measured time cost of limiting retained
source bytes; scoped read reuse could be reconsidered if repeated raw reads
become a common workload.
Retained processed content fell from about 1.39 MB to 180–198 KB with the
candidate and parsed Markdown cache kept alive. The Markdown benchmark now
allocates about 2.29 MB instead of 7.67 MB and measured 10.90–11.14 ms versus
19.4–21.5 ms at baseline. The complete offline facade check with selected file
checks measured 379.7–391.8 ms and 503–513 KB in the benchmark process, versus
528–634 ms and 632–646 KB at baseline. The Go-suite-only check remains
374.0–386.9 ms; its real test/compile subprocess is the main complete-check
cost, and benchmark-process allocations exclude subprocess allocations.
Artifact declarations measured 11.0–11.5 µs and 2,296 B for two 16 MiB files.
Declarations plus current local verification measured 92.4–93.3 ms and about
69.7 KB, against 186.7–188.4 ms and 136.2 KB at baseline. The current flow
therefore removes one 32 MiB local read/hash pass while retaining one local
verification before hosted requests. In the prebuilt local-HTTP facade fixture,
fresh uploads measured 331.9–415.3 ms and matching retries 210.7–217.5 ms. A
retry still reads and verifies local assets and compares streamed remote bytes;
those fixture and remote-comparison costs remain in the complete workflow.
Output capture allocations fell from 8,256 B to 4,176 B for 4 KiB, and from
about 16.78 MB to 8.39 MB for 8 MiB. The 8 MiB timing ranges overlap, while one
allocation and the completed-buffer copy are removed. The returned capture
remains caller-owned in memory; a failed output command can also retain its
diagnostic string. Warm runtime fallback access measured 4.59–4.65 ns and zero
allocations after a separate 1.54–2.14 µs, 1,264 B initialization. The
linker-injected selection remains allocation-free.
Some microbenchmark timings varied substantially between repeated runs on this
shared machine; allocations were steadier. The local delivery server and
filesystem are workload fixtures, not a hosted service. These figures
describe only the recorded toolchain and workloads and do not imply numerical
runtime guarantees. Raw repeated-ID read retention and disk-backed large
diagnostics remain follow-up options if their measured workload frequency
justifies their memory and lifecycle tradeoffs.
1. Extend the same benchmark prefix to declaration/preflight processing of two
deterministic 16 MiB artifact files. Measure prebuilt facade `Deliver`
against a local HTTP fixture for fresh uploads and matching retries.
@@ -583,6 +641,15 @@ are recorded, and the worktree is clean. Completion depends on observed behavior
and reviewed diffs; benchmarks establish improvements within their measured
scope without adding numerical runtime guarantees.
**Completion:** Complete. The full offline suite (`go test ./...`) and compiled
facade/dependency check (`go run ./internal/apicheck`) passed. Focused race
validation passed for the runtime fallback, Go output adapter, and process
boundary. Targeted fixtures also cover cancellation and writer-failure causes.
The canonical internal docs describe Git reader lifetime and source ownership,
validation reuse and delivery verification, Go output ownership, and runtime
fallback reuse. Local links and Markdown whitespace were checked. All stages
are marked complete; this results document is retained for review.
## Open Questions
None required for this implementation. The plan selects scoped batch reading,