Record validation performance baseline
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Computational Efficiency Implementation Plan
|
||||
|
||||
**Status:** Ready for implementation; all stages pending.
|
||||
**Status:** Stage 1 complete; Stages 2–11 pending.
|
||||
|
||||
## Purpose And Baseline
|
||||
|
||||
@@ -51,6 +51,50 @@ initialization mechanism.
|
||||
|
||||
**Owners:** Benchmark files in `internal/validation` and root facade fixtures.
|
||||
|
||||
**Completion:** Complete. Measurement-only benchmarks cover tree metadata,
|
||||
content reads and formatting for 1,000 unique or repeated-ID Go blobs in loose
|
||||
and packed repositories, repeated links to identical 1 MiB Markdown documents,
|
||||
retained content after processing, and a real offline facade `Check`. Fixtures
|
||||
verify committed bytes, unique/repeated object IDs, link counts, and Git pack
|
||||
state. The facade fixture selects Go formatting, local-replacement inspection,
|
||||
and Markdown while using test-owned module and build caches.
|
||||
|
||||
### Baseline Measurement Record
|
||||
|
||||
The baseline is commit `ca571aca4f2fd43798174a6d0d7c5a588b5347c9`, before adding
|
||||
measurement support. Results use Go `go1.26.8 linux/amd64`, Git `2.53.0`, and a
|
||||
Linux x86_64 Intel Core i5-7260U at 2.20 GHz. The command was
|
||||
`go test ./internal/validation . -run '^$' -bench '^BenchmarkPerformance' -benchmem -count=3`.
|
||||
Each candidate-content operation used a fresh empty content cache with tree
|
||||
metadata prepared outside its timer. The loose fixtures had loose objects and
|
||||
no pack; packed fixtures were checked after `git gc --aggressive --prune=now`.
|
||||
The unique Go fixture contains 1,000 distinct, formatted files totaling 49,000
|
||||
bytes. The repeated-ID fixture contains 1,000 paths sharing one 15-byte blob.
|
||||
The Markdown fixture contains two identical 1 MiB documents, a 47,780-byte
|
||||
index with 1,000 repeated links, and 2,144,932 bytes processed per check. The
|
||||
complete-check module contains 906 bytes of Go/module input. Its `GOMODCACHE`
|
||||
and `GOCACHE` are test-owned temporary directories, and module network access
|
||||
is disabled; the build cache is warmed before timing.
|
||||
|
||||
| Measurement | Baseline observations | Final comparison |
|
||||
| --- | --- | --- |
|
||||
| Tree metadata, 1,000 entries | Loose: 27.5–28.2 ms; packed: 25.0–28.1 ms. About 1.28 MB allocated per operation. | Pending Stage 11. |
|
||||
| Unique-blob reads, 49,000 bytes | Loose: 1.86–1.96 s; packed: 2.57–3.08 s. About 34.57 MB allocated per operation. | Pending Stage 11. |
|
||||
| Repeated-ID reads, 15,000 bytes across 1,000 paths | Loose: 2.12–3.00 ms; packed: 3.42–7.75 ms. About 50.7 KB allocated per operation. | Pending Stage 11. |
|
||||
| Formatting, 1,000 unique files | Loose: 1.87–2.94 s; packed: 2.12–4.55 s. About 36.9 MB allocated per operation. | Pending Stage 11. |
|
||||
| Formatting, 1,000 paths sharing one object | Loose: 4.52–12.66 ms; packed: 6.11–8.62 ms. About 1.49 MB allocated per operation. | Pending Stage 11. |
|
||||
| Markdown links and parsing | 19.4–21.5 ms for two identical 1 MiB documents and 1,000 links; about 7.67 MB allocated per operation. | Pending Stage 11. |
|
||||
| Facade check, standard Go suite only | 385–690 ms; 906 module bytes; 336–340 KB allocated in the benchmark process. | Pending Stage 11. |
|
||||
| Facade check with selected file checks | 528–634 ms; 906 module bytes; 632–646 KB allocated in the benchmark process. | Pending Stage 11. |
|
||||
| Retained content after formatting and Markdown parsing | 1,385,040–1,385,616 bytes above the post-GC metadata baseline, with the candidate and Markdown cache kept alive. | Pending Stage 11. |
|
||||
|
||||
Raw output is in temporary storage at `/tmp/releasekit-performance-baseline.txt`.
|
||||
The shared machine produced substantial timing variation, especially for packed
|
||||
content and the full check; allocations and retained-byte observations were
|
||||
more stable. `B/op` for facade checks measures allocations in the benchmark
|
||||
process and does not include allocations inside its Go subprocess. These are
|
||||
workload-specific component measurements, not runtime guarantees.
|
||||
|
||||
1. Add benchmarks named with the `BenchmarkPerformance` prefix. Reuse local Git
|
||||
fixture patterns; adapt shared helper parameters to `testing.TB` only where
|
||||
a benchmark needs them.
|
||||
|
||||
Reference in New Issue
Block a user