Record validation performance baseline

This commit is contained in:
2026-10-09 01:02:05 +00:00
parent ca571aca4f
commit 29e1bab228
6 changed files with 457 additions and 9 deletions

View File

@@ -1,6 +1,6 @@
# Computational Efficiency Implementation Plan
**Status:** Ready for implementation; all stages pending.
**Status:** Stage 1 complete; Stages 2–11 pending.
## Purpose And Baseline
@@ -51,6 +51,50 @@ initialization mechanism.
**Owners:** Benchmark files in `internal/validation` and root facade fixtures.
**Completion:** Complete. Measurement-only benchmarks cover tree metadata,
content reads and formatting for 1,000 unique or repeated-ID Go blobs in loose
and packed repositories, repeated links to identical 1 MiB Markdown documents,
retained content after processing, and a real offline facade `Check`. Fixtures
verify committed bytes, unique/repeated object IDs, link counts, and Git pack
state. The facade fixture selects Go formatting, local-replacement inspection,
and Markdown while using test-owned module and build caches.
### Baseline Measurement Record
The baseline is commit `ca571aca4f2fd43798174a6d0d7c5a588b5347c9`, before adding
measurement support. Results use Go `go1.26.8 linux/amd64`, Git `2.53.0`, and a
Linux x86_64 Intel Core i5-7260U at 2.20 GHz. The command was
`go test ./internal/validation . -run '^$' -bench '^BenchmarkPerformance' -benchmem -count=3`.
Each candidate-content operation used a fresh empty content cache with tree
metadata prepared outside its timer. The loose fixtures had loose objects and
no pack; packed fixtures were checked after `git gc --aggressive --prune=now`.
The unique Go fixture contains 1,000 distinct, formatted files totaling 49,000
bytes. The repeated-ID fixture contains 1,000 paths sharing one 15-byte blob.
The Markdown fixture contains two identical 1 MiB documents, a 47,780-byte
index with 1,000 repeated links, and 2,144,932 bytes processed per check. The
complete-check module contains 906 bytes of Go/module input. Its `GOMODCACHE`
and `GOCACHE` are test-owned temporary directories, and module network access
is disabled; the build cache is warmed before timing.
| Measurement | Baseline observations | Final comparison |
| --- | --- | --- |
| Tree metadata, 1,000 entries | Loose: 27.5–28.2 ms; packed: 25.0–28.1 ms. About 1.28 MB allocated per operation. | Pending Stage 11. |
| Unique-blob reads, 49,000 bytes | Loose: 1.86–1.96 s; packed: 2.57–3.08 s. About 34.57 MB allocated per operation. | Pending Stage 11. |
| Repeated-ID reads, 15,000 bytes across 1,000 paths | Loose: 2.12–3.00 ms; packed: 3.42–7.75 ms. About 50.7 KB allocated per operation. | Pending Stage 11. |
| Formatting, 1,000 unique files | Loose: 1.87–2.94 s; packed: 2.12–4.55 s. About 36.9 MB allocated per operation. | Pending Stage 11. |
| Formatting, 1,000 paths sharing one object | Loose: 4.52–12.66 ms; packed: 6.11–8.62 ms. About 1.49 MB allocated per operation. | Pending Stage 11. |
| Markdown links and parsing | 19.4–21.5 ms for two identical 1 MiB documents and 1,000 links; about 7.67 MB allocated per operation. | Pending Stage 11. |
| Facade check, standard Go suite only | 385–690 ms; 906 module bytes; 336–340 KB allocated in the benchmark process. | Pending Stage 11. |
| Facade check with selected file checks | 528–634 ms; 906 module bytes; 632–646 KB allocated in the benchmark process. | Pending Stage 11. |
| Retained content after formatting and Markdown parsing | 1,385,040–1,385,616 bytes above the post-GC metadata baseline, with the candidate and Markdown cache kept alive. | Pending Stage 11. |
Raw output is in temporary storage at `/tmp/releasekit-performance-baseline.txt`.
The shared machine produced substantial timing variation, especially for packed
content and the full check; allocations and retained-byte observations were
more stable. `B/op` for facade checks measures allocations in the benchmark
process and does not include allocations inside its Go subprocess. These are
workload-specific component measurements, not runtime guarantees.
1. Add benchmarks named with the `BenchmarkPerformance` prefix. Reuse local Git
fixture patterns; adapt shared helper parameters to `testing.TB` only where
a benchmark needs them.