Record validation and rendering baselines

This commit is contained in:
2026-10-09 00:11:00 +00:00
parent 0eb009ad5a
commit 05fe13d80b

View File

@@ -2,7 +2,8 @@
Implement the remaining local work in the [performance roadmap](performance.md)
in order. Each sequential stage is scoped for one Luna coding-agent prompt.
All stages below are pending. Update their status and evidence as work lands.
Stage 1 is complete; Stages 2–14 remain pending. Update status and evidence
as each work unit lands.
The completed inspection plan is preserved at
`481e2d4:docs/roadmap/implementation.md` in Git history.
@@ -93,6 +94,11 @@ retention, or maintenance trade-off; the original percentages are not targets.
## Stage 1 — Establish Validation And Captured-Rendering Benchmarks
**Status: Complete (2026-10-09).** Maintained basic-validation, compiled
rendering, and retained-output benchmarks are committed. The benchmark-only
baseline is `0eb009ad5a3248e2a1e12cad9b3a05906d6b0850`; no production behavior
changed.
**Outcome:** Reproducible baselines for roadmap items 2 and 3; no production
change in this stage.
@@ -509,10 +515,66 @@ rather than rerunning merely to create the final commit.
## Execution Evidence
Implementation has not started. Record benchmark-only baseline revisions,
comparisons, ownership analysis, conditional decisions, and final verification
here as the stages are completed. Keep the feature roadmap focused on target
state and progress rather than duplicating the staged procedure.
### Validation And Compiled-Rendering Baseline
The benchmark-only baseline is commit
`0eb009ad5a3248e2a1e12cad9b3a05906d6b0850` (`Add validation and rendering
benchmarks`), on top of planning baseline
`3fde1f6cbbc2cc5b3dc17779ba44e0af6cd8a836`. The measured source is unchanged
from that commit. No production files changed.
Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark parallelism
4; `go1.26.8`, module directive `go 1.26.0`, ConfigKit `v0.5.0`, JSON Schema
`v6.0.2`, and the backend modules `v3.20260930.0`.
Command (five 500 ms samples per benchmark, without the race detector):
```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
go test -mod=readonly -run '^$' \
-bench 'Benchmark(BasicValidation|CompiledRendering|RetainedRenderedOutput|GoRendererArtifactReferences)$' \
-benchmem -benchtime=500ms -count=5 \
./internal/validate ./internal/prompt
```
Times below list all five `ns/op` samples in command order, followed by median
and range. Allocation figures are `B/op; allocs/op` (ranges where they varied).
| Benchmark case | `ns/op` samples (median; range) | Allocations |
| --- | --- | --- |
| Basic validation, short text | 46.59, 44.98, 43.31, 46.23, 43.19 (44.98; 43.19–46.59) | 0; 0 |
| Basic validation, 1 MiB nonempty | 193859, 329884, 418896, 386126, 378173 (378173; 193859–418896) | 1048576–1048577; 1 |
| Basic validation, ~1 MiB whitespace | 1112498, 1100439, 1175913, 1290678, 1108746 (1112498; 1100439–1290678) | 1048592–1048595; 2 |
| Basic validation, Unicode whitespace | 170.8, 164.2, 158.6, 159.2, 165.2 (164.2; 158.6–170.8) | 16; 1 |
| Compiled rendering, small message | 3931, 8488, 2041, 1902, 2181 (2181; 1902–8488) | 1632; 20 |
| Compiled rendering, 256 KiB artifact message | 285812, 380128, 655925, 664387, 537684 (537684; 285812–664387) | 1050656–1050713; 25 |
| Compiled rendering, repeated across session/messages | 1870972, 1362368, 1142268, 1073474, 1173588 (1173588; 1073474–1870972) | 3945983–3946034; 74 |
| `NewGoRenderer` fallback, one artifact reference | 407143, 629149, 522549, 562200, 470393 (522549; 407143–629149) | 1056409–1056477; 69–70 |
| `NewGoRenderer` fallback, repeated references | 1439883, 1535813, 1300176, 1541795, 1416267 (1439883; 1300176–1541795) | 2897171–2897255; 157 |
| Retained output, four 256 KiB rendered results | 398753, 392206, 384503, 380386, 406691 (392206; 380386–406691) | 1050702–1050724; 25–26 |
For the retained-output benchmark, post-GC live-heap deltas above the same
setup-only fixture were 1048272, 1050176, 1046032, 1049856, and 1049016 bytes
(median 1049016; range 1046032–1050176). The batch contains four separately
rendered 256 KiB messages. This is retained-heap evidence, not an inference
from `B/op`; it includes result structure and allocator/runtime noise. The
initial exploratory run exposed cross-count sink retention in the measurement
baseline; the benchmark now clears prior sinks before measuring setup, and the
recorded values are from the corrected committed fixture.
The short validator case allocates nothing, while the 1 MiB nonempty body has
about one response-sized allocation; the whitespace-only case also retains a
small failure-result allocation. Compiled rendering provides separate small,
artifact-sized, and repeated-reference baselines, with the existing parsing
fallback reported for context. Several render timings show wide ranges on this
machine, so the median/range—not any single run—are the comparison baseline.
The retained-output results consistently remain near 1 MiB for four outputs.
Focused verification passed: `go test -mod=readonly ./internal/validate
./internal/prompt`. Record later before/after comparisons, ownership analysis,
conditional decisions, and final verification in this section as the remaining
stages complete. Keep the feature roadmap focused on target state and progress
rather than duplicating the staged procedure.
## Open Questions