Record validation and rendering baselines
This commit is contained in:
@@ -2,7 +2,8 @@
|
||||
|
||||
Implement the remaining local work in the [performance roadmap](performance.md)
|
||||
in order. Each sequential stage is scoped for one Luna coding-agent prompt.
|
||||
All stages below are pending. Update their status and evidence as work lands.
|
||||
Stage 1 is complete; Stages 2–14 remain pending. Update status and evidence
|
||||
as each work unit lands.
|
||||
The completed inspection plan is preserved at
|
||||
`481e2d4:docs/roadmap/implementation.md` in Git history.
|
||||
|
||||
@@ -93,6 +94,11 @@ retention, or maintenance trade-off; the original percentages are not targets.
|
||||
|
||||
## Stage 1 — Establish Validation And Captured-Rendering Benchmarks
|
||||
|
||||
**Status: Complete (2026-10-09).** Maintained basic-validation, compiled
|
||||
rendering, and retained-output benchmarks are committed. The benchmark-only
|
||||
baseline is `0eb009ad5a3248e2a1e12cad9b3a05906d6b0850`; no production behavior
|
||||
changed.
|
||||
|
||||
**Outcome:** Reproducible baselines for roadmap items 2 and 3; no production
|
||||
change in this stage.
|
||||
|
||||
@@ -509,10 +515,66 @@ rather than rerunning merely to create the final commit.
|
||||
|
||||
## Execution Evidence
|
||||
|
||||
Implementation has not started. Record benchmark-only baseline revisions,
|
||||
comparisons, ownership analysis, conditional decisions, and final verification
|
||||
here as the stages are completed. Keep the feature roadmap focused on target
|
||||
state and progress rather than duplicating the staged procedure.
|
||||
### Validation And Compiled-Rendering Baseline
|
||||
|
||||
The benchmark-only baseline is commit
|
||||
`0eb009ad5a3248e2a1e12cad9b3a05906d6b0850` (`Add validation and rendering
|
||||
benchmarks`), on top of planning baseline
|
||||
`3fde1f6cbbc2cc5b3dc17779ba44e0af6cd8a836`. The measured source is unchanged
|
||||
from that commit. No production files changed.
|
||||
|
||||
Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark parallelism
|
||||
4; `go1.26.8`, module directive `go 1.26.0`, ConfigKit `v0.5.0`, JSON Schema
|
||||
`v6.0.2`, and the backend modules `v3.20260930.0`.
|
||||
|
||||
Command (five 500 ms samples per benchmark, without the race detector):
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \
|
||||
go test -mod=readonly -run '^$' \
|
||||
-bench 'Benchmark(BasicValidation|CompiledRendering|RetainedRenderedOutput|GoRendererArtifactReferences)$' \
|
||||
-benchmem -benchtime=500ms -count=5 \
|
||||
./internal/validate ./internal/prompt
|
||||
```
|
||||
|
||||
Times below list all five `ns/op` samples in command order, followed by median
|
||||
and range. Allocation figures are `B/op; allocs/op` (ranges where they varied).
|
||||
|
||||
| Benchmark case | `ns/op` samples (median; range) | Allocations |
|
||||
| --- | --- | --- |
|
||||
| Basic validation, short text | 46.59, 44.98, 43.31, 46.23, 43.19 (44.98; 43.19–46.59) | 0; 0 |
|
||||
| Basic validation, 1 MiB nonempty | 193859, 329884, 418896, 386126, 378173 (378173; 193859–418896) | 1048576–1048577; 1 |
|
||||
| Basic validation, ~1 MiB whitespace | 1112498, 1100439, 1175913, 1290678, 1108746 (1112498; 1100439–1290678) | 1048592–1048595; 2 |
|
||||
| Basic validation, Unicode whitespace | 170.8, 164.2, 158.6, 159.2, 165.2 (164.2; 158.6–170.8) | 16; 1 |
|
||||
| Compiled rendering, small message | 3931, 8488, 2041, 1902, 2181 (2181; 1902–8488) | 1632; 20 |
|
||||
| Compiled rendering, 256 KiB artifact message | 285812, 380128, 655925, 664387, 537684 (537684; 285812–664387) | 1050656–1050713; 25 |
|
||||
| Compiled rendering, repeated across session/messages | 1870972, 1362368, 1142268, 1073474, 1173588 (1173588; 1073474–1870972) | 3945983–3946034; 74 |
|
||||
| `NewGoRenderer` fallback, one artifact reference | 407143, 629149, 522549, 562200, 470393 (522549; 407143–629149) | 1056409–1056477; 69–70 |
|
||||
| `NewGoRenderer` fallback, repeated references | 1439883, 1535813, 1300176, 1541795, 1416267 (1439883; 1300176–1541795) | 2897171–2897255; 157 |
|
||||
| Retained output, four 256 KiB rendered results | 398753, 392206, 384503, 380386, 406691 (392206; 380386–406691) | 1050702–1050724; 25–26 |
|
||||
|
||||
For the retained-output benchmark, post-GC live-heap deltas above the same
|
||||
setup-only fixture were 1048272, 1050176, 1046032, 1049856, and 1049016 bytes
|
||||
(median 1049016; range 1046032–1050176). The batch contains four separately
|
||||
rendered 256 KiB messages. This is retained-heap evidence, not an inference
|
||||
from `B/op`; it includes result structure and allocator/runtime noise. The
|
||||
initial exploratory run exposed cross-count sink retention in the measurement
|
||||
baseline; the benchmark now clears prior sinks before measuring setup, and the
|
||||
recorded values are from the corrected committed fixture.
|
||||
|
||||
The short validator case allocates nothing, while the 1 MiB nonempty body has
|
||||
about one response-sized allocation; the whitespace-only case also retains a
|
||||
small failure-result allocation. Compiled rendering provides separate small,
|
||||
artifact-sized, and repeated-reference baselines, with the existing parsing
|
||||
fallback reported for context. Several render timings show wide ranges on this
|
||||
machine, so the median/range—not any single run—are the comparison baseline.
|
||||
The retained-output results consistently remain near 1 MiB for four outputs.
|
||||
|
||||
Focused verification passed: `go test -mod=readonly ./internal/validate
|
||||
./internal/prompt`. Record later before/after comparisons, ownership analysis,
|
||||
conditional decisions, and final verification in this section as the remaining
|
||||
stages complete. Keep the feature roadmap focused on target state and progress
|
||||
rather than duplicating the staged procedure.
|
||||
|
||||
## Open Questions
|
||||
|
||||
|
||||
Reference in New Issue
Block a user