From d53021ea2c26b7bccaca9391370ba0d0f6d45cad Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Thu, 8 Oct 2026 23:50:02 +0000 Subject: [PATCH] Request more efficient configuration loading for PromptKit --- docs/roadmap/promptkit-performance-request.md | 176 ++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 docs/roadmap/promptkit-performance-request.md diff --git a/docs/roadmap/promptkit-performance-request.md b/docs/roadmap/promptkit-performance-request.md new file mode 100644 index 0000000..2546be3 --- /dev/null +++ b/docs/roadmap/promptkit-performance-request.md @@ -0,0 +1,176 @@ +# PromptKit Request: More Efficient Configuration Construction + +Status: Submitted for ConfigKit's consideration; no upstream design or delivery +commitment is assumed. + +Date: 2026-10-08. + +## Consumer Need + +PromptKit uses ConfigKit to assemble configuration for an importable Go LLM +prompting library. Catalog and consumer sources supply backend, profile, and +versioned prompt definitions. ConfigKit owns source composition, inheritance, +typed decoding, resolution, and final configuration publication. PromptKit +interprets the final configuration and captures execution-ready definitions. + +We request consideration of lower CPU and allocation costs for this existing +loading workflow, particularly with hundreds of small definitions. The goal is +to benefit consumers through the existing public API, while preserving the +configuration semantics and inspection capabilities they rely on. Acceptance +and the implementation approach remain ConfigKit decisions. + +## Why This Matters + +PromptKit ordinarily constructs an engine once and reuses its captured +configuration across many calls. Construction is therefore startup work, not a +cost repeated for each prompt. Current indexed prompt/profile selection is +already fast, and this request does not concern upstream inference latency. + +Nevertheless, cumulative allocation during construction can matter for large +catalogs, applications creating multiple engines, and repeated configuration +loads. Reducing that work can improve startup and reduce garbage-collection +pressure without requiring every downstream library to own a specialized +configuration loader. + +PromptKit retains ConfigKit's inspection data for explicit operator APIs. +Ordinary prompt execution uses captured definitions rather than analyzing +configuration provenance. Inspection remains useful even though it is not on +the normal execution path. + +## Observed Workload + +PromptKit's committed `internal/configuration/benchmark_test.go` contains +`BenchmarkConfigurationConstruction`. Its `independent` fixture has: + +- 128 independently named profiles, each with an endpoint and model; +- 128 independently named prompts, each with version `1`; +- one message per prompt, containing the small template `Hello {{.name}}`; +- a default-profile reference and a text/no-validation output declaration per + prompt; and +- one in-memory mapping source, with no underlays, imports, inheritance, + external file reads, credentials, or model calls. + +The benchmark reports 256 input and effective entries: 128 profiles plus 128 +prompt versions. This is a definition count, not a count of all configuration +fields or metadata paths. + +Each timed iteration calls PromptKit's real configuration adapter. That adapter +uses `ConfigKit.LoadLayered` with resolution and validation callbacks, then +captures prompt content, compiled templates, hashes, and execution targets. +Input fixture construction occurs outside the timed iteration. Consequently, +the measurements include downstream adapter work and are not a standalone +ConfigKit benchmark. + +Other committed fixtures cover file-backed messages, shared-base profile +inheritance, and four-tier layering. They provide additional downstream cases +for assessing proposed improvements; the principal evidence below uses the +simple independent fixture. + +## Measurements And Reproduction + +Original PromptKit review baseline: +`de6443f3aef4a66169e323e9f9a5f41594fbface`. + +The request preparation repeated the benchmark at PromptKit revision +`eb830bb61346672f0f6c5bce6119b76ccc6d61ac`. The benchmark, configuration adapter, +and `go.mod` are unchanged between these revisions. + +Reproduction environment: + +- ConfigKit: `gitea.maximumdirect.net/eric/ConfigKit v0.5.0`. +- Toolchain: `go1.26.8`, `linux/amd64`. +- Processor: Intel Core i5-7260U CPU at 2.20 GHz. +- Benchmark process parallelism: 4, as reported by the benchmark suffix. +- Workspace mode disabled; installed toolchain and already-acquired module + dependencies used without downloads. + +From a PromptKit checkout at either recorded revision: + +```sh +GOTOOLCHAIN=local GOWORK=off GOPROXY=off \ + go test -mod=readonly -run '^$' \ + -bench '^BenchmarkConfigurationConstruction/independent$' \ + -benchmem -benchtime=500ms -count=5 ./internal/configuration +``` + +Repeated results: + +| Run | Time per construction (ms) | Allocated bytes per construction | Allocations per construction | +| --- | ---: | ---: | ---: | +| 1 | 71.052 | 50,128,254 | 538,582 | +| 2 | 67.740 | 50,139,198 | 538,539 | +| 3 | 66.353 | 50,174,030 | 538,541 | +| 4 | 67.170 | 50,159,959 | 538,542 | +| 5 | 68.209 | 50,167,214 | 538,541 | + +These agree with the original review's approximate 69 ms, 50 MB, and 539,000 +allocations. They measure cumulative allocation per operation, not retained +heap or a memory leak. Timing depends on the machine and runtime environment. + +### Allocation And CPU Profile Observations + +A separate two-second profiled run reproduced approximately 68.7 ms, +50.1 MB, and 538,553 allocations per construction. To generate comparable +profiles from the same PromptKit checkout, placing generated artifacts outside +the repository: + +```sh +profile_dir=$(mktemp -d) +GOTOOLCHAIN=local GOWORK=off GOPROXY=off \ + go test -mod=readonly -run '^$' \ + -bench '^BenchmarkConfigurationConstruction/independent$' \ + -benchtime=2s -cpuprofile="$profile_dir/config.cpu" \ + -memprofile="$profile_dir/config.mem" -o "$profile_dir/config.test" \ + ./internal/configuration +GOTOOLCHAIN=local go tool pprof -top -alloc_space \ + "$profile_dir/config.test" "$profile_dir/config.mem" +GOTOOLCHAIN=local go tool pprof -top \ + "$profile_dir/config.test" "$profile_dir/config.cpu" +``` + +The sampled allocation profile prominently attributes work to ConfigKit's +path and metadata machinery. Selected flat allocation shares were: + +- `clonePathSegments`: approximately 29.9%. +- `ParsePath`: approximately 4.7%. +- `collectFieldInfo`: approximately 4.3%. +- `collectOrigins`: approximately 3.7%. +- `effectivePathIndex.func1`: approximately 3.7%. + +The CPU profile also showed substantial allocation/garbage-collection work and +path formatting (`Path.String`, approximately 17.4% cumulative CPU samples). +These symbol names are profiling evidence, not recommendations for ConfigKit's +internal implementation. Flat allocation and cumulative CPU shares describe +different measures and should not be added together. + +The profile supports investigating ConfigKit as a major contributor; it does +not attribute every byte or millisecond of this assembled benchmark to +ConfigKit. A single fixture size also does not establish an asymptotic scaling +defect. Repeated measurements across representative sizes would help assess +scaling and any proposed improvement. + +## Requested Outcome + +Consider reducing avoidable construction-time work and temporary allocations +for this workload while retaining the capabilities defined by ConfigKit's +[configuration contract](../config.md), including: + +- layered sources and sibling inheritance; +- typed decoding and recorded application resolution; +- effective presence, explicit null, and resource/source context; +- provenance, history, and safe operator inspection; and +- established result/query ownership and secret-handling behavior. + +An improvement through the existing API would be preferable for PromptKit. +The request does not prescribe a cache, data structure, new public API, or +alternative metadata mode. Any optional API change can be assessed against +the downstream need if ConfigKit determines that one is justified. + +Useful evidence of success would be repeatable reductions in CPU or allocation +cost for the recorded fixture and representative related workloads, with +existing semantics intact. No machine-specific threshold, fixed allocation +budget, mandatory complexity guarantee, or additional release gate is requested. + +This request can be considered independently of PromptKit's local runtime +optimizations. PromptKit will continue to use ConfigKit's final configuration +handoff and can evaluate adoption of an upstream improvement when available.