PromptKit Request: More Efficient Configuration Construction

Status: Submitted for ConfigKit's consideration; no upstream design or delivery commitment is assumed.

Date: 2026-10-08.

Consumer Need

PromptKit uses ConfigKit to assemble configuration for an importable Go LLM prompting library. Catalog and consumer sources supply backend, profile, and versioned prompt definitions. ConfigKit owns source composition, inheritance, typed decoding, resolution, and final configuration publication. PromptKit interprets the final configuration and captures execution-ready definitions.

We request consideration of lower CPU and allocation costs for this existing loading workflow, particularly with hundreds of small definitions. The goal is to benefit consumers through the existing public API, while preserving the configuration semantics and inspection capabilities they rely on. Acceptance and the implementation approach remain ConfigKit decisions.

Why This Matters

PromptKit ordinarily constructs an engine once and reuses its captured configuration across many calls. Construction is therefore startup work, not a cost repeated for each prompt. Current indexed prompt/profile selection is already fast, and this request does not concern upstream inference latency.

Nevertheless, cumulative allocation during construction can matter for large catalogs, applications creating multiple engines, and repeated configuration loads. Reducing that work can improve startup and reduce garbage-collection pressure without requiring every downstream library to own a specialized configuration loader.

PromptKit retains ConfigKit's inspection data for explicit operator APIs. Ordinary prompt execution uses captured definitions rather than analyzing configuration provenance. Inspection remains useful even though it is not on the normal execution path.

Observed Workload

PromptKit's committed internal/configuration/benchmark_test.go contains BenchmarkConfigurationConstruction. Its independent fixture has:

128 independently named profiles, each with an endpoint and model;
128 independently named prompts, each with version 1;
one message per prompt, containing the small template Hello {{.name}};
a default-profile reference and a text/no-validation output declaration per prompt; and
one in-memory mapping source, with no underlays, imports, inheritance, external file reads, credentials, or model calls.

The benchmark reports 256 input and effective entries: 128 profiles plus 128 prompt versions. This is a definition count, not a count of all configuration fields or metadata paths.

Each timed iteration calls PromptKit's real configuration adapter. That adapter uses ConfigKit.LoadLayered with resolution and validation callbacks, then captures prompt content, compiled templates, hashes, and execution targets. Input fixture construction occurs outside the timed iteration. Consequently, the measurements include downstream adapter work and are not a standalone ConfigKit benchmark.

Other committed fixtures cover file-backed messages, shared-base profile inheritance, and four-tier layering. They provide additional downstream cases for assessing proposed improvements; the principal evidence below uses the simple independent fixture.

Measurements And Reproduction

Original PromptKit review baseline: de6443f3aef4a66169e323e9f9a5f41594fbface.

The request preparation repeated the benchmark at PromptKit revision eb830bb61346672f0f6c5bce6119b76ccc6d61ac. The benchmark, configuration adapter, and go.mod are unchanged between these revisions.

Reproduction environment:

ConfigKit: gitea.maximumdirect.net/eric/ConfigKit v0.5.0.
Toolchain: go1.26.8, linux/amd64.
Processor: Intel Core i5-7260U CPU at 2.20 GHz.
Benchmark process parallelism: 4, as reported by the benchmark suffix.
Workspace mode disabled; installed toolchain and already-acquired module dependencies used without downloads.

From a PromptKit checkout at either recorded revision:

GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
  go test -mod=readonly -run '^$' \
  -bench '^BenchmarkConfigurationConstruction/independent$' \
  -benchmem -benchtime=500ms -count=5 ./internal/configuration

Repeated results:

Run	Time per construction (ms)	Allocated bytes per construction	Allocations per construction
1	71.052	50,128,254	538,582
2	67.740	50,139,198	538,539
3	66.353	50,174,030	538,541
4	67.170	50,159,959	538,542
5	68.209	50,167,214	538,541

These agree with the original review's approximate 69 ms, 50 MB, and 539,000 allocations. They measure cumulative allocation per operation, not retained heap or a memory leak. Timing depends on the machine and runtime environment.

Allocation And CPU Profile Observations

A separate two-second profiled run reproduced approximately 68.7 ms, 50.1 MB, and 538,553 allocations per construction. To generate comparable profiles from the same PromptKit checkout, placing generated artifacts outside the repository:

profile_dir=$(mktemp -d)
GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
  go test -mod=readonly -run '^$' \
  -bench '^BenchmarkConfigurationConstruction/independent$' \
  -benchtime=2s -cpuprofile="$profile_dir/config.cpu" \
  -memprofile="$profile_dir/config.mem" -o "$profile_dir/config.test" \
  ./internal/configuration
GOTOOLCHAIN=local go tool pprof -top -alloc_space \
  "$profile_dir/config.test" "$profile_dir/config.mem"
GOTOOLCHAIN=local go tool pprof -top \
  "$profile_dir/config.test" "$profile_dir/config.cpu"

The sampled allocation profile prominently attributes work to ConfigKit's path and metadata machinery. Selected flat allocation shares were:

clonePathSegments: approximately 29.9%.
ParsePath: approximately 4.7%.
collectFieldInfo: approximately 4.3%.
collectOrigins: approximately 3.7%.
effectivePathIndex.func1: approximately 3.7%.

The CPU profile also showed substantial allocation/garbage-collection work and path formatting (Path.String, approximately 17.4% cumulative CPU samples). These symbol names are profiling evidence, not recommendations for ConfigKit's internal implementation. Flat allocation and cumulative CPU shares describe different measures and should not be added together.

The profile supports investigating ConfigKit as a major contributor; it does not attribute every byte or millisecond of this assembled benchmark to ConfigKit. A single fixture size also does not establish an asymptotic scaling defect. Repeated measurements across representative sizes would help assess scaling and any proposed improvement.

Requested Outcome

Consider reducing avoidable construction-time work and temporary allocations for this workload while retaining the capabilities defined by ConfigKit's configuration contract, including:

layered sources and sibling inheritance;
typed decoding and recorded application resolution;
effective presence, explicit null, and resource/source context;
provenance, history, and safe operator inspection; and
established result/query ownership and secret-handling behavior.

An improvement through the existing API would be preferable for PromptKit. The request does not prescribe a cache, data structure, new public API, or alternative metadata mode. Any optional API change can be assessed against the downstream need if ConfigKit determines that one is justified.

Useful evidence of success would be repeatable reductions in CPU or allocation cost for the recorded fixture and representative related workloads, with existing semantics intact. No machine-specific threshold, fixed allocation budget, mandatory complexity guarantee, or additional release gate is requested.

This request can be considered independently of PromptKit's local runtime optimizations. PromptKit will continue to use ConfigKit's final configuration handoff and can evaluate adoption of an upstream improvement when available.

7.9 KiB Raw Blame History