ConfigKit/docs/roadmap/promptkit-performance-request.md

# PromptKit Request: More Efficient Configuration Construction

Status: Submitted for ConfigKit's consideration; no upstream design or delivery
commitment is assumed.

Date: 2026-10-08.

## Consumer Need

PromptKit uses ConfigKit to assemble configuration for an importable Go LLM
prompting library. Catalog and consumer sources supply backend, profile, and
versioned prompt definitions. ConfigKit owns source composition, inheritance,
typed decoding, resolution, and final configuration publication. PromptKit
interprets the final configuration and captures execution-ready definitions.

We request consideration of lower CPU and allocation costs for this existing
loading workflow, particularly with hundreds of small definitions. The goal is
to benefit consumers through the existing public API, while preserving the
configuration semantics and inspection capabilities they rely on. Acceptance
and the implementation approach remain ConfigKit decisions.

## Why This Matters

PromptKit ordinarily constructs an engine once and reuses its captured
configuration across many calls. Construction is therefore startup work, not a
cost repeated for each prompt. Current indexed prompt/profile selection is
already fast, and this request does not concern upstream inference latency.

Nevertheless, cumulative allocation during construction can matter for large
catalogs, applications creating multiple engines, and repeated configuration
loads. Reducing that work can improve startup and reduce garbage-collection
pressure without requiring every downstream library to own a specialized
configuration loader.

PromptKit retains ConfigKit's inspection data for explicit operator APIs.
Ordinary prompt execution uses captured definitions rather than analyzing
configuration provenance. Inspection remains useful even though it is not on
the normal execution path.

## Observed Workload

PromptKit's committed `internal/configuration/benchmark_test.go` contains
`BenchmarkConfigurationConstruction`. Its `independent` fixture has:

- 128 independently named profiles, each with an endpoint and model;
- 128 independently named prompts, each with version `1`;
- one message per prompt, containing the small template `Hello {{.name}}`;
- a default-profile reference and a text/no-validation output declaration per
  prompt; and
- one in-memory mapping source, with no underlays, imports, inheritance,
  external file reads, credentials, or model calls.

The benchmark reports 256 input and effective entries: 128 profiles plus 128
prompt versions. This is a definition count, not a count of all configuration
fields or metadata paths.

Each timed iteration calls PromptKit's real configuration adapter. That adapter
uses `ConfigKit.LoadLayered` with resolution and validation callbacks, then
captures prompt content, compiled templates, hashes, and execution targets.
Input fixture construction occurs outside the timed iteration. Consequently,
the measurements include downstream adapter work and are not a standalone
ConfigKit benchmark.

Other committed fixtures cover file-backed messages, shared-base profile
inheritance, and four-tier layering. They provide additional downstream cases
for assessing proposed improvements; the principal evidence below uses the
simple independent fixture.

## Measurements And Reproduction

Original PromptKit review baseline:
`de6443f3aef4a66169e323e9f9a5f41594fbface`.

The request preparation repeated the benchmark at PromptKit revision
`eb830bb61346672f0f6c5bce6119b76ccc6d61ac`. The benchmark, configuration adapter,
and `go.mod` are unchanged between these revisions.

Reproduction environment:

- ConfigKit: `gitea.maximumdirect.net/eric/ConfigKit v0.5.0`.
- Toolchain: `go1.26.8`, `linux/amd64`.
- Processor: Intel Core i5-7260U CPU at 2.20 GHz.
- Benchmark process parallelism: 4, as reported by the benchmark suffix.
- Workspace mode disabled; installed toolchain and already-acquired module
  dependencies used without downloads.

From a PromptKit checkout at either recorded revision:

```sh
GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
  go test -mod=readonly -run '^$' \
  -bench '^BenchmarkConfigurationConstruction/independent$' \
  -benchmem -benchtime=500ms -count=5 ./internal/configuration
```

Repeated results:

| Run | Time per construction (ms) | Allocated bytes per construction | Allocations per construction |
| --- | ---: | ---: | ---: |
| 1 | 71.052 | 50,128,254 | 538,582 |
| 2 | 67.740 | 50,139,198 | 538,539 |
| 3 | 66.353 | 50,174,030 | 538,541 |
| 4 | 67.170 | 50,159,959 | 538,542 |
| 5 | 68.209 | 50,167,214 | 538,541 |

These agree with the original review's approximate 69 ms, 50 MB, and 539,000
allocations. They measure cumulative allocation per operation, not retained
heap or a memory leak. Timing depends on the machine and runtime environment.

### Allocation And CPU Profile Observations

A separate two-second profiled run reproduced approximately 68.7 ms,
50.1 MB, and 538,553 allocations per construction. To generate comparable
profiles from the same PromptKit checkout, placing generated artifacts outside
the repository:

```sh
profile_dir=$(mktemp -d)
GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
  go test -mod=readonly -run '^$' \
  -bench '^BenchmarkConfigurationConstruction/independent$' \
  -benchtime=2s -cpuprofile="$profile_dir/config.cpu" \
  -memprofile="$profile_dir/config.mem" -o "$profile_dir/config.test" \
  ./internal/configuration
GOTOOLCHAIN=local go tool pprof -top -alloc_space \
  "$profile_dir/config.test" "$profile_dir/config.mem"
GOTOOLCHAIN=local go tool pprof -top \
  "$profile_dir/config.test" "$profile_dir/config.cpu"
```

The sampled allocation profile prominently attributes work to ConfigKit's
path and metadata machinery. Selected flat allocation shares were:

- `clonePathSegments`: approximately 29.9%.
- `ParsePath`: approximately 4.7%.
- `collectFieldInfo`: approximately 4.3%.
- `collectOrigins`: approximately 3.7%.
- `effectivePathIndex.func1`: approximately 3.7%.

The CPU profile also showed substantial allocation/garbage-collection work and
path formatting (`Path.String`, approximately 17.4% cumulative CPU samples).
These symbol names are profiling evidence, not recommendations for ConfigKit's
internal implementation. Flat allocation and cumulative CPU shares describe
different measures and should not be added together.

The profile supports investigating ConfigKit as a major contributor; it does
not attribute every byte or millisecond of this assembled benchmark to
ConfigKit. A single fixture size also does not establish an asymptotic scaling
defect. Repeated measurements across representative sizes would help assess
scaling and any proposed improvement.

## Requested Outcome

Consider reducing avoidable construction-time work and temporary allocations
for this workload while retaining the capabilities defined by ConfigKit's
[configuration contract](../config.md), including:

- layered sources and sibling inheritance;
- typed decoding and recorded application resolution;
- effective presence, explicit null, and resource/source context;
- provenance, history, and safe operator inspection; and
- established result/query ownership and secret-handling behavior.

An improvement through the existing API would be preferable for PromptKit.
The request does not prescribe a cache, data structure, new public API, or
alternative metadata mode. Any optional API change can be assessed against
the downstream need if ConfigKit determines that one is justified.

Useful evidence of success would be repeatable reductions in CPU or allocation
cost for the recorded fixture and representative related workloads, with
existing semantics intact. No machine-specific threshold, fixed allocation
budget, mandatory complexity guarantee, or additional release gate is requested.

This request can be considered independently of PromptKit's local runtime
optimizations. PromptKit will continue to use ConfigKit's final configuration
handoff and can evaluate adoption of an upstream improvement when available.