Request more efficient configuration loading for PromptKit

docs/roadmap/promptkit-performance-request.md

@@ -0,0 +1,176 @@
+# PromptKit Request: More Efficient Configuration Construction
+
+Status: Submitted for ConfigKit's consideration; no upstream design or delivery
+commitment is assumed.
+
+Date: 2026-10-08.
+
+## Consumer Need
+
+PromptKit uses ConfigKit to assemble configuration for an importable Go LLM
+prompting library. Catalog and consumer sources supply backend, profile, and
+versioned prompt definitions. ConfigKit owns source composition, inheritance,
+typed decoding, resolution, and final configuration publication. PromptKit
+interprets the final configuration and captures execution-ready definitions.
+
+We request consideration of lower CPU and allocation costs for this existing
+loading workflow, particularly with hundreds of small definitions. The goal is
+to benefit consumers through the existing public API, while preserving the
+configuration semantics and inspection capabilities they rely on. Acceptance
+and the implementation approach remain ConfigKit decisions.
+
+## Why This Matters
+
+PromptKit ordinarily constructs an engine once and reuses its captured
+configuration across many calls. Construction is therefore startup work, not a
+cost repeated for each prompt. Current indexed prompt/profile selection is
+already fast, and this request does not concern upstream inference latency.
+
+Nevertheless, cumulative allocation during construction can matter for large
+catalogs, applications creating multiple engines, and repeated configuration
+loads. Reducing that work can improve startup and reduce garbage-collection
+pressure without requiring every downstream library to own a specialized
+configuration loader.
+
+PromptKit retains ConfigKit's inspection data for explicit operator APIs.
+Ordinary prompt execution uses captured definitions rather than analyzing
+configuration provenance. Inspection remains useful even though it is not on
+the normal execution path.
+
+## Observed Workload
+
+PromptKit's committed `internal/configuration/benchmark_test.go` contains
+`BenchmarkConfigurationConstruction`. Its `independent` fixture has:
+
+- 128 independently named profiles, each with an endpoint and model;
+- 128 independently named prompts, each with version `1`;
+- one message per prompt, containing the small template `Hello {{.name}}`;
+- a default-profile reference and a text/no-validation output declaration per
+  prompt; and
+- one in-memory mapping source, with no underlays, imports, inheritance,
+  external file reads, credentials, or model calls.
+
+The benchmark reports 256 input and effective entries: 128 profiles plus 128
+prompt versions. This is a definition count, not a count of all configuration
+fields or metadata paths.
+
+Each timed iteration calls PromptKit's real configuration adapter. That adapter
+uses `ConfigKit.LoadLayered` with resolution and validation callbacks, then
+captures prompt content, compiled templates, hashes, and execution targets.
+Input fixture construction occurs outside the timed iteration. Consequently,
+the measurements include downstream adapter work and are not a standalone
+ConfigKit benchmark.
+
+Other committed fixtures cover file-backed messages, shared-base profile
+inheritance, and four-tier layering. They provide additional downstream cases
+for assessing proposed improvements; the principal evidence below uses the
+simple independent fixture.
+
+## Measurements And Reproduction
+
+Original PromptKit review baseline:
+`de6443f3aef4a66169e323e9f9a5f41594fbface`.
+
+The request preparation repeated the benchmark at PromptKit revision
+`eb830bb61346672f0f6c5bce6119b76ccc6d61ac`. The benchmark, configuration adapter,
+and `go.mod` are unchanged between these revisions.
+
+Reproduction environment:
+
+- ConfigKit: `gitea.maximumdirect.net/eric/ConfigKit v0.5.0`.
+- Toolchain: `go1.26.8`, `linux/amd64`.
+- Processor: Intel Core i5-7260U CPU at 2.20 GHz.
+- Benchmark process parallelism: 4, as reported by the benchmark suffix.
+- Workspace mode disabled; installed toolchain and already-acquired module
+  dependencies used without downloads.
+
+From a PromptKit checkout at either recorded revision:
+
+```sh
+GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
+  go test -mod=readonly -run '^$' \
+  -bench '^BenchmarkConfigurationConstruction/independent$' \
+  -benchmem -benchtime=500ms -count=5 ./internal/configuration
+```
+
+Repeated results:
+
+| Run | Time per construction (ms) | Allocated bytes per construction | Allocations per construction |
+| --- | ---: | ---: | ---: |
+| 1 | 71.052 | 50,128,254 | 538,582 |
+| 2 | 67.740 | 50,139,198 | 538,539 |
+| 3 | 66.353 | 50,174,030 | 538,541 |
+| 4 | 67.170 | 50,159,959 | 538,542 |
+| 5 | 68.209 | 50,167,214 | 538,541 |
+
+These agree with the original review's approximate 69 ms, 50 MB, and 539,000
+allocations. They measure cumulative allocation per operation, not retained
+heap or a memory leak. Timing depends on the machine and runtime environment.
+
+### Allocation And CPU Profile Observations
+
+A separate two-second profiled run reproduced approximately 68.7 ms,
+50.1 MB, and 538,553 allocations per construction. To generate comparable
+profiles from the same PromptKit checkout, placing generated artifacts outside
+the repository:
+
+```sh
+profile_dir=$(mktemp -d)
+GOTOOLCHAIN=local GOWORK=off GOPROXY=off \
+  go test -mod=readonly -run '^$' \
+  -bench '^BenchmarkConfigurationConstruction/independent$' \
+  -benchtime=2s -cpuprofile="$profile_dir/config.cpu" \
+  -memprofile="$profile_dir/config.mem" -o "$profile_dir/config.test" \
+  ./internal/configuration
+GOTOOLCHAIN=local go tool pprof -top -alloc_space \
+  "$profile_dir/config.test" "$profile_dir/config.mem"
+GOTOOLCHAIN=local go tool pprof -top \
+  "$profile_dir/config.test" "$profile_dir/config.cpu"
+```
+
+The sampled allocation profile prominently attributes work to ConfigKit's
+path and metadata machinery. Selected flat allocation shares were:
+
+- `clonePathSegments`: approximately 29.9%.
+- `ParsePath`: approximately 4.7%.
+- `collectFieldInfo`: approximately 4.3%.
+- `collectOrigins`: approximately 3.7%.
+- `effectivePathIndex.func1`: approximately 3.7%.
+
+The CPU profile also showed substantial allocation/garbage-collection work and
+path formatting (`Path.String`, approximately 17.4% cumulative CPU samples).
+These symbol names are profiling evidence, not recommendations for ConfigKit's
+internal implementation. Flat allocation and cumulative CPU shares describe
+different measures and should not be added together.
+
+The profile supports investigating ConfigKit as a major contributor; it does
+not attribute every byte or millisecond of this assembled benchmark to
+ConfigKit. A single fixture size also does not establish an asymptotic scaling
+defect. Repeated measurements across representative sizes would help assess
+scaling and any proposed improvement.
+
+## Requested Outcome
+
+Consider reducing avoidable construction-time work and temporary allocations
+for this workload while retaining the capabilities defined by ConfigKit's
+[configuration contract](../config.md), including:
+
+- layered sources and sibling inheritance;
+- typed decoding and recorded application resolution;
+- effective presence, explicit null, and resource/source context;
+- provenance, history, and safe operator inspection; and
+- established result/query ownership and secret-handling behavior.
+
+An improvement through the existing API would be preferable for PromptKit.
+The request does not prescribe a cache, data structure, new public API, or
+alternative metadata mode. Any optional API change can be assessed against
+the downstream need if ConfigKit determines that one is justified.
+
+Useful evidence of success would be repeatable reductions in CPU or allocation
+cost for the recorded fixture and representative related workloads, with
+existing semantics intact. No machine-specific threshold, fixed allocation
+budget, mandatory complexity guarantee, or additional release gate is requested.
+
+This request can be considered independently of PromptKit's local runtime
+optimizations. PromptKit will continue to use ConfigKit's final configuration
+handoff and can evaluate adoption of an upstream improvement when available.