promptkit/docs/roadmap/performance.md

Computational Efficiency Improvements

Status: Accepted scope following review. The ConfigKit request (item 1) is complete; local performance implementation has not started.

Purpose

Reduce avoidable CPU work and allocations in Promptkit's existing configuration, preparation, rendering, and execution paths. Large payloads and concurrent library use make redundant full-payload copies especially worth addressing. Prefer small, idiomatic improvements over additional framework machinery.

This roadmap captures the intended outcomes and recommended changes. The implementation plan owns staged work and verification. This roadmap introduces no new public performance guarantee. Existing contracts remain owned by GoDoc, the format reference, and the integration contract. Preserve the boundaries in the architecture policy and configuration ADR.

Review Evidence

The review used existing package benchmarks and temporary offline probes at baseline de6443f. Approximate measurements on the review machine included:

• Configuration construction for 256 effective definitions: 69 ms, 50 MB of cumulative allocations, and roughly 539,000 allocations per construction. Allocation profiling pointed predominantly to ConfigKit's path and metadata machinery. Indexed selection remained fast.
• Basic validation of a 1 MiB nonempty response: an additional 1 MiB allocation and about 0.2 ms, caused by conversion to a string for the whitespace check.
• Isolated template execution with about 272 KiB of output: changing the output writer from a buffer to a string builder reduced cumulative allocation from about 816 KiB to 544 KiB and execution time by about 27%.
• Request encoding with a 1 MiB extra-parameter value: a single-serialization diagnostic roughly halved encoding time and cumulative allocated bytes.
• Reading a 1 MiB regular file: approximately 4.5 MiB of cumulative allocations after warm-up, including repeated buffer growth.

These are local measurements of particular fixtures, not end-to-end provider latency predictions, retained-heap measurements, or acceptance thresholds. Snapshot-copy and inspection opportunities below were identified from code; their benefits still need measurement.

Scope And Target Outcomes

1. Submit A ConfigKit Performance Feature Request

Completed on 2026-10-08. The request is recorded in ConfigKit's docs/roadmap/promptkit-performance-request.md, committed as d53021e. It includes the downstream fixture, dependency and toolchain versions, repeated benchmark results, reproduction commands, and allocation-profile observations, while preserving existing configuration and inspection semantics.

Acceptance, design, and implementation remain with ConfigKit. Upstream improvements will become available through a later ConfigKit dependency upgrade; no further Promptkit-specific work remains for this item at present. Local performance improvements can proceed independently.

2. Avoid Response Copies In Basic Validation

In the validator, perform the empty/whitespace-only check directly on the artifact's byte slice, rather than converting the entire body to a string. bytes.TrimSpace is the recommended standard-library operation.

The target is to remove the response-sized temporary allocation while retaining current Unicode whitespace behavior, validation results, and cancellation handling. Empty, whitespace-only, and nonempty outputs keep their existing meaning; this is not a validation-policy change.

3. Avoid The Final Rendered-Message Copy

Use a string-oriented output writer where the renderer currently accumulates template output in a byte buffer and then obtains a copied string. A local strings.Builder is the recommended approach for message output; apply the same simplification to session rendering where appropriate.

The target is fewer output-sized allocations with unchanged rendered bytes, template behavior, and failure/cancellation handling. Preserve captured parses, request-specific template bindings, and the existing per-render artifact-text cache. No shared writer pool or global render cache is needed for this change.

4. Eliminate Duplicate Serialization Of Provider Extras

Refine request encoding so provider extra-parameter values are not serialized once for validation and again as part of the complete request. Assemble and encode the request without discarding an earlier serialization of the same values.

Preserve the reserved-field checks, accepted input types, wire semantics, useful error context, and existing public error categories. Consolidating encoding is not permission to skip validation or turn invalid requests into transport failures. Continue freezing one serialized request for transport retries.

5. Reduce Artifact File-Read Buffer Growth

Improve allocation in the ordinary artifact reader, which currently grows an initially empty slice despite having the opened file's metadata. Use the opened size as a sensible initial-capacity hint, with safe integer handling, rather than repeatedly copying already-read content.

The actual read still determines the returned content and size. Retain regular file checks, chunked cancellation handling, error behavior, and reading through EOF, including when the file changes after its metadata was obtained. This optimization introduces no new file-size limit or filesystem policy.

6. Consolidate Redundant Prepared-State Copies Where Justified

Review ownership through prepared execution and public conversion. Creation currently builds two deep snapshots, and returning details adds further internal and public copies. Large schemas and nested provider maps can make this expensive. Copying message structures does not itself duplicate their immutable string contents; measure mutable payload copies separately from message length.

Where internal ownership is demonstrably exclusive, consolidate overlapping copies or transfer owned values rather than cloning again. Retain copies needed to isolate engine state, the execution payload, returned details, and injected collaborators. Preserve prepared-execution lifecycle and concurrency behavior. Before removing a copy, identify which component owns each mutable value and whether another component can still access it. In particular, preserve the independence and availability of details after execution or discard described by the public handle contract.

Measure representative cases before selecting the refactor. There is no target copy count and no requirement to remove every defensive copy; retain the simpler design where savings do not justify additional ownership complexity.

7. Keep Identity-Only Inspection Lightweight

Requirement inspection selects a full execution profile, including a deep copy of provider extras, although it uses only the selected profile identity. Review nearby prompt selection for similarly unnecessary copying of data that inspection does not consume.

Use narrower internal selection or defer payload copying where that reduces work without duplicating selection semantics. Preserve exact identity lookup, omitted-selector behavior, partial-profile inspectability, and existing public errors. The target is inspection cost driven by the information being inspected, not unrelated provider payloads. This requires no new public lookup API and does not move provenance analysis into ordinary prompt execution.

Benchmarking And Verification

Use focused, offline before/after benchmarks to establish useful savings in time, allocated bytes, and allocation count. Cover representative small inputs as well as larger messages, responses, files, schemas, and provider maps where they exercise the affected path.

Preserve reproducible fixtures and comparison methods for the temporary probes behind the review figures. Record the baseline and changed revisions, Go and dependency versions, relevant machine details, benchmark commands, and repeated results alongside the comparison. The existing construction benchmark already provides a reproducible fixture; capture equivalent evidence for the other affected paths before using their preliminary figures to judge improvements.

Distinguish cumulative allocations from retained memory. Where relevant, measure retained heap with representative lifetimes and concurrency, especially for builder-backed strings and long-lived prepared handles. A reduction in allocated bytes alone does not establish a reduction in retained memory.

Add coverage for the production captured-template renderer: the existing artifact-reference benchmark uses the reparsing fallback. Include representative assembled preparation and execution with an injected model client so local overhead can be measured independently of provider latency. Use repeatable comparisons rather than machine-specific timing or allocation thresholds.

Follow the testing policy: protect observable behavior and consequential ownership, cancellation, concurrency, and error-boundary risks, reusing existing protection where sufficient. Benchmarks support these changes; they are not additional release gates or public speed guarantees.

Intended End State

ConfigKit has received a reproducible downstream performance request for its consideration. Promptkit avoids the identified unnecessary validation, rendering, encoding, and file-buffer allocations. Prepared-state and inspection copying are simplified where evidence supports a maintainable improvement.

Consumers retain the same substantive API behavior and ownership protections. Existing indexed selection, captured templates, per-render text reuse, operation-local schema reuse, serialized retry reuse, and constant-time queue removal remain useful foundations. New caches, execution limits, configuration semantics, and release gates are not part of this work.