promptkit/docs/roadmap/performance.md

# Computational Efficiency Improvements

Status: Complete (2026-10-09). The ConfigKit request (item 1) was submitted,
and local items 2–7 were implemented or completed with the evidence and
conditional ownership decision recorded in the
[performance implementation plan](https://gitea.maximumdirect.net/eric/promptkit/src/commit/7325a34c070c77c4784ab76c916fc8a4d79b5c46/docs/roadmap/implementation.md). No dependency upgrade or public
performance guarantee was part of this work.

## Purpose

Reduce avoidable CPU work and allocations in Promptkit's existing configuration,
preparation, rendering, and execution paths. Large payloads and concurrent
library use make redundant full-payload copies especially worth addressing.
Prefer small, idiomatic improvements over additional framework machinery.

This roadmap captures the intended outcomes and recommended changes. The
[performance implementation plan](https://gitea.maximumdirect.net/eric/promptkit/src/commit/7325a34c070c77c4784ab76c916fc8a4d79b5c46/docs/roadmap/implementation.md) owns staged work and verification.
This roadmap introduces no new public performance guarantee. Existing contracts
remain owned by GoDoc, the
[format reference](../formats.md), and the
[integration contract](../integrations/openai-compatible-chat.md). Preserve the
boundaries in the [architecture policy](../policy/architecture.md) and
[configuration ADR](../adr/0001-configuration-boundary.md).

## Review Evidence

The review used existing package benchmarks and temporary offline probes at
baseline `de6443f`. Approximate measurements on the review machine included:

- Configuration construction for 256 effective definitions: 69 ms, 50 MB of
  cumulative allocations, and roughly 539,000 allocations per construction.
  Allocation profiling pointed predominantly to ConfigKit's path and metadata
  machinery. Indexed selection remained fast.
- Basic validation of a 1 MiB nonempty response: an additional 1 MiB allocation
  and about 0.2 ms, caused by conversion to a string for the whitespace check.
- Isolated template execution with about 272 KiB of output: changing the
  output writer from a buffer to a string builder reduced cumulative allocation
  from about 816 KiB to 544 KiB and execution time by about 27%.
- Request encoding with a 1 MiB extra-parameter value: a single-serialization
  diagnostic roughly halved encoding time and cumulative allocated bytes.
- Reading a 1 MiB regular file: approximately 4.5 MiB of cumulative allocations
  after warm-up, including repeated buffer growth.

These are local measurements of particular fixtures, not end-to-end provider
latency predictions, retained-heap measurements, or acceptance thresholds.
Snapshot-copy and inspection opportunities below were identified from code;
their benefits still need measurement.

## Scope And Target Outcomes

### 1. Submit A ConfigKit Performance Feature Request

Completed on 2026-10-08. The request is recorded in ConfigKit's
`docs/roadmap/promptkit-performance-request.md`, committed as `d53021e`.
It includes the downstream fixture, dependency and toolchain versions,
repeated benchmark results, reproduction commands, and allocation-profile
observations, while preserving existing configuration and inspection semantics.

Acceptance, design, and implementation remain with ConfigKit. Upstream
improvements will become available through a later ConfigKit dependency upgrade;
no further Promptkit-specific work remains for this item at present. Local
performance improvements can proceed independently.

### 2. Avoid Response Copies In Basic Validation

In the [validator](../../internal/validate/standard_validator.go), perform the
empty/whitespace-only check directly on the artifact's byte slice, rather than
converting the entire body to a string. `bytes.TrimSpace` is the recommended
standard-library operation.

The target is to remove the response-sized temporary allocation while retaining
current Unicode whitespace behavior, validation results, and cancellation
handling. Empty, whitespace-only, and nonempty outputs keep their existing
meaning; this is not a validation-policy change.

### 3. Avoid The Final Rendered-Message Copy

Use a string-oriented output writer where the
[renderer](../../internal/prompt/go_renderer.go) currently accumulates template
output in a byte buffer and then obtains a copied string. A local
`strings.Builder` is the recommended approach for message output; apply the
same simplification to session rendering where appropriate.

The target is fewer output-sized allocations with unchanged rendered bytes,
template behavior, and failure/cancellation handling. Preserve captured parses,
request-specific template bindings, and the existing per-render artifact-text
cache. No shared writer pool or global render cache is needed for this change.

### 4. Eliminate Duplicate Serialization Of Provider Extras

Refine [request encoding](../../internal/llm/openai_compatible_client.go) so
provider extra-parameter values are not serialized once for validation and
again as part of the complete request. Assemble and encode the request without
discarding an earlier serialization of the same values.

Preserve the reserved-field checks, accepted input types, wire semantics, useful
error context, and existing public error categories. Consolidating encoding is
not permission to skip validation or turn invalid requests into transport
failures. Continue freezing one serialized request for transport retries.

### 5. Reduce Artifact File-Read Buffer Growth

Improve allocation in the
[ordinary artifact reader](../../internal/artifact/reader.go), which currently
grows an initially empty slice despite having the opened file's metadata.
Use the opened size as a sensible initial-capacity hint, with safe integer
handling, rather than repeatedly copying already-read content.

The actual read still determines the returned content and size. Retain regular
file checks, chunked cancellation handling, error behavior, and reading through
EOF, including when the file changes after its metadata was obtained. This
optimization introduces no new file-size limit or filesystem policy.

### 6. Consolidate Redundant Prepared-State Copies Where Justified

Review ownership through
[prepared execution](../../internal/usecase/prepared_execution.go) and
[public conversion](../../convert.go). Creation currently builds two deep
snapshots, and returning details adds further internal and public copies.
Large schemas and nested provider maps can make this expensive. Copying message
structures does not itself duplicate their immutable string contents; measure
mutable payload copies separately from message length.

Where internal ownership is demonstrably exclusive, consolidate overlapping
copies or transfer owned values rather than cloning again. Retain copies needed
to isolate engine state, the execution payload, returned details, and injected
collaborators. Preserve prepared-execution lifecycle and concurrency behavior.
Before removing a copy, identify which component owns each mutable value and
whether another component can still access it. In particular, preserve the
independence and availability of details after execution or discard described
by the [public handle contract](../../prepared_execution.go).

Measure representative cases before selecting the refactor. There is no target
copy count and no requirement to remove every defensive copy; retain the simpler
design where savings do not justify additional ownership complexity.

### 7. Keep Identity-Only Inspection Lightweight

[Requirement inspection](../../internal/usecase/requirements_inspection.go)
selects a full execution profile, including a deep copy of provider extras,
although it uses only the selected profile identity. Review nearby prompt
selection for similarly unnecessary copying of data that inspection does not
consume.

Use narrower internal selection or defer payload copying where that reduces
work without duplicating selection semantics. Preserve exact identity lookup,
omitted-selector behavior, partial-profile inspectability, and existing public
errors. The target is inspection cost driven by the information being inspected,
not unrelated provider payloads. This requires no new public lookup API and
does not move provenance analysis into ordinary prompt execution.

## Benchmarking And Verification

Use focused, offline before/after benchmarks to establish useful savings in
time, allocated bytes, and allocation count. Cover representative small inputs
as well as larger messages, responses, files, schemas, and provider maps where
they exercise the affected path.

Preserve reproducible fixtures and comparison methods for the temporary probes
behind the review figures. Record the baseline and changed revisions, Go and
dependency versions, relevant machine details, benchmark commands, and repeated
results alongside the comparison. The existing construction benchmark already
provides a reproducible fixture; capture equivalent evidence for the other
affected paths before using their preliminary figures to judge improvements.

Distinguish cumulative allocations from retained memory. Where relevant,
measure retained heap with representative lifetimes and concurrency, especially
for builder-backed strings and long-lived prepared handles. A reduction in
allocated bytes alone does not establish a reduction in retained memory.

Add coverage for the production captured-template renderer: the existing
artifact-reference benchmark uses the reparsing fallback. Include representative
assembled preparation and execution with an injected model client so local
overhead can be measured independently of provider latency. Use repeatable
comparisons rather than machine-specific timing or allocation thresholds.

Follow the [testing policy](../policy/testing.md): protect observable behavior
and consequential ownership, cancellation, concurrency, and error-boundary
risks, reusing existing protection where sufficient. Benchmarks support these
changes; they are not additional release gates or public speed guarantees.

## Intended End State

ConfigKit has received a reproducible downstream performance request for its
consideration. Promptkit avoids the identified unnecessary validation,
rendering, encoding, and file-buffer allocations. Prepared-state and inspection
copying are simplified where evidence supports a maintainable improvement.

Consumers retain the same substantive API behavior and ownership protections.
Existing indexed selection, captured templates, per-render text reuse,
operation-local schema reuse, serialized retry reuse, and constant-time queue
removal remain useful foundations. New caches, execution limits, configuration
semantics, and release gates are not part of this work.