203 lines
11 KiB
Markdown
203 lines
11 KiB
Markdown
# Computational Efficiency Improvements
|
|
|
|
Status: Accepted scope following review. The ConfigKit request (item 1) is
|
|
complete; local performance implementation has not started.
|
|
|
|
## Purpose
|
|
|
|
Reduce avoidable CPU work and allocations in Promptkit's existing configuration,
|
|
preparation, rendering, and execution paths. Large payloads and concurrent
|
|
library use make redundant full-payload copies especially worth addressing.
|
|
Prefer small, idiomatic improvements over additional framework machinery.
|
|
|
|
This roadmap captures the intended outcomes, recommended changes, and suggested
|
|
order of work. It is not a detailed implementation plan or a new public
|
|
performance guarantee. Existing contracts remain owned by GoDoc, the
|
|
[format reference](../formats.md), and the
|
|
[integration contract](../integrations/openai-compatible-chat.md). Preserve the
|
|
boundaries in the [architecture policy](../policy/architecture.md) and
|
|
[configuration ADR](../adr/0001-configuration-boundary.md).
|
|
|
|
## Review Evidence
|
|
|
|
The review used existing package benchmarks and temporary offline probes at
|
|
baseline `de6443f`. Approximate measurements on the review machine included:
|
|
|
|
- Configuration construction for 256 effective definitions: 69 ms, 50 MB of
|
|
cumulative allocations, and roughly 539,000 allocations per construction.
|
|
Allocation profiling pointed predominantly to ConfigKit's path and metadata
|
|
machinery. Indexed selection remained fast.
|
|
- Basic validation of a 1 MiB nonempty response: an additional 1 MiB allocation
|
|
and about 0.2 ms, caused by conversion to a string for the whitespace check.
|
|
- Isolated template execution with about 272 KiB of output: changing the
|
|
output writer from a buffer to a string builder reduced cumulative allocation
|
|
from about 816 KiB to 544 KiB and execution time by about 27%.
|
|
- Request encoding with a 1 MiB extra-parameter value: a single-serialization
|
|
diagnostic roughly halved encoding time and cumulative allocated bytes.
|
|
- Reading a 1 MiB regular file: approximately 4.5 MiB of cumulative allocations
|
|
after warm-up, including repeated buffer growth.
|
|
|
|
These are local measurements of particular fixtures, not end-to-end provider
|
|
latency predictions, retained-heap measurements, or acceptance thresholds.
|
|
Snapshot-copy and inspection opportunities below were identified from code;
|
|
their benefits still need measurement.
|
|
|
|
## Scope And Target Outcomes
|
|
|
|
### 1. Submit A ConfigKit Performance Feature Request
|
|
|
|
Completed on 2026-10-08. The request is recorded in ConfigKit's
|
|
`docs/roadmap/promptkit-performance-request.md`, committed as `d53021e`.
|
|
It includes the downstream fixture, dependency and toolchain versions,
|
|
repeated benchmark results, reproduction commands, and allocation-profile
|
|
observations, while preserving existing configuration and inspection semantics.
|
|
|
|
Acceptance, design, and implementation remain with ConfigKit. Upstream
|
|
improvements will become available through a later ConfigKit dependency upgrade;
|
|
no further Promptkit-specific work remains for this item at present. Local
|
|
performance improvements can proceed independently.
|
|
|
|
### 2. Avoid Response Copies In Basic Validation
|
|
|
|
In the [validator](../../internal/validate/standard_validator.go), perform the
|
|
empty/whitespace-only check directly on the artifact's byte slice, rather than
|
|
converting the entire body to a string. `bytes.TrimSpace` is the recommended
|
|
standard-library operation.
|
|
|
|
The target is to remove the response-sized temporary allocation while retaining
|
|
current Unicode whitespace behavior, validation results, and cancellation
|
|
handling. Empty, whitespace-only, and nonempty outputs keep their existing
|
|
meaning; this is not a validation-policy change.
|
|
|
|
### 3. Avoid The Final Rendered-Message Copy
|
|
|
|
Use a string-oriented output writer where the
|
|
[renderer](../../internal/prompt/go_renderer.go) currently accumulates template
|
|
output in a byte buffer and then obtains a copied string. A local
|
|
`strings.Builder` is the recommended approach for message output; apply the
|
|
same simplification to session rendering where appropriate.
|
|
|
|
The target is fewer output-sized allocations with unchanged rendered bytes,
|
|
template behavior, and failure/cancellation handling. Preserve captured parses,
|
|
request-specific template bindings, and the existing per-render artifact-text
|
|
cache. No shared writer pool or global render cache is needed for this change.
|
|
|
|
### 4. Eliminate Duplicate Serialization Of Provider Extras
|
|
|
|
Refine [request encoding](../../internal/llm/openai_compatible_client.go) so
|
|
provider extra-parameter values are not serialized once for validation and
|
|
again as part of the complete request. Assemble and encode the request without
|
|
discarding an earlier serialization of the same values.
|
|
|
|
Preserve the reserved-field checks, accepted input types, wire semantics, useful
|
|
error context, and existing public error categories. Consolidating encoding is
|
|
not permission to skip validation or turn invalid requests into transport
|
|
failures. Continue freezing one serialized request for transport retries.
|
|
|
|
### 5. Reduce Artifact File-Read Buffer Growth
|
|
|
|
Improve allocation in the
|
|
[ordinary artifact reader](../../internal/artifact/reader.go), which currently
|
|
grows an initially empty slice despite having the opened file's metadata.
|
|
Use the opened size as a sensible initial-capacity hint, with safe integer
|
|
handling, rather than repeatedly copying already-read content.
|
|
|
|
The actual read still determines the returned content and size. Retain regular
|
|
file checks, chunked cancellation handling, error behavior, and reading through
|
|
EOF, including when the file changes after its metadata was obtained. This
|
|
optimization introduces no new file-size limit or filesystem policy.
|
|
|
|
### 6. Consolidate Redundant Prepared-State Copies Where Justified
|
|
|
|
Review ownership through
|
|
[prepared execution](../../internal/usecase/prepared_execution.go) and
|
|
[public conversion](../../convert.go). Creation currently builds two deep
|
|
snapshots, and returning details adds further internal and public copies.
|
|
Large schemas and nested provider maps can make this expensive. Copying message
|
|
structures does not itself duplicate their immutable string contents; measure
|
|
mutable payload copies separately from message length.
|
|
|
|
Where internal ownership is demonstrably exclusive, consolidate overlapping
|
|
copies or transfer owned values rather than cloning again. Retain copies needed
|
|
to isolate engine state, the execution payload, returned details, and injected
|
|
collaborators. Preserve prepared-execution lifecycle and concurrency behavior.
|
|
Before removing a copy, identify which component owns each mutable value and
|
|
whether another component can still access it. In particular, preserve the
|
|
independence and availability of details after execution or discard described
|
|
by the [public handle contract](../../prepared_execution.go).
|
|
|
|
Measure representative cases before selecting the refactor. There is no target
|
|
copy count and no requirement to remove every defensive copy; retain the simpler
|
|
design where savings do not justify additional ownership complexity.
|
|
|
|
### 7. Keep Identity-Only Inspection Lightweight
|
|
|
|
[Requirement inspection](../../internal/usecase/requirements_inspection.go)
|
|
selects a full execution profile, including a deep copy of provider extras,
|
|
although it uses only the selected profile identity. Review nearby prompt
|
|
selection for similarly unnecessary copying of data that inspection does not
|
|
consume.
|
|
|
|
Use narrower internal selection or defer payload copying where that reduces
|
|
work without duplicating selection semantics. Preserve exact identity lookup,
|
|
omitted-selector behavior, partial-profile inspectability, and existing public
|
|
errors. The target is inspection cost driven by the information being inspected,
|
|
not unrelated provider payloads. This requires no new public lookup API and
|
|
does not move provenance analysis into ordinary prompt execution.
|
|
|
|
## Suggested Order Of Work
|
|
|
|
Start with basic validation and template output (items 2 and 3), then file-read
|
|
capacity (item 5). These are localized changes with directly identifiable
|
|
allocation costs. Follow with request encoding and identity-only inspection
|
|
(items 4 and 7), preserving their error and selection semantics.
|
|
|
|
Keep prepared-state consolidation (item 6) conditional on measured savings and
|
|
a clear ownership analysis; it has the greatest risk of weakening isolation or
|
|
concurrency behavior. The ConfigKit request (item 1) is complete and does not
|
|
block local improvements. This order is a recommendation, not a
|
|
dependency between otherwise independent changes.
|
|
|
|
## Benchmarking And Verification
|
|
|
|
Use focused, offline before/after benchmarks to establish useful savings in
|
|
time, allocated bytes, and allocation count. Cover representative small inputs
|
|
as well as larger messages, responses, files, schemas, and provider maps where
|
|
they exercise the affected path.
|
|
|
|
Preserve reproducible fixtures and comparison methods for the temporary probes
|
|
behind the review figures. Record the baseline and changed revisions, Go and
|
|
dependency versions, relevant machine details, benchmark commands, and repeated
|
|
results alongside the comparison. The existing construction benchmark already
|
|
provides a reproducible fixture; capture equivalent evidence for the other
|
|
affected paths before using their preliminary figures to judge improvements.
|
|
|
|
Distinguish cumulative allocations from retained memory. Where relevant,
|
|
measure retained heap with representative lifetimes and concurrency, especially
|
|
for builder-backed strings and long-lived prepared handles. A reduction in
|
|
allocated bytes alone does not establish a reduction in retained memory.
|
|
|
|
Add coverage for the production captured-template renderer: the existing
|
|
artifact-reference benchmark uses the reparsing fallback. Include representative
|
|
assembled preparation and execution with an injected model client so local
|
|
overhead can be measured independently of provider latency. Use repeatable
|
|
comparisons rather than machine-specific timing or allocation thresholds.
|
|
|
|
Follow the [testing policy](../policy/testing.md): protect observable behavior
|
|
and consequential ownership, cancellation, concurrency, and error-boundary
|
|
risks, reusing existing protection where sufficient. Benchmarks support these
|
|
changes; they are not additional release gates or public speed guarantees.
|
|
|
|
## Intended End State
|
|
|
|
ConfigKit has received a reproducible downstream performance request for its
|
|
consideration. Promptkit avoids the identified unnecessary validation,
|
|
rendering, encoding, and file-buffer allocations. Prepared-state and inspection
|
|
copying are simplified where evidence supports a maintainable improvement.
|
|
|
|
Consumers retain the same substantive API behavior and ownership protections.
|
|
Existing indexed selection, captured templates, per-render text reuse,
|
|
operation-local schema reuse, serialized retry reuse, and constant-time queue
|
|
removal remain useful foundations. New caches, execution limits, configuration
|
|
semantics, and release gates are not part of this work.
|