diff --git a/docs/roadmap/performance.md b/docs/roadmap/performance.md new file mode 100644 index 0000000..5b31e24 --- /dev/null +++ b/docs/roadmap/performance.md @@ -0,0 +1,178 @@ +# Computational Efficiency Improvements + +Status: Proposed scope following the computational-efficiency review. + +## Purpose + +Reduce avoidable CPU work and allocations in Promptkit's existing configuration, +preparation, rendering, and execution paths. Large payloads and concurrent +library use make redundant full-payload copies especially worth addressing. +Prefer small, idiomatic improvements over additional framework machinery. + +This roadmap captures the intended outcomes and recommended changes. It is not +a staged implementation plan or a new public performance guarantee. Existing +contracts remain owned by GoDoc, the [format reference](../formats.md), and the +[integration contract](../integrations/openai-compatible-chat.md). Preserve the +boundaries in the [architecture policy](../policy/architecture.md) and +[configuration ADR](../adr/0001-configuration-boundary.md). + +## Review Evidence + +The review used existing package benchmarks and temporary offline probes at +baseline `de6443f`. Approximate measurements on the review machine included: + +- Configuration construction for 256 effective definitions: 69 ms, 50 MB of + cumulative allocations, and roughly 539,000 allocations per construction. + Allocation profiling pointed predominantly to ConfigKit's path and metadata + machinery. Indexed selection remained fast. +- Basic validation of a 1 MiB nonempty response: an additional 1 MiB allocation + and about 0.2 ms, caused by conversion to a string for the whitespace check. +- Isolated template execution with about 272 KiB of output: changing the + output writer from a buffer to a string builder reduced cumulative allocation + from about 816 KiB to 544 KiB and execution time by about 27%. +- Request encoding with a 1 MiB extra-parameter value: a single-serialization + diagnostic roughly halved encoding time and cumulative allocated bytes. +- Reading a 1 MiB regular file: approximately 4.5 MiB of cumulative allocations + after warm-up, including repeated buffer growth. + +These are local measurements of particular fixtures, not end-to-end provider +latency predictions, retained-heap measurements, or acceptance thresholds. +Snapshot-copy and inspection opportunities below were identified from code; +their benefits still need measurement. + +## Scope And Target Outcomes + +### 1. Submit A ConfigKit Performance Feature Request + +Submit an appropriate feature request for ConfigKit's consideration, describing +Promptkit's downstream configuration-construction workload and the measured +allocation cost. Include the fixture shape, selected ConfigKit version, +benchmark method, and allocation-profile evidence so upstream maintainers can +reproduce and assess it. The existing +[construction benchmark](../../internal/configuration/benchmark_test.go) and +[configuration adapter](../../internal/configuration/load.go) provide the +downstream context. + +Request more efficient construction while retaining the source composition, +resolution, final-value handoff, and operator-inspection capabilities Promptkit +uses. Explain why libraries that build multiple engines or load large catalogs +benefit, and distinguish startup cost from ordinary prompt execution. + +Leave acceptance, internal design, and implementation to ConfigKit. Submission +and upstream consideration are the local outcome for this item; adoption of a +future upstream improvement can be assessed when one is available. Promptkit's +other improvements can proceed independently. Reimplementing ConfigKit's loader +or bypassing its configuration ownership is outside this scope. + +### 2. Avoid Response Copies In Basic Validation + +In the [validator](../../internal/validate/standard_validator.go), perform the +empty/whitespace-only check directly on the artifact's byte slice, rather than +converting the entire body to a string. `bytes.TrimSpace` is the recommended +standard-library operation. + +The target is to remove the response-sized temporary allocation while retaining +current Unicode whitespace behavior, validation results, and cancellation +handling. Empty, whitespace-only, and nonempty outputs keep their existing +meaning; this is not a validation-policy change. + +### 3. Avoid The Final Rendered-Message Copy + +Use a string-oriented output writer where the +[renderer](../../internal/prompt/go_renderer.go) currently accumulates template +output in a byte buffer and then obtains a copied string. A local +`strings.Builder` is the recommended approach for message output; apply the +same simplification to session rendering where appropriate. + +The target is fewer output-sized allocations with unchanged rendered bytes, +template behavior, and failure/cancellation handling. Preserve captured parses, +request-specific template bindings, and the existing per-render artifact-text +cache. No shared writer pool or global render cache is needed for this change. + +### 4. Eliminate Duplicate Serialization Of Provider Extras + +Refine [request encoding](../../internal/llm/openai_compatible_client.go) so +provider extra-parameter values are not serialized once for validation and +again as part of the complete request. Assemble and encode the request without +discarding an earlier serialization of the same values. + +Preserve the reserved-field checks, accepted input types, wire semantics, useful +error context, and existing public error categories. Consolidating encoding is +not permission to skip validation or turn invalid requests into transport +failures. Continue freezing one serialized request for transport retries. + +### 5. Reduce Artifact File-Read Buffer Growth + +Improve allocation in the +[ordinary artifact reader](../../internal/artifact/reader.go), which currently +grows an initially empty slice despite having the opened file's metadata. +Use the opened size as a sensible initial-capacity hint, with safe integer +handling, rather than repeatedly copying already-read content. + +The actual read still determines the returned content and size. Retain regular +file checks, chunked cancellation handling, error behavior, and reading through +EOF, including when the file changes after its metadata was obtained. This +optimization introduces no new file-size limit or filesystem policy. + +### 6. Consolidate Redundant Prepared-State Copies Where Justified + +Review ownership through +[prepared execution](../../internal/usecase/prepared_execution.go) and +[public conversion](../../convert.go). Creation currently builds two deep +snapshots, and returning details adds further internal and public copies. +Large schemas and nested provider maps can make this expensive. + +Where internal ownership is demonstrably exclusive, consolidate overlapping +copies or transfer owned values rather than cloning again. Retain copies needed +to isolate engine state, the execution payload, returned details, and injected +collaborators. Preserve prepared-execution lifecycle and concurrency behavior. + +Measure representative cases before selecting the refactor. There is no target +copy count and no requirement to remove every defensive copy; retain the simpler +design where savings do not justify additional ownership complexity. + +### 7. Keep Identity-Only Inspection Lightweight + +[Requirement inspection](../../internal/usecase/requirements_inspection.go) +selects a full execution profile, including a deep copy of provider extras, +although it uses only the selected profile identity. Review nearby prompt +selection for similarly unnecessary copying of data that inspection does not +consume. + +Use narrower internal selection or defer payload copying where that reduces +work without duplicating selection semantics. Preserve exact identity lookup, +omitted-selector behavior, partial-profile inspectability, and existing public +errors. The target is inspection cost driven by the information being inspected, +not unrelated provider payloads. This requires no new public lookup API and +does not move provenance analysis into ordinary prompt execution. + +## Benchmarking And Verification + +Use focused, offline before/after benchmarks to establish useful savings in +time, allocated bytes, and allocation count. Cover representative small inputs +as well as larger messages, responses, files, schemas, and provider maps where +they exercise the affected path. + +Add coverage for the production captured-template renderer: the existing +artifact-reference benchmark uses the reparsing fallback. Include representative +assembled preparation and execution with an injected model client so local +overhead can be measured independently of provider latency. Use repeatable +comparisons rather than machine-specific timing or allocation thresholds. + +Follow the [testing policy](../policy/testing.md): protect observable behavior +and consequential ownership, cancellation, concurrency, and error-boundary +risks, reusing existing protection where sufficient. Benchmarks support these +changes; they are not additional release gates or public speed guarantees. + +## Intended End State + +ConfigKit has received a reproducible downstream performance request for its +consideration. Promptkit avoids the identified unnecessary validation, +rendering, encoding, and file-buffer allocations. Prepared-state and inspection +copying are simplified where evidence supports a maintainable improvement. + +Consumers retain the same substantive API behavior and ownership protections. +Existing indexed selection, captured templates, per-render text reuse, +operation-local schema reuse, serialized retry reuse, and constant-time queue +removal remain useful foundations. New caches, execution limits, configuration +semantics, and release gates are not part of this work.