Plan computational efficiency improvements

docs/roadmap/performance.md

@@ -0,0 +1,178 @@
+# Computational Efficiency Improvements
+
+Status: Proposed scope following the computational-efficiency review.
+
+## Purpose
+
+Reduce avoidable CPU work and allocations in Promptkit's existing configuration,
+preparation, rendering, and execution paths. Large payloads and concurrent
+library use make redundant full-payload copies especially worth addressing.
+Prefer small, idiomatic improvements over additional framework machinery.
+
+This roadmap captures the intended outcomes and recommended changes. It is not
+a staged implementation plan or a new public performance guarantee. Existing
+contracts remain owned by GoDoc, the [format reference](../formats.md), and the
+[integration contract](../integrations/openai-compatible-chat.md). Preserve the
+boundaries in the [architecture policy](../policy/architecture.md) and
+[configuration ADR](../adr/0001-configuration-boundary.md).
+
+## Review Evidence
+
+The review used existing package benchmarks and temporary offline probes at
+baseline `de6443f`. Approximate measurements on the review machine included:
+
+- Configuration construction for 256 effective definitions: 69 ms, 50 MB of
+  cumulative allocations, and roughly 539,000 allocations per construction.
+  Allocation profiling pointed predominantly to ConfigKit's path and metadata
+  machinery. Indexed selection remained fast.
+- Basic validation of a 1 MiB nonempty response: an additional 1 MiB allocation
+  and about 0.2 ms, caused by conversion to a string for the whitespace check.
+- Isolated template execution with about 272 KiB of output: changing the
+  output writer from a buffer to a string builder reduced cumulative allocation
+  from about 816 KiB to 544 KiB and execution time by about 27%.
+- Request encoding with a 1 MiB extra-parameter value: a single-serialization
+  diagnostic roughly halved encoding time and cumulative allocated bytes.
+- Reading a 1 MiB regular file: approximately 4.5 MiB of cumulative allocations
+  after warm-up, including repeated buffer growth.
+
+These are local measurements of particular fixtures, not end-to-end provider
+latency predictions, retained-heap measurements, or acceptance thresholds.
+Snapshot-copy and inspection opportunities below were identified from code;
+their benefits still need measurement.
+
+## Scope And Target Outcomes
+
+### 1. Submit A ConfigKit Performance Feature Request
+
+Submit an appropriate feature request for ConfigKit's consideration, describing
+Promptkit's downstream configuration-construction workload and the measured
+allocation cost. Include the fixture shape, selected ConfigKit version,
+benchmark method, and allocation-profile evidence so upstream maintainers can
+reproduce and assess it. The existing
+[construction benchmark](../../internal/configuration/benchmark_test.go) and
+[configuration adapter](../../internal/configuration/load.go) provide the
+downstream context.
+
+Request more efficient construction while retaining the source composition,
+resolution, final-value handoff, and operator-inspection capabilities Promptkit
+uses. Explain why libraries that build multiple engines or load large catalogs
+benefit, and distinguish startup cost from ordinary prompt execution.
+
+Leave acceptance, internal design, and implementation to ConfigKit. Submission
+and upstream consideration are the local outcome for this item; adoption of a
+future upstream improvement can be assessed when one is available. Promptkit's
+other improvements can proceed independently. Reimplementing ConfigKit's loader
+or bypassing its configuration ownership is outside this scope.
+
+### 2. Avoid Response Copies In Basic Validation
+
+In the [validator](../../internal/validate/standard_validator.go), perform the
+empty/whitespace-only check directly on the artifact's byte slice, rather than
+converting the entire body to a string. `bytes.TrimSpace` is the recommended
+standard-library operation.
+
+The target is to remove the response-sized temporary allocation while retaining
+current Unicode whitespace behavior, validation results, and cancellation
+handling. Empty, whitespace-only, and nonempty outputs keep their existing
+meaning; this is not a validation-policy change.
+
+### 3. Avoid The Final Rendered-Message Copy
+
+Use a string-oriented output writer where the
+[renderer](../../internal/prompt/go_renderer.go) currently accumulates template
+output in a byte buffer and then obtains a copied string. A local
+`strings.Builder` is the recommended approach for message output; apply the
+same simplification to session rendering where appropriate.
+
+The target is fewer output-sized allocations with unchanged rendered bytes,
+template behavior, and failure/cancellation handling. Preserve captured parses,
+request-specific template bindings, and the existing per-render artifact-text
+cache. No shared writer pool or global render cache is needed for this change.
+
+### 4. Eliminate Duplicate Serialization Of Provider Extras
+
+Refine [request encoding](../../internal/llm/openai_compatible_client.go) so
+provider extra-parameter values are not serialized once for validation and
+again as part of the complete request. Assemble and encode the request without
+discarding an earlier serialization of the same values.
+
+Preserve the reserved-field checks, accepted input types, wire semantics, useful
+error context, and existing public error categories. Consolidating encoding is
+not permission to skip validation or turn invalid requests into transport
+failures. Continue freezing one serialized request for transport retries.
+
+### 5. Reduce Artifact File-Read Buffer Growth
+
+Improve allocation in the
+[ordinary artifact reader](../../internal/artifact/reader.go), which currently
+grows an initially empty slice despite having the opened file's metadata.
+Use the opened size as a sensible initial-capacity hint, with safe integer
+handling, rather than repeatedly copying already-read content.
+
+The actual read still determines the returned content and size. Retain regular
+file checks, chunked cancellation handling, error behavior, and reading through
+EOF, including when the file changes after its metadata was obtained. This
+optimization introduces no new file-size limit or filesystem policy.
+
+### 6. Consolidate Redundant Prepared-State Copies Where Justified
+
+Review ownership through
+[prepared execution](../../internal/usecase/prepared_execution.go) and
+[public conversion](../../convert.go). Creation currently builds two deep
+snapshots, and returning details adds further internal and public copies.
+Large schemas and nested provider maps can make this expensive.
+
+Where internal ownership is demonstrably exclusive, consolidate overlapping
+copies or transfer owned values rather than cloning again. Retain copies needed
+to isolate engine state, the execution payload, returned details, and injected
+collaborators. Preserve prepared-execution lifecycle and concurrency behavior.
+
+Measure representative cases before selecting the refactor. There is no target
+copy count and no requirement to remove every defensive copy; retain the simpler
+design where savings do not justify additional ownership complexity.
+
+### 7. Keep Identity-Only Inspection Lightweight
+
+[Requirement inspection](../../internal/usecase/requirements_inspection.go)
+selects a full execution profile, including a deep copy of provider extras,
+although it uses only the selected profile identity. Review nearby prompt
+selection for similarly unnecessary copying of data that inspection does not
+consume.
+
+Use narrower internal selection or defer payload copying where that reduces
+work without duplicating selection semantics. Preserve exact identity lookup,
+omitted-selector behavior, partial-profile inspectability, and existing public
+errors. The target is inspection cost driven by the information being inspected,
+not unrelated provider payloads. This requires no new public lookup API and
+does not move provenance analysis into ordinary prompt execution.
+
+## Benchmarking And Verification
+
+Use focused, offline before/after benchmarks to establish useful savings in
+time, allocated bytes, and allocation count. Cover representative small inputs
+as well as larger messages, responses, files, schemas, and provider maps where
+they exercise the affected path.
+
+Add coverage for the production captured-template renderer: the existing
+artifact-reference benchmark uses the reparsing fallback. Include representative
+assembled preparation and execution with an injected model client so local
+overhead can be measured independently of provider latency. Use repeatable
+comparisons rather than machine-specific timing or allocation thresholds.
+
+Follow the [testing policy](../policy/testing.md): protect observable behavior
+and consequential ownership, cancellation, concurrency, and error-boundary
+risks, reusing existing protection where sufficient. Benchmarks support these
+changes; they are not additional release gates or public speed guarantees.
+
+## Intended End State
+
+ConfigKit has received a reproducible downstream performance request for its
+consideration. Promptkit avoids the identified unnecessary validation,
+rendering, encoding, and file-buffer allocations. Prepared-state and inspection
+copying are simplified where evidence supports a maintainable improvement.
+
+Consumers retain the same substantive API behavior and ownership protections.
+Existing indexed selection, captured templates, per-render text reuse,
+operation-local schema reuse, serialized retry reuse, and constant-time queue
+removal remain useful foundations. New caches, execution limits, configuration
+semantics, and release gates are not part of this work.