From eb830bb61346672f0f6c5bce6119b76ccc6d61ac Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Thu, 8 Oct 2026 23:44:12 +0000 Subject: [PATCH] Refine performance roadmap after review --- docs/roadmap/performance.md | 42 ++++++++++++++++++++++++++++++++----- 1 file changed, 37 insertions(+), 5 deletions(-) diff --git a/docs/roadmap/performance.md b/docs/roadmap/performance.md index 5b31e24..6759e32 100644 --- a/docs/roadmap/performance.md +++ b/docs/roadmap/performance.md @@ -1,6 +1,6 @@ # Computational Efficiency Improvements -Status: Proposed scope following the computational-efficiency review. +Status: Accepted scope following review; implementation has not started. ## Purpose @@ -9,9 +9,10 @@ preparation, rendering, and execution paths. Large payloads and concurrent library use make redundant full-payload copies especially worth addressing. Prefer small, idiomatic improvements over additional framework machinery. -This roadmap captures the intended outcomes and recommended changes. It is not -a staged implementation plan or a new public performance guarantee. Existing -contracts remain owned by GoDoc, the [format reference](../formats.md), and the +This roadmap captures the intended outcomes, recommended changes, and suggested +order of work. It is not a detailed implementation plan or a new public +performance guarantee. Existing contracts remain owned by GoDoc, the +[format reference](../formats.md), and the [integration contract](../integrations/openai-compatible-chat.md). Preserve the boundaries in the [architecture policy](../policy/architecture.md) and [configuration ADR](../adr/0001-configuration-boundary.md). @@ -120,12 +121,18 @@ Review ownership through [prepared execution](../../internal/usecase/prepared_execution.go) and [public conversion](../../convert.go). Creation currently builds two deep snapshots, and returning details adds further internal and public copies. -Large schemas and nested provider maps can make this expensive. +Large schemas and nested provider maps can make this expensive. Copying message +structures does not itself duplicate their immutable string contents; measure +mutable payload copies separately from message length. Where internal ownership is demonstrably exclusive, consolidate overlapping copies or transfer owned values rather than cloning again. Retain copies needed to isolate engine state, the execution payload, returned details, and injected collaborators. Preserve prepared-execution lifecycle and concurrency behavior. +Before removing a copy, identify which component owns each mutable value and +whether another component can still access it. In particular, preserve the +independence and availability of details after execution or discard described +by the [public handle contract](../../prepared_execution.go). Measure representative cases before selecting the refactor. There is no target copy count and no requirement to remove every defensive copy; retain the simpler @@ -146,6 +153,19 @@ errors. The target is inspection cost driven by the information being inspected, not unrelated provider payloads. This requires no new public lookup API and does not move provenance analysis into ordinary prompt execution. +## Suggested Order Of Work + +Start with basic validation and template output (items 2 and 3), then file-read +capacity (item 5). These are localized changes with directly identifiable +allocation costs. Follow with request encoding and identity-only inspection +(items 4 and 7), preserving their error and selection semantics. + +Keep prepared-state consolidation (item 6) conditional on measured savings and +a clear ownership analysis; it has the greatest risk of weakening isolation or +concurrency behavior. The ConfigKit request (item 1) can proceed independently +and does not block local improvements. This order is a recommendation, not a +dependency between otherwise independent changes. + ## Benchmarking And Verification Use focused, offline before/after benchmarks to establish useful savings in @@ -153,6 +173,18 @@ time, allocated bytes, and allocation count. Cover representative small inputs as well as larger messages, responses, files, schemas, and provider maps where they exercise the affected path. +Preserve reproducible fixtures and comparison methods for the temporary probes +behind the review figures. Record the baseline and changed revisions, Go and +dependency versions, relevant machine details, benchmark commands, and repeated +results alongside the comparison. The existing construction benchmark already +provides a reproducible fixture; capture equivalent evidence for the other +affected paths before using their preliminary figures to judge improvements. + +Distinguish cumulative allocations from retained memory. Where relevant, +measure retained heap with representative lifetimes and concurrency, especially +for builder-backed strings and long-lived prepared handles. A reduction in +allocated bytes alone does not establish a reduction in retained memory. + Add coverage for the production captured-template renderer: the existing artifact-reference benchmark uses the reparsing fallback. Include representative assembled preparation and execution with an injected model client so local