Refine performance roadmap after review

This commit is contained in:
2026-10-08 23:44:12 +00:00
parent dd6d4591ae
commit eb830bb613

View File

@@ -1,6 +1,6 @@
# Computational Efficiency Improvements # Computational Efficiency Improvements
Status: Proposed scope following the computational-efficiency review. Status: Accepted scope following review; implementation has not started.
## Purpose ## Purpose
@@ -9,9 +9,10 @@ preparation, rendering, and execution paths. Large payloads and concurrent
library use make redundant full-payload copies especially worth addressing. library use make redundant full-payload copies especially worth addressing.
Prefer small, idiomatic improvements over additional framework machinery. Prefer small, idiomatic improvements over additional framework machinery.
This roadmap captures the intended outcomes and recommended changes. It is not This roadmap captures the intended outcomes, recommended changes, and suggested
a staged implementation plan or a new public performance guarantee. Existing order of work. It is not a detailed implementation plan or a new public
contracts remain owned by GoDoc, the [format reference](../formats.md), and the performance guarantee. Existing contracts remain owned by GoDoc, the
[format reference](../formats.md), and the
[integration contract](../integrations/openai-compatible-chat.md). Preserve the [integration contract](../integrations/openai-compatible-chat.md). Preserve the
boundaries in the [architecture policy](../policy/architecture.md) and boundaries in the [architecture policy](../policy/architecture.md) and
[configuration ADR](../adr/0001-configuration-boundary.md). [configuration ADR](../adr/0001-configuration-boundary.md).
@@ -120,12 +121,18 @@ Review ownership through
[prepared execution](../../internal/usecase/prepared_execution.go) and [prepared execution](../../internal/usecase/prepared_execution.go) and
[public conversion](../../convert.go). Creation currently builds two deep [public conversion](../../convert.go). Creation currently builds two deep
snapshots, and returning details adds further internal and public copies. snapshots, and returning details adds further internal and public copies.
Large schemas and nested provider maps can make this expensive. Large schemas and nested provider maps can make this expensive. Copying message
structures does not itself duplicate their immutable string contents; measure
mutable payload copies separately from message length.
Where internal ownership is demonstrably exclusive, consolidate overlapping Where internal ownership is demonstrably exclusive, consolidate overlapping
copies or transfer owned values rather than cloning again. Retain copies needed copies or transfer owned values rather than cloning again. Retain copies needed
to isolate engine state, the execution payload, returned details, and injected to isolate engine state, the execution payload, returned details, and injected
collaborators. Preserve prepared-execution lifecycle and concurrency behavior. collaborators. Preserve prepared-execution lifecycle and concurrency behavior.
Before removing a copy, identify which component owns each mutable value and
whether another component can still access it. In particular, preserve the
independence and availability of details after execution or discard described
by the [public handle contract](../../prepared_execution.go).
Measure representative cases before selecting the refactor. There is no target Measure representative cases before selecting the refactor. There is no target
copy count and no requirement to remove every defensive copy; retain the simpler copy count and no requirement to remove every defensive copy; retain the simpler
@@ -146,6 +153,19 @@ errors. The target is inspection cost driven by the information being inspected,
not unrelated provider payloads. This requires no new public lookup API and not unrelated provider payloads. This requires no new public lookup API and
does not move provenance analysis into ordinary prompt execution. does not move provenance analysis into ordinary prompt execution.
## Suggested Order Of Work
Start with basic validation and template output (items 2 and 3), then file-read
capacity (item 5). These are localized changes with directly identifiable
allocation costs. Follow with request encoding and identity-only inspection
(items 4 and 7), preserving their error and selection semantics.
Keep prepared-state consolidation (item 6) conditional on measured savings and
a clear ownership analysis; it has the greatest risk of weakening isolation or
concurrency behavior. The ConfigKit request (item 1) can proceed independently
and does not block local improvements. This order is a recommendation, not a
dependency between otherwise independent changes.
## Benchmarking And Verification ## Benchmarking And Verification
Use focused, offline before/after benchmarks to establish useful savings in Use focused, offline before/after benchmarks to establish useful savings in
@@ -153,6 +173,18 @@ time, allocated bytes, and allocation count. Cover representative small inputs
as well as larger messages, responses, files, schemas, and provider maps where as well as larger messages, responses, files, schemas, and provider maps where
they exercise the affected path. they exercise the affected path.
Preserve reproducible fixtures and comparison methods for the temporary probes
behind the review figures. Record the baseline and changed revisions, Go and
dependency versions, relevant machine details, benchmark commands, and repeated
results alongside the comparison. The existing construction benchmark already
provides a reproducible fixture; capture equivalent evidence for the other
affected paths before using their preliminary figures to judge improvements.
Distinguish cumulative allocations from retained memory. Where relevant,
measure retained heap with representative lifetimes and concurrency, especially
for builder-backed strings and long-lived prepared handles. A reduction in
allocated bytes alone does not establish a reduction in retained memory.
Add coverage for the production captured-template renderer: the existing Add coverage for the production captured-template renderer: the existing
artifact-reference benchmark uses the reparsing fallback. Include representative artifact-reference benchmark uses the reparsing fallback. Include representative
assembled preparation and execution with an injected model client so local assembled preparation and execution with an injected model client so local