Refine performance roadmap after review
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Computational Efficiency Improvements
|
||||
|
||||
Status: Proposed scope following the computational-efficiency review.
|
||||
Status: Accepted scope following review; implementation has not started.
|
||||
|
||||
## Purpose
|
||||
|
||||
@@ -9,9 +9,10 @@ preparation, rendering, and execution paths. Large payloads and concurrent
|
||||
library use make redundant full-payload copies especially worth addressing.
|
||||
Prefer small, idiomatic improvements over additional framework machinery.
|
||||
|
||||
This roadmap captures the intended outcomes and recommended changes. It is not
|
||||
a staged implementation plan or a new public performance guarantee. Existing
|
||||
contracts remain owned by GoDoc, the [format reference](../formats.md), and the
|
||||
This roadmap captures the intended outcomes, recommended changes, and suggested
|
||||
order of work. It is not a detailed implementation plan or a new public
|
||||
performance guarantee. Existing contracts remain owned by GoDoc, the
|
||||
[format reference](../formats.md), and the
|
||||
[integration contract](../integrations/openai-compatible-chat.md). Preserve the
|
||||
boundaries in the [architecture policy](../policy/architecture.md) and
|
||||
[configuration ADR](../adr/0001-configuration-boundary.md).
|
||||
@@ -120,12 +121,18 @@ Review ownership through
|
||||
[prepared execution](../../internal/usecase/prepared_execution.go) and
|
||||
[public conversion](../../convert.go). Creation currently builds two deep
|
||||
snapshots, and returning details adds further internal and public copies.
|
||||
Large schemas and nested provider maps can make this expensive.
|
||||
Large schemas and nested provider maps can make this expensive. Copying message
|
||||
structures does not itself duplicate their immutable string contents; measure
|
||||
mutable payload copies separately from message length.
|
||||
|
||||
Where internal ownership is demonstrably exclusive, consolidate overlapping
|
||||
copies or transfer owned values rather than cloning again. Retain copies needed
|
||||
to isolate engine state, the execution payload, returned details, and injected
|
||||
collaborators. Preserve prepared-execution lifecycle and concurrency behavior.
|
||||
Before removing a copy, identify which component owns each mutable value and
|
||||
whether another component can still access it. In particular, preserve the
|
||||
independence and availability of details after execution or discard described
|
||||
by the [public handle contract](../../prepared_execution.go).
|
||||
|
||||
Measure representative cases before selecting the refactor. There is no target
|
||||
copy count and no requirement to remove every defensive copy; retain the simpler
|
||||
@@ -146,6 +153,19 @@ errors. The target is inspection cost driven by the information being inspected,
|
||||
not unrelated provider payloads. This requires no new public lookup API and
|
||||
does not move provenance analysis into ordinary prompt execution.
|
||||
|
||||
## Suggested Order Of Work
|
||||
|
||||
Start with basic validation and template output (items 2 and 3), then file-read
|
||||
capacity (item 5). These are localized changes with directly identifiable
|
||||
allocation costs. Follow with request encoding and identity-only inspection
|
||||
(items 4 and 7), preserving their error and selection semantics.
|
||||
|
||||
Keep prepared-state consolidation (item 6) conditional on measured savings and
|
||||
a clear ownership analysis; it has the greatest risk of weakening isolation or
|
||||
concurrency behavior. The ConfigKit request (item 1) can proceed independently
|
||||
and does not block local improvements. This order is a recommendation, not a
|
||||
dependency between otherwise independent changes.
|
||||
|
||||
## Benchmarking And Verification
|
||||
|
||||
Use focused, offline before/after benchmarks to establish useful savings in
|
||||
@@ -153,6 +173,18 @@ time, allocated bytes, and allocation count. Cover representative small inputs
|
||||
as well as larger messages, responses, files, schemas, and provider maps where
|
||||
they exercise the affected path.
|
||||
|
||||
Preserve reproducible fixtures and comparison methods for the temporary probes
|
||||
behind the review figures. Record the baseline and changed revisions, Go and
|
||||
dependency versions, relevant machine details, benchmark commands, and repeated
|
||||
results alongside the comparison. The existing construction benchmark already
|
||||
provides a reproducible fixture; capture equivalent evidence for the other
|
||||
affected paths before using their preliminary figures to judge improvements.
|
||||
|
||||
Distinguish cumulative allocations from retained memory. Where relevant,
|
||||
measure retained heap with representative lifetimes and concurrency, especially
|
||||
for builder-backed strings and long-lived prepared handles. A reduction in
|
||||
allocated bytes alone does not establish a reduction in retained memory.
|
||||
|
||||
Add coverage for the production captured-template renderer: the existing
|
||||
artifact-reference benchmark uses the reparsing fallback. Include representative
|
||||
assembled preparation and execution with an injected model client so local
|
||||
|
||||
Reference in New Issue
Block a user