Refine performance roadmap after review
This commit is contained in:
@@ -1,6 +1,6 @@
|
|||||||
# Computational Efficiency Improvements
|
# Computational Efficiency Improvements
|
||||||
|
|
||||||
Status: Proposed scope following the computational-efficiency review.
|
Status: Accepted scope following review; implementation has not started.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
@@ -9,9 +9,10 @@ preparation, rendering, and execution paths. Large payloads and concurrent
|
|||||||
library use make redundant full-payload copies especially worth addressing.
|
library use make redundant full-payload copies especially worth addressing.
|
||||||
Prefer small, idiomatic improvements over additional framework machinery.
|
Prefer small, idiomatic improvements over additional framework machinery.
|
||||||
|
|
||||||
This roadmap captures the intended outcomes and recommended changes. It is not
|
This roadmap captures the intended outcomes, recommended changes, and suggested
|
||||||
a staged implementation plan or a new public performance guarantee. Existing
|
order of work. It is not a detailed implementation plan or a new public
|
||||||
contracts remain owned by GoDoc, the [format reference](../formats.md), and the
|
performance guarantee. Existing contracts remain owned by GoDoc, the
|
||||||
|
[format reference](../formats.md), and the
|
||||||
[integration contract](../integrations/openai-compatible-chat.md). Preserve the
|
[integration contract](../integrations/openai-compatible-chat.md). Preserve the
|
||||||
boundaries in the [architecture policy](../policy/architecture.md) and
|
boundaries in the [architecture policy](../policy/architecture.md) and
|
||||||
[configuration ADR](../adr/0001-configuration-boundary.md).
|
[configuration ADR](../adr/0001-configuration-boundary.md).
|
||||||
@@ -120,12 +121,18 @@ Review ownership through
|
|||||||
[prepared execution](../../internal/usecase/prepared_execution.go) and
|
[prepared execution](../../internal/usecase/prepared_execution.go) and
|
||||||
[public conversion](../../convert.go). Creation currently builds two deep
|
[public conversion](../../convert.go). Creation currently builds two deep
|
||||||
snapshots, and returning details adds further internal and public copies.
|
snapshots, and returning details adds further internal and public copies.
|
||||||
Large schemas and nested provider maps can make this expensive.
|
Large schemas and nested provider maps can make this expensive. Copying message
|
||||||
|
structures does not itself duplicate their immutable string contents; measure
|
||||||
|
mutable payload copies separately from message length.
|
||||||
|
|
||||||
Where internal ownership is demonstrably exclusive, consolidate overlapping
|
Where internal ownership is demonstrably exclusive, consolidate overlapping
|
||||||
copies or transfer owned values rather than cloning again. Retain copies needed
|
copies or transfer owned values rather than cloning again. Retain copies needed
|
||||||
to isolate engine state, the execution payload, returned details, and injected
|
to isolate engine state, the execution payload, returned details, and injected
|
||||||
collaborators. Preserve prepared-execution lifecycle and concurrency behavior.
|
collaborators. Preserve prepared-execution lifecycle and concurrency behavior.
|
||||||
|
Before removing a copy, identify which component owns each mutable value and
|
||||||
|
whether another component can still access it. In particular, preserve the
|
||||||
|
independence and availability of details after execution or discard described
|
||||||
|
by the [public handle contract](../../prepared_execution.go).
|
||||||
|
|
||||||
Measure representative cases before selecting the refactor. There is no target
|
Measure representative cases before selecting the refactor. There is no target
|
||||||
copy count and no requirement to remove every defensive copy; retain the simpler
|
copy count and no requirement to remove every defensive copy; retain the simpler
|
||||||
@@ -146,6 +153,19 @@ errors. The target is inspection cost driven by the information being inspected,
|
|||||||
not unrelated provider payloads. This requires no new public lookup API and
|
not unrelated provider payloads. This requires no new public lookup API and
|
||||||
does not move provenance analysis into ordinary prompt execution.
|
does not move provenance analysis into ordinary prompt execution.
|
||||||
|
|
||||||
|
## Suggested Order Of Work
|
||||||
|
|
||||||
|
Start with basic validation and template output (items 2 and 3), then file-read
|
||||||
|
capacity (item 5). These are localized changes with directly identifiable
|
||||||
|
allocation costs. Follow with request encoding and identity-only inspection
|
||||||
|
(items 4 and 7), preserving their error and selection semantics.
|
||||||
|
|
||||||
|
Keep prepared-state consolidation (item 6) conditional on measured savings and
|
||||||
|
a clear ownership analysis; it has the greatest risk of weakening isolation or
|
||||||
|
concurrency behavior. The ConfigKit request (item 1) can proceed independently
|
||||||
|
and does not block local improvements. This order is a recommendation, not a
|
||||||
|
dependency between otherwise independent changes.
|
||||||
|
|
||||||
## Benchmarking And Verification
|
## Benchmarking And Verification
|
||||||
|
|
||||||
Use focused, offline before/after benchmarks to establish useful savings in
|
Use focused, offline before/after benchmarks to establish useful savings in
|
||||||
@@ -153,6 +173,18 @@ time, allocated bytes, and allocation count. Cover representative small inputs
|
|||||||
as well as larger messages, responses, files, schemas, and provider maps where
|
as well as larger messages, responses, files, schemas, and provider maps where
|
||||||
they exercise the affected path.
|
they exercise the affected path.
|
||||||
|
|
||||||
|
Preserve reproducible fixtures and comparison methods for the temporary probes
|
||||||
|
behind the review figures. Record the baseline and changed revisions, Go and
|
||||||
|
dependency versions, relevant machine details, benchmark commands, and repeated
|
||||||
|
results alongside the comparison. The existing construction benchmark already
|
||||||
|
provides a reproducible fixture; capture equivalent evidence for the other
|
||||||
|
affected paths before using their preliminary figures to judge improvements.
|
||||||
|
|
||||||
|
Distinguish cumulative allocations from retained memory. Where relevant,
|
||||||
|
measure retained heap with representative lifetimes and concurrency, especially
|
||||||
|
for builder-backed strings and long-lived prepared handles. A reduction in
|
||||||
|
allocated bytes alone does not establish a reduction in retained memory.
|
||||||
|
|
||||||
Add coverage for the production captured-template renderer: the existing
|
Add coverage for the production captured-template renderer: the existing
|
||||||
artifact-reference benchmark uses the reparsing fallback. Include representative
|
artifact-reference benchmark uses the reparsing fallback. Include representative
|
||||||
assembled preparation and execution with an injected model client so local
|
assembled preparation and execution with an injected model client so local
|
||||||
|
|||||||
Reference in New Issue
Block a user