Plan the computational efficiency implementation

This commit is contained in:
2026-10-09 00:42:08 +00:00
parent ff90510c38
commit ca571aca4f
2 changed files with 463 additions and 127 deletions

View File

@@ -1,23 +1,24 @@
# Computational Efficiency Roadmap
**Status:** Proposed; implementation pending.
**Status:** Ready for implementation; implementation pending.
## Objective And Scope
## Objective And Ownership
Reduce the time, allocations, retained memory, and repeated I/O of routine
release operations. Implement improvements behind the existing `releasekit`
and `buildinfo` facades, following the
[architecture policy](../policy/architecture.md) and
[routine-release priorities](../adr/0002-routine-releases.md).
release operations. Improvements belong behind the existing `releasekit` and
`buildinfo` facades and follow the [architecture policy](../policy/architecture.md)
and [routine-release priorities](../adr/0002-routine-releases.md).
This roadmap captures the computational efficiency review of commit `db9e24f`
on 2026-10-09. It owns proposed work and sequencing. The
[internal inventory](../internal/overview.md) and focused internal documents
own implemented behavior; Go declarations and GoDoc own public contracts.
This document owns the desired target state and its rationale. The
[implementation plan](implementation.md) owns stages, tasks, validation, and
implementation status. The [internal inventory](../internal/overview.md) and
focused internal documents own implemented mechanics; Go declarations and
GoDoc own public contracts.
## Review Evidence
Temporary offline probes produced the following local measurements:
The computational efficiency review examined commit `db9e24f` on 2026-10-09.
Temporary offline probes produced these local measurements:
| Probe | Result |
| --- | --- |
@@ -27,7 +28,7 @@ Temporary offline probes produced the following local measurements:
| Resolve runtime version through the Go build metadata fallback in a test executable | Approximately 2 microseconds, 1,184 allocated bytes, and seven allocations per call. |
| Resolve an injected runtime version | Zero allocations. |
The blob timings cover content access, with tree metadata discovery measured
Blob timings cover content access, with tree metadata discovery measured
separately. They exclude Go tests, compilation, formatting, and Markdown parsing.
The fixture contained small, loose Git objects. These are component measurements
on one machine; overall release gains depend on consumer workloads. The probes
@@ -36,128 +37,82 @@ were temporary and left the repository unchanged.
Static review also identified repeated artifact hashing and unconditional Go
output copying. Their workload impact remains to be measured.
## Priorities
## Desired Target State
### Git Content Access
Selected file checks share an operation-scoped Git content reader. Requests
remain lazy, use committed object IDs, and consume responses incrementally.
Subprocess startup is amortized across the operation. Metadata-only checks
continue to use the captured tree without starting a content reader.
The reader preserves offline object access, cancellation, contextual inspection
errors, and explicit process cleanup. Each validation operation owns its reader
and reusable data independently.
### Validation Memory And Reuse
One-use regular-file bytes belong to their immediate consumer and are released
from retained references after inspection. Repeated work reuses immutable parsed
or compact results keyed by object ID within the relevant check. Reusable
symlink targets belong to the candidate tree. Link checks apply shared document
data in each source path's own context, preserving findings, source locations,
and relative destinations.
This replaces retention of every inspected raw blob with data that supports
actual repeated work. Explicit repeated raw reads may reread an object through
the batch reader. The same bytes can be interpreted separately by different
checks; reuse within one interpretation preserves object-ID deduplication.
### Artifact Verification
Delivery declarations reuse recorded digests and collect file metadata. One
preflight verification checks current local bytes before hosted requests. This
combines adjacent declaration-time and asset-verification hash passes for both
managed and prebuilt delivery.
Build-result hashing and upload preflight retain their separate responsibilities.
Prebuilt artifacts remain caller-owned. Exact remote content comparison,
targeted reconciliation, additive retries, and partial-delivery outcomes retain
their existing semantics under the [Gitea contract](../integrations/gitea.md).
### Go Command Output
Status-consuming commands return status and useful failure diagnostics.
Output-consuming commands receive ownership of completed captured bytes without
an additional full-buffer copy. Forwarded output, synchronization, command
environments, cancellation, and failure causes retain their existing behavior.
Complete diagnostic capture remains in memory for this implementation. Its
remaining cost is measured with large output. Disk-backed capture is a possible
follow-up when its benefit justifies additional filesystem and lifecycle work.
### Runtime Version Lookup
The runtime implementation resolves its immutable build-metadata fallback once
and shares that result safely across calls. Linker injection retains precedence
and its inexpensive path. Main-module version selection and the development
fallback retain the [runtime facade's semantics](../consumers/runtime-version.md).
The runtime dependency graph remains small and independent of orchestration.
## Priorities And Evaluation
| Priority | Improvement | Expected benefit |
| --- | --- | --- |
| High | Batch selected Git blob reads | Amortize subprocess startup and object access across many files. |
| High | Clarify file-content ownership and selective caching | Reduce copies and memory retained across validation checks. |
| High | Clarify file-content ownership and selective reuse | Reduce copies and memory retained across validation checks. |
| Medium | Consolidate artifact hashing | Reduce repeated full-file reads and SHA-256 calculations. |
| Lower | Avoid unnecessary Go-output copies | Reduce allocation and peak memory for commands with large diagnostics. |
| Low | Cache runtime version fallback | Reduce repeated metadata parsing when consumers call the accessor frequently. |
## Implementation Sequence
### 1. Establish Representative Measurements
- Add a small set of offline benchmarks covering many unique Git blobs,
repeated reads, large source or Markdown files, and artifact hashing.
- Record elapsed time, allocations, retained memory where relevant, and bytes
processed. Separate Git content access from formatting, parsing, and Go work.
- Include repeated object IDs and packed objects to reflect useful cache and
batching cases. Use temporary local Git repositories and local HTTP fixtures.
- Measure representative complete check and delivery workflows locally to
distinguish component gains from overall gains.
Follow the [testing policy](../policy/testing.md) for proportionate behavioral
coverage and fixture isolation. The
[development guide](../development.md) owns validation commands and synthetic
module-cache isolation requirements. Benchmark results inform implementation;
machine-specific timings remain measurements.
### 2. Improve Blob Access And File-Content Ownership
Owners: [Git object access](../../internal/git/source.go),
[candidate tree](../../internal/validation/tree.go), and
[Markdown parsing cache](../../internal/validation/markdown.go).
- Introduce an operation-scoped, streaming Git batch reader for selected blobs.
Retain lazy selection and object-ID deduplication.
- Manage child-process lifetime explicitly, including completion, cancellation,
cleanup, and useful failures for missing local objects or malformed responses.
Preserve the existing offline object-access behavior.
- Define ownership of cached and returned bytes. Use immutable internal reads
where consumers can safely share contents, and preserve isolation where
mutation or caller ownership requires copies.
- Retain reusable data selectively. Formatting generally consumes source once;
Markdown checks already reuse parsed documents. Measure the tradeoff between
retained raw contents and rereads of shared objects or symlink text.
- Consume batch results incrementally to reduce temporary content allocations.
Reuse existing formatting, Markdown, committed-source, and destination-resolution
tests. Extend coverage for realistic batching, cancellation, and ownership risks.
Compare allocation and retained-memory results with the baseline as well as
elapsed time.
### 3. Consolidate Artifact Hashing
Owners: [builder](../../internal/build/build.go),
[delivery declarations](../../internal/delivery/assets.go),
[delivery orchestration](../../internal/delivery/delivery.go), and
[asset verification and upload](../../internal/gitea/upload.go).
A fresh managed delivery currently hashes each missing binary during building,
declaration construction, local verification, and upload preflight. Each pass
streams its input; repeated passes add work proportional to artifact size.
- Consolidate adjacent declaration and asset verification work in one internal
owner and reuse the digest and size it computes.
- Reuse freshly computed metadata within managed delivery while artifact
ownership remains with that operation.
- Preserve meaningful verification boundaries for caller-supplied prebuilt
artifacts, detection of changed contents, and accurate partial-delivery results.
- Preserve exact remote content comparison and targeted reconciliation after
uncertain writes, as described by the
[Gitea integration contract](../integrations/gitea.md).
Measure large-artifact delivery for fresh uploads and retries. Reuse existing
tampering, conflicting-content, partial-delivery, and additive-retry tests.
### 4. Reduce Go-Output Copying
Owner: [Go command execution](../../internal/gotool/gotool.go).
Go command execution currently retains combined output and copies the complete
buffer before returning, including for callers that discard the returned bytes.
- Distinguish output-consuming commands, such as compiler identification, from
commands whose callers consume status and diagnostics.
- Eliminate unnecessary final copies for status-only callers while preserving
forwarded output and failure diagnostics.
- Evaluate disk-backed diagnostic capture if measured large-log workloads
justify its filesystem and lifecycle costs.
Measure allocations and peak memory with ordinary output and large diagnostics.
Preserve cancellation, command failure attribution, and output-writer behavior.
### 5. Cache Runtime Version Fallback
Owner: [runtime version implementation](../../internal/runtimeinfo/version.go).
- Cache the immutable result of the Go build metadata fallback internally using
a proportionate standard-library mechanism.
- Preserve injected-version precedence, main-module version selection, and the
development fallback defined by the runtime facade.
- Keep the [runtime dependency boundary](../policy/architecture.md#runtime-version-boundary)
intact and verify concurrent access if caching introduces shared state.
Compare repeated fallback access with the already inexpensive injected path.
This work has limited impact for consumers that read their version once.
## Completion And Follow-Up
Implement and validate the work in priority order, then report the measured
results and any remaining opportunities for review. Run the applicable existing
behavioral suite and facade checks using the developer guide. Update canonical
internal documentation when each implementation lands.
Evaluate component and complete-workflow gains with offline local fixtures,
including unique and repeated object IDs, packed objects, large files, fresh
delivery, and retries. Record workload descriptions, allocations, elapsed time,
and retained-memory observations. Timings inform implementation and remain
measurements specific to the workload and machine.
Preserve the efficient foundations already present: batched tree metadata,
deduplicated object access, parsed Markdown reuse, streamed artifact I/O, one
selected Go test suite, native artifact reuse for version smoke checks, and
source-only delivery without compilation. Publication guards and exact content
comparisons remain part of the release procedure throughout this work.
Summarize component and complete-workflow results with workload descriptions and
measurement limitations. Record lower-priority opportunities as completed or
deferred according to their measured benefit and maintenance cost. Retire this
roadmap when its work and disposition are recorded in the appropriate owners.
parsed Markdown reuse, streamed artifact I/O, one selected Go test suite, native
artifact reuse for version smoke checks, and source-only delivery without
compilation. Publication guards and exact content comparisons retain their
existing responsibilities throughout this work.