Plan the computational efficiency implementation
This commit is contained in:
@@ -1,23 +1,24 @@
|
||||
# Computational Efficiency Roadmap
|
||||
|
||||
**Status:** Proposed; implementation pending.
|
||||
**Status:** Ready for implementation; implementation pending.
|
||||
|
||||
## Objective And Scope
|
||||
## Objective And Ownership
|
||||
|
||||
Reduce the time, allocations, retained memory, and repeated I/O of routine
|
||||
release operations. Implement improvements behind the existing `releasekit`
|
||||
and `buildinfo` facades, following the
|
||||
[architecture policy](../policy/architecture.md) and
|
||||
[routine-release priorities](../adr/0002-routine-releases.md).
|
||||
release operations. Improvements belong behind the existing `releasekit` and
|
||||
`buildinfo` facades and follow the [architecture policy](../policy/architecture.md)
|
||||
and [routine-release priorities](../adr/0002-routine-releases.md).
|
||||
|
||||
This roadmap captures the computational efficiency review of commit `db9e24f`
|
||||
on 2026-10-09. It owns proposed work and sequencing. The
|
||||
[internal inventory](../internal/overview.md) and focused internal documents
|
||||
own implemented behavior; Go declarations and GoDoc own public contracts.
|
||||
This document owns the desired target state and its rationale. The
|
||||
[implementation plan](implementation.md) owns stages, tasks, validation, and
|
||||
implementation status. The [internal inventory](../internal/overview.md) and
|
||||
focused internal documents own implemented mechanics; Go declarations and
|
||||
GoDoc own public contracts.
|
||||
|
||||
## Review Evidence
|
||||
|
||||
Temporary offline probes produced the following local measurements:
|
||||
The computational efficiency review examined commit `db9e24f` on 2026-10-09.
|
||||
Temporary offline probes produced these local measurements:
|
||||
|
||||
| Probe | Result |
|
||||
| --- | --- |
|
||||
@@ -27,7 +28,7 @@ Temporary offline probes produced the following local measurements:
|
||||
| Resolve runtime version through the Go build metadata fallback in a test executable | Approximately 2 microseconds, 1,184 allocated bytes, and seven allocations per call. |
|
||||
| Resolve an injected runtime version | Zero allocations. |
|
||||
|
||||
The blob timings cover content access, with tree metadata discovery measured
|
||||
Blob timings cover content access, with tree metadata discovery measured
|
||||
separately. They exclude Go tests, compilation, formatting, and Markdown parsing.
|
||||
The fixture contained small, loose Git objects. These are component measurements
|
||||
on one machine; overall release gains depend on consumer workloads. The probes
|
||||
@@ -36,128 +37,82 @@ were temporary and left the repository unchanged.
|
||||
Static review also identified repeated artifact hashing and unconditional Go
|
||||
output copying. Their workload impact remains to be measured.
|
||||
|
||||
## Priorities
|
||||
## Desired Target State
|
||||
|
||||
### Git Content Access
|
||||
|
||||
Selected file checks share an operation-scoped Git content reader. Requests
|
||||
remain lazy, use committed object IDs, and consume responses incrementally.
|
||||
Subprocess startup is amortized across the operation. Metadata-only checks
|
||||
continue to use the captured tree without starting a content reader.
|
||||
|
||||
The reader preserves offline object access, cancellation, contextual inspection
|
||||
errors, and explicit process cleanup. Each validation operation owns its reader
|
||||
and reusable data independently.
|
||||
|
||||
### Validation Memory And Reuse
|
||||
|
||||
One-use regular-file bytes belong to their immediate consumer and are released
|
||||
from retained references after inspection. Repeated work reuses immutable parsed
|
||||
or compact results keyed by object ID within the relevant check. Reusable
|
||||
symlink targets belong to the candidate tree. Link checks apply shared document
|
||||
data in each source path's own context, preserving findings, source locations,
|
||||
and relative destinations.
|
||||
|
||||
This replaces retention of every inspected raw blob with data that supports
|
||||
actual repeated work. Explicit repeated raw reads may reread an object through
|
||||
the batch reader. The same bytes can be interpreted separately by different
|
||||
checks; reuse within one interpretation preserves object-ID deduplication.
|
||||
|
||||
### Artifact Verification
|
||||
|
||||
Delivery declarations reuse recorded digests and collect file metadata. One
|
||||
preflight verification checks current local bytes before hosted requests. This
|
||||
combines adjacent declaration-time and asset-verification hash passes for both
|
||||
managed and prebuilt delivery.
|
||||
|
||||
Build-result hashing and upload preflight retain their separate responsibilities.
|
||||
Prebuilt artifacts remain caller-owned. Exact remote content comparison,
|
||||
targeted reconciliation, additive retries, and partial-delivery outcomes retain
|
||||
their existing semantics under the [Gitea contract](../integrations/gitea.md).
|
||||
|
||||
### Go Command Output
|
||||
|
||||
Status-consuming commands return status and useful failure diagnostics.
|
||||
Output-consuming commands receive ownership of completed captured bytes without
|
||||
an additional full-buffer copy. Forwarded output, synchronization, command
|
||||
environments, cancellation, and failure causes retain their existing behavior.
|
||||
|
||||
Complete diagnostic capture remains in memory for this implementation. Its
|
||||
remaining cost is measured with large output. Disk-backed capture is a possible
|
||||
follow-up when its benefit justifies additional filesystem and lifecycle work.
|
||||
|
||||
### Runtime Version Lookup
|
||||
|
||||
The runtime implementation resolves its immutable build-metadata fallback once
|
||||
and shares that result safely across calls. Linker injection retains precedence
|
||||
and its inexpensive path. Main-module version selection and the development
|
||||
fallback retain the [runtime facade's semantics](../consumers/runtime-version.md).
|
||||
The runtime dependency graph remains small and independent of orchestration.
|
||||
|
||||
## Priorities And Evaluation
|
||||
|
||||
| Priority | Improvement | Expected benefit |
|
||||
| --- | --- | --- |
|
||||
| High | Batch selected Git blob reads | Amortize subprocess startup and object access across many files. |
|
||||
| High | Clarify file-content ownership and selective caching | Reduce copies and memory retained across validation checks. |
|
||||
| High | Clarify file-content ownership and selective reuse | Reduce copies and memory retained across validation checks. |
|
||||
| Medium | Consolidate artifact hashing | Reduce repeated full-file reads and SHA-256 calculations. |
|
||||
| Lower | Avoid unnecessary Go-output copies | Reduce allocation and peak memory for commands with large diagnostics. |
|
||||
| Low | Cache runtime version fallback | Reduce repeated metadata parsing when consumers call the accessor frequently. |
|
||||
|
||||
## Implementation Sequence
|
||||
|
||||
### 1. Establish Representative Measurements
|
||||
|
||||
- Add a small set of offline benchmarks covering many unique Git blobs,
|
||||
repeated reads, large source or Markdown files, and artifact hashing.
|
||||
- Record elapsed time, allocations, retained memory where relevant, and bytes
|
||||
processed. Separate Git content access from formatting, parsing, and Go work.
|
||||
- Include repeated object IDs and packed objects to reflect useful cache and
|
||||
batching cases. Use temporary local Git repositories and local HTTP fixtures.
|
||||
- Measure representative complete check and delivery workflows locally to
|
||||
distinguish component gains from overall gains.
|
||||
|
||||
Follow the [testing policy](../policy/testing.md) for proportionate behavioral
|
||||
coverage and fixture isolation. The
|
||||
[development guide](../development.md) owns validation commands and synthetic
|
||||
module-cache isolation requirements. Benchmark results inform implementation;
|
||||
machine-specific timings remain measurements.
|
||||
|
||||
### 2. Improve Blob Access And File-Content Ownership
|
||||
|
||||
Owners: [Git object access](../../internal/git/source.go),
|
||||
[candidate tree](../../internal/validation/tree.go), and
|
||||
[Markdown parsing cache](../../internal/validation/markdown.go).
|
||||
|
||||
- Introduce an operation-scoped, streaming Git batch reader for selected blobs.
|
||||
Retain lazy selection and object-ID deduplication.
|
||||
- Manage child-process lifetime explicitly, including completion, cancellation,
|
||||
cleanup, and useful failures for missing local objects or malformed responses.
|
||||
Preserve the existing offline object-access behavior.
|
||||
- Define ownership of cached and returned bytes. Use immutable internal reads
|
||||
where consumers can safely share contents, and preserve isolation where
|
||||
mutation or caller ownership requires copies.
|
||||
- Retain reusable data selectively. Formatting generally consumes source once;
|
||||
Markdown checks already reuse parsed documents. Measure the tradeoff between
|
||||
retained raw contents and rereads of shared objects or symlink text.
|
||||
- Consume batch results incrementally to reduce temporary content allocations.
|
||||
|
||||
Reuse existing formatting, Markdown, committed-source, and destination-resolution
|
||||
tests. Extend coverage for realistic batching, cancellation, and ownership risks.
|
||||
Compare allocation and retained-memory results with the baseline as well as
|
||||
elapsed time.
|
||||
|
||||
### 3. Consolidate Artifact Hashing
|
||||
|
||||
Owners: [builder](../../internal/build/build.go),
|
||||
[delivery declarations](../../internal/delivery/assets.go),
|
||||
[delivery orchestration](../../internal/delivery/delivery.go), and
|
||||
[asset verification and upload](../../internal/gitea/upload.go).
|
||||
|
||||
A fresh managed delivery currently hashes each missing binary during building,
|
||||
declaration construction, local verification, and upload preflight. Each pass
|
||||
streams its input; repeated passes add work proportional to artifact size.
|
||||
|
||||
- Consolidate adjacent declaration and asset verification work in one internal
|
||||
owner and reuse the digest and size it computes.
|
||||
- Reuse freshly computed metadata within managed delivery while artifact
|
||||
ownership remains with that operation.
|
||||
- Preserve meaningful verification boundaries for caller-supplied prebuilt
|
||||
artifacts, detection of changed contents, and accurate partial-delivery results.
|
||||
- Preserve exact remote content comparison and targeted reconciliation after
|
||||
uncertain writes, as described by the
|
||||
[Gitea integration contract](../integrations/gitea.md).
|
||||
|
||||
Measure large-artifact delivery for fresh uploads and retries. Reuse existing
|
||||
tampering, conflicting-content, partial-delivery, and additive-retry tests.
|
||||
|
||||
### 4. Reduce Go-Output Copying
|
||||
|
||||
Owner: [Go command execution](../../internal/gotool/gotool.go).
|
||||
|
||||
Go command execution currently retains combined output and copies the complete
|
||||
buffer before returning, including for callers that discard the returned bytes.
|
||||
|
||||
- Distinguish output-consuming commands, such as compiler identification, from
|
||||
commands whose callers consume status and diagnostics.
|
||||
- Eliminate unnecessary final copies for status-only callers while preserving
|
||||
forwarded output and failure diagnostics.
|
||||
- Evaluate disk-backed diagnostic capture if measured large-log workloads
|
||||
justify its filesystem and lifecycle costs.
|
||||
|
||||
Measure allocations and peak memory with ordinary output and large diagnostics.
|
||||
Preserve cancellation, command failure attribution, and output-writer behavior.
|
||||
|
||||
### 5. Cache Runtime Version Fallback
|
||||
|
||||
Owner: [runtime version implementation](../../internal/runtimeinfo/version.go).
|
||||
|
||||
- Cache the immutable result of the Go build metadata fallback internally using
|
||||
a proportionate standard-library mechanism.
|
||||
- Preserve injected-version precedence, main-module version selection, and the
|
||||
development fallback defined by the runtime facade.
|
||||
- Keep the [runtime dependency boundary](../policy/architecture.md#runtime-version-boundary)
|
||||
intact and verify concurrent access if caching introduces shared state.
|
||||
|
||||
Compare repeated fallback access with the already inexpensive injected path.
|
||||
This work has limited impact for consumers that read their version once.
|
||||
|
||||
## Completion And Follow-Up
|
||||
|
||||
Implement and validate the work in priority order, then report the measured
|
||||
results and any remaining opportunities for review. Run the applicable existing
|
||||
behavioral suite and facade checks using the developer guide. Update canonical
|
||||
internal documentation when each implementation lands.
|
||||
Evaluate component and complete-workflow gains with offline local fixtures,
|
||||
including unique and repeated object IDs, packed objects, large files, fresh
|
||||
delivery, and retries. Record workload descriptions, allocations, elapsed time,
|
||||
and retained-memory observations. Timings inform implementation and remain
|
||||
measurements specific to the workload and machine.
|
||||
|
||||
Preserve the efficient foundations already present: batched tree metadata,
|
||||
deduplicated object access, parsed Markdown reuse, streamed artifact I/O, one
|
||||
selected Go test suite, native artifact reuse for version smoke checks, and
|
||||
source-only delivery without compilation. Publication guards and exact content
|
||||
comparisons remain part of the release procedure throughout this work.
|
||||
|
||||
Summarize component and complete-workflow results with workload descriptions and
|
||||
measurement limitations. Record lower-priority opportunities as completed or
|
||||
deferred according to their measured benefit and maintenance cost. Retire this
|
||||
roadmap when its work and disposition are recorded in the appropriate owners.
|
||||
parsed Markdown reuse, streamed artifact I/O, one selected Go test suite, native
|
||||
artifact reuse for version smoke checks, and source-only delivery without
|
||||
compilation. Publication guards and exact content comparisons retain their
|
||||
existing responsibilities throughout this work.
|
||||
|
||||
Reference in New Issue
Block a user