From ca571aca4f2fd43798174a6d0d7c5a588b5347c9 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 9 Oct 2026 00:42:08 +0000 Subject: [PATCH] Plan the computational efficiency implementation --- docs/roadmap/implementation.md | 381 +++++++++++++++++++++++++++++++++ docs/roadmap/performance.md | 209 +++++++----------- 2 files changed, 463 insertions(+), 127 deletions(-) create mode 100644 docs/roadmap/implementation.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..ba6bb59 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,381 @@ +# Computational Efficiency Implementation Plan + +**Status:** Ready for implementation; all stages pending. + +## Purpose And Baseline + +This plan turns the [performance roadmap](performance.md) into ordered work for +a Luna coding agent. The inspected baseline is commit +`ff90510c380b47a23d0ac1cd8c314b1c4366a15b`; this planning change updates only +the two roadmap documents. Recheck the worktree and relevant code before each +stage because subsequent stages build on earlier changes. + +Execute Stage 1 through Stage 11 in order. Each stage is one focused +implementation prompt. Complete its changes, applicable validation, canonical +documentation updates, and local commit under the +[agent policy](../policy/agents.md), then proceed. The sequence ends with a +results report for review. Consumer pilots, hosted operations, and intermediate +releases are outside this implementation sequence. + +Read [development.md](../development.md) and its applicable policy guide. +Apply the [architecture policy](../policy/architecture.md), +[testing policy](../policy/testing.md), and +[documentation policy](../policy/documentation.md). Use graph tools for code +discovery as instructed by `AGENTS.md`. + +## Selected Design And Scope + +| Area | Selected implementation | +| --- | --- | +| Child process | A small piped-process helper in `internal/process`, sharing existing command setup and error wrapping. | +| Git blobs | One lazy `git cat-file --batch` session per candidate tree, with one request/response exchange at a time. | +| Raw file bytes | Caller-owned bytes for regular-file reads; immutable symlink strings cached by object ID. | +| Repeated checks | Compact formatting outcomes, parsed Markdown documents, and replacement summaries cached by object ID within their check. | +| Artifact hashing | Data-only declarations followed by one existing local preflight verification; retain build hashing and missing-upload verification. | +| Go output | Separate status and captured-output methods; transfer completed buffers without copying. | +| Runtime version | A standard-library `sync.OnceValue` cache for the fallback, with an immediate injected-version return. | + +These are internal implementation choices. Preserve public facade signatures, +configuration, CLI semantics, selected check ordering, callback sequencing, +findings, failure causes, publication guards, and delivery recovery. Existing +public contracts remain authoritative. New helpers expose only the internal +operations needed here. File-size selection and existing supported inputs keep +their current semantics. + +The Git session and checker caches belong to one validation operation. Existing +sequential check execution remains sufficient for this design. The runtime +fallback is immutable process data and uses the standard library's concurrent +initialization mechanism. + +## Stage 1: Measure Candidate Content And Validation + +**Owners:** Benchmark files in `internal/validation` and root facade fixtures. + +1. Add benchmarks named with the `BenchmarkPerformance` prefix. Reuse local Git + fixture patterns; adapt shared helper parameters to `testing.TB` only where + a benchmark needs them. +2. Measure content access and formatting for 1,000 unique regular Go blobs. + Include loose and packed objects and a repeated-object-ID case. Measure + Markdown checking with a 1 MiB document, repeated links to it, and identical + contents under different directories. +3. Measure a complete facade `Check` using a temporary committed Go module with + a valid release note, Go formatting, local replacements, and Markdown selected. + Keep its Go suite real and offline. Prepare fixtures outside timed work. + Report allocations and bytes processed, separating content access, tree + metadata, and Go test/compile work. +4. Add one diagnostic retained-memory observation after file processing, with + the candidate and document cache kept alive, to expose raw-content retention. + Keep GC/profile collection outside timing loops. +5. Create a measurement record in this document for baseline and final results. + Include commit, Go/Git versions, fixture sizes, packed/loose state, and cache + state. Keep raw benchmark output or profiles in temporary storage. + +**Validation and completion:** Run these benchmarks with `-run '^$'`, +`-bench '^BenchmarkPerformance'`, `-benchmem`, and `-count=3`. Verify the +fixtures inspect the intended bytes. Synthetic modules use test-owned module +caches under the developer guide. Timing ratios remain reported measurements; +behavioral tests protect content, failures, and ownership. This stage changes +measurement support only. + +## Stage 2: Measure Delivery, Output, And Runtime Lookup + +**Owners:** Benchmark files in `internal/delivery`, `internal/gotool`, +`internal/runtimeinfo`, and root delivery fixtures. + +1. Extend the same benchmark prefix to declaration/preflight processing of two + deterministic 16 MiB artifact files. Measure prebuilt facade `Deliver` + against a local HTTP fixture for fresh uploads and matching retries. + Prepare files outside timed work and reset hosted state for each fresh run. + Stream binary bodies through the fixture's digest/size checks and serve + matching files incrementally so fixture buffering does not obscure allocation + improvements. Describe fixture processing costs in the workflow results. +2. Benchmark completed Go-output capture with modest output and 8 MiB diagnostics, + including forwarding to `io.Discard`. Reuse the combined writer for component + measurements and existing child fixtures for any command measurements. +3. Measure injected version access and repeated real metadata-fallback access. + Preserve benchmark coverage of initialization and repeated access when the + implementation later adds caching. +4. Record time, allocations, bytes processed, and workload/cache settings in the + measurement record. Run the benchmark set with the same flags and repetitions + as Stage 1. Verify delivered bytes and matching-retry behavior. + +**Validation and completion:** Measurements are repeatable offline using local +collaborators and temporary files. Fixture state changes occur outside timed +work where they are setup. These two measurement stages establish the baseline +before production refactoring begins. Default tests remain independent of real +credentials and mutable external services. + +## Stage 3: Add The Piped Child-Process Boundary + +**Owner:** [internal/process](../../internal/process/process.go) and its tests. + +1. Factor existing command construction and command-error wrapping into small + private helpers shared by `Run` and a new internal `StartPiped` operation. + Preserve arguments, directory, environment overlay/removal, stderr routing, + context handling, and wrapped exit errors. +2. `StartPiped` accepts an `Invocation` for command settings and stderr; stdin + and stdout are supplied by pipes. Return a small process handle containing + the input writer and output reader, plus `Cancel` and `Wait` operations. + Keep the underlying `exec.Cmd` private. Use a derived cancellable context so + the owner can abort a blocked exchange without cancelling its caller. +3. Close acquired pipes on setup/start failure. The caller owns input closure, + stdout consumption, and one wait/reap after a successful start. `Cancel` + interrupts the child; `Wait` retains the current context/exit error semantics. + Read stdout to completion before a normal wait. Release the derived context + when the process is reaped; abort paths close active pipes before waiting. +4. Extend the existing test executable helper for an exchange that returns + output before stdin EOF. Test real argument/environment propagation, normal + EOF completion, cancellation after a readiness signal, start failure, and + a nonzero child exit. Synchronize cancellation with the child instead of + depending on sleep duration. + +**Validation and completion:** Run `internal/process` tests. Existing `Run` +behavior still passes, and the piped helper supports repeated exchanges followed +by deterministic cleanup. Update the process inventory description when needed. + +## Stage 4: Implement The Git Batch Reader + +**Owners:** A focused reader file and tests under `internal/git`, reusing +[source.go](../../internal/git/source.go). + +1. Add an internal `BlobReader` that stores a repository directory and parent + context. Create its process lazily on the first content request through + `process.StartPiped` with `git cat-file --batch`. +2. Use the same child-only `GIT_NO_LAZY_FETCH=1` and `GIT_ALLOW_PROTOCOL=` + settings as `readLocalObjects`. Keep existing standalone `ReadBlob`, + `ReadCommitFile`, and batched metadata functions for their current callers. +3. Implement `Read(ctx, objectID) ([]byte, error)` as one serialized exchange: + validate the ID using the existing helper, write its newline-terminated + request, read the response header, validate object/type/size, read exactly + the declared bytes, and consume the framing newline. Compare hexadecimal + IDs without narrowing the existing accepted case conventions. +4. Allocate the response body once and transfer it to the caller. Use buffered + header reading and `io.ReadFull`; blob contents can contain arbitrary bytes + and newlines. Read only requested bodies and avoid whole-session buffering. + Size conversion errors reflect representability on the host. +5. Treat Git's missing-object response as a contextual inspection error. Preserve + stderr and process causes when the child fails. A broken protocol or I/O + exchange aborts that session and reaps the child; return the original cause. +6. Honor the read context during blocked I/O, for example by registering + `context.AfterFunc` to cancel the child and stopping that registration after + the exchange. Coordinate callback completion when the stop function reports + it has begun; stopping alone does not wait for that callback. If cancellation + races completion, preserve its cause and abort the session. Read captured + stderr after reaping to avoid writer races. +7. Add idempotent `Close`: an unused reader has nothing to close; a healthy + reader closes stdin, finishes stdout consumption, and waits; an aborted + reader closes pipes, cancels, and waits. Preserve unexpected exit failures; + deliberate abort cleanup retains the original read failure. + +**Validation and completion:** Test repeated reads from loose and packed Git +objects, empty/binary bodies, and missing objects. Reuse the partial-clone test +approach to verify absent objects cause errors with no transport or object-store +population. Use a small child fixture for truncated framing and cancellation +mid-response. Assert content and cleanup outcomes; benchmark process reuse +without making a subprocess count a public contract. Run Git and process tests. + +## Stage 5: Give Candidate Validation A Scoped Reader + +**Owners:** [tree.go](../../internal/validation/tree.go), +[validation.go](../../internal/validation/validation.go), and tree fixtures. + +1. Give `CandidateTree` one lazy `BlobReader` and an internal `Close` method. + `ReadCandidateTree` still reads metadata only. Route blob misses through the + reader while temporarily retaining the existing byte-cache behavior. +2. The shared validation executor owns closure on every return after loading + a tree. Complete cleanup before product checks or build callbacks run. + Metadata-only selections leave the content reader unstarted. +3. Preserve an earlier check's `Failure` and cancellation cause during cleanup. + If cleanup reveals an unexpected child failure after successful checks, + return a contextual `Failure` attributed to the last content-reading check. + Track that check in the executor; normal intentional cancellation during + cleanup adds no independent failure classification. +4. Update direct tree users in tests and benchmarks to register cleanup or close + explicitly. Update synthetic tree fixtures to support lazy reader creation. + Keep tree paths, source capture, symlink resolution, and check order intact. + +**Validation and completion:** Run validation, candidate, publication, and facade +candidate tests. Cover early check failure and cancellation with cleanup, and +preserve workspace/file-size metadata behavior. Re-run content benchmarks with +reader startup and closure included in the content-operation timing. Update +[validation mechanics](../internal/validation.md) for the landed lifecycle. + +## Stage 6: Release One-Use Source Bytes And Reuse Formatting Results + +**Owners:** `internal/validation/tree.go`, `validation.go`, and destination tests. + +1. Remove the tree's general raw-blob cache. A regular-file `ReadFile` returns + its newly read, caller-owned body without a cache copy. Its GoDoc describes + that ownership and the scoped reader; independently repeated raw reads can + request the same object again. +2. Cache committed symlink targets as owned immutable strings keyed by object + ID. Add a private target-reading helper used by destination resolution and + module-symlink handling. Preserve original target bytes and traversal order. + Explicit `ReadFile` of a symlink still returns independently owned bytes. +3. In `checkGoFormat`, cache the formatted/unformatted result for successfully + inspected objects by object ID. Release source and formatted bytes from + retained references after comparing them. Emit a finding at every selected + unformatted path, including duplicate-content paths. Parse/read errors still + stop with the current path and cause; check cancellation for each entry. +4. Adapt tests that construct the old cache directly. Preserve observable + isolation: mutating returned bytes cannot alter later validation or reads. + Reuse destination-cycle and repeated-symlink tests for shared target strings. + +**Validation and completion:** Run validation tests and formatting/destination +benchmarks. Confirm many unique Go files no longer remain as retained source +buffers after formatting, and duplicate-content files retain per-path findings. +Record the tradeoff for explicit repeated raw reads. Update validation mechanics +with the new ownership and reuse choices. + +## Stage 7: Reuse Owned Markdown And Module Results + +**Owners:** [markdown.go](../../internal/validation/markdown.go), +[markdown_check.go](../../internal/validation/markdown_check.go), and +[local_replacements.go](../../internal/validation/local_replacements.go). + +1. Key parsed Markdown reuse by resolved regular blob object ID. Keep a path + lookup when useful for callers, backed by the shared document records. + Relative link resolution and findings use the current source path; a shared + parsed document stores content-relative locations, links, and anchors only. +2. Give retained link destinations and titles their own string storage, using + `strings.Clone` at extraction where needed, so parser/source views cannot + keep a large raw body alive. Cache parsed records, release the AST and source + from retained references, and preserve heading collision rules, image behavior, + and use-site locations. +3. Refactor module-declaration selection to identify the resolved regular entry + before reading its body. After successful parsing, cache a compact list of + filesystem replacements by resolved object ID: target string and source line. + Own retained target strings and release raw bytes and the parsed syntax tree + from retained references. +4. Apply each cached replacement summary to the current declaration path when + constructing findings. Resolve each symlink declaration in its own path + context. On read or parse failure, preserve the current declaration's path + and underlying cause; preserve findings order and versioned-replacement rules. + +**Validation and completion:** Run Markdown/module and facade validation tests. +Add behavioral cases for identical Markdown in different directories and two +module declarations resolving to one object, including multiple findings with +correct paths/lines. Reuse parser and boundary/cycle coverage. Re-run +large-document and repeated-content benchmarks and retained-memory observations. +Update the validation document's cache description to match implemented ownership. + +## Stage 8: Consolidate Delivery Preflight Hashing + +**Owners:** [delivery declarations](../../internal/delivery/assets.go), +[delivery orchestration](../../internal/delivery/delivery.go), and their tests. + +1. Make `declaredBuildAssets` perform structural declaration checks, stat files, + reuse the recorded artifact SHA-256, and construct expected assets and the + manifest. Remove its file-opening/hash pass. Keep target, filename, regular + file, declared digest format, and manifest validation at the appropriate + existing owner. +2. Keep one `gitea.VerifyLocalAssets` call in `Deliver` after declaration and + build-identity checks and before hosted requests. This verifies actual bytes + and sizes against recorded identities for both managed and prebuilt assets. +3. Keep builder hashing, CLI receipt verification, and missing-upload preflight + in their current phases. Managed delivery reuses its fresh build metadata for + declarations; prebuilt delivery reuses its caller's recorded digests only as + expected identities until current bytes have been verified. +4. Move the declaration helper's tampering assertion to delivery preflight's + behavioral owner. Verify same-size changed contents fail before any hosted + request. Reuse source-only, retained-prebuilt, managed cleanup, conflict, + partial-upload, and retry coverage. + +**Validation and completion:** Run delivery, Gitea, facade delivery, and CLI +receipt/delivery tests. Re-run artifact and complete delivery benchmarks. A fresh +missing binary has three hash passes including a managed build, or two within +prebuilt delivery; these are explanatory measurements of the chosen flow. +Streaming upload, exact remote comparison, and later upload verification remain +intact. Update the inventory's delivery mechanics where needed. + +## Stage 9: Transfer Completed Go Output Without Copies + +**Owners:** [gotool.go](../../internal/gotool/gotool.go), validation/build callers, +and Go tool fixture tests. + +1. Give `Tool` a status-returning `Run(...args) error` and an output-returning + `Output(...args) ([]byte, error)`, with one private execution path. Keep their + existing directory, mode, target, writer, and argument inputs. +2. On completion, a status-only success releases its capture. An output-consuming + call receives the completed buffer's backing bytes with ownership transferred + from the private writer. Replace `combinedWriter.Bytes` copying with a small + synchronized ownership-transfer operation used after child output has finished. +3. Preserve complete failure diagnostics, original causes, combined-output + synchronization, and forwarding errors. If `Output` fails, return its captured + bytes and diagnostic error as before. Status failures format the same useful + diagnostics without creating an unnecessary intermediate snapshot. +4. Convert validation test/vet/compile and artifact-build calls to status-only + `Run`. Use `Output` for compiler identification. Adapt preparation failure + formatting to the same completed-buffer ownership discipline, preserving its + network-capable environment. Leave configuration discovery's separate stdout + JSON and stderr handling intact. +5. Keep full capture in memory. Measure remaining large-log retention and record + the disk-backed alternative as follow-up evidence, as scoped by the roadmap. + +**Validation and completion:** Extend existing process/Go fixtures for successful +capture, large failure output, cancellation, and a failing output writer. Verify +forwarded bytes and meaningful error content/cause. Run Go tool, validation, +build, preparation, and affected CLI tests. Re-run output benchmarks and update +[Go execution mechanics](../internal/gotool.md). + +## Stage 10: Cache The Runtime Metadata Fallback + +**Owners:** [runtimeinfo/version.go](../../internal/runtimeinfo/version.go) and +runtime/buildinfo tests. + +1. Preserve `injectedVersion` and its linker identity. `Version` checks and + returns injection first, then calls a fallback accessor created with + `sync.OnceValue`. +2. Keep the existing pure `version(injected, readBuildInfo)` selection helper. + Add a small private fallback factory accepting the metadata reader and + returning the cached accessor. Production supplies `debug.ReadBuildInfo`; + tests create independent cache instances with controlled metadata. +3. Reuse selection tests. Add repeated/concurrent access coverage with a + counting reader to verify reuse and stable results. Test unavailable and + development metadata fallback through isolated instances. Keep shared + production state immutable during tests. + +**Validation and completion:** Run runtime/buildinfo tests, the existing consumer +build-and-run injection test, and focused race validation of the fallback cache. +Run facade/dependency checks. Re-run version benchmarks after initializing the +cache and report initialization separately from repeated access. The runtime +facade and its dependency boundary retain their existing public shape. + +## Stage 11: Verify The Complete Implementation And Record Results + +**Owners:** Affected tests and canonical internal documentation; roadmap status. + +1. Run the complete offline test suite and compiled facade/dependency checks + using the [developer guide](../development.md). Resolve remaining failures + within this performance scope. Reuse targeted race results where code is + unchanged; broaden them only for a concrete remaining concurrency risk. +2. Run the Stage 1 and Stage 2 benchmark set again with comparable toolchain, + fixtures, cache state, repetitions, and measurement boundaries. Compare + component and complete `Check`/`Deliver` results. Record memory and explicit + reread tradeoffs as well as time and allocation improvements. +3. Record final results alongside the baseline in this document. Explain the + remaining Go compilation/test cost, remote-comparison work, and diagnostic + retention where measurements show they matter. Separate measured effects + from expectations and include machine/workload limitations. +4. Check canonical internal docs describe landed reader lifetime, source/result + ownership, verification flow, output ownership, and runtime reuse. Preserve + existing public contracts; update a public document only if its explanation + became inaccurate. Check local links and prose whitespace. +5. Mark each stage's actual completion, update the feature roadmap status, and + commit the completed work under repository policy. End with a concise report + of changes, tests, performance results, and remaining follow-up opportunities. + Retain these planning documents with their results for the final review. + +**Completion evidence:** All five target improvements are implemented, relevant +existing behavior passes, the public facade checks pass, comparable measurements +are recorded, and the worktree is clean. Completion depends on observed behavior +and reviewed diffs; benchmarks establish improvements within their measured +scope without adding numerical runtime guarantees. + +## Open Questions + +None required for this implementation. The plan selects scoped batch reading, +result-based reuse, preserved verification boundaries, in-memory diagnostics, +and cached runtime fallback. Disk-backed diagnostics remain a follow-up option +whose benefit and lifecycle costs can be assessed from the recorded output +measurements. diff --git a/docs/roadmap/performance.md b/docs/roadmap/performance.md index c52cc93..1322e20 100644 --- a/docs/roadmap/performance.md +++ b/docs/roadmap/performance.md @@ -1,23 +1,24 @@ # Computational Efficiency Roadmap -**Status:** Proposed; implementation pending. +**Status:** Ready for implementation; implementation pending. -## Objective And Scope +## Objective And Ownership Reduce the time, allocations, retained memory, and repeated I/O of routine -release operations. Implement improvements behind the existing `releasekit` -and `buildinfo` facades, following the -[architecture policy](../policy/architecture.md) and -[routine-release priorities](../adr/0002-routine-releases.md). +release operations. Improvements belong behind the existing `releasekit` and +`buildinfo` facades and follow the [architecture policy](../policy/architecture.md) +and [routine-release priorities](../adr/0002-routine-releases.md). -This roadmap captures the computational efficiency review of commit `db9e24f` -on 2026-10-09. It owns proposed work and sequencing. The -[internal inventory](../internal/overview.md) and focused internal documents -own implemented behavior; Go declarations and GoDoc own public contracts. +This document owns the desired target state and its rationale. The +[implementation plan](implementation.md) owns stages, tasks, validation, and +implementation status. The [internal inventory](../internal/overview.md) and +focused internal documents own implemented mechanics; Go declarations and +GoDoc own public contracts. ## Review Evidence -Temporary offline probes produced the following local measurements: +The computational efficiency review examined commit `db9e24f` on 2026-10-09. +Temporary offline probes produced these local measurements: | Probe | Result | | --- | --- | @@ -27,7 +28,7 @@ Temporary offline probes produced the following local measurements: | Resolve runtime version through the Go build metadata fallback in a test executable | Approximately 2 microseconds, 1,184 allocated bytes, and seven allocations per call. | | Resolve an injected runtime version | Zero allocations. | -The blob timings cover content access, with tree metadata discovery measured +Blob timings cover content access, with tree metadata discovery measured separately. They exclude Go tests, compilation, formatting, and Markdown parsing. The fixture contained small, loose Git objects. These are component measurements on one machine; overall release gains depend on consumer workloads. The probes @@ -36,128 +37,82 @@ were temporary and left the repository unchanged. Static review also identified repeated artifact hashing and unconditional Go output copying. Their workload impact remains to be measured. -## Priorities +## Desired Target State + +### Git Content Access + +Selected file checks share an operation-scoped Git content reader. Requests +remain lazy, use committed object IDs, and consume responses incrementally. +Subprocess startup is amortized across the operation. Metadata-only checks +continue to use the captured tree without starting a content reader. + +The reader preserves offline object access, cancellation, contextual inspection +errors, and explicit process cleanup. Each validation operation owns its reader +and reusable data independently. + +### Validation Memory And Reuse + +One-use regular-file bytes belong to their immediate consumer and are released +from retained references after inspection. Repeated work reuses immutable parsed +or compact results keyed by object ID within the relevant check. Reusable +symlink targets belong to the candidate tree. Link checks apply shared document +data in each source path's own context, preserving findings, source locations, +and relative destinations. + +This replaces retention of every inspected raw blob with data that supports +actual repeated work. Explicit repeated raw reads may reread an object through +the batch reader. The same bytes can be interpreted separately by different +checks; reuse within one interpretation preserves object-ID deduplication. + +### Artifact Verification + +Delivery declarations reuse recorded digests and collect file metadata. One +preflight verification checks current local bytes before hosted requests. This +combines adjacent declaration-time and asset-verification hash passes for both +managed and prebuilt delivery. + +Build-result hashing and upload preflight retain their separate responsibilities. +Prebuilt artifacts remain caller-owned. Exact remote content comparison, +targeted reconciliation, additive retries, and partial-delivery outcomes retain +their existing semantics under the [Gitea contract](../integrations/gitea.md). + +### Go Command Output + +Status-consuming commands return status and useful failure diagnostics. +Output-consuming commands receive ownership of completed captured bytes without +an additional full-buffer copy. Forwarded output, synchronization, command +environments, cancellation, and failure causes retain their existing behavior. + +Complete diagnostic capture remains in memory for this implementation. Its +remaining cost is measured with large output. Disk-backed capture is a possible +follow-up when its benefit justifies additional filesystem and lifecycle work. + +### Runtime Version Lookup + +The runtime implementation resolves its immutable build-metadata fallback once +and shares that result safely across calls. Linker injection retains precedence +and its inexpensive path. Main-module version selection and the development +fallback retain the [runtime facade's semantics](../consumers/runtime-version.md). +The runtime dependency graph remains small and independent of orchestration. + +## Priorities And Evaluation | Priority | Improvement | Expected benefit | | --- | --- | --- | | High | Batch selected Git blob reads | Amortize subprocess startup and object access across many files. | -| High | Clarify file-content ownership and selective caching | Reduce copies and memory retained across validation checks. | +| High | Clarify file-content ownership and selective reuse | Reduce copies and memory retained across validation checks. | | Medium | Consolidate artifact hashing | Reduce repeated full-file reads and SHA-256 calculations. | | Lower | Avoid unnecessary Go-output copies | Reduce allocation and peak memory for commands with large diagnostics. | | Low | Cache runtime version fallback | Reduce repeated metadata parsing when consumers call the accessor frequently. | -## Implementation Sequence - -### 1. Establish Representative Measurements - -- Add a small set of offline benchmarks covering many unique Git blobs, - repeated reads, large source or Markdown files, and artifact hashing. -- Record elapsed time, allocations, retained memory where relevant, and bytes - processed. Separate Git content access from formatting, parsing, and Go work. -- Include repeated object IDs and packed objects to reflect useful cache and - batching cases. Use temporary local Git repositories and local HTTP fixtures. -- Measure representative complete check and delivery workflows locally to - distinguish component gains from overall gains. - -Follow the [testing policy](../policy/testing.md) for proportionate behavioral -coverage and fixture isolation. The -[development guide](../development.md) owns validation commands and synthetic -module-cache isolation requirements. Benchmark results inform implementation; -machine-specific timings remain measurements. - -### 2. Improve Blob Access And File-Content Ownership - -Owners: [Git object access](../../internal/git/source.go), -[candidate tree](../../internal/validation/tree.go), and -[Markdown parsing cache](../../internal/validation/markdown.go). - -- Introduce an operation-scoped, streaming Git batch reader for selected blobs. - Retain lazy selection and object-ID deduplication. -- Manage child-process lifetime explicitly, including completion, cancellation, - cleanup, and useful failures for missing local objects or malformed responses. - Preserve the existing offline object-access behavior. -- Define ownership of cached and returned bytes. Use immutable internal reads - where consumers can safely share contents, and preserve isolation where - mutation or caller ownership requires copies. -- Retain reusable data selectively. Formatting generally consumes source once; - Markdown checks already reuse parsed documents. Measure the tradeoff between - retained raw contents and rereads of shared objects or symlink text. -- Consume batch results incrementally to reduce temporary content allocations. - -Reuse existing formatting, Markdown, committed-source, and destination-resolution -tests. Extend coverage for realistic batching, cancellation, and ownership risks. -Compare allocation and retained-memory results with the baseline as well as -elapsed time. - -### 3. Consolidate Artifact Hashing - -Owners: [builder](../../internal/build/build.go), -[delivery declarations](../../internal/delivery/assets.go), -[delivery orchestration](../../internal/delivery/delivery.go), and -[asset verification and upload](../../internal/gitea/upload.go). - -A fresh managed delivery currently hashes each missing binary during building, -declaration construction, local verification, and upload preflight. Each pass -streams its input; repeated passes add work proportional to artifact size. - -- Consolidate adjacent declaration and asset verification work in one internal - owner and reuse the digest and size it computes. -- Reuse freshly computed metadata within managed delivery while artifact - ownership remains with that operation. -- Preserve meaningful verification boundaries for caller-supplied prebuilt - artifacts, detection of changed contents, and accurate partial-delivery results. -- Preserve exact remote content comparison and targeted reconciliation after - uncertain writes, as described by the - [Gitea integration contract](../integrations/gitea.md). - -Measure large-artifact delivery for fresh uploads and retries. Reuse existing -tampering, conflicting-content, partial-delivery, and additive-retry tests. - -### 4. Reduce Go-Output Copying - -Owner: [Go command execution](../../internal/gotool/gotool.go). - -Go command execution currently retains combined output and copies the complete -buffer before returning, including for callers that discard the returned bytes. - -- Distinguish output-consuming commands, such as compiler identification, from - commands whose callers consume status and diagnostics. -- Eliminate unnecessary final copies for status-only callers while preserving - forwarded output and failure diagnostics. -- Evaluate disk-backed diagnostic capture if measured large-log workloads - justify its filesystem and lifecycle costs. - -Measure allocations and peak memory with ordinary output and large diagnostics. -Preserve cancellation, command failure attribution, and output-writer behavior. - -### 5. Cache Runtime Version Fallback - -Owner: [runtime version implementation](../../internal/runtimeinfo/version.go). - -- Cache the immutable result of the Go build metadata fallback internally using - a proportionate standard-library mechanism. -- Preserve injected-version precedence, main-module version selection, and the - development fallback defined by the runtime facade. -- Keep the [runtime dependency boundary](../policy/architecture.md#runtime-version-boundary) - intact and verify concurrent access if caching introduces shared state. - -Compare repeated fallback access with the already inexpensive injected path. -This work has limited impact for consumers that read their version once. - -## Completion And Follow-Up - -Implement and validate the work in priority order, then report the measured -results and any remaining opportunities for review. Run the applicable existing -behavioral suite and facade checks using the developer guide. Update canonical -internal documentation when each implementation lands. +Evaluate component and complete-workflow gains with offline local fixtures, +including unique and repeated object IDs, packed objects, large files, fresh +delivery, and retries. Record workload descriptions, allocations, elapsed time, +and retained-memory observations. Timings inform implementation and remain +measurements specific to the workload and machine. Preserve the efficient foundations already present: batched tree metadata, -deduplicated object access, parsed Markdown reuse, streamed artifact I/O, one -selected Go test suite, native artifact reuse for version smoke checks, and -source-only delivery without compilation. Publication guards and exact content -comparisons remain part of the release procedure throughout this work. - -Summarize component and complete-workflow results with workload descriptions and -measurement limitations. Record lower-priority opportunities as completed or -deferred according to their measured benefit and maintenance cost. Retire this -roadmap when its work and disposition are recorded in the appropriate owners. +parsed Markdown reuse, streamed artifact I/O, one selected Go test suite, native +artifact reuse for version smoke checks, and source-only delivery without +compilation. Publication guards and exact content comparisons retain their +existing responsibilities throughout this work.