Files
ReleaseKit/docs/roadmap/implementation.md

545 lines
35 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Computational Efficiency Implementation Plan
**Status:** Stages 1–7 complete; Stages 8–11 pending.
## Purpose And Baseline
This plan turns the [performance roadmap](performance.md) into ordered work for
a Luna coding agent. The inspected baseline is commit
`ff90510c380b47a23d0ac1cd8c314b1c4366a15b`; this planning change updates only
the two roadmap documents. Recheck the worktree and relevant code before each
stage because subsequent stages build on earlier changes.
Execute Stage 1 through Stage 11 in order. Each stage is one focused
implementation prompt. Complete its changes, applicable validation, canonical
documentation updates, and local commit under the
[agent policy](../policy/agents.md), then proceed. The sequence ends with a
results report for review. Consumer pilots, hosted operations, and intermediate
releases are outside this implementation sequence.
Read [development.md](../development.md) and its applicable policy guide.
Apply the [architecture policy](../policy/architecture.md),
[testing policy](../policy/testing.md), and
[documentation policy](../policy/documentation.md). Use graph tools for code
discovery as instructed by `AGENTS.md`.
## Selected Design And Scope
| Area | Selected implementation |
| --- | --- |
| Child process | A small piped-process helper in `internal/process`, sharing existing command setup and error wrapping. |
| Git blobs | One lazy `git cat-file --batch` session per candidate tree, with one request/response exchange at a time. |
| Raw file bytes | Caller-owned bytes for regular-file reads; immutable symlink strings cached by object ID. |
| Repeated checks | Compact formatting outcomes, parsed Markdown documents, and replacement summaries cached by object ID within their check. |
| Artifact hashing | Data-only declarations followed by one existing local preflight verification; retain build hashing and missing-upload verification. |
| Go output | Separate status and captured-output methods; transfer completed buffers without copying. |
| Runtime version | A standard-library `sync.OnceValue` cache for the fallback, with an immediate injected-version return. |
These are internal implementation choices. Preserve public facade signatures,
configuration, CLI semantics, selected check ordering, callback sequencing,
findings, failure causes, publication guards, and delivery recovery. Existing
public contracts remain authoritative. New helpers expose only the internal
operations needed here. File-size selection and existing supported inputs keep
their current semantics.
The Git session and checker caches belong to one validation operation. Existing
sequential check execution remains sufficient for this design. The runtime
fallback is immutable process data and uses the standard library's concurrent
initialization mechanism.
## Stage 1: Measure Candidate Content And Validation
**Owners:** Benchmark files in `internal/validation` and root facade fixtures.
**Completion:** Complete. Measurement-only benchmarks cover tree metadata,
content reads and formatting for 1,000 unique or repeated-ID Go blobs in loose
and packed repositories, repeated links to identical 1 MiB Markdown documents,
retained content after processing, and a real offline facade `Check`. Fixtures
verify committed bytes, unique/repeated object IDs, link counts, and Git pack
state. The facade fixture selects Go formatting, local-replacement inspection,
and Markdown while using test-owned module and build caches.
### Baseline Measurement Record
The baseline is commit `ca571aca4f2fd43798174a6d0d7c5a588b5347c9`, before adding
measurement support. Results use Go `go1.26.8 linux/amd64`, Git `2.53.0`, and a
Linux x86_64 Intel Core i5-7260U at 2.20 GHz. The command was
`go test ./internal/validation . -run '^$' -bench '^BenchmarkPerformance' -benchmem -count=3`.
Each candidate-content operation used a fresh empty content cache with tree
metadata prepared outside its timer. The loose fixtures had loose objects and
no pack; packed fixtures were checked after `git gc --aggressive --prune=now`.
The unique Go fixture contains 1,000 distinct, formatted files totaling 49,000
bytes. The repeated-ID fixture contains 1,000 paths sharing one 15-byte blob.
The Markdown fixture contains two identical 1 MiB documents, a 47,780-byte
index with 1,000 repeated links, and 2,144,932 bytes processed per check. The
complete-check module contains 906 bytes of Go/module input. Its `GOMODCACHE`
and `GOCACHE` are test-owned temporary directories, and module network access
is disabled; the build cache is warmed before timing.
| Measurement | Baseline observations | Final comparison |
| --- | --- | --- |
| Tree metadata, 1,000 entries | Loose: 27.5–28.2 ms; packed: 25.0–28.1 ms. About 1.28 MB allocated per operation. | Pending Stage 11. |
| Unique-blob reads, 49,000 bytes | Loose: 1.86–1.96 s; packed: 2.57–3.08 s. About 34.57 MB allocated per operation. | Pending Stage 11. |
| Repeated-ID reads, 15,000 bytes across 1,000 paths | Loose: 2.12–3.00 ms; packed: 3.42–7.75 ms. About 50.7 KB allocated per operation. | Pending Stage 11. |
| Formatting, 1,000 unique files | Loose: 1.87–2.94 s; packed: 2.12–4.55 s. About 36.9 MB allocated per operation. | Pending Stage 11. |
| Formatting, 1,000 paths sharing one object | Loose: 4.52–12.66 ms; packed: 6.11–8.62 ms. About 1.49 MB allocated per operation. | Pending Stage 11. |
| Markdown links and parsing | 19.4–21.5 ms for two identical 1 MiB documents and 1,000 links; about 7.67 MB allocated per operation. | Pending Stage 11. |
| Facade check, standard Go suite only | 385–690 ms; 906 module bytes; 336–340 KB allocated in the benchmark process. | Pending Stage 11. |
| Facade check with selected file checks | 528–634 ms; 906 module bytes; 632–646 KB allocated in the benchmark process. | Pending Stage 11. |
| Retained content after formatting and Markdown parsing | 1,385,040–1,385,616 bytes above the post-GC metadata baseline, with the candidate and Markdown cache kept alive. | Pending Stage 11. |
Raw output is in temporary storage at `/tmp/releasekit-performance-baseline.txt`.
The shared machine produced substantial timing variation, especially for packed
content and the full check; allocations and retained-byte observations were
more stable. `B/op` for facade checks measures allocations in the benchmark
process and does not include allocations inside its Go subprocess. These are
workload-specific component measurements, not runtime guarantees.
1. Add benchmarks named with the `BenchmarkPerformance` prefix. Reuse local Git
fixture patterns; adapt shared helper parameters to `testing.TB` only where
a benchmark needs them.
2. Measure content access and formatting for 1,000 unique regular Go blobs.
Include loose and packed objects and a repeated-object-ID case. Measure
Markdown checking with a 1 MiB document, repeated links to it, and identical
contents under different directories.
3. Measure a complete facade `Check` using a temporary committed Go module with
a valid release note, Go formatting, local replacements, and Markdown selected.
Keep its Go suite real and offline. Prepare fixtures outside timed work.
Report allocations and bytes processed, separating content access, tree
metadata, and Go test/compile work.
4. Add one diagnostic retained-memory observation after file processing, with
the candidate and document cache kept alive, to expose raw-content retention.
Keep GC/profile collection outside timing loops.
5. Create a measurement record in this document for baseline and final results.
Include commit, Go/Git versions, fixture sizes, packed/loose state, and cache
state. Keep raw benchmark output or profiles in temporary storage.
**Validation and completion:** Run these benchmarks with `-run '^$'`,
`-bench '^BenchmarkPerformance'`, `-benchmem`, and `-count=3`. Verify the
fixtures inspect the intended bytes. Synthetic modules use test-owned module
caches under the developer guide. Timing ratios remain reported measurements;
behavioral tests protect content, failures, and ownership. This stage changes
measurement support only.
## Stage 2: Measure Delivery, Output, And Runtime Lookup
**Owners:** Benchmark files in `internal/delivery`, `internal/gotool`,
`internal/runtimeinfo`, and root delivery fixtures.
**Completion:** Complete. Measurements cover declarations and local
verification of two deterministic 16 MiB files, prebuilt facade delivery for
fresh uploads and matching retries, combined output capture and forwarding at
4 KiB and 8 MiB, and injected and real metadata version access. The local HTTP
fixture hashes and counts incoming multipart bytes incrementally and streams
matching files from immutable fixture data. It verifies uploaded identities and
that retries match all three declared assets without uploading again.
### Delivery, Output, And Runtime Baseline
The baseline is commit `29e1bab`, before production refactoring, using the same
Go, Git, operating system, processor, benchmark command, and three repetitions
recorded above. Artifact declaration and preflight fixtures contain two
deterministic 16 MiB files (33,554,432 binary bytes per operation); setup and
initial writes are outside the timers. Declaration currently hashes each file,
and declaration plus local verification therefore measures both hashing passes.
The facade benchmark uses a committed tagged source fixture and prebuilt
artifacts, so its time includes local source checks, hashing, HTTP requests, and
streamed body comparison or upload, but no Go compilation. Fresh hosted state is
reset outside the timed region for every operation. Retry state is prepared
before timing. The fixture server retains only digest and size metadata for
uploads and streams known matching bytes directly from shared fixture content.
Output measurements call the combined writer directly, with forwarding to
`io.Discard`, then capture its completed buffer. They include the current
completed-output copy. Runtime fallback measurements call `Version()` repeatedly
against real build metadata; injected access measures the selection helper's
injected-version branch. The benchmark process reports its own allocations,
including local HTTP fixture work; streamed artifact bodies are not buffered by
the fixture server.
| Measurement | Baseline observations | Final comparison |
| --- | --- | --- |
| Artifact declarations, two 16 MiB files | 94.3–95.4 ms; 33,554,432 bytes processed; about 68.9 KB allocated. | Pending Stage 11. |
| Declarations plus local verification | 186.7–188.4 ms; 33,554,432 binary bytes per each of two passes; about 136.2 KB allocated. | Pending Stage 11. |
| Facade delivery, fresh uploads | 429–447 ms; 32 MiB binary payload, two files plus manifest; 1.43–1.46 MB allocated. | Pending Stage 11. |
| Facade delivery, matching retry | 309.1–310.0 ms; 32 MiB of local verification plus streamed remote comparisons; 1.21–1.22 MB allocated. | Pending Stage 11. |
| Combined output, 4 KiB and `io.Discard` | 1.82–1.87 μs; 8,256 B and 3 allocations per operation. | Pending Stage 11. |
| Combined output, 8 MiB and `io.Discard` | 2.02–2.36 ms; 16.78 MB and 3 allocations per operation, including the completed capture copy. | Pending Stage 11. |
| Injected version selection | 0.745–0.754 ns; 0 B and 0 allocations. | Pending Stage 11. |
| Repeated real metadata fallback | 1.438–1.474 μs; 1,184 B and 7 allocations per operation. | Pending Stage 11. |
The delivery observations include local HTTP and filesystem costs specific to
these fixtures. The output component benchmark does not include child startup;
the real delivery benchmark does include HTTP request handling. These results
describe measured workloads and do not establish runtime guarantees.
1. Extend the same benchmark prefix to declaration/preflight processing of two
deterministic 16 MiB artifact files. Measure prebuilt facade `Deliver`
against a local HTTP fixture for fresh uploads and matching retries.
Prepare files outside timed work and reset hosted state for each fresh run.
Stream binary bodies through the fixture's digest/size checks and serve
matching files incrementally so fixture buffering does not obscure allocation
improvements. Describe fixture processing costs in the workflow results.
2. Benchmark completed Go-output capture with modest output and 8 MiB diagnostics,
including forwarding to `io.Discard`. Reuse the combined writer for component
measurements and existing child fixtures for any command measurements.
3. Measure injected version access and repeated real metadata-fallback access.
Preserve benchmark coverage of initialization and repeated access when the
implementation later adds caching.
4. Record time, allocations, bytes processed, and workload/cache settings in the
measurement record. Run the benchmark set with the same flags and repetitions
as Stage 1. Verify delivered bytes and matching-retry behavior.
**Validation and completion:** Measurements are repeatable offline using local
collaborators and temporary files. Fixture state changes occur outside timed
work where they are setup. These two measurement stages establish the baseline
before production refactoring begins. Default tests remain independent of real
credentials and mutable external services.
## Stage 3: Add The Piped Child-Process Boundary
**Owner:** [internal/process](../../internal/process/process.go) and its tests.
**Completion:** Complete. `Run` and `StartPiped` share command construction.
Piped children expose caller-owned input and output pipes, cancellation, and a
single underlying wait/reap while preserving parent-context and exit errors.
Tests cover argument, directory, and environment propagation; an exchange that
responds before input EOF; synchronized cancellation; startup failure; and a
nonzero exit. The package inventory now describes the pipe lifecycle.
1. Factor existing command construction and command-error wrapping into small
private helpers shared by `Run` and a new internal `StartPiped` operation.
Preserve arguments, directory, environment overlay/removal, stderr routing,
context handling, and wrapped exit errors.
2. `StartPiped` accepts an `Invocation` for command settings and stderr; stdin
and stdout are supplied by pipes. Return a small process handle containing
the input writer and output reader, plus `Cancel` and `Wait` operations.
Keep the underlying `exec.Cmd` private. Use a derived cancellable context so
the owner can abort a blocked exchange without cancelling its caller.
3. Close acquired pipes on setup/start failure. The caller owns input closure,
stdout consumption, and one wait/reap after a successful start. `Cancel`
interrupts the child; `Wait` retains the current context/exit error semantics.
Read stdout to completion before a normal wait. Release the derived context
when the process is reaped; abort paths close active pipes before waiting.
4. Extend the existing test executable helper for an exchange that returns
output before stdin EOF. Test real argument/environment propagation, normal
EOF completion, cancellation after a readiness signal, start failure, and
a nonzero child exit. Synchronize cancellation with the child instead of
depending on sleep duration.
**Validation and completion:** Run `internal/process` tests. Existing `Run`
behavior still passes, and the piped helper supports repeated exchanges followed
by deterministic cleanup. Update the process inventory description when needed.
## Stage 4: Implement The Git Batch Reader
**Owners:** A focused reader file and tests under `internal/git`, reusing
[source.go](../../internal/git/source.go).
**Completion:** Complete. `BlobReader` lazily starts one local-only
`git cat-file --batch` process, serializes request/response exchanges, and
returns caller-owned body bytes after validating the object ID, blob type,
representable size, and framing. Missing objects produce contextual errors
without invalidating a healthy session. Broken exchanges and cancellation close
the pipes and reap the child, preserving the original failure, Git stderr, and
process causes. Existing standalone blob and committed-file readers remain in
place. Tests cover repeated loose and packed reads, empty and binary bodies,
missing objects, partial clones, malformed/truncated responses, cancellation,
and child failures.
The process-reuse benchmark used one 9-byte blob. A new reader per read took
1.98–2.24 ms and allocated 37.9 KB in 190 allocations per operation; a shared
session took 57–82 μs and allocated 425 B in 8 allocations. The fresh-session
measurement includes process startup and clean close per read. The shared-session
timer includes lazy startup on its first read; final cleanup is outside the
timer. This is a fixture-specific measurement, not a subprocess-count contract.
1. Add an internal `BlobReader` that stores a repository directory and parent
context. Create its process lazily on the first content request through
`process.StartPiped` with `git cat-file --batch`.
2. Use the same child-only `GIT_NO_LAZY_FETCH=1` and `GIT_ALLOW_PROTOCOL=`
settings as `readLocalObjects`. Keep existing standalone `ReadBlob`,
`ReadCommitFile`, and batched metadata functions for their current callers.
3. Implement `Read(ctx, objectID) ([]byte, error)` as one serialized exchange:
validate the ID using the existing helper, write its newline-terminated
request, read the response header, validate object/type/size, read exactly
the declared bytes, and consume the framing newline. Compare hexadecimal
IDs without narrowing the existing accepted case conventions.
4. Allocate the response body once and transfer it to the caller. Use buffered
header reading and `io.ReadFull`; blob contents can contain arbitrary bytes
and newlines. Read only requested bodies and avoid whole-session buffering.
Size conversion errors reflect representability on the host.
5. Treat Git's missing-object response as a contextual inspection error. Preserve
stderr and process causes when the child fails. A broken protocol or I/O
exchange aborts that session and reaps the child; return the original cause.
6. Honor the read context during blocked I/O, for example by registering
`context.AfterFunc` to cancel the child and stopping that registration after
the exchange. Coordinate callback completion when the stop function reports
it has begun; stopping alone does not wait for that callback. If cancellation
races completion, preserve its cause and abort the session. Read captured
stderr after reaping to avoid writer races.
7. Add idempotent `Close`: an unused reader has nothing to close; a healthy
reader closes stdin, finishes stdout consumption, and waits; an aborted
reader closes pipes, cancels, and waits. Preserve unexpected exit failures;
deliberate abort cleanup retains the original read failure.
**Validation and completion:** Test repeated reads from loose and packed Git
objects, empty/binary bodies, and missing objects. Reuse the partial-clone test
approach to verify absent objects cause errors with no transport or object-store
population. Use a small child fixture for truncated framing and cancellation
mid-response. Assert content and cleanup outcomes; benchmark process reuse
without making a subprocess count a public contract. Run Git and process tests.
## Stage 5: Give Candidate Validation A Scoped Reader
**Owners:** [tree.go](../../internal/validation/tree.go),
[validation.go](../../internal/validation/validation.go), and tree fixtures.
1. Give `CandidateTree` one lazy `BlobReader` and an internal `Close` method.
`ReadCandidateTree` still reads metadata only. Route blob misses through the
reader while temporarily retaining the existing byte-cache behavior.
2. The shared validation executor owns closure on every return after loading
a tree. Complete cleanup before product checks or build callbacks run.
Metadata-only selections leave the content reader unstarted.
3. Preserve an earlier check's `Failure` and cancellation cause during cleanup.
If cleanup reveals an unexpected child failure after successful checks,
return a contextual `Failure` attributed to the last content-reading check.
Track that check in the executor; normal intentional cancellation during
cleanup adds no independent failure classification.
4. Update direct tree users in tests and benchmarks to register cleanup or close
explicitly. Update synthetic tree fixtures to support lazy reader creation.
Keep tree paths, source capture, symlink resolution, and check order intact.
**Validation and completion:** Run validation, candidate, publication, and facade
candidate tests. Cover early check failure and cancellation with cleanup, and
preserve workspace/file-size metadata behavior. Re-run content benchmarks with
reader startup and closure included in the content-operation timing. Update
[validation mechanics](../internal/validation.md) for the landed lifecycle.
**Completion:** Complete. `CandidateTree` owns a lazy batch reader and exposes
idempotent cleanup. The shared executor closes it on every post-load return,
before product checks or build callbacks; metadata-only runs leave it unstarted.
Earlier check failures and cancellation remain primary, while an unexpected
reader exit after successful content checks becomes a contextual failure for
the last content-reading check. Direct tree users close their readers. The
temporary object-ID byte cache remains until Stage 6. Validation, candidate,
and root facade tests passed. Content benchmark timings now include reader
startup and close (Go 1.26.8, Git 2.53.0, Linux x86_64 Intel Core i5-7260U,
three repetitions): unique reads took 59.8–77.9 ms loose and 22.8–43.0 ms
packed; unique-file formatting took 114–188 ms loose and 33.7–55.7 ms packed;
repeated-ID reads took 2.18–6.45 ms, repeated-ID formatting took 4.75–6.62 ms,
and the Markdown workload took 15.0–16.9 ms. Compared with the Stage 1 timings,
these measurements reflect the shared Git process and include its lifecycle;
timing varies on the shared machine.
## Stage 6: Release One-Use Source Bytes And Reuse Formatting Results
**Owners:** `internal/validation/tree.go`, `validation.go`, and destination tests.
1. Remove the tree's general raw-blob cache. A regular-file `ReadFile` returns
its newly read, caller-owned body without a cache copy. Its GoDoc describes
that ownership and the scoped reader; independently repeated raw reads can
request the same object again.
2. Cache committed symlink targets as owned immutable strings keyed by object
ID. Add a private target-reading helper used by destination resolution and
module-symlink handling. Preserve original target bytes and traversal order.
Explicit `ReadFile` of a symlink still returns independently owned bytes.
3. In `checkGoFormat`, cache the formatted/unformatted result for successfully
inspected objects by object ID. Release source and formatted bytes from
retained references after comparing them. Emit a finding at every selected
unformatted path, including duplicate-content paths. Parse/read errors still
stop with the current path and cause; check cancellation for each entry.
4. Adapt tests that construct the old cache directly. Preserve observable
isolation: mutating returned bytes cannot alter later validation or reads.
Reuse destination-cycle and repeated-symlink tests for shared target strings.
**Validation and completion:** Run validation tests and formatting/destination
benchmarks. Confirm many unique Go files no longer remain as retained source
buffers after formatting, and duplicate-content files retain per-path findings.
Record the tradeoff for explicit repeated raw reads. Update validation mechanics
with the new ownership and reuse choices.
**Completion:** Complete. Regular-file reads now return independent bytes and
are not retained by `CandidateTree`; repeated explicit reads therefore transfer
the blob again. The tree caches owned symlink-target strings by object ID, and
explicit symlink reads still return independent byte slices. Go formatting
caches only successful formatted/unformatted outcomes by object ID, checks
cancellation per path, and reports every unformatted path even when contents
repeat. Validation tests and the formatting/Markdown destination benchmarks
passed. On the same Go 1.26.8, Git 2.53.0, Linux x86_64 Intel Core i5-7260U
setup, three repetitions measured unique-file formatting at 75.0–75.9 ms loose
and 37.9–42.5 ms packed, repeated-ID formatting at 1.97–2.17 ms, and the
Markdown workload at 16.3–17.0 ms. Formatting used about 2.89 MB/op for unique
files and 121 KB/op for repeated IDs. The retained-memory observation was
129,496 bytes with the candidate and Markdown cache alive, compared with
1,385,040–1,385,616 bytes in the baseline fixture. This retains Markdown parser
data by path until Stage 7; it is a workload-specific heap observation.
## Stage 7: Reuse Owned Markdown And Module Results
**Owners:** [markdown.go](../../internal/validation/markdown.go),
[markdown_check.go](../../internal/validation/markdown_check.go), and
[local_replacements.go](../../internal/validation/local_replacements.go).
1. Key parsed Markdown reuse by resolved regular blob object ID. Keep a path
lookup when useful for callers, backed by the shared document records.
Relative link resolution and findings use the current source path; a shared
parsed document stores content-relative locations, links, and anchors only.
2. Give retained link destinations and titles their own string storage, using
`strings.Clone` at extraction where needed, so parser/source views cannot
keep a large raw body alive. Cache parsed records, release the AST and source
from retained references, and preserve heading collision rules, image behavior,
and use-site locations.
3. Refactor module-declaration selection to identify the resolved regular entry
before reading its body. After successful parsing, cache a compact list of
filesystem replacements by resolved object ID: target string and source line.
Own retained target strings and release raw bytes and the parsed syntax tree
from retained references.
4. Apply each cached replacement summary to the current declaration path when
constructing findings. Resolve each symlink declaration in its own path
context. On read or parse failure, preserve the current declaration's path
and underlying cause; preserve findings order and versioned-replacement rules.
**Validation and completion:** Run Markdown/module and facade validation tests.
Add behavioral cases for identical Markdown in different directories and two
module declarations resolving to one object, including multiple findings with
correct paths/lines. Reuse parser and boundary/cycle coverage. Re-run
large-document and repeated-content benchmarks and retained-memory observations.
Update the validation document's cache description to match implemented ownership.
**Completion:** Complete. Parsed Markdown documents are shared by regular blob
object ID with path lookups; destinations and titles own their string storage,
and link locations remain content-relative while resolution uses each current
source path. Module declarations resolve to a regular committed entry before
reading; successfully parsed filesystem replacements are cached as owned
target/line summaries by body object ID and applied to every declaration path.
Validation, candidate, and facade tests passed, including shared Markdown
resolved from distinct directories and two module symlinks receiving the
correct findings at both paths and lines. Three large-document runs measured
10.85–11.22 ms and 2.29 MB/op for two identical 1 MiB documents and 1,000
repeated links. Three retained-memory observations measured 110,568–120,280
bytes with the candidate and parsed Markdown cache alive. These fixture results
include the retained per-document links, anchors, and candidate metadata; they
are workload-specific observations on the recorded shared machine.
## Stage 8: Consolidate Delivery Preflight Hashing
**Owners:** [delivery declarations](../../internal/delivery/assets.go),
[delivery orchestration](../../internal/delivery/delivery.go), and their tests.
1. Make `declaredBuildAssets` perform structural declaration checks, stat files,
reuse the recorded artifact SHA-256, and construct expected assets and the
manifest. Remove its file-opening/hash pass. Keep target, filename, regular
file, declared digest format, and manifest validation at the appropriate
existing owner.
2. Keep one `gitea.VerifyLocalAssets` call in `Deliver` after declaration and
build-identity checks and before hosted requests. This verifies actual bytes
and sizes against recorded identities for both managed and prebuilt assets.
3. Keep builder hashing, CLI receipt verification, and missing-upload preflight
in their current phases. Managed delivery reuses its fresh build metadata for
declarations; prebuilt delivery reuses its caller's recorded digests only as
expected identities until current bytes have been verified.
4. Move the declaration helper's tampering assertion to delivery preflight's
behavioral owner. Verify same-size changed contents fail before any hosted
request. Reuse source-only, retained-prebuilt, managed cleanup, conflict,
partial-upload, and retry coverage.
**Validation and completion:** Run delivery, Gitea, facade delivery, and CLI
receipt/delivery tests. Re-run artifact and complete delivery benchmarks. A fresh
missing binary has three hash passes including a managed build, or two within
prebuilt delivery; these are explanatory measurements of the chosen flow.
Streaming upload, exact remote comparison, and later upload verification remain
intact. Update the inventory's delivery mechanics where needed.
## Stage 9: Transfer Completed Go Output Without Copies
**Owners:** [gotool.go](../../internal/gotool/gotool.go), validation/build callers,
and Go tool fixture tests.
1. Give `Tool` a status-returning `Run(...args) error` and an output-returning
`Output(...args) ([]byte, error)`, with one private execution path. Keep their
existing directory, mode, target, writer, and argument inputs.
2. On completion, a status-only success releases its capture. An output-consuming
call receives the completed buffer's backing bytes with ownership transferred
from the private writer. Replace `combinedWriter.Bytes` copying with a small
synchronized ownership-transfer operation used after child output has finished.
3. Preserve complete failure diagnostics, original causes, combined-output
synchronization, and forwarding errors. If `Output` fails, return its captured
bytes and diagnostic error as before. Status failures format the same useful
diagnostics without creating an unnecessary intermediate snapshot.
4. Convert validation test/vet/compile and artifact-build calls to status-only
`Run`. Use `Output` for compiler identification. Adapt preparation failure
formatting to the same completed-buffer ownership discipline, preserving its
network-capable environment. Leave configuration discovery's separate stdout
JSON and stderr handling intact.
5. Keep full capture in memory. Measure remaining large-log retention and record
the disk-backed alternative as follow-up evidence, as scoped by the roadmap.
**Validation and completion:** Extend existing process/Go fixtures for successful
capture, large failure output, cancellation, and a failing output writer. Verify
forwarded bytes and meaningful error content/cause. Run Go tool, validation,
build, preparation, and affected CLI tests. Re-run output benchmarks and update
[Go execution mechanics](../internal/gotool.md).
## Stage 10: Cache The Runtime Metadata Fallback
**Owners:** [runtimeinfo/version.go](../../internal/runtimeinfo/version.go) and
runtime/buildinfo tests.
1. Preserve `injectedVersion` and its linker identity. `Version` checks and
returns injection first, then calls a fallback accessor created with
`sync.OnceValue`.
2. Keep the existing pure `version(injected, readBuildInfo)` selection helper.
Add a small private fallback factory accepting the metadata reader and
returning the cached accessor. Production supplies `debug.ReadBuildInfo`;
tests create independent cache instances with controlled metadata.
3. Reuse selection tests. Add repeated/concurrent access coverage with a
counting reader to verify reuse and stable results. Test unavailable and
development metadata fallback through isolated instances. Keep shared
production state immutable during tests.
**Validation and completion:** Run runtime/buildinfo tests, the existing consumer
build-and-run injection test, and focused race validation of the fallback cache.
Run facade/dependency checks. Re-run version benchmarks after initializing the
cache and report initialization separately from repeated access. The runtime
facade and its dependency boundary retain their existing public shape.
## Stage 11: Verify The Complete Implementation And Record Results
**Owners:** Affected tests and canonical internal documentation; roadmap status.
1. Run the complete offline test suite and compiled facade/dependency checks
using the [developer guide](../development.md). Resolve remaining failures
within this performance scope. Reuse targeted race results where code is
unchanged; broaden them only for a concrete remaining concurrency risk.
2. Run the Stage 1 and Stage 2 benchmark set again with comparable toolchain,
fixtures, cache state, repetitions, and measurement boundaries. Compare
component and complete `Check`/`Deliver` results. Record memory and explicit
reread tradeoffs as well as time and allocation improvements.
3. Record final results alongside the baseline in this document. Explain the
remaining Go compilation/test cost, remote-comparison work, and diagnostic
retention where measurements show they matter. Separate measured effects
from expectations and include machine/workload limitations.
4. Check canonical internal docs describe landed reader lifetime, source/result
ownership, verification flow, output ownership, and runtime reuse. Preserve
existing public contracts; update a public document only if its explanation
became inaccurate. Check local links and prose whitespace.
5. Mark each stage's actual completion, update the feature roadmap status, and
commit the completed work under repository policy. End with a concise report
of changes, tests, performance results, and remaining follow-up opportunities.
Retain these planning documents with their results for the final review.
**Completion evidence:** All five target improvements are implemented, relevant
existing behavior passes, the public facade checks pass, comparable measurements
are recorded, and the worktree is clean. Completion depends on observed behavior
and reviewed diffs; benchmarks establish improvements within their measured
scope without adding numerical runtime guarantees.
## Open Questions
None required for this implementation. The plan selects scoped batch reading,
result-based reuse, preserved verification boundaries, in-memory diagnostics,
and cached runtime fallback. Disk-backed diagnostics remain a follow-up option
whose benefit and lifecycle costs can be assessed from the recorded output
measurements.