Files
ReleaseKit/docs/roadmap/audit.md

416 lines
25 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# ReleaseKit pre-release audit
Date: 2026-10-06. Reviewed source: `536a57916f6565ae8194c4563577dfcebd6cfb69`.
Status: historical review evidence and selected target direction for the
[implementation plan](implementation.md).
Audience: maintainers deciding the shared release standard before `v0.1.0`.
## Assessment
ReleaseKit's practical goals are to minimize application-owned release code,
duplication across the suite, and the time developers spend on release processes.
Shared conventions serve these goals by giving routine decisions one owner.
Applications supply product facts and product-specific checks; ReleaseKit owns
the procedure, sensible defaults, and recovery guidance.
Evaluate recommendations by the application code they remove, the shared logic
they consolidate, and the recurring decisions they eliminate. Standardization,
configuration options, and diagnostic detail are useful when they reduce that
work. Keep routine release decisions inside ReleaseKit.
The core direction is sound. Retain the publication/delivery separation, narrow
public facades, standard artifact naming, version injection, and content-aware
retries. Fix the reproduced validation and recovery defects in F1–F2 before
`v0.1.0`. Treat F3 as a focused improvement to internal build defaults.
Build-environment reproducibility, hardware-specific configurability, and
extensive build inspection are outside the product's optimization goals. Use the
available compatible Go toolchain and shared baseline settings. Evaluate new
configuration and deployment machinery by the application code and developer
work they save. Different toolchains may produce different bytes; existing
published content remains protected by digest comparison.
The implementation covers most mechanics in the [product roadmap](releasekit.md).
Representative adoption should demonstrate the
[architectural success criteria](../policy/architecture.md#success-criteria)
through deleted release code, fewer duplicated checks, and fewer routine decisions.
F4–F6 identify opportunities to centralize validation, delivery, and CI setup.
The existing source transaction supports these goals. Annotated tags preserve
the checked-in note and provide an exact object for recovery. Requiring the
candidate on authoritative `main` keeps branch publication in preparation.
Separate delivery lets source publication finish independently of hosting.
Consumer migrations should adopt this common process and retain product identity,
target selection, compatibility decisions, and product-specific checks.
## Scope and evidence
Reviewed the development guide; agent, architecture, testing, documentation, and
release policies; facade ADR; roadmap; public declarations; CLI/configuration
and consumer guides; publication, build, delivery, Git and Gitea implementations;
and their relevant tests and workflow examples. The
[release policy](../policy/release.md) governs ReleaseKit's own releases.
The facade and integration contracts define consumer behavior.
Also inspected the release scripts, module declarations, and available delivery
YAML in the six sibling projects named by the roadmap. This read-only sample
informed the assessment of adoption needs.
Existing extra checks should be retained according to their product-specific value.
For example, PromptKit has product-specific executable examples, whereas
Narratio currently requires Go 1.26 and several peers require Go 1.25.5.
Validation on Linux/amd64 with Go 1.26.8:
- `GOTOOLCHAIN=local GOPROXY=off GOSUMDB=off go test -mod=readonly -count=1 ./...`
passed on the reviewed checkout.
- `go run ./internal/apicheck` passed.
- Isolated probes against an archive of the reviewed commit reproduced F1–F3
below. They used temporary consumer repositories and local bare Git remotes.
The investigative probes ran in a disposable checkout. Their setup and outcomes
are recorded with each finding for future regressions. Validation covered local
fixtures on Go 1.26.8. Live Gitea/Woodpecker integration, published-module
installation, and minimum-Go-1.25 compatibility remain unverified. These results
belong to the original source review; this revision updates its recommendations.
## Goal coverage
| Goal | Assessment and evidence |
| --- | --- |
| One standard source-publication transaction | Substantially implemented in [Publish](../../internal/publish/publish.go): captured candidate, authoritative `main`, annotated tag, exact object/ref push, and bounded recovery. F2 is a correctness exception. |
| One validation of the intended candidate | Callback/default validation plus mutable-state rechecks are implemented in [candidate](../../internal/candidate/check.go). F1 can silently narrow the default suite; F4 concerns the selected customization model. |
| Consumer-selected binaries or source-only releases | [Build](../../internal/build/build.go) supports target pairs and empty sets. Source-only delivery creates no binary manifest. This is the right distinction. |
| Shared runtime version, build flags, naming, and smoke check | Implemented through [buildinfo](../../buildinfo/version.go), the builder, and external consumer build tests. F3 identifies an opportunity for predictable internal defaults. |
| Immutable hosted contents and recoverable partial delivery | [Delivery](../../internal/delivery/delivery.go) and [Gitea](../../internal/gitea/) compare notes and asset bytes, reject conflicts, and add missing assets. F7 identifies an outcome-reporting inconsistency. |
| Public contract ownership and small runtime dependency graph | Root and runtime facades follow [ADR 0001](../adr/0001-public-facade.md); compiled boundary checks pass. Keep these boundaries. |
| Thin application integration and less repeated infrastructure | Representative consumer adoption remains to be evaluated. F5–F6 identify remaining orchestration and setup duplication. |
| Ready deployment path | Prepared templates and local fixtures exist. Immutable deployment pins, credentials, activated workflows, and live verification remain explicitly open in the roadmap. |
## Findings
### F1 — High: ambient Go flags can turn candidate validation into a false pass
**Evidence.** `runDefaultValidation` in
[internal/candidate/check.go](../../internal/candidate/check.go) controls module
network access, workspace discovery, and toolchain selection, but inherits
`GOFLAGS`. The default command supplies `-mod=readonly -count=1`; neither flag
cancels a test-name filter. Configured checks also inherit the environment through
[RunChecks](../../cmd/releasekit/internal/cli/config.go). ReleaseKit's own
[profile](../../releasekit.json) explicitly supplies the test command without
controlling `GOFLAGS`.
**Reproduction.** Create a clean `main` fixture with a matching committed note
and one `TestFails` that calls `t.Fatal`. `Check` with no callback fails as
expected. Set `GOFLAGS=-run=^$` and invoke the same operation against the same
commit: it returns nil and reports `[no tests to run]`.
The [Go command documentation](https://pkg.go.dev/cmd/go#hdr-Environment_variables)
confirms that `GOFLAGS` supplies defaults to commands that recognize its flags.
**Consequence.** A normal developer setting used for focused testing can cause a
release to pass after skipping the default suite. The reproduction uses an
unchanged committed candidate and an ordinary shell setting.
**Recommendation.** Make the standard validation command own its effective Go
flags and host target, including values supplied through Go's persistent
configuration. Preserve caches and legitimate dependency preparation settings.
Share internal Go-environment handling where the semantics match. Preserve
the settings appropriate to native tests and static release builds separately.
Apply the same baseline to ReleaseKit's own profile. Retain a regression showing
that a failing test remains effective under an ambient filter. F4 and Design
Choice 1 describe how product-specific checks compose with this baseline.
### F2 — High: automatic push recovery can query a different endpoint and report false absence
**Evidence.** [reconcilePush](../../internal/publish/publish.go) calls
`Endpoint.QueryTag` using the captured URL and command-scoped remote.
[QueryTag](../../internal/git/tag.go) runs the query directly.
[Resolve/Recheck](../../internal/git/remote.go) call `verifyScopedURLs` before
publication. Git still applies current URL rewrite configuration to the captured
literal URL. The manual recovery instructions in
[the runbook](../release.md#resolve-failures) already require a URL-equality check.
The automatic path lacks the equivalent check.
**Reproduction.** In a temporary publication fixture, wrap Git so the actual tag
push succeeds, then add a local `url.<empty-bare-repository>.insteadOf` rule for
the captured endpoint and return exit 77 to simulate a lost acknowledgment.
`Publish` returns `state=not_published`, `phase=reconciliation`, and says the
authoritative tag is absent. Direct inspection of the original bare repository
shows the exact tag object returned in `PublicationResult` is present.
**Consequence.** The result gives incorrect recovery advice after an ambiguous
push. It converts a query to the wrong endpoint into proof that the intended
endpoint lacks the published tag. This contradicts the existing recovery
contract for the authoritative endpoint.
**Recommendation.** Within the existing bounded recovery context, verify that the
captured endpoint still resolves to itself before the one authoritative tag
query. If verification fails, preserve the push error and local tag and return
`unknown`. Keep recovery bound to the captured endpoint. Cover the
successful-publication/false-absence reproduction and ordinary recovery against
an unchanged endpoint. Preserve successful push as the completion point.
### F3 — Internal improvement: ambient CPU settings change binary compatibility
**Evidence.** `goEnvironment` in [internal/build/build.go](../../internal/build/build.go)
standardizes several Go settings and inherits architecture feature variables,
`GOEXPERIMENT`, and the persistent Go environment file.
**Reproduction.** Build the same committed consumer and `v1.2.3` for
`linux/amd64` twice, once with `GOAMD64=v1` and once with `GOAMD64=v2`.
Both builds succeed with `SourceVerified=true` and the filename
`fixture-v1.2.3-linux-amd64`. Their SHA-256 digests differ, and `go version -m`
records the respective feature levels. Go documents these as different target
microarchitecture requirements in its
[environment reference](https://pkg.go.dev/cmd/go#hdr-Environment_variables).
**Consequence.** A developer's ambient CPU setting can narrow a binary's hardware
compatibility. Different build inputs can also produce a digest conflict during
a delivery retry. Existing digest comparison correctly preserves published bytes.
The practical benefit of addressing this finding is predictable hardware
compatibility with fewer settings for application developers to manage.
**Recommendation.** Use ordinary baseline CPU settings and shared build flags
internally, with the available compatible Go toolchain. Control incidental
settings that affect expected compatibility or validation. Keep the application
profile focused on its package, binary name, and targets. Exact toolchain
pinning and detailed build metadata remain optional operational choices.
When a retry requires the original bytes, preserve and reuse those artifacts.
Design Choice 2 compares local toolchain use with a centrally maintained build
environment for consumers that benefit from shared setup.
### F4 — Design priority: adding a custom check replaces the entire standard suite
**Evidence.** [CLI routing](../../cmd/releasekit/main.go) installs a validation
callback whenever `checks` is nonempty. [Validate](../../internal/candidate/check.go)
then skips its default test command. This agrees with the
[configuration contract](../config.md), and the Go API deliberately permits a
replacement `Validate` callback.
**Consequence.** Adding one product-specific example check requires the consumer
to remember and reproduce the standard test command, environment, and ordering.
The profile becomes a small release program that repeats shared policy. Migrating
to built-in tests requires removing their invocation from existing custom scripts
to preserve one suite run.
**Recommendation.** Run the shared offline suite once, followed by optional
product-specific checks. Move redundant suite invocations out of consumer scripts.
Retain checks for actual product behavior, such as PromptKit's examples. Add
alternative test-invocation settings when a concrete consumer requirement
justifies them. This choice changes the current customization contract; Design
Choice 1 records the replacement-callback alternative.
### F5 — Design priority: the CLI owns reusable tagged-build delivery orchestration
**Evidence.** [runDelivery](../../cmd/releasekit/main.go) chooses a temporary
output directory, invokes the public builder, converts receipts, and then calls
`Deliver`. The [public delivery request](../../delivery.go) accepts an already
produced `BuildResult`. A Go consumer wanting the normal build-and-deliver
operation must reproduce orchestration already implemented by the CLI. The
default CLI path also builds and runs the native smoke executable
before `Deliver` verifies that HEAD matches the selected local tag.
**Consequence.** Temporary storage, build sequencing, tagged-source preflight,
and cleanup have more than one potential owner. An invalid tagged checkout can
consume a complete cross-build before it is rejected. The CLI owns domain coordination
alongside its parsing, presentation, and exit-status responsibilities.
**Recommendation.** Put the ordinary verify-tagged-source, build-selected-targets,
and deliver sequence behind the orchestration facade. Verify the local tagged
source before expensive compilation, then retain the necessary checks before
remote effects. Let the CLI adapt JSON, credentials, and output to that operation.
Use the same build declaration for candidate validation and delivery. Support
caller-provided builds for concrete needs such as retrying with retained
artifacts, as discussed in Design Choice 3. Keep separate building useful for
local development. The ordinary consumer workflow should complete delivery in
one invocation; advanced artifact handoffs can use private coordination data
when needed.
### F6 — Adoption priority: shared deployment still depends on copied mechanics
**Evidence.** The [source-only](../../examples/woodpecker/tag-delivery-source-only.yml)
and [binary](../../examples/woodpecker/tag-delivery-binary.yml) templates repeat
fetch/checkout guards, local exclusions, cache setup, and tool installation.
The binary template adds an explicit build/receipt/deliver sequence even though
the CLI already supports combined delivery. ReleaseKit's
[own workflow](../../examples/woodpecker/releasekit-source-delivery.yml) has a
separate first-release bootstrap. Consumer pins remain placeholders as documented.
**Consequence.** Copied CI mechanics require maintenance in each consumer and
can drift. A shared installation and update convention would simplify local
publication and CI delivery.
**Recommendation.** Provide one maintained integration entry point and one
default delivery sequence for source-only and binary profiles. Consumers select
the ReleaseKit version, supply product settings, and provision credentials.
Centralize installation and update instructions. Use versioned source
installation for the shared CI command. Evaluate a maintained image by the
additional setup and maintenance effort it saves.
Keep dependency preparation separate from offline validation/builds. A split
build/delivery flow remains useful for an actual credential boundary or artifact
handoff. See Design Choice 4 for packaging alternatives.
`TestWoodpeckerDeliveryCommandSequences` in
[the CLI workflow tests](../../cmd/releasekit/woodpecker_test.go) executes
equivalent commands;
[the ReleaseKit workflow test](../../cmd/releasekit/releasekit_delivery_workflow_test.go)
similarly reconstructs the sequence. Execution of the maintained YAML and
installation step remains unverified. A YAML drift can leave these tests green.
Validate the selected integration during adoption and keep a small check tied to
the maintained artifact. Use that evidence to maintain the shared integration.
### F7 — Medium: delivery asset classifications overlap inconsistently
**Evidence.** In [UploadMissing](../../internal/gitea/upload.go), a failed
reconciliation list adds an asset to `Uncertain` while leaving it in `Pending`.
The `AssetMissing` reconciliation branch does the same for an unresolved write;
the default/unverifiable branch removes it from `Pending`. A reconciled conflict
can also remain pending. The existing
`TestUploadMissingReportsUnknownWhenReconciliationReadFails` in
[upload_test.go](../../internal/gitea/upload_test.go) explicitly expects the
same asset in both pending and uncertain collections.
**Consequence.** Overlapping categories require special handling by consumers.
The CLI can present one file as both pending and uncertain, while a
slightly different failure reports it only as uncertain. The overall unknown
state remains conservative. Consistent categories would simplify recovery
reporting and automation.
**Recommendation.** Prefer mutually exclusive current-outcome categories:
confirmed matching, uploaded, known pending, conflicting, and uncertain. Move an
asset out of pending when its outcome becomes uncertain or conflicting, and
clarify the facade GoDoc. Design Choice 5 records the alternative superset
interpretation.
### F8 — Maintainability: artifact rules have several implementation owners
**Evidence.** Filename construction occurs in `artifactFilename` in
[the builder](../../internal/build/build.go), `receiptArtifactFilename` in
[CLI receipt handling](../../cmd/releasekit/internal/cli/receipt.go), and
`declaredBuildAssets` in [delivery](../../internal/delivery/assets.go).
Target normalization and filename-component validation are also repeated across
these boundaries. CLI receipt handling carries substantial domain validation
alongside JSON serialization. Build invocation uses its own `exec.CommandContext`
path while default validation uses [process.Run](../../internal/process/process.go),
with independently constructed Go environments.
**Consequence.** A change to target identity, naming, or build settings can leave
the builder, receipt reader, and delivery validator disagreeing. F1 and F3 show
why divergent Go-environment ownership matters beyond cosmetic duplication.
**Recommendation.** Give internal build/artifact rules one owner and reuse them
from receipt and delivery adapters. Share Go invocation/environment construction
where the semantics match. Keep the explicit facade conversions required
by the ADR. Focus refactoring on the small internal helpers that own these rules.
## Target state
Retain the existing source transaction: committed stable-version note, clean
`main`, candidate already on authoritative `main`, one validation, selected-target
build verification, exact annotated-tag publication, and bounded recovery.
Source publication finishes independently of hosted delivery. Keep additive,
content-checked delivery and lightweight runtime version access.
Make ordinary integration a versioned tool invocation plus a small product
profile and credential provisioning. The profile supplies product identity,
selected binaries and targets, delivery destination, and product-specific checks.
ReleaseKit owns the standard suite, routine build settings, delivery sequence,
and recovery reporting. Product developers choose versions and write
compatibility notes. Evaluate each proposed setting by the work it saves them.
The [implementation plan](implementation.md) owns the stages, API changes,
regression coverage, and completion criteria needed to reach this target.
Representative adoption should demonstrate fewer scripts, duplicated tests,
setup steps, and recurring release decisions.
Feature expansion should follow demonstrated consumer needs and measurable
reductions in application maintenance or developer effort.
## Design Choices
The implementation plan selects the recommended directions below for the
upcoming sprint. Alternatives record the tradeoffs for future consumer needs.
These choices describe the target behavior; current contracts change as their
implementation lands.
### 1. Should product checks supplement or replace standard validation?
**Selected direction:** Run the shared offline Go suite once, followed by optional
product-specific checks. ReleaseKit owns the command and its routine environment
settings. Migrate repeated suite invocations out of application scripts. Add
alternative invocation settings when an actual consumer requirement establishes
their value. An application adding an example check should declare that check
and receive the standard suite automatically. This gives developers one ordinary
validation path to understand.
**Viable alternative:** Keep a replacement callback or complete validation
command for products with unusual test layouts. This is a simple composition
boundary and can accommodate existing product test infrastructure. It assigns
more release responsibility to the application, so its value depends on the
specific consumer need.
### 2. How should ReleaseKit choose build settings?
**Selected direction:** Use the available compatible Go toolchain, ordinary
baseline CPU settings, and shared build flags. Handle incidental settings internally
so developers can rely on the selected target's expected compatibility. Keep
application configuration small. Toolchain-dependent byte differences are
acceptable; digest checks preserve existing published content, and retained
artifacts support retries requiring the same bytes. ReleaseKit can own baseline
settings internally while applications declare their packages and targets.
**Viable alternative:** A shared CI image can supply a centrally selected Go
version and build environment when this makes installation and support easier.
Maintainers own image updates; applications use the shared integration. This is
useful where a common environment reduces operational work. Its value should be
measured against image maintenance and the suite's differing Go requirements.
Explicit hardware variants can be added for a demonstrated product need.
### 3. Should the library own build-and-deliver, or accept only completed builds?
**Selected direction:** Provide one facade operation for tagged-source verification,
selected-target builds, temporary storage, checksums, uploads, and cleanup.
Use it from the CLI so both interfaces share the procedure. Retain separate
building for local development and support prebuilt delivery where a concrete
handoff or retry requirement justifies it. Routine delivery should require one
invocation, with ReleaseKit managing the artifact lifecycle. This minimizes
consumer orchestration code.
**Viable alternative:** Keep separate build and delivery primitives with a
maintained composition helper. This suits consumers that require credential
isolation between stages or need to retain artifacts for later delivery. Those
consumers accept responsibility for the artifact lifecycle in exchange for the
operational separation they need.
### 4. How should the shared executable and CI integration be distributed?
**Selected direction:** Choose the simplest maintained entry point for installation,
updates, and delivery. Use versioned source installation and a shared CI delivery
command. The command prepares the tagged checkout, reads its product profile,
prepares dependencies for selected binaries, and invokes ordinary facade delivery.
Centralize reusable setup and provide one clear upgrade procedure. Applications
select a ReleaseKit version and supply product settings and credentials; shared
integration owns routine setup.
Assess packaging by the consumer code and recurring developer work it removes.
**Viable alternative:** Publish a maintained CI image containing ReleaseKit and
a compatible Go/Git environment when it substantially reduces setup. The image
provides a ready-to-run integration and centralizes environment maintenance.
It also creates an image publication and update responsibility, which should be
reflected in the release procedure. Either packaging choice should give
applications a small configuration and a routine invocation.
### 5. How should delivery results support simple recovery?
**Selected direction:** Use exclusive outcome categories: matching, uploaded, known
pending, conflicting, and uncertain. Present ordinary callers with completion or
a clear failure and recovery instruction. Keep asset details available for
diagnostics. Derive recovery guidance centrally so each application can present
the result directly. Consistent categories simplify implementation and
interpretation.
**Viable alternative:** Pending includes every unfinished declaration, with
conflicting/uncertain as diagnostic subsets. This can simplify a progress UI
when defined explicitly and applied in every branch. Consumers must then
subtract blocked subsets before deriving retry work. Apply the chosen
meaning consistently across all outcomes.