Plan the shared validation foundation

docs/roadmap/implementation.md

@@ -0,0 +1,393 @@
+# Validation model implementation plan
+
+Status: Proposed implementation sequence; the API rollout question below needs
+resolution before the stages that expose optional selections.
+
+This plan implements the enabling work in [model.md](model.md), guided by
+[ADR 0003](../adr/0003-declarative-validation.md). It is written for a Luna
+coding agent executing one stage per prompt, in numerical order. The
+[v0.2.0 roadmap](v0.2.0.md) owns the scope and defaults of the individual
+capabilities. Their checker implementations, module-selected CI tooling, and
+the publication endpoint expectation are subsequent feature work.
+
+The foundation provides shared execution, diagnostics, committed-file access,
+and a concrete design for equivalent Go and profile declarations. Completion
+of this plan establishes that foundation; completion of the individual
+validation features makes their selections operational.
+
+## Instructions for each stage
+
+- Start with [development.md](../development.md), the
+  [agent policy](../policy/agents.md), and the policies applicable to the stage.
+  Inspect the worktree and reuse relevant validation already available.
+- Use the codebase graph to locate current symbols. The paths below identify
+  the inspected baseline and may change during implementation.
+- Implement the stage's focused outcome with its behavioral coverage and
+  applicable documentation. Commit completed units according to the agent
+  policy and report the resulting commit and remaining work.
+- Continue through the sequence using local fixtures and automated checks.
+  Consumer migrations, hosted actions, intermediate releases, and human review
+  are outside these stages. Architecture review and next-step decisions follow
+  the completed implementation.
+- Keep internal collaboration proportionate: ordinary structs, functions, and
+  explicit conversions. Preserve the root and `buildinfo` facade boundaries.
+  Current API contracts and selected user decisions determine compatibility.
+
+## Design to carry through the stages
+
+### Validation declarations
+
+The intended complete facade shape is:
+
+```go
+type ValidationSpec struct {
+    Go                 GoValidationSpec
+    GoFormat           bool
+    Markdown           *MarkdownValidationSpec
+    ModuleDistribution ModuleDistributionValidationSpec
+    FileSize           *FileSizeValidationSpec
+}
+
+type GoValidationSpec struct {
+    Race           bool
+    FullVet        bool
+    CompileTargets []Target
+}
+
+type MarkdownValidationSpec struct {
+    EnforceCheckoutBoundary bool
+}
+
+type ModuleDistributionValidationSpec struct {
+    WorkspaceFiles    bool
+    LocalReplacements bool
+}
+
+type FileSizeValidationSpec struct {
+    LimitBytes *int64
+    Exclusions []string
+}
+```
+
+`ReleaseRequest.Validation` holds a `ValidationSpec` value. `Target` is the
+existing facade-owned operating-system/architecture pair. The corresponding
+profile keys are `validation.go.race`, `validation.go.full_vet`,
+`validation.go.compile_targets`, `validation.go_format`,
+`validation.markdown.enforce_checkout_boundary`,
+`validation.module_distribution.workspace_files`,
+`validation.module_distribution.local_replacements`,
+`validation.file_size.limit_bytes`, and `validation.file_size.exclusions`.
+Profile compilation targets use the existing `GOOS/GOARCH` string convention.
+
+Value groups preserve zero-value defaults. A non-nil Markdown or file-size
+group enables its check; an empty JSON object has the same meaning. Each
+distribution boolean independently selects inspection for the corresponding
+distribution rule. Omitted optional groups select current default validation.
+A nil byte limit selects 1,000,000 bytes when file-size checking is enabled;
+an explicit zero selects zero bytes. Negative byte limits are invalid because
+byte counts are nonnegative. Exclusions identify literal repository-relative
+file or directory paths; a directory exclusion includes its committed
+descendants. Preserve Git path spelling and use slash-separated paths.
+
+These declarations describe the complete intended model. Stage 5 implements
+the rollout selected under Open Questions. Public settings acquire operational
+behavior with their capability implementations, and current documentation
+describes the implemented boundary precisely.
+
+### Execution and ownership
+
+- `internal/candidate` retains capture and rechecks. Its existing `Validate`
+  entry point delegates shared validation to `internal/validation` and then
+  invokes `AdditionalChecks` once with the captured candidate.
+- `internal/validation` owns internal settings, the shared executor, command
+  coordination, and diagnostic records. It receives the captured repository
+  directory and commit explicitly. It depends on internal Go and Git helpers;
+  internal production packages retain internal contracts.
+- Root facade helpers perform request and diagnostic conversions. `Check` and
+  `Publish` share those helpers. The CLI maps its own JSON settings to the root
+  facade in `cmd/releasekit`, preserving its helper-package dependency boundary.
+- Shared validation runs one base Go suite, selected full vet, selected package
+  compilation, and selected file checks, followed by product-specific checks.
+  Artifact builds retain their existing after-validation position. The
+  foundation executes the currently implemented portion of this procedure.
+- Native commands retain the current offline environment. Future race
+  selection changes that one suite; full vet adds `-vet=off` to testing and a
+  separate vet invocation. Package compilation uses static target mode with
+  `CGO_ENABLED=0`, independently of artifact declarations.
+- Preserve fail-stop behavior, context propagation, candidate rechecks, local
+  `Check` behavior, and publication phase/state meanings. Cross-check ordering
+  is internal procedure except for the relationships selected in the roadmap.
+
+### Diagnostic model
+
+Add facade-owned `ValidationCheck` (a string type), `ValidationFinding`, and
+`ValidationError`. The error carries `Check`, `Findings`, and `Cause`, implements
+`Error() string` and `Unwrap() error`, and is returned as a pointer. Findings
+carry `Path`, `Line`, `Column`, and `Message`, with `Target` and `Directive` for
+applicable file checks. A nullable `FileSizeMeasurement` containing
+`Bytes` and `LimitBytes` supplies structured size details. Each named type
+belongs to the root facade, with corresponding internal records and conversions.
+
+Use check identities `go-test`, `go-vet`, `package-compile`, `go-format`,
+`markdown-links`, `workspace-files`, `local-replacements`, and `file-size` as
+their checks land. Findings use repository-relative paths and one-based source
+locations where available; zero locations mean unavailable. Human-readable
+messages supply detail without a promise of exact wording.
+
+The shared error represents a failed validation check and preserves its
+underlying cause. Candidate guards, artifact failures, and publication outcomes
+retain their existing classification. Existing additional-check errors retain
+their cause and current wrapping; custom callbacks can return facade errors
+without losing their identity. Successful out-of-scope Markdown observations
+will use the existing output writer when that feature lands, leaving successful
+validation successful.
+
+### Committed-file access
+
+Provide an internal candidate-tree reader using the captured commit. Enumerate
+the full tree with NUL-delimited Git output and record path, object ID, entry
+kind/mode, and blob size where available. Read blob bytes by object ID, preserving
+binary bytes and filenames containing spaces, tabs, or newlines. Cache reusable
+metadata within the validation operation; read contents when needed.
+
+Regular tracked files include executable files, vendor contents, and nested
+modules. The inventory applies no feature-specific exclusion. Symlinks expose
+their committed link text as blob contents. Gitlinks expose the recorded
+submodule commit and carry no parent-tree file size; the reader enumerates the
+parent candidate's committed tree. Traversing a submodule checkout would inspect
+different source and require a separately defined capability.
+
+Formatting and Markdown source parsing operate on regular source blobs. File
+size can measure regular and symlink blobs. Supply an internal path-resolution
+helper for future Markdown destinations that resolves symlink text within the
+candidate tree, classifies destinations outside the checkout, and reports
+missing entries or cycles. A submodule's contents are outside the parent
+candidate's inspected tree. The Markdown feature owns whether a resulting
+observation is a finding or an out-of-scope report under its selected rule.
+The helpers read neither symlink targets nor submodule contents from the live
+filesystem. Their tests use synthetic trees and local Git repositories.
+
+## Stage 1: Extract the shared validation executor
+
+**Outcome:** One internal owner runs current default validation, with current
+candidate and publication behavior preserved.
+
+1. Inspect `internal/candidate/check.go`, `internal/publish/publish.go`,
+   `internal/gotool`, and their existing tests. Record the capture, validation,
+   additional-check, build, and recheck boundaries in implementation notes.
+2. Move default Go validation into `internal/validation`. Accept context,
+   captured repository/commit, output, and internal settings explicitly. Start
+   with ordinary zero-value settings and the existing default invocation.
+3. Keep `candidate.Validate` as the shared bridge used by local checks and
+   publication. Retain additional checks and after-validation builds in their
+   existing responsibilities. Reuse `gotool.New` and its environment handling.
+4. Consolidate root callback/request conversion where `check.go` and
+   `publish.go` currently duplicate it, using small private helpers.
+
+**Verification:** Run affected candidate, publication, and facade tests. Add
+coverage only for gaps in default suite execution, callback ordering, failure
+stopping before builds/tag creation, and cancellation. Existing publication
+recovery tests retain ownership of remote-state behavior. Update the internal
+inventory to reflect the implemented executor.
+
+## Stage 2: Introduce internal validation diagnostics
+
+**Outcome:** The executor returns a small structured failure with an underlying
+cause, independently of public error conversion.
+
+1. Add internal check identity, finding, measurement, and failure records to
+   `internal/validation`, corresponding to the diagnostic design above.
+2. Attribute Go setup/test failures to the base test check. Preserve process
+   errors and canceled contexts through `Unwrap` and ordinary wrapping.
+3. Keep candidate commit context in the candidate wrapper. Keep additional
+   callback errors and artifact failures in their current flow.
+4. Define one internal formatting routine for the failure's useful summary;
+   avoid duplicating diagnostic prose at each call site.
+
+**Verification:** Protect `errors.Is` for cancellation and sentinel causes,
+`errors.As` for subprocess errors, and distinguish shared failures from callback
+and artifact failures. Test semantic fields and cause preservation rather than
+complete error text.
+
+## Stage 3: Add committed candidate-file enumeration
+
+**Outcome:** File-oriented validators can share an offline reader bound to the
+captured commit.
+
+1. Add Git tree/blob operations to the existing internal Git boundary and a
+   small candidate-tree reader used by `internal/validation`. Select a separate
+   internal package only if it clarifies that responsibility.
+2. Implement enumeration and lazy blob reads according to the committed-file
+   design. Bind queries to the captured commit and preserve Git/process causes.
+3. Represent regular files, symlinks, and submodules explicitly. Keep feature
+   filtering in the respective future checker.
+4. Make the reader available to the executor for selected file checks. Default
+   Go-only validation does not need to load all repository blobs.
+
+**Verification:** Use real temporary Git repositories to show that bytes and
+sizes come from the selected commit despite changed worktree files, that unusual
+filenames survive enumeration, and that vendor/nested-module files are present.
+Include symlink and gitlink entries without requiring a network or neighboring
+checkout. Confirm inspection leaves repository contents unchanged.
+
+## Stage 4: Add candidate destination-resolution primitives
+
+**Outcome:** Future Markdown validation can resolve committed local destinations
+and classify checkout boundaries without filesystem traversal.
+
+1. Implement a small internal resolver for repository-relative destinations
+   against the candidate inventory. Keep URL decoding and Markdown parsing in
+   the later Markdown capability.
+2. Resolve committed symlink text relative to its containing path. Distinguish
+   existing entries, missing entries, outside-checkout paths, submodule contents,
+   and symlink cycles. Classification is data for the caller.
+3. Preserve ordinary directory destinations where the committed tree contains
+   descendants. Return the resolved identity needed for future fragment lookup.
+4. Keep inspection errors separate from normal resolution observations.
+
+**Verification:** Cover relative paths, parent traversal within and outside the
+root, existing directories, symlinks pointing inside/outside the root, cycles,
+and submodule traversal. Show identical classifications with neighboring files
+present and absent. These tests own path resolution; later Markdown tests own
+syntax, anchors, and selected failure policy.
+
+## Stage 5: Add facade declarations and shared request conversion
+
+**Outcome:** The public model has an agreed representation, shared conversion,
+and cause-preserving errors across `Check` and `Publish`.
+
+1. Apply the API rollout decision under Open Questions. For the recommended
+   incremental rollout, implement the public diagnostic types now and retain
+   the complete specification above as the blueprint for capability stages.
+   Introduce `ReleaseRequest.Validation`, `ValidationSpec`, and supported
+   settings with the first capability implementation. This avoids exporting an
+   empty placeholder or a selection with no implementation.
+2. Define internal counterparts for the intended settings where required by
+   actual coordination code. Use explicit field/target conversions through
+   the same root helper for both operations. Add remaining fields as used.
+3. Convert internal failures into facade-owned `*ValidationError`, using typed
+   recognition and explicit diagnostic-field conversion. Retain the original
+   wrapped error as the cause so candidate context, underlying identity, and
+   `errors.Is`/`errors.As` behavior survive the conversion. Render the useful
+   failure once rather than repeating the same context at each wrapper.
+4. Keep source publication result conversion unchanged. Extend GoDoc for the
+   errors and any implemented settings, with links to current consumer guidance.
+5. Extend compiled public API checks for facade-owned named types and dependency
+   direction. Preserve the small independent `buildinfo` dependency graph.
+
+**Verification:** External-package tests cover typed error access and
+`errors.Is`/`errors.As` through both facades. Reuse existing integration fixtures
+for a failing suite and an additional-check cause; avoid a new generic test
+framework. Run `go run ./internal/apicheck`.
+
+## Stage 6: Establish equivalent profile mapping and CLI presentation
+
+**Outcome:** Profile conversion has one owner and the CLI reports structured
+validation failures consistently while retaining current exit classification.
+
+1. Inspect `cmd/releasekit/internal/cli/config.go` and
+   `cmd/releasekit/main.go`. Share profile-to-request conversion for check and
+   publication through a small command-layer helper. Preserve additional-command
+   argument/environment handling and build/delivery mapping.
+2. Under the incremental rollout, add each `validation` setting with its
+   capability. Use CLI-owned JSON structs and explicit conversion at the command
+   boundary. Reuse ordinary JSON decoding and existing unknown-field rejection.
+   The schema blueprint above guides later additions; profile settings acquire
+   behavior at the same time as their Go counterparts.
+3. Add shared CLI presentation for facade validation errors. Show check identity,
+   useful findings and cause context once. Retain publication phase/state and
+   uncertain-outcome reporting. Present validation failures through the existing
+   status 1 path and malformed profile settings through the existing status 2
+   path; preserve other statuses.
+4. Update current CLI documentation for the diagnostic behavior that has landed.
+   Update configuration documentation only for implemented settings.
+
+**Verification:** Exercise default/omitted settings and existing additional
+commands. Check useful diagnostics and exit classifications for both operations,
+including a publication validation failure before tag creation. Capability work
+will add setting-specific mapping/equivalence cases at this established boundary.
+
+## Stage 7: Record capability integration recipes and consumer shapes
+
+**Outcome:** Subsequent feature prompts can use the foundation without redesigning
+its ownership or inventing a public extension framework.
+
+1. Add focused internal guidance for adding a typed setting, facade/profile
+   conversion, ordinary checker function, identity/findings, and behavioral
+   coverage through the shared executor. Update the inventory and link to that
+   owner for mechanics.
+2. Record the Go coordination recipe: construct one test command with selected
+   flags; select native full vet when requested; use static package compilation
+   for declared targets after test/vet. Reuse target/environment handling while
+   keeping artifact metadata and version smoke tests in the builder.
+3. Record how file checks consume the candidate reader and return findings.
+   Keep individual rule semantics in the release roadmap until implemented.
+   Markdown adopts PromptKit's current conventions as its starting specification;
+   parser/anchor implementation belongs to the Markdown feature prompt.
+4. Add prospective Go/profile examples under this roadmap for the three consumer
+   shapes: Audita with omitted optional validation and existing artifact builds;
+   ConfigKit with race, full vet, and two Linux compilation targets; PromptKit
+   with shared selections plus substantive example assertions in `checks`.
+   Label prospective fields clearly. Maintained copyable consumer examples gain
+   selections when those selections are implemented.
+5. Compare those declarations with local consumer scripts and profiles read-only.
+   Record which duplicated mechanics each later capability removes. ConfigKit's
+   separate toolchain-support commitments remain consumer compatibility decisions.
+
+**Verification:** Check the proposed declaration pairs against the schema design
+and the ownership table in `model.md`. Confirm product checks still fit the
+existing callback/profile extension point. Runtime tests are self-contained and
+independent of the three sibling checkouts.
+
+## Stage 8: Verify and document the completed foundation
+
+**Outcome:** The enabling refactor is complete, its current boundary is clear,
+and later feature implementation can proceed through the shared model.
+
+1. Review the assembled request flow, cause conversions, candidate inspection,
+   CLI presentation, and facade imports against ADR 0003 and the architecture
+   policy. Address concrete defects within the stages' scope.
+2. Run `go test ./...` and `go run ./internal/apicheck` from the repository root.
+   Add focused integration coverage only for a remaining risk, such as default
+   behavior diverging between API/profile or check/publication paths.
+3. Update current GoDoc, consumer guidance, CLI reference, and internal owners
+   for implemented behavior. Keep future setting examples and pending capability
+   contracts in roadmaps. Check local links and Markdown whitespace.
+4. Update `model.md` and this plan's status to identify the completed foundation
+   and the individual capability work remaining in `v0.2.0.md`. A foundation
+   completion is distinct from a completed v0.2.0 release.
+5. Commit the completed unit and report its verification, remaining capability
+   work, and clean or preserved worktree state. The subsequent architecture
+   review and testing/next-step decisions belong to the user after this sequence.
+
+## Open Questions
+
+### Should the complete optional specification be exported in the enabling sprint?
+
+**Recommendation:** Adopt the complete schema as the design and introduce each
+public setting with its functioning capability. Execute the shared refactor,
+diagnostics, candidate readers, and conversion boundaries in this plan first.
+The first capability then introduces `ValidationSpec` and the request/profile
+field; subsequent capabilities add their supported selections. This keeps Go
+and profile behavior equivalent, avoids public placeholders and temporary
+unsupported-option machinery, and lets each feature own meaningful coverage.
+The model's full target is reached as the feature implementations land.
+
+**Viable alternative:** Export the full schema during the enabling sprint and
+report an explicitly selected, unimplemented check as unavailable before
+validation proceeds. This establishes the complete configuration shape at once
+and supports declaration-level integration work. It needs a small temporary
+internal availability check and clear current documentation; otherwise an
+enabled selection could be silently omitted. Remove each availability guard
+when its checker lands. Zero/default declarations continue to work.
+
+**Another viable alternative:** Include all optional validation checker bodies
+in this sprint, splitting Markdown parsing and destination/anchor validation
+into separate additional stages. This delivers a fully operational specification
+at once and avoids transitional API behavior, while broadening this enabling
+plan into most of the validation feature work in the release roadmap. It still
+leaves CI tooling and the expected publication endpoint as separate work.
+
+Stages 1–4 are independent of this choice. Stages 5–8 describe the recommended
+rollout. The choice needs resolution before an implementation agent treats the
+public API rollout as settled.