eric/ConfigKit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add an everyday check command and contributor test map

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-07 01:40:39 +00:00
parent cb5b4356aa
commit 994a9ea0e2
3 changed files with 172 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files

74
docs/development.md
View File

@@ -1,21 +1,16 @@
# Development
This is the contributor entry point for ConfigKit. Read the task-specific owners
below before making changes. Its public
loader currently covers strict selected sources, explicit path/glob import graphs,
composition, selected profile overlays, typed defaults, schema-version checks,
explicit CLI overrides, recorded application resolution, semantic validation,
layered metadata, secret
classification, typed environment references, explicit secrets bootstrap, and
redacted effective rendering and historical value explanation. The
[component overview](internal/overview.md) owns the implementation inventory.
Start here when contributing to ConfigKit. Read the task-specific owners below
before making changes. The [component overview](internal/overview.md) maps
implementation responsibilities; the [configuration contract](config.md) owns
public behavior.
## Initial orientation
1. Inspect the working tree and preserve unrelated changes.
2. Read the architecture policy for code or design work.
3. Read the policy, contract, and internal documents applicable to the task.
4. Inspect relevant implementation and tests where they exist before changing them.
4. Inspect the relevant implementation and behavioral tests before changing them.
## Task-specific reading guide
@@ -24,7 +19,7 @@ redacted effective rendering and historical value explanation. The
| Repository orientation or component responsibility | [Architecture policy](policy/architecture.md) and [component overview](internal/overview.md) |
| Documentation changes | [Documentation policy](policy/documentation.md) and the owning document |
| Architecture decisions or future work | [Documentation policy](policy/documentation.md), [initial target record](roadmap/initial.md), and relevant ADRs listed below |
| Code and test changes | [Architecture policy](policy/architecture.md), [testing policy](policy/testing.md), and owning contracts/tests as they are introduced |
| Code and test changes | [Architecture policy](policy/architecture.md), [testing policy](policy/testing.md), and the owning contract and tests from the map below |
| Examples or copyable assets | The owning implemented contract and [consumer adoption guide](consumers/README.md) |
| Release preparation | [Release policy](policy/release.md), [release procedure](release.md), and [release-readiness evidence](roadmap/release-readiness.md) |
@@ -33,7 +28,6 @@ Accepted decisions:
- [Separate composition and overlay](adr/0001-separate-composition-and-overlay.md)
- [Own YAML semantics behind a private parser boundary](adr/0002-own-yaml-semantics.md)
- [Preserve presence and record application resolution](adr/0003-preserve-presence-and-resolution-history.md)
- [Secret references and explicit bootstrap](adr/0004-secret-references-and-explicit-bootstrap.md)
- [Explicit overrides and safe explanation](adr/0005-explicit-overrides-and-safe-explanation.md)
@@ -51,27 +45,51 @@ environment:
go mod download
```
Run the library baseline from the repository root:
Run the everyday check with Bash 3.2 or newer and Git:
```sh
go test ./...
go vet ./...
go build ./...
git diff --check
./scripts/check.sh
```
The default tests must remain deterministic and runnable without network access or
credentials after dependency acquisition. Also inspect new untracked files, verify
changed local links and heading fragments, and review paths, examples, ownership,
and current-versus-planned claims. The commands above do not check links or
untracked content. Do not use an application binary build as the library baseline.
Add executable-example and other checks when those artifacts are introduced; do
not document nonexistent tools as operational.
[The script](../scripts/check.sh) runs from the repository root regardless of the
invocation directory. It checks Go formatting without rewriting files, checks
staged/unstaged and untracked whitespace, then runs `go test ./...`,
`go vet ./...`, and `go build ./...`, stopping on the first failure. It uses the
installed Go toolchain by default, honors an explicit `GOTOOLCHAIN`, and disables
dependency downloads. Acquire dependencies and any selected toolchain beforehand.
Format reported Go files with `gofmt -w <files>`.
Once code exists, use focused checks while iterating and validation proportionate
to the change under the testing policy. Documentation-only work does not require
Go checks unless it changes behavior-bearing examples or commands they validate.
Release validation is a separate, stricter gate governed by release policy.
The default suite includes executable examples, maintained consumer fixtures,
local documentation-link checks, and the dependency import-boundary check.
It must stay deterministic and runnable without network access or credentials.
Still inspect new/untracked content and review paths, examples, ownership, and
current-versus-planned claims; automated checks do not replace content review.
Use focused checks while iterating, with validation proportionate to the change
under the testing policy. Documentation-only work normally needs the local-link
check below and whitespace review; behavior-bearing examples and commands also
need execution. Use `go test -race ./...` for ownership or concurrency changes.
The separate [release checker](release.md#4-validate-the-source-candidate) adds
two-toolchain, module-hygiene, race, and cross-build gates.
## Task-to-test map
Commands run from the repository root after dependency acquisition. These are
starting points for focused iteration; run the everyday check before handing off
code changes. Use the component overview to find the corresponding implementation.
| Task | Behavioral test owners | Focused command |
| --- | --- | --- |
| YAML syntax, diagnostics, parser limits | [Adapter tests](../internal/yaml/parse_test.go), [source translation](../parse_test.go) | `go test ./internal/yaml` and `go test . -run '^TestParseSource'` |
| File/reader access, imports, globs, containment | [Sources](../source_test.go), [FIFO](../source_fifo_test.go), [imports](../imports_test.go), [globs](../glob_test.go) | `go test . -run 'Source|Import|Glob|Filesystem|Nonregular|RegularFile'` |
| Composition, profiles, CLI precedence | [Composition](../compose_test.go), [overlay](../overlay_test.go), [profiles](../profile_test.go), [overrides](../override_test.go) | `go test . -run 'Compose|Overlay|Profile|Override'` |
| Schema, defaults, projection, cloning | [Schema](../schema_test.go), [decoding](../decode_test.go), [projection](../projection_test.go) | `go test . -run 'Schema|Decode|Projection|Clone|Inline|CompareDocument'` |
| Lifecycle, resolution, validation, callback ownership | [Loading](../load_test.go), [resolution](../resolution_test.go), [validation](../validation_test.go), [ownership](../callback_ownership_test.go), [independence](../independence_test.go) | `go test . -run 'Load|Resolution|Validation|Validator|Hook|Callback|Parallel'` |
| Paths, metadata, provenance | [Paths](../path_test.go), [metadata](../metadata_test.go), [provenance](../provenance_test.go) | `go test . -run 'Path|Metadata|Provenance|Queries'` |
| Secrets, bootstrap, redaction, explanation | [References](../secret_reference_test.go), [bootstrap](../secrets_test.go), [rendering](../render_test.go), [explanation](../explain_test.go) | `go test . -run 'Secret|Bootstrap|Effective|Explain|Explanation|History|Sensitivity'` |
| Consumer examples and migration fixtures | [Consumer tests](../examples/consumers/consumers_test.go), [API examples](../example_test.go) | `go test ./examples/consumers` and `go test . -run '^Example'` |
| Documentation links and dependency isolation | [Link checks](../documentation_test.go), [import boundary](../import_boundary_test.go) | `go test . -run '^(TestLocalDocumentationLinks|TestDependencyImportBoundary)$'` |
| Check-command behavior | [Shell regression](../release_check_test.go) | `go test . -run '^TestCheckScripts'` |
## Fuzz validation

166
release_check_test.go
View File

@@ -9,91 +9,95 @@ import (
"testing"
)
func TestReleaseCheckerShellHandlesFilenamesAndStopsOnFailure(t *testing.T) {
script, err := os.ReadFile("scripts/release-check.sh")
if err != nil {
t.Fatal(err)
}
for _, fail := range []bool{false, true} {
name := "success"
if fail {
name = "failed-go-check"
}
t.Run(name, func(t *testing.T) {
root := t.TempDir()
bin := t.TempDir()
log := filepath.Join(bin, "gofmt.args")
calls := filepath.Join(bin, "go.calls")
for _, directory := range []string{"scripts", "nested"} {
if err := os.Mkdir(filepath.Join(root, directory), 0700); err != nil {
t.Fatal(err)
}
}
write := func(path string, data []byte, mode os.FileMode) {
t.Helper()
if err := os.WriteFile(path, data, mode); err != nil {
t.Fatal(err)
}
}
write(filepath.Join(root, "scripts", "release-check.sh"), script, 0700)
write(filepath.Join(root, "go.mod"), []byte("module example.invalid/check\n"), 0600)
write(filepath.Join(root, "go.sum"), nil, 0600)
files := []string{"nested/a space.go", "nested/a\nnewline.go"}
for _, file := range files {
write(filepath.Join(root, file), []byte("package example\n"), 0600)
}
// Isolate expensive toolchain commands: this test owns shell orchestration,
// not Go's compiler or the production tests exercised by the real checker.
write(filepath.Join(bin, "go"), []byte("#!/bin/sh\nprintf '%s\\n' \"$*\" >> \"$CONFIGKIT_TEST_GO_LOG\"\nif [ \"${CONFIGKIT_TEST_FAIL-}\" = yes ] && [ \"${1-}\" = test ]; then exit 37; fi\n"), 0700)
write(filepath.Join(bin, "gofmt"), []byte("#!/bin/sh\nprintf '%s\\000' \"$@\" > \"$CONFIGKIT_TEST_FMT_LOG\"\n"), 0700)
init := exec.Command("git", "init", "-q")
init.Dir = root
if out, err := init.CombinedOutput(); err != nil {
t.Fatalf("git init: %v %s", err, out)
}
// HEAD is required by the whitespace gate. Commit only the module baseline.
for _, args := range [][]string{{"add", "go.mod", "go.sum"}, {"-c", "user.name=Test", "-c", "user.email=test@example.invalid", "-c", "commit.gpgsign=false", "commit", "-qm", "Test fixture"}} {
cmd := exec.Command("git", args...)
cmd.Dir = root
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("git fixture: %v %s", err, out)
}
}
cmd := exec.Command("/bin/bash", filepath.Join(root, "scripts", "release-check.sh"))
cmd.Env = append(os.Environ(), "PATH="+bin+string(os.PathListSeparator)+os.Getenv("PATH"), "CONFIGKIT_BASELINE_TOOLCHAIN=test-baseline", "CONFIGKIT_NEWER_TOOLCHAIN=test-newer", "CONFIGKIT_TEST_FMT_LOG="+log, "CONFIGKIT_TEST_GO_LOG="+calls)
if fail {
cmd.Env = append(cmd.Env, "CONFIGKIT_TEST_FAIL=yes")
}
out, err := cmd.CombinedOutput()
if fail {
if err == nil {
t.Fatal("checker ignored failed Go test")
}
} else if err != nil {
t.Fatalf("checker: %v %s", err, out)
}
arguments, err := os.ReadFile(log)
func TestCheckScriptsHandleFilenamesAndStopOnFailure(t *testing.T) {
for _, scriptName := range []string{"release-check.sh", "check.sh"} {
t.Run(scriptName, func(t *testing.T) {
script, err := os.ReadFile("scripts/" + scriptName)
if err != nil {
t.Fatal(err)
}
parts := bytes.Split(arguments, []byte{0})
for _, file := range files {
found := false
for _, arg := range parts {
if string(arg) == file {
found = true
for _, name := range []string{"success", "failed-go-check", "untracked-whitespace"} {
fail := name != "success"
t.Run(name, func(t *testing.T) {
root := t.TempDir()
bin := t.TempDir()
log := filepath.Join(bin, "gofmt.args")
calls := filepath.Join(bin, "go.calls")
for _, directory := range []string{"scripts", "nested"} {
if err := os.Mkdir(filepath.Join(root, directory), 0700); err != nil {
t.Fatal(err)
}
}
}
if !found {
t.Fatalf("filename was split: %q", file)
}
}
invoked, err := os.ReadFile(calls)
if err != nil {
t.Fatal(err)
}
if fail && strings.Contains(string(invoked), "vet ./...") {
t.Fatal("checker continued after failed test")
write := func(path string, data []byte, mode os.FileMode) {
t.Helper()
if err := os.WriteFile(path, data, mode); err != nil {
t.Fatal(err)
}
}
write(filepath.Join(root, "scripts", scriptName), script, 0700)
write(filepath.Join(root, "go.mod"), []byte("module example.invalid/check\n"), 0600)
write(filepath.Join(root, "go.sum"), nil, 0600)
files := []string{"nested/a space.go", "nested/a\nnewline.go"}
for _, file := range files {
write(filepath.Join(root, file), []byte("package example\n"), 0600)
}
if name == "untracked-whitespace" {
write(filepath.Join(root, "notes.md"), []byte("trailing space \n"), 0600)
}
// Isolate expensive toolchain commands: this test owns shell orchestration,
// not Go's compiler or the production tests exercised by the real checker.
write(filepath.Join(bin, "go"), []byte("#!/bin/sh\nprintf '%s\\n' \"$*\" >> \"$CONFIGKIT_TEST_GO_LOG\"\nif [ \"${CONFIGKIT_TEST_FAIL-}\" = yes ] && [ \"${1-}\" = test ]; then exit 37; fi\n"), 0700)
write(filepath.Join(bin, "gofmt"), []byte("#!/bin/sh\nprintf '%s\\000' \"$@\" > \"$CONFIGKIT_TEST_FMT_LOG\"\n"), 0700)
init := exec.Command("git", "init", "-q")
init.Dir = root
if out, err := init.CombinedOutput(); err != nil {
t.Fatalf("git init: %v %s", err, out)
}
// HEAD is required by the whitespace gate. Commit only the module baseline.
for _, args := range [][]string{{"add", "go.mod", "go.sum"}, {"-c", "user.name=Test", "-c", "user.email=test@example.invalid", "-c", "commit.gpgsign=false", "commit", "-qm", "Test fixture"}} {
cmd := exec.Command("git", args...)
cmd.Dir = root
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("git fixture: %v %s", err, out)
}
}
cmd := exec.Command("/bin/bash", filepath.Join(root, "scripts", scriptName))
cmd.Env = append(os.Environ(), "PATH="+bin+string(os.PathListSeparator)+os.Getenv("PATH"), "CONFIGKIT_BASELINE_TOOLCHAIN=test-baseline", "CONFIGKIT_NEWER_TOOLCHAIN=test-newer", "CONFIGKIT_TEST_FMT_LOG="+log, "CONFIGKIT_TEST_GO_LOG="+calls)
if name == "failed-go-check" {
cmd.Env = append(cmd.Env, "CONFIGKIT_TEST_FAIL=yes")
}
out, err := cmd.CombinedOutput()
if fail {
if err == nil {
t.Fatal("checker ignored failed Go test")
}
} else if err != nil {
t.Fatalf("checker: %v %s", err, out)
}
arguments, err := os.ReadFile(log)
if err != nil {
t.Fatal(err)
}
parts := bytes.Split(arguments, []byte{0})
for _, file := range files {
found := false
for _, arg := range parts {
if string(arg) == file {
found = true
}
}
if !found {
t.Fatalf("filename was split: %q", file)
}
}
invoked, err := os.ReadFile(calls)
if err != nil {
t.Fatal(err)
}
if fail && strings.Contains(string(invoked), "vet ./...") {
t.Fatal("checker continued after failed test")
}
})
}
})
}

41
scripts/check.sh Executable file
View File

@@ -0,0 +1,41 @@
#!/usr/bin/env bash
set -euo pipefail
repository_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)
cd "$repository_dir"
# Use the installed toolchain unless the caller explicitly selects another.
# Toolchains and dependencies must already be available locally.
export GOTOOLCHAIN=${GOTOOLCHAIN:-local}
export GOPROXY=off
go version
go_files=()
while IFS= read -r -d '' go_file; do
if [[ -f "$go_file" ]]; then
go_files+=("$go_file")
fi
done < <(git ls-files --cached --others --exclude-standard -z -- '*.go')
if ((${#go_files[@]})); then
unformatted=$(gofmt -l -- "${go_files[@]}")
if [[ -n "$unformatted" ]]; then
printf 'gofmt required for:\n%s\n' "$unformatted" >&2
exit 1
fi
fi
git diff --check HEAD --
while IFS= read -r -d '' candidate_file; do
# --no-index returns 1 for differing content even when whitespace is clean;
# whitespace errors return 2. Preserve other failures as well.
git diff --no-index --check -- /dev/null "$candidate_file" || {
check_status=$?
if ((check_status != 1)); then
exit "$check_status"
fi
}
done < <(git ls-files --others --exclude-standard -z)
go test ./...
go vet ./...
go build ./...