eric/ConfigKit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add explicit secrets directory bootstrap

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-06 21:29:25 +00:00
parent 828fdd38eb
commit 16f4212770
7 changed files with 845 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@@ -9,8 +9,9 @@ can overlay one selected fixed profile and check an application schema version.
returns detached metadata and safe diagnostics.
Applications may perform audited defaulting and normalization through a resolver,
then report domain errors and warnings through read-only semantic validation. Secret
tags and a path predicate drive deterministic redacted effective YAML.
Overrides, secret references/bootstrap, and value explanation are still under
tags and a path predicate drive deterministic redacted effective YAML. Explicit
secrets-directory bootstrap is available as a separate startup operation.
Overrides, typed secret references, and value explanation are still under
development.
```go

35
docs/config.md
View File

@@ -144,6 +144,41 @@ configuration values. Applications must still keep source labels, resolution
reasons, and diagnostic messages value-free because ConfigKit cannot infer and
sanitize secrets embedded in arbitrary application prose.
## Explicit secrets-directory bootstrap
`LoadSecretsDir(directory)` is a separate, opt-in startup operation. It is never
called by `Load`, is not a YAML option, and does not participate in configuration
validation. Applications choose the native directory path and should invoke the
operation once before starting concurrent work. The selected path may itself be a
symlink; ConfigKit resolves it to an existing canonical directory before examining
its immediate entries.
The bootstrap enumerates at most 1,024 entries and sorts them by filename. Every
entry must have an ASCII environment-variable name and be a regular, nonsymlink
file. Metadata for all entries is validated before file content is read or process
state is changed. Existing environment variables win, including variables whose
value is empty; their files are not opened. Each eligible file is limited to 1 MiB,
and eligible content is limited to 8 MiB in total. ConfigKit removes exactly one
terminal LF or CRLF, preserves all other whitespace, accepts an empty value, and
rejects NUL bytes.
All eligible files are read successfully before application begins. Presence is
checked again immediately before each `os.Setenv`: a variable populated since
preflight is skipped. A write failure stops application without rollback, so prior
writes remain and later eligible entries are `not_applied`. This operation is not
atomic and does not provide compare-and-set semantics for concurrent environment
mutation.
`SecretsReport.Entries` returns detached, filename-ordered `SecretFileResult`
records with `loaded`, `skipped_existing`, `not_applied`, or `failed` status.
`ExistingEmpty` distinguishes an empty existing winner. `SecretsReport.WriteTo`
renders only quoted names, canonical paths, statuses, and that empty-state flag;
it never retains or renders file contents. Failures return the non-nil partial
report with a `*SecretsError` containing detached `secrets.*` diagnostics and no
raw filesystem or environment error chain. Applications should retain the report
alongside the later configuration result and explicitly present blocked and failed
files when appropriate; ConfigKit stores no implicit global report.
### Recorded application resolution
An optional `Definition.Resolve` callback runs after successful typed decoding. It

19
docs/internal/extraction.md
View File

@@ -300,6 +300,16 @@ Assigned to the explicit bootstrap operation:
fixed budgets. Logging, configuration-driven implicit invocation, private-mode
conventions, and application API-key plumbing remain in Narratio.
Implementation record (2026-09-06): `LoadSecretsDir`, `readSecretsEntries`,
`readSecretValue`, and the private environment/file seams in `secrets.go` adapt
Narratio's bounded-read and preserve-existing mechanisms at revision
`b68435128be1e24b94fc7ec259d6e9093669fd99`. The corresponding tests retain its
valid-file, existing-variable (including empty), missing/permission/read,
nonregular, symlink, and byte-limit cases. ConfigKit validates invalid entries
rather than skipping them, permits ordinary deployment permission modes, resolves
the selected directory symlink, performs complete preflight before mutation, and
emits per-file safe records without application configuration or logging.
## WeatherReporter evidence
### Secrets bootstrap
@@ -324,6 +334,15 @@ Assigned to the explicit bootstrap operation:
entries are not read. Bootstrap is explicitly invoked and remains independent of
YAML loading and semantic validation.
Implementation record (2026-09-06): the same `secrets.go` state machine adapts
WeatherReporter's `loadSecrets`/`applySecrets` separation, environment-name check,
regular-file rejection, and exact one-LF/CRLF removal at revision
`5d306a202471e10fe98923a1df27d0b0fce5c666`. Tests adapt its newline matrix,
invalid-entry and preflight-no-write cases, and controlled application failure.
The overwrite/rollback expectation is intentionally replaced by existing-winner,
presence-recheck, and partial-progress statuses; no configuration loader invokes
the bootstrap.
### Defaults, explicit presence, and module options
Assigned to consumer-shaped examples and migration checks:

31
docs/roadmap/implementation.md
View File

@@ -1164,7 +1164,36 @@ Acceptance:
## Stage 16 — Extract explicit secrets-directory bootstrap
Status: pending.
Status: complete.
Completion evidence (2026-09-06):
- Added the separate, opt-in `LoadSecretsDir` startup boundary in
[`secrets.go`](../../secrets.go), with detached per-file results, stable status
values, safe `SecretsError` diagnostics, and a deterministic quoted report
renderer. `Load` remains process-state-free and has no secrets-directory option.
- Adapted WeatherReporter's read/apply separation, filename/type checks, exact
one-LF/CRLF removal, and failure cases together with Narratio's bounded reads and
existing-variable preservation. The [extraction inventory](../internal/extraction.md)
records both inspected revisions, reused symbols/cases, and the intentional
removal of overwrite, rollback, permission-mode, implicit configuration, and
logging policies.
- Implemented bounded immediate enumeration (1,024 entries), canonical directory
selection, sorted metadata validation, 1 MiB eligible-file and 8 MiB aggregate
reads, NUL rejection, full preflight before writes, and a per-write presence
recheck. Existing empty values are distinguished, skipped files are never opened,
and apply failures retain accurate partial progress without rollback.
- Added real filesystem/environment and private-seam coverage in
[`secrets_test.go`](../../secrets_test.go) for canonical ordering, selected
directory symlinks, newline/empty values, invalid names/types, all bounds,
missing/permission/read failures, no-write preflight failures, newly populated
variables, controlled apply failure, defensive copies, non-disclosure, and writer
failures. The [configuration contract](../config.md#explicit-secrets-directory-bootstrap)
documents startup-only mutation, non-atomicity, reporting blocked files, and
retaining reports alongside configuration results. Verified `go test ./...`,
`go test -race ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and
`git diff --check` with Go 1.26.7 on linux/amd64. No deviations were required;
environment-backed typed secret references remain later work.
Read: [secret bootstrap contract](initial.md#secret-references-and-explicit-bootstrap),
architecture responsibility boundary, testing policy, and WeatherReporter's

10
records.go
View File

@@ -117,6 +117,16 @@ const (
CodeValidateWarning Code = "validate.warning"
CodeValidateInvalidPath Code = "validate.invalid_path"
CodeValidateMutation Code = "validate.mutation"
CodeSecretsInvalidPath Code = "secrets.invalid_path"
CodeSecretsMissing Code = "secrets.missing"
CodeSecretsPermission Code = "secrets.permission"
CodeSecretsRead Code = "secrets.read"
CodeSecretsNotRegular Code = "secrets.not_regular"
CodeSecretsInvalidName Code = "secrets.invalid_name"
CodeSecretsLimitExceeded Code = "secrets.limit_exceeded"
CodeSecretsInvalidValue Code = "secrets.invalid_value"
CodeSecretsApplyFailed Code = "secrets.apply_failed"
)
// Location identifies a position in a configuration source. Zero line and column

371
secrets.go Normal file
View File

@@ -0,0 +1,371 @@
package configkit
import (
"errors"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
)
const (
secretsEntryLimit = 1024
secretFileByteLimit = int64(1 << 20)
secretsByteLimit = int64(8 << 20)
)
// SecretFileStatus describes the outcome for one file considered by
// LoadSecretsDir.
type SecretFileStatus string
const (
SecretFileLoaded SecretFileStatus = "loaded"
SecretFileSkippedExisting SecretFileStatus = "skipped_existing"
SecretFileNotApplied SecretFileStatus = "not_applied"
SecretFileFailed SecretFileStatus = "failed"
)
// SecretFileResult is the safe, value-free bootstrap result for one directory
// entry. ExistingEmpty is meaningful when Status is SecretFileSkippedExisting.
type SecretFileResult struct {
Name string
Path string
Status SecretFileStatus
ExistingEmpty bool
}
// SecretsReport is a detached snapshot of an explicit secrets-directory
// bootstrap. It does not retain file contents or environment values.
type SecretsReport struct {
entries []SecretFileResult
}
// Entries returns a copy of the report's filename-ordered entries.
func (r *SecretsReport) Entries() []SecretFileResult {
if r == nil {
return nil
}
return slices.Clone(r.entries)
}
// WriteTo renders one safe, deterministic line per entry. Names and paths are
// quoted so control characters cannot affect terminal output.
func (r *SecretsReport) WriteTo(output io.Writer) (int64, error) {
if output == nil {
return 0, errors.New("configkit: secrets report output is nil")
}
if r == nil {
return 0, nil
}
var total int64
for _, entry := range r.entries {
line := "status=" + strconv.Quote(string(entry.Status)) +
" name=" + strconv.Quote(entry.Name) +
" path=" + strconv.Quote(entry.Path) +
" existing_empty=" + strconv.FormatBool(entry.ExistingEmpty) + "\n"
written, err := io.WriteString(output, line)
total += int64(written)
if err != nil {
return total, err
}
if written != len(line) {
return total, io.ErrShortWrite
}
}
return total, nil
}
// SecretsError reports safe structured diagnostics for a failed explicit
// secrets-directory bootstrap. It intentionally does not unwrap filesystem or
// environment errors.
type SecretsError struct {
diagnostics []Diagnostic
}
// Error implements error without rendering filenames, paths, or file contents.
func (e *SecretsError) Error() string {
if e == nil {
return "configkit: secrets bootstrap failed"
}
count := 0
for _, diagnostic := range e.diagnostics {
if diagnostic.Severity == SeverityError {
count++
}
}
if count == 1 {
return "configkit: secrets bootstrap failed with 1 error"
}
return fmt.Sprintf("configkit: secrets bootstrap failed with %d errors", count)
}
// Diagnostics returns a deep copy of the bootstrap diagnostics.
func (e *SecretsError) Diagnostics() []Diagnostic {
if e == nil {
return nil
}
return cloneDiagnostics(e.diagnostics)
}
type secretsEnvironment struct {
lookup func(string) (string, bool)
set func(string, string) error
}
type secretsAccess struct {
openDirectory func(string) (*os.File, error)
lstat func(string) (fs.FileInfo, error)
openFile func(string) (*os.File, error)
}
type stagedSecret struct {
entry int
value string
}
// LoadSecretsDir explicitly loads immediate regular files from directory into
// previously unset process environment variables. Applications should call it
// once during startup, before concurrent work; Load never calls it.
func LoadSecretsDir(directory string) (*SecretsReport, error) {
return loadSecretsDir(directory, secretsEnvironment{lookup: os.LookupEnv, set: os.Setenv}, secretsAccess{
openDirectory: os.Open,
lstat: os.Lstat,
openFile: os.Open,
})
}
func loadSecretsDir(directory string, environment secretsEnvironment, access secretsAccess) (*SecretsReport, error) {
report := &SecretsReport{}
if strings.TrimSpace(directory) == "" {
return report, secretsFailure(CodeSecretsInvalidPath, "secrets directory path is required")
}
canonical, err := canonicalExistingPath(directory)
if err != nil {
return report, secretsFailure(classifySecretsFilesystemError(err), "secrets directory could not be canonicalized: "+strconv.Quote(cleanAbsolutePath(directory)))
}
info, err := os.Stat(canonical)
if err != nil {
return report, secretsFailure(classifySecretsFilesystemError(err), "secrets directory could not be inspected: "+strconv.Quote(canonical))
}
if !info.IsDir() {
return report, secretsFailure(CodeSecretsInvalidPath, "secrets directory is not a directory: "+strconv.Quote(canonical))
}
directoryFile, err := access.openDirectory(canonical)
if err != nil {
return report, secretsFailure(classifySecretsFilesystemError(err), "secrets directory could not be opened: "+strconv.Quote(canonical))
}
entries, readErr := readSecretsEntries(directoryFile)
closeErr := directoryFile.Close()
if readErr != nil {
return report, secretsFailure(classifySecretsFilesystemError(readErr), "secrets directory could not be enumerated: "+strconv.Quote(canonical))
}
if closeErr != nil {
return report, secretsFailure(CodeSecretsRead, "secrets directory could not be closed: "+strconv.Quote(canonical))
}
slices.SortFunc(entries, func(left, right fs.DirEntry) int { return strings.Compare(left.Name(), right.Name()) })
report.entries = make([]SecretFileResult, len(entries))
for index, entry := range entries {
name := entry.Name()
report.entries[index] = SecretFileResult{Name: name, Path: filepath.Join(canonical, name), Status: SecretFileNotApplied}
}
if len(entries) > secretsEntryLimit {
return report, secretsFailure(CodeSecretsLimitExceeded, "secrets directory exceeds the 1024-entry limit: "+strconv.Quote(canonical))
}
for index := range report.entries {
entry := &report.entries[index]
if !validEnvironmentName(entry.Name) {
entry.Status = SecretFileFailed
return report, secretsFailure(CodeSecretsInvalidName, "secret filename is not a valid environment name: "+strconv.Quote(entry.Path))
}
info, err := access.lstat(entry.Path)
if err != nil {
entry.Status = SecretFileFailed
return report, secretsFailure(classifySecretsFilesystemError(err), "secret file could not be inspected: "+strconv.Quote(entry.Path))
}
if info.Mode()&fs.ModeSymlink != 0 || !info.Mode().IsRegular() {
entry.Status = SecretFileFailed
return report, secretsFailure(CodeSecretsNotRegular, "secret entry is not a regular file: "+strconv.Quote(entry.Path))
}
}
staged := make([]stagedSecret, 0, len(report.entries))
var consumed int64
for index := range report.entries {
entry := &report.entries[index]
if existing, present := environment.lookup(entry.Name); present {
entry.Status = SecretFileSkippedExisting
entry.ExistingEmpty = existing == ""
continue
}
remaining := secretsByteLimit - consumed
value, used, err := readSecretValue(entry.Path, remaining, access)
if err != nil {
entry.Status = SecretFileFailed
return report, secretsFailure(secretReadCode(err), secretReadMessage(err)+": "+strconv.Quote(entry.Path))
}
consumed += used
staged = append(staged, stagedSecret{entry: index, value: value})
}
for _, secret := range staged {
entry := &report.entries[secret.entry]
if existing, present := environment.lookup(entry.Name); present {
entry.Status = SecretFileSkippedExisting
entry.ExistingEmpty = existing == ""
continue
}
if err := environment.set(entry.Name, secret.value); err != nil {
entry.Status = SecretFileFailed
return report, secretsFailure(CodeSecretsApplyFailed, "secret environment variable could not be set for file: "+strconv.Quote(entry.Path))
}
entry.Status = SecretFileLoaded
}
return report, nil
}
func readSecretsEntries(directory *os.File) ([]fs.DirEntry, error) {
entries := make([]fs.DirEntry, 0)
for len(entries) <= secretsEntryLimit {
batch, err := directory.ReadDir(secretsEntryLimit + 1 - len(entries))
entries = append(entries, batch...)
if err == io.EOF {
return entries, nil
}
if err != nil {
return entries, err
}
if len(entries) > secretsEntryLimit {
return entries, nil
}
}
return entries, nil
}
type secretReadFailure int
const (
secretReadFilesystem secretReadFailure = iota
secretReadFileLimit
secretReadTotalLimit
secretReadNUL
)
type secretReadError struct {
kind secretReadFailure
err error
}
func (e *secretReadError) Error() string { return "secret read failed" }
func readSecretValue(path string, remaining int64, access secretsAccess) (string, int64, error) {
if remaining < 0 {
return "", 0, &secretReadError{kind: secretReadTotalLimit}
}
limit := min(secretFileByteLimit, remaining)
file, err := access.openFile(path)
if err != nil {
return "", 0, &secretReadError{kind: secretReadFilesystem, err: err}
}
data, readErr := io.ReadAll(io.LimitReader(file, limit+1))
closeErr := file.Close()
if readErr != nil {
return "", 0, &secretReadError{kind: secretReadFilesystem, err: readErr}
}
if closeErr != nil {
return "", 0, &secretReadError{kind: secretReadFilesystem, err: closeErr}
}
if int64(len(data)) > limit {
kind := secretReadFileLimit
if remaining < secretFileByteLimit {
kind = secretReadTotalLimit
}
return "", 0, &secretReadError{kind: kind}
}
if slices.Contains(data, byte(0)) {
return "", 0, &secretReadError{kind: secretReadNUL}
}
value := string(data)
if strings.HasSuffix(value, "\r\n") {
value = strings.TrimSuffix(value, "\r\n")
} else {
value = strings.TrimSuffix(value, "\n")
}
return value, int64(len(data)), nil
}
func validEnvironmentName(name string) bool {
if name == "" || !asciiEnvironmentFirst(name[0]) {
return false
}
for index := 1; index < len(name); index++ {
character := name[index]
if !asciiEnvironmentFirst(character) && (character < '0' || character > '9') {
return false
}
}
return true
}
func asciiEnvironmentFirst(character byte) bool {
return character == '_' || character >= 'A' && character <= 'Z' || character >= 'a' && character <= 'z'
}
func cleanAbsolutePath(path string) string {
absolute, err := filepath.Abs(path)
if err != nil {
return filepath.Clean(path)
}
return filepath.Clean(absolute)
}
func classifySecretsFilesystemError(err error) Code {
switch {
case errors.Is(err, fs.ErrNotExist):
return CodeSecretsMissing
case errors.Is(err, fs.ErrPermission):
return CodeSecretsPermission
default:
return CodeSecretsRead
}
}
func secretReadCode(err error) Code {
var failure *secretReadError
if errors.As(err, &failure) {
switch failure.kind {
case secretReadFileLimit, secretReadTotalLimit:
return CodeSecretsLimitExceeded
case secretReadNUL:
return CodeSecretsInvalidValue
case secretReadFilesystem:
return classifySecretsFilesystemError(failure.err)
}
}
return CodeSecretsRead
}
func secretReadMessage(err error) string {
var failure *secretReadError
if errors.As(err, &failure) {
switch failure.kind {
case secretReadFileLimit:
return "secret file exceeds the 1 MiB limit"
case secretReadTotalLimit:
return "eligible secret files exceed the 8 MiB total limit"
case secretReadNUL:
return "secret file contains a NUL byte"
}
}
return "secret file could not be read"
}
func secretsFailure(code Code, message string) *SecretsError {
return &SecretsError{diagnostics: []Diagnostic{{Code: code, Severity: SeverityError, Message: message}}}
}

377
secrets_test.go Normal file
View File

@@ -0,0 +1,377 @@
package configkit
import (
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"reflect"
"runtime"
"strings"
"testing"
)
func TestLoadSecretsDirLoadsCanonicalSortedValues(t *testing.T) {
directory := t.TempDir()
writeSecretTestFile(t, directory, "CONFIGKIT_SECRET_Z", "last\n\n")
writeSecretTestFile(t, directory, "CONFIGKIT_SECRET_A", "first\r\n")
writeSecretTestFile(t, directory, "CONFIGKIT_SECRET_EMPTY", "")
selected := directory
if runtime.GOOS != "windows" {
selected = filepath.Join(t.TempDir(), "selected")
if err := os.Symlink(directory, selected); err != nil {
t.Fatalf("Symlink() error = %v", err)
}
}
for _, name := range []string{"CONFIGKIT_SECRET_A", "CONFIGKIT_SECRET_EMPTY", "CONFIGKIT_SECRET_Z"} {
t.Setenv(name, "temporary")
if err := os.Unsetenv(name); err != nil {
t.Fatalf("Unsetenv(%q) error = %v", name, err)
}
}
report, err := LoadSecretsDir(selected)
if err != nil {
t.Fatalf("LoadSecretsDir() error = %v", err)
}
if got := os.Getenv("CONFIGKIT_SECRET_A"); got != "first" {
t.Fatalf("LF value = %q, want first", got)
}
if got := os.Getenv("CONFIGKIT_SECRET_Z"); got != "last\n" {
t.Fatalf("two-LF value = %q, want one retained LF", got)
}
if value, present := os.LookupEnv("CONFIGKIT_SECRET_EMPTY"); !present || value != "" {
t.Fatalf("empty value = %q, %v; want present empty", value, present)
}
canonical := canonicalTestPath(t, directory)
want := []SecretFileResult{
{Name: "CONFIGKIT_SECRET_A", Path: filepath.Join(canonical, "CONFIGKIT_SECRET_A"), Status: SecretFileLoaded},
{Name: "CONFIGKIT_SECRET_EMPTY", Path: filepath.Join(canonical, "CONFIGKIT_SECRET_EMPTY"), Status: SecretFileLoaded},
{Name: "CONFIGKIT_SECRET_Z", Path: filepath.Join(canonical, "CONFIGKIT_SECRET_Z"), Status: SecretFileLoaded},
}
if got := report.Entries(); !reflect.DeepEqual(got, want) {
t.Fatalf("Entries() = %#v, want %#v", got, want)
}
entries := report.Entries()
entries[0].Name = "mutated"
if report.Entries()[0].Name != want[0].Name {
t.Fatal("Entries() did not return a detached slice")
}
}
func TestLoadSecretsDirPreservesExistingWithoutReading(t *testing.T) {
directory := t.TempDir()
writeSecretTestFile(t, directory, "EMPTY_WINNER", strings.Repeat("CANARY", int(secretFileByteLimit)))
writeSecretTestFile(t, directory, "VALUE_WINNER", "CANARY-CONTENT")
environment := map[string]string{"EMPTY_WINNER": "", "VALUE_WINNER": "existing"}
opened := 0
access := testSecretsAccess()
access.openFile = func(string) (*os.File, error) {
opened++
return nil, errors.New("must not open")
}
report, err := loadSecretsDir(directory, mapSecretsEnvironment(environment), access)
if err != nil {
t.Fatalf("loadSecretsDir() error = %v", err)
}
if opened != 0 {
t.Fatalf("opened skipped files %d times", opened)
}
want := []SecretFileResult{
{Name: "EMPTY_WINNER", Path: filepath.Join(canonicalTestPath(t, directory), "EMPTY_WINNER"), Status: SecretFileSkippedExisting, ExistingEmpty: true},
{Name: "VALUE_WINNER", Path: filepath.Join(canonicalTestPath(t, directory), "VALUE_WINNER"), Status: SecretFileSkippedExisting},
}
if got := report.Entries(); !reflect.DeepEqual(got, want) {
t.Fatalf("Entries() = %#v, want %#v", got, want)
}
}
func TestLoadSecretsDirValidatesAllMetadataBeforeReadingOrWriting(t *testing.T) {
for _, test := range []struct {
name string
make func(*testing.T, string)
code Code
}{
{name: "invalid name", make: func(t *testing.T, directory string) { writeSecretTestFile(t, directory, "1INVALID", "CANARY") }, code: CodeSecretsInvalidName},
{name: "directory", make: func(t *testing.T, directory string) {
if err := os.Mkdir(filepath.Join(directory, "BROKEN"), 0o700); err != nil {
t.Fatal(err)
}
}, code: CodeSecretsNotRegular},
{name: "symlink", make: func(t *testing.T, directory string) {
if runtime.GOOS == "windows" {
t.Skip("symlink setup differs on Windows")
}
writeSecretTestFile(t, directory, "TARGET", "CANARY")
if err := os.Symlink(filepath.Join(directory, "TARGET"), filepath.Join(directory, "BROKEN")); err != nil {
t.Fatal(err)
}
}, code: CodeSecretsNotRegular},
} {
t.Run(test.name, func(t *testing.T) {
directory := t.TempDir()
writeSecretTestFile(t, directory, "A_VALID", "first")
test.make(t, directory)
writes := 0
access := testSecretsAccess()
opened := 0
access.openFile = func(path string) (*os.File, error) { opened++; return os.Open(path) }
report, err := loadSecretsDir(directory, secretsEnvironment{
lookup: func(string) (string, bool) { return "", false },
set: func(string, string) error { writes++; return nil },
}, access)
requireSecretsCode(t, err, test.code)
if writes != 0 || opened != 0 {
t.Fatalf("writes = %d, opens = %d; want no preflight activity", writes, opened)
}
if report == nil || len(report.Entries()) < 2 {
t.Fatalf("report = %#v, want enumerated entries", report)
}
})
}
}
func TestLoadSecretsDirPreflightFailuresDoNotWrite(t *testing.T) {
for _, test := range []struct {
name string
body string
code Code
}{
{name: "file limit", body: strings.Repeat("x", int(secretFileByteLimit)+1), code: CodeSecretsLimitExceeded},
{name: "NUL", body: "before\x00CANARY", code: CodeSecretsInvalidValue},
} {
t.Run(test.name, func(t *testing.T) {
directory := t.TempDir()
writeSecretTestFile(t, directory, "A_FIRST", "first")
writeSecretTestFile(t, directory, "Z_FAILED", test.body)
writes := 0
report, err := loadSecretsDir(directory, secretsEnvironment{
lookup: func(string) (string, bool) { return "", false },
set: func(string, string) error { writes++; return nil },
}, testSecretsAccess())
requireSecretsCode(t, err, test.code)
if writes != 0 {
t.Fatalf("writes = %d, want zero", writes)
}
entries := report.Entries()
if entries[0].Status != SecretFileNotApplied || entries[1].Status != SecretFileFailed {
t.Fatalf("statuses = %#v", entries)
}
})
}
}
func TestLoadSecretsDirClassifiesReadFailuresWithoutLeakingThem(t *testing.T) {
for _, test := range []struct {
name string
err error
code Code
}{
{name: "permission", err: fs.ErrPermission, code: CodeSecretsPermission},
{name: "missing", err: fs.ErrNotExist, code: CodeSecretsMissing},
{name: "other", err: errors.New("READ-ERROR-CANARY"), code: CodeSecretsRead},
} {
t.Run(test.name, func(t *testing.T) {
directory := t.TempDir()
writeSecretTestFile(t, directory, "SECRET", "CREDENTIAL-CANARY")
access := testSecretsAccess()
access.openFile = func(string) (*os.File, error) { return nil, test.err }
writes := 0
report, err := loadSecretsDir(directory, secretsEnvironment{
lookup: func(string) (string, bool) { return "", false },
set: func(string, string) error { writes++; return nil },
}, access)
requireSecretsCode(t, err, test.code)
if writes != 0 || report.Entries()[0].Status != SecretFileFailed {
t.Fatalf("writes = %d, entries = %#v", writes, report.Entries())
}
if strings.Contains(fmt.Sprintf("%v %#v", err, err), "CANARY") {
t.Fatalf("error leaked underlying failure: %v %#v", err, err)
}
})
}
}
func TestLoadSecretsDirEnforcesEnumerationAndTotalBounds(t *testing.T) {
t.Run("entries", func(t *testing.T) {
directory := t.TempDir()
for index := 0; index <= secretsEntryLimit; index++ {
writeSecretTestFile(t, directory, fmt.Sprintf("SECRET_%04d", index), "")
}
report, err := loadSecretsDir(directory, mapSecretsEnvironment(map[string]string{}), testSecretsAccess())
requireSecretsCode(t, err, CodeSecretsLimitExceeded)
if len(report.Entries()) != secretsEntryLimit+1 {
t.Fatalf("entries = %d, want %d bounded enumerated entries", len(report.Entries()), secretsEntryLimit+1)
}
})
t.Run("total bytes", func(t *testing.T) {
directory := t.TempDir()
for index := 0; index < 9; index++ {
writeSecretTestFile(t, directory, fmt.Sprintf("SECRET_%02d", index), strings.Repeat("x", int(secretFileByteLimit)))
}
writes := 0
report, err := loadSecretsDir(directory, secretsEnvironment{
lookup: func(string) (string, bool) { return "", false },
set: func(string, string) error { writes++; return nil },
}, testSecretsAccess())
requireSecretsCode(t, err, CodeSecretsLimitExceeded)
if writes != 0 || report.Entries()[8].Status != SecretFileFailed {
t.Fatalf("writes = %d, final status = %q", writes, report.Entries()[8].Status)
}
})
}
func TestLoadSecretsDirApplyReportsPartialFailureAndPresenceRace(t *testing.T) {
directory := t.TempDir()
for _, name := range []string{"A_LOADED", "B_RACED", "C_FAILED", "D_WAITING"} {
writeSecretTestFile(t, directory, name, "CANARY-"+name)
}
values := map[string]string{}
lookups := map[string]int{}
environment := secretsEnvironment{
lookup: func(name string) (string, bool) {
lookups[name]++
if name == "B_RACED" && lookups[name] == 2 {
values[name] = ""
}
value, present := values[name]
return value, present
},
set: func(name, value string) error {
if name == "C_FAILED" {
return errors.New("CANARY set error")
}
values[name] = value
return nil
},
}
report, err := loadSecretsDir(directory, environment, testSecretsAccess())
requireSecretsCode(t, err, CodeSecretsApplyFailed)
entries := report.Entries()
wantStatuses := []SecretFileStatus{SecretFileLoaded, SecretFileSkippedExisting, SecretFileFailed, SecretFileNotApplied}
for index, want := range wantStatuses {
if entries[index].Status != want {
t.Fatalf("entry %d status = %q, want %q", index, entries[index].Status, want)
}
}
if !entries[1].ExistingEmpty || values["A_LOADED"] != "CANARY-A_LOADED" {
t.Fatalf("partial report/state = %#v, %#v", entries, values)
}
if _, present := values["D_WAITING"]; present {
t.Fatal("entry after apply failure was written")
}
}
func TestLoadSecretsDirFailuresAreSafeAndDetached(t *testing.T) {
directory := t.TempDir()
writeSecretTestFile(t, directory, "SECRET", "CREDENTIAL-CANARY\x00")
report, err := loadSecretsDir(directory, mapSecretsEnvironment(map[string]string{}), testSecretsAccess())
secretsErr := requireSecretsCode(t, err, CodeSecretsInvalidValue)
diagnostics := secretsErr.Diagnostics()
diagnostics[0].Message = "mutated"
if secretsErr.Diagnostics()[0].Message == "mutated" {
t.Fatal("Diagnostics() did not return a detached slice")
}
var rendered strings.Builder
if _, writeErr := report.WriteTo(&rendered); writeErr != nil {
t.Fatal(writeErr)
}
jsonReport, marshalErr := json.Marshal(report)
if marshalErr != nil {
t.Fatal(marshalErr)
}
outputs := []string{err.Error(), fmt.Sprintf("%v %#v", err, err), fmt.Sprintf("%v %#v", report, report), rendered.String(), string(jsonReport)}
for _, output := range outputs {
if strings.Contains(output, "CREDENTIAL-CANARY") {
t.Fatalf("safe output leaked content: %q", output)
}
}
if secretsErr.Diagnostics()[0].Primary != nil || secretsErr.Diagnostics()[0].Path.String() != "" {
t.Fatalf("bootstrap diagnostic fabricated source location/path: %#v", secretsErr.Diagnostics()[0])
}
}
func TestSecretsReportWriteToPropagatesWriterFailures(t *testing.T) {
report := &SecretsReport{entries: []SecretFileResult{{Name: "A\nB", Path: "/tmp/A\nB", Status: SecretFileLoaded}}}
var output strings.Builder
written, err := report.WriteTo(&output)
if err != nil || written != int64(output.Len()) || strings.Contains(output.String(), "A\nB") || !strings.Contains(output.String(), `A\nB`) {
t.Fatalf("WriteTo() = (%d, %v), output %q", written, err, output.String())
}
failure := errors.New("writer CANARY")
written, err = report.WriteTo(errorAfterWriter{limit: 5, err: failure})
if written != 5 || !errors.Is(err, failure) {
t.Fatalf("failing WriteTo() = (%d, %v)", written, err)
}
written, err = report.WriteTo(shortWriter{})
if written == 0 || !errors.Is(err, io.ErrShortWrite) {
t.Fatalf("short WriteTo() = (%d, %v)", written, err)
}
}
func TestLoadSecretsDirRejectsMissingAndInvalidDirectory(t *testing.T) {
report, err := LoadSecretsDir("")
requireSecretsCode(t, err, CodeSecretsInvalidPath)
if report == nil || len(report.Entries()) != 0 {
t.Fatalf("empty path report = %#v", report)
}
report, err = LoadSecretsDir(filepath.Join(t.TempDir(), "missing"))
requireSecretsCode(t, err, CodeSecretsMissing)
if report == nil || len(report.Entries()) != 0 {
t.Fatalf("missing path report = %#v", report)
}
file := filepath.Join(t.TempDir(), "FILE")
if err := os.WriteFile(file, nil, 0o600); err != nil {
t.Fatal(err)
}
_, err = LoadSecretsDir(file)
requireSecretsCode(t, err, CodeSecretsInvalidPath)
}
func testSecretsAccess() secretsAccess {
return secretsAccess{openDirectory: os.Open, lstat: os.Lstat, openFile: os.Open}
}
func mapSecretsEnvironment(values map[string]string) secretsEnvironment {
return secretsEnvironment{
lookup: func(name string) (string, bool) { value, present := values[name]; return value, present },
set: func(name, value string) error { values[name] = value; return nil },
}
}
func writeSecretTestFile(t *testing.T, directory, name, value string) {
t.Helper()
if err := os.WriteFile(filepath.Join(directory, name), []byte(value), 0o600); err != nil {
t.Fatalf("WriteFile(%q) error = %v", name, err)
}
}
func requireSecretsCode(t *testing.T, err error, code Code) *SecretsError {
t.Helper()
var secretsErr *SecretsError
if !errors.As(err, &secretsErr) {
t.Fatalf("error = %T %v, want *SecretsError", err, err)
}
diagnostics := secretsErr.Diagnostics()
if len(diagnostics) != 1 || diagnostics[0].Code != code {
t.Fatalf("diagnostics = %#v, want code %q", diagnostics, code)
}
return secretsErr
}
type errorAfterWriter struct {
limit int
err error
}
func (w errorAfterWriter) Write(data []byte) (int, error) {
return min(w.limit, len(data)), w.err
}