diff --git a/README.md b/README.md index 47352d7..67c23ed 100644 --- a/README.md +++ b/README.md @@ -9,8 +9,9 @@ can overlay one selected fixed profile and check an application schema version. returns detached metadata and safe diagnostics. Applications may perform audited defaulting and normalization through a resolver, then report domain errors and warnings through read-only semantic validation. Secret -tags and a path predicate drive deterministic redacted effective YAML. -Overrides, secret references/bootstrap, and value explanation are still under +tags and a path predicate drive deterministic redacted effective YAML. Explicit +secrets-directory bootstrap is available as a separate startup operation. +Overrides, typed secret references, and value explanation are still under development. ```go diff --git a/docs/config.md b/docs/config.md index 9557a77..8dd1f45 100644 --- a/docs/config.md +++ b/docs/config.md @@ -144,6 +144,41 @@ configuration values. Applications must still keep source labels, resolution reasons, and diagnostic messages value-free because ConfigKit cannot infer and sanitize secrets embedded in arbitrary application prose. +## Explicit secrets-directory bootstrap + +`LoadSecretsDir(directory)` is a separate, opt-in startup operation. It is never +called by `Load`, is not a YAML option, and does not participate in configuration +validation. Applications choose the native directory path and should invoke the +operation once before starting concurrent work. The selected path may itself be a +symlink; ConfigKit resolves it to an existing canonical directory before examining +its immediate entries. + +The bootstrap enumerates at most 1,024 entries and sorts them by filename. Every +entry must have an ASCII environment-variable name and be a regular, nonsymlink +file. Metadata for all entries is validated before file content is read or process +state is changed. Existing environment variables win, including variables whose +value is empty; their files are not opened. Each eligible file is limited to 1 MiB, +and eligible content is limited to 8 MiB in total. ConfigKit removes exactly one +terminal LF or CRLF, preserves all other whitespace, accepts an empty value, and +rejects NUL bytes. + +All eligible files are read successfully before application begins. Presence is +checked again immediately before each `os.Setenv`: a variable populated since +preflight is skipped. A write failure stops application without rollback, so prior +writes remain and later eligible entries are `not_applied`. This operation is not +atomic and does not provide compare-and-set semantics for concurrent environment +mutation. + +`SecretsReport.Entries` returns detached, filename-ordered `SecretFileResult` +records with `loaded`, `skipped_existing`, `not_applied`, or `failed` status. +`ExistingEmpty` distinguishes an empty existing winner. `SecretsReport.WriteTo` +renders only quoted names, canonical paths, statuses, and that empty-state flag; +it never retains or renders file contents. Failures return the non-nil partial +report with a `*SecretsError` containing detached `secrets.*` diagnostics and no +raw filesystem or environment error chain. Applications should retain the report +alongside the later configuration result and explicitly present blocked and failed +files when appropriate; ConfigKit stores no implicit global report. + ### Recorded application resolution An optional `Definition.Resolve` callback runs after successful typed decoding. It diff --git a/docs/internal/extraction.md b/docs/internal/extraction.md index 3eeb831..5fc2d58 100644 --- a/docs/internal/extraction.md +++ b/docs/internal/extraction.md @@ -300,6 +300,16 @@ Assigned to the explicit bootstrap operation: fixed budgets. Logging, configuration-driven implicit invocation, private-mode conventions, and application API-key plumbing remain in Narratio. +Implementation record (2026-09-06): `LoadSecretsDir`, `readSecretsEntries`, +`readSecretValue`, and the private environment/file seams in `secrets.go` adapt +Narratio's bounded-read and preserve-existing mechanisms at revision +`b68435128be1e24b94fc7ec259d6e9093669fd99`. The corresponding tests retain its +valid-file, existing-variable (including empty), missing/permission/read, +nonregular, symlink, and byte-limit cases. ConfigKit validates invalid entries +rather than skipping them, permits ordinary deployment permission modes, resolves +the selected directory symlink, performs complete preflight before mutation, and +emits per-file safe records without application configuration or logging. + ## WeatherReporter evidence ### Secrets bootstrap @@ -324,6 +334,15 @@ Assigned to the explicit bootstrap operation: entries are not read. Bootstrap is explicitly invoked and remains independent of YAML loading and semantic validation. +Implementation record (2026-09-06): the same `secrets.go` state machine adapts +WeatherReporter's `loadSecrets`/`applySecrets` separation, environment-name check, +regular-file rejection, and exact one-LF/CRLF removal at revision +`5d306a202471e10fe98923a1df27d0b0fce5c666`. Tests adapt its newline matrix, +invalid-entry and preflight-no-write cases, and controlled application failure. +The overwrite/rollback expectation is intentionally replaced by existing-winner, +presence-recheck, and partial-progress statuses; no configuration loader invokes +the bootstrap. + ### Defaults, explicit presence, and module options Assigned to consumer-shaped examples and migration checks: diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index ff1970d..7cb0b80 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1164,7 +1164,36 @@ Acceptance: ## Stage 16 — Extract explicit secrets-directory bootstrap -Status: pending. +Status: complete. + +Completion evidence (2026-09-06): + +- Added the separate, opt-in `LoadSecretsDir` startup boundary in + [`secrets.go`](../../secrets.go), with detached per-file results, stable status + values, safe `SecretsError` diagnostics, and a deterministic quoted report + renderer. `Load` remains process-state-free and has no secrets-directory option. +- Adapted WeatherReporter's read/apply separation, filename/type checks, exact + one-LF/CRLF removal, and failure cases together with Narratio's bounded reads and + existing-variable preservation. The [extraction inventory](../internal/extraction.md) + records both inspected revisions, reused symbols/cases, and the intentional + removal of overwrite, rollback, permission-mode, implicit configuration, and + logging policies. +- Implemented bounded immediate enumeration (1,024 entries), canonical directory + selection, sorted metadata validation, 1 MiB eligible-file and 8 MiB aggregate + reads, NUL rejection, full preflight before writes, and a per-write presence + recheck. Existing empty values are distinguished, skipped files are never opened, + and apply failures retain accurate partial progress without rollback. +- Added real filesystem/environment and private-seam coverage in + [`secrets_test.go`](../../secrets_test.go) for canonical ordering, selected + directory symlinks, newline/empty values, invalid names/types, all bounds, + missing/permission/read failures, no-write preflight failures, newly populated + variables, controlled apply failure, defensive copies, non-disclosure, and writer + failures. The [configuration contract](../config.md#explicit-secrets-directory-bootstrap) + documents startup-only mutation, non-atomicity, reporting blocked files, and + retaining reports alongside configuration results. Verified `go test ./...`, + `go test -race ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and + `git diff --check` with Go 1.26.7 on linux/amd64. No deviations were required; + environment-backed typed secret references remain later work. Read: [secret bootstrap contract](initial.md#secret-references-and-explicit-bootstrap), architecture responsibility boundary, testing policy, and WeatherReporter's diff --git a/records.go b/records.go index 5b04d6b..03ddedb 100644 --- a/records.go +++ b/records.go @@ -117,6 +117,16 @@ const ( CodeValidateWarning Code = "validate.warning" CodeValidateInvalidPath Code = "validate.invalid_path" CodeValidateMutation Code = "validate.mutation" + + CodeSecretsInvalidPath Code = "secrets.invalid_path" + CodeSecretsMissing Code = "secrets.missing" + CodeSecretsPermission Code = "secrets.permission" + CodeSecretsRead Code = "secrets.read" + CodeSecretsNotRegular Code = "secrets.not_regular" + CodeSecretsInvalidName Code = "secrets.invalid_name" + CodeSecretsLimitExceeded Code = "secrets.limit_exceeded" + CodeSecretsInvalidValue Code = "secrets.invalid_value" + CodeSecretsApplyFailed Code = "secrets.apply_failed" ) // Location identifies a position in a configuration source. Zero line and column diff --git a/secrets.go b/secrets.go new file mode 100644 index 0000000..43af75e --- /dev/null +++ b/secrets.go @@ -0,0 +1,371 @@ +package configkit + +import ( + "errors" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + "slices" + "strconv" + "strings" +) + +const ( + secretsEntryLimit = 1024 + secretFileByteLimit = int64(1 << 20) + secretsByteLimit = int64(8 << 20) +) + +// SecretFileStatus describes the outcome for one file considered by +// LoadSecretsDir. +type SecretFileStatus string + +const ( + SecretFileLoaded SecretFileStatus = "loaded" + SecretFileSkippedExisting SecretFileStatus = "skipped_existing" + SecretFileNotApplied SecretFileStatus = "not_applied" + SecretFileFailed SecretFileStatus = "failed" +) + +// SecretFileResult is the safe, value-free bootstrap result for one directory +// entry. ExistingEmpty is meaningful when Status is SecretFileSkippedExisting. +type SecretFileResult struct { + Name string + Path string + Status SecretFileStatus + ExistingEmpty bool +} + +// SecretsReport is a detached snapshot of an explicit secrets-directory +// bootstrap. It does not retain file contents or environment values. +type SecretsReport struct { + entries []SecretFileResult +} + +// Entries returns a copy of the report's filename-ordered entries. +func (r *SecretsReport) Entries() []SecretFileResult { + if r == nil { + return nil + } + return slices.Clone(r.entries) +} + +// WriteTo renders one safe, deterministic line per entry. Names and paths are +// quoted so control characters cannot affect terminal output. +func (r *SecretsReport) WriteTo(output io.Writer) (int64, error) { + if output == nil { + return 0, errors.New("configkit: secrets report output is nil") + } + if r == nil { + return 0, nil + } + var total int64 + for _, entry := range r.entries { + line := "status=" + strconv.Quote(string(entry.Status)) + + " name=" + strconv.Quote(entry.Name) + + " path=" + strconv.Quote(entry.Path) + + " existing_empty=" + strconv.FormatBool(entry.ExistingEmpty) + "\n" + written, err := io.WriteString(output, line) + total += int64(written) + if err != nil { + return total, err + } + if written != len(line) { + return total, io.ErrShortWrite + } + } + return total, nil +} + +// SecretsError reports safe structured diagnostics for a failed explicit +// secrets-directory bootstrap. It intentionally does not unwrap filesystem or +// environment errors. +type SecretsError struct { + diagnostics []Diagnostic +} + +// Error implements error without rendering filenames, paths, or file contents. +func (e *SecretsError) Error() string { + if e == nil { + return "configkit: secrets bootstrap failed" + } + count := 0 + for _, diagnostic := range e.diagnostics { + if diagnostic.Severity == SeverityError { + count++ + } + } + if count == 1 { + return "configkit: secrets bootstrap failed with 1 error" + } + return fmt.Sprintf("configkit: secrets bootstrap failed with %d errors", count) +} + +// Diagnostics returns a deep copy of the bootstrap diagnostics. +func (e *SecretsError) Diagnostics() []Diagnostic { + if e == nil { + return nil + } + return cloneDiagnostics(e.diagnostics) +} + +type secretsEnvironment struct { + lookup func(string) (string, bool) + set func(string, string) error +} + +type secretsAccess struct { + openDirectory func(string) (*os.File, error) + lstat func(string) (fs.FileInfo, error) + openFile func(string) (*os.File, error) +} + +type stagedSecret struct { + entry int + value string +} + +// LoadSecretsDir explicitly loads immediate regular files from directory into +// previously unset process environment variables. Applications should call it +// once during startup, before concurrent work; Load never calls it. +func LoadSecretsDir(directory string) (*SecretsReport, error) { + return loadSecretsDir(directory, secretsEnvironment{lookup: os.LookupEnv, set: os.Setenv}, secretsAccess{ + openDirectory: os.Open, + lstat: os.Lstat, + openFile: os.Open, + }) +} + +func loadSecretsDir(directory string, environment secretsEnvironment, access secretsAccess) (*SecretsReport, error) { + report := &SecretsReport{} + if strings.TrimSpace(directory) == "" { + return report, secretsFailure(CodeSecretsInvalidPath, "secrets directory path is required") + } + + canonical, err := canonicalExistingPath(directory) + if err != nil { + return report, secretsFailure(classifySecretsFilesystemError(err), "secrets directory could not be canonicalized: "+strconv.Quote(cleanAbsolutePath(directory))) + } + info, err := os.Stat(canonical) + if err != nil { + return report, secretsFailure(classifySecretsFilesystemError(err), "secrets directory could not be inspected: "+strconv.Quote(canonical)) + } + if !info.IsDir() { + return report, secretsFailure(CodeSecretsInvalidPath, "secrets directory is not a directory: "+strconv.Quote(canonical)) + } + + directoryFile, err := access.openDirectory(canonical) + if err != nil { + return report, secretsFailure(classifySecretsFilesystemError(err), "secrets directory could not be opened: "+strconv.Quote(canonical)) + } + entries, readErr := readSecretsEntries(directoryFile) + closeErr := directoryFile.Close() + if readErr != nil { + return report, secretsFailure(classifySecretsFilesystemError(readErr), "secrets directory could not be enumerated: "+strconv.Quote(canonical)) + } + if closeErr != nil { + return report, secretsFailure(CodeSecretsRead, "secrets directory could not be closed: "+strconv.Quote(canonical)) + } + slices.SortFunc(entries, func(left, right fs.DirEntry) int { return strings.Compare(left.Name(), right.Name()) }) + report.entries = make([]SecretFileResult, len(entries)) + for index, entry := range entries { + name := entry.Name() + report.entries[index] = SecretFileResult{Name: name, Path: filepath.Join(canonical, name), Status: SecretFileNotApplied} + } + if len(entries) > secretsEntryLimit { + return report, secretsFailure(CodeSecretsLimitExceeded, "secrets directory exceeds the 1024-entry limit: "+strconv.Quote(canonical)) + } + + for index := range report.entries { + entry := &report.entries[index] + if !validEnvironmentName(entry.Name) { + entry.Status = SecretFileFailed + return report, secretsFailure(CodeSecretsInvalidName, "secret filename is not a valid environment name: "+strconv.Quote(entry.Path)) + } + info, err := access.lstat(entry.Path) + if err != nil { + entry.Status = SecretFileFailed + return report, secretsFailure(classifySecretsFilesystemError(err), "secret file could not be inspected: "+strconv.Quote(entry.Path)) + } + if info.Mode()&fs.ModeSymlink != 0 || !info.Mode().IsRegular() { + entry.Status = SecretFileFailed + return report, secretsFailure(CodeSecretsNotRegular, "secret entry is not a regular file: "+strconv.Quote(entry.Path)) + } + } + + staged := make([]stagedSecret, 0, len(report.entries)) + var consumed int64 + for index := range report.entries { + entry := &report.entries[index] + if existing, present := environment.lookup(entry.Name); present { + entry.Status = SecretFileSkippedExisting + entry.ExistingEmpty = existing == "" + continue + } + remaining := secretsByteLimit - consumed + value, used, err := readSecretValue(entry.Path, remaining, access) + if err != nil { + entry.Status = SecretFileFailed + return report, secretsFailure(secretReadCode(err), secretReadMessage(err)+": "+strconv.Quote(entry.Path)) + } + consumed += used + staged = append(staged, stagedSecret{entry: index, value: value}) + } + + for _, secret := range staged { + entry := &report.entries[secret.entry] + if existing, present := environment.lookup(entry.Name); present { + entry.Status = SecretFileSkippedExisting + entry.ExistingEmpty = existing == "" + continue + } + if err := environment.set(entry.Name, secret.value); err != nil { + entry.Status = SecretFileFailed + return report, secretsFailure(CodeSecretsApplyFailed, "secret environment variable could not be set for file: "+strconv.Quote(entry.Path)) + } + entry.Status = SecretFileLoaded + } + return report, nil +} + +func readSecretsEntries(directory *os.File) ([]fs.DirEntry, error) { + entries := make([]fs.DirEntry, 0) + for len(entries) <= secretsEntryLimit { + batch, err := directory.ReadDir(secretsEntryLimit + 1 - len(entries)) + entries = append(entries, batch...) + if err == io.EOF { + return entries, nil + } + if err != nil { + return entries, err + } + if len(entries) > secretsEntryLimit { + return entries, nil + } + } + return entries, nil +} + +type secretReadFailure int + +const ( + secretReadFilesystem secretReadFailure = iota + secretReadFileLimit + secretReadTotalLimit + secretReadNUL +) + +type secretReadError struct { + kind secretReadFailure + err error +} + +func (e *secretReadError) Error() string { return "secret read failed" } + +func readSecretValue(path string, remaining int64, access secretsAccess) (string, int64, error) { + if remaining < 0 { + return "", 0, &secretReadError{kind: secretReadTotalLimit} + } + limit := min(secretFileByteLimit, remaining) + file, err := access.openFile(path) + if err != nil { + return "", 0, &secretReadError{kind: secretReadFilesystem, err: err} + } + data, readErr := io.ReadAll(io.LimitReader(file, limit+1)) + closeErr := file.Close() + if readErr != nil { + return "", 0, &secretReadError{kind: secretReadFilesystem, err: readErr} + } + if closeErr != nil { + return "", 0, &secretReadError{kind: secretReadFilesystem, err: closeErr} + } + if int64(len(data)) > limit { + kind := secretReadFileLimit + if remaining < secretFileByteLimit { + kind = secretReadTotalLimit + } + return "", 0, &secretReadError{kind: kind} + } + if slices.Contains(data, byte(0)) { + return "", 0, &secretReadError{kind: secretReadNUL} + } + value := string(data) + if strings.HasSuffix(value, "\r\n") { + value = strings.TrimSuffix(value, "\r\n") + } else { + value = strings.TrimSuffix(value, "\n") + } + return value, int64(len(data)), nil +} + +func validEnvironmentName(name string) bool { + if name == "" || !asciiEnvironmentFirst(name[0]) { + return false + } + for index := 1; index < len(name); index++ { + character := name[index] + if !asciiEnvironmentFirst(character) && (character < '0' || character > '9') { + return false + } + } + return true +} + +func asciiEnvironmentFirst(character byte) bool { + return character == '_' || character >= 'A' && character <= 'Z' || character >= 'a' && character <= 'z' +} + +func cleanAbsolutePath(path string) string { + absolute, err := filepath.Abs(path) + if err != nil { + return filepath.Clean(path) + } + return filepath.Clean(absolute) +} + +func classifySecretsFilesystemError(err error) Code { + switch { + case errors.Is(err, fs.ErrNotExist): + return CodeSecretsMissing + case errors.Is(err, fs.ErrPermission): + return CodeSecretsPermission + default: + return CodeSecretsRead + } +} + +func secretReadCode(err error) Code { + var failure *secretReadError + if errors.As(err, &failure) { + switch failure.kind { + case secretReadFileLimit, secretReadTotalLimit: + return CodeSecretsLimitExceeded + case secretReadNUL: + return CodeSecretsInvalidValue + case secretReadFilesystem: + return classifySecretsFilesystemError(failure.err) + } + } + return CodeSecretsRead +} + +func secretReadMessage(err error) string { + var failure *secretReadError + if errors.As(err, &failure) { + switch failure.kind { + case secretReadFileLimit: + return "secret file exceeds the 1 MiB limit" + case secretReadTotalLimit: + return "eligible secret files exceed the 8 MiB total limit" + case secretReadNUL: + return "secret file contains a NUL byte" + } + } + return "secret file could not be read" +} + +func secretsFailure(code Code, message string) *SecretsError { + return &SecretsError{diagnostics: []Diagnostic{{Code: code, Severity: SeverityError, Message: message}}} +} diff --git a/secrets_test.go b/secrets_test.go new file mode 100644 index 0000000..4750266 --- /dev/null +++ b/secrets_test.go @@ -0,0 +1,377 @@ +package configkit + +import ( + "encoding/json" + "errors" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + "reflect" + "runtime" + "strings" + "testing" +) + +func TestLoadSecretsDirLoadsCanonicalSortedValues(t *testing.T) { + directory := t.TempDir() + writeSecretTestFile(t, directory, "CONFIGKIT_SECRET_Z", "last\n\n") + writeSecretTestFile(t, directory, "CONFIGKIT_SECRET_A", "first\r\n") + writeSecretTestFile(t, directory, "CONFIGKIT_SECRET_EMPTY", "") + + selected := directory + if runtime.GOOS != "windows" { + selected = filepath.Join(t.TempDir(), "selected") + if err := os.Symlink(directory, selected); err != nil { + t.Fatalf("Symlink() error = %v", err) + } + } + for _, name := range []string{"CONFIGKIT_SECRET_A", "CONFIGKIT_SECRET_EMPTY", "CONFIGKIT_SECRET_Z"} { + t.Setenv(name, "temporary") + if err := os.Unsetenv(name); err != nil { + t.Fatalf("Unsetenv(%q) error = %v", name, err) + } + } + + report, err := LoadSecretsDir(selected) + if err != nil { + t.Fatalf("LoadSecretsDir() error = %v", err) + } + if got := os.Getenv("CONFIGKIT_SECRET_A"); got != "first" { + t.Fatalf("LF value = %q, want first", got) + } + if got := os.Getenv("CONFIGKIT_SECRET_Z"); got != "last\n" { + t.Fatalf("two-LF value = %q, want one retained LF", got) + } + if value, present := os.LookupEnv("CONFIGKIT_SECRET_EMPTY"); !present || value != "" { + t.Fatalf("empty value = %q, %v; want present empty", value, present) + } + + canonical := canonicalTestPath(t, directory) + want := []SecretFileResult{ + {Name: "CONFIGKIT_SECRET_A", Path: filepath.Join(canonical, "CONFIGKIT_SECRET_A"), Status: SecretFileLoaded}, + {Name: "CONFIGKIT_SECRET_EMPTY", Path: filepath.Join(canonical, "CONFIGKIT_SECRET_EMPTY"), Status: SecretFileLoaded}, + {Name: "CONFIGKIT_SECRET_Z", Path: filepath.Join(canonical, "CONFIGKIT_SECRET_Z"), Status: SecretFileLoaded}, + } + if got := report.Entries(); !reflect.DeepEqual(got, want) { + t.Fatalf("Entries() = %#v, want %#v", got, want) + } + entries := report.Entries() + entries[0].Name = "mutated" + if report.Entries()[0].Name != want[0].Name { + t.Fatal("Entries() did not return a detached slice") + } +} + +func TestLoadSecretsDirPreservesExistingWithoutReading(t *testing.T) { + directory := t.TempDir() + writeSecretTestFile(t, directory, "EMPTY_WINNER", strings.Repeat("CANARY", int(secretFileByteLimit))) + writeSecretTestFile(t, directory, "VALUE_WINNER", "CANARY-CONTENT") + environment := map[string]string{"EMPTY_WINNER": "", "VALUE_WINNER": "existing"} + opened := 0 + access := testSecretsAccess() + access.openFile = func(string) (*os.File, error) { + opened++ + return nil, errors.New("must not open") + } + + report, err := loadSecretsDir(directory, mapSecretsEnvironment(environment), access) + if err != nil { + t.Fatalf("loadSecretsDir() error = %v", err) + } + if opened != 0 { + t.Fatalf("opened skipped files %d times", opened) + } + want := []SecretFileResult{ + {Name: "EMPTY_WINNER", Path: filepath.Join(canonicalTestPath(t, directory), "EMPTY_WINNER"), Status: SecretFileSkippedExisting, ExistingEmpty: true}, + {Name: "VALUE_WINNER", Path: filepath.Join(canonicalTestPath(t, directory), "VALUE_WINNER"), Status: SecretFileSkippedExisting}, + } + if got := report.Entries(); !reflect.DeepEqual(got, want) { + t.Fatalf("Entries() = %#v, want %#v", got, want) + } +} + +func TestLoadSecretsDirValidatesAllMetadataBeforeReadingOrWriting(t *testing.T) { + for _, test := range []struct { + name string + make func(*testing.T, string) + code Code + }{ + {name: "invalid name", make: func(t *testing.T, directory string) { writeSecretTestFile(t, directory, "1INVALID", "CANARY") }, code: CodeSecretsInvalidName}, + {name: "directory", make: func(t *testing.T, directory string) { + if err := os.Mkdir(filepath.Join(directory, "BROKEN"), 0o700); err != nil { + t.Fatal(err) + } + }, code: CodeSecretsNotRegular}, + {name: "symlink", make: func(t *testing.T, directory string) { + if runtime.GOOS == "windows" { + t.Skip("symlink setup differs on Windows") + } + writeSecretTestFile(t, directory, "TARGET", "CANARY") + if err := os.Symlink(filepath.Join(directory, "TARGET"), filepath.Join(directory, "BROKEN")); err != nil { + t.Fatal(err) + } + }, code: CodeSecretsNotRegular}, + } { + t.Run(test.name, func(t *testing.T) { + directory := t.TempDir() + writeSecretTestFile(t, directory, "A_VALID", "first") + test.make(t, directory) + writes := 0 + access := testSecretsAccess() + opened := 0 + access.openFile = func(path string) (*os.File, error) { opened++; return os.Open(path) } + report, err := loadSecretsDir(directory, secretsEnvironment{ + lookup: func(string) (string, bool) { return "", false }, + set: func(string, string) error { writes++; return nil }, + }, access) + requireSecretsCode(t, err, test.code) + if writes != 0 || opened != 0 { + t.Fatalf("writes = %d, opens = %d; want no preflight activity", writes, opened) + } + if report == nil || len(report.Entries()) < 2 { + t.Fatalf("report = %#v, want enumerated entries", report) + } + }) + } +} + +func TestLoadSecretsDirPreflightFailuresDoNotWrite(t *testing.T) { + for _, test := range []struct { + name string + body string + code Code + }{ + {name: "file limit", body: strings.Repeat("x", int(secretFileByteLimit)+1), code: CodeSecretsLimitExceeded}, + {name: "NUL", body: "before\x00CANARY", code: CodeSecretsInvalidValue}, + } { + t.Run(test.name, func(t *testing.T) { + directory := t.TempDir() + writeSecretTestFile(t, directory, "A_FIRST", "first") + writeSecretTestFile(t, directory, "Z_FAILED", test.body) + writes := 0 + report, err := loadSecretsDir(directory, secretsEnvironment{ + lookup: func(string) (string, bool) { return "", false }, + set: func(string, string) error { writes++; return nil }, + }, testSecretsAccess()) + requireSecretsCode(t, err, test.code) + if writes != 0 { + t.Fatalf("writes = %d, want zero", writes) + } + entries := report.Entries() + if entries[0].Status != SecretFileNotApplied || entries[1].Status != SecretFileFailed { + t.Fatalf("statuses = %#v", entries) + } + }) + } +} + +func TestLoadSecretsDirClassifiesReadFailuresWithoutLeakingThem(t *testing.T) { + for _, test := range []struct { + name string + err error + code Code + }{ + {name: "permission", err: fs.ErrPermission, code: CodeSecretsPermission}, + {name: "missing", err: fs.ErrNotExist, code: CodeSecretsMissing}, + {name: "other", err: errors.New("READ-ERROR-CANARY"), code: CodeSecretsRead}, + } { + t.Run(test.name, func(t *testing.T) { + directory := t.TempDir() + writeSecretTestFile(t, directory, "SECRET", "CREDENTIAL-CANARY") + access := testSecretsAccess() + access.openFile = func(string) (*os.File, error) { return nil, test.err } + writes := 0 + report, err := loadSecretsDir(directory, secretsEnvironment{ + lookup: func(string) (string, bool) { return "", false }, + set: func(string, string) error { writes++; return nil }, + }, access) + requireSecretsCode(t, err, test.code) + if writes != 0 || report.Entries()[0].Status != SecretFileFailed { + t.Fatalf("writes = %d, entries = %#v", writes, report.Entries()) + } + if strings.Contains(fmt.Sprintf("%v %#v", err, err), "CANARY") { + t.Fatalf("error leaked underlying failure: %v %#v", err, err) + } + }) + } +} + +func TestLoadSecretsDirEnforcesEnumerationAndTotalBounds(t *testing.T) { + t.Run("entries", func(t *testing.T) { + directory := t.TempDir() + for index := 0; index <= secretsEntryLimit; index++ { + writeSecretTestFile(t, directory, fmt.Sprintf("SECRET_%04d", index), "") + } + report, err := loadSecretsDir(directory, mapSecretsEnvironment(map[string]string{}), testSecretsAccess()) + requireSecretsCode(t, err, CodeSecretsLimitExceeded) + if len(report.Entries()) != secretsEntryLimit+1 { + t.Fatalf("entries = %d, want %d bounded enumerated entries", len(report.Entries()), secretsEntryLimit+1) + } + }) + + t.Run("total bytes", func(t *testing.T) { + directory := t.TempDir() + for index := 0; index < 9; index++ { + writeSecretTestFile(t, directory, fmt.Sprintf("SECRET_%02d", index), strings.Repeat("x", int(secretFileByteLimit))) + } + writes := 0 + report, err := loadSecretsDir(directory, secretsEnvironment{ + lookup: func(string) (string, bool) { return "", false }, + set: func(string, string) error { writes++; return nil }, + }, testSecretsAccess()) + requireSecretsCode(t, err, CodeSecretsLimitExceeded) + if writes != 0 || report.Entries()[8].Status != SecretFileFailed { + t.Fatalf("writes = %d, final status = %q", writes, report.Entries()[8].Status) + } + }) +} + +func TestLoadSecretsDirApplyReportsPartialFailureAndPresenceRace(t *testing.T) { + directory := t.TempDir() + for _, name := range []string{"A_LOADED", "B_RACED", "C_FAILED", "D_WAITING"} { + writeSecretTestFile(t, directory, name, "CANARY-"+name) + } + values := map[string]string{} + lookups := map[string]int{} + environment := secretsEnvironment{ + lookup: func(name string) (string, bool) { + lookups[name]++ + if name == "B_RACED" && lookups[name] == 2 { + values[name] = "" + } + value, present := values[name] + return value, present + }, + set: func(name, value string) error { + if name == "C_FAILED" { + return errors.New("CANARY set error") + } + values[name] = value + return nil + }, + } + + report, err := loadSecretsDir(directory, environment, testSecretsAccess()) + requireSecretsCode(t, err, CodeSecretsApplyFailed) + entries := report.Entries() + wantStatuses := []SecretFileStatus{SecretFileLoaded, SecretFileSkippedExisting, SecretFileFailed, SecretFileNotApplied} + for index, want := range wantStatuses { + if entries[index].Status != want { + t.Fatalf("entry %d status = %q, want %q", index, entries[index].Status, want) + } + } + if !entries[1].ExistingEmpty || values["A_LOADED"] != "CANARY-A_LOADED" { + t.Fatalf("partial report/state = %#v, %#v", entries, values) + } + if _, present := values["D_WAITING"]; present { + t.Fatal("entry after apply failure was written") + } +} + +func TestLoadSecretsDirFailuresAreSafeAndDetached(t *testing.T) { + directory := t.TempDir() + writeSecretTestFile(t, directory, "SECRET", "CREDENTIAL-CANARY\x00") + report, err := loadSecretsDir(directory, mapSecretsEnvironment(map[string]string{}), testSecretsAccess()) + secretsErr := requireSecretsCode(t, err, CodeSecretsInvalidValue) + diagnostics := secretsErr.Diagnostics() + diagnostics[0].Message = "mutated" + if secretsErr.Diagnostics()[0].Message == "mutated" { + t.Fatal("Diagnostics() did not return a detached slice") + } + var rendered strings.Builder + if _, writeErr := report.WriteTo(&rendered); writeErr != nil { + t.Fatal(writeErr) + } + jsonReport, marshalErr := json.Marshal(report) + if marshalErr != nil { + t.Fatal(marshalErr) + } + outputs := []string{err.Error(), fmt.Sprintf("%v %#v", err, err), fmt.Sprintf("%v %#v", report, report), rendered.String(), string(jsonReport)} + for _, output := range outputs { + if strings.Contains(output, "CREDENTIAL-CANARY") { + t.Fatalf("safe output leaked content: %q", output) + } + } + if secretsErr.Diagnostics()[0].Primary != nil || secretsErr.Diagnostics()[0].Path.String() != "" { + t.Fatalf("bootstrap diagnostic fabricated source location/path: %#v", secretsErr.Diagnostics()[0]) + } +} + +func TestSecretsReportWriteToPropagatesWriterFailures(t *testing.T) { + report := &SecretsReport{entries: []SecretFileResult{{Name: "A\nB", Path: "/tmp/A\nB", Status: SecretFileLoaded}}} + var output strings.Builder + written, err := report.WriteTo(&output) + if err != nil || written != int64(output.Len()) || strings.Contains(output.String(), "A\nB") || !strings.Contains(output.String(), `A\nB`) { + t.Fatalf("WriteTo() = (%d, %v), output %q", written, err, output.String()) + } + failure := errors.New("writer CANARY") + written, err = report.WriteTo(errorAfterWriter{limit: 5, err: failure}) + if written != 5 || !errors.Is(err, failure) { + t.Fatalf("failing WriteTo() = (%d, %v)", written, err) + } + written, err = report.WriteTo(shortWriter{}) + if written == 0 || !errors.Is(err, io.ErrShortWrite) { + t.Fatalf("short WriteTo() = (%d, %v)", written, err) + } +} + +func TestLoadSecretsDirRejectsMissingAndInvalidDirectory(t *testing.T) { + report, err := LoadSecretsDir("") + requireSecretsCode(t, err, CodeSecretsInvalidPath) + if report == nil || len(report.Entries()) != 0 { + t.Fatalf("empty path report = %#v", report) + } + report, err = LoadSecretsDir(filepath.Join(t.TempDir(), "missing")) + requireSecretsCode(t, err, CodeSecretsMissing) + if report == nil || len(report.Entries()) != 0 { + t.Fatalf("missing path report = %#v", report) + } + file := filepath.Join(t.TempDir(), "FILE") + if err := os.WriteFile(file, nil, 0o600); err != nil { + t.Fatal(err) + } + _, err = LoadSecretsDir(file) + requireSecretsCode(t, err, CodeSecretsInvalidPath) +} + +func testSecretsAccess() secretsAccess { + return secretsAccess{openDirectory: os.Open, lstat: os.Lstat, openFile: os.Open} +} + +func mapSecretsEnvironment(values map[string]string) secretsEnvironment { + return secretsEnvironment{ + lookup: func(name string) (string, bool) { value, present := values[name]; return value, present }, + set: func(name, value string) error { values[name] = value; return nil }, + } +} + +func writeSecretTestFile(t *testing.T, directory, name, value string) { + t.Helper() + if err := os.WriteFile(filepath.Join(directory, name), []byte(value), 0o600); err != nil { + t.Fatalf("WriteFile(%q) error = %v", name, err) + } +} + +func requireSecretsCode(t *testing.T, err error, code Code) *SecretsError { + t.Helper() + var secretsErr *SecretsError + if !errors.As(err, &secretsErr) { + t.Fatalf("error = %T %v, want *SecretsError", err, err) + } + diagnostics := secretsErr.Diagnostics() + if len(diagnostics) != 1 || diagnostics[0].Code != code { + t.Fatalf("diagnostics = %#v, want code %q", diagnostics, code) + } + return secretsErr +} + +type errorAfterWriter struct { + limit int + err error +} + +func (w errorAfterWriter) Write(data []byte) (int, error) { + return min(w.limit, len(data)), w.err +}