Complete bounded glob imports

This commit is contained in:
2026-09-06 20:51:21 +00:00
parent 9f04c09be3
commit 71a73bcfd8
15 changed files with 679 additions and 82 deletions

View File

@@ -3,12 +3,13 @@
An opinionated Go library for applications that use YAML configuration files. An opinionated Go library for applications that use YAML configuration files.
ConfigKit is in early implementation. Its current loader accepts one strict YAML ConfigKit is in early implementation. Its current loader accepts one strict YAML
source (or explicit defaults-only input), optionally traverses explicit path imports, source (or explicit defaults-only input), optionally traverses explicit path and glob
strictly composes their disjoint definitions, applies typed defaults, and can check imports, strictly composes their disjoint definitions, applies typed defaults, and
an application schema version. It returns detached metadata and safe diagnostics. can check an application schema version. It returns detached metadata and safe
diagnostics.
Applications may perform audited defaulting and normalization through a resolver, Applications may perform audited defaulting and normalization through a resolver,
then report domain errors and warnings through read-only semantic validation. then report domain errors and warnings through read-only semantic validation.
Glob imports, profiles, overrides, secrets, and rendering are still under development. Profiles, overrides, secrets, and rendering are still under development.
```go ```go
type Config struct { type Config struct {

View File

@@ -22,9 +22,10 @@ File sources can observe file changes between consumptions; byte sources cannot.
Each source is limited to 8 MiB. An imported load is limited to 64 MiB across all Each source is limited to 8 MiB. An imported load is limited to 64 MiB across all
accepted sources, 1,024 accepted sources, 64 active import edges, and 1,000,000 accepted sources, 1,024 accepted sources, 64 active import edges, and 1,000,000
input nodes in aggregate. Reads request at most the remaining allowance plus one byte input nodes in aggregate. Glob expansion separately permits 1,000,000 visited
so excess data is rejected rather than silently truncated. Data returned together directory entries, including nonmatches. Reads request at most the remaining
with a read error is not accepted as partial input. allowance plus one byte so excess data is rejected rather than silently truncated.
Data returned together with a read error is not accepted as partial input.
## Single-source loading ## Single-source loading
@@ -50,10 +51,10 @@ contributes no source, and continues through defaults, decode, and projection. A
missing file remains an error; it is never interpreted as no input. missing file remains an error; it is never interpreted as no input.
The current `Options` boundary accepts a defaults factory, `VersionRule`, resolver, The current `Options` boundary accepts a defaults factory, `VersionRule`, resolver,
semantic validator, explicit path imports, and import filesystem context. Active semantic validator, explicit path and glob imports, and filesystem context. Active
profiles, standalone filesystem context, overrides, and dynamic secret classification profiles, overrides, and dynamic secret classification return `options.unavailable`
return `options.unavailable` because their lifecycle stages are not implemented. because their lifecycle stages are not implemented. The reserved profile spelling
The reserved profile spelling `default` selects no active profile. `NoSource` `default` selects no active profile. `NoSource`
additionally rejects imports, active profiles, and filesystem context as invalid additionally rejects imports, active profiles, and filesystem context as invalid
combinations. Input containing a reserved `_configkit` mapping key is rejected while combinations. Input containing a reserved `_configkit` mapping key is rejected while
imports are disabled. imports are disabled.
@@ -180,16 +181,14 @@ traversal order. Traversal order never authorizes replacement. Composition clone
the resulting tree and its ordered claim records rather than mutating or aliasing an the resulting tree and its ordered claim records rather than mutating or aliasing an
input document. input document.
## Explicit path imports ## Explicit imports
Imports are opt-in. With `Options.Imports` enabled, a document may reserve its root Imports are opt-in. With `Options.Imports` enabled, a document may reserve its root
`_configkit` mapping for an `imports` sequence. ConfigKit removes that mapping before `_configkit` mapping for an `imports` sequence. ConfigKit removes that mapping before
application decoding. Each sequence item is a mapping with exactly one nonempty application decoding. Each sequence item is a mapping with exactly one nonempty
string `path` or `glob` plus an optional boolean `optional`; all other shapes and string `path` or `glob` plus an optional boolean `optional`; all other shapes and
fields are `import.invalid`. `glob` is recognized but temporarily returns fields are `import.invalid`. A `_configkit` key below the root is invalid. With
`import.unsupported` until bounded glob expansion is implemented. A `_configkit` imports disabled, any occurrence remains `import.disabled`.
key below the root is invalid. With imports disabled, any occurrence remains
`import.disabled`.
Path declarations use slash-separated relative paths. Absolute paths and network Path declarations use slash-separated relative paths. Absolute paths and network
URLs are invalid. Each path resolves from its declaring file's canonical directory, URLs are invalid. Each path resolves from its declaring file's canonical directory,
@@ -199,16 +198,29 @@ permits replacement. Required missing paths fail; `optional: true` suppresses on
target nonexistence, not invalid paths, containment failures, directories, target nonexistence, not invalid paths, containment failures, directories,
permissions, parse errors, or composition errors. permissions, parse errors, or composition errors.
Glob declarations use the slash-based Go `path.Match` grammar: `*`, `?`, character
classes, and backslash escapes are supported, while recursive `**` syntax is invalid.
Backslashes escape pattern characters rather than acting as directory separators.
ConfigKit validates the complete pattern before filesystem traversal, walks only
directories selected by preceding segments, and reads wildcard directories in
bounded batches. Every examined entry consumes the separate visit budget, including
nonmatches. Matches are sorted lexicographically by slash-relative spelling within
each declaration before depth-first source traversal. A required zero-match glob is
`import.missing`; `optional: true` suppresses only zero matches. Invalid patterns,
walk failures, matched directories, invalid documents, escapes, and repeated targets
remain errors. Final candidates are bounded by the remaining source limit before
their contents are accumulated.
The configuration root defaults to a selected file's canonical directory. A file The configuration root defaults to a selected file's canonical directory. A file
source may supply `FilesystemContext.Root` to widen that root but may not supply source may supply `FilesystemContext.Root` to widen that root but may not supply
`BaseDir`. Byte and reader sources must supply both `BaseDir` and `Root` as absolute `BaseDir`. Whenever byte and reader sources supply filesystem context, they must
existing directories when imports are enabled; their diagnostic labels, including provide both `BaseDir` and `Root` as absolute existing directories; their diagnostic
URI-looking labels, never imply filesystem paths. The base and selected file must be labels, including URI-looking labels, never imply filesystem paths. The base and
inside the canonical root. ConfigKit canonicalizes symlinks, checks containment with selected file must be inside the canonical root. ConfigKit canonicalizes symlinks,
relative paths, and opens the canonical root-relative target through a library-owned checks containment with relative paths, and opens the canonical root-relative target
root handle. Contained relative or absolute symlink targets are accepted; escapes through a library-owned root handle. Contained relative or absolute symlink targets
are rejected. Canonical paths define identity, so hard links at distinct paths remain are accepted; escapes are rejected. Canonical paths define identity, so hard links at
distinct sources. distinct paths remain distinct sources.
An active-stack repeat is `import.cycle` and reports the complete cycle's import An active-stack repeat is `import.cycle` and reports the complete cycle's import
sites. A previously visited canonical path is `import.repeated` and reports both sites. A previously visited canonical path is `import.repeated` and reports both

View File

@@ -2,7 +2,7 @@
This is the contributor entry point for ConfigKit. Read the task-specific owners This is the contributor entry point for ConfigKit. Read the task-specific owners
below before making changes. The repository is in early implementation: its public below before making changes. The repository is in early implementation: its public
loader currently covers strict selected sources, explicit path-import graphs, loader currently covers strict selected sources, explicit path/glob import graphs,
composition, typed defaults, schema-version checks, recorded application resolution, composition, typed defaults, schema-version checks, recorded application resolution,
semantic validation, and metadata. The semantic validation, and metadata. The
[component overview](internal/overview.md) owns the implementation inventory. [component overview](internal/overview.md) owns the implementation inventory.

View File

@@ -124,6 +124,16 @@ fragment version-placement enforcement. The planned glob syntax is parsed but
remains explicitly unavailable for its next owning work; hard links intentionally remains explicitly unavailable for its next owning work; hard links intentionally
replace Narratio's same-file rejection with path identity. replace Narratio's same-file rejection with path identity.
Glob-import implementation record (2026-09-06): ConfigKit retains the extracted
declaration order, import failures, composition, and identity cases while adding the
new `expandImportGlob`, `readGlobMatches`, and pattern-validation mechanisms in
`glob.go`. Focused cases cover lexical ordering, multiple declarations, overlap,
directories, hard links, contained and escaping symlinks, unmatched declarations,
malformed and recursive patterns, escaped metacharacters, character classes,
permission failures, explicit `conf.d` use, and entry/candidate budgets. This work
has no Narratio production equivalent; it uses the planned standard-library
`path.Match` and bounded `File.ReadDir` adaptation rather than its flat loader.
### Profiles and overlay ### Profiles and overlay
Assigned to selected-profile overlay: Assigned to selected-profile overlay:

View File

@@ -20,8 +20,9 @@ module and package documentation boundary. Its implemented public primitives are
- strict schema-driven decoding over detached defaults in `decode.go`, including - strict schema-driven decoding over detached defaults in `decode.go`, including
mapping/sequence semantics, null and numeric policy, safe codec errors, presence, mapping/sequence semantics, null and numeric policy, safe codec errors, presence,
baseline/final trees, and initial default/input origins; and baseline/final trees, and initial default/input origins; and
- public lifecycle coordination in `load.go`, bounded canonical path-import DFS in - public lifecycle coordination in `load.go`, bounded canonical import DFS in
`imports.go`, version-report contexts in `validation.go`, and detached `imports.go`, pattern-directed glob expansion in `glob.go`, version-report contexts
in `validation.go`, and detached
base/default metadata assembly in `load_metadata.go`; and base/default metadata assembly in `load_metadata.go`; and
- recorded application defaulting and normalization in `resolution.go`, including - recorded application defaulting and normalization in `resolution.go`, including
explicitness queries, helper verification, structural mutation audit, coalesced explicitness queries, helper verification, structural mutation audit, coalesced

View File

@@ -878,7 +878,7 @@ Completion evidence (2026-09-06):
tests cover nested/list-order traversal, composition conflicts, version placement, tests cover nested/list-order traversal, composition conflicts, version placement,
directives, optional/required paths, explicit roots, cycles, diamonds, symlinks, directives, optional/required paths, explicit roots, cycles, diamonds, symlinks,
hard links, parse/decode attribution, file types, context rules, and reduced limit hard links, parse/decode attribution, file types, context rules, and reduced limit
boundaries. The [current path-import contract](../config.md#explicit-path-imports) boundaries. The [current path-import contract](../config.md#explicit-imports)
owns implemented behavior; glob expansion remains pending. Verified `go test ./...`, owns implemented behavior; glob expansion remains pending. Verified `go test ./...`,
`go test -race ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and `go test -race ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and
`git diff --check` with Go 1.26.7 on linux/amd64. `git diff --check` with Go 1.26.7 on linux/amd64.
@@ -917,7 +917,36 @@ Acceptance:
## Stage 12 — Complete glob imports and filesystem option handling ## Stage 12 — Complete glob imports and filesystem option handling
Status: pending. Status: complete.
Completion evidence (2026-09-06):
- Removed the temporary glob gate and added prevalidated Go `path.Match` semantics
for `*`, `?`, character classes, and escapes, with recursive unescaped `**`, URLs,
absolute paths, empty segments, and malformed classes rejected before any walk.
- Added a pattern-directed walker that resolves literal segments directly and reads
wildcard directories through positive bounded `File.ReadDir` batches. It charges
every examined entry, including nonmatches, canonicalizes and contains traversed
directories and final targets, sorts slash-relative matches, and bounds final
candidates against remaining source capacity before traversal.
- Reused the path-import DFS for every sorted match, preserving declaration ordering,
nested depth-first imports, canonical alias/repetition handling, hard-link path
identity, source IDs, composition, version placement, and all existing byte/node/
source/depth budgets. Optional glob declarations suppress only zero matches, never
invalid syntax, walk/permission failures, matched directories, or symlink escapes.
- Completed filesystem option validation independently of import/profile selection:
file inputs reject `BaseDir` and may select a containing root; byte/readers require
existing absolute base/root pairs whenever context is supplied; `NoSource` rejects
context while the reserved `default` profile remains inactive.
- Tests cover lexical and multi-declaration ordering, deterministic repeated loads,
overlapping matches, directories, hard links, contained absolute and escaping
symlink directories, required/optional no-match behavior, invalid patterns in empty
directories, escaped metacharacters, character classes, visible permission errors,
explicit `conf.d` imports without discovery, nonmatching-entry exhaustion, candidate
bounds, and filesystem combinations. The
[current import contract](../config.md#explicit-imports) owns the completed behavior.
Verified `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`,
`go mod tidy -diff`, and `git diff --check` with Go 1.26.7 on linux/amd64.
Read: feature import syntax and source-context API. Read: feature import syntax and source-context API.

View File

@@ -475,30 +475,9 @@ the surviving sequence, not an attempted correspondence with previous indices.
## Explicit imports and source identity ## Explicit imports and source identity
Explicit path-import traversal, source identity, containment, budgets, recursive Explicit path and glob imports, recursive traversal, source identity, containment,
directives, and the temporary glob gate are implemented and owned by the filesystem context, and budgets are implemented and now owned by the
[current configuration contract](../config.md#explicit-path-imports). Bounded glob [current configuration contract](../config.md#explicit-imports).
expansion remains target behavior. Illustrative future fragment:
```yaml
_configkit:
imports:
- glob: fragments/outputs/*.yml
optional: true
```
- Expand declarations in list order, sorting each glob's matches
lexicographically by slash-separated relative path, with no locale sorting.
Use the slash-based [Go `path.Match` grammar](https://pkg.go.dev/path#Match)
(`*`, `?`, character classes, and backslash escapes), with recursive `**`
syntax forbidden. Backslashes escape pattern characters, not directory separators.
- A required unmatched glob is an error. Optional suppresses only zero matches.
Invalid patterns, permission
failures, directories, and invalid matched documents remain errors.
- Glob traversal counts examined directory entries, including nonmatches, against
its separate work budget; only accepted matches contribute to sorted source order.
Expansion remains pattern-directed and must preserve the current path-import
identity, containment, repetition, and depth-first traversal rules.
## Profiles ## Profiles

205
glob.go Normal file
View File

@@ -0,0 +1,205 @@
package configkit
import (
"errors"
"io"
"io/fs"
"os"
"path"
"path/filepath"
"slices"
"strings"
)
const globReadBatch = 128
type globCandidate struct {
absolute string
relative string
}
func (s *importTraversal) expandImportGlob(basePath string, declaration importDeclaration) ([]string, []Diagnostic) {
location := declaration.location
segments, valid := validGlobPattern(declaration.glob)
if !valid {
return nil, []Diagnostic{importDiagnostic(CodeImportInvalid, "import glob pattern is invalid or uses recursive ** syntax", &location)}
}
candidates := []globCandidate{{absolute: basePath}}
for index, segment := range segments {
last := index == len(segments)-1
literal, patterned := globLiteralSegment(segment)
var next []globCandidate
for _, candidate := range candidates {
var names []string
if patterned {
matched, diagnostics := s.readGlobMatches(candidate.absolute, segment, location)
if len(diagnostics) != 0 {
return nil, diagnostics
}
names = matched
} else {
names = []string{literal}
}
for _, name := range names {
absolute := filepath.Join(candidate.absolute, filepath.FromSlash(name))
relative := path.Join(candidate.relative, name)
if !containedPath(s.rootPath, filepath.Clean(absolute)) {
return nil, []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "glob traversal is outside the configuration root", &location)}
}
canonical, err := canonicalExistingPath(absolute)
if err != nil {
if errors.Is(err, fs.ErrNotExist) && !patterned {
continue
}
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob match could not be canonicalized", location)}
}
if !containedPath(s.rootPath, canonical) {
return nil, []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "glob traversal resolves outside the configuration root", &location)}
}
if last {
next = append(next, globCandidate{absolute: canonical, relative: relative})
if len(s.sources)+len(next) > s.limits.sources {
return nil, []Diagnostic{importDiagnostic(CodeImportLimitExceeded, "glob matches exceed the remaining source-count limit", &location)}
}
continue
}
info, err := os.Stat(canonical)
if err != nil {
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob traversal target could not be inspected", location)}
}
if info.IsDir() {
next = append(next, globCandidate{absolute: canonical, relative: relative})
}
}
}
candidates = next
if len(candidates) == 0 {
return nil, nil
}
}
matches := make([]string, len(candidates))
for index, candidate := range candidates {
matches[index] = candidate.relative
}
slices.Sort(matches)
return matches, nil
}
func (s *importTraversal) readGlobMatches(directory, pattern string, location Location) ([]string, []Diagnostic) {
canonical, err := canonicalExistingPath(directory)
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
return nil, nil
}
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob directory could not be canonicalized", location)}
}
if !containedPath(s.rootPath, canonical) {
return nil, []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "glob directory resolves outside the configuration root", &location)}
}
relative, err := filepath.Rel(s.rootPath, canonical)
if err != nil || filepath.IsAbs(relative) || relative == ".." || strings.HasPrefix(relative, ".."+string(filepath.Separator)) {
return nil, []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "glob directory could not be addressed inside the configuration root", &location)}
}
directoryFile, err := s.root.Open(relative)
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
return nil, nil
}
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob directory could not be opened", location)}
}
var matches []string
for {
entries, readErr := directoryFile.ReadDir(globReadBatch)
for _, entry := range entries {
s.usedEntries++
if s.usedEntries > s.limits.entries {
_ = directoryFile.Close()
return nil, []Diagnostic{importDiagnostic(CodeImportLimitExceeded, "glob traversal exceeds the directory-entry visit limit", &location)}
}
matched, matchErr := path.Match(pattern, entry.Name())
if matchErr != nil {
_ = directoryFile.Close()
return nil, []Diagnostic{importDiagnostic(CodeImportInvalid, "import glob pattern is invalid", &location)}
}
if matched {
matches = append(matches, entry.Name())
}
}
if errors.Is(readErr, io.EOF) {
break
}
if readErr != nil {
_ = directoryFile.Close()
return nil, []Diagnostic{globFilesystemDiagnostic(readErr, "glob directory could not be read", location)}
}
}
if err := directoryFile.Close(); err != nil {
return nil, []Diagnostic{globFilesystemDiagnostic(err, "glob directory could not be closed", location)}
}
slices.Sort(matches)
return matches, nil
}
func validGlobPattern(pattern string) ([]string, bool) {
if !validImportPath(pattern) {
return nil, false
}
segments := strings.Split(pattern, "/")
for _, segment := range segments {
if segment == "" || hasRecursiveStars(segment) {
return nil, false
}
if _, err := path.Match(segment, ""); err != nil {
return nil, false
}
}
return segments, true
}
func hasRecursiveStars(segment string) bool {
inClass := false
for index := 0; index < len(segment); index++ {
switch segment[index] {
case '\\':
index++
case '[':
inClass = true
case ']':
inClass = false
case '*':
if !inClass && index+1 < len(segment) && segment[index+1] == '*' {
return true
}
}
}
return false
}
func globLiteralSegment(segment string) (string, bool) {
var literal strings.Builder
for index := 0; index < len(segment); index++ {
switch segment[index] {
case '*', '?', '[':
return "", true
case '\\':
index++
if index < len(segment) {
literal.WriteByte(segment[index])
}
default:
literal.WriteByte(segment[index])
}
}
return literal.String(), false
}
func globFilesystemDiagnostic(err error, message string, location Location) Diagnostic {
code := CodeImportRead
if errors.Is(err, fs.ErrPermission) {
code = CodeImportPermission
}
return importDiagnostic(code, message, &location)
}

273
glob_test.go Normal file
View File

@@ -0,0 +1,273 @@
package configkit
import (
"os"
"path/filepath"
"reflect"
"runtime"
"testing"
)
func TestLoadGlobImportsUseLexicalDeclarationOrderAndNestedDFS(t *testing.T) {
directory := t.TempDir()
mainPath := writeImportFile(t, directory, "main.yml", `_configkit:
imports:
- glob: fragments/*.yml
- path: after.yml
items: {root: root}
`)
aPath := writeImportFile(t, directory, "fragments/a.yml", `_configkit:
imports: [{path: nested/child.yml}]
items: {alpha: alpha}
`)
childPath := writeImportFile(t, directory, "fragments/nested/child.yml", "items: {child: child}\n")
bPath := writeImportFile(t, directory, "fragments/b.yml", "items: {beta: beta}\n")
afterPath := writeImportFile(t, directory, "after.yml", "items: {after: after}\n")
var previous []SourceInfo
for run := 0; run < 2; run++ {
result, err := Load[importTestConfig](File(mainPath), Options[importTestConfig]{Imports: true})
if err != nil {
t.Fatal(err)
}
wantItems := map[string]string{"root": "root", "alpha": "alpha", "child": "child", "beta": "beta", "after": "after"}
if !reflect.DeepEqual(result.Config.Items, wantItems) {
t.Fatalf("items = %#v, want %#v", result.Config.Items, wantItems)
}
sources := result.Sources()
wantPaths := []string{
canonicalTestPath(t, mainPath), canonicalTestPath(t, aPath),
canonicalTestPath(t, childPath), canonicalTestPath(t, bPath),
canonicalTestPath(t, afterPath),
}
gotPaths := make([]string, len(sources))
for index, source := range sources {
gotPaths[index] = source.Path
}
if !reflect.DeepEqual(gotPaths, wantPaths) {
t.Fatalf("source paths = %#v, want %#v", gotPaths, wantPaths)
}
if run != 0 && !reflect.DeepEqual(sources, previous) {
t.Fatalf("repeated traversal changed sources: %#v != %#v", sources, previous)
}
previous = sources
}
}
func TestLoadGlobImportsSupportEscapesClassesAndExplicitConfD(t *testing.T) {
directory := t.TempDir()
mainPath := writeImportFile(t, directory, "main.yml", `_configkit:
imports:
- glob: 'conf.d/literal\*.yml'
- glob: 'conf.d/item-[ab].yml'
- glob: 'conf.d/question?.yml'
items: {root: root}
`)
writeImportFile(t, directory, "conf.d/literal*.yml", "items: {literal: literal}\n")
writeImportFile(t, directory, "conf.d/item-b.yml", "items: {b: b}\n")
writeImportFile(t, directory, "conf.d/item-a.yml", "items: {a: a}\n")
writeImportFile(t, directory, "conf.d/question1.yml", "items: {question: question}\n")
result, err := Load[importTestConfig](File(mainPath), Options[importTestConfig]{Imports: true})
if err != nil {
t.Fatal(err)
}
if want := map[string]string{"root": "root", "literal": "literal", "a": "a", "b": "b", "question": "question"}; !reflect.DeepEqual(result.Config.Items, want) {
t.Fatalf("items = %#v, want %#v", result.Config.Items, want)
}
ignoredPath := writeImportFile(t, directory, "conf.d/not-discovered.yml", "unknown: value\n")
_ = ignoredPath
plainPath := writeImportFile(t, directory, "plain.yml", "items: {root: only}\n")
plain, err := Load[importTestConfig](File(plainPath), Options[importTestConfig]{Imports: true})
if err != nil || len(plain.Sources()) != 1 || plain.Config.Items["root"] != "only" {
t.Fatalf("implicit discovery result = %#v, %v", plain, err)
}
}
func TestLoadGlobImportsRejectInvalidAndUnmatchedPatterns(t *testing.T) {
tests := []struct {
name string
pattern string
code Code
}{
{name: "bad class", pattern: "empty/[", code: CodeImportInvalid},
{name: "recursive star segment", pattern: "empty/**/value.yml", code: CodeImportInvalid},
{name: "recursive adjacent stars", pattern: "empty/value**.yml", code: CodeImportInvalid},
{name: "trailing separator", pattern: "empty/", code: CodeImportInvalid},
{name: "absolute", pattern: "/tmp/*.yml", code: CodeImportInvalid},
{name: "URL", pattern: "https://example.invalid/*.yml", code: CodeImportInvalid},
{name: "missing parent escape", pattern: "../missing/*.yml", code: CodeImportOutsideRoot},
{name: "unmatched", pattern: "empty/*.yml", code: CodeImportMissing},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
directory := t.TempDir()
if err := os.Mkdir(filepath.Join(directory, "empty"), 0o755); err != nil {
t.Fatal(err)
}
mainPath := writeGlobRoot(t, directory, test.pattern, false)
requireLoadCode(t, File(mainPath), Options[importTestConfig]{Imports: true}, test.code)
})
}
directory := t.TempDir()
if err := os.Mkdir(filepath.Join(directory, "empty"), 0o755); err != nil {
t.Fatal(err)
}
mainPath := writeGlobRoot(t, directory, "empty/*.yml", true)
result, err := Load[importTestConfig](File(mainPath), Options[importTestConfig]{Imports: true})
if err != nil || len(result.Sources()) != 1 {
t.Fatalf("optional unmatched glob = %#v, %v", result, err)
}
}
func TestLoadGlobImportsPreserveIdentityAndFileRules(t *testing.T) {
t.Run("overlapping declarations", func(t *testing.T) {
directory := t.TempDir()
writeImportFile(t, directory, "conf/value.yml", "items: {value: value}\n")
mainPath := writeImportFile(t, directory, "main.yml", `_configkit:
imports:
- glob: conf/*.yml
- glob: conf/value.*
`)
requireLoadCode(t, File(mainPath), Options[importTestConfig]{Imports: true}, CodeImportRepeated)
})
t.Run("matched directory", func(t *testing.T) {
directory := t.TempDir()
if err := os.MkdirAll(filepath.Join(directory, "conf/matched.yml"), 0o755); err != nil {
t.Fatal(err)
}
mainPath := writeGlobRoot(t, directory, "conf/*.yml", true)
requireLoadCode(t, File(mainPath), Options[importTestConfig]{Imports: true}, CodeSourceNotRegular)
})
t.Run("hard links stay distinct", func(t *testing.T) {
directory := t.TempDir()
one := writeImportFile(t, directory, "conf/one.yml", "items: {same: value}\n")
if err := os.Link(one, filepath.Join(directory, "conf/two.yml")); err != nil {
t.Skipf("hard links unavailable: %v", err)
}
mainPath := writeGlobRoot(t, directory, "conf/*.yml", false)
requireLoadCode(t, File(mainPath), Options[importTestConfig]{Imports: true}, CodeComposeConflict)
})
}
func TestLoadGlobImportsConfineSymlinkDirectories(t *testing.T) {
if runtime.GOOS != "linux" && runtime.GOOS != "darwin" {
t.Skip("initial filesystem contract covers Linux and macOS")
}
t.Run("contained absolute target", func(t *testing.T) {
root := t.TempDir()
target := filepath.Join(root, "target")
writeImportFile(t, target, "value.yml", "items: {linked: linked}\n")
if err := os.Symlink(target, filepath.Join(root, "linked")); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
mainPath := writeGlobRoot(t, root, "linked/*.yml", false)
result, err := Load[importTestConfig](File(mainPath), Options[importTestConfig]{Imports: true})
if err != nil || result.Config.Items["linked"] != "linked" {
t.Fatalf("contained directory symlink = %#v, %v", result, err)
}
})
t.Run("escaping target", func(t *testing.T) {
root := t.TempDir()
outside := t.TempDir()
writeImportFile(t, outside, "value.yml", "items: {escaped: escaped}\n")
if err := os.Symlink(outside, filepath.Join(root, "linked")); err != nil {
t.Skipf("symlinks unavailable: %v", err)
}
mainPath := writeGlobRoot(t, root, "linked/*.yml", true)
requireLoadCode(t, File(mainPath), Options[importTestConfig]{Imports: true}, CodeImportOutsideRoot)
})
}
func TestOptionalGlobDoesNotSuppressDirectoryPermissionFailure(t *testing.T) {
directory := t.TempDir()
locked := filepath.Join(directory, "locked")
if err := os.Mkdir(locked, 0o700); err != nil {
t.Fatal(err)
}
if err := os.Chmod(locked, 0); err != nil {
t.Skipf("cannot create unreadable directory fixture: %v", err)
}
defer func() { _ = os.Chmod(locked, 0o700) }()
if opened, err := os.Open(locked); err == nil {
_ = opened.Close()
t.Skip("filesystem does not enforce the unreadable-directory fixture")
}
mainPath := writeGlobRoot(t, directory, "locked/*.yml", true)
requireLoadCode(t, File(mainPath), Options[importTestConfig]{Imports: true}, CodeImportPermission)
}
func TestGlobTraversalBoundsNonmatchesAndAcceptedCandidates(t *testing.T) {
directory := t.TempDir()
for _, name := range []string{"one.txt", "two.txt", "three.txt"} {
writeImportFile(t, directory, "many/"+name, "ignored\n")
}
mainPath := writeGlobRoot(t, directory, "many/*.yml", true)
main := parseImportMain(t, mainPath)
limits := defaultImportLimits()
limits.entries = 2
_, _, diagnostics := loadImportedDocuments(File(mainPath), nil, main, nil, limits)
if len(diagnostics) != 1 || diagnostics[0].Code != CodeImportLimitExceeded {
t.Fatalf("entry-budget diagnostics = %#v", diagnostics)
}
for _, name := range []string{"a.yml", "b.yml"} {
writeImportFile(t, directory, "matches/"+name, "items: {"+name[:1]+": value}\n")
}
mainPath = writeGlobRoot(t, directory, "matches/*.yml", false)
main = parseImportMain(t, mainPath)
limits = defaultImportLimits()
limits.sources = 2
_, _, diagnostics = loadImportedDocuments(File(mainPath), nil, main, nil, limits)
if len(diagnostics) != 1 || diagnostics[0].Code != CodeImportLimitExceeded {
t.Fatalf("candidate-budget diagnostics = %#v", diagnostics)
}
}
func TestFilesystemContextIsValidatedWithoutImportsOrActiveProfile(t *testing.T) {
directory := t.TempDir()
mainPath := writeImportFile(t, directory, "main.yml", "items: {root: root}\n")
result, err := Load[importTestConfig](File(mainPath), Options[importTestConfig]{
Profile: "default", Filesystem: &FilesystemContext{Root: directory},
})
if err != nil || result.Config.Items["root"] != "root" {
t.Fatalf("file filesystem context = %#v, %v", result, err)
}
requireLoadCode(t, File(mainPath), Options[importTestConfig]{Filesystem: &FilesystemContext{BaseDir: directory, Root: directory}}, CodeOptionsInvalid)
requireLoadCode(t, File(mainPath), Options[importTestConfig]{Filesystem: &FilesystemContext{Root: filepath.Join(directory, "missing")}}, CodeImportInvalid)
input := Bytes("memory.yml", []byte("items: {root: memory}\n"))
result, err = Load[importTestConfig](input, Options[importTestConfig]{Filesystem: &FilesystemContext{BaseDir: directory, Root: directory}})
if err != nil || result.Config.Items["root"] != "memory" {
t.Fatalf("byte filesystem context = %#v, %v", result, err)
}
requireLoadCode(t, Bytes("memory.yml", []byte("{}\n")), Options[importTestConfig]{Filesystem: &FilesystemContext{BaseDir: directory}}, CodeOptionsInvalid)
defaultsOnly, err := Load[importTestConfig](NoSource(), Options[importTestConfig]{Profile: "default"})
if err != nil || defaultsOnly == nil {
t.Fatalf("default profile with NoSource = %#v, %v", defaultsOnly, err)
}
if runtime.GOOS == "linux" || runtime.GOOS == "darwin" {
selectedLink := filepath.Join(directory, "selected-link.yml")
if err := os.Symlink(mainPath, selectedLink); err == nil {
linked, err := Load[importTestConfig](File(selectedLink), Options[importTestConfig]{Filesystem: &FilesystemContext{Root: directory}})
if err != nil || linked.Sources()[0].Path != canonicalTestPath(t, mainPath) {
t.Fatalf("selected symlink identity = %#v, %v", linked, err)
}
}
}
}
func writeGlobRoot(t *testing.T, directory, pattern string, optional bool) string {
t.Helper()
optionalLine := ""
if optional {
optionalLine = "\n optional: true"
}
return writeImportFile(t, directory, "main.yml", "_configkit:\n imports:\n - glob: '"+pattern+"'"+optionalLine+"\n")
}

View File

@@ -5,7 +5,6 @@ import (
"errors" "errors"
"io" "io"
"io/fs" "io/fs"
"net/url"
"os" "os"
"path" "path"
"path/filepath" "path/filepath"
@@ -19,6 +18,7 @@ import (
const ( const (
maxImportDepth = 64 maxImportDepth = 64
maxImportSources = 1024 maxImportSources = 1024
maxGlobEntries = 1_000_000
) )
type importLimits struct { type importLimits struct {
@@ -26,12 +26,13 @@ type importLimits struct {
sources int sources int
bytes int64 bytes int64
nodes int nodes int
entries int
} }
func defaultImportLimits() importLimits { func defaultImportLimits() importLimits {
return importLimits{ return importLimits{
depth: maxImportDepth, sources: maxImportSources, depth: maxImportDepth, sources: maxImportSources,
bytes: maxTotalBytes, nodes: maxDocumentNodes, bytes: maxTotalBytes, nodes: maxDocumentNodes, entries: maxGlobEntries,
} }
} }
@@ -53,13 +54,14 @@ type importTraversal struct {
limits importLimits limits importLimits
rule *VersionRule rule *VersionRule
documents []*document.Node documents []*document.Node
sources []SourceInfo sources []SourceInfo
active map[string]int active map[string]int
visited map[string]importRoute visited map[string]importRoute
stack []importRoute stack []importRoute
usedBytes int64 usedBytes int64
usedNodes int usedNodes int
usedEntries int
} }
func loadImportedDocuments(source Source, context *FilesystemContext, main parsedSource, rule *VersionRule, limits importLimits) (*composedDocument, []SourceInfo, []Diagnostic) { func loadImportedDocuments(source Source, context *FilesystemContext, main parsedSource, rule *VersionRule, limits importLimits) (*composedDocument, []SourceInfo, []Diagnostic) {
@@ -69,7 +71,9 @@ func loadImportedDocuments(source Source, context *FilesystemContext, main parse
} }
root, err := os.OpenRoot(rootPath) root, err := os.OpenRoot(rootPath)
if err != nil { if err != nil {
return nil, sourceInfos(main.source), []Diagnostic{importDiagnostic(CodeImportInvalid, "configuration root could not be opened", nil)} diagnostic := globFilesystemDiagnostic(err, "configuration root could not be opened", Location{})
diagnostic.Primary = nil
return nil, sourceInfos(main.source), []Diagnostic{diagnostic}
} }
mainInfo := cloneSourceInfo(*main.source) mainInfo := cloneSourceInfo(*main.source)
@@ -107,12 +111,49 @@ func loadImportedDocuments(source Source, context *FilesystemContext, main parse
return nil, cloneSourceInfos(state.sources), diagnostics return nil, cloneSourceInfos(state.sources), diagnostics
} }
if closeErr != nil { if closeErr != nil {
return nil, cloneSourceInfos(state.sources), []Diagnostic{importDiagnostic(CodeImportInvalid, "configuration root could not be closed", nil)} return nil, cloneSourceInfos(state.sources), []Diagnostic{importDiagnostic(CodeImportRead, "configuration root could not be closed", nil)}
} }
composed, diagnostics := composeDocuments(state.documents...) composed, diagnostics := composeDocuments(state.documents...)
return composed, cloneSourceInfos(state.sources), diagnostics return composed, cloneSourceInfos(state.sources), diagnostics
} }
func validateFilesystemContext(source Source, context *FilesystemContext, main parsedSource) *Diagnostic {
rootPath, _, _, diagnostic := importFilesystem(source, context, main)
if diagnostic != nil {
return diagnostic
}
root, err := os.OpenRoot(rootPath)
if err != nil {
value := globFilesystemDiagnostic(err, "configuration root could not be opened", Location{})
value.Primary = nil
return &value
}
if err := root.Close(); err != nil {
value := importDiagnostic(CodeImportRead, "configuration root could not be closed", nil)
return &value
}
return nil
}
func filesystemOptionDiagnostics(source Source, context *FilesystemContext) []Diagnostic {
switch source.kind {
case sourceNone:
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "NoSource cannot use filesystem context")}
case sourceFile:
if context.BaseDir != "" {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "file sources determine their own import base directory")}
}
case sourceBytes, sourceReader:
if context.BaseDir == "" || context.Root == "" {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "byte and reader filesystem context requires base and root directories")}
}
if !filepath.IsAbs(context.BaseDir) || !filepath.IsAbs(context.Root) {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "byte and reader filesystem base and root directories must be absolute")}
}
}
return nil
}
func importFilesystem(source Source, context *FilesystemContext, main parsedSource) (rootPath, basePath, mainPath string, diagnostic *Diagnostic) { func importFilesystem(source Source, context *FilesystemContext, main parsedSource) (rootPath, basePath, mainPath string, diagnostic *Diagnostic) {
if main.source == nil || main.document == nil { if main.source == nil || main.document == nil {
value := importDiagnostic(CodeImportInvalid, "imports require a selected configuration source", nil) value := importDiagnostic(CodeImportInvalid, "imports require a selected configuration source", nil)
@@ -209,8 +250,26 @@ func (s *importTraversal) checkInitialLimits() *Diagnostic {
func (s *importTraversal) walkDeclarations(basePath string, declarations []importDeclaration, depth int) []Diagnostic { func (s *importTraversal) walkDeclarations(basePath string, declarations []importDeclaration, depth int) []Diagnostic {
for _, declaration := range declarations { for _, declaration := range declarations {
if declaration.glob != "" { if declaration.glob != "" {
location := declaration.location matches, diagnostics := s.expandImportGlob(basePath, declaration)
return []Diagnostic{importDiagnostic(CodeImportUnsupported, "glob imports are not available in this release boundary", &location)} if len(diagnostics) != 0 {
return diagnostics
}
if len(matches) == 0 {
if declaration.optional {
continue
}
location := declaration.location
return []Diagnostic{importDiagnostic(CodeImportMissing, "required import glob matched no files", &location)}
}
for _, match := range matches {
matched := declaration
matched.path = match
matched.glob = ""
if diagnostics := s.visitPath(basePath, matched, depth+1); len(diagnostics) != 0 {
return diagnostics
}
}
continue
} }
if diagnostics := s.visitPath(basePath, declaration, depth+1); len(diagnostics) != 0 { if diagnostics := s.visitPath(basePath, declaration, depth+1); len(diagnostics) != 0 {
return diagnostics return diagnostics
@@ -518,8 +577,29 @@ func validImportPath(value string) bool {
if value == "" || strings.TrimSpace(value) != value || path.IsAbs(value) || strings.HasPrefix(value, "//") { if value == "" || strings.TrimSpace(value) != value || path.IsAbs(value) || strings.HasPrefix(value, "//") {
return false return false
} }
parsed, err := url.Parse(value) return !hasImportScheme(value)
return err == nil && parsed.Scheme == "" }
func hasImportScheme(value string) bool {
colon := strings.IndexByte(value, ':')
slash := strings.IndexByte(value, '/')
if colon <= 0 || (slash >= 0 && slash < colon) {
return false
}
for index := 0; index < colon; index++ {
character := value[index]
if index == 0 {
if (character < 'a' || character > 'z') && (character < 'A' || character > 'Z') {
return false
}
continue
}
if (character < 'a' || character > 'z') && (character < 'A' || character > 'Z') &&
(character < '0' || character > '9') && character != '+' && character != '-' && character != '.' {
return false
}
}
return true
} }
func sourceInfos(source *SourceInfo) []SourceInfo { func sourceInfos(source *SourceInfo) []SourceInfo {

View File

@@ -18,7 +18,8 @@ type importTestConfig struct {
Nested string `yaml:"nested"` Nested string `yaml:"nested"`
Second string `yaml:"second"` Second string `yaml:"second"`
} `yaml:"settings"` } `yaml:"settings"`
Version string `yaml:"version"` Version string `yaml:"version"`
Items map[string]string `yaml:"items"`
} }
func TestLoadTraversesPathImportsDepthFirstAndComposesOrigins(t *testing.T) { func TestLoadTraversesPathImportsDepthFirstAndComposesOrigins(t *testing.T) {
@@ -121,7 +122,6 @@ func TestLoadImportDirectiveValidationAndTemporaryGlobGate(t *testing.T) {
{name: "two targets", body: "_configkit: {imports: [{path: a.yml, glob: '*.yml'}]}\n", code: CodeImportInvalid}, {name: "two targets", body: "_configkit: {imports: [{path: a.yml, glob: '*.yml'}]}\n", code: CodeImportInvalid},
{name: "unknown declaration field", body: "_configkit: {imports: [{path: a.yml, extra: true}]}\n", code: CodeImportInvalid}, {name: "unknown declaration field", body: "_configkit: {imports: [{path: a.yml, extra: true}]}\n", code: CodeImportInvalid},
{name: "optional kind", body: "_configkit: {imports: [{path: a.yml, optional: 'yes'}]}\n", code: CodeImportInvalid}, {name: "optional kind", body: "_configkit: {imports: [{path: a.yml, optional: 'yes'}]}\n", code: CodeImportInvalid},
{name: "glob gate", body: "_configkit: {imports: [{glob: '*.yml'}]}\n", code: CodeImportUnsupported},
{name: "nested directive", body: "nested: {_configkit: {}}\n", code: CodeImportInvalid}, {name: "nested directive", body: "nested: {_configkit: {}}\n", code: CodeImportInvalid},
} }
for _, test := range tests { for _, test := range tests {

20
load.go
View File

@@ -17,8 +17,8 @@ type Definition[T any] struct {
} }
// Options contains the application definition and per-load input selections. // Options contains the application definition and per-load input selections.
// Imports enables explicit path imports; glob imports and later precedence layers // Imports enables explicit path and glob imports; later precedence layers remain
// remain unavailable until their owning features are implemented. // unavailable until their owning features are implemented.
type Options[T any] struct { type Options[T any] struct {
Definition Definition[T] Definition Definition[T]
Imports bool Imports bool
@@ -127,6 +127,12 @@ func Load[T any](source Source, supplied ...Options[T]) (*Result[T], error) {
if parsed.source != nil { if parsed.source != nil {
accumulator.sources = append(accumulator.sources, *parsed.source) accumulator.sources = append(accumulator.sources, *parsed.source)
} }
if options.Filesystem != nil {
if diagnostic := validateFilesystemContext(source, options.Filesystem, parsed); diagnostic != nil {
accumulator.add(*diagnostic)
return nil, accumulator.loadError()
}
}
} }
if !options.Imports && inputDocument != nil { if !options.Imports && inputDocument != nil {
directiveDiagnostics := disabledDirectiveDiagnostics(inputDocument, Path{}) directiveDiagnostics := disabledDirectiveDiagnostics(inputDocument, Path{})
@@ -210,14 +216,8 @@ func selectOptions[T any](source Source, supplied []Options[T]) (Options[T], []D
if options.Profile != "" && options.Profile != "default" { if options.Profile != "" && options.Profile != "default" {
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsUnavailable, "configuration profiles are not available in this release boundary")) diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsUnavailable, "configuration profiles are not available in this release boundary"))
} }
if options.Filesystem != nil && !options.Imports { if options.Filesystem != nil {
code := CodeOptionsUnavailable diagnostics = append(diagnostics, filesystemOptionDiagnostics(source, options.Filesystem)...)
message := "filesystem context is not available in this release boundary"
if source.kind == sourceNone {
code = CodeOptionsInvalid
message = "NoSource cannot use filesystem context"
}
diagnostics = append(diagnostics, optionDiagnostic(code, message))
} }
if len(options.Overrides) != 0 { if len(options.Overrides) != 0 {
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsUnavailable, "configuration overrides are not available in this release boundary")) diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsUnavailable, "configuration overrides are not available in this release boundary"))

View File

@@ -225,7 +225,7 @@ func TestLoadRejectsUnavailableOptionsAndDisabledDirectives(t *testing.T) {
{name: "secret classifier", source: validSource(), options: []Options[config]{{Definition: Definition[config]{Secret: func(Path) bool { return false }}}}, code: CodeOptionsUnavailable}, {name: "secret classifier", source: validSource(), options: []Options[config]{{Definition: Definition[config]{Secret: func(Path) bool { return false }}}}, code: CodeOptionsUnavailable},
{name: "imports without filesystem context", source: validSource(), options: []Options[config]{{Imports: true}}, code: CodeOptionsInvalid}, {name: "imports without filesystem context", source: validSource(), options: []Options[config]{{Imports: true}}, code: CodeOptionsInvalid},
{name: "profile", source: validSource(), options: []Options[config]{{Profile: "testing"}}, code: CodeOptionsUnavailable}, {name: "profile", source: validSource(), options: []Options[config]{{Profile: "testing"}}, code: CodeOptionsUnavailable},
{name: "filesystem", source: validSource(), options: []Options[config]{{Filesystem: &FilesystemContext{}}}, code: CodeOptionsUnavailable}, {name: "incomplete filesystem", source: validSource(), options: []Options[config]{{Filesystem: &FilesystemContext{}}}, code: CodeOptionsInvalid},
{name: "overrides", source: validSource(), options: []Options[config]{{Overrides: []Override{{Path: Keys("name"), Value: "other", Flag: "--name"}}}}, code: CodeOptionsUnavailable}, {name: "overrides", source: validSource(), options: []Options[config]{{Overrides: []Override{{Path: Keys("name"), Value: "other", Flag: "--name"}}}}, code: CodeOptionsUnavailable},
{name: "missing version checker", source: validSource(), options: []Options[config]{{Definition: Definition[config]{Version: &VersionRule{Path: Keys("version")}}}}, code: CodeOptionsInvalid}, {name: "missing version checker", source: validSource(), options: []Options[config]{{Definition: Definition[config]{Version: &VersionRule{Path: Keys("version")}}}}, code: CodeOptionsInvalid},
{name: "no-source filesystem", source: NoSource(), options: []Options[config]{{Filesystem: &FilesystemContext{}}}, code: CodeOptionsInvalid}, {name: "no-source filesystem", source: NoSource(), options: []Options[config]{{Filesystem: &FilesystemContext{}}}, code: CodeOptionsInvalid},

View File

@@ -78,8 +78,9 @@ const (
CodeImportDisabled Code = "import.disabled" CodeImportDisabled Code = "import.disabled"
CodeImportInvalid Code = "import.invalid" CodeImportInvalid Code = "import.invalid"
CodeImportUnsupported Code = "import.unsupported"
CodeImportMissing Code = "import.missing" CodeImportMissing Code = "import.missing"
CodeImportPermission Code = "import.permission"
CodeImportRead Code = "import.read"
CodeImportOutsideRoot Code = "import.outside_root" CodeImportOutsideRoot Code = "import.outside_root"
CodeImportCycle Code = "import.cycle" CodeImportCycle Code = "import.cycle"
CodeImportRepeated Code = "import.repeated" CodeImportRepeated Code = "import.repeated"

View File

@@ -150,6 +150,12 @@ func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead
_ = file.Close() _ = file.Close()
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceNotRegular, sourceID, "source is not a regular file")} return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceNotRegular, sourceID, "source is not a regular file")}
} }
canonical, err := filepath.EvalSymlinks(info.Path)
if err != nil {
_ = file.Close()
return sourceRead{info: info}, []Diagnostic{classifySourceError(err, sourceID)}
}
info.Path = canonical
if stat.Size() > limit { if stat.Size() > limit {
_ = file.Close() _ = file.Close()
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceLimitExceeded, sourceID, "source exceeds the configured byte limit")} return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceLimitExceeded, sourceID, "source exceeds the configured byte limit")}