Complete bounded glob imports
This commit is contained in:
@@ -22,9 +22,10 @@ File sources can observe file changes between consumptions; byte sources cannot.
|
||||
|
||||
Each source is limited to 8 MiB. An imported load is limited to 64 MiB across all
|
||||
accepted sources, 1,024 accepted sources, 64 active import edges, and 1,000,000
|
||||
input nodes in aggregate. Reads request at most the remaining allowance plus one byte
|
||||
so excess data is rejected rather than silently truncated. Data returned together
|
||||
with a read error is not accepted as partial input.
|
||||
input nodes in aggregate. Glob expansion separately permits 1,000,000 visited
|
||||
directory entries, including nonmatches. Reads request at most the remaining
|
||||
allowance plus one byte so excess data is rejected rather than silently truncated.
|
||||
Data returned together with a read error is not accepted as partial input.
|
||||
|
||||
## Single-source loading
|
||||
|
||||
@@ -50,10 +51,10 @@ contributes no source, and continues through defaults, decode, and projection. A
|
||||
missing file remains an error; it is never interpreted as no input.
|
||||
|
||||
The current `Options` boundary accepts a defaults factory, `VersionRule`, resolver,
|
||||
semantic validator, explicit path imports, and import filesystem context. Active
|
||||
profiles, standalone filesystem context, overrides, and dynamic secret classification
|
||||
return `options.unavailable` because their lifecycle stages are not implemented.
|
||||
The reserved profile spelling `default` selects no active profile. `NoSource`
|
||||
semantic validator, explicit path and glob imports, and filesystem context. Active
|
||||
profiles, overrides, and dynamic secret classification return `options.unavailable`
|
||||
because their lifecycle stages are not implemented. The reserved profile spelling
|
||||
`default` selects no active profile. `NoSource`
|
||||
additionally rejects imports, active profiles, and filesystem context as invalid
|
||||
combinations. Input containing a reserved `_configkit` mapping key is rejected while
|
||||
imports are disabled.
|
||||
@@ -180,16 +181,14 @@ traversal order. Traversal order never authorizes replacement. Composition clone
|
||||
the resulting tree and its ordered claim records rather than mutating or aliasing an
|
||||
input document.
|
||||
|
||||
## Explicit path imports
|
||||
## Explicit imports
|
||||
|
||||
Imports are opt-in. With `Options.Imports` enabled, a document may reserve its root
|
||||
`_configkit` mapping for an `imports` sequence. ConfigKit removes that mapping before
|
||||
application decoding. Each sequence item is a mapping with exactly one nonempty
|
||||
string `path` or `glob` plus an optional boolean `optional`; all other shapes and
|
||||
fields are `import.invalid`. `glob` is recognized but temporarily returns
|
||||
`import.unsupported` until bounded glob expansion is implemented. A `_configkit`
|
||||
key below the root is invalid. With imports disabled, any occurrence remains
|
||||
`import.disabled`.
|
||||
fields are `import.invalid`. A `_configkit` key below the root is invalid. With
|
||||
imports disabled, any occurrence remains `import.disabled`.
|
||||
|
||||
Path declarations use slash-separated relative paths. Absolute paths and network
|
||||
URLs are invalid. Each path resolves from its declaring file's canonical directory,
|
||||
@@ -199,16 +198,29 @@ permits replacement. Required missing paths fail; `optional: true` suppresses on
|
||||
target nonexistence, not invalid paths, containment failures, directories,
|
||||
permissions, parse errors, or composition errors.
|
||||
|
||||
Glob declarations use the slash-based Go `path.Match` grammar: `*`, `?`, character
|
||||
classes, and backslash escapes are supported, while recursive `**` syntax is invalid.
|
||||
Backslashes escape pattern characters rather than acting as directory separators.
|
||||
ConfigKit validates the complete pattern before filesystem traversal, walks only
|
||||
directories selected by preceding segments, and reads wildcard directories in
|
||||
bounded batches. Every examined entry consumes the separate visit budget, including
|
||||
nonmatches. Matches are sorted lexicographically by slash-relative spelling within
|
||||
each declaration before depth-first source traversal. A required zero-match glob is
|
||||
`import.missing`; `optional: true` suppresses only zero matches. Invalid patterns,
|
||||
walk failures, matched directories, invalid documents, escapes, and repeated targets
|
||||
remain errors. Final candidates are bounded by the remaining source limit before
|
||||
their contents are accumulated.
|
||||
|
||||
The configuration root defaults to a selected file's canonical directory. A file
|
||||
source may supply `FilesystemContext.Root` to widen that root but may not supply
|
||||
`BaseDir`. Byte and reader sources must supply both `BaseDir` and `Root` as absolute
|
||||
existing directories when imports are enabled; their diagnostic labels, including
|
||||
URI-looking labels, never imply filesystem paths. The base and selected file must be
|
||||
inside the canonical root. ConfigKit canonicalizes symlinks, checks containment with
|
||||
relative paths, and opens the canonical root-relative target through a library-owned
|
||||
root handle. Contained relative or absolute symlink targets are accepted; escapes
|
||||
are rejected. Canonical paths define identity, so hard links at distinct paths remain
|
||||
distinct sources.
|
||||
`BaseDir`. Whenever byte and reader sources supply filesystem context, they must
|
||||
provide both `BaseDir` and `Root` as absolute existing directories; their diagnostic
|
||||
labels, including URI-looking labels, never imply filesystem paths. The base and
|
||||
selected file must be inside the canonical root. ConfigKit canonicalizes symlinks,
|
||||
checks containment with relative paths, and opens the canonical root-relative target
|
||||
through a library-owned root handle. Contained relative or absolute symlink targets
|
||||
are accepted; escapes are rejected. Canonical paths define identity, so hard links at
|
||||
distinct paths remain distinct sources.
|
||||
|
||||
An active-stack repeat is `import.cycle` and reports the complete cycle's import
|
||||
sites. A previously visited canonical path is `import.repeated` and reports both
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
This is the contributor entry point for ConfigKit. Read the task-specific owners
|
||||
below before making changes. The repository is in early implementation: its public
|
||||
loader currently covers strict selected sources, explicit path-import graphs,
|
||||
loader currently covers strict selected sources, explicit path/glob import graphs,
|
||||
composition, typed defaults, schema-version checks, recorded application resolution,
|
||||
semantic validation, and metadata. The
|
||||
[component overview](internal/overview.md) owns the implementation inventory.
|
||||
|
||||
@@ -124,6 +124,16 @@ fragment version-placement enforcement. The planned glob syntax is parsed but
|
||||
remains explicitly unavailable for its next owning work; hard links intentionally
|
||||
replace Narratio's same-file rejection with path identity.
|
||||
|
||||
Glob-import implementation record (2026-09-06): ConfigKit retains the extracted
|
||||
declaration order, import failures, composition, and identity cases while adding the
|
||||
new `expandImportGlob`, `readGlobMatches`, and pattern-validation mechanisms in
|
||||
`glob.go`. Focused cases cover lexical ordering, multiple declarations, overlap,
|
||||
directories, hard links, contained and escaping symlinks, unmatched declarations,
|
||||
malformed and recursive patterns, escaped metacharacters, character classes,
|
||||
permission failures, explicit `conf.d` use, and entry/candidate budgets. This work
|
||||
has no Narratio production equivalent; it uses the planned standard-library
|
||||
`path.Match` and bounded `File.ReadDir` adaptation rather than its flat loader.
|
||||
|
||||
### Profiles and overlay
|
||||
|
||||
Assigned to selected-profile overlay:
|
||||
|
||||
@@ -20,8 +20,9 @@ module and package documentation boundary. Its implemented public primitives are
|
||||
- strict schema-driven decoding over detached defaults in `decode.go`, including
|
||||
mapping/sequence semantics, null and numeric policy, safe codec errors, presence,
|
||||
baseline/final trees, and initial default/input origins; and
|
||||
- public lifecycle coordination in `load.go`, bounded canonical path-import DFS in
|
||||
`imports.go`, version-report contexts in `validation.go`, and detached
|
||||
- public lifecycle coordination in `load.go`, bounded canonical import DFS in
|
||||
`imports.go`, pattern-directed glob expansion in `glob.go`, version-report contexts
|
||||
in `validation.go`, and detached
|
||||
base/default metadata assembly in `load_metadata.go`; and
|
||||
- recorded application defaulting and normalization in `resolution.go`, including
|
||||
explicitness queries, helper verification, structural mutation audit, coalesced
|
||||
|
||||
@@ -878,7 +878,7 @@ Completion evidence (2026-09-06):
|
||||
tests cover nested/list-order traversal, composition conflicts, version placement,
|
||||
directives, optional/required paths, explicit roots, cycles, diamonds, symlinks,
|
||||
hard links, parse/decode attribution, file types, context rules, and reduced limit
|
||||
boundaries. The [current path-import contract](../config.md#explicit-path-imports)
|
||||
boundaries. The [current path-import contract](../config.md#explicit-imports)
|
||||
owns implemented behavior; glob expansion remains pending. Verified `go test ./...`,
|
||||
`go test -race ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and
|
||||
`git diff --check` with Go 1.26.7 on linux/amd64.
|
||||
@@ -917,7 +917,36 @@ Acceptance:
|
||||
|
||||
## Stage 12 — Complete glob imports and filesystem option handling
|
||||
|
||||
Status: pending.
|
||||
Status: complete.
|
||||
|
||||
Completion evidence (2026-09-06):
|
||||
|
||||
- Removed the temporary glob gate and added prevalidated Go `path.Match` semantics
|
||||
for `*`, `?`, character classes, and escapes, with recursive unescaped `**`, URLs,
|
||||
absolute paths, empty segments, and malformed classes rejected before any walk.
|
||||
- Added a pattern-directed walker that resolves literal segments directly and reads
|
||||
wildcard directories through positive bounded `File.ReadDir` batches. It charges
|
||||
every examined entry, including nonmatches, canonicalizes and contains traversed
|
||||
directories and final targets, sorts slash-relative matches, and bounds final
|
||||
candidates against remaining source capacity before traversal.
|
||||
- Reused the path-import DFS for every sorted match, preserving declaration ordering,
|
||||
nested depth-first imports, canonical alias/repetition handling, hard-link path
|
||||
identity, source IDs, composition, version placement, and all existing byte/node/
|
||||
source/depth budgets. Optional glob declarations suppress only zero matches, never
|
||||
invalid syntax, walk/permission failures, matched directories, or symlink escapes.
|
||||
- Completed filesystem option validation independently of import/profile selection:
|
||||
file inputs reject `BaseDir` and may select a containing root; byte/readers require
|
||||
existing absolute base/root pairs whenever context is supplied; `NoSource` rejects
|
||||
context while the reserved `default` profile remains inactive.
|
||||
- Tests cover lexical and multi-declaration ordering, deterministic repeated loads,
|
||||
overlapping matches, directories, hard links, contained absolute and escaping
|
||||
symlink directories, required/optional no-match behavior, invalid patterns in empty
|
||||
directories, escaped metacharacters, character classes, visible permission errors,
|
||||
explicit `conf.d` imports without discovery, nonmatching-entry exhaustion, candidate
|
||||
bounds, and filesystem combinations. The
|
||||
[current import contract](../config.md#explicit-imports) owns the completed behavior.
|
||||
Verified `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`,
|
||||
`go mod tidy -diff`, and `git diff --check` with Go 1.26.7 on linux/amd64.
|
||||
|
||||
Read: feature import syntax and source-context API.
|
||||
|
||||
|
||||
@@ -475,30 +475,9 @@ the surviving sequence, not an attempted correspondence with previous indices.
|
||||
|
||||
## Explicit imports and source identity
|
||||
|
||||
Explicit path-import traversal, source identity, containment, budgets, recursive
|
||||
directives, and the temporary glob gate are implemented and owned by the
|
||||
[current configuration contract](../config.md#explicit-path-imports). Bounded glob
|
||||
expansion remains target behavior. Illustrative future fragment:
|
||||
|
||||
```yaml
|
||||
_configkit:
|
||||
imports:
|
||||
- glob: fragments/outputs/*.yml
|
||||
optional: true
|
||||
```
|
||||
|
||||
- Expand declarations in list order, sorting each glob's matches
|
||||
lexicographically by slash-separated relative path, with no locale sorting.
|
||||
Use the slash-based [Go `path.Match` grammar](https://pkg.go.dev/path#Match)
|
||||
(`*`, `?`, character classes, and backslash escapes), with recursive `**`
|
||||
syntax forbidden. Backslashes escape pattern characters, not directory separators.
|
||||
- A required unmatched glob is an error. Optional suppresses only zero matches.
|
||||
Invalid patterns, permission
|
||||
failures, directories, and invalid matched documents remain errors.
|
||||
- Glob traversal counts examined directory entries, including nonmatches, against
|
||||
its separate work budget; only accepted matches contribute to sorted source order.
|
||||
Expansion remains pattern-directed and must preserve the current path-import
|
||||
identity, containment, repetition, and depth-first traversal rules.
|
||||
Explicit path and glob imports, recursive traversal, source identity, containment,
|
||||
filesystem context, and budgets are implemented and now owned by the
|
||||
[current configuration contract](../config.md#explicit-imports).
|
||||
|
||||
## Profiles
|
||||
|
||||
|
||||
Reference in New Issue
Block a user