Expose native filesystem sources through public loading API

This commit is contained in:
2026-09-29 17:36:20 +00:00
parent 6111cb23a1
commit 1ff3c812a7
18 changed files with 430 additions and 36 deletions

View File

@@ -43,10 +43,10 @@ func toEngineLocations(values []Location) []engine.Location {
return result
}
func fromEngineSourceInfo(value engine.SourceInfo) SourceInfo {
return SourceInfo{ID: value.ID, Label: value.Label, Path: value.Path, Layer: Layer(value.Layer), ImportedFrom: fromEngineLocationPointer(value.ImportedFrom)}
return SourceInfo{ID: value.ID, Label: value.Label, Path: value.Path, Layer: Layer(value.Layer), Kind: SourceKind(value.Kind), ImportedFrom: fromEngineLocationPointer(value.ImportedFrom)}
}
func toEngineSourceInfo(value SourceInfo) engine.SourceInfo {
return engine.SourceInfo{ID: value.ID, Label: value.Label, Path: value.Path, Layer: engine.Layer(value.Layer), ImportedFrom: toEngineLocationPointer(value.ImportedFrom)}
return engine.SourceInfo{ID: value.ID, Label: value.Label, Path: value.Path, Layer: engine.Layer(value.Layer), Kind: engine.SourceKind(value.Kind), ImportedFrom: toEngineLocationPointer(value.ImportedFrom)}
}
func fromEngineOrigin(value engine.Origin) Origin {
return Origin{Layer: Layer(value.Layer), Locations: fromEngineLocations(value.Locations), Operation: ChangeKind(value.Operation)}

View File

@@ -12,13 +12,14 @@ ConfigKit exposes opaque sources through these constructors:
| Constructor | Current contract |
| --- | --- |
| `File(path)` | Requires a nonempty path, opens a regular file for each consumption, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. |
| `NewFilesystem(label, filesystem).File(name)` | Borrows a caller-owned `fs.FS` under a safe nonempty display label. `name` must be a non-root slash-relative `fs.ValidPath`. Each load reopens the selected regular file; copies of the `Filesystem` preserve namespace identity. Separate constructors create separate namespaces even for the same label or filesystem. |
| `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. |
| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. |
| `NoSource()` | Selects defaults-only loading, creates no source record, and marks no paths explicit. |
The zero `Source` is invalid. Constructor argument failures are retained inside the
opaque source and become safe structured diagnostics when the source is consumed.
File sources can observe file changes between consumptions; byte sources cannot.
File and filesystem sources can observe changes between consumptions; byte sources cannot.
Each source is limited to 8 MiB. A load using imports or an active overlay is limited
to 64 MiB across all accepted sources, 1,024 accepted sources, 64 active import
@@ -50,7 +51,7 @@ validate options → parse the selected source → traverse/compose enabled impo
→ snapshot configuration and metadata
```
`File`, `Bytes`, and `Reader` each contribute the first base source with deterministic
`File`, `Filesystem.File`, `Bytes`, and `Reader` each contribute the first base source with deterministic
ID `source-1`; enabled imports continue in depth-first order as `source-2` onward.
`NoSource` skips parsing, supplies an absent version to a configured version checker,
contributes no source, and continues through defaults, decode, and projection. A
@@ -66,7 +67,11 @@ disabled.
Every source, options, schema, version, defaults, decode, resolution, or validation
failure returns a nil result and a `*LoadError`. Successful results contain caller-owned `Config`, detached
`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Mutating
`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Each source
record's `Kind` identifies file, filesystem, bytes, reader, or CLI acquisition
independently of its precedence `Layer`. Host file paths remain canonical;
filesystem paths are slash-relative within their namespace; synthetic inputs have
no path. Mutating
the returned configuration or inspection records cannot alter ConfigKit's private
snapshot. ConfigKit performs no logging. `Result.WriteEffective` exposes only the
frozen redacted projection.
@@ -79,7 +84,8 @@ validation, and configuration/metadata snapshotting have completed. A reported
load failure returns diagnostics and a nil result, as described above.
Before returning on success or failure, ConfigKit closes the filesystem handles
it opened for loading, including files and directory/root handles. The returned
it opened for loading, including files and directory/root handles. Caller-owned
filesystems are borrowed, never closed. The returned
configuration and inspection metadata require no close operation and remain
usable independently of the source files. A supplied `Reader` remains
caller-owned and is never closed by ConfigKit.
@@ -582,18 +588,25 @@ walk failures, matched directories, invalid documents, escapes, and repeated tar
remain errors. Final candidates are bounded by the remaining source limit before
their contents are accumulated.
The configuration root defaults to a selected file's canonical directory. A file
source may supply `FilesystemContext.Root` to widen that root but may not supply
`BaseDir`. Whenever byte and reader sources supply filesystem context, they must
provide both `BaseDir` and `Root` as absolute existing directories; their diagnostic
labels, including URI-looking labels, never imply filesystem paths. The base and
selected file must be inside the canonical root. Nonregular targets are rejected before opening to avoid blocking on FIFOs;
For host files, the configuration root defaults to the selected file's canonical
directory. A host file may supply `FilesystemContext.Root` to widen that root but
may not supply `BaseDir` or a native `FS`. Byte and reader inputs with host context
must provide both `BaseDir` and `Root` as absolute existing directories.
For `Filesystem.File`, the default root is `.` in its namespace and the base is
the selected file's directory. An explicit context must use the same `Filesystem`
namespace and may set `Root` to contain the file and its base; it may not supply
`BaseDir`. Byte and reader inputs with native context provide both `BaseDir` and
`Root` as slash-relative `fs.ValidPath` directories in `FS` (`.` means its root),
with the base inside the root. Context paths are checked even when imports and
overlays are inactive. Diagnostic labels, including URI-looking labels, never
imply filesystem paths. Nonregular targets are rejected before opening to avoid blocking on FIFOs;
post-open regular-file checks remain in place under the stable-input assumption.
ConfigKit canonicalizes symlinks,
checks containment with relative paths, and opens the canonical root-relative target
through a library-owned root handle. Contained relative or absolute symlink targets
are accepted; escapes are rejected. Canonical paths define identity, so hard links at
distinct paths remain distinct sources.
distinct paths remain distinct sources. Native filesystems use slash-path operations
and their own symlink behavior; imports stay in their selected namespace and root.
An active-stack repeat is `import.cycle` and reports the complete cycle's import
sites. A previously visited canonical path is `import.repeated` and reports both

View File

@@ -312,6 +312,14 @@ everyday checker. No public constructors land until the next stage.
## Stage 4: Expose native filesystem sources through the facade
**Completed 2026-09-29.** The root API now exposes reusable native filesystem
namespaces, namespace-aware contexts, and source acquisition kinds. Ordinary
loads support native files, opt-in imports, selected overlays, and byte/reader
contexts while retaining host semantics. Public tests cover embedded fixtures,
`fstest.MapFS`, same-label namespace isolation, wrapper copies and reuse,
constructor/context diagnostics, and host/CLI source kinds. The everyday checker
passed, including public-boundary and documentation checks.
1. Add `Filesystem`, `NewFilesystem`, `Filesystem.File`, and the additive FS
context field at root and engine boundaries. Constructors follow deferred
diagnostics; copying the wrapper preserves namespace identity.

142
facade_filesystem_test.go Normal file
View File

@@ -0,0 +1,142 @@
package configkit_test
import (
"embed"
"io/fs"
"reflect"
"strconv"
"strings"
"testing"
"testing/fstest"
configkit "gitea.maximumdirect.net/eric/ConfigKit"
)
//go:embed testdata/filesystem
var filesystemFixtures embed.FS
type filesystemFacadeConfig struct {
Items map[string]string `yaml:"items"`
}
func TestFacadeFilesystemSourceImportsOverlayAndReuse(t *testing.T) {
fixture, err := fs.Sub(filesystemFixtures, "testdata/filesystem")
if err != nil {
t.Fatal(err)
}
namespace := configkit.NewFilesystem("embedded", fixture)
copyOfNamespace := namespace
source := namespace.File("config/main.yml")
for range 2 {
result, err := configkit.Load(source, configkit.Options[filesystemFacadeConfig]{
Imports: true, Overlay: "testing", Filesystem: &configkit.FilesystemContext{FS: &copyOfNamespace},
})
if err != nil {
t.Fatal(err)
}
if want := (map[string]string{"main": "selected", "shared": "default"}); !reflect.DeepEqual(result.Config.Items, want) {
t.Fatalf("items = %#v, want %#v", result.Config.Items, want)
}
sources := result.Sources()
if len(sources) != 3 {
t.Fatalf("sources = %#v", sources)
}
for index, want := range []string{"config/main.yml", "shared.yml", "overlays/testing.yml"} {
if sources[index].ID != "source-"+strconv.Itoa(index+1) || sources[index].Path != want || sources[index].Kind != configkit.SourceKindFilesystem {
t.Errorf("source %d = %#v", index, sources[index])
}
if sources[index].Label != "embedded/"+want {
t.Errorf("source label = %q", sources[index].Label)
}
}
if sources[1].ImportedFrom == nil || sources[1].ImportedFrom.SourceID != sources[0].ID || sources[1].ImportedFrom.Line == 0 {
t.Fatalf("import location = %#v", sources[1].ImportedFrom)
}
}
}
func TestFacadeFilesystemContextsAndIdentity(t *testing.T) {
first := configkit.NewFilesystem("same", fstest.MapFS{
"root/config/main.yml": &fstest.MapFile{Data: []byte("configkit: {imports: [{path: ../shared.yml}]}\n")},
"root/shared.yml": &fstest.MapFile{Data: []byte("items: {value: first}\n")},
"root/overlays/testing.yml": &fstest.MapFile{Data: []byte("items: {overlay: first}\n")},
})
second := configkit.NewFilesystem("same", fstest.MapFS{
"root/config/main.yml": &fstest.MapFile{Data: []byte("items: {value: second}\n")},
})
context := &configkit.FilesystemContext{Root: "root", FS: &first}
result, err := configkit.Load(first.File("root/config/main.yml"), configkit.Options[filesystemFacadeConfig]{Imports: true, Overlay: "testing", Filesystem: context})
if err != nil || result.Config.Items["value"] != "first" || result.Config.Items["overlay"] != "first" {
t.Fatalf("contained load = %#v, %v", result, err)
}
result, err = configkit.Load[filesystemFacadeConfig](second.File("root/config/main.yml"))
if err != nil || result.Config.Items["value"] != "second" {
t.Fatalf("same-label independent namespace = %#v, %v", result, err)
}
checkCode(t, second.File("root/config/main.yml"), configkit.Options[filesystemFacadeConfig]{Filesystem: context}, configkit.CodeOptionsInvalid)
checkCode(t, first.File("root/config/main.yml"), configkit.Options[filesystemFacadeConfig]{Filesystem: &configkit.FilesystemContext{Root: "root/config", FS: &first}, Imports: true}, configkit.CodeImportOutsideRoot)
checkCode(t, configkit.File("testdata/filesystem/config/main.yml"), configkit.Options[filesystemFacadeConfig]{Filesystem: context}, configkit.CodeOptionsInvalid)
for _, input := range []struct {
name string
source configkit.Source
kind configkit.SourceKind
}{
{"bytes", configkit.Bytes("memory", []byte("configkit: {imports: [{path: shared.yml}]}\n")), configkit.SourceKindBytes},
{"reader", configkit.Reader("stream", strings.NewReader("configkit: {imports: [{path: shared.yml}]}\n")), configkit.SourceKindReader},
} {
t.Run(input.name, func(t *testing.T) {
result, err := configkit.Load(input.source, configkit.Options[filesystemFacadeConfig]{Imports: true, Filesystem: &configkit.FilesystemContext{Root: "root", BaseDir: "root", FS: &first}})
if err != nil || result.Config.Items["value"] != "first" {
t.Fatalf("synthetic import = %#v, %v", result, err)
}
if got := result.Sources(); len(got) != 2 || got[0].Kind != input.kind || got[0].Path != "" || got[1].Kind != configkit.SourceKindFilesystem {
t.Fatalf("sources = %#v", got)
}
})
}
checkCode(t, configkit.Bytes("memory", []byte("items: {}\n")), configkit.Options[filesystemFacadeConfig]{Filesystem: &configkit.FilesystemContext{Root: "root", BaseDir: ".", FS: &first}}, configkit.CodeImportOutsideRoot)
result, err = configkit.Load(configkit.Bytes("memory", []byte("items: {value: main}\n")), configkit.Options[filesystemFacadeConfig]{
Overlay: "testing", Filesystem: &configkit.FilesystemContext{Root: "root", BaseDir: "root", FS: &first},
})
if err != nil || result.Config.Items["value"] != "main" || result.Config.Items["overlay"] != "first" {
t.Fatalf("synthetic selected overlay = %#v, %v", result, err)
}
}
func TestFacadeSourceKindsIncludeHostAndCLI(t *testing.T) {
result, err := configkit.Load(configkit.File("testdata/filesystem/shared.yml"), configkit.Options[filesystemFacadeConfig]{
Overrides: []configkit.Override{{Path: configkit.Keys("items", "shared"), Value: "changed", Flag: "--shared"}},
})
if err != nil {
t.Fatal(err)
}
sources := result.Sources()
if len(sources) != 2 || sources[0].Kind != configkit.SourceKindFile || sources[0].Path == "" || sources[1].Kind != configkit.SourceKindCLI || sources[1].Path != "" {
t.Fatalf("host and CLI sources = %#v", sources)
}
}
func TestFacadeFilesystemConstructorDiagnostics(t *testing.T) {
var nilFilesystem fs.FS
var zero configkit.Filesystem
for _, source := range []configkit.Source{
zero.File("main.yml"),
configkit.NewFilesystem("", fstest.MapFS{}).File("main.yml"),
configkit.NewFilesystem("bad\nlabel", fstest.MapFS{}).File("main.yml"),
configkit.NewFilesystem("good", nilFilesystem).File("main.yml"),
configkit.NewFilesystem("good", fstest.MapFS{}).File("."),
configkit.NewFilesystem("good", fstest.MapFS{}).File("../outside.yml"),
} {
checkCode(t, source, configkit.Options[filesystemFacadeConfig]{}, configkit.CodeSourceInvalid)
}
}
func checkCode(t *testing.T, source configkit.Source, options configkit.Options[filesystemFacadeConfig], want configkit.Code) {
t.Helper()
_, err := configkit.Load(source, options)
loadError, ok := err.(*configkit.LoadError)
if !ok || len(loadError.Diagnostics()) == 0 || loadError.Diagnostics()[0].Code != want {
t.Fatalf("error = %#v, want %s", err, want)
}
}

View File

@@ -1,12 +1,21 @@
package engine
import (
"io/fs"
"os"
"path"
"path/filepath"
"strings"
)
func validateFilesystemContext(source Source, context *FilesystemContext, main parsedSource) *Diagnostic {
if nativeContext(source, context) {
access, basePath, mainPath, diagnostic := nativeFilesystemAccess(source, context)
if diagnostic != nil {
return diagnostic
}
return validateNativeDirectories(access, basePath, mainPath)
}
rootPath, _, _, diagnostic := importFilesystem(source, context, main)
if diagnostic != nil {
return diagnostic
@@ -29,20 +38,87 @@ func filesystemOptionDiagnostics(source Source, context *FilesystemContext) []Di
case sourceNone:
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "NoSource cannot use filesystem context")}
case sourceFile:
if context.FS != nil {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "host file sources cannot use a native filesystem context")}
}
if context.BaseDir != "" {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "file sources determine their own import base directory")}
}
case sourceFilesystem:
if context.FS == nil || context.FS.namespace == nil || source.namespace == nil || context.FS.namespace.token != source.namespace.token {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "filesystem file context must use the source namespace")}
}
if context.BaseDir != "" {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "file sources determine their own import base directory")}
}
if context.Root != "" && !fs.ValidPath(context.Root) {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "filesystem root must be a slash-relative valid path")}
}
case sourceBytes, sourceReader:
if context.BaseDir == "" || context.Root == "" {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "byte and reader filesystem context requires base and root directories")}
}
if !filepath.IsAbs(context.BaseDir) || !filepath.IsAbs(context.Root) {
if context.FS != nil {
if context.FS.namespace == nil || !fs.ValidPath(context.BaseDir) || !fs.ValidPath(context.Root) {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "filesystem base and root must be slash-relative valid paths")}
}
} else if !filepath.IsAbs(context.BaseDir) || !filepath.IsAbs(context.Root) {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "byte and reader filesystem base and root directories must be absolute")}
}
}
return nil
}
func nativeContext(source Source, context *FilesystemContext) bool {
return source.kind == sourceFilesystem || context != nil && context.FS != nil
}
// nativeFilesystemAccess resolves namespace context without opening caller
// resources. File inputs always derive their base from the selected file.
func nativeFilesystemAccess(source Source, context *FilesystemContext) (*filesystemAccess, string, string, *Diagnostic) {
var namespace *nativeNamespace
rootPath := "."
basePath := "."
mainPath := ""
if source.kind == sourceFilesystem {
namespace = source.namespace
mainPath = source.path
basePath = path.Dir(source.path)
if context != nil && context.Root != "" {
rootPath = context.Root
}
} else if context != nil && context.FS != nil {
namespace = context.FS.namespace
rootPath, basePath = context.Root, context.BaseDir
}
if namespace == nil || nilFilesystem(namespace.filesystem) || !fs.ValidPath(rootPath) || !fs.ValidPath(basePath) {
value := optionDiagnostic(CodeOptionsInvalid, "filesystem context is invalid")
return nil, "", "", &value
}
access := newNativeAccess(namespace.filesystem, rootPath, namespace.token)
access.label = namespace.label
if !access.contains(basePath) || mainPath != "" && !access.contains(mainPath) {
value := importDiagnostic(CodeImportOutsideRoot, "selected source and import base must stay inside the configuration root", nil)
return nil, "", "", &value
}
return access, basePath, mainPath, nil
}
func validateNativeDirectories(access *filesystemAccess, basePath, mainPath string) *Diagnostic {
if !access.contains(basePath) || mainPath != "" && !access.contains(mainPath) {
value := importDiagnostic(CodeImportOutsideRoot, "selected source and import base must stay inside the configuration root", nil)
return &value
}
for _, directory := range []string{access.rootPath, basePath} {
info, err := access.stat(directory)
if err != nil || !info.IsDir() {
value := importDiagnostic(CodeImportInvalid, "configuration root and import base must be directories", nil)
return &value
}
}
return nil
}
func importFilesystem(source Source, context *FilesystemContext, main parsedSource) (rootPath, basePath, mainPath string, diagnostic *Diagnostic) {
if main.source == nil || main.document == nil {
value := importDiagnostic(CodeImportInvalid, "imports require a selected configuration source", nil)

View File

@@ -37,6 +37,7 @@ type filesystemAccess struct {
root *os.Root
native fs.FS
token *namespaceToken
label string
}
func openHostAccess(rootPath string) (*filesystemAccess, error) {

View File

@@ -58,9 +58,12 @@ type importTraversal struct {
}
func loadImportedDocuments(source Source, context *FilesystemContext, main parsedSource, rule *VersionRule, limits importLimits) (*composedDocument, []SourceInfo, []Diagnostic) {
if source.kind == sourceFilesystem {
access := newNativeAccess(source.namespace.filesystem, ".", source.namespace.token)
return loadImportedWithAccess(access, access.dir(source.path), source.path, main, rule, limits, &importAccounting{})
if nativeContext(source, context) {
access, basePath, mainPath, diagnostic := nativeFilesystemAccess(source, context)
if diagnostic != nil {
return nil, sourceInfos(main.source), []Diagnostic{*diagnostic}
}
return loadImportedWithAccess(access, basePath, mainPath, main, rule, limits, &importAccounting{})
}
rootPath, basePath, mainPath, diagnostic := importFilesystem(source, context, main)
if diagnostic != nil {
@@ -231,7 +234,13 @@ func (s *importTraversal) visitPath(basePath string, declaration importDeclarati
return []Diagnostic{importDiagnostic(CodeImportOutsideRoot, "import target could not be addressed inside the configuration root", &location)}
}
sourceID := "source-" + strconv.Itoa(s.accounting.sources+1)
info := SourceInfo{ID: sourceID, Label: declaration.path, Path: canonical, Layer: LayerBase, ImportedFrom: &location}
kind := SourceKindFile
label := declaration.path
if s.access.native != nil {
kind = SourceKindFilesystem
label = s.access.label + "/" + canonical
}
info := SourceInfo{ID: sourceID, Label: label, Path: canonical, Layer: LayerBase, Kind: kind, ImportedFrom: &location}
s.sources = append(s.sources, info)
data, skipped, diagnostic := s.readImport(canonical, sourceID, declaration.optional)
if diagnostic != nil {

View File

@@ -33,6 +33,7 @@ type Options[T any] struct {
type FilesystemContext struct {
BaseDir string
Root string
FS *Filesystem
}
// Override describes one explicitly supplied application override. Value uses

View File

@@ -44,7 +44,7 @@ func prepareOverrides(base *composedDocument, sources []SourceInfo, overrides []
allSources := cloneSourceInfos(sources)
for index, override := range overrides {
sourceID := "source-" + strconv.Itoa(len(sources)+index+1)
info := SourceInfo{ID: sourceID, Label: override.Flag, Layer: LayerCLI}
info := SourceInfo{ID: sourceID, Label: override.Flag, Layer: LayerCLI, Kind: SourceKindCLI}
allSources = append(allSources, info)
prepared[index].source = info
}

View File

@@ -11,6 +11,18 @@ const (
LayerResolution Layer = "resolution"
)
// SourceKind identifies source acquisition independently of precedence.
type SourceKind string
const (
SourceKindFile SourceKind = "file"
SourceKindFilesystem SourceKind = "filesystem"
SourceKindBytes SourceKind = "bytes"
SourceKindReader SourceKind = "reader"
SourceKindMapping SourceKind = "mapping"
SourceKindCLI SourceKind = "cli"
)
// ChangeKind identifies a recorded operation. The zero value is
// used for input and default origins.
type ChangeKind string
@@ -156,14 +168,16 @@ type Location struct {
Column int
}
// SourceInfo describes a source discovered during a load. Path is a canonical
// local path when one exists. ImportedFrom identifies the declaration that
// included this source.
// SourceInfo describes a source discovered during a load. Kind identifies
// acquisition independently of Layer. Path is a canonical host path or a
// namespace-relative filesystem path; synthetic sources have no Path.
// ImportedFrom identifies the declaration that included this source.
type SourceInfo struct {
ID string
Label string
Path string
Layer Layer
Kind SourceKind
ImportedFrom *Location
}

View File

@@ -38,8 +38,14 @@ func loadSelectedOverlayDocument(source Source, context *FilesystemContext, main
}
func loadSelectedOverlayInto(source Source, context *FilesystemContext, main parsedSource, base *assemblyAccumulator, sources []SourceInfo, name string, rule *VersionRule, limits importLimits) ([]SourceInfo, []Diagnostic) {
if source.kind == sourceFilesystem {
access := newNativeAccess(source.namespace.filesystem, ".", source.namespace.token)
if nativeContext(source, context) {
access, basePath, mainPath, diagnostic := nativeFilesystemAccess(source, context)
if diagnostic != nil {
return cloneSourceInfos(sources), []Diagnostic{*diagnostic}
}
if diagnostic := validateNativeDirectories(access, basePath, mainPath); diagnostic != nil {
return cloneSourceInfos(sources), []Diagnostic{*diagnostic}
}
return loadSelectedOverlayWithAccess(access, base, sources, name, rule, limits)
}
rootPath, _, _, diagnostic := importFilesystem(source, context, main)
@@ -92,7 +98,13 @@ func loadSelectedOverlayWithAccess(access *filesystemAccess, base *assemblyAccum
sourceID := "source-" + strconv.Itoa(len(sources)+1)
overlayLocation := &Location{SourceID: sourceID}
overlaySource := SourceInfo{ID: sourceID, Label: "overlays/" + name + ".yml", Path: canonical, Layer: LayerOverlay}
kind := SourceKindFile
label := "overlays/" + name + ".yml"
if access.native != nil {
kind = SourceKindFilesystem
label = access.label + "/" + canonical
}
overlaySource := SourceInfo{ID: sourceID, Label: label, Path: canonical, Layer: LayerOverlay, Kind: kind}
sources = append(cloneSourceInfos(sources), overlaySource)
data, diagnostic := readOverlayFile(access, canonical, sourceID, limits.bytes-base.current.inputBytes)
if diagnostic != nil {

View File

@@ -16,8 +16,8 @@ const (
maxTotalBytes = 64 << 20
)
// Source is an opaque configuration input. Construct sources with File, Bytes,
// Reader, or NoSource; the zero value is invalid.
// Source is an opaque configuration input. Construct sources with File,
// Filesystem.File, Bytes, Reader, or NoSource; the zero value is invalid.
type Source struct {
kind sourceKind
path string
@@ -47,8 +47,52 @@ const (
creationInvalidLabel
creationInvalidReader
creationTooLarge
creationInvalidFilesystem
creationInvalidFilesystemLabel
creationInvalidFilesystemPath
)
// Filesystem is a reusable opaque namespace for caller-owned fs.FS inputs.
// Copies preserve the namespace identity used for explicit contexts.
type Filesystem struct {
namespace *nativeNamespace
createErr sourceCreationError
}
// NewFilesystem records a borrowed filesystem without opening it. Invalid
// arguments are reported when a File source is consumed.
func NewFilesystem(label string, filesystem fs.FS) Filesystem {
result := Filesystem{}
if !validOverrideLabel(label) {
result.createErr = creationInvalidFilesystemLabel
} else if nilFilesystem(filesystem) {
result.createErr = creationInvalidFilesystem
} else {
result.namespace = newNativeNamespace(label, filesystem)
}
return result
}
// File selects a non-root slash-relative file within this namespace.
func (f Filesystem) File(name string) Source {
if f.createErr != creationValid {
return Source{kind: sourceFilesystem, createErr: f.createErr}
}
return filesystemFile(f.namespace, name)
}
func nilFilesystem(filesystem fs.FS) bool {
if filesystem == nil {
return true
}
value := reflect.ValueOf(filesystem)
switch value.Kind() {
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return value.IsNil()
}
return false
}
type readerSource struct {
input io.Reader
used atomic.Bool
@@ -63,8 +107,11 @@ func File(path string) Source {
}
func filesystemFile(namespace *nativeNamespace, name string) Source {
if namespace == nil || namespace.filesystem == nil || name == "." || !fs.ValidPath(name) {
return Source{kind: sourceFilesystem, createErr: creationInvalidPath}
if namespace == nil || nilFilesystem(namespace.filesystem) {
return Source{kind: sourceFilesystem, createErr: creationInvalidFilesystem}
}
if name == "." || !fs.ValidPath(name) {
return Source{kind: sourceFilesystem, createErr: creationInvalidFilesystemPath}
}
return Source{kind: sourceFilesystem, path: name, label: namespace.label + "/" + name, namespace: namespace}
}
@@ -132,7 +179,7 @@ func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead
if limit < 0 {
limit = 0
}
info := &SourceInfo{ID: sourceID, Label: source.label, Layer: LayerBase}
info := &SourceInfo{ID: sourceID, Label: source.label, Layer: LayerBase, Kind: sourceSourceKind(source.kind)}
var input io.Reader
switch source.kind {
@@ -159,6 +206,7 @@ func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead
if err != nil {
return sourceRead{info: info}, []Diagnostic{classifySourceError(err, sourceID)}
}
info.Path = canonical
data, failure := readRegularFile(canonical, access.stat, access.open, limit)
if failure != nil {
return sourceRead{info: info}, []Diagnostic{sourceReadDiagnostic(failure, sourceID, CodeSourceLimitExceeded, "source exceeds the configured byte limit")}
@@ -199,11 +247,32 @@ func sourceConstructorDiagnostic(source Source) *Diagnostic {
case creationTooLarge:
code = CodeSourceLimitExceeded
message = "source exceeds the configured byte limit"
case creationInvalidFilesystem:
message = "filesystem must not be nil"
case creationInvalidFilesystemLabel:
message = "filesystem label must be nonempty and safe to display"
case creationInvalidFilesystemPath:
message = "filesystem source path must be a non-root slash-relative valid path"
}
diagnostic := sourceDiagnostic(code, "", message)
return &diagnostic
}
func sourceSourceKind(kind sourceKind) SourceKind {
switch kind {
case sourceFile:
return SourceKindFile
case sourceFilesystem:
return SourceKindFilesystem
case sourceBytes:
return SourceKindBytes
case sourceReader:
return SourceKindReader
default:
return ""
}
}
func classifySourceError(err error, sourceID string) Diagnostic {
switch {
case errors.Is(err, fs.ErrNotExist):

View File

@@ -28,10 +28,14 @@ type Options[T any] struct {
Overrides []Override
}
// FilesystemContext supplies filesystem roots for imports and overlays.
// FilesystemContext supplies roots for imports and overlays. With FS nil,
// BaseDir and Root use absolute host paths. With FS set, they use slash-relative
// paths in that namespace ("." denotes its root). File sources derive BaseDir
// from the selected file; a native file's explicit Root limits its scope.
type FilesystemContext struct {
BaseDir string
Root string
FS *Filesystem
}
// Override describes one explicitly supplied application override. Value uses
@@ -223,6 +227,10 @@ func toEngineOptions[T any](options Options[T]) engine.Options[T] {
}
if options.Filesystem != nil {
result.Filesystem = &engine.FilesystemContext{BaseDir: options.Filesystem.BaseDir, Root: options.Filesystem.Root}
if options.Filesystem.FS != nil {
filesystem := options.Filesystem.FS.value
result.Filesystem.FS = &filesystem
}
}
if options.Overrides != nil {
result.Overrides = make([]engine.Override, len(options.Overrides))

View File

@@ -11,6 +11,19 @@ const (
LayerResolution Layer = "resolution"
)
// SourceKind identifies how a configuration source was acquired, independent
// of its precedence layer.
type SourceKind string
const (
SourceKindFile SourceKind = "file"
SourceKindFilesystem SourceKind = "filesystem"
SourceKindBytes SourceKind = "bytes"
SourceKindReader SourceKind = "reader"
SourceKindMapping SourceKind = "mapping"
SourceKindCLI SourceKind = "cli"
)
// ChangeKind identifies a recorded operation. The zero value is
// used for input and default origins.
type ChangeKind string
@@ -156,14 +169,16 @@ type Location struct {
Column int
}
// SourceInfo describes a source discovered during a load. Path is a canonical
// local path when one exists. ImportedFrom identifies the declaration that
// included this source.
// SourceInfo describes a source discovered during a load. Kind identifies its
// acquisition, independent of Layer. Path is canonical for a host file and
// namespace-relative for a filesystem file; synthetic sources have no Path.
// ImportedFrom identifies the declaration that included this source.
type SourceInfo struct {
ID string
Label string
Path string
Layer Layer
Kind SourceKind
ImportedFrom *Location
}

View File

@@ -2,14 +2,31 @@ package configkit
import (
"io"
"io/fs"
"gitea.maximumdirect.net/eric/ConfigKit/internal/engine"
)
// Source is an opaque configuration input. Construct sources with File, Bytes,
// Reader, or NoSource; the zero value is invalid.
// Source is an opaque configuration input. Construct sources with File,
// Filesystem.File, Bytes, Reader, or NoSource; the zero value is invalid.
type Source struct{ value engine.Source }
// Filesystem is a reusable opaque namespace for a caller-owned fs.FS. Copies
// preserve identity; separately constructed values are distinct even if their
// labels or underlying filesystems match. The filesystem is never closed.
type Filesystem struct{ value engine.Filesystem }
// NewFilesystem borrows filesystem without opening it. Label must be nonempty
// and safe to display; a nil filesystem or invalid label is reported when a
// source from the returned namespace is consumed.
func NewFilesystem(label string, filesystem fs.FS) Filesystem {
return Filesystem{value: engine.NewFilesystem(label, filesystem)}
}
// File selects a non-root fs.ValidPath name in this filesystem. Each load
// reopens the file; constructor problems are reported when the source is used.
func (f Filesystem) File(name string) Source { return Source{value: f.value.File(name)} }
// File returns a source that reopens path for every load attempt.
func File(path string) Source { return Source{value: engine.File(path)} }

5
testdata/filesystem/config/main.yml vendored Normal file
View File

@@ -0,0 +1,5 @@
configkit:
imports:
- path: ../shared.yml
items:
main: base

View File

@@ -0,0 +1,2 @@
items:
main: selected

2
testdata/filesystem/shared.yml vendored Normal file
View File

@@ -0,0 +1,2 @@
items:
shared: default