Add snapshotted mapping configuration sources

This commit is contained in:
2026-09-29 17:43:19 +00:00
parent 1ff3c812a7
commit 90c6e8ed7e
12 changed files with 454 additions and 159 deletions

View File

@@ -15,11 +15,18 @@ ConfigKit exposes opaque sources through these constructors:
| `NewFilesystem(label, filesystem).File(name)` | Borrows a caller-owned `fs.FS` under a safe nonempty display label. `name` must be a non-root slash-relative `fs.ValidPath`. Each load reopens the selected regular file; copies of the `Filesystem` preserve namespace identity. Separate constructors create separate namespaces even for the same label or filesystem. | | `NewFilesystem(label, filesystem).File(name)` | Borrows a caller-owned `fs.FS` under a safe nonempty display label. `name` must be a non-root slash-relative `fs.ValidPath`. Each load reopens the selected regular file; copies of the `Filesystem` preserve namespace identity. Separate constructors create separate namespaces even for the same label or filesystem. |
| `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. | | `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. |
| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. | | `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. |
| `Mapping(label, values)` | Requires a safe nonempty label and a non-nil root `map[string]any`. Snapshots supported built-in values into an owned mapping document at construction; `{}` is valid. Constructor conversion failures become safe deferred diagnostics. |
| `NoSource()` | Selects defaults-only loading, creates no source record, and marks no paths explicit. | | `NoSource()` | Selects defaults-only loading, creates no source record, and marks no paths explicit. |
The zero `Source` is invalid. Constructor argument failures are retained inside the The zero `Source` is invalid. Constructor argument failures are retained inside the
opaque source and become safe structured diagnostics when the source is consumed. opaque source and become safe structured diagnostics when the source is consumed.
File and filesystem sources can observe changes between consumptions; byte sources cannot. File and filesystem sources can observe changes between consumptions; byte and
mapping sources cannot. Mapping accepts nil, built-in string, boolean, signed or
unsigned integer, finite float, `[]any`, and `map[string]any` values recursively.
Typed nil maps and slices represent explicit null; empty collections remain empty.
Keys are sorted for stable provenance, integer decimal spelling stays exact, and
cycles or unsupported types are rejected without exposing values. Mapping input
uses the same directive and version rules as YAML, not CLI override restrictions.
Each source is limited to 8 MiB. A load using imports or an active overlay is limited Each source is limited to 8 MiB. A load using imports or an active overlay is limited
to 64 MiB across all accepted sources, 1,024 accepted sources, 64 active import to 64 MiB across all accepted sources, 1,024 accepted sources, 64 active import
@@ -27,6 +34,8 @@ edges, and 1,000,000 input nodes in aggregate. Glob expansion separately permits
1,000,000 visited directory entries, including nonmatches. Reads request at most the 1,000,000 visited directory entries, including nonmatches. Reads request at most the
remaining allowance plus one byte so excess data is rejected rather than silently remaining allowance plus one byte so excess data is rejected rather than silently
truncated. Data returned together with a read error is not accepted as partial input. truncated. Data returned together with a read error is not accepted as partial input.
For mapping sources, key and string bytes use the same accounting as programmatic
overrides, and the converted tree uses the same node and depth bounds.
## Single-source loading ## Single-source loading
@@ -51,7 +60,7 @@ validate options → parse the selected source → traverse/compose enabled impo
→ snapshot configuration and metadata → snapshot configuration and metadata
``` ```
`File`, `Filesystem.File`, `Bytes`, and `Reader` each contribute the first base source with deterministic `File`, `Filesystem.File`, `Bytes`, `Reader`, and `Mapping` each contribute the first base source with deterministic
ID `source-1`; enabled imports continue in depth-first order as `source-2` onward. ID `source-1`; enabled imports continue in depth-first order as `source-2` onward.
`NoSource` skips parsing, supplies an absent version to a configured version checker, `NoSource` skips parsing, supplies an absent version to a configured version checker,
contributes no source, and continues through defaults, decode, and projection. A contributes no source, and continues through defaults, decode, and projection. A
@@ -68,7 +77,7 @@ disabled.
Every source, options, schema, version, defaults, decode, resolution, or validation Every source, options, schema, version, defaults, decode, resolution, or validation
failure returns a nil result and a `*LoadError`. Successful results contain caller-owned `Config`, detached failure returns a nil result and a `*LoadError`. Successful results contain caller-owned `Config`, detached
`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Each source `Metadata`, a defensive source list, and defensive nonfatal diagnostics. Each source
record's `Kind` identifies file, filesystem, bytes, reader, or CLI acquisition record's `Kind` identifies file, filesystem, bytes, reader, mapping, or CLI acquisition
independently of its precedence `Layer`. Host file paths remain canonical; independently of its precedence `Layer`. Host file paths remain canonical;
filesystem paths are slash-relative within their namespace; synthetic inputs have filesystem paths are slash-relative within their namespace; synthetic inputs have
no path. Mutating no path. Mutating
@@ -590,12 +599,12 @@ their contents are accumulated.
For host files, the configuration root defaults to the selected file's canonical For host files, the configuration root defaults to the selected file's canonical
directory. A host file may supply `FilesystemContext.Root` to widen that root but directory. A host file may supply `FilesystemContext.Root` to widen that root but
may not supply `BaseDir` or a native `FS`. Byte and reader inputs with host context may not supply `BaseDir` or a native `FS`. Byte, reader, and mapping inputs with host context
must provide both `BaseDir` and `Root` as absolute existing directories. must provide both `BaseDir` and `Root` as absolute existing directories.
For `Filesystem.File`, the default root is `.` in its namespace and the base is For `Filesystem.File`, the default root is `.` in its namespace and the base is
the selected file's directory. An explicit context must use the same `Filesystem` the selected file's directory. An explicit context must use the same `Filesystem`
namespace and may set `Root` to contain the file and its base; it may not supply namespace and may set `Root` to contain the file and its base; it may not supply
`BaseDir`. Byte and reader inputs with native context provide both `BaseDir` and `BaseDir`. Byte, reader, and mapping inputs with native context provide both `BaseDir` and
`Root` as slash-relative `fs.ValidPath` directories in `FS` (`.` means its root), `Root` as slash-relative `fs.ValidPath` directories in `FS` (`.` means its root),
with the base inside the root. Context paths are checked even when imports and with the base inside the root. Context paths are checked even when imports and
overlays are inactive. Diagnostic labels, including URI-looking labels, never overlays are inactive. Diagnostic labels, including URI-looking labels, never

View File

@@ -336,6 +336,16 @@ their contracts. Run the everyday checker, including public-boundary tests.
## Stage 5: Add snapshotted mapping sources ## Stage 5: Add snapshotted mapping sources
**Completed 2026-09-29.** `Mapping` now snapshots supported built-in values at
construction, retaining safe deferred diagnostics for invalid roots, values,
cycles, nonfinite floats, and limits. A shared private converter preserves CLI
override behavior through a separate adapter. Mapping inputs participate in
ordinary directives, imports, version checks, decoding, and inspection with
their own source kind and synthetic locations. Public tests cover YAML
equivalence, reuse after caller mutation, null/empty/zero distinctions, exact
integers, explicit filesystem context, and safe failures. The everyday checker
and engine race suite passed.
1. Extract reusable value conversion from `override_value.go`; keep CLI-specific 1. Extract reusable value conversion from `override_value.go`; keep CLI-specific
path, label, and diagnostic handling in its adapter. path, label, and diagnostic handling in its adapter.
2. Implement `Mapping` at root and engine boundaries with constructor-time 2. Implement `Mapping` at root and engine boundaries with constructor-time

133
facade_mapping_test.go Normal file
View File

@@ -0,0 +1,133 @@
package configkit_test
import (
"math"
"reflect"
"strings"
"testing"
"testing/fstest"
configkit "gitea.maximumdirect.net/eric/ConfigKit"
)
type mappingFacadeConfig struct {
Enabled bool `yaml:"enabled"`
Count uint64 `yaml:"count"`
Optional *string `yaml:"optional"`
Items []string `yaml:"items"`
Name string `yaml:"name"`
}
func TestFacadeMappingSnapshotsAndMatchesYAML(t *testing.T) {
items := []any{"one", "two"}
values := map[string]any{
"name": "original", "enabled": false, "count": ^uint64(0),
"optional": nil, "items": items,
}
source := configkit.Mapping("programmatic", values)
values["name"] = "mutated"
items[0] = "mutated"
for range 2 {
result, err := configkit.Load[mappingFacadeConfig](source)
if err != nil {
t.Fatal(err)
}
want := mappingFacadeConfig{Count: ^uint64(0), Items: []string{"one", "two"}, Name: "original"}
if !reflect.DeepEqual(result.Config, want) {
t.Fatalf("mapping config = %#v, want %#v", result.Config, want)
}
if sources := result.Sources(); len(sources) != 1 || sources[0].ID != "source-1" || sources[0].Kind != configkit.SourceKindMapping || sources[0].Layer != configkit.LayerBase || sources[0].Path != "" {
t.Fatalf("mapping sources = %#v", sources)
}
info, ok := result.Metadata().Lookup(configkit.Keys("name"))
if !ok || len(info.Origin.Locations) != 1 || info.Origin.Locations[0].SourceID != "source-1" || info.Origin.Locations[0].Line != 0 {
t.Fatalf("mapping name origin = %#v, %v", info, ok)
}
fromYAML, err := configkit.Load[mappingFacadeConfig](configkit.Bytes("equivalent", []byte("enabled: false\ncount: 18446744073709551615\noptional: null\nitems: [one, two]\nname: original\n")))
if err != nil || !reflect.DeepEqual(result.Config, fromYAML.Config) {
t.Fatalf("equivalent YAML = %#v, %v", fromYAML, err)
}
}
}
func TestFacadeMappingNullEmptyAndVersion(t *testing.T) {
result, err := configkit.Load[mappingFacadeConfig](configkit.Mapping("empty", map[string]any{}))
if err != nil || len(result.Sources()) != 1 {
t.Fatalf("empty mapping = %#v, %v", result, err)
}
var nilSequence []any
var nilMap map[string]any
var checked string
result, err = configkit.Load(configkit.Mapping("nullable", map[string]any{
"items": nilSequence, "optional": nilMap, "enabled": false, "count": uint64(0),
}), configkit.Options[mappingFacadeConfig]{Definition: configkit.Definition[mappingFacadeConfig]{
Version: &configkit.VersionRule{Path: configkit.Keys("count"), Check: func(version configkit.Version, validation *configkit.Validation) error {
checked = version.Text
return nil
}},
}})
if err != nil || checked != "0" || result.Config.Items != nil || result.Config.Optional != nil {
t.Fatalf("null and zero mapping = %#v, checked %q, %v", result, checked, err)
}
info, ok := result.Metadata().Lookup(configkit.Keys("items"))
if !ok || !info.Null || !info.Explicit {
t.Fatalf("null metadata = %#v, %v", info, ok)
}
}
func TestFacadeMappingImportsUseExplicitContext(t *testing.T) {
namespace := configkit.NewFilesystem("bundle", fstest.MapFS{
"root/shared.yml": &fstest.MapFile{Data: []byte("name: imported\n")},
})
source := configkit.Mapping("programmatic", map[string]any{
"configkit": map[string]any{"imports": []any{map[string]any{"path": "shared.yml"}}},
"enabled": true,
})
result, err := configkit.Load(source, configkit.Options[mappingFacadeConfig]{
Imports: true, Filesystem: &configkit.FilesystemContext{Root: "root", BaseDir: "root", FS: &namespace},
})
if err != nil || result.Config.Name != "imported" || !result.Config.Enabled {
t.Fatalf("mapping import = %#v, %v", result, err)
}
if sources := result.Sources(); len(sources) != 2 || sources[0].Kind != configkit.SourceKindMapping || sources[1].Kind != configkit.SourceKindFilesystem || sources[1].Layer != configkit.LayerBase {
t.Fatalf("mapping import sources = %#v", sources)
}
checkMappingCode(t, source, configkit.Options[mappingFacadeConfig]{Imports: true}, configkit.CodeOptionsInvalid)
checkMappingCode(t, source, configkit.Options[mappingFacadeConfig]{}, configkit.CodeImportDisabled)
}
func TestFacadeMappingDeferredValueDiagnostics(t *testing.T) {
cycle := map[string]any{}
cycle["secret"] = cycle
for _, test := range []struct {
name string
source configkit.Source
want configkit.Code
path string
}{
{"nil root", configkit.Mapping("nil", nil), configkit.CodeSourceInvalid, ""},
{"bad label", configkit.Mapping("bad\nlabel", map[string]any{}), configkit.CodeSourceInvalid, ""},
{"unsupported", configkit.Mapping("bad", map[string]any{"secret": struct{ Value string }{"canary-value"}}), configkit.CodeSourceInvalid, "/secret"},
{"nonfinite", configkit.Mapping("bad", map[string]any{"count": math.NaN()}), configkit.CodeSourceInvalid, "/count"},
{"cycle", configkit.Mapping("bad", cycle), configkit.CodeSourceInvalid, "/secret"},
{"too many bytes", configkit.Mapping("bad", map[string]any{"name": strings.Repeat("x", 8<<20+1)}), configkit.CodeSourceLimitExceeded, "/name"},
} {
t.Run(test.name, func(t *testing.T) {
loadError := checkMappingCode(t, test.source, configkit.Options[mappingFacadeConfig]{}, test.want)
diagnostic := loadError.Diagnostics()[0]
if diagnostic.Path.String() != test.path || diagnostic.Primary == nil || diagnostic.Primary.SourceID != "source-1" || strings.Contains(diagnostic.Message, "canary-value") {
t.Fatalf("diagnostic = %#v", diagnostic)
}
})
}
}
func checkMappingCode(t *testing.T, source configkit.Source, options configkit.Options[mappingFacadeConfig], want configkit.Code) *configkit.LoadError {
t.Helper()
_, err := configkit.Load(source, options)
loadError, ok := err.(*configkit.LoadError)
if !ok || len(loadError.Diagnostics()) == 0 || loadError.Diagnostics()[0].Code != want {
t.Fatalf("error = %#v, want %s", err, want)
}
return loadError
}

View File

@@ -54,7 +54,7 @@ func filesystemOptionDiagnostics(source Source, context *FilesystemContext) []Di
if context.Root != "" && !fs.ValidPath(context.Root) { if context.Root != "" && !fs.ValidPath(context.Root) {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "filesystem root must be a slash-relative valid path")} return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "filesystem root must be a slash-relative valid path")}
} }
case sourceBytes, sourceReader: case sourceBytes, sourceReader, sourceMapping:
if context.BaseDir == "" || context.Root == "" { if context.BaseDir == "" || context.Root == "" {
return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "byte and reader filesystem context requires base and root directories")} return []Diagnostic{optionDiagnostic(CodeOptionsInvalid, "byte and reader filesystem context requires base and root directories")}
} }

View File

@@ -22,8 +22,8 @@ func selectOptions[T any](source Source, supplied []Options[T]) (Options[T], []D
if source.kind == sourceNone && options.Overlay != "" && options.Overlay != "default" { if source.kind == sourceNone && options.Overlay != "" && options.Overlay != "default" {
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsInvalid, "NoSource cannot select an active overlay")) diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsInvalid, "NoSource cannot select an active overlay"))
} }
if options.Filesystem == nil && (source.kind == sourceBytes || source.kind == sourceReader) && (options.Imports || activeOverlay(options.Overlay)) { if options.Filesystem == nil && (source.kind == sourceBytes || source.kind == sourceReader || source.kind == sourceMapping) && (options.Imports || activeOverlay(options.Overlay)) {
diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsInvalid, "byte and reader imports or overlays require filesystem context")) diagnostics = append(diagnostics, optionDiagnostic(CodeOptionsInvalid, "synthetic-source imports or overlays require filesystem context"))
} }
if definition.Version != nil { if definition.Version != nil {
if !definition.Version.Path.valid() { if !definition.Version.Path.valid() {

View File

@@ -94,11 +94,12 @@ func prepareOverrides(base *composedDocument, sources []SourceInfo, overrides []
} }
pathAccepted[index] = true pathAccepted[index] = true
builder := overrideValueBuilder{sourceID: prepared[index].source.ID, active: make(map[valueVisit]bool)} builder := valueBuilder{sourceID: prepared[index].source.ID, active: make(map[valueVisit]bool)}
value, issue := builder.build(override.Value, override.Path, len(override.Path.segments)) value, issue := builder.build(override.Value, override.Path, len(override.Path.segments))
if issue != nil { if issue != nil {
issue.Primary = location diagnostic := issue.overrideDiagnostic()
diagnostics = append(diagnostics, *issue) diagnostic.Primary = location
diagnostics = append(diagnostics, diagnostic)
continue continue
} }
if relative, ok := overrideDirectivePath(value, Path{}); ok { if relative, ok := overrideDirectivePath(value, Path{}); ok {

View File

@@ -1,150 +1,21 @@
package engine package engine
import ( func (issue *valueIssue) overrideDiagnostic() Diagnostic {
"math" code := CodeOverrideInvalidValue
"reflect" var message string
"slices" switch issue.kind {
"strconv" case valueDepthLimit:
code, message = CodeOverrideLimitExceeded, "configuration override exceeds the depth limit"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document" case valueNodeLimit:
) code, message = CodeOverrideLimitExceeded, "configuration override exceeds the node limit"
case valueByteLimit:
type overrideValueBuilder struct { code, message = CodeOverrideLimitExceeded, "configuration override exceeds the record string-byte limit"
sourceID string case valueNonFinite:
active map[valueVisit]bool message = "configuration override float must be finite"
bytes int64 case valueCycle:
nodes int message = "configuration override values must not contain cycles"
}
func (b *overrideValueBuilder) build(value any, path Path, depth int) (*document.Node, *Diagnostic) {
if depth > maxDocumentDepth {
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path, "configuration override exceeds the depth limit")
}
b.nodes++
if b.nodes > maxDocumentNodes {
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path, "configuration override exceeds the node limit")
}
location := document.Location{SourceID: b.sourceID}
switch value := value.(type) {
case nil:
return &document.Node{Kind: document.Null, Location: location}, nil
case string:
if !b.addBytes(len(value)) {
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path, "configuration override exceeds the record string-byte limit")
}
return &document.Node{Kind: document.String, Text: value, Location: location}, nil
case bool:
return &document.Node{Kind: document.Boolean, Text: strconv.FormatBool(value), Location: location}, nil
case int:
return overrideNumber(strconv.FormatInt(int64(value), 10), location), nil
case int8:
return overrideNumber(strconv.FormatInt(int64(value), 10), location), nil
case int16:
return overrideNumber(strconv.FormatInt(int64(value), 10), location), nil
case int32:
return overrideNumber(strconv.FormatInt(int64(value), 10), location), nil
case int64:
return overrideNumber(strconv.FormatInt(value, 10), location), nil
case uint:
return overrideNumber(strconv.FormatUint(uint64(value), 10), location), nil
case uint8:
return overrideNumber(strconv.FormatUint(uint64(value), 10), location), nil
case uint16:
return overrideNumber(strconv.FormatUint(uint64(value), 10), location), nil
case uint32:
return overrideNumber(strconv.FormatUint(uint64(value), 10), location), nil
case uint64:
return overrideNumber(strconv.FormatUint(value, 10), location), nil
case float32:
if math.IsNaN(float64(value)) || math.IsInf(float64(value), 0) {
return nil, overrideValueDiagnostic(CodeOverrideInvalidValue, path, "configuration override float must be finite")
}
return overrideNumber(strconv.FormatFloat(float64(value), 'g', -1, 32), location), nil
case float64:
if math.IsNaN(value) || math.IsInf(value, 0) {
return nil, overrideValueDiagnostic(CodeOverrideInvalidValue, path, "configuration override float must be finite")
}
return overrideNumber(strconv.FormatFloat(value, 'g', -1, 64), location), nil
case []any:
if value == nil {
return &document.Node{Kind: document.Null, Location: location}, nil
}
leave, diagnostic := b.enter(reflect.ValueOf(value), path)
if diagnostic != nil {
return nil, diagnostic
}
defer leave()
if len(value) > maxDocumentNodes-b.nodes {
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path, "configuration override exceeds the node limit")
}
node := &document.Node{Kind: document.Sequence, Location: location, Sequence: make([]*document.Node, len(value))}
for index, child := range value {
built, diagnostic := b.build(child, path.Index(index), depth+1)
if diagnostic != nil {
return nil, diagnostic
}
node.Sequence[index] = built
}
return node, nil
case map[string]any:
if value == nil {
return &document.Node{Kind: document.Null, Location: location}, nil
}
leave, diagnostic := b.enter(reflect.ValueOf(value), path)
if diagnostic != nil {
return nil, diagnostic
}
defer leave()
if len(value) > (maxDocumentNodes-b.nodes)/2 {
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path, "configuration override exceeds the node limit")
}
keys := make([]string, 0, len(value))
for key := range value {
keys = append(keys, key)
}
slices.Sort(keys)
node := &document.Node{Kind: document.Mapping, Location: location, Mapping: make([]document.Entry, 0, len(keys))}
for _, key := range keys {
b.nodes++ // mapping key
if !b.addBytes(len(key)) {
return nil, overrideValueDiagnostic(CodeOverrideLimitExceeded, path.Key(key), "configuration override exceeds the record string-byte limit")
}
built, diagnostic := b.build(value[key], path.Key(key), depth+1)
if diagnostic != nil {
return nil, diagnostic
}
node.Mapping = append(node.Mapping, document.Entry{Key: key, KeyLocation: location, ValueLocation: location, Value: built})
}
return node, nil
default: default:
return nil, overrideValueDiagnostic(CodeOverrideInvalidValue, path, "configuration override value has an unsupported type") message = "configuration override value has an unsupported type"
} }
} return Diagnostic{Code: code, Severity: SeverityError, Path: issue.path.clone(), Message: message}
func (b *overrideValueBuilder) addBytes(count int) bool {
if count < 0 || int64(count) > maxSourceBytes-b.bytes {
return false
}
b.bytes += int64(count)
return true
}
func (b *overrideValueBuilder) enter(value reflect.Value, path Path) (func(), *Diagnostic) {
visit, ok := visitFor(value)
if !ok {
return func() {}, nil
}
if b.active[visit] {
return nil, overrideValueDiagnostic(CodeOverrideInvalidValue, path, "configuration override values must not contain cycles")
}
b.active[visit] = true
return func() { delete(b.active, visit) }, nil
}
func overrideValueDiagnostic(code Code, path Path, message string) *Diagnostic {
return &Diagnostic{Code: code, Severity: SeverityError, Path: path.clone(), Message: message}
}
func overrideNumber(text string, location document.Location) *document.Node {
return &document.Node{Kind: document.Number, Text: text, Location: location}
} }

View File

@@ -53,6 +53,9 @@ func parseSource(source Source, sourceID string, limits parseLimits) (parsedSour
if len(diagnostics) != 0 || read.info == nil { if len(diagnostics) != 0 || read.info == nil {
return parsedSource{source: read.info}, diagnostics return parsedSource{source: read.info}, diagnostics
} }
if read.document != nil {
return parsedSource{document: read.document, source: read.info, bytes: read.bytes}, nil
}
root, issues := yamladapter.Parse(bytes.NewReader(read.data), sourceID, limits.document) root, issues := yamladapter.Parse(bytes.NewReader(read.data), sourceID, limits.document)
if len(issues) != 0 { if len(issues) != 0 {

View File

@@ -9,6 +9,8 @@ import (
"path/filepath" "path/filepath"
"reflect" "reflect"
"sync/atomic" "sync/atomic"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
) )
const ( const (
@@ -25,6 +27,10 @@ type Source struct {
namespace *nativeNamespace namespace *nativeNamespace
data []byte data []byte
reader *readerSource reader *readerSource
mapping *document.Node
mapIssue *valueIssue
mapBytes int64
mapNodes int
createErr sourceCreationError createErr sourceCreationError
} }
@@ -36,6 +42,7 @@ const (
sourceFilesystem sourceFilesystem
sourceBytes sourceBytes
sourceReader sourceReader
sourceMapping
sourceNone sourceNone
) )
@@ -50,6 +57,7 @@ const (
creationInvalidFilesystem creationInvalidFilesystem
creationInvalidFilesystemLabel creationInvalidFilesystemLabel
creationInvalidFilesystemPath creationInvalidFilesystemPath
creationInvalidMappingRoot
) )
// Filesystem is a reusable opaque namespace for caller-owned fs.FS inputs. // Filesystem is a reusable opaque namespace for caller-owned fs.FS inputs.
@@ -148,6 +156,24 @@ func Reader(label string, input io.Reader) Source {
return source return source
} }
// Mapping snapshots supported built-in values into an owned document at
// construction. Invalid values are retained as safe deferred diagnostics.
func Mapping(label string, values map[string]any) Source {
source := Source{kind: sourceMapping, label: label}
if !validOverrideLabel(label) {
source.createErr = creationInvalidLabel
return source
}
if values == nil {
source.createErr = creationInvalidMappingRoot
return source
}
builder := valueBuilder{active: make(map[valueVisit]bool)}
source.mapping, source.mapIssue = builder.build(values, Path{}, 0)
source.mapBytes, source.mapNodes = builder.bytes, builder.nodes
return source
}
// NoSource explicitly selects a defaults-only input. It creates no source record. // NoSource explicitly selects a defaults-only input. It creates no source record.
func NoSource() Source { func NoSource() Source {
return Source{kind: sourceNone} return Source{kind: sourceNone}
@@ -163,14 +189,24 @@ func defaultSourceLimits() sourceLimits {
} }
type sourceRead struct { type sourceRead struct {
data []byte data []byte
info *SourceInfo document *document.Node
bytes int64
info *SourceInfo
} }
func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead, []Diagnostic) { func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead, []Diagnostic) {
if diagnostic := sourceConstructorDiagnostic(source); diagnostic != nil { if diagnostic := sourceConstructorDiagnostic(source); diagnostic != nil {
if source.kind == sourceMapping {
diagnostic.Primary = &Location{SourceID: sourceID}
return sourceRead{info: mappingSourceInfo(source, sourceID)}, []Diagnostic{*diagnostic}
}
return sourceRead{}, []Diagnostic{*diagnostic} return sourceRead{}, []Diagnostic{*diagnostic}
} }
if source.kind == sourceMapping && source.mapIssue != nil {
diagnostic := source.mapIssue.sourceDiagnostic(sourceID)
return sourceRead{info: mappingSourceInfo(source, sourceID)}, []Diagnostic{diagnostic}
}
if source.kind == sourceNone { if source.kind == sourceNone {
return sourceRead{}, nil return sourceRead{}, nil
} }
@@ -219,6 +255,13 @@ func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceReaderUsed, sourceID, "reader source has already been consumed")} return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceReaderUsed, sourceID, "reader source has already been consumed")}
} }
input = source.reader.input input = source.reader.input
case sourceMapping:
if source.mapBytes > limit || source.mapNodes > maxDocumentNodes {
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceLimitExceeded, sourceID, "mapping source exceeds the configured input limit")}
}
owned := document.Clone(source.mapping)
setDocumentSourceID(owned, sourceID)
return sourceRead{document: owned, bytes: source.mapBytes, info: info}, nil
default: default:
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceInvalid, sourceID, "source is invalid")} return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceInvalid, sourceID, "source is invalid")}
} }
@@ -229,6 +272,14 @@ func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead
return sourceRead{data: bytes.Clone(data), info: info}, nil return sourceRead{data: bytes.Clone(data), info: info}, nil
} }
func mappingSourceInfo(source Source, sourceID string) *SourceInfo {
label := source.label
if !validOverrideLabel(label) {
label = "mapping"
}
return &SourceInfo{ID: sourceID, Label: label, Layer: LayerBase, Kind: SourceKindMapping}
}
func sourceConstructorDiagnostic(source Source) *Diagnostic { func sourceConstructorDiagnostic(source Source) *Diagnostic {
var message string var message string
code := CodeSourceInvalid code := CodeSourceInvalid
@@ -242,6 +293,9 @@ func sourceConstructorDiagnostic(source Source) *Diagnostic {
message = "source path must not be empty" message = "source path must not be empty"
case creationInvalidLabel: case creationInvalidLabel:
message = "source label must not be empty" message = "source label must not be empty"
if source.kind == sourceMapping {
message = "mapping source label must be nonempty and safe to display"
}
case creationInvalidReader: case creationInvalidReader:
message = "source reader must not be nil" message = "source reader must not be nil"
case creationTooLarge: case creationTooLarge:
@@ -253,6 +307,8 @@ func sourceConstructorDiagnostic(source Source) *Diagnostic {
message = "filesystem label must be nonempty and safe to display" message = "filesystem label must be nonempty and safe to display"
case creationInvalidFilesystemPath: case creationInvalidFilesystemPath:
message = "filesystem source path must be a non-root slash-relative valid path" message = "filesystem source path must be a non-root slash-relative valid path"
case creationInvalidMappingRoot:
message = "mapping source root must be a non-nil mapping"
} }
diagnostic := sourceDiagnostic(code, "", message) diagnostic := sourceDiagnostic(code, "", message)
return &diagnostic return &diagnostic
@@ -268,11 +324,29 @@ func sourceSourceKind(kind sourceKind) SourceKind {
return SourceKindBytes return SourceKindBytes
case sourceReader: case sourceReader:
return SourceKindReader return SourceKindReader
case sourceMapping:
return SourceKindMapping
default: default:
return "" return ""
} }
} }
func setDocumentSourceID(node *document.Node, sourceID string) {
if node == nil {
return
}
node.Location.SourceID = sourceID
for index := range node.Mapping {
entry := &node.Mapping[index]
entry.KeyLocation.SourceID = sourceID
entry.ValueLocation.SourceID = sourceID
setDocumentSourceID(entry.Value, sourceID)
}
for _, child := range node.Sequence {
setDocumentSourceID(child, sourceID)
}
}
func classifySourceError(err error, sourceID string) Diagnostic { func classifySourceError(err error, sourceID string) Diagnostic {
switch { switch {
case errors.Is(err, fs.ErrNotExist): case errors.Is(err, fs.ErrNotExist):

View File

@@ -0,0 +1,19 @@
package engine
func (issue *valueIssue) sourceDiagnostic(sourceID string) Diagnostic {
code := CodeSourceInvalid
message := "mapping source has an unsupported value"
switch issue.kind {
case valueDepthLimit:
code, message = CodeSourceLimitExceeded, "mapping source exceeds the depth limit"
case valueNodeLimit:
code, message = CodeSourceLimitExceeded, "mapping source exceeds the node limit"
case valueByteLimit:
code, message = CodeSourceLimitExceeded, "mapping source exceeds the string-byte limit"
case valueNonFinite:
message = "mapping source float must be finite"
case valueCycle:
message = "mapping source values must not contain cycles"
}
return Diagnostic{Code: code, Severity: SeverityError, Path: issue.path.clone(), Message: message, Primary: &Location{SourceID: sourceID}}
}

166
internal/engine/value.go Normal file
View File

@@ -0,0 +1,166 @@
package engine
import (
"math"
"reflect"
"slices"
"strconv"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
)
type valueBuilder struct {
sourceID string
active map[valueVisit]bool
bytes int64
nodes int
}
type valueIssueKind uint8
const (
valueDepthLimit valueIssueKind = iota
valueNodeLimit
valueByteLimit
valueNonFinite
valueUnsupported
valueCycle
)
type valueIssue struct {
kind valueIssueKind
path Path
}
func newValueIssue(kind valueIssueKind, path Path) *valueIssue {
return &valueIssue{kind: kind, path: path.clone()}
}
func (b *valueBuilder) build(value any, path Path, depth int) (*document.Node, *valueIssue) {
if depth > maxDocumentDepth {
return nil, newValueIssue(valueDepthLimit, path)
}
b.nodes++
if b.nodes > maxDocumentNodes {
return nil, newValueIssue(valueNodeLimit, path)
}
location := document.Location{SourceID: b.sourceID}
switch value := value.(type) {
case nil:
return &document.Node{Kind: document.Null, Location: location}, nil
case string:
if !b.addBytes(len(value)) {
return nil, newValueIssue(valueByteLimit, path)
}
return &document.Node{Kind: document.String, Text: value, Location: location}, nil
case bool:
return &document.Node{Kind: document.Boolean, Text: strconv.FormatBool(value), Location: location}, nil
case int:
return valueNumber(strconv.FormatInt(int64(value), 10), location), nil
case int8:
return valueNumber(strconv.FormatInt(int64(value), 10), location), nil
case int16:
return valueNumber(strconv.FormatInt(int64(value), 10), location), nil
case int32:
return valueNumber(strconv.FormatInt(int64(value), 10), location), nil
case int64:
return valueNumber(strconv.FormatInt(value, 10), location), nil
case uint:
return valueNumber(strconv.FormatUint(uint64(value), 10), location), nil
case uint8:
return valueNumber(strconv.FormatUint(uint64(value), 10), location), nil
case uint16:
return valueNumber(strconv.FormatUint(uint64(value), 10), location), nil
case uint32:
return valueNumber(strconv.FormatUint(uint64(value), 10), location), nil
case uint64:
return valueNumber(strconv.FormatUint(value, 10), location), nil
case float32:
if math.IsNaN(float64(value)) || math.IsInf(float64(value), 0) {
return nil, newValueIssue(valueNonFinite, path)
}
return valueNumber(strconv.FormatFloat(float64(value), 'g', -1, 32), location), nil
case float64:
if math.IsNaN(value) || math.IsInf(value, 0) {
return nil, newValueIssue(valueNonFinite, path)
}
return valueNumber(strconv.FormatFloat(value, 'g', -1, 64), location), nil
case []any:
if value == nil {
return &document.Node{Kind: document.Null, Location: location}, nil
}
leave, diagnostic := b.enter(reflect.ValueOf(value), path)
if diagnostic != nil {
return nil, diagnostic
}
defer leave()
if len(value) > maxDocumentNodes-b.nodes {
return nil, newValueIssue(valueNodeLimit, path)
}
node := &document.Node{Kind: document.Sequence, Location: location, Sequence: make([]*document.Node, len(value))}
for index, child := range value {
built, diagnostic := b.build(child, path.Index(index), depth+1)
if diagnostic != nil {
return nil, diagnostic
}
node.Sequence[index] = built
}
return node, nil
case map[string]any:
if value == nil {
return &document.Node{Kind: document.Null, Location: location}, nil
}
leave, diagnostic := b.enter(reflect.ValueOf(value), path)
if diagnostic != nil {
return nil, diagnostic
}
defer leave()
if len(value) > (maxDocumentNodes-b.nodes)/2 {
return nil, newValueIssue(valueNodeLimit, path)
}
keys := make([]string, 0, len(value))
for key := range value {
keys = append(keys, key)
}
slices.Sort(keys)
node := &document.Node{Kind: document.Mapping, Location: location, Mapping: make([]document.Entry, 0, len(keys))}
for _, key := range keys {
b.nodes++ // mapping key
if !b.addBytes(len(key)) {
return nil, newValueIssue(valueByteLimit, path.Key(key))
}
built, diagnostic := b.build(value[key], path.Key(key), depth+1)
if diagnostic != nil {
return nil, diagnostic
}
node.Mapping = append(node.Mapping, document.Entry{Key: key, KeyLocation: location, ValueLocation: location, Value: built})
}
return node, nil
default:
return nil, newValueIssue(valueUnsupported, path)
}
}
func (b *valueBuilder) addBytes(count int) bool {
if count < 0 || int64(count) > maxSourceBytes-b.bytes {
return false
}
b.bytes += int64(count)
return true
}
func (b *valueBuilder) enter(value reflect.Value, path Path) (func(), *valueIssue) {
visit, ok := visitFor(value)
if !ok {
return func() {}, nil
}
if b.active[visit] {
return nil, newValueIssue(valueCycle, path)
}
b.active[visit] = true
return func() { delete(b.active, visit) }, nil
}
func valueNumber(text string, location document.Location) *document.Node {
return &document.Node{Kind: document.Number, Text: text, Location: location}
}

View File

@@ -8,7 +8,7 @@ import (
) )
// Source is an opaque configuration input. Construct sources with File, // Source is an opaque configuration input. Construct sources with File,
// Filesystem.File, Bytes, Reader, or NoSource; the zero value is invalid. // Filesystem.File, Bytes, Reader, Mapping, or NoSource; the zero value is invalid.
type Source struct{ value engine.Source } type Source struct{ value engine.Source }
// Filesystem is a reusable opaque namespace for a caller-owned fs.FS. Copies // Filesystem is a reusable opaque namespace for a caller-owned fs.FS. Copies
@@ -38,6 +38,15 @@ func Bytes(label string, data []byte) Source { return Source{value: engine.Bytes
// Copies of the returned Source share the same single-use claim. // Copies of the returned Source share the same single-use claim.
func Reader(label string, input io.Reader) Source { return Source{value: engine.Reader(label, input)} } func Reader(label string, input io.Reader) Source { return Source{value: engine.Reader(label, input)} }
// Mapping snapshots supported built-in values immediately. A nil root map is
// invalid; an empty map is a valid empty document. Values may be nil, built-in
// scalars, []any, or map[string]any recursively. Unsupported values and cycles
// are reported when the source is consumed. Label must be nonempty and safe to
// display. Later caller mutation cannot change the captured document.
func Mapping(label string, values map[string]any) Source {
return Source{value: engine.Mapping(label, values)}
}
// NoSource explicitly selects a defaults-only input. It creates no source record. // NoSource explicitly selects a defaults-only input. It creates no source record.
func NoSource() Source { return Source{value: engine.NoSource()} } func NoSource() Source { return Source{value: engine.NoSource()} }