eric/ConfigKit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add bounded sources and strict YAML parsing

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-06 19:30:31 +00:00
parent 14b10cb6a5
commit 7c78da9cd2
17 changed files with 1287 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
doc.go
View File

@@ -1,8 +1,8 @@
// Package configkit provides foundational types for deterministic,
// application-owned configuration loading in Go programs.
//
// Semantic paths, metadata and provenance records, safe diagnostics, and structured
// load errors are available. Source, parsing, decoding, composition, and loading
// operations will be introduced with their implemented contracts; no usable
// configuration loader is available yet.
// Semantic paths, opaque source constructors, metadata and provenance records, safe
// diagnostics, and structured load errors are available. YAML input is validated
// behind a private parser boundary according to docs/config.md. Typed decoding,
// composition, and loading operations are not available yet.
package configkit

79
docs/config.md Normal file
View File

@@ -0,0 +1,79 @@
# Configuration Input Contract
This document owns ConfigKit's currently implemented source-input and YAML syntax
contracts. The parser and owned document representation are private; a public typed
loading lifecycle has not been implemented yet.
## Sources
ConfigKit exposes opaque sources through these constructors:
| Constructor | Current contract |
| --- | --- |
| `File(path)` | Requires a nonempty path, opens a regular file for each consumption, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. |
| `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. |
| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. |
| `NoSource()` | Represents an explicit absence of input and creates no source record. Its defaults-only lifecycle behavior will become available with `Load`. |
The zero `Source` is invalid. Constructor argument failures are retained inside the
opaque source and become safe structured diagnostics when the source is consumed.
File sources can observe file changes between consumptions; byte sources cannot.
Each source is limited to 8 MiB. A load will be limited to 64 MiB across all source
and override input when those layers are implemented. Reads request at most the
remaining allowance plus one byte so excess data is rejected rather than silently
truncated. Data returned together with a read error is not accepted as partial input.
## YAML documents
The private YAML boundary currently accepts exactly one nonempty document whose root
is a mapping. Comments and block or flow collections are supported. It rejects:
- empty input or an empty document;
- every additional document, including an empty one;
- duplicate decoded mapping keys;
- non-string mapping keys;
- explicit tags, anchors, aliases, and merge keys;
- a quoted or plain mapping key equal to `<<`; and
- input exceeding the fixed depth or node limits.
Mapping key comparison is exact: it performs no case folding or Unicode
normalization. The private document tree retains declaration order, exact validated
number spelling, source identifiers, and separate key/value line and column
locations. It contains no upstream parser nodes or types.
Root depth is zero. Input depth is limited to 256 and each parsed input tree to
1,000,000 nodes, counting mapping keys as input nodes. The upstream parser also has
a private calibrated depth guard; passing that guard never bypasses ConfigKit's tree
limits.
## Scalars
Scalar meaning is determined by ConfigKit rather than upstream coercion:
| Input form | Meaning |
| --- | --- |
| Quoted, literal, or folded scalar | String |
| Plain `true` or `false` | Boolean |
| Plain `null`, `~`, or an empty mapping value | Null |
| JSON decimal number | Number, retaining its exact spelling |
| Date/time-looking or other ordinary plain text | String |
JSON numbers permit an optional minus, a zero or nonzero decimal integer without a
leading zero, an optional fraction with at least one digit, and an optional decimal
exponent. Hexadecimal, octal, digit separators, leading plus, leading/trailing-dot
forms, incomplete exponents, and non-finite values are rejected. Legacy boolean
words (`yes`, `no`, `on`, and `off`) and non-lowercase variants of the reserved
boolean/null words are rejected unless quoted.
Mapping keys use the same scalar classification and must classify as strings. Quote
keys that otherwise classify as a boolean, number, or null.
## Safe failures
Source failures use stable `source.*` codes and YAML failures use stable `yaml.*`
codes declared by the root package. Diagnostics contain safe classifications,
semantic paths when applicable, and available source coordinates. Raw parser,
conversion, reader-error, and input text is not forwarded. Source labels and
diagnostic messages are application-visible and therefore must never contain
secrets; terminal control characters are escaped by `WriteDiagnostics`.

8
docs/internal/extraction.md
View File

@@ -53,6 +53,14 @@ Assigned to the YAML/source implementation:
ConfigKit's scalar grammar and limits, and replace raw parser errors with safe
diagnostics. Narratio's application decoders and YAML node types are not copied.
Implementation record (2026-09-06): the labeled byte/reader ownership and strict
single-document cases above were generalized into `source.go`, `parse.go`, and
`internal/yaml`. No production line was copied verbatim. ConfigKit adds byte and
tree ceilings, shared atomic reader claims, regular-file checks, exact numeric
lexemes, parser-depth calibration, ConfigKit-owned locations, safe diagnostic
codes, and an AST import-boundary regression. Application typed decoding, parser
errors, YAML node types, and session-specific loaders were not retained.
### Strict composition
Assigned to document composition:

11
docs/internal/overview.md
View File

@@ -5,10 +5,15 @@ module and package documentation boundary. Its implemented public primitives are
- immutable semantic `Path` construction and strict JSON Pointer parsing;
- detached provenance and diagnostic record types with empty `Metadata` queries;
- canonical, control-safe diagnostic rendering and structured `LoadError` access.
- canonical, control-safe diagnostic rendering and structured `LoadError` access;
- opaque file, byte, reader, and no-input source construction and bounded reads;
- a parser-independent ordered tree in `internal/document`; and
- the sole upstream YAML integration in `internal/yaml`, which translates validated
input into that owned tree and exposes no parser types outside its boundary.
It intentionally exposes no loading, source, parsing, or decoding API yet. Private
metadata population will be integrated with loading when that lifecycle exists.
The [configuration input contract](../config.md) owns implemented source and YAML
behavior. ConfigKit intentionally exposes no typed loading or decoding API yet.
Private metadata population will be integrated when that lifecycle exists.
The source-extraction inventory records the application foundations assigned to
future implementation work:

27
docs/roadmap/implementation.md
View File

@@ -2,7 +2,7 @@
## Status and ownership
Status: implementation in progress; Stages 1–2 are complete and later stages are
Status: implementation in progress; Stages 1–3 are complete and later stages are
pending.
Audience: a `gpt-5.6-terra` coding agent implementing one stage per prompt, in
@@ -383,7 +383,30 @@ Acceptance:
## Stage 3 — Extract source reading and enforce YAML policy
Status: pending.
Status: complete.
Completion evidence (2026-09-06):
- Added opaque source constructors and bounded file/byte/reader acquisition in
[`source.go`](../../source.go), including byte snapshots, regular-file checks,
deterministic I/O codes, caller-owned reader lifetime, and one atomic claim
shared by source copies. `NoSource` creates no source record.
- Pinned `go.yaml.in/yaml/v4 v4.0.0-rc.6` solely behind `internal/yaml` and added
the parser-independent ordered tree in `internal/document`. The adapter uses the
streaming loader, checks for a second document explicitly, applies a calibrated
private parser depth plugin, enforces owned depth/node limits, preserves numeric
spelling and locations, and returns only value-free internal issues.
- Generalized the strict-source/document cases recorded in the [extraction
inventory](../internal/extraction.md), created the [current configuration input
contract](../config.md), and added an AST import-boundary regression. No typed
decoder, composition, filesystem graph, or public `Load` was introduced.
- Tests cover real file failures and reopening, byte ownership, fragmented/error
readers, exact/overflow byte limits, concurrent copied-source claims, all YAML
policy categories, trailing/empty documents, duplicate locations, lexical wide
numbers, block/flow depth calibration, node limits, safe parser errors, and
adapter isolation. Verified `go test ./...`, `go vet ./...`, `go build ./...`,
`go mod tidy -diff`, and `git diff --check` with Go 1.26.7 on linux/amd64. No
deviations were required; typed binding and decoding remain later work.
Read: feature [YAML policy](initial.md#yaml-and-scalar-policy),
[source constructors](initial.md#entry-point-and-reusable-application-definition).

44
docs/roadmap/initial.md
View File

@@ -423,31 +423,10 @@ configuration is never returned as a successful, usable result.
## YAML and scalar policy
The supported model is mappings with string keys, sequences, scalars, and null,
with YAML comments and block/flow syntax. Require exactly one document with a
mapping root. Reject empty input, additional documents (including empty ones),
duplicate mapping keys, explicit tags, anchors, aliases, and merge keys (`<<`).
Mapping keys must resolve to strings under this scalar policy; quote keys
that would otherwise be booleans, numbers, or null. Compare decoded string keys
without case folding or Unicode normalization. A quoted `"<<"` is also reserved and rejected.
Use a deliberately limited scalar vocabulary independent of parser defaults:
- Quoted and block scalars are strings.
- Plain `true` and `false` are booleans; plain `null`, `~`, and an empty value
are null. Other case variants of these reserved literals are rejected when used as
plain values.
- Numbers use JSON decimal syntax: optional minus, integer part with no leading
zero except zero itself, optional fraction, and optional decimal exponent.
- Reject YAML numeric extensions such as hexadecimal, octal, digit separators,
leading plus, and non-finite values. Reject legacy boolean words such as
`yes`, `no`, `on`, and `off` unless quoted (case-insensitively).
- Date/time-looking plain text remains a string, never an implicit timestamp.
Other ordinary plain text is a string.
Unsupported scalar forms must produce a policy error rather than inherit a
parser conversion. Preserve numeric lexical information until decoding so
large integers are not rounded through an intermediate floating-point value.
Source ownership, the supported YAML subset, scalar grammar, fixed input limits,
and safe failure behavior are implemented. The [current configuration input
contract](../config.md) now owns those definitions. Typed scalar conversion into
application values remains target work in [typed decoding](#typed-decoding).
## Paths and presence
@@ -770,11 +749,6 @@ type FilesystemContext struct {
BaseDir string
Root string
}
func File(path string) Source
func Bytes(label string, data []byte) Source
func Reader(label string, input io.Reader) Source
func NoSource() Source
```
`Load` accepts zero or one options value; multiple values are an options error,
@@ -792,11 +766,11 @@ result. Callbacks must not retain these contexts or configuration pointers; shar
closures must be safe for concurrent calls if the application loads concurrently.
ConfigKit does not synchronize application-owned captured state.
`Source` is opaque with the constructors above, not an arbitrary provider
interface. `Bytes` snapshots its input; `Reader` is consumed during the load and
is not closed by ConfigKit. A reader source is single-use. File sources are read
on each load. Constructor argument errors surface as load diagnostics. Applications must use
non-secret source labels because labels appear in diagnostics and source reports.
`Source` and its constructors are implemented; [`source.go`](../../source.go) and
the [configuration input contract](../config.md#sources) own their current behavior.
The loading and filesystem-option behavior below remains target work.
Applications must use non-secret source labels because labels appear in diagnostics
and source reports.
Filesystem context follows the [source rules](#explicit-imports-and-source-identity):
a file's own directory supplies its base, and an explicit context can set its
root. Supplying `BaseDir` for a file is an options error; bytes/readers require

2
go.mod
View File

@@ -1,3 +1,5 @@
module gitea.maximumdirect.net/eric/ConfigKit
go 1.25.0
require go.yaml.in/yaml/v4 v4.0.0-rc.6

2
go.sum Normal file
View File

@@ -0,0 +1,2 @@
go.yaml.in/yaml/v4 v4.0.0-rc.6 h1:1h7H1ohdUh93/FyE4YaDa1Zh64K6VVbjF4K6WUxMtH4=
go.yaml.in/yaml/v4 v4.0.0-rc.6/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=

50
import_boundary_test.go Normal file
View File

@@ -0,0 +1,50 @@
package configkit
import (
"go/parser"
"go/token"
"io/fs"
"path/filepath"
"strconv"
"strings"
"testing"
)
func TestDependencyImportBoundary(t *testing.T) {
t.Parallel()
err := filepath.WalkDir(".", func(path string, entry fs.DirEntry, walkErr error) error {
if walkErr != nil {
return walkErr
}
if entry.IsDir() {
if path == ".git" {
return filepath.SkipDir
}
return nil
}
if filepath.Ext(path) != ".go" {
return nil
}
file, err := parser.ParseFile(token.NewFileSet(), path, nil, parser.ImportsOnly)
if err != nil {
return err
}
for _, imported := range file.Imports {
importPath, err := strconv.Unquote(imported.Path.Value)
if err != nil {
return err
}
if strings.HasPrefix(importPath, "go.yaml.in/yaml/") && !strings.HasPrefix(filepath.ToSlash(path), "internal/yaml/") {
t.Errorf("%s imports private YAML dependency %s", path, importPath)
}
if strings.HasPrefix(importPath, "github.com/google/go-cmp/") && !strings.HasSuffix(path, "_test.go") {
t.Errorf("%s imports test-only dependency %s", path, importPath)
}
}
return nil
})
if err != nil {
t.Fatal(err)
}
}

62
internal/document/document.go Normal file
View File

@@ -0,0 +1,62 @@
// Package document defines ConfigKit's parser-independent configuration tree.
package document
// Kind identifies a configuration node category.
type Kind uint8
const (
Mapping Kind = iota + 1
Sequence
String
Boolean
Number
Null
)
// Location identifies a source coordinate. Root depth and unavailable coordinates
// are represented independently by callers; line and column are one-based when set.
type Location struct {
SourceID string
Line int
Column int
}
// Entry is an ordered mapping entry with separate key and value locations.
type Entry struct {
Key string
KeyLocation Location
ValueLocation Location
Value *Node
}
// Node is ConfigKit's owned configuration representation. Number Text retains its
// exact validated decimal spelling; Boolean Text is "true" or "false".
type Node struct {
Kind Kind
Location Location
Text string
Mapping []Entry
Sequence []*Node
}
// Clone returns a detached copy of a node tree.
func Clone(node *Node) *Node {
if node == nil {
return nil
}
cloned := *node
if node.Mapping != nil {
cloned.Mapping = make([]Entry, len(node.Mapping))
for i, entry := range node.Mapping {
cloned.Mapping[i] = entry
cloned.Mapping[i].Value = Clone(entry.Value)
}
}
if node.Sequence != nil {
cloned.Sequence = make([]*Node, len(node.Sequence))
for i, child := range node.Sequence {
cloned.Sequence[i] = Clone(child)
}
}
return &cloned
}

274
internal/yaml/parse.go Normal file
View File

@@ -0,0 +1,274 @@
// Package yaml adapts the private upstream parser to ConfigKit's document model.
package yaml
import (
"errors"
"io"
"regexp"
"strings"
"go.yaml.in/yaml/v4"
"go.yaml.in/yaml/v4/plugin/limit"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
)
const (
codeInvalid = "yaml.invalid"
codeEmpty = "yaml.empty"
codeMultipleDocuments = "yaml.multiple_documents"
codeRootType = "yaml.root_type"
codeDuplicateKey = "yaml.duplicate_key"
codeTag = "yaml.tag"
codeAnchor = "yaml.anchor"
codeAlias = "yaml.alias"
codeMergeKey = "yaml.merge_key"
codeScalar = "yaml.scalar"
codeDepthLimit = "yaml.depth_limit"
codeNodeLimit = "yaml.node_limit"
)
var (
jsonNumberPattern = regexp.MustCompile(`^-?(0|[1-9][0-9]*)(\.[0-9]+)?([eE][+-]?[0-9]+)?$`)
decimalLikePattern = regexp.MustCompile(`^[+-]?[0-9][0-9_]*(\.[0-9_]*)?([eE][+-]?[0-9_]*)?$`)
prefixedNumberPattern = regexp.MustCompile(`^[+-]?0[xXoO][0-9A-Fa-f_]+$`)
leadingDecimalPattern = regexp.MustCompile(`^[+-]?\.[0-9_]+$`)
nonFiniteNumberPattern = regexp.MustCompile(`(?i)^[+-]?\.(inf|nan)$`)
errParserDepth = errors.New("configkit yaml parser depth limit")
)
// Limits controls private parser and translated-tree resource bounds.
type Limits struct {
Depth int
Nodes int
ParserDepth int
}
// Issue is a parser-independent, value-free YAML diagnostic.
type Issue struct {
Code string
Message string
Line int
Column int
Related []document.Location
}
// Parse loads exactly one policy-valid mapping document.
func Parse(input io.Reader, sourceID string, limits Limits) (*document.Node, []Issue) {
parserDepth := limits.ParserDepth
if parserDepth <= 0 {
parserDepth = limits.Depth + 2
}
depthExceeded := false
plugin := limit.New(limit.DepthFunc(func(depth int, _ *limit.DepthContext) error {
if depth > parserDepth {
depthExceeded = true
return errParserDepth
}
return nil
}))
loader, err := yaml.NewLoader(input, yaml.WithPlugin(plugin))
if err != nil {
return nil, []Issue{{Code: codeInvalid, Message: "YAML parser could not be initialized"}}
}
var parsed yaml.Node
if err := loader.Load(&parsed); err != nil {
if errors.Is(err, io.EOF) {
return nil, []Issue{{Code: codeEmpty, Message: "configuration input is empty"}}
}
return nil, []Issue{parserIssue(err, depthExceeded)}
}
var trailing yaml.Node
if err := loader.Load(&trailing); err == nil {
return nil, []Issue{{Code: codeMultipleDocuments, Message: "configuration input must contain exactly one document", Line: trailing.Line, Column: trailing.Column}}
} else if !errors.Is(err, io.EOF) {
return nil, []Issue{parserIssue(err, depthExceeded)}
}
root := documentRoot(&parsed)
if root == nil || (root.Kind == yaml.ScalarNode && root.Value == "" && root.ShortTag() == "!!null") {
return nil, []Issue{{Code: codeEmpty, Message: "configuration document is empty", Line: parsed.Line, Column: parsed.Column}}
}
if root.Kind != yaml.MappingNode {
return nil, []Issue{{Code: codeRootType, Message: "configuration document root must be a mapping", Line: root.Line, Column: root.Column}}
}
state := translationState{sourceID: sourceID, limits: limits}
translated, issue := state.translate(root, 0)
if issue != nil {
return nil, []Issue{*issue}
}
return translated, nil
}
type translationState struct {
sourceID string
limits Limits
nodes int
}
func (s *translationState) translate(node *yaml.Node, depth int) (*document.Node, *Issue) {
if depth > s.limits.Depth {
return nil, nodeIssue(codeDepthLimit, "configuration exceeds the nesting depth limit", node)
}
if node.Style&yaml.TaggedStyle != 0 {
return nil, nodeIssue(codeTag, "explicit YAML tags are not supported", node)
}
if node.Anchor != "" {
return nil, nodeIssue(codeAnchor, "YAML anchors are not supported", node)
}
if node.Kind == yaml.AliasNode || node.Alias != nil {
return nil, nodeIssue(codeAlias, "YAML aliases are not supported", node)
}
if issue := s.chargeNode(node); issue != nil {
return nil, issue
}
location := document.Location{SourceID: s.sourceID, Line: node.Line, Column: node.Column}
switch node.Kind {
case yaml.MappingNode:
if len(node.Content)%2 != 0 {
return nil, nodeIssue(codeInvalid, "mapping structure is invalid", node)
}
result := &document.Node{Kind: document.Mapping, Location: location, Mapping: make([]document.Entry, 0, len(node.Content)/2)}
seen := make(map[string]document.Location, len(node.Content)/2)
for i := 0; i < len(node.Content); i += 2 {
keyNode, valueNode := node.Content[i], node.Content[i+1]
if issue := s.chargeNode(keyNode); issue != nil {
return nil, issue
}
key, issue := mappingKey(keyNode)
if issue != nil {
return nil, issue
}
if key == "<<" {
return nil, nodeIssue(codeMergeKey, "YAML merge keys are not supported", keyNode)
}
keyLocation := document.Location{SourceID: s.sourceID, Line: keyNode.Line, Column: keyNode.Column}
if previous, ok := seen[key]; ok {
issue := nodeIssue(codeDuplicateKey, "mapping key is defined more than once", keyNode)
issue.Related = []document.Location{previous}
return nil, issue
}
seen[key] = keyLocation
value, issue := s.translate(valueNode, depth+1)
if issue != nil {
return nil, issue
}
result.Mapping = append(result.Mapping, document.Entry{
Key: key,
KeyLocation: keyLocation,
ValueLocation: value.Location,
Value: value,
})
}
return result, nil
case yaml.SequenceNode:
result := &document.Node{Kind: document.Sequence, Location: location, Sequence: make([]*document.Node, 0, len(node.Content))}
for _, child := range node.Content {
translated, issue := s.translate(child, depth+1)
if issue != nil {
return nil, issue
}
result.Sequence = append(result.Sequence, translated)
}
return result, nil
case yaml.ScalarNode:
kind, text, issue := scalar(node)
if issue != nil {
return nil, issue
}
return &document.Node{Kind: kind, Location: location, Text: text}, nil
default:
return nil, nodeIssue(codeInvalid, "YAML node kind is not supported", node)
}
}
func (s *translationState) chargeNode(node *yaml.Node) *Issue {
s.nodes++
if s.limits.Nodes >= 0 && s.nodes > s.limits.Nodes {
return nodeIssue(codeNodeLimit, "configuration exceeds the node limit", node)
}
return nil
}
func mappingKey(node *yaml.Node) (string, *Issue) {
if node.Kind == yaml.AliasNode || node.Alias != nil {
return "", nodeIssue(codeAlias, "YAML aliases are not supported", node)
}
if node.Kind != yaml.ScalarNode {
return "", nodeIssue(codeScalar, "mapping keys must be strings", node)
}
if node.Style&yaml.TaggedStyle != 0 {
return "", nodeIssue(codeTag, "explicit YAML tags are not supported", node)
}
if node.Anchor != "" {
return "", nodeIssue(codeAnchor, "YAML anchors are not supported", node)
}
kind, text, issue := scalar(node)
if issue != nil {
return "", issue
}
if kind != document.String {
return "", nodeIssue(codeScalar, "mapping keys must be strings", node)
}
return text, nil
}
func scalar(node *yaml.Node) (document.Kind, string, *Issue) {
if node.Style&(yaml.DoubleQuotedStyle|yaml.SingleQuotedStyle|yaml.LiteralStyle|yaml.FoldedStyle) != 0 {
return document.String, node.Value, nil
}
value := node.Value
switch value {
case "":
if node.ShortTag() == "!!null" {
return document.Null, "", nil
}
case "null", "~":
return document.Null, "", nil
case "true", "false":
return document.Boolean, value, nil
}
lower := strings.ToLower(value)
if lower == "true" || lower == "false" || lower == "null" || lower == "yes" || lower == "no" || lower == "on" || lower == "off" {
return 0, "", nodeIssue(codeScalar, "plain scalar uses an unsupported reserved form", node)
}
if jsonNumberPattern.MatchString(value) {
return document.Number, value, nil
}
if decimalLikePattern.MatchString(value) || prefixedNumberPattern.MatchString(value) ||
leadingDecimalPattern.MatchString(value) || nonFiniteNumberPattern.MatchString(value) {
return 0, "", nodeIssue(codeScalar, "plain scalar uses an unsupported numeric form", node)
}
if tag := node.ShortTag(); tag == "!!int" || tag == "!!float" {
return 0, "", nodeIssue(codeScalar, "plain scalar uses an unsupported numeric form", node)
}
return document.String, value, nil
}
func documentRoot(node *yaml.Node) *yaml.Node {
if node.Kind == yaml.DocumentNode && len(node.Content) == 1 {
return node.Content[0]
}
return node
}
func parserIssue(err error, depthExceeded bool) Issue {
code := codeInvalid
if depthExceeded || errors.Is(err, errParserDepth) {
code = codeDepthLimit
}
issue := Issue{Code: code, Message: "configuration contains invalid YAML"}
var loadError *yaml.LoadError
if errors.As(err, &loadError) {
issue.Line = loadError.Mark.Line
issue.Column = loadError.Mark.Column
}
return issue
}
func nodeIssue(code, message string, node *yaml.Node) *Issue {
return &Issue{Code: code, Message: message, Line: node.Line, Column: node.Column}
}

200
internal/yaml/parse_test.go Normal file
View File

@@ -0,0 +1,200 @@
package yaml
import (
"io"
"strings"
"testing"
"go.yaml.in/yaml/v4"
"go.yaml.in/yaml/v4/plugin/limit"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
)
func TestParsePreservesDocumentValuesAndLocations(t *testing.T) {
t.Parallel()
input := "# comment\nwide: 9223372036854775808\nwhen: 2026-09-06\nquoted: \"yes\"\nflow: [true, null, ~, false]\nempty:\n"
root, issues := Parse(strings.NewReader(input), "source-1", generousLimits())
if len(issues) != 0 {
t.Fatalf("Parse() issues = %#v", issues)
}
if root.Kind != document.Mapping || len(root.Mapping) != 5 {
t.Fatalf("root = %#v", root)
}
if root.Mapping[0].Value.Kind != document.Number || root.Mapping[0].Value.Text != "9223372036854775808" {
t.Fatalf("wide number = %#v", root.Mapping[0].Value)
}
if root.Mapping[0].KeyLocation.Line != 2 || root.Mapping[0].ValueLocation.Column != 7 {
t.Fatalf("locations = %#v", root.Mapping[0])
}
if root.Mapping[1].Value.Kind != document.String || root.Mapping[1].Value.Text != "2026-09-06" {
t.Fatalf("timestamp-like text = %#v", root.Mapping[1].Value)
}
if root.Mapping[3].Value.Sequence[1].Kind != document.Null || root.Mapping[3].Value.Sequence[2].Kind != document.Null {
t.Fatalf("null spellings = %#v", root.Mapping[3].Value.Sequence)
}
if root.Mapping[4].Value.Kind != document.Null {
t.Fatalf("empty value = %#v", root.Mapping[4].Value)
}
}
func TestParsePolicyTable(t *testing.T) {
t.Parallel()
tests := []struct {
name string
input string
code string
}{
{name: "empty input", input: "", code: codeEmpty},
{name: "empty document", input: "---\n", code: codeEmpty},
{name: "sequence root", input: "- value\n", code: codeRootType},
{name: "scalar root", input: "value\n", code: codeRootType},
{name: "second document", input: "a: 1\n---\nb: 2\n", code: codeMultipleDocuments},
{name: "empty second document", input: "a: 1\n---\n", code: codeMultipleDocuments},
{name: "duplicate key", input: "a: 1\na: 2\n", code: codeDuplicateKey},
{name: "nested duplicate", input: "a: {b: 1, b: 2}\n", code: codeDuplicateKey},
{name: "explicit standard tag", input: "a: !!str value\n", code: codeTag},
{name: "explicit custom tag", input: "a: !custom value\n", code: codeTag},
{name: "anchor", input: "a: &name value\n", code: codeAnchor},
{name: "alias source anchor", input: "a: &name value\nb: *name\n", code: codeAnchor},
{name: "merge key", input: "<<: value\n", code: codeMergeKey},
{name: "quoted merge key", input: "\"<<\": value\n", code: codeMergeKey},
{name: "boolean key", input: "true: value\n", code: codeScalar},
{name: "number key", input: "1: value\n", code: codeScalar},
{name: "null key", input: "null: value\n", code: codeScalar},
{name: "legacy yes", input: "a: yes\n", code: codeScalar},
{name: "legacy mixed", input: "a: OfF\n", code: codeScalar},
{name: "reserved case", input: "a: TRUE\n", code: codeScalar},
{name: "hex", input: "a: 0x10\n", code: codeScalar},
{name: "octal", input: "a: 0o10\n", code: codeScalar},
{name: "separator", input: "a: 1_000\n", code: codeScalar},
{name: "leading plus", input: "a: +1\n", code: codeScalar},
{name: "leading zero", input: "a: 01\n", code: codeScalar},
{name: "negative leading zero", input: "a: -01\n", code: codeScalar},
{name: "leading dot", input: "a: .5\n", code: codeScalar},
{name: "trailing dot", input: "a: 1.\n", code: codeScalar},
{name: "incomplete exponent", input: "a: 1e\n", code: codeScalar},
{name: "incomplete signed exponent", input: "a: 1e+\n", code: codeScalar},
{name: "infinity", input: "a: .inf\n", code: codeScalar},
{name: "nan", input: "a: .NaN\n", code: codeScalar},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
_, issues := Parse(strings.NewReader(test.input), "source-1", generousLimits())
if len(issues) != 1 || issues[0].Code != test.code {
t.Fatalf("issues = %#v, want code %q", issues, test.code)
}
})
}
}
func TestTranslationRejectsAliasNodes(t *testing.T) {
t.Parallel()
node := &yaml.Node{Kind: yaml.AliasNode, Line: 2, Column: 3}
state := translationState{sourceID: "source-1", limits: generousLimits()}
if _, issue := state.translate(node, 0); issue == nil || issue.Code != codeAlias {
t.Fatalf("alias issue = %#v", issue)
}
}
func TestParseAcceptsSupportedSyntax(t *testing.T) {
t.Parallel()
inputs := []string{
"plain: text\nquoted_bool: 'ON'\nquoted_number: \"01\"\nblock: |\n text\n",
"{zero: 0, negative: -1, fraction: 1.25, exponent: 1e+3, boolean: false}\n",
"\"true\": value\n\"1\": number key\n\"null\": null key\n",
}
for _, input := range inputs {
if _, issues := Parse(strings.NewReader(input), "source-1", generousLimits()); len(issues) != 0 {
t.Errorf("Parse(%q) issues = %#v", input, issues)
}
}
}
func TestDuplicateKeyReportsBothLocations(t *testing.T) {
t.Parallel()
_, issues := Parse(strings.NewReader("same: first\nsame: second\n"), "source-1", generousLimits())
if len(issues) != 1 || issues[0].Code != codeDuplicateKey || issues[0].Line != 2 || len(issues[0].Related) != 1 || issues[0].Related[0].Line != 1 {
t.Fatalf("duplicate issues = %#v", issues)
}
}
func TestParseEnforcesTreeAndParserLimits(t *testing.T) {
t.Parallel()
for _, input := range []string{"a:\n b: value\n", "{a: {b: value}}\n"} {
if _, issues := Parse(strings.NewReader(input), "source-1", Limits{Depth: 2, Nodes: 20, ParserDepth: 20}); len(issues) != 0 {
t.Fatalf("depth boundary rejected for %q: %#v", input, issues)
}
if _, issues := Parse(strings.NewReader(input), "source-1", Limits{Depth: 1, Nodes: 20, ParserDepth: 20}); len(issues) != 1 || issues[0].Code != codeDepthLimit {
t.Fatalf("depth overflow for %q = %#v", input, issues)
}
}
if _, issues := Parse(strings.NewReader("a: 1\n"), "source-1", Limits{Depth: 2, Nodes: 3, ParserDepth: 20}); len(issues) != 0 {
t.Fatalf("node boundary rejected: %#v", issues)
}
if _, issues := Parse(strings.NewReader("a: 1\n"), "source-1", Limits{Depth: 2, Nodes: 2, ParserDepth: 20}); len(issues) != 1 || issues[0].Code != codeNodeLimit {
t.Fatalf("node overflow = %#v", issues)
}
if _, issues := Parse(strings.NewReader("a:\n b:\n c: value\n"), "source-1", Limits{Depth: 20, Nodes: 20, ParserDepth: 1}); len(issues) != 1 || issues[0].Code != codeDepthLimit {
t.Fatalf("parser depth overflow = %#v", issues)
}
}
func TestParserDepthCalibrationForBlockAndFlow(t *testing.T) {
t.Parallel()
inputs := []string{"a:\n b: value\n", "{a: {b: value}}\n"}
for _, input := range inputs {
maximum := observedParserDepth(t, input)
if maximum > 4 {
t.Fatalf("parser depth %d exceeds root-zero tree depth plus safety offset", maximum)
}
if _, issues := Parse(strings.NewReader(input), "source-1", Limits{Depth: 2, Nodes: 20, ParserDepth: maximum}); len(issues) != 0 {
t.Fatalf("parser boundary %d rejected %q: %#v", maximum, input, issues)
}
if _, issues := Parse(strings.NewReader(input), "source-1", Limits{Depth: 20, Nodes: 20, ParserDepth: maximum - 1}); len(issues) != 1 || issues[0].Code != codeDepthLimit {
t.Fatalf("parser overflow at %d for %q: %#v", maximum-1, input, issues)
}
}
}
func observedParserDepth(t *testing.T, input string) int {
t.Helper()
maximum := 0
depthPlugin := limit.New(limit.DepthFunc(func(depth int, _ *limit.DepthContext) error {
maximum = max(maximum, depth)
return nil
}))
loader, err := yaml.NewLoader(strings.NewReader(input), yaml.WithPlugin(depthPlugin))
if err != nil {
t.Fatal(err)
}
var node yaml.Node
if err := loader.Load(&node); err != nil && err != io.EOF {
t.Fatal(err)
}
return maximum
}
func TestMalformedParserDiagnosticIsSafe(t *testing.T) {
t.Parallel()
const canary = "CANARY_SECRET_VALUE"
_, issues := Parse(strings.NewReader("value: ["+canary+"\n"), "source-1", generousLimits())
if len(issues) != 1 || issues[0].Code != codeInvalid || issues[0].Line == 0 {
t.Fatalf("issues = %#v", issues)
}
if strings.Contains(issues[0].Message, canary) {
t.Fatal("parser diagnostic exposed input")
}
}
func generousLimits() Limits {
return Limits{Depth: 32, Nodes: 1_000, ParserDepth: 64}
}

66
parse.go Normal file
View File

@@ -0,0 +1,66 @@
package configkit
import (
"bytes"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
yamladapter "gitea.maximumdirect.net/eric/ConfigKit/internal/yaml"
)
const (
maxDocumentDepth = 256
maxDocumentNodes = 1_000_000
)
type parseLimits struct {
source sourceLimits
document yamladapter.Limits
}
func defaultParseLimits() parseLimits {
return parseLimits{
source: defaultSourceLimits(),
document: yamladapter.Limits{
Depth: maxDocumentDepth,
Nodes: maxDocumentNodes,
},
}
}
type parsedSource struct {
document *document.Node
source *SourceInfo
}
func parseSource(source Source, sourceID string, limits parseLimits) (parsedSource, []Diagnostic) {
read, diagnostics := readSource(source, sourceID, limits.source)
if len(diagnostics) != 0 || read.info == nil {
return parsedSource{source: read.info}, diagnostics
}
root, issues := yamladapter.Parse(bytes.NewReader(read.data), sourceID, limits.document)
if len(issues) != 0 {
diagnostics := make([]Diagnostic, len(issues))
for i, issue := range issues {
diagnostics[i] = Diagnostic{
Code: Code(issue.Code),
Severity: SeverityError,
Message: issue.Message,
Primary: &Location{
SourceID: sourceID,
Line: issue.Line,
Column: issue.Column,
},
}
for _, related := range issue.Related {
diagnostics[i].Related = append(diagnostics[i].Related, Location{
SourceID: related.SourceID,
Line: related.Line,
Column: related.Column,
})
}
}
return parsedSource{source: read.info}, canonicalDiagnostics(diagnostics)
}
return parsedSource{document: root, source: read.info}, nil
}

38
parse_test.go Normal file
View File

@@ -0,0 +1,38 @@
package configkit
import (
"strings"
"testing"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
"gitea.maximumdirect.net/eric/ConfigKit/internal/yaml"
)
func TestParseSourceOwnsDocumentAndSafeDiagnostics(t *testing.T) {
t.Parallel()
limits := defaultParseLimits()
parsed, diagnostics := parseSource(Bytes("memory.yml", []byte("name: config\n")), "source-1", limits)
if len(diagnostics) != 0 || parsed.source == nil || parsed.source.Label != "memory.yml" || parsed.document.Kind != document.Mapping {
t.Fatalf("parse result = %#v, diagnostics = %#v", parsed, diagnostics)
}
const canary = "CANARY_PRIVATE_INPUT"
failed, diagnostics := parseSource(Bytes("unsafe\nlabel", []byte("name: ["+canary+"\n")), "source-1", limits)
if failed.document != nil || failed.source == nil || len(diagnostics) != 1 || diagnostics[0].Code != CodeYAMLInvalid {
t.Fatalf("failed parse = %#v, diagnostics = %#v", failed, diagnostics)
}
var output strings.Builder
if err := WriteDiagnostics(&output, diagnostics); err != nil {
t.Fatal(err)
}
if strings.Contains(output.String(), canary) || strings.Count(output.String(), "\n") != 1 {
t.Fatalf("unsafe rendered diagnostic = %q", output.String())
}
limits.document = yaml.Limits{Depth: 1, Nodes: 20, ParserDepth: 20}
_, diagnostics = parseSource(Bytes("deep", []byte("a:\n b: value\n")), "source-1", limits)
if len(diagnostics) != 1 || diagnostics[0].Code != CodeYAMLDepthLimit || diagnostics[0].Primary.Line == 0 {
t.Fatalf("depth diagnostics = %#v", diagnostics)
}
}

23
records.go
View File

@@ -31,6 +31,29 @@ const (
// Code is a stable, machine-readable diagnostic classification.
type Code string
const (
CodeSourceInvalid Code = "source.invalid"
CodeSourceMissing Code = "source.missing"
CodeSourcePermission Code = "source.permission"
CodeSourceRead Code = "source.read"
CodeSourceNotRegular Code = "source.not_regular"
CodeSourceLimitExceeded Code = "source.limit_exceeded"
CodeSourceReaderUsed Code = "source.reader_used"
CodeYAMLInvalid Code = "yaml.invalid"
CodeYAMLEmpty Code = "yaml.empty"
CodeYAMLMultipleDocuments Code = "yaml.multiple_documents"
CodeYAMLRootType Code = "yaml.root_type"
CodeYAMLDuplicateKey Code = "yaml.duplicate_key"
CodeYAMLTag Code = "yaml.tag"
CodeYAMLAnchor Code = "yaml.anchor"
CodeYAMLAlias Code = "yaml.alias"
CodeYAMLMergeKey Code = "yaml.merge_key"
CodeYAMLScalar Code = "yaml.scalar"
CodeYAMLDepthLimit Code = "yaml.depth_limit"
CodeYAMLNodeLimit Code = "yaml.node_limit"
)
// Location identifies a position in a configuration source. Zero line and column
// values mean that coordinates are unavailable or synthetic.
type Location struct {

235
source.go Normal file
View File

@@ -0,0 +1,235 @@
package configkit
import (
"bytes"
"errors"
"io"
"io/fs"
"os"
"path/filepath"
"reflect"
"sync/atomic"
)
const (
maxSourceBytes = 8 << 20
maxTotalBytes = 64 << 20
)
// Source is an opaque configuration input. Construct sources with File, Bytes,
// Reader, or NoSource; the zero value is invalid.
type Source struct {
kind sourceKind
path string
label string
data []byte
reader *readerSource
createErr sourceCreationError
}
type sourceKind uint8
const (
sourceInvalid sourceKind = iota
sourceFile
sourceBytes
sourceReader
sourceNone
)
type sourceCreationError uint8
const (
creationValid sourceCreationError = iota
creationInvalidPath
creationInvalidLabel
creationInvalidReader
creationTooLarge
)
type readerSource struct {
input io.Reader
used atomic.Bool
}
// File returns a source that reopens path for every load attempt.
func File(path string) Source {
if path == "" {
return Source{kind: sourceFile, createErr: creationInvalidPath}
}
return Source{kind: sourceFile, path: path, label: path}
}
// Bytes snapshots data immediately. Label must be nonempty and safe to display;
// constructor problems are reported when the source is consumed.
func Bytes(label string, data []byte) Source {
source := Source{kind: sourceBytes, label: label}
if label == "" {
source.createErr = creationInvalidLabel
return source
}
if len(data) > maxSourceBytes {
source.createErr = creationTooLarge
return source
}
source.data = bytes.Clone(data)
return source
}
// Reader returns a single-use source. ConfigKit consumes input but never closes it.
// Copies of the returned Source share the same single-use claim.
func Reader(label string, input io.Reader) Source {
source := Source{kind: sourceReader, label: label}
if label == "" {
source.createErr = creationInvalidLabel
return source
}
if nilReader(input) {
source.createErr = creationInvalidReader
return source
}
source.reader = &readerSource{input: input}
return source
}
// NoSource explicitly selects a defaults-only input. It creates no source record.
func NoSource() Source {
return Source{kind: sourceNone}
}
type sourceLimits struct {
perSource int64
remaining int64
}
func defaultSourceLimits() sourceLimits {
return sourceLimits{perSource: maxSourceBytes, remaining: maxTotalBytes}
}
type sourceRead struct {
data []byte
info *SourceInfo
}
func readSource(source Source, sourceID string, limits sourceLimits) (sourceRead, []Diagnostic) {
if diagnostic := sourceConstructorDiagnostic(source); diagnostic != nil {
return sourceRead{}, []Diagnostic{*diagnostic}
}
if source.kind == sourceNone {
return sourceRead{}, nil
}
limit := min(limits.perSource, limits.remaining)
if limit < 0 {
limit = 0
}
info := &SourceInfo{ID: sourceID, Label: source.label, Layer: LayerBase}
var (
input io.Reader
close func() error
)
switch source.kind {
case sourceFile:
absolute, err := filepath.Abs(source.path)
if err != nil {
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceRead, sourceID, "source path could not be resolved")}
}
info.Path = filepath.Clean(absolute)
file, err := os.Open(source.path)
if err != nil {
return sourceRead{info: info}, []Diagnostic{classifySourceError(err, sourceID)}
}
close = file.Close
stat, err := file.Stat()
if err != nil {
_ = file.Close()
return sourceRead{info: info}, []Diagnostic{classifySourceError(err, sourceID)}
}
if !stat.Mode().IsRegular() {
_ = file.Close()
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceNotRegular, sourceID, "source is not a regular file")}
}
if stat.Size() > limit {
_ = file.Close()
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceLimitExceeded, sourceID, "source exceeds the configured byte limit")}
}
input = file
case sourceBytes:
input = bytes.NewReader(source.data)
case sourceReader:
if !source.reader.used.CompareAndSwap(false, true) {
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceReaderUsed, sourceID, "reader source has already been consumed")}
}
input = source.reader.input
default:
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceInvalid, sourceID, "source is invalid")}
}
if close != nil {
defer func() { _ = close() }()
}
data, err := io.ReadAll(io.LimitReader(input, limit+1))
if err != nil {
return sourceRead{info: info}, []Diagnostic{classifySourceError(err, sourceID)}
}
if int64(len(data)) > limit {
return sourceRead{info: info}, []Diagnostic{sourceDiagnostic(CodeSourceLimitExceeded, sourceID, "source exceeds the configured byte limit")}
}
return sourceRead{data: bytes.Clone(data), info: info}, nil
}
func sourceConstructorDiagnostic(source Source) *Diagnostic {
var message string
code := CodeSourceInvalid
switch source.createErr {
case creationValid:
if source.kind != sourceInvalid {
return nil
}
message = "source is invalid"
case creationInvalidPath:
message = "source path must not be empty"
case creationInvalidLabel:
message = "source label must not be empty"
case creationInvalidReader:
message = "source reader must not be nil"
case creationTooLarge:
code = CodeSourceLimitExceeded
message = "source exceeds the configured byte limit"
}
diagnostic := sourceDiagnostic(code, "", message)
return &diagnostic
}
func classifySourceError(err error, sourceID string) Diagnostic {
switch {
case errors.Is(err, fs.ErrNotExist):
return sourceDiagnostic(CodeSourceMissing, sourceID, "source does not exist")
case errors.Is(err, fs.ErrPermission):
return sourceDiagnostic(CodeSourcePermission, sourceID, "source cannot be accessed")
default:
return sourceDiagnostic(CodeSourceRead, sourceID, "source could not be read")
}
}
func sourceDiagnostic(code Code, sourceID, message string) Diagnostic {
diagnostic := Diagnostic{Code: code, Severity: SeverityError, Message: message}
if sourceID != "" {
diagnostic.Primary = &Location{SourceID: sourceID}
}
return diagnostic
}
func nilReader(input io.Reader) bool {
if input == nil {
return true
}
value := reflect.ValueOf(input)
switch value.Kind() {
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return value.IsNil()
default:
return false
}
}

202
source_test.go Normal file
View File

@@ -0,0 +1,202 @@
package configkit
import (
"errors"
"io"
"os"
"path/filepath"
"strings"
"sync"
"testing"
)
func TestSourceConstructorsAndOwnership(t *testing.T) {
t.Parallel()
data := []byte("value: before\n")
source := Bytes("memory", data)
copy(data, "value: after!\n")
read, diagnostics := readSource(source, "source-1", defaultSourceLimits())
if len(diagnostics) != 0 || string(read.data) != "value: before\n" {
t.Fatalf("byte snapshot = %q, diagnostics = %#v", read.data, diagnostics)
}
read.data[0] = 'X'
again, diagnostics := readSource(source, "source-1", defaultSourceLimits())
if len(diagnostics) != 0 || string(again.data) != "value: before\n" {
t.Fatal("returned bytes alias source snapshot")
}
closer := &observedReadCloser{Reader: strings.NewReader("value: reader\n")}
readerSource := Reader("stream", closer)
read, diagnostics = readSource(readerSource, "source-1", defaultSourceLimits())
if len(diagnostics) != 0 || string(read.data) != "value: reader\n" || closer.closed {
t.Fatalf("reader result = %q, closed=%v diagnostics=%#v", read.data, closer.closed, diagnostics)
}
_, diagnostics = readSource(readerSource, "source-1", defaultSourceLimits())
if len(diagnostics) != 1 || diagnostics[0].Code != CodeSourceReaderUsed {
t.Fatalf("second reader diagnostics = %#v", diagnostics)
}
if read, diagnostics := readSource(NoSource(), "source-1", defaultSourceLimits()); len(diagnostics) != 0 || read.info != nil || read.data != nil {
t.Fatalf("NoSource read = %#v, %#v", read, diagnostics)
}
}
func TestFileSourcesReopenAndClassifyFailures(t *testing.T) {
t.Parallel()
directory := t.TempDir()
path := filepath.Join(directory, "config.yml")
if err := os.WriteFile(path, []byte("value: first\n"), 0o600); err != nil {
t.Fatal(err)
}
source := File(path)
first, diagnostics := readSource(source, "source-1", defaultSourceLimits())
if len(diagnostics) != 0 || string(first.data) != "value: first\n" || first.info.Path != path {
t.Fatalf("first read = %#v, %#v", first, diagnostics)
}
if err := os.WriteFile(path, []byte("value: second\n"), 0o600); err != nil {
t.Fatal(err)
}
second, diagnostics := readSource(source, "source-1", defaultSourceLimits())
if len(diagnostics) != 0 || string(second.data) != "value: second\n" {
t.Fatalf("second read = %#v, %#v", second, diagnostics)
}
missing, diagnostics := readSource(File(filepath.Join(directory, "missing.yml")), "source-2", defaultSourceLimits())
if len(diagnostics) != 1 || diagnostics[0].Code != CodeSourceMissing || missing.info == nil {
t.Fatalf("missing diagnostics = %#v, info=%#v", diagnostics, missing.info)
}
_, diagnostics = readSource(File(directory), "source-3", defaultSourceLimits())
if len(diagnostics) != 1 || diagnostics[0].Code != CodeSourceNotRegular {
t.Fatalf("directory diagnostics = %#v", diagnostics)
}
permissionPath := filepath.Join(directory, "private.yml")
if err := os.WriteFile(permissionPath, []byte("value: private\n"), 0); err != nil {
t.Fatal(err)
}
_, diagnostics = readSource(File(permissionPath), "source-4", defaultSourceLimits())
if os.Geteuid() != 0 {
if len(diagnostics) != 1 || diagnostics[0].Code != CodeSourcePermission {
t.Fatalf("permission diagnostics = %#v", diagnostics)
}
}
}
func TestSourceReadLimitsAndErrors(t *testing.T) {
t.Parallel()
limits := sourceLimits{perSource: 5, remaining: 20}
read, diagnostics := readSource(Reader("exact", &fragmentReader{data: []byte("12345"), width: 1}), "source-1", limits)
if len(diagnostics) != 0 || string(read.data) != "12345" {
t.Fatalf("exact limit read = %q, %#v", read.data, diagnostics)
}
_, diagnostics = readSource(Reader("over", strings.NewReader("123456")), "source-2", limits)
if len(diagnostics) != 1 || diagnostics[0].Code != CodeSourceLimitExceeded {
t.Fatalf("over-limit diagnostics = %#v", diagnostics)
}
_, diagnostics = readSource(Reader("total", strings.NewReader("1234")), "source-3", sourceLimits{perSource: 10, remaining: 3})
if len(diagnostics) != 1 || diagnostics[0].Code != CodeSourceLimitExceeded {
t.Fatalf("total-limit diagnostics = %#v", diagnostics)
}
want := errors.New("read canary: must not escape")
_, diagnostics = readSource(Reader("failing", &dataErrorReader{data: []byte("partial"), err: want}), "source-4", limits)
if len(diagnostics) != 1 || diagnostics[0].Code != CodeSourceRead || strings.Contains(diagnostics[0].Message, "canary") {
t.Fatalf("read-error diagnostics = %#v", diagnostics)
}
}
func TestReaderSourceCopiesCompeteForOneClaim(t *testing.T) {
t.Parallel()
source := Reader("shared", strings.NewReader("value: one\n"))
copyOfSource := source
start := make(chan struct{})
results := make(chan []Diagnostic, 2)
var group sync.WaitGroup
for _, candidate := range []Source{source, copyOfSource} {
group.Add(1)
go func(candidate Source) {
defer group.Done()
<-start
_, diagnostics := readSource(candidate, "source-1", defaultSourceLimits())
results <- diagnostics
}(candidate)
}
close(start)
group.Wait()
close(results)
successes, used := 0, 0
for diagnostics := range results {
if len(diagnostics) == 0 {
successes++
} else if len(diagnostics) == 1 && diagnostics[0].Code == CodeSourceReaderUsed {
used++
}
}
if successes != 1 || used != 1 {
t.Fatalf("successes=%d reader-used=%d", successes, used)
}
}
func TestInvalidSourcesFailWithoutConsumption(t *testing.T) {
t.Parallel()
var typedNil *strings.Reader
for name, source := range map[string]Source{
"zero": {},
"empty file": File(""),
"empty bytes": Bytes("", nil),
"nil reader": Reader("reader", nil),
"typed nil": Reader("reader", typedNil),
"large bytes": Bytes("large", make([]byte, maxSourceBytes+1)),
} {
t.Run(name, func(t *testing.T) {
_, diagnostics := readSource(source, "source-1", defaultSourceLimits())
if len(diagnostics) != 1 {
t.Fatalf("diagnostics = %#v", diagnostics)
}
})
}
}
type observedReadCloser struct {
io.Reader
closed bool
}
func (r *observedReadCloser) Close() error {
r.closed = true
return nil
}
type fragmentReader struct {
data []byte
width int
}
func (r *fragmentReader) Read(p []byte) (int, error) {
if len(r.data) == 0 {
return 0, io.EOF
}
n := min(len(p), r.width, len(r.data))
copy(p, r.data[:n])
r.data = r.data[n:]
return n, nil
}
type dataErrorReader struct {
data []byte
err error
}
func (r *dataErrorReader) Read(p []byte) (int, error) {
if len(r.data) == 0 {
return 0, r.err
}
n := copy(p, r.data)
r.data = r.data[n:]
return n, r.err
}