Implement canonical projection and detached copies

This commit is contained in:
2026-09-06 19:44:36 +00:00
parent 2b65876a7d
commit 15172294b5
8 changed files with 1059 additions and 23 deletions

View File

@@ -101,8 +101,39 @@ tokens become indices only in a sequence. Sequence indices cannot have leading
zeroes; fixed-array indices are bounds checked. Dynamic `any` and inline-map entries
remain schema-valid for application-defined descendants.
Descriptors do not yet decode documents, clone defaults, or project application
values. Those operations will reuse this graph as their implementation lands.
Descriptors do not yet decode documents. Document decoding will reuse this graph as
its implementation lands.
## Typed projection and ownership
ConfigKit can internally project a schema-valid application value directly into its
owned canonical tree without YAML serialization. Projection includes every bound
field, including ordinary zero values; `omitempty` has no effect. Struct fields keep
binding order, dynamic map keys sort by decoded string, and sequences keep their
order. Integers retain exact base-10 values, finite floats use their destination
precision, durations use `time.Duration.String`, and paired text scalars use their
canonical marshaled text.
Nil pointers, maps, slices, and dynamic values project as null. Non-nil empty maps
and slices remain empty containers, preserving the distinction. Recursive acyclic
objects are supported. Cyclic values, non-finite floats, unsupported dynamic concrete
types, invalid codec output, and depth/node-limit violations fail with safe schema
classifications that omit codec error text.
The same descriptor graph creates detached typed copies. Pointers, maps, slices,
arrays, nested dynamic values, and shared references are recursively copied; shared
pointer identity is not preserved as configuration meaning. Paired text scalars are
rebuilt by unmarshaling their canonical text into fresh values, detaching private
backing storage. Ignored and unexported struct fields are zeroed rather than copied.
Private structural comparison operates on canonical trees. Mapping additions,
removals, and descendant replacements retain semantic paths. A sequence is atomic:
any sequence change reports the sequence path and never invents correspondence
between old and new indices. Cross-kind and null/container changes replace the
affected path.
These mechanisms currently support later decoding, resolution auditing, metadata,
and rendering. They are not a public generic conversion or cloning API.
## Safe failures

View File

@@ -11,10 +11,13 @@ module and package documentation boundary. Its implemented public primitives are
- the sole upstream YAML integration in `internal/yaml`, which translates validated
input into that owned tree and exposes no parser types outside its boundary; and
- a per-operation private reflection descriptor graph in `schema.go` for supported
types, YAML field binding, inlining, secret tags, recursion, and semantic paths.
types, YAML field binding, inlining, secret tags, recursion, and semantic paths;
and
- descriptor-driven canonical projection, detached typed cloning, cycle/limit
enforcement, and structural comparison in `projection.go`.
The [configuration input contract](../config.md) owns implemented source and YAML
behavior. ConfigKit intentionally exposes no typed loading or decoding API yet.
The [configuration contract](../config.md) owns implemented input, schema, and
ownership behavior. ConfigKit intentionally exposes no typed loading or decoding API yet.
Private metadata population will be integrated when that lifecycle exists.
The source-extraction inventory records the application foundations assigned to

View File

@@ -2,7 +2,7 @@
## Status and ownership
Status: implementation in progress; Stages 1–4 are complete and later stages are
Status: implementation in progress; Stages 1–5 are complete and later stages are
pending.
Audience: a `gpt-5.6-terra` coding agent implementing one stage per prompt, in
@@ -512,7 +512,30 @@ Acceptance:
## Stage 5 — Implement canonical projection and detached copies
Status: pending.
Status: complete.
Completion evidence (2026-09-06):
- Added descriptor-driven canonical typed projection and independently owned cloning
in [`projection.go`](../../projection.go), without YAML serialization or a second
binding model. Zero values, nil/empty distinctions, exact integers, destination-
precision finite floats, durations, text scalars, dynamic concrete values, and
recursive acyclic objects retain their configuration meaning.
- Added active-graph cycle detection, private depth/node enforcement, safe custom
codec and dynamic-type failures, zeroing of ignored state, reconstruction of text
scalars into fresh instances, and non-preservation of shared pointer identity.
- Added canonical-tree structural comparison with semantic mapping additions,
removals and replacements, cross-kind replacement, and whole-sequence atomicity.
The [current ownership contract](../config.md#typed-projection-and-ownership) and
component inventory now own the implemented behavior.
- Focused tests mutate every source alias after cloning, cover nil versus empty,
wide integers, duration/text canonicalization, recursive acyclic values and cycles,
unsupported dynamic types, non-finite floats, value-free codec failures, inline
physical members/conflicts, projection bounds, ignored state, and meaningful
comparison paths without index-wise sequence matching. Verified `go test ./...`,
`go vet ./...`, `go build ./...`, `go mod tidy -diff`, and `git diff --check` with
Go 1.26.7 on linux/amd64. No deviations were required; defaults and typed decoding
remain later work.
Read: feature [ownership](initial.md#recorded-resolution-and-validation) and
[type/projection contract](initial.md#supported-types-and-projection).

View File

@@ -645,6 +645,11 @@ subsequently mutate, not language-enforced immutability of exported Go values.
Callers should treat returned configuration as read-only; later caller mutations
are outside the captured metadata snapshot.
Canonical typed projection, detached cloning, and structural comparison are now
implemented as private foundations. The [current ownership contract](../config.md#typed-projection-and-ownership)
owns those mechanics; publishing independently captured results remains lifecycle
work.
Application semantic validation runs after successful resolution and collects
independent errors in a standard diagnostic form. Validators are read-only and
own domain requirements; checking live S3 buckets or starting runtime clients
@@ -1304,20 +1309,12 @@ sequence length. Ignored and unexported fields stay outside configuration projec
hooks must not use them to carry runtime resources or configuration needing
inspection. Non-configuration runtime state belongs in separate application types.
One field-binding model supplies defaults, decoding, path lookup, change auditing,
and rendering. `omitempty` never removes projected zero values. Preserve nil
versus empty maps/slices where represented by the type. Shared pointer identity
inside application values is not a configuration semantic; cyclic values are
unsupported and independent load/snapshot ownership must still hold.
`time.Duration` projects to a canonical duration string. A custom scalar's text
marshal method supplies its deterministic canonical string; unmarshalling must
accept that representation and preserve its configuration meaning. These methods
must be side-effect-free with respect to shared state. Custom scalars are atomic
for comparison/history; their internal fields are not traversed. Reject types
with only a text decoder and no supported inspection representation. Conversion
or projection failures produce safe diagnostics. This is a bounded scalar seam,
not a custom structural decoder or arbitrary reflection extension framework.
Canonical projection, detached copying, cycle and finite-value enforcement, custom
scalar reconstruction, and structural comparison are implemented and owned by the
[current ownership contract](../config.md#typed-projection-and-ownership). The same
field-binding model must still supply defaults, decoding, change auditing, and
rendering. Application text methods must remain side-effect-free with respect to
shared state; this is a bounded scalar seam, not a custom structural decoder.
## Internal organization

567
projection.go Normal file
View File

@@ -0,0 +1,567 @@
package configkit
import (
"encoding"
"math"
"reflect"
"slices"
"strconv"
"time"
"unicode/utf8"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
)
type projectionLimits struct {
depth int
nodes int
}
func defaultProjectionLimits() projectionLimits {
return projectionLimits{depth: maxDocumentDepth, nodes: maxDocumentNodes}
}
type projectionIssue struct {
code Code
path Path
message string
}
type valueVisit struct {
typeOf reflect.Type
kind reflect.Kind
ptr uintptr
}
type valueOperation struct {
schema *schemaDescriptor
limits projectionLimits
active map[valueVisit]bool
nodes int
}
func projectConfiguration(schema *schemaDescriptor, value reflect.Value, limits projectionLimits) (*document.Node, *projectionIssue) {
if schema == nil || schema.root == nil || !value.IsValid() || value.Type() != schema.root.typeOf {
return nil, &projectionIssue{code: CodeSchemaUnsupportedType, message: "configuration value does not match its schema"}
}
operation := valueOperation{schema: schema, limits: limits, active: make(map[valueVisit]bool)}
return operation.project(value, schema.root, Path{}, 0)
}
func cloneConfiguration(schema *schemaDescriptor, value reflect.Value, limits projectionLimits) (reflect.Value, *projectionIssue) {
if schema == nil || schema.root == nil || !value.IsValid() || value.Type() != schema.root.typeOf {
return reflect.Value{}, &projectionIssue{code: CodeSchemaUnsupportedType, message: "configuration value does not match its schema"}
}
operation := valueOperation{schema: schema, limits: limits, active: make(map[valueVisit]bool)}
return operation.clone(value, schema.root, Path{}, 0)
}
func (o *valueOperation) project(value reflect.Value, descriptor *typeDescriptor, path Path, depth int) (*document.Node, *projectionIssue) {
if issue := o.checkDepth(path, depth); issue != nil {
return nil, issue
}
if descriptor.kind == descriptorPointer {
if value.IsNil() {
return o.node(document.Null, path, depth)
}
leave, issue := o.enter(value, path)
if issue != nil {
return nil, issue
}
defer leave()
return o.project(value.Elem(), descriptor.element, path, depth)
}
switch descriptor.kind {
case descriptorStruct:
result, issue := o.node(document.Mapping, path, depth)
if issue != nil {
return nil, issue
}
result.Mapping = make([]document.Entry, 0, len(descriptor.fields))
seen := make(map[string]bool, len(descriptor.fields))
for _, member := range descriptor.members {
memberValue := value.Field(member.index)
if member.inline {
if isNilValue(memberValue) {
continue
}
inline, issue := o.project(memberValue, member.descriptor, path, depth)
if issue != nil {
return nil, issue
}
if inline.Kind != document.Mapping {
return nil, &projectionIssue{code: CodeSchemaUnsupportedType, path: path.clone(), message: "inline value did not project to a mapping"}
}
o.nodes-- // The flattened container is not retained in the output tree.
for _, entry := range inline.Mapping {
if seen[entry.Key] {
return nil, &projectionIssue{code: CodeSchemaAmbiguousField, path: path.Key(entry.Key), message: "inline value conflicts with a declared field"}
}
seen[entry.Key] = true
result.Mapping = append(result.Mapping, entry)
}
continue
}
childPath := path.Key(member.name)
if seen[member.name] {
return nil, &projectionIssue{code: CodeSchemaAmbiguousField, path: childPath, message: "inline value conflicts with a declared field"}
}
if issue := o.charge(childPath); issue != nil { // mapping key
return nil, issue
}
child, issue := o.project(memberValue, member.descriptor, childPath, depth+1)
if issue != nil {
return nil, issue
}
seen[member.name] = true
result.Mapping = append(result.Mapping, document.Entry{Key: member.name, Value: child})
}
return result, nil
case descriptorMap:
if value.IsNil() {
return o.node(document.Null, path, depth)
}
leave, issue := o.enter(value, path)
if issue != nil {
return nil, issue
}
defer leave()
result, issue := o.node(document.Mapping, path, depth)
if issue != nil {
return nil, issue
}
result.Mapping = make([]document.Entry, 0, value.Len())
keys := value.MapKeys()
slices.SortFunc(keys, func(left, right reflect.Value) int {
return cmpString(left.String(), right.String())
})
for _, key := range keys {
childPath := path.Key(key.String())
if issue := o.charge(childPath); issue != nil {
return nil, issue
}
child, issue := o.project(value.MapIndex(key), descriptor.element, childPath, depth+1)
if issue != nil {
return nil, issue
}
result.Mapping = append(result.Mapping, document.Entry{Key: key.String(), Value: child})
}
return result, nil
case descriptorSlice:
if value.IsNil() {
return o.node(document.Null, path, depth)
}
leave, issue := o.enter(value, path)
if issue != nil {
return nil, issue
}
defer leave()
fallthrough
case descriptorArray:
result, issue := o.node(document.Sequence, path, depth)
if issue != nil {
return nil, issue
}
result.Sequence = make([]*document.Node, 0, value.Len())
for i := 0; i < value.Len(); i++ {
childPath := path.Index(i)
child, issue := o.project(value.Index(i), descriptor.element, childPath, depth+1)
if issue != nil {
return nil, issue
}
result.Sequence = append(result.Sequence, child)
}
return result, nil
case descriptorDynamic:
if value.IsNil() {
return o.node(document.Null, path, depth)
}
concrete := value.Elem()
dynamic, issue := o.dynamicDescriptor(concrete.Type(), path)
if issue != nil {
return nil, issue
}
return o.project(concrete, dynamic, path, depth)
case descriptorString:
node, issue := o.node(document.String, path, depth)
if issue == nil {
node.Text = value.String()
}
return node, issue
case descriptorBoolean:
node, issue := o.node(document.Boolean, path, depth)
if issue == nil {
node.Text = strconv.FormatBool(value.Bool())
}
return node, issue
case descriptorSigned:
node, issue := o.node(document.Number, path, depth)
if issue == nil {
node.Text = strconv.FormatInt(value.Int(), 10)
}
return node, issue
case descriptorUnsigned:
node, issue := o.node(document.Number, path, depth)
if issue == nil {
node.Text = strconv.FormatUint(value.Uint(), 10)
}
return node, issue
case descriptorFloat:
if math.IsNaN(value.Float()) || math.IsInf(value.Float(), 0) {
return nil, &projectionIssue{code: CodeSchemaNonFinite, path: path.clone(), message: "configuration float must be finite"}
}
node, issue := o.node(document.Number, path, depth)
if issue == nil {
node.Text = strconv.FormatFloat(value.Float(), 'g', -1, value.Type().Bits())
}
return node, issue
case descriptorDuration:
node, issue := o.node(document.String, path, depth)
if issue == nil {
node.Text = time.Duration(value.Int()).String()
}
return node, issue
case descriptorText:
text, issue := marshalCanonicalText(value, path)
if issue != nil {
return nil, issue
}
node, nodeIssue := o.node(document.String, path, depth)
if nodeIssue == nil {
node.Text = text
}
return node, nodeIssue
default:
return nil, &projectionIssue{code: CodeSchemaUnsupportedType, path: path.clone(), message: "configuration type is not supported"}
}
}
func (o *valueOperation) clone(value reflect.Value, descriptor *typeDescriptor, path Path, depth int) (reflect.Value, *projectionIssue) {
if issue := o.checkDepth(path, depth); issue != nil {
return reflect.Value{}, issue
}
if descriptor.kind == descriptorPointer {
if value.IsNil() {
return reflect.Zero(value.Type()), nil
}
leave, issue := o.enter(value, path)
if issue != nil {
return reflect.Value{}, issue
}
defer leave()
child, issue := o.clone(value.Elem(), descriptor.element, path, depth)
if issue != nil {
return reflect.Value{}, issue
}
result := reflect.New(value.Type().Elem())
result.Elem().Set(child)
return result, nil
}
switch descriptor.kind {
case descriptorStruct:
result := reflect.New(value.Type()).Elem()
for _, member := range descriptor.members {
memberPath, memberDepth := path, depth
if !member.inline {
memberPath, memberDepth = path.Key(member.name), depth+1
}
cloned, issue := o.clone(value.Field(member.index), member.descriptor, memberPath, memberDepth)
if issue != nil {
return reflect.Value{}, issue
}
result.Field(member.index).Set(cloned)
}
return result, nil
case descriptorMap:
if value.IsNil() {
return reflect.Zero(value.Type()), nil
}
leave, issue := o.enter(value, path)
if issue != nil {
return reflect.Value{}, issue
}
defer leave()
result := reflect.MakeMapWithSize(value.Type(), value.Len())
for _, key := range value.MapKeys() {
cloned, issue := o.clone(value.MapIndex(key), descriptor.element, path.Key(key.String()), depth+1)
if issue != nil {
return reflect.Value{}, issue
}
result.SetMapIndex(key, cloned)
}
return result, nil
case descriptorSlice:
if value.IsNil() {
return reflect.Zero(value.Type()), nil
}
leave, issue := o.enter(value, path)
if issue != nil {
return reflect.Value{}, issue
}
defer leave()
result := reflect.MakeSlice(value.Type(), value.Len(), value.Len())
for i := 0; i < value.Len(); i++ {
cloned, issue := o.clone(value.Index(i), descriptor.element, path.Index(i), depth+1)
if issue != nil {
return reflect.Value{}, issue
}
result.Index(i).Set(cloned)
}
return result, nil
case descriptorArray:
result := reflect.New(value.Type()).Elem()
for i := 0; i < value.Len(); i++ {
cloned, issue := o.clone(value.Index(i), descriptor.element, path.Index(i), depth+1)
if issue != nil {
return reflect.Value{}, issue
}
result.Index(i).Set(cloned)
}
return result, nil
case descriptorDynamic:
if value.IsNil() {
return reflect.Zero(value.Type()), nil
}
concrete := value.Elem()
dynamic, issue := o.dynamicDescriptor(concrete.Type(), path)
if issue != nil {
return reflect.Value{}, issue
}
cloned, issue := o.clone(concrete, dynamic, path, depth)
if issue != nil {
return reflect.Value{}, issue
}
result := reflect.New(value.Type()).Elem()
result.Set(cloned)
return result, nil
case descriptorText:
text, issue := marshalCanonicalText(value, path)
if issue != nil {
return reflect.Value{}, issue
}
result := reflect.New(value.Type()).Elem()
unmarshaler := result.Addr().Interface().(encoding.TextUnmarshaler)
if err := unmarshaler.UnmarshalText([]byte(text)); err != nil {
return reflect.Value{}, &projectionIssue{code: CodeSchemaCodec, path: path.clone(), message: "custom text scalar could not be reconstructed"}
}
return result, nil
case descriptorFloat:
if math.IsNaN(value.Float()) || math.IsInf(value.Float(), 0) {
return reflect.Value{}, &projectionIssue{code: CodeSchemaNonFinite, path: path.clone(), message: "configuration float must be finite"}
}
fallthrough
default:
result := reflect.New(value.Type()).Elem()
result.Set(value)
return result, nil
}
}
func (o *valueOperation) dynamicDescriptor(current reflect.Type, path Path) (*typeDescriptor, *projectionIssue) {
builder := schemaBuilder{cache: o.schema.types, building: make(map[reflect.Type]bool)}
descriptor, issue := builder.describe(current, path)
if issue != nil {
return nil, &projectionIssue{code: CodeSchemaDynamicType, path: path.clone(), message: "dynamic configuration value has an unsupported type"}
}
return descriptor, nil
}
func (o *valueOperation) node(kind document.Kind, path Path, depth int) (*document.Node, *projectionIssue) {
if issue := o.checkDepth(path, depth); issue != nil {
return nil, issue
}
if issue := o.charge(path); issue != nil {
return nil, issue
}
return &document.Node{Kind: kind}, nil
}
func (o *valueOperation) charge(path Path) *projectionIssue {
o.nodes++
if o.limits.nodes >= 0 && o.nodes > o.limits.nodes {
return &projectionIssue{code: CodeSchemaProjectionLimit, path: path.clone(), message: "configuration projection exceeds the node limit"}
}
return nil
}
func (o *valueOperation) checkDepth(path Path, depth int) *projectionIssue {
if depth > o.limits.depth {
return &projectionIssue{code: CodeSchemaProjectionLimit, path: path.clone(), message: "configuration projection exceeds the depth limit"}
}
return nil
}
func (o *valueOperation) enter(value reflect.Value, path Path) (func(), *projectionIssue) {
visit, ok := visitFor(value)
if !ok {
return func() {}, nil
}
if o.active[visit] {
return nil, &projectionIssue{code: CodeSchemaCycle, path: path.clone(), message: "configuration values must not contain cycles"}
}
o.active[visit] = true
return func() { delete(o.active, visit) }, nil
}
func visitFor(value reflect.Value) (valueVisit, bool) {
if isNilValue(value) {
return valueVisit{}, false
}
visit := valueVisit{typeOf: value.Type(), kind: value.Kind()}
switch value.Kind() {
case reflect.Pointer, reflect.Slice:
if value.Kind() == reflect.Slice && value.Len() == 0 {
return valueVisit{}, false
}
visit.ptr = value.Pointer()
case reflect.Map:
if value.Len() == 0 {
return valueVisit{}, false
}
visit.ptr = uintptr(value.UnsafePointer())
default:
return valueVisit{}, false
}
return visit, visit.ptr != 0
}
func isNilValue(value reflect.Value) bool {
switch value.Kind() {
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return value.IsNil()
default:
return false
}
}
func marshalCanonicalText(value reflect.Value, path Path) (string, *projectionIssue) {
var marshaler encoding.TextMarshaler
if value.CanInterface() {
marshaler, _ = value.Interface().(encoding.TextMarshaler)
}
if marshaler == nil {
addressable := reflect.New(value.Type())
addressable.Elem().Set(value)
marshaler, _ = addressable.Interface().(encoding.TextMarshaler)
}
if marshaler == nil {
return "", &projectionIssue{code: CodeSchemaCodec, path: path.clone(), message: "custom text scalar has no usable text marshaler"}
}
text, err := marshaler.MarshalText()
if err != nil || !utf8.Valid(text) {
return "", &projectionIssue{code: CodeSchemaCodec, path: path.clone(), message: "custom text scalar could not be projected"}
}
return string(text), nil
}
func cmpString(left, right string) int {
switch {
case left < right:
return -1
case left > right:
return 1
default:
return 0
}
}
type documentDeltaKind uint8
const (
documentAdded documentDeltaKind = iota + 1
documentRemoved
documentReplaced
)
type documentDelta struct {
path Path
kind documentDeltaKind
}
func compareDocumentTrees(before, after *document.Node) []documentDelta {
var deltas []documentDelta
compareDocumentNode(before, after, Path{}, &deltas)
slices.SortFunc(deltas, func(left, right documentDelta) int {
if order := cmpString(left.path.String(), right.path.String()); order != 0 {
return order
}
return int(left.kind) - int(right.kind)
})
return deltas
}
func compareDocumentNode(before, after *document.Node, path Path, deltas *[]documentDelta) {
if before == nil && after == nil {
return
}
if before == nil {
*deltas = append(*deltas, documentDelta{path: path.clone(), kind: documentAdded})
return
}
if after == nil {
*deltas = append(*deltas, documentDelta{path: path.clone(), kind: documentRemoved})
return
}
if before.Kind != after.Kind {
*deltas = append(*deltas, documentDelta{path: path.clone(), kind: documentReplaced})
return
}
switch before.Kind {
case document.Mapping:
beforeEntries := make(map[string]*document.Node, len(before.Mapping))
afterEntries := make(map[string]*document.Node, len(after.Mapping))
keys := make(map[string]bool, len(before.Mapping)+len(after.Mapping))
for _, entry := range before.Mapping {
beforeEntries[entry.Key] = entry.Value
keys[entry.Key] = true
}
for _, entry := range after.Mapping {
afterEntries[entry.Key] = entry.Value
keys[entry.Key] = true
}
ordered := make([]string, 0, len(keys))
for key := range keys {
ordered = append(ordered, key)
}
slices.Sort(ordered)
for _, key := range ordered {
compareDocumentNode(beforeEntries[key], afterEntries[key], path.Key(key), deltas)
}
case document.Sequence:
if !documentNodesEqual(before, after) {
*deltas = append(*deltas, documentDelta{path: path.clone(), kind: documentReplaced})
}
default:
if before.Text != after.Text {
*deltas = append(*deltas, documentDelta{path: path.clone(), kind: documentReplaced})
}
}
}
func documentNodesEqual(left, right *document.Node) bool {
if left == nil || right == nil {
return left == right
}
if left.Kind != right.Kind || left.Text != right.Text || len(left.Mapping) != len(right.Mapping) || len(left.Sequence) != len(right.Sequence) {
return false
}
if left.Kind == document.Mapping {
leftMap := make(map[string]*document.Node, len(left.Mapping))
for _, entry := range left.Mapping {
leftMap[entry.Key] = entry.Value
}
for _, entry := range right.Mapping {
if !documentNodesEqual(leftMap[entry.Key], entry.Value) {
return false
}
}
}
for i := range left.Sequence {
if !documentNodesEqual(left.Sequence[i], right.Sequence[i]) {
return false
}
}
return true
}

394
projection_test.go Normal file
View File

@@ -0,0 +1,394 @@
package configkit
import (
"errors"
"math"
"reflect"
"strings"
"testing"
"time"
"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
)
type mutableText struct {
data []byte
}
func (value mutableText) MarshalText() ([]byte, error) {
return value.data, nil
}
func (value *mutableText) UnmarshalText(text []byte) error {
value.data = append(value.data[:0], text...)
return nil
}
type failingText struct {
failUnmarshal bool
}
func (value failingText) MarshalText() ([]byte, error) {
return []byte("CANARY_CODEC_TEXT"), errors.New("CANARY_MARSHAL_ERROR")
}
func (value *failingText) UnmarshalText([]byte) error {
if value.failUnmarshal {
return errors.New("CANARY_UNMARSHAL_ERROR")
}
return nil
}
type reconstructFailText string
func (value reconstructFailText) MarshalText() ([]byte, error) {
return []byte(value), nil
}
func (value *reconstructFailText) UnmarshalText([]byte) error {
return errors.New("CANARY_RECONSTRUCT_ERROR")
}
func TestProjectionPreservesCanonicalTypesAndNilEmptyDistinctions(t *testing.T) {
t.Parallel()
type nested struct {
Value string `yaml:"value"`
}
type config struct {
Zero int `yaml:"zero"`
Wide uint64 `yaml:"wide"`
Delay time.Duration `yaml:"delay"`
Pointer *nested `yaml:"pointer"`
NilPointer *nested `yaml:"nil_pointer"`
EmptyMap map[string]string `yaml:"empty_map"`
NilMap map[string]string `yaml:"nil_map"`
EmptySlice []string `yaml:"empty_slice"`
NilSlice []string `yaml:"nil_slice"`
Objects map[string]*nested `yaml:"objects"`
Fixed [2]int `yaml:"fixed"`
Custom mutableText `yaml:"custom"`
Dynamic any `yaml:"dynamic"`
Ignored *nested `yaml:"-"`
private *nested
}
value := config{
Wide: math.MaxUint64,
Delay: 1500 * time.Millisecond,
Pointer: &nested{Value: "pointer"},
EmptyMap: map[string]string{},
EmptySlice: []string{},
Objects: map[string]*nested{
"z": {Value: "last"},
"a": {Value: "first"},
},
Fixed: [2]int{1, 2},
Custom: mutableText{data: []byte("canonical")},
Dynamic: map[string]any{"count": int64(9), "items": []any{"x", false}},
Ignored: &nested{Value: "ignored"},
private: &nested{Value: "private"},
}
schema := mustSchemaFor[config](t)
projected, issue := projectConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits())
if issue != nil {
t.Fatalf("projectConfiguration() issue = %#v", issue)
}
assertDocument(t, mappingValue(t, projected, "zero"), document.Number, "0")
assertDocument(t, mappingValue(t, projected, "wide"), document.Number, "18446744073709551615")
assertDocument(t, mappingValue(t, projected, "delay"), document.String, "1.5s")
assertDocument(t, mappingValue(t, projected, "custom"), document.String, "canonical")
if mappingValue(t, projected, "nil_pointer").Kind != document.Null || mappingValue(t, projected, "nil_map").Kind != document.Null || mappingValue(t, projected, "nil_slice").Kind != document.Null {
t.Fatal("nil values did not project as null")
}
if empty := mappingValue(t, projected, "empty_map"); empty.Kind != document.Mapping || empty.Mapping == nil || len(empty.Mapping) != 0 {
t.Fatalf("empty map = %#v", empty)
}
if empty := mappingValue(t, projected, "empty_slice"); empty.Kind != document.Sequence || empty.Sequence == nil || len(empty.Sequence) != 0 {
t.Fatalf("empty slice = %#v", empty)
}
objects := mappingValue(t, projected, "objects")
if len(objects.Mapping) != 2 || objects.Mapping[0].Key != "a" || objects.Mapping[1].Key != "z" {
t.Fatalf("map order = %#v", objects.Mapping)
}
if hasMappingKey(projected, "ignored") || hasMappingKey(projected, "private") {
t.Fatal("ignored field entered projection")
}
clonedValue, cloneIssue := cloneConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits())
if cloneIssue != nil {
t.Fatal(cloneIssue.message)
}
cloned := clonedValue.Interface().(config)
if cloned.NilMap != nil || cloned.NilSlice != nil || cloned.NilPointer != nil || cloned.EmptyMap == nil || cloned.EmptySlice == nil {
t.Fatalf("clone lost nil/empty distinctions: %#v", cloned)
}
}
func TestCloneConfigurationDetachesAllSupportedMutableValues(t *testing.T) {
t.Parallel()
type nested struct {
Values []string `yaml:"values"`
}
type config struct {
First *nested `yaml:"first"`
Second *nested `yaml:"second"`
Map map[string]*nested `yaml:"map"`
Slice []map[string]int `yaml:"slice"`
Custom mutableText `yaml:"custom"`
Dynamic any `yaml:"dynamic"`
Ignored *nested `yaml:"-"`
}
shared := &nested{Values: []string{"original"}}
value := config{
First: shared,
Second: shared,
Map: map[string]*nested{"entry": {Values: []string{"map"}}},
Slice: []map[string]int{{"count": 1}},
Custom: mutableText{data: []byte("text")},
Dynamic: map[string]any{
"items": []any{map[string]any{"name": "dynamic"}},
},
Ignored: &nested{Values: []string{"ignored"}},
}
schema := mustSchemaFor[config](t)
clonedValue, issue := cloneConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits())
if issue != nil {
t.Fatalf("cloneConfiguration() issue = %#v", issue)
}
cloned := clonedValue.Interface().(config)
if cloned.First == cloned.Second || cloned.First == shared || cloned.Custom.data == nil || &cloned.Custom.data[0] == &value.Custom.data[0] {
t.Fatal("clone retained mutable identity")
}
if cloned.Ignored != nil {
t.Fatal("ignored field was retained")
}
value.First.Values[0] = "changed"
value.Map["entry"].Values[0] = "changed"
value.Slice[0]["count"] = 2
value.Custom.data[0] = 'X'
value.Dynamic.(map[string]any)["items"].([]any)[0].(map[string]any)["name"] = "changed"
if cloned.First.Values[0] != "original" || cloned.Map["entry"].Values[0] != "map" || cloned.Slice[0]["count"] != 1 || string(cloned.Custom.data) != "text" || cloned.Dynamic.(map[string]any)["items"].([]any)[0].(map[string]any)["name"] != "dynamic" {
t.Fatalf("clone changed after source mutation: %#v", cloned)
}
}
func TestProjectionAndCloneRejectCyclesAndInvalidDynamicValues(t *testing.T) {
t.Parallel()
type recursive struct {
Next *recursive `yaml:"next"`
}
type config struct {
Tree *recursive `yaml:"tree"`
Dynamic any `yaml:"dynamic"`
}
cyclic := &recursive{}
cyclic.Next = cyclic
schema := mustSchemaFor[config](t)
acyclic := config{Tree: &recursive{Next: &recursive{}}}
if _, issue := projectConfiguration(schema, reflect.ValueOf(acyclic), defaultProjectionLimits()); issue != nil {
t.Fatalf("acyclic recursive projection issue = %#v", issue)
}
if _, issue := cloneConfiguration(schema, reflect.ValueOf(acyclic), defaultProjectionLimits()); issue != nil {
t.Fatalf("acyclic recursive clone issue = %#v", issue)
}
value := config{Tree: cyclic}
if _, issue := projectConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits()); issue == nil || issue.code != CodeSchemaCycle || issue.path.String() != "/tree/next" {
t.Fatalf("projection cycle issue = %#v", issue)
}
if _, issue := cloneConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits()); issue == nil || issue.code != CodeSchemaCycle {
t.Fatalf("clone cycle issue = %#v", issue)
}
dynamicCycle := map[string]any{}
dynamicCycle["self"] = dynamicCycle
value = config{Dynamic: dynamicCycle}
if _, issue := projectConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits()); issue == nil || issue.code != CodeSchemaCycle {
t.Fatalf("dynamic cycle issue = %#v", issue)
}
value = config{Dynamic: make(chan int)}
if _, issue := projectConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits()); issue == nil || issue.code != CodeSchemaDynamicType {
t.Fatalf("dynamic type issue = %#v", issue)
}
if _, issue := cloneConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits()); issue == nil || issue.code != CodeSchemaDynamicType {
t.Fatalf("dynamic clone issue = %#v", issue)
}
}
func TestProjectionRejectsNonFiniteLimitsAndCodecFailuresSafely(t *testing.T) {
t.Parallel()
type numbers struct {
Value float64 `yaml:"value"`
}
numberSchema := mustSchemaFor[numbers](t)
for _, value := range []float64{math.NaN(), math.Inf(1), math.Inf(-1)} {
if _, issue := projectConfiguration(numberSchema, reflect.ValueOf(numbers{Value: value}), defaultProjectionLimits()); issue == nil || issue.code != CodeSchemaNonFinite {
t.Fatalf("non-finite issue = %#v", issue)
}
}
type codecConfig struct {
Value failingText `yaml:"value"`
}
codecSchema := mustSchemaFor[codecConfig](t)
if _, issue := projectConfiguration(codecSchema, reflect.ValueOf(codecConfig{}), defaultProjectionLimits()); issue == nil || issue.code != CodeSchemaCodec || strings.Contains(issue.message, "CANARY") {
t.Fatalf("codec projection issue = %#v", issue)
}
type reconstructConfig struct {
Value reconstructFailText `yaml:"value"`
}
reconstructSchema := mustSchemaFor[reconstructConfig](t)
if _, issue := cloneConfiguration(reconstructSchema, reflect.ValueOf(reconstructConfig{Value: "safe-name"}), defaultProjectionLimits()); issue == nil || issue.code != CodeSchemaCodec || strings.Contains(issue.message, "CANARY") {
t.Fatalf("codec clone issue = %#v", issue)
}
type simple struct {
Value string `yaml:"value"`
}
simpleSchema := mustSchemaFor[simple](t)
if _, issue := projectConfiguration(simpleSchema, reflect.ValueOf(simple{}), projectionLimits{depth: 1, nodes: 3}); issue != nil {
t.Fatalf("exact node limit rejected: %#v", issue)
}
if _, issue := projectConfiguration(simpleSchema, reflect.ValueOf(simple{}), projectionLimits{depth: 1, nodes: 2}); issue == nil || issue.code != CodeSchemaProjectionLimit {
t.Fatalf("node overflow issue = %#v", issue)
}
if _, issue := projectConfiguration(simpleSchema, reflect.ValueOf(simple{}), projectionLimits{depth: 0, nodes: 3}); issue == nil || issue.code != CodeSchemaProjectionLimit {
t.Fatalf("depth overflow issue = %#v", issue)
}
}
func TestInlineProjectionAndCloneUsePhysicalMembers(t *testing.T) {
t.Parallel()
type Embedded struct {
Name string `yaml:"name"`
Items map[string]string `yaml:"items"`
}
type config struct {
*Embedded `yaml:",inline"`
Extras map[string]any `yaml:",inline"`
}
value := config{
Embedded: &Embedded{Name: "value", Items: map[string]string{"a": "b"}},
Extras: map[string]any{"extra": []string{"one"}},
}
schema := mustSchemaFor[config](t)
projected, issue := projectConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits())
if issue != nil || !hasMappingKey(projected, "name") || !hasMappingKey(projected, "items") || !hasMappingKey(projected, "extra") {
t.Fatalf("inline projection = %#v, issue=%#v", projected, issue)
}
clonedValue, issue := cloneConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits())
if issue != nil {
t.Fatal(issue.message)
}
cloned := clonedValue.Interface().(config)
value.Items["a"] = "changed"
value.Extras["extra"].([]string)[0] = "changed"
if cloned.Items["a"] != "b" || cloned.Extras["extra"].([]string)[0] != "one" {
t.Fatal("inline clone retained aliases")
}
value.Extras["name"] = "collision"
if _, issue := projectConfiguration(schema, reflect.ValueOf(value), defaultProjectionLimits()); issue == nil || issue.code != CodeSchemaAmbiguousField || issue.path.String() != "/name" {
t.Fatalf("inline collision issue = %#v", issue)
}
}
func TestCompareDocumentTreesTracksSemanticChanges(t *testing.T) {
t.Parallel()
type nested struct {
Value string `yaml:"value"`
}
type config struct {
Nested nested `yaml:"nested"`
Mapping map[string]int `yaml:"mapping"`
Sequence []string `yaml:"sequence"`
Maybe map[string]int `yaml:"maybe"`
}
schema := mustSchemaFor[config](t)
before, issue := projectConfiguration(schema, reflect.ValueOf(config{
Nested: nested{Value: "before"},
Mapping: map[string]int{"remove": 1, "same": 2},
Sequence: []string{"a", "b"},
}), defaultProjectionLimits())
if issue != nil {
t.Fatal(issue.message)
}
after, issue := projectConfiguration(schema, reflect.ValueOf(config{
Nested: nested{Value: "after"},
Mapping: map[string]int{"add": 3, "same": 2},
Sequence: []string{"a", "changed"},
Maybe: map[string]int{},
}), defaultProjectionLimits())
if issue != nil {
t.Fatal(issue.message)
}
deltas := compareDocumentTrees(before, after)
want := []struct {
path string
kind documentDeltaKind
}{
{path: "/mapping/add", kind: documentAdded},
{path: "/mapping/remove", kind: documentRemoved},
{path: "/maybe", kind: documentReplaced},
{path: "/nested/value", kind: documentReplaced},
{path: "/sequence", kind: documentReplaced},
}
if len(deltas) != len(want) {
t.Fatalf("deltas = %#v", deltas)
}
for i := range want {
if deltas[i].path.String() != want[i].path || deltas[i].kind != want[i].kind {
t.Fatalf("delta[%d] = %#v, want %#v", i, deltas[i], want[i])
}
}
for _, delta := range deltas {
if strings.HasPrefix(delta.path.String(), "/sequence/") {
t.Fatal("sequence comparison created index correspondence")
}
}
if equal := compareDocumentTrees(after, document.Clone(after)); len(equal) != 0 {
t.Fatalf("equal trees changed = %#v", equal)
}
}
func mustSchemaFor[T any](t *testing.T) *schemaDescriptor {
t.Helper()
schema, issue := buildSchema(reflect.TypeFor[T]())
if issue != nil {
t.Fatalf("buildSchema() issue = %#v", issue)
}
return schema
}
func mappingValue(t *testing.T, node *document.Node, key string) *document.Node {
t.Helper()
for _, entry := range node.Mapping {
if entry.Key == key {
return entry.Value
}
}
t.Fatalf("mapping key %q missing", key)
return nil
}
func hasMappingKey(node *document.Node, key string) bool {
for _, entry := range node.Mapping {
if entry.Key == key {
return true
}
}
return false
}
func assertDocument(t *testing.T, node *document.Node, kind document.Kind, text string) {
t.Helper()
if node.Kind != kind || node.Text != text {
t.Fatalf("node = %#v, want kind=%v text=%q", node, kind, text)
}
}

View File

@@ -59,6 +59,11 @@ const (
CodeSchemaTag Code = "schema.tag"
CodeSchemaTextCodec Code = "schema.text_codec"
CodeSchemaInline Code = "schema.inline"
CodeSchemaCycle Code = "schema.cycle"
CodeSchemaProjectionLimit Code = "schema.projection_limit"
CodeSchemaCodec Code = "schema.codec"
CodeSchemaNonFinite Code = "schema.non_finite"
CodeSchemaDynamicType Code = "schema.dynamic_type"
)
// Location identifies a position in a configuration source. Zero line and column

View File

@@ -28,7 +28,8 @@ const (
)
type schemaDescriptor struct {
root *typeDescriptor
root *typeDescriptor
types map[reflect.Type]*typeDescriptor
}
type typeDescriptor struct {
@@ -36,11 +37,19 @@ type typeDescriptor struct {
kind descriptorKind
element *typeDescriptor
length int
members []structMember
fields []fieldDescriptor
fieldByName map[string]int
inlineMap *fieldDescriptor
}
type structMember struct {
index int
name string
descriptor *typeDescriptor
inline bool
}
type fieldDescriptor struct {
name string
index []int
@@ -76,7 +85,7 @@ func buildSchema(root reflect.Type) (*schemaDescriptor, *schemaIssue) {
if descriptor.kind != descriptorStruct && descriptor.kind != descriptorMap {
return nil, &schemaIssue{code: CodeSchemaRoot, message: "configuration root must be a structural struct or string-keyed map"}
}
return &schemaDescriptor{root: descriptor}, nil
return &schemaDescriptor{root: descriptor, types: builder.cache}, nil
}
type schemaBuilder struct {
@@ -228,6 +237,11 @@ func (b *schemaBuilder) describeFields(descriptor *typeDescriptor, current refle
}
descriptor.inlineMap = &nested
}
memberType, memberIssue := b.describe(field.Type, path)
if memberIssue != nil {
return memberIssue
}
descriptor.members = append(descriptor.members, structMember{index: i, descriptor: memberType, inline: true})
case reflect.Map:
if inlineType.Key().Kind() != reflect.String {
return &schemaIssue{code: CodeSchemaInline, path: path.clone(), message: "inline map must have string keys"}
@@ -241,6 +255,7 @@ func (b *schemaBuilder) describeFields(descriptor *typeDescriptor, current refle
return childIssue
}
descriptor.inlineMap = &fieldDescriptor{index: []int{i}, descriptor: fieldType, secret: secret, inlineMap: true}
descriptor.members = append(descriptor.members, structMember{index: i, descriptor: fieldType, inline: true})
default:
return &schemaIssue{code: CodeSchemaInline, path: path.clone(), message: "inline field must be a struct, pointer to struct, or string-keyed map"}
}
@@ -255,6 +270,7 @@ func (b *schemaBuilder) describeFields(descriptor *typeDescriptor, current refle
if issue := addField(descriptor, fieldDescriptor{name: name, index: []int{i}, descriptor: fieldType, secret: secret}, path); issue != nil {
return issue
}
descriptor.members = append(descriptor.members, structMember{index: i, name: name, descriptor: fieldType})
}
return nil
}