ConfigKit/docs/config.md

Configuration Input Contract

This document owns ConfigKit's currently implemented source-input, YAML syntax, schema, typed-decoding, loading, metadata, and ownership contracts. The parser, owned document representation, and decoder remain private behind the public loading lifecycle.

Sources

ConfigKit exposes opaque sources through these constructors:

Constructor	Current contract
File(path)	Requires a nonempty path, opens a regular file for each consumption, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail.
Bytes(label, data)	Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source.
Reader(label, input)	Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused.
NoSource()	Selects defaults-only loading, creates no source record, and marks no paths explicit.

The zero Source is invalid. Constructor argument failures are retained inside the opaque source and become safe structured diagnostics when the source is consumed. File sources can observe file changes between consumptions; byte sources cannot.

Each source is limited to 8 MiB. A load using imports or an active profile is limited to 64 MiB across all accepted sources, 1,024 accepted sources, 64 active import edges, and 1,000,000 input nodes in aggregate. Glob expansion separately permits 1,000,000 visited directory entries, including nonmatches. Reads request at most the remaining allowance plus one byte so excess data is rejected rather than silently truncated. Data returned together with a read error is not accepted as partial input.

Single-source loading

Load[T](source, options...) accepts zero or one Options[T]. The configuration root T must satisfy the application schema contract below. With no options it starts from the Go zero value. A Definition[T].Defaults factory, when supplied, runs exactly once after successful source parsing, schema construction, optional schema-version checking, and selected-profile loading. Its result is detached before input is applied.

The current lifecycle is:

validate options → parse the selected source → traverse/compose enabled imports
  → build schema → check the optional application version → apply a selected profile
  → create and clone defaults → strictly decode → recorded application resolution
  → semantic validation → snapshot configuration and metadata

File, Bytes, and Reader each contribute the first base source with deterministic ID source-1; enabled imports continue in depth-first order as source-2 onward. NoSource skips parsing, supplies an absent version to a configured version checker, contributes no source, and continues through defaults, decode, and projection. A missing file remains an error; it is never interpreted as no input.

The current Options boundary accepts a defaults factory, VersionRule, resolver, semantic validator, explicit path and glob imports, one selected profile, and filesystem context. Overrides and dynamic secret classification return options.unavailable because their lifecycle stages are not implemented. NoSource rejects imports, active profiles, and filesystem context as invalid combinations. Input containing a reserved _configkit mapping key is rejected while imports are disabled.

Every source, options, schema, version, defaults, decode, resolution, or validation failure returns a nil result and a *LoadError. Successful results contain caller-owned Config, detached Metadata, a defensive source list, and defensive nonfatal diagnostics. Mutating the returned configuration or inspection records cannot alter ConfigKit's private snapshot. ConfigKit performs no logging and exposes no effective writer yet.

Application schema versions

VersionRule declares a semantic path and non-nil checker. Before defaults or typed decoding, the checker receives a Version preserving absent, explicit null, or string input. Any other input kind is a version.type error and the checker does not run. ConfigKit does not interpret the text or impose a universal version path.

The checker reports safe, value-free errors and warnings through Validation.Error and Validation.Warning; ConfigKit resolves primary and related paths to exact or nearest input locations. Reported errors fail loading. If the checker returns an error without reporting one, ConfigKit emits a generic version.hook_failed diagnostic and suppresses the returned error text. Warnings alone are retained on a successful result. Callback panics remain programming failures.

Layered input/default metadata

Result.Metadata describes every final canonical path. Exact surviving input paths are explicit and have a base or profile origin; omitted paths have a default origin. An explicit parent mapping does not make omitted children explicit. Null state and available source coordinates are retained. History records the Go-zero or application default baseline followed by exact base or profile definitions, including equal-valued input. Profile replacement removes displaced descendants from effective metadata while retaining their base definition and the parent replacement operation in history. All metadata queries and returned nested records are detached.

Recorded application resolution

An optional Definition.Resolve callback runs after successful typed decoding. It receives the working configuration and a per-load Resolution context. The context's IsExplicit query uses surviving input presence, so explicit false, zero, empty, and null are never confused with absence or static defaults.

Default assigns and records a derived value only when its path is not explicit. Normalize always assigns and records. For transformations performed directly by application code, Resolution.Record declares a derived-default or normalization operation; a recorded container covers changes to its descendants. Records require a schema-valid path, valid kind, safe nonempty reason, and schema-valid dependency paths. Repeated records at one path are coalesced: reasons and dependencies retain their first occurrence, the last helper value is authoritative, and any normalization makes the combined operation a normalization.

ConfigKit projects the configuration before and after the callback. Every net change must have an exact or ancestor record. Helper-assigned values must agree with the final value at their declared path, which supports ordinary map-entry copy/writeback while detecting forgotten writeback or unrelated targets. Invalid paths, kinds, values, records, helper targets, and unrecorded mutations fail the load. Equal-valued derived assignments may still record application intent even when there is no structural change.

Resolution.Error and Resolution.Warning attach exact or nearest available input locations. Reported errors fail the load; warnings survive on a successful result. A returned resolver error adds one generic resolve.hook_failed error only when the callback did not report an error, and its text is never exposed. Callback panics are programming failures. Resolution and validation contexts are callback-scoped; using a nil, zero, or retained context outside its active callback panics with fixed, value-free text.

Successful records update effective origins and append detached metadata history without changing input explicitness. Removed paths leave effective metadata but retain history. ConfigKit privately snapshots each recorded before/after subtree for later safe explanation; caller mutations of Result.Config cannot alter those snapshots.

Semantic validation

An optional Definition.Validate callback runs after successful resolution and receives the resolved configuration plus a per-load Validation context. It can collect multiple independent validate.error and validate.warning reports. Warnings continue the lifecycle and are returned on success; any error returns a nil result with the full safe diagnostic and source report.

Primary and related paths must be valid for the application schema. Omitted declared fields and missing string-map or dynamic keys are valid report targets; unknown struct fields and malformed paths produce validate.invalid_path. ConfigKit attaches the exact surviving input location when one exists, otherwise the nearest input container location, or no primary location for defaults-only synthetic state.

Validation is read-only. ConfigKit projects the configuration immediately before and after the callback and returns validate.mutation at every detected net-change path, including nested mutable state. A value that can no longer be projected is also a mutation failure. No mutated or otherwise invalid configuration is published. Callback panics remain programming failures, and retained contexts use the same fixed closed-context panic as version and resolution callbacks.

Only schema-bound exported state participates in loading, resolution audit, validation audit, metadata, and snapshots. Ignored or unexported ordinary fields are zero in returned detached values. Applications retain responsibility for domain rules and runtime initialization; validators should not start clients or depend on live services as a substitute for configuration validation.

Strict document composition

ConfigKit strictly composes parsed base documents when Options.Imports is true. Application defaults do not participate in this operation.

Existing node	Incoming node	Result
Absent	Absent	Absent
Absent	Present	Retain the incoming node
Present	Absent	Retain the existing node
Mapping	Mapping	Recurse by mapping key
Any other pair of present nodes	Any other pair of present nodes	compose.conflict

Every non-mapping overlap is a conflict, even when values are equal. Nulls and sequences are present atomic values, so they also conflict when repeated; sequences are never joined by index. Disjoint entries under shared mappings compose, making stable keyed mappings the natural representation for independently contributed entries.

An empty mapping contributes no child definitions. Two empty mappings compose, and an empty mapping neither clears nor conflicts with another mapping's children. Shared mapping containers retain all contributing value locations as their composite base origin. A conflict reports the first claim as its primary location and every remaining claim as related locations, in deterministic source and declaration traversal order. Traversal order never authorizes replacement. Composition clones the resulting tree and its ordered claim records rather than mutating or aliasing an input document.

Selected profiles

Options.Profile selects at most one overlay by name. Names must match [a-zA-Z0-9_-]+; they are not paths. The empty name and exact lowercase name default select no profile, and profiles/default.yml is never loaded implicitly. Every other name resolves only to <configuration-root>/profiles/<name>.yml, with no extension fallback, case normalization, inheritance, or profile stack. A selected missing profile is an error.

The profile is a separately policy-validated YAML mapping. It is loaded after the base import graph is composed and its schema version is checked, but before defaults and typed decoding. Profile bytes, nodes, and source records count toward the same aggregate limits as base inputs. Its canonical path must remain within the configured root and must not duplicate a file already present in the base graph.

Overlay is intentionally distinct from strict composition:

Existing lower node	Incoming profile node	Result
Absent	Value	Retain the profile value
Value	Absent	Retain the lower value
Mapping	Mapping	Recurse by mapping key
Any other pair of present nodes	Any other pair of present nodes	Replace the entire lower node

Scalars, sequences, and null are atomic. Thus a profile sequence replaces rather than extends a base sequence, [] clears it, null is an explicit replacement, and cross-kind replacement is allowed when the final application schema accepts it. Replacing a subtree removes all displaced descendants from effective presence and origin data; history retains the displaced definitions and identifies the parent replacement. Surviving profile paths have profile origins, while recursively shared mapping containers retain their contributing locations.

A profile may not contain _configkit anywhere, contribute the configured schema version, or replace an ancestor of that version with a scalar or null. These rules apply even if the supplied version text would be unchanged. Profiles cannot mask a base parse, composition, or version failure because those checks run first.

Explicit imports

Imports are opt-in. With Options.Imports enabled, a document may reserve its root _configkit mapping for an imports sequence. ConfigKit removes that mapping before application decoding. Each sequence item is a mapping with exactly one nonempty string path or glob plus an optional boolean optional; all other shapes and fields are import.invalid. A _configkit key below the root is invalid. With imports disabled, any occurrence remains import.disabled.

Path declarations use slash-separated relative paths. Absolute paths and network URLs are invalid. Each path resolves from its declaring file's canonical directory, and files are traversed depth-first in declaration order: a file contributes its own body before its imports. Order stabilizes source records and diagnostics but never permits replacement. Required missing paths fail; optional: true suppresses only target nonexistence, not invalid paths, containment failures, directories, permissions, parse errors, or composition errors.

Glob declarations use the slash-based Go path.Match grammar: *, ?, character classes, and backslash escapes are supported, while recursive ** syntax is invalid. Backslashes escape pattern characters rather than acting as directory separators. ConfigKit validates the complete pattern before filesystem traversal, walks only directories selected by preceding segments, and reads wildcard directories in bounded batches. Every examined entry consumes the separate visit budget, including nonmatches. Matches are sorted lexicographically by slash-relative spelling within each declaration before depth-first source traversal. A required zero-match glob is import.missing; optional: true suppresses only zero matches. Invalid patterns, walk failures, matched directories, invalid documents, escapes, and repeated targets remain errors. Final candidates are bounded by the remaining source limit before their contents are accumulated.

The configuration root defaults to a selected file's canonical directory. A file source may supply FilesystemContext.Root to widen that root but may not supply BaseDir. Whenever byte and reader sources supply filesystem context, they must provide both BaseDir and Root as absolute existing directories; their diagnostic labels, including URI-looking labels, never imply filesystem paths. The base and selected file must be inside the canonical root. ConfigKit canonicalizes symlinks, checks containment with relative paths, and opens the canonical root-relative target through a library-owned root handle. Contained relative or absolute symlink targets are accepted; escapes are rejected. Canonical paths define identity, so hard links at distinct paths remain distinct sources.

An active-stack repeat is import.cycle and reports the complete cycle's import sites. A previously visited canonical path is import.repeated and reports both inclusion routes, including diamonds and aliases. Each accepted file is read once. Imported fragments may contain their own root directives, but may not claim the application's schema-version path or replace one of its ancestors; the version must come from the selected source. No directory is scanned implicitly, including one named conf.d, and ConfigKit performs no remote fetching.

YAML documents

The private YAML boundary currently accepts exactly one nonempty document whose root is a mapping. Comments and block or flow collections are supported. It rejects:

• empty input or an empty document;
• every additional document, including an empty one;
• duplicate decoded mapping keys;
• non-string mapping keys;
• explicit tags, anchors, aliases, and merge keys;
• a quoted or plain mapping key equal to <<; and
• input exceeding the fixed depth or node limits.

Mapping key comparison is exact: it performs no case folding or Unicode normalization. The private document tree retains declaration order, exact validated number spelling, source identifiers, and separate key/value line and column locations. It contains no upstream parser nodes or types.

Root depth is zero. Input depth is limited to 256 and each parsed input tree to 1,000,000 nodes, counting mapping keys as input nodes. The upstream parser also has a private calibrated depth guard; passing that guard never bypasses ConfigKit's tree limits.

Scalars

Scalar meaning is determined by ConfigKit rather than upstream coercion:

Input form	Meaning
Quoted, literal, or folded scalar	String
Plain true or false	Boolean
Plain null, ~, or an empty mapping value	Null
JSON decimal number	Number, retaining its exact spelling
Date/time-looking or other ordinary plain text	String

JSON numbers permit an optional minus, a zero or nonzero decimal integer without a leading zero, an optional fraction with at least one digit, and an optional decimal exponent. Hexadecimal, octal, digit separators, leading plus, leading/trailing-dot forms, incomplete exponents, and non-finite values are rejected. Legacy boolean words (yes, no, on, and off) and non-lowercase variants of the reserved boolean/null words are rejected unless quoted.

Mapping keys use the same scalar classification and must classify as strings. Quote keys that otherwise classify as a boolean, number, or null.

Application schema binding

ConfigKit builds a private reflection descriptor graph for each schema operation; it keeps no global mutable descriptor cache. A configuration root must be a struct or string-keyed map. Nested supported shapes are:

• booleans, strings, signed and unsigned integers, and finite floating-point types;
• time.Duration and paired encoding.TextMarshaler/TextUnmarshaler scalars;
• pointers, structs, string-keyed maps, slices, fixed arrays, and any; and
• recursive type definitions whose eventual application values are acyclic.

Named scalar types retain their Go identity. uintptr, complex numbers, functions, channels, unsafe pointers, non-string map keys, and nonempty interfaces are not configuration types. A custom text scalar must provide both directions on its value or pointer method set. Building a descriptor only inspects method sets; it never executes application methods or parser-specific hooks.

Exported struct fields use an explicit yaml name or the lowercase Go field name. Unexported fields and yaml:"-" are ignored. omitempty is accepted but does not change binding or future projection. An explicit yaml:",inline" flattens a struct or pointer-to-struct, or designates at most one string-keyed inline map for otherwise unmatched names. Ambiguous names, named inline fields, multiple inline maps, and unsupported or malformed tag options are schema errors. Embedded fields are ordinary named fields unless explicitly inlined. configkit:"secret" is the only current ConfigKit field tag and records sensitivity for later lifecycle work.

Schema path traversal uses these same bindings. Explicit key segments address mappings, explicit index segments address sequences, and parsed numeric pointer tokens become indices only in a sequence. Sequence indices cannot have leading zeroes; fixed-array indices are bounds checked. Dynamic any and inline-map entries remain schema-valid for application-defined descendants.

Descriptors are reused by typed decoding, projection, and cloning so those operations cannot acquire different field bindings.

Defaults and typed decoding

The private decoder starts from an independently owned clone of the application's typed defaults and applies only explicitly present input nodes. Defaults therefore never become explicit, while equal-valued input remains explicit. Successful decode produces detached typed state, baseline and final canonical trees, an input-presence index, and default/input origins for the later public loading lifecycle.

Configured input	Effect on the typed baseline
Absent field	Retain the default
Explicit false, zero, or empty string	Replace the default
Partial mapping or string-keyed map	Recurse and retain omitted default children or entries
Empty mapping	Retain existing mapping children
Sequence, including an empty sequence	Replace the entire sequence
Pointer object	Recurse into a non-nil default; otherwise allocate an object whose omitted children have ordinary Go zero values
Allowed null	Replace the value with nil and suppress its default

Null is allowed only for pointers, maps, slices, and dynamic any values. It is an error for scalars, structs, and arrays. New map entries and sequence elements begin from their Go zero values; ConfigKit does not invent entry-level defaults.

Decoding is strict at every typed depth. Unknown struct fields, type mismatches, invalid nulls, numeric overflow, and arrays with the wrong length are errors at the original semantic path and source coordinates. String-keyed maps accept arbitrary keys but preserve the strict schema of their values; inline maps accept unmatched keys according to their element descriptor. Multiple independent field failures are collected without publishing the partially decoded object.

Booleans and strings are not coerced. Integer destinations require integer syntax and a value in the destination type's range. Float destinations accept finite JSON numbers representable by their destination precision. Dynamic numbers become int64 for integer syntax and float64 for fractional or exponent syntax, with range and finiteness checks. Named scalar types retain their identity. Durations decode only from strings accepted by time.ParseDuration; integers are rejected. Paired text scalars decode only from strings through encoding.TextUnmarshaler, and codec failures are classified without exposing application error text or input values.

This decoder is available only through Load; ConfigKit exposes no generic document conversion API.

Typed projection and ownership

ConfigKit can internally project a schema-valid application value directly into its owned canonical tree without YAML serialization. Projection includes every bound field, including ordinary zero values; omitempty has no effect. Struct fields keep binding order, dynamic map keys sort by decoded string, and sequences keep their order. Integers retain exact base-10 values, finite floats use their destination precision, durations use time.Duration.String, and paired text scalars use their canonical marshaled text.

Nil pointers, maps, slices, and dynamic values project as null. Non-nil empty maps and slices remain empty containers, preserving the distinction. Recursive acyclic objects are supported. Cyclic values, non-finite floats, unsupported dynamic concrete types, invalid codec output, and depth/node-limit violations fail with safe schema classifications that omit codec error text.

The same descriptor graph creates detached typed copies. Pointers, maps, slices, arrays, nested dynamic values, and shared references are recursively copied; shared pointer identity is not preserved as configuration meaning. Paired text scalars are rebuilt by unmarshaling their canonical text into fresh values, detaching private backing storage. Ignored and unexported struct fields are zeroed rather than copied.

Private structural comparison operates on canonical trees. Mapping additions, removals, and descendant replacements retain semantic paths. A sequence is atomic: any sequence change reports the sequence path and never invents correspondence between old and new indices. Cross-kind and null/container changes replace the affected path.

These mechanisms currently support later decoding, resolution auditing, metadata, and rendering. They are not a public generic conversion or cloning API.

Safe failures

Source failures use stable source.* codes and YAML failures use stable yaml.* codes declared by the root package. Diagnostics contain safe classifications, semantic paths when applicable, and available source coordinates. Raw parser, conversion, reader-error, and input text is not forwarded. Source labels and diagnostic messages are application-visible and therefore must never contain secrets; terminal control characters are escaped by WriteDiagnostics.