ConfigKit/docs/config.md

# Configuration Input Contract

This document owns ConfigKit's currently implemented source-input, YAML syntax,
schema, typed-decoding, loading, metadata, and ownership contracts. The parser,
owned document representation, and decoder remain private behind the public loading
lifecycle.

## Sources

ConfigKit exposes opaque sources through these constructors:

| Constructor | Current contract |
| --- | --- |
| `File(path)` | Requires a nonempty path, opens a regular file for each consumption, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. |
| `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. |
| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. |
| `NoSource()` | Selects defaults-only loading, creates no source record, and marks no paths explicit. |

The zero `Source` is invalid. Constructor argument failures are retained inside the
opaque source and become safe structured diagnostics when the source is consumed.
File sources can observe file changes between consumptions; byte sources cannot.

Each source is limited to 8 MiB. A load using imports or an active profile is limited
to 64 MiB across all accepted sources, 1,024 accepted sources, 64 active import
edges, and 1,000,000 input nodes in aggregate. Glob expansion separately permits
1,000,000 visited directory entries, including nonmatches. Reads request at most the
remaining allowance plus one byte so excess data is rejected rather than silently
truncated. Data returned together with a read error is not accepted as partial input.

## Single-source loading

`Load[T](source, options...)` accepts zero or one `Options[T]`. The configuration
root `T` must satisfy the application schema contract below. With no options it
starts from the Go zero value. A `Definition[T].Defaults` factory, when supplied,
runs exactly once after successful source parsing, schema construction, optional
schema-version checking, and selected-profile loading. Its result is detached before
input is applied.

The current lifecycle is:

```text
validate options → parse the selected source → traverse/compose enabled imports
  → build schema → check the optional application version → apply a selected profile
  → create and clone defaults → strictly decode → recorded application resolution
  → semantic validation → snapshot configuration and metadata
```

`File`, `Bytes`, and `Reader` each contribute the first base source with deterministic
ID `source-1`; enabled imports continue in depth-first order as `source-2` onward.
`NoSource` skips parsing, supplies an absent version to a configured version checker,
contributes no source, and continues through defaults, decode, and projection. A
missing file remains an error; it is never interpreted as no input.

The current `Options` boundary accepts a defaults factory, `VersionRule`, resolver,
semantic validator, explicit path and glob imports, one selected profile, and
filesystem context. Overrides and dynamic secret classification return
`options.unavailable` because their lifecycle stages are not implemented. `NoSource`
rejects imports, active profiles, and filesystem context as invalid combinations.
Input containing a reserved `_configkit` mapping key is rejected while imports are
disabled.

Every source, options, schema, version, defaults, decode, resolution, or validation
failure returns a nil result and a `*LoadError`. Successful results contain caller-owned `Config`, detached
`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Mutating
the returned configuration or inspection records cannot alter ConfigKit's private
snapshot. ConfigKit performs no logging and exposes no effective writer yet.

### Application schema versions

`VersionRule` declares a semantic path and non-nil checker. Before defaults or typed
decoding, the checker receives a `Version` preserving absent, explicit null, or
string input. Any other input kind is a `version.type` error and the checker does not
run. ConfigKit does not interpret the text or impose a universal version path.

The checker reports safe, value-free errors and warnings through `Validation.Error`
and `Validation.Warning`; ConfigKit resolves primary and related paths to exact or
nearest input locations. Reported errors fail loading. If the checker returns an
error without reporting one, ConfigKit emits a generic `version.hook_failed`
diagnostic and suppresses the returned error text. Warnings alone are retained on a
successful result. Callback panics remain programming failures.

### Layered input/default metadata

`Result.Metadata` describes every final canonical path. Exact surviving input paths
are explicit and have a base or profile origin; omitted paths have a default origin.
An explicit parent mapping does not make omitted children explicit. Null state and
available source coordinates are retained. `History` records the Go-zero or application
default baseline followed by exact base or profile definitions, including
equal-valued input. Profile replacement removes displaced descendants from effective
metadata while retaining their base definition and the parent replacement operation
in history.
All metadata queries and returned nested records are detached.

### Recorded application resolution

An optional `Definition.Resolve` callback runs after successful typed decoding. It
receives the working configuration and a per-load `Resolution` context. The context's
`IsExplicit` query uses surviving input presence, so explicit false, zero, empty, and
null are never confused with absence or static defaults.

`Default` assigns and records a derived value only when its path is not explicit.
`Normalize` always assigns and records. For transformations performed directly by
application code, `Resolution.Record` declares a derived-default or normalization
operation; a recorded container covers changes to its descendants. Records require
a schema-valid path, valid kind, safe nonempty reason, and schema-valid dependency
paths. Repeated records at one path are coalesced: reasons and dependencies retain
their first occurrence, the last helper value is authoritative, and any normalization
makes the combined operation a normalization.

ConfigKit projects the configuration before and after the callback. Every net change
must have an exact or ancestor record. Helper-assigned values must agree with the
final value at their declared path, which supports ordinary map-entry copy/writeback
while detecting forgotten writeback or unrelated targets. Invalid paths, kinds,
values, records, helper targets, and unrecorded mutations fail the load. Equal-valued
derived assignments may still record application intent even when there is no
structural change.

`Resolution.Error` and `Resolution.Warning` attach exact or nearest available input
locations. Reported errors fail the load; warnings survive on a successful result. A
returned resolver error adds one generic `resolve.hook_failed` error only when the
callback did not report an error, and its text is never exposed. Callback panics are
programming failures. Resolution and validation contexts are callback-scoped; using
a nil, zero, or retained context outside its active callback panics with fixed,
value-free text.

Successful records update effective origins and append detached metadata history
without changing input explicitness. Removed paths leave effective metadata but
retain history. ConfigKit privately snapshots each recorded before/after subtree for
later safe explanation; caller mutations of `Result.Config` cannot alter those
snapshots.

### Semantic validation

An optional `Definition.Validate` callback runs after successful resolution and
receives the resolved configuration plus a per-load `Validation` context. It can
collect multiple independent `validate.error` and `validate.warning` reports.
Warnings continue the lifecycle and are returned on success; any error returns a nil
result with the full safe diagnostic and source report.

Primary and related paths must be valid for the application schema. Omitted declared
fields and missing string-map or dynamic keys are valid report targets; unknown
struct fields and malformed paths produce `validate.invalid_path`. ConfigKit attaches
the exact surviving input location when one exists, otherwise the nearest input
container location, or no primary location for defaults-only synthetic state.

Validation is read-only. ConfigKit projects the configuration immediately before and
after the callback and returns `validate.mutation` at every detected net-change path,
including nested mutable state. A value that can no longer be projected is also a
mutation failure. No mutated or otherwise invalid configuration is published.
Callback panics remain programming failures, and retained contexts use the same
fixed closed-context panic as version and resolution callbacks.

Only schema-bound exported state participates in loading, resolution audit,
validation audit, metadata, and snapshots. Ignored or unexported ordinary fields are
zero in returned detached values. Applications retain responsibility for domain
rules and runtime initialization; validators should not start clients or depend on
live services as a substitute for configuration validation.

## Strict document composition

ConfigKit strictly composes parsed base documents when `Options.Imports` is true.
Application defaults do not participate in this operation.

| Existing node | Incoming node | Result |
| --- | --- | --- |
| Absent | Absent | Absent |
| Absent | Present | Retain the incoming node |
| Present | Absent | Retain the existing node |
| Mapping | Mapping | Recurse by mapping key |
| Any other pair of present nodes | Any other pair of present nodes | `compose.conflict` |

Every non-mapping overlap is a conflict, even when values are equal. Nulls and
sequences are present atomic values, so they also conflict when repeated; sequences
are never joined by index. Disjoint entries under shared mappings compose, making
stable keyed mappings the natural representation for independently contributed
entries.

An empty mapping contributes no child definitions. Two empty mappings compose, and
an empty mapping neither clears nor conflicts with another mapping's children.
Shared mapping containers retain all contributing value locations as their composite
base origin. A conflict reports the first claim as its primary location and every
remaining claim as related locations, in deterministic source and declaration
traversal order. Traversal order never authorizes replacement. Composition clones
the resulting tree and its ordered claim records rather than mutating or aliasing an
input document.

## Selected profiles

`Options.Profile` selects at most one overlay by name. Names must match
`[a-zA-Z0-9_-]+`; they are not paths. The empty name and exact lowercase name
`default` select no profile, and `profiles/default.yml` is never loaded implicitly.
Every other name resolves only to `<configuration-root>/profiles/<name>.yml`, with
no extension fallback, case normalization, inheritance, or profile stack. A selected
missing profile is an error.

The profile is a separately policy-validated YAML mapping. It is loaded after the
base import graph is composed and its schema version is checked, but before defaults
and typed decoding. Profile bytes, nodes, and source records count toward the same
aggregate limits as base inputs. Its canonical path must remain within the configured
root and must not duplicate a file already present in the base graph.

Overlay is intentionally distinct from strict composition:

| Existing lower node | Incoming profile node | Result |
| --- | --- | --- |
| Absent | Value | Retain the profile value |
| Value | Absent | Retain the lower value |
| Mapping | Mapping | Recurse by mapping key |
| Any other pair of present nodes | Any other pair of present nodes | Replace the entire lower node |

Scalars, sequences, and null are atomic. Thus a profile sequence replaces rather
than extends a base sequence, `[]` clears it, null is an explicit replacement, and
cross-kind replacement is allowed when the final application schema accepts it.
Replacing a subtree removes all displaced descendants from effective presence and
origin data; history retains the displaced definitions and identifies the parent
replacement. Surviving profile paths have profile origins, while recursively shared
mapping containers retain their contributing locations.

A profile may not contain `_configkit` anywhere, contribute the configured schema
version, or replace an ancestor of that version with a scalar or null. These rules
apply even if the supplied version text would be unchanged. Profiles cannot mask a
base parse, composition, or version failure because those checks run first.

## Explicit imports

Imports are opt-in. With `Options.Imports` enabled, a document may reserve its root
`_configkit` mapping for an `imports` sequence. ConfigKit removes that mapping before
application decoding. Each sequence item is a mapping with exactly one nonempty
string `path` or `glob` plus an optional boolean `optional`; all other shapes and
fields are `import.invalid`. A `_configkit` key below the root is invalid. With
imports disabled, any occurrence remains `import.disabled`.

Path declarations use slash-separated relative paths. Absolute paths and network
URLs are invalid. Each path resolves from its declaring file's canonical directory,
and files are traversed depth-first in declaration order: a file contributes its own
body before its imports. Order stabilizes source records and diagnostics but never
permits replacement. Required missing paths fail; `optional: true` suppresses only
target nonexistence, not invalid paths, containment failures, directories,
permissions, parse errors, or composition errors.

Glob declarations use the slash-based Go `path.Match` grammar: `*`, `?`, character
classes, and backslash escapes are supported, while recursive `**` syntax is invalid.
Backslashes escape pattern characters rather than acting as directory separators.
ConfigKit validates the complete pattern before filesystem traversal, walks only
directories selected by preceding segments, and reads wildcard directories in
bounded batches. Every examined entry consumes the separate visit budget, including
nonmatches. Matches are sorted lexicographically by slash-relative spelling within
each declaration before depth-first source traversal. A required zero-match glob is
`import.missing`; `optional: true` suppresses only zero matches. Invalid patterns,
walk failures, matched directories, invalid documents, escapes, and repeated targets
remain errors. Final candidates are bounded by the remaining source limit before
their contents are accumulated.

The configuration root defaults to a selected file's canonical directory. A file
source may supply `FilesystemContext.Root` to widen that root but may not supply
`BaseDir`. Whenever byte and reader sources supply filesystem context, they must
provide both `BaseDir` and `Root` as absolute existing directories; their diagnostic
labels, including URI-looking labels, never imply filesystem paths. The base and
selected file must be inside the canonical root. ConfigKit canonicalizes symlinks,
checks containment with relative paths, and opens the canonical root-relative target
through a library-owned root handle. Contained relative or absolute symlink targets
are accepted; escapes are rejected. Canonical paths define identity, so hard links at
distinct paths remain distinct sources.

An active-stack repeat is `import.cycle` and reports the complete cycle's import
sites. A previously visited canonical path is `import.repeated` and reports both
inclusion routes, including diamonds and aliases. Each accepted file is read once.
Imported fragments may contain their own root directives, but may not claim the
application's schema-version path or replace one of its ancestors; the version must
come from the selected source. No directory is scanned implicitly, including one
named `conf.d`, and ConfigKit performs no remote fetching.

## YAML documents

The private YAML boundary currently accepts exactly one nonempty document whose root
is a mapping. Comments and block or flow collections are supported. It rejects:

- empty input or an empty document;
- every additional document, including an empty one;
- duplicate decoded mapping keys;
- non-string mapping keys;
- explicit tags, anchors, aliases, and merge keys;
- a quoted or plain mapping key equal to `<<`; and
- input exceeding the fixed depth or node limits.

Mapping key comparison is exact: it performs no case folding or Unicode
normalization. The private document tree retains declaration order, exact validated
number spelling, source identifiers, and separate key/value line and column
locations. It contains no upstream parser nodes or types.

Root depth is zero. Input depth is limited to 256 and each parsed input tree to
1,000,000 nodes, counting mapping keys as input nodes. The upstream parser also has
a private calibrated depth guard; passing that guard never bypasses ConfigKit's tree
limits.

## Scalars

Scalar meaning is determined by ConfigKit rather than upstream coercion:

| Input form | Meaning |
| --- | --- |
| Quoted, literal, or folded scalar | String |
| Plain `true` or `false` | Boolean |
| Plain `null`, `~`, or an empty mapping value | Null |
| JSON decimal number | Number, retaining its exact spelling |
| Date/time-looking or other ordinary plain text | String |

JSON numbers permit an optional minus, a zero or nonzero decimal integer without a
leading zero, an optional fraction with at least one digit, and an optional decimal
exponent. Hexadecimal, octal, digit separators, leading plus, leading/trailing-dot
forms, incomplete exponents, and non-finite values are rejected. Legacy boolean
words (`yes`, `no`, `on`, and `off`) and non-lowercase variants of the reserved
boolean/null words are rejected unless quoted.

Mapping keys use the same scalar classification and must classify as strings. Quote
keys that otherwise classify as a boolean, number, or null.

## Application schema binding

ConfigKit builds a private reflection descriptor graph for each schema operation;
it keeps no global mutable descriptor cache. A configuration root must be a struct
or string-keyed map. Nested supported shapes are:

- booleans, strings, signed and unsigned integers, and finite floating-point types;
- `time.Duration` and paired `encoding.TextMarshaler`/`TextUnmarshaler` scalars;
- pointers, structs, string-keyed maps, slices, fixed arrays, and `any`; and
- recursive type definitions whose eventual application values are acyclic.

Named scalar types retain their Go identity. `uintptr`, complex numbers, functions,
channels, unsafe pointers, non-string map keys, and nonempty interfaces are not
configuration types. A custom text scalar must provide both directions on its value
or pointer method set. Building a descriptor only inspects method sets; it never
executes application methods or parser-specific hooks.

Exported struct fields use an explicit `yaml` name or the lowercase Go field name.
Unexported fields and `yaml:"-"` are ignored. `omitempty` is accepted but does not
change binding or future projection. An explicit `yaml:",inline"` flattens a struct
or pointer-to-struct, or designates at most one string-keyed inline map for otherwise
unmatched names. Ambiguous names, named inline fields, multiple inline maps, and
unsupported or malformed tag options are schema errors. Embedded fields are ordinary
named fields unless explicitly inlined. `configkit:"secret"` is the only current
ConfigKit field tag and records sensitivity for later lifecycle work.

Schema path traversal uses these same bindings. Explicit key segments address
mappings, explicit index segments address sequences, and parsed numeric pointer
tokens become indices only in a sequence. Sequence indices cannot have leading
zeroes; fixed-array indices are bounds checked. Dynamic `any` and inline-map entries
remain schema-valid for application-defined descendants.

Descriptors are reused by typed decoding, projection, and cloning so those operations
cannot acquire different field bindings.

## Defaults and typed decoding

The private decoder starts from an independently owned clone of the application's
typed defaults and applies only explicitly present input nodes. Defaults therefore
never become explicit, while equal-valued input remains explicit. Successful decode
produces detached typed state, baseline and final canonical trees, an input-presence
index, and default/input origins for the later public loading lifecycle.

| Configured input | Effect on the typed baseline |
| --- | --- |
| Absent field | Retain the default |
| Explicit false, zero, or empty string | Replace the default |
| Partial mapping or string-keyed map | Recurse and retain omitted default children or entries |
| Empty mapping | Retain existing mapping children |
| Sequence, including an empty sequence | Replace the entire sequence |
| Pointer object | Recurse into a non-nil default; otherwise allocate an object whose omitted children have ordinary Go zero values |
| Allowed null | Replace the value with nil and suppress its default |

Null is allowed only for pointers, maps, slices, and dynamic `any` values. It is an
error for scalars, structs, and arrays. New map entries and sequence elements begin
from their Go zero values; ConfigKit does not invent entry-level defaults.

Decoding is strict at every typed depth. Unknown struct fields, type mismatches,
invalid nulls, numeric overflow, and arrays with the wrong length are errors at the
original semantic path and source coordinates. String-keyed maps accept arbitrary
keys but preserve the strict schema of their values; inline maps accept unmatched
keys according to their element descriptor. Multiple independent field failures are
collected without publishing the partially decoded object.

Booleans and strings are not coerced. Integer destinations require integer syntax
and a value in the destination type's range. Float destinations accept finite JSON
numbers representable by their destination precision. Dynamic numbers become
`int64` for integer syntax and `float64` for fractional or exponent syntax, with
range and finiteness checks. Named scalar types retain their identity. Durations
decode only from strings accepted by `time.ParseDuration`; integers are rejected.
Paired text scalars decode only from strings through `encoding.TextUnmarshaler`, and
codec failures are classified without exposing application error text or input
values.

This decoder is available only through `Load`; ConfigKit exposes no generic document
conversion API.

## Typed projection and ownership

ConfigKit can internally project a schema-valid application value directly into its
owned canonical tree without YAML serialization. Projection includes every bound
field, including ordinary zero values; `omitempty` has no effect. Struct fields keep
binding order, dynamic map keys sort by decoded string, and sequences keep their
order. Integers retain exact base-10 values, finite floats use their destination
precision, durations use `time.Duration.String`, and paired text scalars use their
canonical marshaled text.

Nil pointers, maps, slices, and dynamic values project as null. Non-nil empty maps
and slices remain empty containers, preserving the distinction. Recursive acyclic
objects are supported. Cyclic values, non-finite floats, unsupported dynamic concrete
types, invalid codec output, and depth/node-limit violations fail with safe schema
classifications that omit codec error text.

The same descriptor graph creates detached typed copies. Pointers, maps, slices,
arrays, nested dynamic values, and shared references are recursively copied; shared
pointer identity is not preserved as configuration meaning. Paired text scalars are
rebuilt by unmarshaling their canonical text into fresh values, detaching private
backing storage. Ignored and unexported struct fields are zeroed rather than copied.

Private structural comparison operates on canonical trees. Mapping additions,
removals, and descendant replacements retain semantic paths. A sequence is atomic:
any sequence change reports the sequence path and never invents correspondence
between old and new indices. Cross-kind and null/container changes replace the
affected path.

These mechanisms currently support later decoding, resolution auditing, metadata,
and rendering. They are not a public generic conversion or cloning API.

## Safe failures

Source failures use stable `source.*` codes and YAML failures use stable `yaml.*`
codes declared by the root package. Diagnostics contain safe classifications,
semantic paths when applicable, and available source coordinates. Raw parser,
conversion, reader-error, and input text is not forwarded. Source labels and
diagnostic messages are application-visible and therefore must never contain
secrets; terminal control characters are escaped by `WriteDiagnostics`.