1226 lines
75 KiB
Markdown
1226 lines
75 KiB
Markdown
# Configuration Input Contract
|
|
|
|
This document owns ConfigKit's currently implemented source-input, YAML syntax,
|
|
schema, typed-decoding, loading, metadata, redaction, rendering, and ownership
|
|
contracts. The parser, owned document representation, and decoder remain private
|
|
behind the public loading lifecycle.
|
|
|
|
## Sources
|
|
|
|
ConfigKit exposes opaque sources through these constructors:
|
|
|
|
| Constructor | Current contract |
|
|
| --- | --- |
|
|
| `File(path)` | Requires a nonempty path, opens a regular file for each consumption, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. |
|
|
| `NewFilesystem(label, filesystem).File(name)` | Borrows a caller-owned `fs.FS` under a safe nonempty display label. `name` must be a non-root slash-relative `fs.ValidPath`. Each load reopens the selected regular file; copies of the `Filesystem` preserve namespace identity. Separate constructors create separate namespaces even for the same label or filesystem. |
|
|
| `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. |
|
|
| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. |
|
|
| `Mapping(label, values)` | Requires a safe nonempty label and a non-nil root `map[string]any`. Snapshots supported built-in values into an owned mapping document at construction; `{}` is valid. Constructor conversion failures become safe deferred diagnostics. |
|
|
| `NoSource()` | Selects defaults-only loading, creates no source record, and marks no paths explicit. |
|
|
|
|
The zero `Source` is invalid. Constructor argument failures are retained inside the
|
|
opaque source and become safe structured diagnostics when the source is consumed.
|
|
File and filesystem sources can observe changes between consumptions; byte and
|
|
mapping sources cannot. Mapping accepts nil, built-in string, boolean, signed or
|
|
unsigned integer, finite float, `[]any`, and `map[string]any` values recursively.
|
|
Typed nil maps and slices represent explicit null; empty collections remain empty.
|
|
Keys are sorted for stable provenance, integer decimal spelling stays exact, and
|
|
cycles or unsupported types are rejected without exposing values. Mapping input
|
|
uses the same directive and version rules as YAML, not CLI override restrictions.
|
|
|
|
Each source is limited to 8 MiB. A load using imports or an active overlay is limited
|
|
to 64 MiB across all accepted sources, 1,024 accepted sources, 64 active import
|
|
edges, and 1,000,000 input nodes in aggregate. Glob expansion separately permits
|
|
1,000,000 visited directory entries, including nonmatches. Reads request at most the
|
|
remaining allowance plus one byte so excess data is rejected rather than silently
|
|
truncated. Data returned together with a read error is not accepted as partial input.
|
|
For mapping sources, key and string bytes use the same accounting as programmatic
|
|
overrides, and the converted tree uses the same node and depth bounds.
|
|
|
|
## Single-source loading
|
|
|
|
`Load[T](source, options...)` accepts zero or one `Options[T]`. The configuration
|
|
root `T` must satisfy the application schema contract below. With no options it
|
|
starts from the Go zero value. A `Definition[T].Defaults` factory, when supplied,
|
|
runs exactly once after successful source parsing, schema construction, optional
|
|
schema-version checking, and selected-overlay loading. Its result is detached before
|
|
input is applied. Resolver-provided mutable values are detached before secret lookup
|
|
and validation, and a final detached copy prevents equal-valued aliases introduced
|
|
by validators from escaping into the returned configuration. The publication copy
|
|
preserves already validated secret state without rereading the environment.
|
|
|
|
The current lifecycle is:
|
|
|
|
```text
|
|
validate options → parse the selected source → traverse/compose enabled imports
|
|
→ build schema → check the optional application version → apply a selected overlay
|
|
→ apply explicit CLI overrides → expand sibling inheritance
|
|
→ create and clone defaults → strictly decode
|
|
→ recorded application resolution
|
|
→ resolve typed secret references → semantic validation → classify secrets
|
|
→ snapshot configuration and metadata
|
|
```
|
|
|
|
`File`, `Filesystem.File`, `Bytes`, `Reader`, and `Mapping` each contribute the first base source with deterministic
|
|
ID `source-1`; enabled imports continue in depth-first order as `source-2` onward.
|
|
`NoSource` skips parsing, supplies an absent version to a configured version checker,
|
|
contributes no source, and continues through defaults, decode, and projection. A
|
|
missing file remains an error; it is never interpreted as no input.
|
|
|
|
`Options` accepts the application definition, ordinary-source `Destination` and
|
|
`PlacementMode`, explicit imports, one selected overlay, filesystem
|
|
context, explicit CLI overrides, and shared source-version controls. The definition
|
|
may register strict sequence composition paths through
|
|
`SequenceCompositions`. Its zero-valued placement mode places the source mapping
|
|
at the destination; `sequence-item` explicitly wraps the mapping as one item at a
|
|
non-root destination. Unknown modes, invalid destinations, and invalid or
|
|
conflicting sequence registrations fail before source acquisition. Overlay and
|
|
CLI coordinates remain rooted at the application root. `NoSource` rejects
|
|
imports, active overlays, and filesystem context as invalid combinations but
|
|
accepts overrides. Input containing a reserved `configkit` mapping key is rejected
|
|
while imports are disabled.
|
|
|
|
Every source, options, schema, version, defaults, decode, resolution, or validation
|
|
failure returns a nil result and a `*LoadError`. Successful results contain caller-owned `Config`, detached
|
|
`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Each source
|
|
record's `Kind` identifies file, filesystem, bytes, reader, mapping, or CLI acquisition
|
|
independently of its precedence `Layer`. Host file paths remain canonical;
|
|
filesystem paths are slash-relative within their namespace; synthetic inputs have
|
|
no path. Mutating
|
|
the returned configuration or inspection records cannot alter ConfigKit's private
|
|
snapshot. ConfigKit performs no logging. `Result.WriteEffective` exposes only the
|
|
frozen redacted projection.
|
|
|
|
### Completion and resource lifetime
|
|
|
|
`Load` is synchronous and complete on return. Success means input assembly,
|
|
defaulting, typed decoding, configured application resolution, secret lookup,
|
|
validation, and configuration/metadata snapshotting have completed. A reported
|
|
load failure returns diagnostics and a nil result, as described above.
|
|
|
|
Before returning on success or failure, ConfigKit closes the filesystem handles
|
|
it opened for loading, including files and directory/root handles. Caller-owned
|
|
filesystems are borrowed, never closed. The returned
|
|
configuration and inspection metadata require no close operation and remain
|
|
usable independently of the source files. A supplied `Reader` remains
|
|
caller-owned and is never closed by ConfigKit.
|
|
|
|
ConfigKit performs no background work after returning: it does not watch inputs,
|
|
refresh secrets, or mutate published results. Inspection and rendering run only
|
|
when explicitly called and use the retained snapshot. Applications normally load
|
|
once near startup, but may explicitly load again using sources that satisfy their
|
|
documented reuse rules. Additional loads produce independently owned results.
|
|
|
|
Completion is not a transaction over external state. Files and environment
|
|
variables are observed during processing rather than as one simultaneous external
|
|
snapshot. ConfigKit does not roll back application callback effects or manage work
|
|
or resources created by application callbacks. The explicit secrets-directory
|
|
bootstrap remains a separate operation invoked by the application.
|
|
|
|
## Layered loading
|
|
|
|
`LoadLayered[T](main, options...)` accepts zero or one `LayeredOptions[T]` value.
|
|
Its `main` is an `Input` containing a source, import opt-in, and optional
|
|
filesystem context. A zero source is invalid; `Input{Source: NoSource()}` omits
|
|
the main document. Each `Underlay` contains inputs that compose strictly as
|
|
one tier, including each input's enabled imports. Empty groups contribute
|
|
nothing. Underlay groups are listed from lowest to highest priority. Repeating
|
|
a file within one group remains an import/repetition error; the same reusable
|
|
source may be selected again in another tier.
|
|
|
|
ConfigKit acquires and composes the main input first, constructs the schema,
|
|
and checks the optional application version from the main tier only. Underlay
|
|
roots cannot claim that version path (`underlay.version`); imported fragments
|
|
retain `import.version` protection. It then acquires and composes underlay
|
|
groups in order, recursively combines the groups, and applies the main tier,
|
|
one selected overlay, and explicit CLI overrides. Scalar, sequence, and null
|
|
definitions replace lower values between tiers; mappings combine recursively.
|
|
Syntax and strict-composition errors inside a tier cannot be masked by a
|
|
higher-priority tier. Defaults, typed decoding, resolution, secret lookup,
|
|
validation, and publication run once on the final assembled input.
|
|
|
|
An active layered overlay requires `LayeredOptions.OverlayFilesystem` explicitly,
|
|
independent of the main source's namespace. Its name and root-relative lookup
|
|
follow the selected-overlay rules below; inactive overlays ignore the context.
|
|
The same path in a different filesystem namespace is not a repeated file.
|
|
Ordinary `Load` continues to infer its selected-overlay context from its source.
|
|
|
|
Source IDs follow acquisition order—main and its depth-first imports, then each
|
|
underlay tier's inputs and depth-first imports, then overlay and CLI—even though
|
|
history follows precedence order. Source `Layer` describes precedence while
|
|
`Kind` describes acquisition. Underlay records use `LayerUnderlay` and a
|
|
one-based `UnderlayIndex` giving their original tier position; all other records
|
|
have index zero. Explicit empty groups and `NoSource` consume no source ID.
|
|
Byte, source, node, and glob-entry budgets are shared across the load, while
|
|
the existing per-source and import-depth bounds still apply.
|
|
|
|
When a source mapping is explicitly placed as an item, its effective destination
|
|
is the sequence container. After assembly assigns an index, nested source
|
|
destinations are projected through that item. Physical source locations and
|
|
resource contexts continue to identify the declaring source.
|
|
|
|
#### Input destinations
|
|
|
|
`Input.Destination` places the source's application mapping beneath a path
|
|
relative to the configuration root. Its zero or empty value means root, so
|
|
existing inputs keep their behavior. Main inputs and underlay inputs use the
|
|
same destination model. Destination segments are literal mapping keys: dots,
|
|
slashes, and numeric-looking text are not split or interpreted as sequence
|
|
indexes. Explicit index segments and paths that introduce the reserved
|
|
`configkit` or mapping-valued `extends` structure are invalid input options and
|
|
produce a structured `LoadError` before source acquisition.
|
|
|
|
Source-root `configkit.imports` directives are processed before placement. An
|
|
import without an explicit destination inherits the effective destination of
|
|
its importing document, including through nested imports. An import may instead
|
|
declare a relative destination as a sequence of literal mapping keys; its keys
|
|
append to the importing document's effective destination. Omission and `[]`
|
|
add no keys. Nested imports continue relative to their immediate importing
|
|
document, each declaration is independent of its siblings, and all matches of
|
|
one glob receive the same resulting destination. This logical placement does
|
|
not change how the physical import path resolves from its declaring file.
|
|
|
|
A valid destination on `NoSource()` contributes nothing; an actual empty
|
|
mapping materializes its destination containers. Once placed, contributions
|
|
follow the same strict composition within a tier and recursive precedence
|
|
behavior described above. A selected overlay and CLI overrides remain rooted at
|
|
the application root and are not moved beneath the main input's destination.
|
|
|
|
`SourceInfo.Destination` reports each acquired source's effective mapping or
|
|
sequence-container path, including imports. `PlacementMode` reports how that
|
|
source was placed; mapping is the zero value, and CLI/overlay sources report
|
|
mapping unless item placement was applied. `DestinationUnresolved` is true only
|
|
when an enclosing item index needed to name the source's destination has not yet
|
|
been assigned. In that case `Destination` reports the nearest resolved enclosing
|
|
collection container and omits the unresolved local suffix. A source's own item
|
|
placement still has an exact destination at its sequence container. Completed
|
|
loads project nested source destinations through their owning items. Callback and
|
|
failed-load snapshots preserve the evidence available at that point; a fallback
|
|
snapshot remains unresolved, while a later exact snapshot is false. Returned
|
|
source records and paths are detached. Placement does not change source IDs,
|
|
acquisition identity, physical source locations, or passive resource contexts.
|
|
Introduced ancestor mappings use the contributing document's root location as a
|
|
source fallback; authored content retains its original source coordinates.
|
|
|
|
The maintained [layered consumer example](consumers/README.md#native-filesystem-and-layered-input)
|
|
shows a main host file, two underlay tiers, embedded defaults, a mapping source,
|
|
and explicit resource access without a second configuration pass.
|
|
The accepted rationale for input placement and its relative-import extension
|
|
is recorded in [ADR 0011](adr/0011-input-destinations.md) and
|
|
[ADR 0012](adr/0012-relative-import-destinations.md).
|
|
|
|
### Sibling inheritance
|
|
|
|
Any input mapping may declare `extends: name` to inherit from the mapping named
|
|
`name` among its immediate siblings. Names are exact keys, including the empty
|
|
string; lookup does not search ancestor or unrelated scopes. The document root
|
|
and a sequence-element mapping have no sibling namespace for their own string
|
|
relationship, though mappings nested inside them can have one. An `extends`
|
|
value of null cancels a relationship without looking up a parent. Other value
|
|
types, missing or non-mapping parents, and cycles fail the load with value-free
|
|
diagnostics.
|
|
|
|
Within each sibling scope, direct mapping definitions resolve by dependency
|
|
readiness. A parent is resolved before a child; among currently eligible
|
|
definitions, the lexically smallest canonical semantic pointer is selected,
|
|
including newly eligible definitions as soon as their parent completes. This
|
|
ordering governs inheritance operations and their history, controls, and
|
|
transfers. It does not reorder the mappings in the resulting configuration.
|
|
|
|
`extends` is reserved structural input across ordinary and layered loads, every
|
|
source kind, imports, selected overlays, and explicit CLI overrides. It is
|
|
consumed before typed decoding and need not appear in the application's Go
|
|
types. Applications previously using `extends` as an ordinary YAML field must
|
|
rename that key and its field. No collection registration or keyword setup is
|
|
required.
|
|
|
|
ConfigKit completes strict composition within tiers and precedence assembly
|
|
between tiers before resolving relationships. An omitted `extends` retains a
|
|
lower-priority relationship; `extends: null` cancels it while preserving the
|
|
child's own settings. Superseded references are historical evidence, not active
|
|
relationships to validate. Each child recursively overlays its own settings
|
|
over its resolved parent: explicit zero, false, empty string, sequence, and
|
|
null values replace inherited values, while mappings combine recursively.
|
|
The child retains its key. All effective relationships resolve, including
|
|
entries the application may never select for execution. Partial parent
|
|
definitions undergo normal typed decoding and application-owned validation,
|
|
not a special structural exemption.
|
|
|
|
Enclosing relationships resolve first. Nested instructions carried by a parent
|
|
are then resolved separately at each destination, so a child's nested sibling
|
|
override can change an inherited nested relationship. Typed defaults and
|
|
application-derived values do not start another inheritance pass. Inherited
|
|
input counts as explicit presence for `Metadata.IsExplicit`,
|
|
`Resolution.IsExplicit`, and absence-based resolver defaults, even for false,
|
|
zero, empty, or null input. `Origin.Inheritance` distinguishes it from a direct
|
|
declaration with concrete source, destination, and controlling directive paths.
|
|
A direct child replacement has its own effective origin; displaced ancestry
|
|
remains inspectable in history. An inherited relative resource keeps the
|
|
declaring source's passive `SourceContext`, not the child's directory.
|
|
|
|
The optional schema-version checker still runs once against the main input at
|
|
its original point. Inheritance cannot add, replace, or consume the protected
|
|
version path; an unrelated relationship inside an enclosing mapping does not
|
|
invalidate an unchanged version. Expansion uses the decoded projection
|
|
node/depth limits without charging copied values as acquired sources;
|
|
retained evidence uses the existing history-node limit. See the
|
|
[consumer example](consumers/README.md#named-sibling-inheritance) for one-load
|
|
usage.
|
|
|
|
### Source resource contexts
|
|
|
|
`Result.SourceContext(id)` and callback-scoped `Resolution.SourceContext(id)`
|
|
return a passive `ResourceContext` for a source with filesystem context, or
|
|
false for unknown and contextless IDs. The result query is nil-safe; the
|
|
resolution query follows the callback's lifetime guard. Queries perform no
|
|
filesystem access. Obtain IDs from exact origin locations rather than guessing
|
|
one context for a composite origin or an application-derived value.
|
|
|
|
`ResourceContext.FS` is rooted at the source's configured root, and `BaseDir`
|
|
is its slash-relative directory inside that root. This includes imported
|
|
sources, selected overlays, host files even without enabled imports, native
|
|
files, and byte, reader, or mapping inputs with an explicit context. CLI and
|
|
contextless synthetic inputs have no resource context. To open a relative
|
|
resource, join `BaseDir` with the application resource name using slash-path
|
|
operations and open that path through `FS`.
|
|
|
|
For a resource-valued field, query `result.Metadata().Lookup(fieldPath)`, inspect
|
|
its origin locations, choose a source ID only when the application's policy can
|
|
identify the intended contributor, then call `result.SourceContext(id)`. The
|
|
context belongs to that source even when the field survived from a lower-priority
|
|
underlay. A composite origin has no automatically selected resource base.
|
|
|
|
Native contexts borrow the caller's filesystem within the configured root.
|
|
Host contexts retain a canonical root path, not an open root handle; each
|
|
resource `Open` checks containment and opens a fresh handle that the returned
|
|
file's `Close` releases. Applications own and close the files they open.
|
|
Resources are application data, not configuration input, and do not consume
|
|
configuration byte or node budgets. The loaded configuration and metadata stay
|
|
usable if source files later change or disappear. Later resource operations
|
|
observe the backing filesystem then and may fail if files move, disappear, or
|
|
a caller-owned filesystem closes. This is not a snapshot or universal sandbox.
|
|
|
|
### Application schema versions
|
|
|
|
`VersionRule` declares a semantic path and non-nil checker. Before defaults or typed
|
|
decoding, the checker receives a `Version` preserving absent, explicit null, string,
|
|
or strict decimal-integer input. Strings and integers supply their exact spelling in
|
|
`Version.Text`; ConfigKit does not impose a machine-integer range. Fractional or
|
|
exponent numbers, booleans, mappings, and sequences produce `version.type`, and the
|
|
checker does not run. ConfigKit does not interpret the text or impose a universal
|
|
version path.
|
|
|
|
Integer versions use the strict grammar `^-?(0|[1-9][0-9]*)$`. Their spelling is
|
|
preserved exactly, including `-0`, and application policy decides which identifiers
|
|
are supported. A numeric version has no built-in `int64` range limit. Existing parser
|
|
policy still rejects leading zeroes, leading plus signs, hexadecimal notation, and
|
|
other non-JSON scalar forms before version checking.
|
|
|
|
At exactly the configured semantic path, accepted integer input can decode into a
|
|
plain Go string or named string without a text codec, including pointer wrappers.
|
|
This is a load-local exception: siblings and unrelated fields, dynamic values,
|
|
integer and float destinations, durations, `Secret`, and text codecs retain ordinary
|
|
strict decoding. The retained source and history stay numeric while typed projection
|
|
and effective YAML contain the resulting string; explanation records this through
|
|
the normal decoding reconciliation event. Fragment, overlay, and CLI ownership
|
|
protections apply before this conversion.
|
|
|
|
The checker reports safe, value-free errors and warnings through `Validation.Error`
|
|
and `Validation.Warning`; ConfigKit resolves primary and related paths to exact or
|
|
nearest input locations. Reported errors fail loading. If the checker returns an
|
|
error without reporting one, ConfigKit emits a generic `version.hook_failed`
|
|
diagnostic and suppresses the returned error text. Warnings alone are retained on a
|
|
successful result. Callback panics remain programming failures.
|
|
|
|
#### Shared source-version checks
|
|
|
|
`Options.SourceVersionMode` and `LayeredOptions.SourceVersionMode` select how
|
|
one `Definition.Version` policy applies across source documents. The zero value
|
|
`SourceVersionModeDefault` preserves the existing main-only check at its
|
|
effective path and existing protected-path behavior. `SourceVersionModeIfPresent`
|
|
checks each participating document that declares a version and permits absent
|
|
non-main declarations. `SourceVersionModeRequired` rejects a participating
|
|
document without a declaration. Enabled modes require a usable `VersionRule`
|
|
and checker; an unknown mode or unusable rule is an options error before source
|
|
access or callbacks.
|
|
|
|
In enabled modes, `Definition.Version.Path` is interpreted in each acquired
|
|
document's own coordinates, before placement and composition. The main document,
|
|
direct underlay documents, imported documents (including transitive and glob
|
|
matches), and the selected overlay participate. `NoSource`, absent optional
|
|
imports, empty underlay groups, defaults, resolution output, inheritance, and
|
|
synthetic CLI overrides are not documents to check. An absent declaration in an
|
|
actual main document still reaches the checker in if-present mode; required mode
|
|
rejects it first. With `NoSource`, the existing single absent-main checker
|
|
behavior remains in both enabled modes. If-present omissions in non-main
|
|
documents skip the checker. Explicit null is present and reaches the checker;
|
|
unsupported selected values produce `version.type`. No equality with the main
|
|
declaration or non-null rule is imposed. Acceptance establishes only that the
|
|
declaration is accepted by the consumer's checker; it does not establish that
|
|
each fragment conforms to a complete application schema. Final assembled
|
|
decoding and application validation still run once.
|
|
|
|
A header that is absent below a structural control follows the same omission
|
|
policy. Actual header/control overlap is rejected, and ordinary structural
|
|
validation and main-version ownership protection continue independently.
|
|
|
|
Accepted non-main declarations are consumed from their owned payload before
|
|
composition; their authored ancestors remain, even if empty. The main
|
|
declaration remains in the application. Ordinary release metadata should use a
|
|
different path. Enabled main protection uses the main input's effective path:
|
|
its `Destination` prefixed to the source-local `VersionRule.Path`. This protects
|
|
the remaining payload against imports, underlays, inheritance, and CLI
|
|
overrides. Thus a mounted main looks up a local header while ownership checks
|
|
protect its destination-prefixed application path. Main checker diagnostics and
|
|
declaration locations use source-local coordinates; ordinary payload
|
|
diagnostics continue to use effective paths. Default-mode diagnostics are
|
|
unchanged.
|
|
|
|
`SourceInfo.VersionCheck` is nil when enabled declaration processing did not
|
|
occur. Otherwise it records a source-local `Path`, the observed `Version` state
|
|
(`Present`, `Null`, and, when disclosed, exact `Text`), an actual or nearest
|
|
available `Location`, and whether text was redacted. A non-nil record with
|
|
`Present == false` records observed absence; it does not say the checker ran or
|
|
accepted a declaration. Absent evidence still identifies the full requested
|
|
header path, while its location identifies the nearest available source node.
|
|
During callbacks and in failed-load source records,
|
|
text is conservatively withheld because final classification may not be
|
|
available. Successful final source records reflect completed sensitivity
|
|
classification at the declaration's mounted path and ancestors, including
|
|
historical classification. Redaction empties `Text` while retaining presence
|
|
and null state. Returned copies are detached; evidence does not reopen inputs.
|
|
Consult [ADR 0013](adr/0013-shared-source-version-checks.md) for the accepted
|
|
decision and the [development guide](development.md#baseline-validation) for
|
|
the completed implementation and remediation records in Git history.
|
|
|
|
### Layered metadata
|
|
|
|
`Result.Metadata` describes every final canonical path. Queries retain explicit
|
|
key/index distinctions using actual effective and historical container shapes;
|
|
only parsed numeric tokens are resolved contextually. A descendant that never
|
|
existed is not historical merely because an atomic ancestor was replaced. Exact
|
|
surviving direct or inherited input paths are explicit and retain their source
|
|
layer and derivation; omitted paths have a default origin.
|
|
An explicit parent mapping does not make omitted children explicit. Null state and
|
|
available source coordinates are retained. `History` records the Go-zero or application
|
|
default baseline followed by exact base or overlay definitions, including
|
|
equal-valued input. Overlay replacement removes displaced descendants from effective
|
|
metadata while retaining their base definition and the parent replacement operation
|
|
in history. All metadata queries and returned nested records are detached.
|
|
|
|
`Metadata.HistoricalPaths` enumerates observed paths, including removed paths,
|
|
without reloading input or resolving unselected credentials. Consumers can use
|
|
it with `History` to validate ownership of declarations hidden by an overlay;
|
|
domain ownership rules remain application responsibilities. The method's Go
|
|
documentation in [metadata.go](../metadata.go) defines enumeration and path-ordering
|
|
details.
|
|
|
|
History is chronological: a path can have a synthetic default record, its exact
|
|
base definition, an overlay replacement, structural inheritance, and a recorded
|
|
resolution operation. A descendant
|
|
query uses the ancestor's path for a whole-container or whole-sequence replacement;
|
|
the descendant can therefore retain history after `Lookup` stops reporting it as
|
|
effective. Equal-valued input is still an explicit base definition, while a derived
|
|
value for absent input remains non-explicit.
|
|
|
|
Appending registered records during strict composition assigns each contribution
|
|
its assembled sequence index in history. A later precedence tier that supplies a
|
|
sequence records replacement at the sequence path; history does not imply
|
|
identity-based merging across tiers.
|
|
|
|
A mapping assembled from the selected file and imports has one composite base origin
|
|
whose locations retain source traversal order. A mapping also visited by an overlay
|
|
has an overlay-layer composite origin with the lower contributing locations followed
|
|
by the overlay location; an empty mapping is still an explicit container even though
|
|
it contributes no children. Resolution origins are synthetic: they have layer
|
|
`resolution`, no fabricated file location, and a `derived_default` or `normalization`
|
|
operation. Reasons and dependency paths retain first occurrence order with duplicates
|
|
removed. Source records remain ordered as selected source, depth-first imports, the
|
|
selected overlay, then synthetic CLI records in supplied order.
|
|
|
|
Inheritance adds no acquired source or precedence layer. `Origin.Inheritance`
|
|
lists concrete transfers from contributor paths through destinations, with
|
|
each step's controlling `extends` path and location. `History` retains source
|
|
applications at their original paths and projects parent contributions, child
|
|
replacements, and consumed controls at destination paths after source assembly
|
|
and before typed decoding. `HistoricalPaths` includes consumed directives and
|
|
displaced inherited descendants. A control can have history even when `Lookup`
|
|
reports it absent. Queries return detached records.
|
|
|
|
### Secret classification and effective rendering
|
|
|
|
Secret classification combines `configkit:"secret"` field tags with the optional
|
|
pure `Definition.Secret(Path) bool` predicate. Tags on concrete structs inside
|
|
supported dynamic values are retained in typed projections, including superseded
|
|
defaults and pre-resolution values. A tagged or predicate-classified
|
|
container makes all descendants sensitive. ConfigKit evaluates final and historical
|
|
paths, so replacing or deleting a definition does not erase its classification. The
|
|
predicate can add sensitivity but cannot declassify a tagged path.
|
|
|
|
Recorded resolution dependencies propagate sensitivity to their derived target.
|
|
Propagation reaches a fixed point, so record/map iteration order does not affect the
|
|
result and dependency cycles terminate. A dependency on a classified descendant or
|
|
on a container that contains one is treated conservatively as sensitive. Applications
|
|
must classify secret-producing transformations whose dependencies are not recorded.
|
|
Inherited value and control paths link source and destination classifications
|
|
through chains and nested copies; unrelated sibling fields are not classified
|
|
merely because another field is secret. Displaced historical evidence remains
|
|
redacted. Secret references are copied only as unresolved names; credential
|
|
lookup stays after decoding and application resolution.
|
|
|
|
`Result.WriteEffective` writes the frozen resolved snapshot as deterministic redacted
|
|
YAML. Mappings sort by decoded key string, sequences retain order, indentation is two
|
|
spaces with noncompact sequence indentation, line endings are LF, the document has no
|
|
start or end marker, and output ends with a newline. Nil pointers/maps/slices remain
|
|
`null`; non-nil empty maps and sequences remain `{}` and `[]`. A secret root is
|
|
replaced by the quoted string `'[REDACTED]'`. Redacted output is intended for
|
|
inspection and is not promised to be reloadable.
|
|
|
|
Rendering uses only a safe adapter tree built from the already completed canonical
|
|
typed projection. It does not invoke application YAML marshal hooks or text codecs;
|
|
all text-codec projection has succeeded before `Load` returns. Caller mutation of
|
|
`Result.Config` cannot change later output. Writer failures, including short writes,
|
|
are returned and may leave partial redacted output; internal dumper failures are
|
|
translated to a fixed safe error. There is no unredacted rendering option and
|
|
ConfigKit performs no implicit logging.
|
|
|
|
`Result.Config` is the explicit trusted boundary and contains ordinary typed values,
|
|
including secrets. Metadata records, diagnostics, and load errors contain no
|
|
configuration values. Explanation snapshots deliberately expose non-secret
|
|
configuration history after applying the redaction contract below. Applications
|
|
must still keep source labels, resolution reasons, and diagnostic messages value-free
|
|
because ConfigKit cannot infer and sanitize secrets embedded in arbitrary application
|
|
prose.
|
|
|
|
### Historical-value explanation
|
|
|
|
`Result.Explain(path)` returns a detached `Explanation` for any known effective or
|
|
historical path. Its effective value and each chronological before/after value are
|
|
`ValueSnapshot` records containing deterministic YAML text rather than mutable Go or
|
|
parser values. `Present=false` with empty YAML means absence or deletion; explicit
|
|
null is present and renders as `null`. The history starts with the canonical typed
|
|
application-default or Go-zero baseline, then includes only relevant base sources in
|
|
traversal order, the selected overlay, supplied CLI records, and audited net
|
|
resolution operations. Structural inheritance events appear after source replay
|
|
and before typed reconciliation, including consumed or canceled `extends` input
|
|
and equal-valued child replacements. Overlapping parent/child resolution records
|
|
are rendered
|
|
as one net subtree event with combined reasons and dependencies, rather than
|
|
pretending that the recorded assignments ran in lexical order. Explicit equal
|
|
writes remain visible. When input replay
|
|
differs from the final typed input, a `ChangeDecoding` event records the transition
|
|
before domain resolution. Its origin retains the decoded path's source/default
|
|
attribution; its reason identifies static-default retention and canonical decoding.
|
|
This built-in inspection operation is not accepted by `Resolution.Record`.
|
|
|
|
Base, overlay, and CLI history preserves the policy-validated input representation,
|
|
including a superseded value that could not have decoded into the final Go field.
|
|
Baseline, effective, and resolution values use canonical typed projection. A
|
|
decoding event bridges replayed input and canonical typed values, so intermediate
|
|
null/scalar replacements cannot incorrectly imply permanent default removal. Mapping
|
|
queries reconstruct each source or override contribution, while sequences remain
|
|
atomic: a descendant history identifies its presence before and after the containing
|
|
sequence replacement without implying element-wise merging. Removed paths remain
|
|
explainable; unknown and malformed paths return false.
|
|
|
|
Every snapshot applies the same historical tag, predicate, and recorded-dependency
|
|
sensitivity closure as effective rendering before YAML is exposed. A redacted scalar
|
|
is the quoted marker `'[REDACTED]'`, and a container snapshot has `Redacted=true` if
|
|
it contains a masked descendant. Typed `Secret` snapshots contain only the configured
|
|
environment name. `SecretReference` reports that name only for a successfully
|
|
resolved effective reference, and is omitted when its path is classified. Credential
|
|
values are never retained in explanation history.
|
|
|
|
`Result.WriteExplanation(output, path)` renders the safe structured result with the
|
|
canonical JSON-pointer path, canonical-versus-input representation labels, effective
|
|
source, composite locations, chronological values, resolution reasons and
|
|
dependencies, and successful secret-reference lookup attribution. Control characters
|
|
in paths, source identifiers and labels, reasons, dependencies, and reference names
|
|
are escaped. An unknown path returns a fixed safe error before writing anything;
|
|
writer errors and short writes are returned and may leave partial output. There is no
|
|
unredacted mode.
|
|
|
|
Retained history consists of immutable source applications and audited subtree
|
|
deltas, rather than a full tree copy per field. Shared nodes count once toward a
|
|
4,000,000-node retained-history limit; exceeding it fails loading with
|
|
`explain.limit_exceeded`. Replaying one requested path is performed on demand, and
|
|
the returned YAML strings and nested records cannot mutate later explanations.
|
|
|
|
Application commands should render the separate `SecretsReport` immediately after
|
|
an explicit `LoadSecretsDir` bootstrap, then use `WriteDiagnostics`,
|
|
`WriteEffective`, or `WriteExplanation` on the later load result as appropriate.
|
|
ConfigKit does not infer bootstrap outcomes from a secret reference and does not
|
|
bundle these independent reports into a command framework.
|
|
|
|
## Effective configuration diffs
|
|
|
|
`configkit.Diff(left, right)` compares two completed successful loads of the same
|
|
configuration type. `left` is the earlier state and `right` is the later state.
|
|
Applications load both sides with the sources and options they choose; to inspect
|
|
an overlay's impact, load the same base once without the overlay and once with it
|
|
selected. The [consumer guide](consumers/README.md#effective-configuration-diffs)
|
|
links to maintained examples. The public API returns safe changed-region records;
|
|
applications own their presentation.
|
|
|
|
Comparison uses the frozen canonical effective inspection values, not mutable
|
|
`Result.Config`, source text, provenance, or history. It does not reopen sources,
|
|
reread environment variables, or rerun defaults or callbacks. Equal effective
|
|
values produce no records even when sources, explicit-input status, defaults, or
|
|
resolution histories differ; for example, explicit zero and defaulted zero are
|
|
equal. Mappings compare recursively, sequences compare atomically, and scalar,
|
|
kind, or null/container replacements report their affected region. Records use
|
|
the existing comparator's canonical pointer ordering. `Present=false` with empty
|
|
YAML means absence; explicit null is present and renders as `null`.
|
|
|
|
Before either side is returned, sensitivity from both loads is applied to both
|
|
snapshots, including historical and dependency-derived classification. A
|
|
classification on either side protects the corresponding values on both sides.
|
|
`Redacted` reports masking, while `HiddenChanged` reports an actual concealed
|
|
value or presence change; it is not implied merely by redaction. A structural
|
|
replacement that prevents safe correspondence masks the complete reported
|
|
region, and an absent side remains absent. Record membership establishes that a
|
|
change occurred even when safe snapshots render identically. Literal
|
|
marker-looking input is ordinary input; consumers use the record fields rather
|
|
than interpreting marker text.
|
|
|
|
Typed secret references contribute their configured environment names to the
|
|
frozen comparison, never resolved credential bytes. Credential rotation behind
|
|
an unchanged reference is therefore not detected. The records contain changed
|
|
regions only, not paired full-document context; independently calling
|
|
`WriteEffective` on each result does not create cross-protected context. Nil or
|
|
uninitialized results return an error with no records. A failed `Load` retains
|
|
its normal nil-result/`LoadError` behavior, which callers handle before invoking
|
|
`Diff`.
|
|
|
|
## Typed secret references
|
|
|
|
Credential-bearing fields use the opaque `Secret` type and store an environment
|
|
variable name in YAML, such as `api_key: OPENROUTER_API_KEY`. `SecretEnv(name)`
|
|
constructs the same unresolved reference for application defaults or recorded
|
|
resolution. Names must match `[A-Za-z_][A-Za-z0-9_]*`; they are public
|
|
configuration and remain visible through `EnvName`, canonical projection, history,
|
|
and effective output unless the path is separately classified for redaction.
|
|
|
|
A non-pointer `Secret` is required. An absent, null, or invalid reference, or an
|
|
unset or empty environment value, fails with a safe `secret.*` diagnostic at the
|
|
configuration path. `*Secret` is optional only when nil; every non-nil reference
|
|
must resolve to a nonempty value. Secret references work in typed structs, maps,
|
|
pointers, slices, and arrays. A dynamic `any` value does not establish secret
|
|
semantics and cannot contain `Secret`; applications must use a typed slot.
|
|
|
|
ConfigKit resolves only the effective names after recorded application resolution
|
|
and before semantic validation. Each unique name is read with `os.LookupEnv` once
|
|
per load. Validation can inspect the credential through the deliberately named
|
|
`Value()` accessor and distinguish success with `IsResolved()`. Later loads read
|
|
the environment again, and cloned defaults are rebuilt from names so a previously
|
|
resolved result cannot retain a stale credential. `Load` never writes to the
|
|
environment.
|
|
|
|
Credential bytes are held only in the per-load lookup state and the trusted returned
|
|
configuration, and are returned only by `Value`. Secret formatting uses a safe
|
|
public-name representation, JSON marshals only the name, and projection, metadata,
|
|
histories, diagnostics, effective output,
|
|
and resolution snapshots never retain the credential. Validation remains read-only:
|
|
in addition to ordinary projected-value comparison, ConfigKit checks the private
|
|
resolved state against this load's lookup result. Validator mutation checks also
|
|
reject changes to projected field sensitivity, even when a replacement dynamic
|
|
concrete type preserves the same textual values. Applications must keep their
|
|
validation messages value-free and must not include `Value()` in diagnostics.
|
|
|
|
For example, a typed pipeline field is declared as:
|
|
|
|
```go
|
|
type PipelineConfig struct {
|
|
APIKey configkit.Secret `yaml:"api_key"`
|
|
}
|
|
```
|
|
|
|
The executable `ExampleSecretEnv` demonstrates loading and inspecting the safe
|
|
reference state without printing its value.
|
|
|
|
## Explicit secrets-directory bootstrap
|
|
|
|
`LoadSecretsDir(directory)` is a separate, opt-in startup operation. It is never
|
|
called by `Load`, is not a YAML option, and does not participate in configuration
|
|
validation. Applications choose the native directory path and should invoke the
|
|
operation once before starting concurrent work. The selected path may itself be a
|
|
symlink; ConfigKit resolves it to an existing canonical directory before examining
|
|
its immediate entries.
|
|
|
|
The bootstrap enumerates at most 1,024 entries, including dot-prefixed entries.
|
|
Entries whose names begin with `.` (files, directories, or symlinks, such as `.git`
|
|
and `.gitignore`) are ignored before name or metadata validation, are never opened,
|
|
and are omitted from the report. Remaining entries are sorted by filename; each
|
|
must have an ASCII environment-variable name and be a regular, nonsymlink
|
|
file. Metadata for all remaining entries is validated before file content is read
|
|
or process state is changed. Existing environment variables win, including variables whose
|
|
value is empty; their files are not opened. Each eligible file is limited to 1 MiB,
|
|
and eligible content is limited to 8 MiB in total. ConfigKit removes exactly one
|
|
terminal LF or CRLF, preserves all other whitespace, accepts an empty value, and
|
|
rejects NUL bytes.
|
|
|
|
All eligible files are read successfully before application begins. Presence is
|
|
checked again immediately before each `os.Setenv`: a variable populated since
|
|
preflight is skipped. A write failure stops application without rollback, so prior
|
|
writes remain and later eligible entries are `not_applied`. This operation is not
|
|
atomic and does not provide compare-and-set semantics for concurrent environment
|
|
mutation.
|
|
|
|
`SecretsReport.Entries` returns detached, filename-ordered `SecretFileResult`
|
|
records with `loaded`, `skipped_existing`, `not_applied`, or `failed` status.
|
|
`ExistingEmpty` distinguishes an empty existing winner. `SecretsReport.WriteTo`
|
|
renders only quoted names, canonical paths, statuses, and that empty-state flag;
|
|
it never retains or renders file contents. Failures return the non-nil partial
|
|
report with a `*SecretsError` containing detached `secrets.*` diagnostics and no
|
|
raw filesystem or environment error chain. Applications should retain the report
|
|
alongside the later configuration result and explicitly present blocked and failed
|
|
files when appropriate; ConfigKit stores no implicit global report.
|
|
|
|
### Recorded application resolution
|
|
|
|
An optional `Definition.Resolve` callback runs after successful typed decoding. It
|
|
receives the working configuration and a per-load `Resolution` context. The context's
|
|
`IsExplicit` query uses surviving input presence, so explicit false, zero, empty, and
|
|
null are never confused with absence or static defaults.
|
|
|
|
`Resolution.InputInfo(path)` returns detached effective metadata from the fixed
|
|
decoded entry state, before application resolution runs. It includes retained static
|
|
defaults, preserves explicit null, and uses the same exact semantic path matching as
|
|
`Metadata.Lookup`; missing, malformed, and key/index-shape-mismatched paths return
|
|
false without diagnostics. Its `Explicit` value agrees with `IsExplicit` for every
|
|
found entry path. Resolution changes, additions, removals, and moves do not alter
|
|
this entry view.
|
|
|
|
`Resolution.Source(id)` returns a detached record for an already acquired source in
|
|
the same load: the main source, imports, selected overlay, and supplied CLI records
|
|
where applicable. Empty and unknown IDs return false. It never reloads input, reads
|
|
the filesystem, resolves symlinks, or looks up credentials. A source path exists
|
|
only when the acquired source has one; defaults and byte, reader, and CLI sources do
|
|
not acquire an invented file path. Applications select a base directory from exact
|
|
origin locations and these source records according to their own policy.
|
|
|
|
Both queries are callback-scoped like the rest of `Resolution`: nil, zero, or
|
|
retained contexts panic with the fixed closed-context failure outside the callback.
|
|
Returned field and source records are detached and remain usable after callback
|
|
exit; changing them cannot affect later queries, results, or another load.
|
|
|
|
`Default` assigns and records a derived value only when its path is not explicit.
|
|
`Normalize` always assigns and records. For transformations performed directly by
|
|
application code, `Resolution.Record` declares a derived-default or normalization
|
|
operation; a recorded container covers changes to its descendants. Records require
|
|
a schema-valid path, valid kind, safe nonempty reason, and schema-valid dependency
|
|
paths. Repeated records at one path are coalesced: reasons and dependencies retain
|
|
their first occurrence, the last helper value is authoritative, and any normalization
|
|
makes the combined operation a normalization.
|
|
|
|
ConfigKit projects the configuration before and after the callback. Every net change
|
|
must have an exact or ancestor record. Helper-assigned values must agree with the
|
|
final value at their declared path, which supports ordinary map-entry copy/writeback
|
|
while detecting forgotten writeback or unrelated targets. Invalid paths, kinds,
|
|
values, records, helper targets, and unrecorded mutations fail the load. Equal-valued
|
|
derived assignments may still record application intent even when there is no
|
|
structural change.
|
|
|
|
`Resolution.Error` and `Resolution.Warning` attach exact or nearest available input
|
|
locations. Reported errors fail the load; warnings survive on a successful result. A
|
|
returned resolver error adds one generic `resolve.hook_failed` error only when the
|
|
callback did not report an error, and its text is never exposed. Callback panics are
|
|
programming failures. Resolution and validation contexts are callback-scoped; using
|
|
a nil, zero, or retained context outside its active callback panics with fixed,
|
|
value-free text.
|
|
|
|
Successful records update effective origins and append detached metadata history
|
|
without changing input explicitness. Removed paths leave effective metadata but
|
|
retain history. ConfigKit privately snapshots each recorded before/after subtree for
|
|
later safe explanation; caller mutations of `Result.Config` cannot alter those
|
|
snapshots.
|
|
|
|
### Semantic validation
|
|
|
|
An optional `Definition.Validate` callback runs after successful resolution and
|
|
receives the resolved configuration plus a per-load `Validation` context. It can
|
|
collect multiple independent `validate.error` and `validate.warning` reports.
|
|
Warnings continue the lifecycle and are returned on success; any error returns a nil
|
|
result with the full safe diagnostic and source report.
|
|
|
|
Primary and related paths must be valid for the application schema. Omitted declared
|
|
fields and missing string-map or dynamic keys are valid report targets; unknown
|
|
struct fields and malformed paths produce `validate.invalid_path`. ConfigKit attaches
|
|
the exact surviving input location when one exists, otherwise the nearest input
|
|
container location, or no primary location for defaults-only synthetic state.
|
|
|
|
Validation is read-only. ConfigKit projects the configuration immediately before and
|
|
after the callback and returns `validate.mutation` at every detected net-change path,
|
|
including nested mutable state. A value that can no longer be projected is also a
|
|
mutation failure. No mutated or otherwise invalid configuration is published.
|
|
Callback panics remain programming failures, and retained contexts use the same
|
|
fixed closed-context panic as version and resolution callbacks.
|
|
|
|
Only schema-bound exported state participates in loading, resolution audit,
|
|
validation audit, metadata, and snapshots. Ignored or unexported ordinary fields are
|
|
zero in returned detached values. Applications retain responsibility for domain
|
|
rules and runtime initialization; validators should not start clients or depend on
|
|
live services as a substitute for configuration validation.
|
|
|
|
## Strict document composition
|
|
|
|
ConfigKit strictly composes parsed base documents when `Options.Imports` is true.
|
|
Application defaults do not participate in this operation.
|
|
|
|
| Existing node | Incoming node | Result |
|
|
| --- | --- | --- |
|
|
| Absent | Absent | Absent |
|
|
| Absent | Present | Retain the incoming node |
|
|
| Present | Absent | Retain the existing node |
|
|
| Mapping | Mapping | Recurse by mapping key |
|
|
| Any other pair of present nodes | Any other pair of present nodes | `compose.conflict` |
|
|
|
|
Every non-mapping overlap is a conflict, even when values are equal, except for
|
|
sequence/sequence contributions at a registered sequence-composition path. Nulls
|
|
remain atomic conflicts. Unregistered sequences conflict when repeated and are
|
|
never joined by index. Disjoint entries under shared mappings compose, making
|
|
stable keyed mappings the natural representation for independently contributed
|
|
entries.
|
|
|
|
`Definition.SequenceCompositions` registers literal mapping paths and one literal
|
|
`IdentityField` for sequences that should compose independently authored records.
|
|
Registration paths follow input-destination key rules: parsed numeric tokens are
|
|
literal mapping keys, explicit index segments and reserved `configkit` or
|
|
mapping-valued `extends` segments are invalid, and the root registration is
|
|
harmless and unused. The identity field is one literal string key; neither the
|
|
field name nor identity strings are normalized.
|
|
At each registered path, every actual sequence contribution in a strict tier is
|
|
validated, including a singleton contribution: each record must be a mapping with
|
|
a present string identity field. Distinct exact identity strings append in source
|
|
and list order; duplicate identities conflict even if the records are otherwise
|
|
equal or disjoint. Empty identity strings are valid. Malformed records produce
|
|
`compose.invalid`; duplicate identities produce `compose.conflict`, with the
|
|
sequence container and source locations identifying a duplicate without exposing
|
|
identity values. Missing paths and non-sequence values do not fail by themselves.
|
|
Identical registrations coalesce; conflicting registrations at one path and
|
|
invalid registration paths produce `options.invalid`. Unused and root registrations
|
|
are allowed and have no effect.
|
|
|
|
Registration changes strict composition only. Each underlay group and the main
|
|
tier has independent duplicate-identity state. Precedence assembly is unchanged:
|
|
a present higher-priority sequence replaces the lower sequence as one value,
|
|
including at a registered path. Overlays, CLI overrides, defaults, inheritance,
|
|
and application resolution do not run the registered identity check. The accepted
|
|
rationale is recorded in [ADR 0014](adr/0014-sequence-placement-and-composition.md).
|
|
The [maintained prompt catalog](consumers/README.md#identified-prompt-sequences)
|
|
shows item path and glob imports, helper-supplied identity, and main-tier
|
|
replacement of a lower-priority sequence.
|
|
When an enabled source-version mode is selected, declarations are checked in each
|
|
source's own coordinates before placement and composition. Accepted non-main
|
|
headers are consumed before strict composition and cannot supply a registered
|
|
record identity. Final typed decoding and application validation still run on the
|
|
completed configuration.
|
|
|
|
An empty mapping contributes no child definitions. Two empty mappings compose, and
|
|
an empty mapping neither clears nor conflicts with another mapping's children.
|
|
Shared mapping containers retain all contributing value locations as their composite
|
|
base origin. A conflict reports the first claim as its primary location and every
|
|
remaining claim as related locations, in deterministic source and declaration
|
|
traversal order. Traversal order never authorizes replacement. Composition clones
|
|
the resulting tree and its ordered claim records rather than mutating or aliasing an
|
|
input document.
|
|
|
|
## Selected overlays
|
|
|
|
`Options.Overlay` selects at most one overlay by name. Names must match
|
|
`[a-zA-Z0-9_-]+`; they are not paths. The empty name and exact lowercase name
|
|
`default` select no overlay, and `overlays/default.yml` is never loaded implicitly.
|
|
Every other name resolves only to `<configuration-root>/overlays/<name>.yml`, with
|
|
no extension fallback, case normalization, inheritance between overlay files,
|
|
or overlay stack. A selected missing overlay is an error.
|
|
|
|
The overlay is a separately policy-validated YAML mapping. It is loaded after the
|
|
base import graph is composed and its schema version is checked, but before defaults
|
|
and typed decoding. Overlay bytes, nodes, and source records count toward the same
|
|
aggregate limits as base inputs. Its canonical path must remain within the configured
|
|
root and must not duplicate a file already present in the base graph.
|
|
|
|
Overlay is intentionally distinct from strict composition:
|
|
|
|
| Existing lower node | Incoming overlay node | Result |
|
|
| --- | --- | --- |
|
|
| Absent | Value | Retain the overlay value |
|
|
| Value | Absent | Retain the lower value |
|
|
| Mapping | Mapping | Recurse by mapping key |
|
|
| Any other pair of present nodes | Any other pair of present nodes | Replace the entire lower node |
|
|
|
|
Scalars, sequences, and null are atomic. Thus an overlay sequence replaces rather
|
|
than extends a base sequence, `[]` clears it, null is an explicit replacement, and
|
|
cross-kind replacement is allowed when the final application schema accepts it.
|
|
Replacing a subtree removes all displaced descendants from effective presence and
|
|
origin data; history retains the displaced definitions and identifies the parent
|
|
replacement. Surviving overlay paths have overlay origins, while recursively shared
|
|
mapping containers retain their contributing locations.
|
|
|
|
An overlay may not contain `configkit` anywhere, contribute the configured schema
|
|
version, or replace an ancestor of that version with a scalar or null. These rules
|
|
apply even if the supplied version text would be unchanged. Overlays cannot mask a
|
|
base parse, composition, or version failure because those checks run first.
|
|
|
|
## Explicit CLI overrides
|
|
|
|
`Options.Overrides` accepts only records the application knows were explicitly set
|
|
by its CLI parser. Each `Override` supplies a non-root mapping `Path`, a detached
|
|
`Value`, and a required safe display `Flag` such as `--workers`; ConfigKit has no CLI
|
|
parser dependency. Parser defaults must not be submitted as overrides, and `Flag`
|
|
must never contain the raw argument or configured value. General environment
|
|
variable overlays are an explicit non-goal: environment access is reserved for
|
|
typed secret references.
|
|
|
|
Overrides apply after the selected overlay and before typed defaults/decoding, so
|
|
the input precedence is application defaults < composed base < selected overlay <
|
|
CLI. Recorded resolution, typed secret lookup, and semantic validation follow the
|
|
effective override tree. Overrides also work with `NoSource`.
|
|
|
|
Values are converted directly into ConfigKit's owned tree without YAML parsing or
|
|
string coercion. Supported values are `nil`, built-in strings and booleans, all
|
|
built-in signed and unsigned integer widths except `uintptr`, finite `float32` and
|
|
`float64`, and recursive `[]any`/`map[string]any`. Strings stay strings; duration and
|
|
secret fields therefore receive string values. Nil is explicit null; empty strings,
|
|
false, and zero remain intentional; empty sequences replace whole lists; and mapping
|
|
values recursively overlay existing mappings. Map keys are sorted canonically and
|
|
sequence order is preserved.
|
|
|
|
All records are detached and validated before any is applied. Root and explicit
|
|
sequence-index paths are invalid; parsed numeric tokens remain mapping keys. A path
|
|
may create missing mapping ancestors but cannot traverse an existing scalar, null,
|
|
or sequence. Duplicate and ancestor/descendant-overlapping records are rejected.
|
|
Neither paths nor values may create `configkit`, and no record may overlap the
|
|
configured schema-version path. Unknown fields and effective type mismatches remain
|
|
ordinary strict-decoder errors.
|
|
|
|
At most 1,024 records are accepted, and each record creates one synthetic CLI
|
|
source after file/overlay sources in supplied order. Override sources share the
|
|
1,024-source ceiling. A record has an 8 MiB string-byte limit; override strings and
|
|
source bytes share the 64 MiB aggregate limit. Synthetic mapping keys and values
|
|
count toward the existing 1,000,000-node and depth-256 ceilings. Cycles, named or
|
|
unsupported types, nonfinite floats, and limit excesses fail with safe `override.*`
|
|
diagnostics. Diagnostics and provenance identify the safe `Flag` source, never raw
|
|
arguments or values.
|
|
|
|
Override origins are explicit CLI origins. History retains replaced base/overlay
|
|
definitions and records equal-value replacements as well as structural changes.
|
|
Caller-owned maps and slices are snapshotted during loading and must not be mutated
|
|
concurrently with the call. The executable `ExampleOverride` shows translating a
|
|
parser's explicit-set state into records while preserving an explicitly supplied
|
|
zero.
|
|
|
|
## Explicit imports
|
|
|
|
Imports are opt-in. With `Options.Imports` enabled, a document may reserve its root
|
|
`configkit` mapping for an `imports` sequence. ConfigKit removes that mapping before
|
|
application decoding. Each sequence item is a mapping with exactly one `path`,
|
|
`dir`, or `glob` selector. `path` and `dir` accept a string or a list of strings;
|
|
`glob` accepts one path pattern string. The optional `required` boolean controls
|
|
whether an empty selection or a missing named path is an error. Its default is
|
|
false, so missing imports are absent unless `required: true` is specified. The
|
|
removed `optional` field is invalid. `destination` is an optional sequence of
|
|
strings. Every destination string is one literal mapping key and follows the
|
|
input-destination validation rules above; quote numeric-looking keys so YAML
|
|
reads them as strings. Null, scalar destinations, and non-string members are
|
|
`import.invalid`, as are all other unsupported shapes and fields. A `configkit`
|
|
key below the root is invalid. With imports disabled, any occurrence remains
|
|
`import.disabled`.
|
|
|
|
The directive key is `configkit`, not the `_configkit` spelling used in v0.1.0.
|
|
There is no legacy alias: `_configkit` is ordinary application data, subject to
|
|
normal schema validation, and does not trigger imports. When upgrading, rename
|
|
directive keys in root files and imported fragments, and rename any application
|
|
fields previously named `configkit` to avoid the newly reserved key.
|
|
|
|
Path declarations use slash-separated relative paths. Absolute paths and network
|
|
URLs are invalid. Each path resolves from its declaring file's canonical directory.
|
|
An empty `path: []` selects nothing; it succeeds even when `required: true` because
|
|
there are no named targets to be missing. A nonempty required path list verifies
|
|
every named target, including targets later excluded from selection.
|
|
|
|
Directory declarations search the listed literal directory roots. Missing roots
|
|
contribute no candidates; supplied inaccessible or invalid roots fail. `pattern`
|
|
accepts one basename pattern or a list, matching filenames only. Omission selects
|
|
all filenames; an explicit empty list selects none. `exclude` accepts one basename
|
|
pattern or a list and removes matching final candidates. Patterns use Go's
|
|
case-sensitive `path.Match` grammar. `recursive` enables descendant traversal;
|
|
`depth` is valid only when recursion is effective, with direct files at depth 1 and
|
|
depth 0 or omission meaning unlimited traversal subject to library budgets.
|
|
`follow` controls discovered symlink entries and `hidden` controls discovered
|
|
names beginning with a dot. Both default to true. Explicit roots retain their
|
|
normal access behavior.
|
|
|
|
Glob declarations use the slash-based Go `path.Match` grammar: `*`, `?`, character
|
|
classes, and backslash escapes are supported, while recursive `**` syntax is
|
|
invalid. Backslashes escape pattern characters rather than acting as directory
|
|
separators. ConfigKit validates the complete pattern before filesystem traversal,
|
|
walks only directories selected by preceding segments, and reads wildcard
|
|
directories in bounded batches. Every examined entry consumes the separate visit
|
|
budget, including nonmatches. Literal components are explicit path components;
|
|
`follow` and `hidden` filter only wildcard-discovered entries. `exclude` filters
|
|
final candidate basenames.
|
|
|
|
Acquisition visits declarations in authored order and each declaration's selected
|
|
paths in namespace-path lexical order, preserving source-record and version-callback
|
|
order. Ordinary mapping contributions in a main tier are assembled dependency-first:
|
|
ConfigKit selects the lexically smallest stable source key among the contributions
|
|
whose imports are complete. Independent branches may interleave, and the main body
|
|
may be assembled after its imports. Sequence-item contributions in that tier are
|
|
nodes in the same schedule; each selected item first assembles its own strict
|
|
mapping scope, with helpers before the bodies that import them. Those local scopes
|
|
do not use application collection policies and are not flattened into the outer
|
|
schedule. Item positions and diagnostic paths follow the resulting assembly order.
|
|
Each underlay group likewise acquires its inputs before dependency-ready assembly
|
|
in one strict scope. Independent groups remain separate precedence tiers; a higher
|
|
tier still replaces a lower tier's sequence values wholesale. Assembly order never
|
|
permits replacement.
|
|
|
|
An empty selection is allowed by default. `required: true` makes an empty directory
|
|
or glob selection fail with `import.missing`; it also makes missing named paths
|
|
fail. Requiredness does not suppress invalid paths, containment failures,
|
|
directories selected as files, permissions, parse errors, or composition errors.
|
|
|
|
Each declaration may include `placement: mapping` or
|
|
`placement: sequence-item`. Omission means `mapping` for that declaration,
|
|
including every match of a glob; placement is not inherited. Sequence-item
|
|
placement requires a nonempty effective destination and wraps the imported
|
|
mapping as one element of a sequence there. It does not split source sequences
|
|
or infer identity fields. Repeated sequences still conflict unless their path is
|
|
registered through `Definition.SequenceCompositions`; registration does not
|
|
change placement or precedence behavior. Sequence records retain source
|
|
locations through placement and history; indexes describe their assembled
|
|
positions while source coordinates continue to identify the declaring files.
|
|
Header/version policy and final application validation retain their existing
|
|
owners and timing; sequence registration is a strict-composition rule only.
|
|
|
|
An item import first opens a local mapping scope. Its body and ordinary helper
|
|
imports compose within that record. An omitted helper destination or
|
|
`destination: []` places the helper at the record root; other destinations are
|
|
relative to the immediate importing source. A helper may supply fields such as
|
|
the record's `id`. An explicit nested `sequence-item` import creates a sequence
|
|
inside the record, and its placement is independent of the enclosing item.
|
|
At the record root, an item import must name a nonempty nested destination.
|
|
Physical paths continue to resolve from the file that declares each import.
|
|
An actual empty mapping contributes an empty item, while an unmatched declaration
|
|
with default or explicit `required: false` contributes nothing.
|
|
|
|
For host files, the configuration root defaults to the selected file's canonical
|
|
directory. A host file may supply `FilesystemContext.Root` to widen that root but
|
|
may not supply `BaseDir` or a native `FS`. Byte, reader, and mapping inputs with host context
|
|
must provide both `BaseDir` and `Root` as absolute existing directories.
|
|
For `Filesystem.File`, the default root is `.` in its namespace and the base is
|
|
the selected file's directory. An explicit context must use the same `Filesystem`
|
|
namespace and may set `Root` to contain the file and its base; it may not supply
|
|
`BaseDir`. Byte, reader, and mapping inputs with native context provide both `BaseDir` and
|
|
`Root` as slash-relative `fs.ValidPath` directories in `FS` (`.` means its root),
|
|
with the base inside the root. Context paths are checked even when imports and
|
|
overlays are inactive. Diagnostic labels, including URI-looking labels, never
|
|
imply filesystem paths. Nonregular targets are rejected before opening to avoid blocking on FIFOs;
|
|
post-open regular-file checks remain in place under the stable-input assumption.
|
|
ConfigKit canonicalizes symlinks,
|
|
checks containment with relative paths, and opens the canonical root-relative target
|
|
through a library-owned root handle. Contained relative or absolute symlink targets
|
|
are accepted; escapes are rejected. Canonical paths define identity, so hard links at
|
|
distinct paths remain distinct sources. Native filesystems use slash-path operations
|
|
and their own symlink behavior; imports stay in their selected namespace and root.
|
|
|
|
An active-stack repeat is `import.cycle` and reports the complete cycle's import
|
|
sites. A previously visited canonical path is `import.repeated` and reports both
|
|
inclusion routes, including diamonds and aliases. Each accepted file is read once.
|
|
Imported fragments may contain their own root directives, but may not claim the
|
|
application's schema-version path or replace one of its ancestors; the version must
|
|
come from the selected source. No directory is scanned implicitly, including one
|
|
named `conf.d`, and ConfigKit performs no remote fetching.
|
|
|
|
## YAML documents
|
|
|
|
The private YAML boundary currently accepts exactly one nonempty document whose root
|
|
is a mapping. Comments and block or flow collections are supported. It rejects:
|
|
|
|
- empty input or an empty document;
|
|
- every additional document, including an empty one;
|
|
- duplicate decoded mapping keys;
|
|
- non-string mapping keys;
|
|
- explicit tags, anchors, aliases, and merge keys;
|
|
- a quoted or plain mapping key equal to `<<`; and
|
|
- input exceeding the fixed depth or node limits.
|
|
|
|
Mapping key comparison is exact: it performs no case folding or Unicode
|
|
normalization. The private document tree retains declaration order, exact validated
|
|
number spelling, source identifiers, and separate key/value line and column
|
|
locations. It contains no upstream parser nodes or types.
|
|
|
|
Root depth is zero. Input depth is limited to 256 and each parsed input tree to
|
|
1,000,000 nodes, counting mapping keys as input nodes. The upstream parser also has
|
|
a private calibrated depth guard; passing that guard never bypasses ConfigKit's tree
|
|
limits.
|
|
|
|
## Scalars
|
|
|
|
Scalar meaning is determined by ConfigKit rather than upstream coercion:
|
|
|
|
| Input form | Meaning |
|
|
| --- | --- |
|
|
| Quoted, literal, or folded scalar | String |
|
|
| Plain `true` or `false` | Boolean |
|
|
| Plain `null`, `~`, or an empty mapping value | Null |
|
|
| JSON decimal number | Number, retaining its exact spelling |
|
|
| Date/time-looking or other ordinary plain text | String |
|
|
|
|
JSON numbers permit an optional minus, a zero or nonzero decimal integer without a
|
|
leading zero, an optional fraction with at least one digit, and an optional decimal
|
|
exponent. Hexadecimal, octal, digit separators, leading plus, leading/trailing-dot
|
|
forms, incomplete exponents, and non-finite values are rejected. Legacy boolean
|
|
words (`yes`, `no`, `on`, and `off`) and non-lowercase variants of the reserved
|
|
boolean/null words are rejected unless quoted.
|
|
|
|
Mapping keys use the same scalar classification and must classify as strings. Quote
|
|
keys that otherwise classify as a boolean, number, or null.
|
|
|
|
## Application schema binding
|
|
|
|
ConfigKit builds a private reflection descriptor graph for each schema operation;
|
|
it keeps no global mutable descriptor cache. A configuration root must be a struct
|
|
or string-keyed map. Nested supported shapes are:
|
|
|
|
- booleans, strings, signed and unsigned integers, and finite floating-point types;
|
|
- `time.Duration`, typed `Secret` references, and paired
|
|
`encoding.TextMarshaler`/`TextUnmarshaler` scalars;
|
|
- pointers, structs, string-keyed maps, slices, fixed arrays, and `any`; and
|
|
- recursive type definitions whose eventual application values are acyclic.
|
|
|
|
Named scalar types retain their Go identity. `uintptr`, complex numbers, functions,
|
|
channels, unsafe pointers, non-string map keys, and nonempty interfaces are not
|
|
configuration types. A custom text scalar must provide both directions on its value
|
|
or pointer method set. Building a descriptor only inspects method sets; it never
|
|
executes application methods or parser-specific hooks.
|
|
|
|
Exported struct fields use an explicit `yaml` name or the lowercase Go field name.
|
|
Unexported fields and `yaml:"-"` are ignored. `omitempty` is accepted but does not
|
|
change binding or projection. An explicit `yaml:",inline"` flattens a struct
|
|
or pointer-to-struct, or designates at most one string-keyed inline map for otherwise
|
|
unmatched names. Ambiguous names, named inline fields, multiple inline maps, and
|
|
unsupported or malformed tag options are schema errors. Inline pointers must
|
|
terminate in a struct; pointer-to-map and recursive pointer-only inline declarations
|
|
are rejected before the defaults factory runs. Embedded fields are ordinary
|
|
named fields unless explicitly inlined. `configkit:"secret"` is the only current
|
|
ConfigKit field tag and marks its bound field or container for secret classification
|
|
and redacted inspection.
|
|
|
|
Schema path traversal uses these same bindings. Explicit key segments address
|
|
mappings, explicit index segments address sequences, and parsed numeric pointer
|
|
tokens become indices only in a sequence. Sequence indices must use canonical
|
|
nonnegative decimal spelling: no signs or leading zeroes except `0` itself.
|
|
Tokens such as `+0` and `-0` remain literal keys in mappings but do not identify
|
|
sequence elements. Fixed-array indices are bounds checked. Dynamic `any` and inline-map entries
|
|
remain schema-valid for application-defined descendants, but dynamic values cannot
|
|
introduce typed secret references.
|
|
|
|
Descriptors are reused by typed decoding, projection, and cloning so those operations
|
|
cannot acquire different field bindings.
|
|
|
|
### Canonical module options
|
|
|
|
Applications with a known module catalog represent each configured item as an object
|
|
with an explicit module identifier and an options mapping. Ordinary typed structs,
|
|
including a small union of optional option fields, use the same strict binding above;
|
|
application validation decides which fields and combinations apply to each module.
|
|
String shorthand, a library module registry, and custom structural decoder hooks are
|
|
not supported alternatives.
|
|
|
|
When a runtime-selected catalog genuinely requires dynamic options,
|
|
`map[string]any` retains ConfigKit's canonical dynamic scalars. In particular,
|
|
integers are `int64`; application validation owns allowed keys, value constraints,
|
|
range checks, and conversion to runtime `int` or catalog objects. ConfigKit does not
|
|
provide generic subtree conversion, and consumers must not add a YAML marshal/decode
|
|
round trip to exercise supported behavior. The
|
|
[consumer adoption guide](consumers/README.md) contains compiled typed and dynamic
|
|
examples.
|
|
|
|
## Defaults and typed decoding
|
|
|
|
The private decoder starts from an independently owned clone of the application's
|
|
typed defaults and applies only explicitly present input nodes. Defaults therefore
|
|
never become explicit, while equal-valued input remains explicit. Successful decode
|
|
produces detached typed state, baseline and final canonical trees, an input-presence
|
|
index, and default/input origins for the later public loading lifecycle.
|
|
|
|
| Configured input | Effect on the typed baseline |
|
|
| --- | --- |
|
|
| Absent field | Retain the default |
|
|
| Explicit false, zero, or empty string | Replace the default |
|
|
| Partial mapping or string-keyed map | Recurse and retain omitted default children or entries |
|
|
| Empty mapping | Retain existing mapping children |
|
|
| Sequence, including an empty sequence | Replace the entire sequence |
|
|
| Pointer object | Recurse into a non-nil default; otherwise allocate an object whose omitted children have ordinary Go zero values |
|
|
| Allowed null | Replace the value with nil and suppress its default |
|
|
|
|
Null is allowed only for pointers, maps, slices, and dynamic `any` values. It is an
|
|
error for scalars, structs, and arrays. New map entries and sequence elements begin
|
|
from their Go zero values; ConfigKit does not invent entry-level defaults.
|
|
|
|
Decoding is strict at every typed depth. Unknown struct fields, type mismatches,
|
|
invalid nulls, numeric overflow, and arrays with the wrong length are errors at the
|
|
original semantic path and source coordinates. String-keyed maps accept arbitrary
|
|
keys but preserve the strict schema of their values; inline maps accept unmatched
|
|
keys according to their element descriptor. Multiple independent field failures are
|
|
collected without publishing the partially decoded object.
|
|
|
|
Booleans and strings are not generally coerced. At exactly a configured
|
|
`VersionRule.Path`, strict decimal-integer input may decode into a plain `string`, a
|
|
named string without a text codec, or pointer wrappers around those types. The
|
|
exception is load-local and does not apply to unrelated fields or loads without a
|
|
version rule. Integer destinations require integer syntax
|
|
and a value in the destination type's range. Float destinations accept finite JSON
|
|
numbers representable by their destination precision. Dynamic numbers become
|
|
`int64` for integer syntax and `float64` for fractional or exponent syntax, with
|
|
range and finiteness checks. Named scalar types retain their identity. Durations
|
|
decode only from strings accepted by `time.ParseDuration`; integers are rejected.
|
|
Paired text scalars decode only from strings through `encoding.TextUnmarshaler`, and
|
|
codec failures are classified without exposing application error text or input
|
|
values.
|
|
|
|
This decoder is available only through `Load`; ConfigKit exposes no generic document
|
|
conversion API.
|
|
|
|
## Typed projection and ownership
|
|
|
|
ConfigKit can internally project a schema-valid application value directly into its
|
|
owned canonical tree without YAML serialization. Projection includes every bound
|
|
field, including ordinary zero values; `omitempty` has no effect. Struct fields keep
|
|
binding order, dynamic map keys sort by decoded string, and sequences keep their
|
|
order. Integers retain exact base-10 values, finite floats use their destination
|
|
precision, durations use `time.Duration.String`, and paired text scalars use their
|
|
canonical marshaled text.
|
|
|
|
Nil pointers, maps, slices, and dynamic values project as null. Non-nil empty maps
|
|
and slices remain empty containers, preserving the distinction. Recursive acyclic
|
|
objects are supported. Cyclic values, non-finite floats, unsupported dynamic concrete
|
|
types, invalid codec output, and depth/node-limit violations fail with safe schema
|
|
classifications that omit codec error text.
|
|
|
|
The same descriptor graph creates detached typed copies. Pointers, maps, slices,
|
|
arrays, nested dynamic values, and shared references are recursively copied; shared
|
|
pointer identity is not preserved as configuration meaning. Paired text scalars are
|
|
rebuilt by unmarshaling their canonical text into fresh values, detaching private
|
|
backing storage. Ignored and unexported struct fields are zeroed rather than copied.
|
|
|
|
Private structural comparison operates on canonical trees. Mapping additions,
|
|
removals, and descendant replacements retain semantic paths. A sequence is atomic:
|
|
any sequence change reports the sequence path and never invents correspondence
|
|
between old and new indices. Cross-kind and null/container changes replace the
|
|
affected path.
|
|
|
|
These mechanisms support decoding, resolution auditing, metadata,
|
|
and rendering. They are not a public generic conversion or cloning API.
|
|
|
|
## Safe failures
|
|
|
|
Source failures use stable `source.*` codes and YAML failures use stable `yaml.*`
|
|
codes declared by the root package. Diagnostics contain safe classifications,
|
|
semantic paths when applicable, and available source coordinates. Raw parser,
|
|
conversion, reader-error, and input text is not forwarded. Source labels and
|
|
diagnostic messages are application-visible and therefore must never contain
|
|
secrets; terminal control characters are escaped by `WriteDiagnostics`.
|