703 lines
41 KiB
Markdown
703 lines
41 KiB
Markdown
# Configuration Input Contract
|
|
|
|
This document owns ConfigKit's currently implemented source-input, YAML syntax,
|
|
schema, typed-decoding, loading, metadata, redaction, rendering, and ownership
|
|
contracts. The parser, owned document representation, and decoder remain private
|
|
behind the public loading lifecycle.
|
|
|
|
## Sources
|
|
|
|
ConfigKit exposes opaque sources through these constructors:
|
|
|
|
| Constructor | Current contract |
|
|
| --- | --- |
|
|
| `File(path)` | Requires a nonempty path, opens a regular file for each consumption, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. |
|
|
| `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. |
|
|
| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. |
|
|
| `NoSource()` | Selects defaults-only loading, creates no source record, and marks no paths explicit. |
|
|
|
|
The zero `Source` is invalid. Constructor argument failures are retained inside the
|
|
opaque source and become safe structured diagnostics when the source is consumed.
|
|
File sources can observe file changes between consumptions; byte sources cannot.
|
|
|
|
Each source is limited to 8 MiB. A load using imports or an active profile is limited
|
|
to 64 MiB across all accepted sources, 1,024 accepted sources, 64 active import
|
|
edges, and 1,000,000 input nodes in aggregate. Glob expansion separately permits
|
|
1,000,000 visited directory entries, including nonmatches. Reads request at most the
|
|
remaining allowance plus one byte so excess data is rejected rather than silently
|
|
truncated. Data returned together with a read error is not accepted as partial input.
|
|
|
|
## Single-source loading
|
|
|
|
`Load[T](source, options...)` accepts zero or one `Options[T]`. The configuration
|
|
root `T` must satisfy the application schema contract below. With no options it
|
|
starts from the Go zero value. A `Definition[T].Defaults` factory, when supplied,
|
|
runs exactly once after successful source parsing, schema construction, optional
|
|
schema-version checking, and selected-profile loading. Its result is detached before
|
|
input is applied. Resolver-provided mutable values are detached before secret lookup
|
|
and validation, and a final detached copy prevents equal-valued aliases introduced
|
|
by validators from escaping into the returned configuration. The publication copy
|
|
preserves already validated secret state without rereading the environment.
|
|
|
|
The current lifecycle is:
|
|
|
|
```text
|
|
validate options → parse the selected source → traverse/compose enabled imports
|
|
→ build schema → check the optional application version → apply a selected profile
|
|
→ apply explicit CLI overrides → create and clone defaults → strictly decode
|
|
→ recorded application resolution
|
|
→ resolve typed secret references → semantic validation → classify secrets
|
|
→ snapshot configuration and metadata
|
|
```
|
|
|
|
`File`, `Bytes`, and `Reader` each contribute the first base source with deterministic
|
|
ID `source-1`; enabled imports continue in depth-first order as `source-2` onward.
|
|
`NoSource` skips parsing, supplies an absent version to a configured version checker,
|
|
contributes no source, and continues through defaults, decode, and projection. A
|
|
missing file remains an error; it is never interpreted as no input.
|
|
|
|
The current `Options` boundary accepts a defaults factory, `VersionRule`, resolver,
|
|
semantic validator, explicit path and glob imports, one selected profile, and
|
|
filesystem context. It also accepts explicit CLI override records, static secret
|
|
tags, and a dynamic path predicate. `NoSource` rejects imports, active profiles,
|
|
and filesystem context as invalid combinations but accepts overrides. Input
|
|
containing a reserved `_configkit` mapping key is rejected while imports are
|
|
disabled.
|
|
|
|
Every source, options, schema, version, defaults, decode, resolution, or validation
|
|
failure returns a nil result and a `*LoadError`. Successful results contain caller-owned `Config`, detached
|
|
`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Mutating
|
|
the returned configuration or inspection records cannot alter ConfigKit's private
|
|
snapshot. ConfigKit performs no logging. `Result.WriteEffective` exposes only the
|
|
frozen redacted projection.
|
|
|
|
### Application schema versions
|
|
|
|
`VersionRule` declares a semantic path and non-nil checker. Before defaults or typed
|
|
decoding, the checker receives a `Version` preserving absent, explicit null, or
|
|
string input. Any other input kind is a `version.type` error and the checker does not
|
|
run. ConfigKit does not interpret the text or impose a universal version path.
|
|
|
|
The checker reports safe, value-free errors and warnings through `Validation.Error`
|
|
and `Validation.Warning`; ConfigKit resolves primary and related paths to exact or
|
|
nearest input locations. Reported errors fail loading. If the checker returns an
|
|
error without reporting one, ConfigKit emits a generic `version.hook_failed`
|
|
diagnostic and suppresses the returned error text. Warnings alone are retained on a
|
|
successful result. Callback panics remain programming failures.
|
|
|
|
### Layered metadata
|
|
|
|
`Result.Metadata` describes every final canonical path. Exact surviving input paths
|
|
are explicit and have a base or profile origin; omitted paths have a default origin.
|
|
An explicit parent mapping does not make omitted children explicit. Null state and
|
|
available source coordinates are retained. `History` records the Go-zero or application
|
|
default baseline followed by exact base or profile definitions, including
|
|
equal-valued input. Profile replacement removes displaced descendants from effective
|
|
metadata while retaining their base definition and the parent replacement operation
|
|
in history. All metadata queries and returned nested records are detached.
|
|
|
|
History is chronological: a path can have a synthetic default record, its exact base
|
|
definition, a profile replacement, and a recorded resolution operation. A descendant
|
|
query uses the ancestor's path for a whole-container or whole-sequence replacement;
|
|
the descendant can therefore retain history after `Lookup` stops reporting it as
|
|
effective. Equal-valued input is still an explicit base definition, while a derived
|
|
value for absent input remains non-explicit.
|
|
|
|
A mapping assembled from the selected file and imports has one composite base origin
|
|
whose locations retain source traversal order. A mapping also visited by a profile
|
|
has a profile-layer composite origin with the lower contributing locations followed
|
|
by the profile location; an empty mapping is still an explicit container even though
|
|
it contributes no children. Resolution origins are synthetic: they have layer
|
|
`resolution`, no fabricated file location, and a `derived_default` or `normalization`
|
|
operation. Reasons and dependency paths retain first occurrence order with duplicates
|
|
removed. Source records remain ordered as selected source, depth-first imports, the
|
|
selected profile, then synthetic CLI records in supplied order.
|
|
|
|
### Secret classification and effective rendering
|
|
|
|
Secret classification combines `configkit:"secret"` field tags with the optional
|
|
pure `Definition.Secret(Path) bool` predicate. Tags on concrete structs inside
|
|
supported dynamic values are retained in typed projections, including superseded
|
|
defaults and pre-resolution values. A tagged or predicate-classified
|
|
container makes all descendants sensitive. ConfigKit evaluates final and historical
|
|
paths, so replacing or deleting a definition does not erase its classification. The
|
|
predicate can add sensitivity but cannot declassify a tagged path.
|
|
|
|
Recorded resolution dependencies propagate sensitivity to their derived target.
|
|
Propagation reaches a fixed point, so record/map iteration order does not affect the
|
|
result and dependency cycles terminate. A dependency on a classified descendant or
|
|
on a container that contains one is treated conservatively as sensitive. Applications
|
|
must classify secret-producing transformations whose dependencies are not recorded.
|
|
|
|
`Result.WriteEffective` writes the frozen resolved snapshot as deterministic redacted
|
|
YAML. Mappings sort by decoded key string, sequences retain order, indentation is two
|
|
spaces with noncompact sequence indentation, line endings are LF, the document has no
|
|
start or end marker, and output ends with a newline. Nil pointers/maps/slices remain
|
|
`null`; non-nil empty maps and sequences remain `{}` and `[]`. A secret root is
|
|
replaced by the quoted string `'[REDACTED]'`. Redacted output is intended for
|
|
inspection and is not promised to be reloadable.
|
|
|
|
Rendering uses only a safe adapter tree built from the already completed canonical
|
|
typed projection. It does not invoke application YAML marshal hooks or text codecs;
|
|
all text-codec projection has succeeded before `Load` returns. Caller mutation of
|
|
`Result.Config` cannot change later output. Writer failures, including short writes,
|
|
are returned and may leave partial redacted output; internal dumper failures are
|
|
translated to a fixed safe error. There is no unredacted rendering option and
|
|
ConfigKit performs no implicit logging.
|
|
|
|
`Result.Config` is the explicit trusted boundary and contains ordinary typed values,
|
|
including secrets. Metadata records, diagnostics, and load errors contain no
|
|
configuration values. Explanation snapshots deliberately expose non-secret
|
|
configuration history after applying the redaction contract below. Applications
|
|
must still keep source labels, resolution reasons, and diagnostic messages value-free
|
|
because ConfigKit cannot infer and sanitize secrets embedded in arbitrary application
|
|
prose.
|
|
|
|
### Historical-value explanation
|
|
|
|
`Result.Explain(path)` returns a detached `Explanation` for any known effective or
|
|
historical path. Its effective value and each chronological before/after value are
|
|
`ValueSnapshot` records containing deterministic YAML text rather than mutable Go or
|
|
parser values. `Present=false` with empty YAML means absence or deletion; explicit
|
|
null is present and renders as `null`. The history starts with the canonical typed
|
|
application-default or Go-zero baseline, then includes only relevant base sources in
|
|
traversal order, the selected profile, supplied CLI records, and audited net
|
|
resolution operations. Overlapping parent/child resolution records are rendered
|
|
as one net subtree event with combined reasons and dependencies, rather than
|
|
pretending that the recorded assignments ran in lexical order. Explicit equal
|
|
writes remain visible. When input replay
|
|
differs from the final typed input, a `ChangeDecoding` event records the transition
|
|
before domain resolution. Its origin retains the decoded path's source/default
|
|
attribution; its reason identifies static-default retention and canonical decoding.
|
|
This built-in inspection operation is not accepted by `Resolution.Record`.
|
|
|
|
Base, profile, and CLI history preserves the policy-validated input representation,
|
|
including a superseded value that could not have decoded into the final Go field.
|
|
Baseline, effective, and resolution values use canonical typed projection. A
|
|
decoding event bridges replayed input and canonical typed values, so intermediate
|
|
null/scalar replacements cannot incorrectly imply permanent default removal. Mapping
|
|
queries reconstruct each source or override contribution, while sequences remain
|
|
atomic: a descendant history identifies its presence before and after the containing
|
|
sequence replacement without implying element-wise merging. Removed paths remain
|
|
explainable; unknown and malformed paths return false.
|
|
|
|
Every snapshot applies the same historical tag, predicate, and recorded-dependency
|
|
sensitivity closure as effective rendering before YAML is exposed. A redacted scalar
|
|
is the quoted marker `'[REDACTED]'`, and a container snapshot has `Redacted=true` if
|
|
it contains a masked descendant. Typed `Secret` snapshots contain only the configured
|
|
environment name. `SecretReference` reports that name only for a successfully
|
|
resolved effective reference, and is omitted when its path is classified. Credential
|
|
values are never retained in explanation history.
|
|
|
|
`Result.WriteExplanation(output, path)` renders the safe structured result with the
|
|
canonical JSON-pointer path, canonical-versus-input representation labels, effective
|
|
source, composite locations, chronological values, resolution reasons and
|
|
dependencies, and successful secret-reference lookup attribution. Control characters
|
|
in paths, source identifiers and labels, reasons, dependencies, and reference names
|
|
are escaped. An unknown path returns a fixed safe error before writing anything;
|
|
writer errors and short writes are returned and may leave partial output. There is no
|
|
unredacted mode.
|
|
|
|
Retained history consists of immutable source applications and audited subtree
|
|
deltas, rather than a full tree copy per field. Shared nodes count once toward a
|
|
4,000,000-node retained-history limit; exceeding it fails loading with
|
|
`explain.limit_exceeded`. Replaying one requested path is performed on demand, and
|
|
the returned YAML strings and nested records cannot mutate later explanations.
|
|
|
|
Application commands should render the separate `SecretsReport` immediately after
|
|
an explicit `LoadSecretsDir` bootstrap, then use `WriteDiagnostics`,
|
|
`WriteEffective`, or `WriteExplanation` on the later load result as appropriate.
|
|
ConfigKit does not infer bootstrap outcomes from a secret reference and does not
|
|
bundle these independent reports into a command framework.
|
|
|
|
## Typed secret references
|
|
|
|
Credential-bearing fields use the opaque `Secret` type and store an environment
|
|
variable name in YAML, such as `api_key: OPENROUTER_API_KEY`. `SecretEnv(name)`
|
|
constructs the same unresolved reference for application defaults or recorded
|
|
resolution. Names must match `[A-Za-z_][A-Za-z0-9_]*`; they are public
|
|
configuration and remain visible through `EnvName`, canonical projection, history,
|
|
and effective output unless the path is separately classified for redaction.
|
|
|
|
A non-pointer `Secret` is required. An absent, null, or invalid reference, or an
|
|
unset or empty environment value, fails with a safe `secret.*` diagnostic at the
|
|
configuration path. `*Secret` is optional only when nil; every non-nil reference
|
|
must resolve to a nonempty value. Secret references work in typed structs, maps,
|
|
pointers, slices, and arrays. A dynamic `any` value does not establish secret
|
|
semantics and cannot contain `Secret`; applications must use a typed slot.
|
|
|
|
ConfigKit resolves only the effective names after recorded application resolution
|
|
and before semantic validation. Each unique name is read with `os.LookupEnv` once
|
|
per load. Validation can inspect the credential through the deliberately named
|
|
`Value()` accessor and distinguish success with `IsResolved()`. Later loads read
|
|
the environment again, and cloned defaults are rebuilt from names so a previously
|
|
resolved result cannot retain a stale credential. `Load` never writes to the
|
|
environment.
|
|
|
|
Credential bytes are held only in the per-load lookup state and the trusted returned
|
|
configuration, and are returned only by `Value`. Secret formatting uses a safe
|
|
public-name representation, JSON marshals only the name, and projection, metadata,
|
|
histories, diagnostics, effective output,
|
|
and resolution snapshots never retain the credential. Validation remains read-only:
|
|
in addition to ordinary projected-value comparison, ConfigKit checks the private
|
|
resolved state against this load's lookup result. Applications must keep their
|
|
validation messages value-free and must not include `Value()` in diagnostics.
|
|
|
|
For example, a typed pipeline field is declared as:
|
|
|
|
```go
|
|
type PipelineConfig struct {
|
|
APIKey configkit.Secret `yaml:"api_key"`
|
|
}
|
|
```
|
|
|
|
The executable `ExampleSecretEnv` demonstrates loading and inspecting the safe
|
|
reference state without printing its value.
|
|
|
|
## Explicit secrets-directory bootstrap
|
|
|
|
`LoadSecretsDir(directory)` is a separate, opt-in startup operation. It is never
|
|
called by `Load`, is not a YAML option, and does not participate in configuration
|
|
validation. Applications choose the native directory path and should invoke the
|
|
operation once before starting concurrent work. The selected path may itself be a
|
|
symlink; ConfigKit resolves it to an existing canonical directory before examining
|
|
its immediate entries.
|
|
|
|
The bootstrap enumerates at most 1,024 entries and sorts them by filename. Every
|
|
entry must have an ASCII environment-variable name and be a regular, nonsymlink
|
|
file. Metadata for all entries is validated before file content is read or process
|
|
state is changed. Existing environment variables win, including variables whose
|
|
value is empty; their files are not opened. Each eligible file is limited to 1 MiB,
|
|
and eligible content is limited to 8 MiB in total. ConfigKit removes exactly one
|
|
terminal LF or CRLF, preserves all other whitespace, accepts an empty value, and
|
|
rejects NUL bytes.
|
|
|
|
All eligible files are read successfully before application begins. Presence is
|
|
checked again immediately before each `os.Setenv`: a variable populated since
|
|
preflight is skipped. A write failure stops application without rollback, so prior
|
|
writes remain and later eligible entries are `not_applied`. This operation is not
|
|
atomic and does not provide compare-and-set semantics for concurrent environment
|
|
mutation.
|
|
|
|
`SecretsReport.Entries` returns detached, filename-ordered `SecretFileResult`
|
|
records with `loaded`, `skipped_existing`, `not_applied`, or `failed` status.
|
|
`ExistingEmpty` distinguishes an empty existing winner. `SecretsReport.WriteTo`
|
|
renders only quoted names, canonical paths, statuses, and that empty-state flag;
|
|
it never retains or renders file contents. Failures return the non-nil partial
|
|
report with a `*SecretsError` containing detached `secrets.*` diagnostics and no
|
|
raw filesystem or environment error chain. Applications should retain the report
|
|
alongside the later configuration result and explicitly present blocked and failed
|
|
files when appropriate; ConfigKit stores no implicit global report.
|
|
|
|
### Recorded application resolution
|
|
|
|
An optional `Definition.Resolve` callback runs after successful typed decoding. It
|
|
receives the working configuration and a per-load `Resolution` context. The context's
|
|
`IsExplicit` query uses surviving input presence, so explicit false, zero, empty, and
|
|
null are never confused with absence or static defaults.
|
|
|
|
`Default` assigns and records a derived value only when its path is not explicit.
|
|
`Normalize` always assigns and records. For transformations performed directly by
|
|
application code, `Resolution.Record` declares a derived-default or normalization
|
|
operation; a recorded container covers changes to its descendants. Records require
|
|
a schema-valid path, valid kind, safe nonempty reason, and schema-valid dependency
|
|
paths. Repeated records at one path are coalesced: reasons and dependencies retain
|
|
their first occurrence, the last helper value is authoritative, and any normalization
|
|
makes the combined operation a normalization.
|
|
|
|
ConfigKit projects the configuration before and after the callback. Every net change
|
|
must have an exact or ancestor record. Helper-assigned values must agree with the
|
|
final value at their declared path, which supports ordinary map-entry copy/writeback
|
|
while detecting forgotten writeback or unrelated targets. Invalid paths, kinds,
|
|
values, records, helper targets, and unrecorded mutations fail the load. Equal-valued
|
|
derived assignments may still record application intent even when there is no
|
|
structural change.
|
|
|
|
`Resolution.Error` and `Resolution.Warning` attach exact or nearest available input
|
|
locations. Reported errors fail the load; warnings survive on a successful result. A
|
|
returned resolver error adds one generic `resolve.hook_failed` error only when the
|
|
callback did not report an error, and its text is never exposed. Callback panics are
|
|
programming failures. Resolution and validation contexts are callback-scoped; using
|
|
a nil, zero, or retained context outside its active callback panics with fixed,
|
|
value-free text.
|
|
|
|
Successful records update effective origins and append detached metadata history
|
|
without changing input explicitness. Removed paths leave effective metadata but
|
|
retain history. ConfigKit privately snapshots each recorded before/after subtree for
|
|
later safe explanation; caller mutations of `Result.Config` cannot alter those
|
|
snapshots.
|
|
|
|
### Semantic validation
|
|
|
|
An optional `Definition.Validate` callback runs after successful resolution and
|
|
receives the resolved configuration plus a per-load `Validation` context. It can
|
|
collect multiple independent `validate.error` and `validate.warning` reports.
|
|
Warnings continue the lifecycle and are returned on success; any error returns a nil
|
|
result with the full safe diagnostic and source report.
|
|
|
|
Primary and related paths must be valid for the application schema. Omitted declared
|
|
fields and missing string-map or dynamic keys are valid report targets; unknown
|
|
struct fields and malformed paths produce `validate.invalid_path`. ConfigKit attaches
|
|
the exact surviving input location when one exists, otherwise the nearest input
|
|
container location, or no primary location for defaults-only synthetic state.
|
|
|
|
Validation is read-only. ConfigKit projects the configuration immediately before and
|
|
after the callback and returns `validate.mutation` at every detected net-change path,
|
|
including nested mutable state. A value that can no longer be projected is also a
|
|
mutation failure. No mutated or otherwise invalid configuration is published.
|
|
Callback panics remain programming failures, and retained contexts use the same
|
|
fixed closed-context panic as version and resolution callbacks.
|
|
|
|
Only schema-bound exported state participates in loading, resolution audit,
|
|
validation audit, metadata, and snapshots. Ignored or unexported ordinary fields are
|
|
zero in returned detached values. Applications retain responsibility for domain
|
|
rules and runtime initialization; validators should not start clients or depend on
|
|
live services as a substitute for configuration validation.
|
|
|
|
## Strict document composition
|
|
|
|
ConfigKit strictly composes parsed base documents when `Options.Imports` is true.
|
|
Application defaults do not participate in this operation.
|
|
|
|
| Existing node | Incoming node | Result |
|
|
| --- | --- | --- |
|
|
| Absent | Absent | Absent |
|
|
| Absent | Present | Retain the incoming node |
|
|
| Present | Absent | Retain the existing node |
|
|
| Mapping | Mapping | Recurse by mapping key |
|
|
| Any other pair of present nodes | Any other pair of present nodes | `compose.conflict` |
|
|
|
|
Every non-mapping overlap is a conflict, even when values are equal. Nulls and
|
|
sequences are present atomic values, so they also conflict when repeated; sequences
|
|
are never joined by index. Disjoint entries under shared mappings compose, making
|
|
stable keyed mappings the natural representation for independently contributed
|
|
entries.
|
|
|
|
An empty mapping contributes no child definitions. Two empty mappings compose, and
|
|
an empty mapping neither clears nor conflicts with another mapping's children.
|
|
Shared mapping containers retain all contributing value locations as their composite
|
|
base origin. A conflict reports the first claim as its primary location and every
|
|
remaining claim as related locations, in deterministic source and declaration
|
|
traversal order. Traversal order never authorizes replacement. Composition clones
|
|
the resulting tree and its ordered claim records rather than mutating or aliasing an
|
|
input document.
|
|
|
|
## Selected profiles
|
|
|
|
`Options.Profile` selects at most one overlay by name. Names must match
|
|
`[a-zA-Z0-9_-]+`; they are not paths. The empty name and exact lowercase name
|
|
`default` select no profile, and `profiles/default.yml` is never loaded implicitly.
|
|
Every other name resolves only to `<configuration-root>/profiles/<name>.yml`, with
|
|
no extension fallback, case normalization, inheritance, or profile stack. A selected
|
|
missing profile is an error.
|
|
|
|
The profile is a separately policy-validated YAML mapping. It is loaded after the
|
|
base import graph is composed and its schema version is checked, but before defaults
|
|
and typed decoding. Profile bytes, nodes, and source records count toward the same
|
|
aggregate limits as base inputs. Its canonical path must remain within the configured
|
|
root and must not duplicate a file already present in the base graph.
|
|
|
|
Overlay is intentionally distinct from strict composition:
|
|
|
|
| Existing lower node | Incoming profile node | Result |
|
|
| --- | --- | --- |
|
|
| Absent | Value | Retain the profile value |
|
|
| Value | Absent | Retain the lower value |
|
|
| Mapping | Mapping | Recurse by mapping key |
|
|
| Any other pair of present nodes | Any other pair of present nodes | Replace the entire lower node |
|
|
|
|
Scalars, sequences, and null are atomic. Thus a profile sequence replaces rather
|
|
than extends a base sequence, `[]` clears it, null is an explicit replacement, and
|
|
cross-kind replacement is allowed when the final application schema accepts it.
|
|
Replacing a subtree removes all displaced descendants from effective presence and
|
|
origin data; history retains the displaced definitions and identifies the parent
|
|
replacement. Surviving profile paths have profile origins, while recursively shared
|
|
mapping containers retain their contributing locations.
|
|
|
|
A profile may not contain `_configkit` anywhere, contribute the configured schema
|
|
version, or replace an ancestor of that version with a scalar or null. These rules
|
|
apply even if the supplied version text would be unchanged. Profiles cannot mask a
|
|
base parse, composition, or version failure because those checks run first.
|
|
|
|
## Explicit CLI overrides
|
|
|
|
`Options.Overrides` accepts only records the application knows were explicitly set
|
|
by its CLI parser. Each `Override` supplies a non-root mapping `Path`, a detached
|
|
`Value`, and a required safe display `Flag` such as `--workers`; ConfigKit has no CLI
|
|
parser dependency. Parser defaults must not be submitted as overrides, and `Flag`
|
|
must never contain the raw argument or configured value. General environment
|
|
variable overlays are an explicit non-goal: environment access is reserved for
|
|
typed secret references.
|
|
|
|
Overrides apply after the selected profile and before typed defaults/decoding, so
|
|
the input precedence is application defaults < composed base < selected profile <
|
|
CLI. Recorded resolution, typed secret lookup, and semantic validation follow the
|
|
effective override tree. Overrides also work with `NoSource`.
|
|
|
|
Values are converted directly into ConfigKit's owned tree without YAML parsing or
|
|
string coercion. Supported values are `nil`, built-in strings and booleans, all
|
|
built-in signed and unsigned integer widths except `uintptr`, finite `float32` and
|
|
`float64`, and recursive `[]any`/`map[string]any`. Strings stay strings; duration and
|
|
secret fields therefore receive string values. Nil is explicit null; empty strings,
|
|
false, and zero remain intentional; empty sequences replace whole lists; and mapping
|
|
values recursively overlay existing mappings. Map keys are sorted canonically and
|
|
sequence order is preserved.
|
|
|
|
All records are detached and validated before any is applied. Root and explicit
|
|
sequence-index paths are invalid; parsed numeric tokens remain mapping keys. A path
|
|
may create missing mapping ancestors but cannot traverse an existing scalar, null,
|
|
or sequence. Duplicate and ancestor/descendant-overlapping records are rejected.
|
|
Neither paths nor values may create `_configkit`, and no record may overlap the
|
|
configured schema-version path. Unknown fields and effective type mismatches remain
|
|
ordinary strict-decoder errors.
|
|
|
|
At most 1,024 records are accepted, and each record creates one synthetic CLI
|
|
source after file/profile sources in supplied order. Override sources share the
|
|
1,024-source ceiling. A record has an 8 MiB string-byte limit; override strings and
|
|
source bytes share the 64 MiB aggregate limit. Synthetic mapping keys and values
|
|
count toward the existing 1,000,000-node and depth-256 ceilings. Cycles, named or
|
|
unsupported types, nonfinite floats, and limit excesses fail with safe `override.*`
|
|
diagnostics. Diagnostics and provenance identify the safe `Flag` source, never raw
|
|
arguments or values.
|
|
|
|
Override origins are explicit CLI origins. History retains replaced base/profile
|
|
definitions and records equal-value replacements as well as structural changes.
|
|
Caller-owned maps and slices are snapshotted during loading and must not be mutated
|
|
concurrently with the call. The executable `ExampleOverride` shows translating a
|
|
parser's explicit-set state into records while preserving an explicitly supplied
|
|
zero.
|
|
|
|
## Explicit imports
|
|
|
|
Imports are opt-in. With `Options.Imports` enabled, a document may reserve its root
|
|
`_configkit` mapping for an `imports` sequence. ConfigKit removes that mapping before
|
|
application decoding. Each sequence item is a mapping with exactly one nonempty
|
|
string `path` or `glob` plus an optional boolean `optional`; all other shapes and
|
|
fields are `import.invalid`. A `_configkit` key below the root is invalid. With
|
|
imports disabled, any occurrence remains `import.disabled`.
|
|
|
|
Path declarations use slash-separated relative paths. Absolute paths and network
|
|
URLs are invalid. Each path resolves from its declaring file's canonical directory,
|
|
and files are traversed depth-first in declaration order: a file contributes its own
|
|
body before its imports. Order stabilizes source records and diagnostics but never
|
|
permits replacement. Required missing paths fail; `optional: true` suppresses only
|
|
target nonexistence, not invalid paths, containment failures, directories,
|
|
permissions, parse errors, or composition errors.
|
|
|
|
Glob declarations use the slash-based Go `path.Match` grammar: `*`, `?`, character
|
|
classes, and backslash escapes are supported, while recursive `**` syntax is invalid.
|
|
Backslashes escape pattern characters rather than acting as directory separators.
|
|
ConfigKit validates the complete pattern before filesystem traversal, walks only
|
|
directories selected by preceding segments, and reads wildcard directories in
|
|
bounded batches. Every examined entry consumes the separate visit budget, including
|
|
nonmatches. Matches are sorted lexicographically by slash-relative spelling within
|
|
each declaration before depth-first source traversal. A required zero-match glob is
|
|
`import.missing`; `optional: true` suppresses only zero matches. Invalid patterns,
|
|
walk failures, matched directories, invalid documents, escapes, and repeated targets
|
|
remain errors. Final candidates are bounded by the remaining source limit before
|
|
their contents are accumulated.
|
|
|
|
The configuration root defaults to a selected file's canonical directory. A file
|
|
source may supply `FilesystemContext.Root` to widen that root but may not supply
|
|
`BaseDir`. Whenever byte and reader sources supply filesystem context, they must
|
|
provide both `BaseDir` and `Root` as absolute existing directories; their diagnostic
|
|
labels, including URI-looking labels, never imply filesystem paths. The base and
|
|
selected file must be inside the canonical root. Nonregular targets are rejected before opening to avoid blocking on FIFOs;
|
|
post-open regular-file checks remain in place under the stable-input assumption.
|
|
ConfigKit canonicalizes symlinks,
|
|
checks containment with relative paths, and opens the canonical root-relative target
|
|
through a library-owned root handle. Contained relative or absolute symlink targets
|
|
are accepted; escapes are rejected. Canonical paths define identity, so hard links at
|
|
distinct paths remain distinct sources.
|
|
|
|
An active-stack repeat is `import.cycle` and reports the complete cycle's import
|
|
sites. A previously visited canonical path is `import.repeated` and reports both
|
|
inclusion routes, including diamonds and aliases. Each accepted file is read once.
|
|
Imported fragments may contain their own root directives, but may not claim the
|
|
application's schema-version path or replace one of its ancestors; the version must
|
|
come from the selected source. No directory is scanned implicitly, including one
|
|
named `conf.d`, and ConfigKit performs no remote fetching.
|
|
|
|
## YAML documents
|
|
|
|
The private YAML boundary currently accepts exactly one nonempty document whose root
|
|
is a mapping. Comments and block or flow collections are supported. It rejects:
|
|
|
|
- empty input or an empty document;
|
|
- every additional document, including an empty one;
|
|
- duplicate decoded mapping keys;
|
|
- non-string mapping keys;
|
|
- explicit tags, anchors, aliases, and merge keys;
|
|
- a quoted or plain mapping key equal to `<<`; and
|
|
- input exceeding the fixed depth or node limits.
|
|
|
|
Mapping key comparison is exact: it performs no case folding or Unicode
|
|
normalization. The private document tree retains declaration order, exact validated
|
|
number spelling, source identifiers, and separate key/value line and column
|
|
locations. It contains no upstream parser nodes or types.
|
|
|
|
Root depth is zero. Input depth is limited to 256 and each parsed input tree to
|
|
1,000,000 nodes, counting mapping keys as input nodes. The upstream parser also has
|
|
a private calibrated depth guard; passing that guard never bypasses ConfigKit's tree
|
|
limits.
|
|
|
|
## Scalars
|
|
|
|
Scalar meaning is determined by ConfigKit rather than upstream coercion:
|
|
|
|
| Input form | Meaning |
|
|
| --- | --- |
|
|
| Quoted, literal, or folded scalar | String |
|
|
| Plain `true` or `false` | Boolean |
|
|
| Plain `null`, `~`, or an empty mapping value | Null |
|
|
| JSON decimal number | Number, retaining its exact spelling |
|
|
| Date/time-looking or other ordinary plain text | String |
|
|
|
|
JSON numbers permit an optional minus, a zero or nonzero decimal integer without a
|
|
leading zero, an optional fraction with at least one digit, and an optional decimal
|
|
exponent. Hexadecimal, octal, digit separators, leading plus, leading/trailing-dot
|
|
forms, incomplete exponents, and non-finite values are rejected. Legacy boolean
|
|
words (`yes`, `no`, `on`, and `off`) and non-lowercase variants of the reserved
|
|
boolean/null words are rejected unless quoted.
|
|
|
|
Mapping keys use the same scalar classification and must classify as strings. Quote
|
|
keys that otherwise classify as a boolean, number, or null.
|
|
|
|
## Application schema binding
|
|
|
|
ConfigKit builds a private reflection descriptor graph for each schema operation;
|
|
it keeps no global mutable descriptor cache. A configuration root must be a struct
|
|
or string-keyed map. Nested supported shapes are:
|
|
|
|
- booleans, strings, signed and unsigned integers, and finite floating-point types;
|
|
- `time.Duration`, typed `Secret` references, and paired
|
|
`encoding.TextMarshaler`/`TextUnmarshaler` scalars;
|
|
- pointers, structs, string-keyed maps, slices, fixed arrays, and `any`; and
|
|
- recursive type definitions whose eventual application values are acyclic.
|
|
|
|
Named scalar types retain their Go identity. `uintptr`, complex numbers, functions,
|
|
channels, unsafe pointers, non-string map keys, and nonempty interfaces are not
|
|
configuration types. A custom text scalar must provide both directions on its value
|
|
or pointer method set. Building a descriptor only inspects method sets; it never
|
|
executes application methods or parser-specific hooks.
|
|
|
|
Exported struct fields use an explicit `yaml` name or the lowercase Go field name.
|
|
Unexported fields and `yaml:"-"` are ignored. `omitempty` is accepted but does not
|
|
change binding or projection. An explicit `yaml:",inline"` flattens a struct
|
|
or pointer-to-struct, or designates at most one string-keyed inline map for otherwise
|
|
unmatched names. Ambiguous names, named inline fields, multiple inline maps, and
|
|
unsupported or malformed tag options are schema errors. Embedded fields are ordinary
|
|
named fields unless explicitly inlined. `configkit:"secret"` is the only current
|
|
ConfigKit field tag and marks its bound field or container for secret classification
|
|
and redacted inspection.
|
|
|
|
Schema path traversal uses these same bindings. Explicit key segments address
|
|
mappings, explicit index segments address sequences, and parsed numeric pointer
|
|
tokens become indices only in a sequence. Sequence indices cannot have leading
|
|
zeroes; fixed-array indices are bounds checked. Dynamic `any` and inline-map entries
|
|
remain schema-valid for application-defined descendants, but dynamic values cannot
|
|
introduce typed secret references.
|
|
|
|
Descriptors are reused by typed decoding, projection, and cloning so those operations
|
|
cannot acquire different field bindings.
|
|
|
|
### Canonical module options
|
|
|
|
Applications with a known module catalog represent each configured item as an object
|
|
with an explicit module identifier and an options mapping. Ordinary typed structs,
|
|
including a small union of optional option fields, use the same strict binding above;
|
|
application validation decides which fields and combinations apply to each module.
|
|
String shorthand, a library module registry, and custom structural decoder hooks are
|
|
not supported alternatives.
|
|
|
|
When a runtime-selected catalog genuinely requires dynamic options,
|
|
`map[string]any` retains ConfigKit's canonical dynamic scalars. In particular,
|
|
integers are `int64`; application validation owns allowed keys, value constraints,
|
|
range checks, and conversion to runtime `int` or catalog objects. ConfigKit does not
|
|
provide generic subtree conversion, and consumers must not add a YAML marshal/decode
|
|
round trip to exercise supported behavior. The
|
|
[consumer adoption guide](consumers/README.md) contains compiled typed and dynamic
|
|
examples.
|
|
|
|
## Defaults and typed decoding
|
|
|
|
The private decoder starts from an independently owned clone of the application's
|
|
typed defaults and applies only explicitly present input nodes. Defaults therefore
|
|
never become explicit, while equal-valued input remains explicit. Successful decode
|
|
produces detached typed state, baseline and final canonical trees, an input-presence
|
|
index, and default/input origins for the later public loading lifecycle.
|
|
|
|
| Configured input | Effect on the typed baseline |
|
|
| --- | --- |
|
|
| Absent field | Retain the default |
|
|
| Explicit false, zero, or empty string | Replace the default |
|
|
| Partial mapping or string-keyed map | Recurse and retain omitted default children or entries |
|
|
| Empty mapping | Retain existing mapping children |
|
|
| Sequence, including an empty sequence | Replace the entire sequence |
|
|
| Pointer object | Recurse into a non-nil default; otherwise allocate an object whose omitted children have ordinary Go zero values |
|
|
| Allowed null | Replace the value with nil and suppress its default |
|
|
|
|
Null is allowed only for pointers, maps, slices, and dynamic `any` values. It is an
|
|
error for scalars, structs, and arrays. New map entries and sequence elements begin
|
|
from their Go zero values; ConfigKit does not invent entry-level defaults.
|
|
|
|
Decoding is strict at every typed depth. Unknown struct fields, type mismatches,
|
|
invalid nulls, numeric overflow, and arrays with the wrong length are errors at the
|
|
original semantic path and source coordinates. String-keyed maps accept arbitrary
|
|
keys but preserve the strict schema of their values; inline maps accept unmatched
|
|
keys according to their element descriptor. Multiple independent field failures are
|
|
collected without publishing the partially decoded object.
|
|
|
|
Booleans and strings are not coerced. Integer destinations require integer syntax
|
|
and a value in the destination type's range. Float destinations accept finite JSON
|
|
numbers representable by their destination precision. Dynamic numbers become
|
|
`int64` for integer syntax and `float64` for fractional or exponent syntax, with
|
|
range and finiteness checks. Named scalar types retain their identity. Durations
|
|
decode only from strings accepted by `time.ParseDuration`; integers are rejected.
|
|
Paired text scalars decode only from strings through `encoding.TextUnmarshaler`, and
|
|
codec failures are classified without exposing application error text or input
|
|
values.
|
|
|
|
This decoder is available only through `Load`; ConfigKit exposes no generic document
|
|
conversion API.
|
|
|
|
## Typed projection and ownership
|
|
|
|
ConfigKit can internally project a schema-valid application value directly into its
|
|
owned canonical tree without YAML serialization. Projection includes every bound
|
|
field, including ordinary zero values; `omitempty` has no effect. Struct fields keep
|
|
binding order, dynamic map keys sort by decoded string, and sequences keep their
|
|
order. Integers retain exact base-10 values, finite floats use their destination
|
|
precision, durations use `time.Duration.String`, and paired text scalars use their
|
|
canonical marshaled text.
|
|
|
|
Nil pointers, maps, slices, and dynamic values project as null. Non-nil empty maps
|
|
and slices remain empty containers, preserving the distinction. Recursive acyclic
|
|
objects are supported. Cyclic values, non-finite floats, unsupported dynamic concrete
|
|
types, invalid codec output, and depth/node-limit violations fail with safe schema
|
|
classifications that omit codec error text.
|
|
|
|
The same descriptor graph creates detached typed copies. Pointers, maps, slices,
|
|
arrays, nested dynamic values, and shared references are recursively copied; shared
|
|
pointer identity is not preserved as configuration meaning. Paired text scalars are
|
|
rebuilt by unmarshaling their canonical text into fresh values, detaching private
|
|
backing storage. Ignored and unexported struct fields are zeroed rather than copied.
|
|
|
|
Private structural comparison operates on canonical trees. Mapping additions,
|
|
removals, and descendant replacements retain semantic paths. A sequence is atomic:
|
|
any sequence change reports the sequence path and never invents correspondence
|
|
between old and new indices. Cross-kind and null/container changes replace the
|
|
affected path.
|
|
|
|
These mechanisms support decoding, resolution auditing, metadata,
|
|
and rendering. They are not a public generic conversion or cloning API.
|
|
|
|
## Safe failures
|
|
|
|
Source failures use stable `source.*` codes and YAML failures use stable `yaml.*`
|
|
codes declared by the root package. Diagnostics contain safe classifications,
|
|
semantic paths when applicable, and available source coordinates. Raw parser,
|
|
conversion, reader-error, and input text is not forwarded. Source labels and
|
|
diagnostic messages are application-visible and therefore must never contain
|
|
secrets; terminal control characters are escaped by `WriteDiagnostics`.
|