Files

Eric Rakestraw 7c78da9cd2 Add bounded sources and strict YAML parsing

2026-09-06 19:30:31 +00:00

77 KiB

Raw Blame History

ConfigKit — Initial Target Design

Status and ownership

This roadmap describes the targeted end-state for ConfigKit's initial release, provisionally v0.1.0. It is future design, not a claim of implemented behavior or a stable v1 compatibility promise. Audience: maintainers and implementation agents.

This document owns target behavior, boundaries, and deferred scope. The implementation plan owns stages, dependencies, acceptance gates, and progress. Do not add implementation steps or stage status here.

The architecture policy owns binding contributor constraints. Accepted rationale is recorded in ADRs for composition versus overlay, parser independence, and presence and resolution history under the documentation policy. As features land, move their normative behavior to current contracts and replace the corresponding roadmap text with links. Retire completed roadmap material during release preparation; these files are not permanent duplicate specifications.

Purpose and principles

ConfigKit provides a consistent configuration lifecycle for Go applications using YAML, including Seriatim, Audita, Narratio, WeatherFeeder, WeatherReporter, and Distributor. Its goal is one deterministic, explainable resolution model that works for a single file and scales to explicitly imported fragments.

YAML supplies syntax; ConfigKit owns composition, precedence, presence, conversion, and error policy.
Preserve a source-located tree through composition and overlay. Do not first collapse input into map[string]any or application structs.
Keep application types ordinary. Pointers express application nullability, not a universal workaround for distinguishing absent from false or zero.
Separate strict composition from intentional overlay.
Keep the public API small and the parser private. Avoid a general provider or merge-strategy framework.

Success criteria

Reduce application infrastructure: Can applications delete substantial configuration infrastructure while retaining clear ownership of their domain rules?
Unify configuration conventions: Do consumers adopt one consistent configuration model, with only the flexibility needed to express genuine application requirements?

The architecture policy owns these coequal criteria and their governing rules. The target must satisfy both: the shared lifecycle removes configuration plumbing, while common syntax, presence, composition, precedence, and decoding rules establish consistent conventions. Application types, interpretation, defaults, and semantic validation retain the flexibility needed to describe domain requirements.

Consumer migrations are expected, including intentional configuration syntax changes. Existing code and tests are extraction foundations and evidence of needs, not compatibility obligations for ConfigKit. Prefer canonical object forms over consumer-specific shorthand where they express the same information. Do not add alternate parsers or merge rules merely to preserve an existing convention.

Evaluate an extension by identifying its concrete domain requirement, explaining why the existing primitives cannot accommodate it cleanly, and showing how it preserves one coherent model. If application code must remain, distinguish domain interpretation and validation from duplicated configuration infrastructure. A restriction that makes useful information inexpressible is not justified by consistency alone.

The consumer examples and adoption work must demonstrate both reduced plumbing and convergence on the model. Narratio extraction and simpler consumers should supply concrete evidence; feature count, unchanged legacy syntax, and fewer lines inside ConfigKit are not measures of success. Detailed consumer migrations and execution steps belong in the implementation plan.

Responsibility boundary

ConfigKit owns	Applications own
Local file and labeled bytes/reader inputs	Main-file search paths, bootstrap timing, and CLI parsing/wiring
YAML policy, explicit import traversal, composition, profiles, and explicit CLI overrides	Configuration types and field meanings
Presence, source locations, resolution records, and safe diagnostics	Static defaults, derived defaults, and normalization rules
Secret-reference resolution and explicit directory bootstrap	Credential use and process startup coordination
Strict typed decoding and lifecycle coordination	Semantic validation and schema-version policy
Source reporting, safe history/explanation, and redacted effective output	Remote acquisition, migrations, runtime initialization, and resource checks

Applications may fetch remote data and pass labeled bytes to ConfigKit. This provides useful provenance without making ConfigKit responsible for network availability, credentials, or caching. Narratio's pipeline/campaign/session fallback rules remain application semantics; the extraction boundary is described below.

Existing implementation foundation

Narratio is the primary extraction source and intended first substantial consumer of ConfigKit. Prefer copying and generalizing applicable existing code and tests over independently rebuilding equivalent mechanisms. Reuse includes the behavior captured by Narratio's tests, not just production code. ConfigKit's target contracts govern the result; reuse does not require preserving incidental application structure or incompatible behavior.

The references below were inspected on 2026-09-06 at Narratio commit b68435128be1e24b94fc7ec259d6e9093669fd99. Paths refer to the sibling checkout ../narratio relative to the ConfigKit repository root; the links require that checkout layout. They are extraction references, not a runtime or build dependency. Recheck them against the source revision actually used for extraction.

Foundation	Source references	Intended treatment
File and labeled reader/bytes loading; strict field checking	load.go: `decodeStrictYAML`, `decodeStrictYAMLFromReader`, `LoadSessionBytesWithOptions`	Extract reusable input handling and generalize decoding behind ConfigKit's private parser boundary
Node parsing, strict composition, overlays, and canonical rendering	composition.go: `parseCompositionDocument`, `buildCompositionNode`, `mergeAdditiveCompositions`, `mergeOverlayComposition`, `canonicalYAML`	Extract algorithms and tests; replace upstream node kinds, path spelling, scalar semantics, and incompatible merge rules
Explicit imports and profile selection	pipeline_composition.go: `loadPipelineCompositionSources`, `loadPipelineImports`, `loadPipelineProfileOverlays`, `selectPipelineProfile`	Reuse traversal/selection mechanics with ConfigKit's recursive graph, containment, directive syntax, and fixed selected-profile convention
Effective value/source inspection	effective_pipeline.go: `EffectivePipelineValues`, `EffectivePipelineSources`, `MarshalEffectivePipeline`; config_commands.go	Reuse deterministic projection, ownership, and side-effect-free inspection cases; retain application commands and generated-domain attribution in Narratio
Explicit field presence	normalize.go: `NormalizeConfig.UnmarshalYAML`	Generalize the requirement into library metadata; replace the application-specific parser hook
Default application	load.go: `applyPipelineDefaults` and related helpers; defaults.go	Reuse applicable mechanics and behavioral cases while keeping actual default values and domain rules application-owned
Resolution and source attribution	load.go: `Resolve`, `mergeCampaignSession`, `selectStableInput`; config.go: `ResolvedInputFile`, `SessionSource`	Adapt source tracking into generic provenance; retain campaign/session fallback and domain types in Narratio
Application-supplied remote bytes	config_loader.go	Use as evidence for labeled input support; keep fetching, storage clients, and remote metadata acquisition outside ConfigKit
Existing behavioral coverage	normalize_test.go, load_validate_test.go, campaign_config_test.go	Extract relevant cases and minimal fixtures; preserve domain-specific assertions in consumer tests

Additional extraction coverage includes composition tests, import tests, profile tests, strict-loader tests, and inspection tests. These are reusable behavior evidence, not an instruction to preserve every assertion.

Compatibility and adaptation

Extraction must distinguish preserved behavior, intentionally changed behavior, and application-only behavior. Record the source revision and relevant symbols with the extraction work so reviewers can compare the implementation and tests. Generalized code must not import Narratio's internal packages or copy its domain model into ConfigKit's public API.

Known adaptation points include:

Some Narratio defaulting helpers use an empty string to trigger a default. Adapt these to ConfigKit's presence and defaults rules rather than losing explicitly configured empty values.
decodeStrictYAMLFromReader now rejects trailing documents. Preserve and generalize its strict-loader regressions under ConfigKit's single-document policy.
The custom normalization decoder and pointer-based presence conventions need to give way to generic metadata while preserving genuine domain nullability.
Existing composition nodes and effective-field ownership are substantial foundations, but do not retain the full line/column and historical-value records ConfigKit requires. Keep application-generated family/campaign attribution as domain resolution with recorded dependencies.
Narratio rejects overlapping empty mappings, null overlays, and cross-kind overlays. ConfigKit's operation table deliberately permits those cases. Adapt the implementation and expectations together.
Narratio declares import paths and profiles in a root composition envelope, supports a declared default profile, and loads every declared overlay. ConfigKit uses _configkit.imports, recursive imports/globs, and only the selected fixed profile path; default means no overlay. Do not preserve an alternate convention.
Narratio uses os.SameFile to reject filesystem aliases. ConfigKit uses canonical path identity, treats hard links as distinct paths, and enforces root containment.
Narratio's owned tree still embeds upstream YAML kinds/tags and dotted/indexed paths. Its final binding and effective inspection reserialize/reparse YAML. Reuse algorithms without importing those dependencies into ConfigKit's document model or replacing direct descriptor-driven decoding and projection.
Narratio's configuration diffs and digests are useful consumer evidence, but remain outside ConfigKit's initial scope. Their implementation does not supply full historical-value explanation or authorize a new public API.

Each intentional behavior change needs focused tests and an explicit migration account for the affected consumer. Do not copy an old expectation merely because it passes in Narratio, or discard a useful case because the implementation moves. Use representative Narratio configurations to verify preserved outcomes and identify intentional differences. Exercise a simpler consumer as well so the public API remains practical for single-document applications.

Capabilities requiring broader implementation

The inspected Narratio loader supplies explicit flat imports, strict composition, profile overlays, source ownership, and effective inspection in addition to its pipeline/campaign/session domain coordination. Prefer extracting those foundations rather than rebuilding equivalent mechanisms.

ConfigKit still requires new or substantially revised work for recursive import graphs and globs, bounded canonical containment, its conservative YAML policy, schema-aware defaults/direct decoding, comprehensive location and value history, recorded domain resolution, generic secret classification, and safe structured reporting. Existing provenance is a starting point, not full explainability.

Automatic conf.d discovery remains deferred. Existing application relationships and declared profile paths do not redefine ConfigKit's import/profile conventions. The implementation plan owns extraction tasks and ordering.

Consumer fit and intentional migrations

These are adoption targets, not promises of unchanged consumer behavior or completed integrations. Source links refer to sibling checkouts as in the Narratio foundation section. The findings reflect the inspected implementations on 2026-09-06 and must be rechecked at migration time.

Consumer and evidence	Useful ConfigKit fit	Intentional migration or retained application responsibility
Narratio: configuration loader	Shared loading, presence, defaults coordination, and source attribution	Replace ad hoc presence/empty-value defaults; retain campaign/session domain resolution
Notarius: file model, environment handling	Strict nested models, named pipelines, presence-aware values, and diagnostics	Use canonical object bindings/references and string schema versions; adapt dynamic integer consumers to `int64`; migrate configuration-field environment overrides to files/profiles; keep catalog resolution and runtime invocation choices application-owned
WeatherReporter: loader, module options	Defaults-only loading, durations, report maps, list presence, and typed module options	Replace string shorthand with object entries and private presence flags with metadata; translate explicitly set configuration flags into ConfigKit overrides; reuse secrets-directory loading through explicit ConfigKit bootstrap
Seriatim: configuration surfaces, speaker rules, autocorrect rules	Simple typed loading of ordered YAML rule files, normalization, and cross-rule diagnostics	Adopt strict YAML rejection deliberately; retain matching/replacement algorithms, resource checks, and CLI/environment runtime settings; no central application YAML file is required

WeatherReporter's updated configuration tests and external constructed-config tests cover explicit empty lists, consistent loaded/constructed behavior, and normalization without mutating source options. Preserve these domain outcomes through ConfigKit's presence, detached ownership, and application validation. ConfigKit permits an empty list structurally; a particular consumer may reject it semantically.

The initial model standardizes syntax and resolution while preserving application field meanings. Module shorthand and integer schema versions are syntax migration issues; module-specific option schemas are genuine domain variation addressed below. ConfigKit configuration profiles are overlays, not substitutes for application-owned LLM or pipeline profile concepts.

External values and invocation state

Configuration-field CLI flags use the explicit override API. Applications own parsing flags and selecting sources/profiles. Separate invocation state, such as an input artifact or execution lane, remains application-owned. General environment-variable configuration overlays are an explicit non-goal, not deferred work. Environment access is reserved for named secret references.

Do not inject hidden overrides through captured environment/CLI values in default factories, resolver hooks, or validators. Domain resolution follows the declared input layers. Mutating Result.Config cannot update a validated snapshot.

Secret references and explicit bootstrap

Secret-bearing configuration fields use ConfigKit's opaque Secret type. Their YAML value is an environment-variable name, for example api_key: OPENROUTER_API_KEY. Names are public configuration; resolved credentials are not configuration-tree values. Literal-secret input, interpolation, fallback literal values, dotenv parsing, and implicit directory discovery are unsupported. Names must match [A-Za-z_][A-Za-z0-9_]*. A string that happens to be a valid name is always a reference; ConfigKit cannot detect credentials mistakenly written in other fields.

func SecretEnv(name string) Secret
func (s Secret) EnvName() string
func (s Secret) Value() string
func (s Secret) IsResolved() bool

SecretEnv creates an unresolved reference; invalid names fail the containing load. The zero value is an absent reference, never a literal secret. A non-pointer Secret is required: absent, null, empty/invalid name, unset environment variable, or an empty environment value fails with a safe secret.* diagnostic. Use *Secret for optional credentials: nil is allowed, but every non-nil reference must resolve to a nonempty value. Applications validate conditional requirements. Only typed fields/maps/slices establish secret-reference semantics; dynamic any strings are ordinary values and must migrate to typed secret slots when needed.

Recognize Secret before ordinary struct/text binding. Projection, comparison, cloning for defaults, and resolution auditing operate on the name only. Resolve the surviving references after recorded application resolution and before semantic validation, using one os.LookupEnv result per unique name per load. A copied previous result used as defaults is re-resolved; it cannot carry stale credentials. Validation can use Value(); resolution works with names. In addition to ordinary projection auditing, reject validator changes to a secret's private resolved state by comparing it with this load's lookup result. A same-name unresolved replacement must not evade the read-only validator contract. Value() returns empty on unresolved references; IsResolved() distinguishes that state. Private credential storage is excluded from every tree, historical snapshot, diagnostic, default format, JSON/text serialization, and inspection path. Provide safe formatting for all fmt verbs and name-only JSON marshaling; expose no credential decoder/codec. Only the deliberately named Value() accessor returns a credential. Applications own its use and must not embed it in hook messages. Secret references themselves need no configkit:"secret" tag: the existing tag/predicate classifies additional sensitive or derived values and can intentionally hide a reference name.

The separate, opt-in bootstrap operation below loads a directory into the process environment. Load never calls it or changes process-global state. Applications choose a directory and call bootstrap once during startup before concurrent work. Bootstrap paths are native application-selected paths, independent of import roots. An empty or missing directory path is an error; an existing empty directory succeeds.

func LoadSecretsDir(directory string) (*SecretsReport, error)
func (r *SecretsReport) Entries() []SecretFileResult
func (r *SecretsReport) WriteTo(output io.Writer) (int64, error)
type SecretsError struct { /* private safe diagnostic storage */ }
func (e *SecretsError) Error() string
func (e *SecretsError) Diagnostics() []Diagnostic
type SecretFileResult struct {
    Name, Path string
    Status SecretFileStatus
    ExistingEmpty bool
}
type SecretFileStatus string
// Constants use values "loaded", "skipped_existing", "not_applied", "failed":
// SecretFileLoaded, SecretFileSkippedExisting, SecretFileNotApplied, SecretFileFailed.

Extract applicable code and tests directly from WeatherReporter's secrets loader, inspected at revision 5d306a202471e10fe98923a1df27d0b0fce5c666. Reuse its filename checks, file-type rejection, single terminal LF/CRLF removal, and separation of loadSecrets from applySecrets. WeatherReporter now preloads file contents, validates its configuration, then applies environment writes with attempted rollback. ConfigKit preserves existing variables, reports partial apply failure without rollback, and keeps bootstrap separate from configuration loading. Record extraction symbols, revision and adaptations during implementation.

Narratio's secrets loader at the revision recorded above is a complementary foundation: loadSecretsFromConfigWithDependencies preserves existing variables including empty ones, and readAPIKeyFile uses bounded reads. Its tests provide reusable failure and environment-preservation cases. Adapt its immediate writes and aggregate/log reporting to ConfigKit's preflight and per-file structured report. Narratio skips invalid names, removes all trailing CR/LF characters, rejects a symlink at the selected directory path, and enforces restrictive permission bits. Those policies are not inherited: use ConfigKit's name/type/newline/directory rules and fixed bounds. ConfigKit does not add a permission-mode gate in the initial API; applications own deployment permissions. Extract bounded-read mechanisms without copying Narratio's application dependency framework or logger.

Process only immediate entries, sorted by filename. Reject invalid names, symlinks, directories, and nonregular files, including entries whose names are already set. Allow a symlink in the explicitly selected directory path by resolving it to a canonical directory first. Validate all entry metadata before mutation. os.LookupEnv presence wins, including an empty existing variable: skip reading that file's contents and report SkippedExisting with ExistingEmpty set accurately. Read eligible files with bounds, remove exactly one terminal CRLF or LF, preserve all other whitespace, and reject NUL bytes. Empty file values are allowed at bootstrap, but fail if subsequently used by a required reference. Preflight all eligible reads before any os.Setenv. No writes occur on preflight failure. At apply time recheck presence before each write and skip newly set names. Concurrent mutation is unsupported; this is not an atomic compare-and-set facility.

Return a non-nil report even on failure. Successful writes are Loaded, blocked files are SkippedExisting, the failed entry is Failed, and eligible but unwritten entries are NotApplied. Directory-level failures have a safe error without a fabricated entry. Stop on the first deterministic failure; reports include all entries enumerated so far, with unprocessed entries NotApplied. An apply failure preserves earlier successful writes and reports partial progress; do not attempt rollback that could overwrite unrelated process changes. Errors are *SecretsError with safe secrets.* diagnostic codes and no raw unwrap chain. Errors and report rendering contain names, paths, statuses, and safe classifications, never contents.

Report entries and diagnostic slices are defensive copies. Bootstrap diagnostics include safely quoted canonical paths in messages with no invented configuration source ID or source location. Report formatting escapes control characters and propagates short writes/errors.

The report is the authoritative bootstrap provenance: a blocked file explicitly identifies the existing process variable as the winner, even when empty. Keep it alongside the load result and render it before configuration explanation in an application command. Secret-resolution metadata records the variable name and resolved, unset, or empty lookup outcome (failures in diagnostics). It does not claim which external actor populated the process environment. Reports are separate snapshots, not accepted as evidence of the environment's later state; there is no implicit global report or stale filename attribution in Load.

CLI overrides

Precedence is application defaults < composed base < selected profile < explicit CLI overrides. Domain normalization follows those inputs and is recorded.

type Override struct {
    Path Path
    Value any
    Flag string
}
// Options[T].Overrides is []Override.

Applications provide records only for flags explicitly set by the caller; parser defaults are not overrides. Flag is a required, non-secret display name, such as --workers, never the raw argument or value. ConfigKit has no CLI parser dependency. Values are detached and converted to ConfigKit-owned nodes before composition: accept built-in string, bool, integer and unsigned integer widths, finite floats, nil, []any, and map[string]any recursively. Strings remain strings; do not parse YAML fragments or coerce textual booleans/numbers. Durations and secret references use strings. Reject other types, cycles, nonfinite numbers, and limits violations with safe override.* diagnostics.

Validate every path/record before applying any. Reject root paths, sequence-index traversal, duplicate paths, and ancestor/descendant overlaps within the override list. Numeric tokens are mapping keys only here. Mapping-key paths may create missing mapping ancestors, but cannot traverse an existing scalar/null/sequence. The target path overlays with the same rules as profiles: mappings recursively merge; sequences and all other replacements are atomic. Use a whole sequence value to replace a list. Preserve supplied list order; mapping entries use canonical key order. Unknown struct fields and effective type mismatches fail normal strict decoding. Never allow _configkit directives or writes to the configured schema version path, including ancestor replacement. Overrides also work with NoSource().

Each record contributes a synthetic CLI source after file/profile sources, in record order, labeled with Flag, with no file coordinates. Explicit false, zero, empty string, null, and empty list remain present. Snapshot caller-owned values before applying them. History includes replaced inputs even for equal-value writes. Neither callers nor hooks may mutate supplied data concurrently with loading.

Resolution lifecycle

application selects source and loading options
  → parse and enforce YAML policy for every source
  → discover explicit imports and strictly compose the base
  → application schema-version check, when configured
  → apply the selected profile
  → apply explicit CLI overrides to the node tree
  → apply static defaults and strictly decode
  → apply recorded application resolution and normalization
  → resolve surviving secret references from the environment
  → collect application semantic validation diagnostics
  → return independently owned configuration and metadata

Value precedence is defaults < composed base < selected profile < CLI overrides. Resolution is an application transformation after those layers, not another user override source. Future migration support belongs after base composition/version inspection and before profile application and typed decoding; migrations are not part of the initial release.

YAML policy applies to every document even when a later layer would replace its values. Schema/type checks apply to the effective input tree; application validation runs only after successful decoding and resolution. An invalid configuration is never returned as a successful, usable result.

YAML and scalar policy

Source ownership, the supported YAML subset, scalar grammar, fixed input limits, and safe failure behavior are implemented. The current configuration input contract now owns those definitions. Typed scalar conversion into application values remains target work in typed decoding.

Paths and presence

Semantic path construction, JSON Pointer parsing, and current metadata query declarations are implemented. Their current contracts are owned by the Go documentation in path.go and metadata.go. The remaining presence behavior below is target lifecycle work.

Distinguish absent, explicitly null, explicitly zero/false/empty, and other explicit values. An explicit value equal to its default remains explicit. Presence queries describe the surviving input contribution after overlays; separate history queries describe earlier appearances. A parent mapping's presence does not make all of its children explicit. Replacing a subtree removes its descendants from effective presence and origin queries while retaining historical replacement records.

Application normalization preserves the fact that a surviving input was explicit. A derived value for an absent input remains non-explicit. Its final origin is a resolution operation, with input origins retained separately.

Composition and overlay

Composition combines the main file and explicitly imported fragments as peers. Repeated semantic definitions are errors, even when their values are equal. Both conflicting locations must be reported. Prefer mappings keyed by stable identifiers when entries need to be contributed independently.

The following table is exhaustive by node category. “Value” includes scalar, sequence, mapping, and null. Absent means no node, not null.

Existing/lower node	Incoming/higher node	Strict composition	Overlay
Absent	Absent	Absent	Absent
Absent	Value	Incoming	Incoming
Value	Absent	Existing	Existing
Mapping	Mapping	Recurse by key	Recurse by key
Any other pair of present nodes	Any other pair of present nodes	Collision	Replace entire existing node

Two empty mappings compose successfully. An empty mapping contributes no child changes to an existing mapping; it does not clear that mapping. A mapping can replace a lower scalar, sequence, or null during overlay, subject to the final schema. Sequences are atomic: two definitions collide in composition and the higher sequence replaces the entire lower sequence in an overlay. [] clears a sequence. There is no append, union, index merge, or mapping-clear operator.

Replacing a parent must not leave effective metadata for removed children. Sequence histories describe whole-sequence replacement; index origins refer to the surviving sequence, not an attempted correspondence with previous indices.

Explicit imports and source identity

Imports are opt-in through the loading API. Reserve the root _configkit mapping for library directives and remove it before application decoding. Initially its only permitted field is imports, a sequence of declarations. Illustrative fragment:

_configkit:
  imports:
    - path: fragments/server.yml
    - glob: fragments/outputs/*.yml
      optional: true

Each declaration has exactly one nonempty path or glob; optional is a bool and defaults to false. Unknown directive fields are errors. When imports are disabled, the presence of _configkit is an error, not an ignored directive.

Relative paths resolve from the declaring file's directory. Absolute import paths and network URLs are rejected.
Expand declarations in list order, sorting each glob's matches lexicographically by slash-separated relative path, with no locale sorting. Use the slash-based Go path.Match grammar (*, ?, character classes, and backslash escapes), with recursive ** syntax forbidden. Backslashes escape pattern characters, not directory separators.
Traverse depth-first: process a file's own contribution, then its declarations in order. Ordering stabilizes source lists and diagnostics; it never permits duplicate definitions to override one another.
A required missing path or unmatched glob is an error. Optional suppresses only a nonexistent target or zero matches. Invalid patterns, permission failures, directories, and invalid matched documents remain errors.
The configuration root defaults to the main file's directory; callers may explicitly supply another root. Resolve symlinks and cleaned absolute paths for identity and containment. Imports and profiles must stay inside that canonical root. Retain declared paths and import-site locations for messages. Relative and absolute symlink targets are permitted when their canonical target is contained; the spelling of a symlink target alone does not determine containment.
An active-stack repeat is a cycle; report the complete cycle and import sites. A previously visited canonical file outside the active stack is a repeated import error, including diamond graphs and overlapping glob matches. Report both inclusion routes. Hard links at distinct canonical paths remain distinct sources; this is path identity, not inode deduplication.
Read each accepted source once per load. Do not promise an atomic snapshot of a filesystem being modified concurrently. Bound input bytes, node nesting, import depth, and source count with documented limits and structured errors. Excess input is an error, never an implicitly truncated document. Glob traversal counts examined directory entries, including nonmatches, against its work budget; only accepted matches contribute to the sorted source order.

Bytes/reader sources require a diagnostic label. Imports and profiles for these sources require an explicit filesystem base directory and root; never infer a filesystem location from a label such as an S3 URI. Without that context they remain unavailable. ConfigKit does not fetch remote imports.

Debian-style automatic conf.d discovery is deferred. A caller may explicitly import a file or glob inside a directory named conf.d, but that name has no special behavior. Do not scan adjacent directories implicitly.

Profiles

Load only the composed base unless the application selects a profile. The API accepts a name matching [a-zA-Z0-9_-]+, not an arbitrary path. The empty name and reserved lowercase name default both mean no overlay; a literal profiles/default.yml is never loaded. Other names resolve exactly to <configuration-root>/profiles/<name>.yml, without extension fallback or case normalization. A selected missing profile is an error.

A profile is one policy-validated YAML mapping applied using the overlay table. It must not contain _configkit, introduce imports, or alter the application's declared schema-version path. Reject these directives even if their values would be unchanged. There is no profile inheritance or multiple-profile stack. Applications own flags and commands that select this behavior.

Static defaults and null

Applications supply a defaults factory that creates a fresh configuration per load. Mutable maps, slices, and pointers must not be shared between factory results. ConfigKit must not retain mutable aliases between returned loads or between the application result and its internal metadata snapshots.

Defaults obey the same recursive mapping/atomic replacement rules as a lower layer, rather than incidental decoder behavior:

Configured input	Effect on defaults
Absent field	Keep the default
Explicit false, zero, or empty string	Replace the default
Partial mapping, including a string-keyed Go map	Retain omitted default children/entries
`{}` over an existing mapping	Retain its default children
Sequence, including `[]`	Replace the entire default sequence
Partial object represented by a pointer	Retain children of a non-nil default object; otherwise allocate and use ordinary zero values for omitted children
Allowed null	Replace with nil; suppress lower defaults

The factory supplies actual defaults; ConfigKit does not invent per-entry defaults for newly introduced map entries or sequence elements. Application resolution may supply those. Go zero values are part of the initial typed baseline; do not imply that the application explicitly declared every zero as a domain default. Avoid serialization through omitempty that would erase baseline values or presence information.

Null is valid for pointers, maps, slices, and dynamic any values. It is invalid for ordinary scalar fields, structs, and arrays. Thus workers: null cannot silently leave an integer default intact. Applications can further reject null through validation. Null never means reset to the application default.

Typed decoding

Strictness must be enforced by ConfigKit across the effective tree:

Unknown struct fields, type mismatches, overflow, and invalid null are errors.
Support ordinary exported fields and yaml names/ignore tags. Untagged field names use the lowercase Go field name. Embedded structs follow the same rule unless explicitly tagged yaml:",inline"; inline struct fields contribute to the containing mapping. Permit at most one inline string-keyed map per object, which receives otherwise unmatched keys. Duplicate declared field names and default inline-map entries colliding with declared fields are schema errors. Without an inline map, unmatched keys remain errors. Ignore omitempty for decoding, defaults, and effective inspection.
String-keyed maps accept application-defined keys. Struct values inside them remain strict. map[string]any permits arbitrary keys and recursively accepts the supported scalar/container model; application validation owns its schema.
Do not coerce numbers or booleans to strings, or quoted numbers to numeric fields. Integer targets require integer syntax and range checks. Float targets accept finite decimal numbers subject to representable range checks.
Dynamic integer values use checked int64, and fractional/exponent values use checked finite float64; values outside those ranges require a typed schema or a quoted string rather than silent rounding of integer nodes.
Support time.Duration from a duration string. Custom scalar types must implement both encoding.TextUnmarshaler and encoding.TextMarshaler so decoding and inspection have an agreed representation. Duration integers are rejected. See the type and projection contract.
Parser-specific unmarshal hooks are not a public extension mechanism. Do not let them bypass key, null, or type policy. Custom structural decoders are deferred; unsupported type shapes fail with a clear schema diagnostic.

Keep parser nodes and errors internal under the upstream isolation contract. The field-binding model must also support defaults, provenance, secret classification, and rendering consistently.

Recorded resolution and validation

Application-owned resolution runs after decoding and may derive absent values or normalize explicitly configured ones. Provide a narrow resolution context that exposes metadata and records each changed semantic path, operation kind (derived default or normalization), and a safe reason. Dependency paths may be recorded when a value is derived from other fields.

A resolver may mutate the typed configuration, but ConfigKit compares its before/after configuration projection and requires records for changed paths. Unrecorded changes are resolution errors. Container replacement records cover changed descendants. Capture previous/final values internally from the actual configuration rather than trusting separately supplied values. Application resolvers own meaningful reasons; a diff alone cannot explain intent. Reasons must not embed secrets. The proposed resolution API provides helpers to combine assignments and records. Snapshot comparison verifies net changes during the hook; it does not reconstruct intermediate assignments. A record describes the hook's before/final transition and declared intent, not an automatically recovered event log. Equal-valued derived-default assignments may still have a record identifying their intent. Multiple records at the same path are coalesced for that hook, retaining declared reasons and dependencies.

The final configuration, effective output, presence queries, and explanation records must agree. Do not render the pre-resolution YAML tree as the final configuration. ConfigKit promises an independently owned snapshot it does not subsequently mutate, not language-enforced immutability of exported Go values. Callers should treat returned configuration as read-only; later caller mutations are outside the captured metadata snapshot.

Application semantic validation runs after successful resolution and collects independent errors in a standard diagnostic form. Validators are read-only and own domain requirements; checking live S3 buckets or starting runtime clients is outside this lifecycle. Do not run later hooks on an invalid typed value.

Provenance, diagnostics, and redaction

Safe diagnostic/provenance value records, canonical diagnostic rendering, and LoadError access are implemented. Their current contracts are owned by records.go and diagnostic.go. Attaching those records to the loading lifecycle remains target work below.

Retain input sources, layer, line/column where available, import sites, effective origins, overridden definition records, and resolution operations. Container origins may be composite; never invent one file location for a mapping assembled from several sources. Defaults and resolution have synthetic origins rather than fabricated file coordinates. Preserve definition history without exposing mutable parser nodes or unrestricted raw value references through the API.

Diagnostics have stable codes, severity, semantic path when available, a safe message, primary location, and related locations. Sort deterministically by source traversal order, location, path, and code. Expose structured errors for programmatic inspection; human message wording need not be a compatibility contract. Fatal parse/graph failures may stop dependent work; collect independent schema and semantic errors where continuing is meaningful.

On any error, loading returns no usable configuration. A structured error may carry safe diagnostics and sources discovered so far. Successful results carry configuration, metadata, sources, and nonfatal diagnostics. Avoid ambiguous partial-success results or two competing authoritative error lists.

Redaction is part of the initial lifecycle, including parse and decode failures. Never blindly forward parser/custom-decoder error text or source snippets that may contain values. Report safe expected-type and location information instead. Applications likewise must supply value-free validation messages and resolution reasons. ConfigKit cannot sanitize secrets embedded in arbitrary application prose reliably.

Use configkit:"secret" on fields and an application path-classification hook for dynamic entries. A secret container makes descendants secret. Secret classification applies conservatively across current and historical definitions of a path: replacing a secret does not declassify its history. It propagates from recorded secret dependencies to derived outputs. Applications remain responsible for marking secret-producing transformations whose dependencies are not recorded.

Provide source reporting and redacted effective rendering in the initial target. Render from a canonical projection of the final typed snapshot, including zero values and null/empty distinctions supported by the schema. Redacted output is for inspection and is not promised to be reloadable. ConfigKit renderers must not expose raw secrets; returning typed configuration to the application is an intentional trusted boundary. ConfigKit does not log configuration on its own.

Provide full safe historical-value explanation in the initial release, as defined below. Defer config diffs and fingerprints. A future fingerprint design must specify canonicalization, secret handling, and disclosure risks before claiming identity or reproducibility guarantees.

Application schema versions

Applications may declare a schema-version path (for example /schema_version) and a pre-decode checker. The main document owns this value; fragments omit it, and profiles/CLI overrides cannot supply or change it. Check fragment placement before composition so a version supplied only by an import is also rejected. ConfigKit enforces that placement when the application enables the mechanism. The application decides whether a version is required and which values it accepts. ConfigKit does not require one universal version field or interpret an application's version as the library version. Automatic migration is deferred.

Consumer API design

The following is a proposed public API, not implemented Go reference documentation or a frozen naming commitment. Declarations omit method bodies and private fields. Consumer snippets assume the indicated standard-library imports and a configkit import using the module path selected in the implementation plan. They are illustrative target examples, not currently runnable examples. The implementation plan owns their conversion into maintained executable examples and verification against consumer models.

Entry point and reusable application definition

func Load[T any](source Source, options ...Options[T]) (*Result[T], error)

type Definition[T any] struct {
    Defaults func() T
    Resolve  func(*T, *Resolution) error
    Validate func(*T, *Validation)
    Version  *VersionRule
    Secret   func(Path) bool
}

type Options[T any] struct {
    Definition Definition[T]
    Imports    bool
    Profile    string
    Filesystem *FilesystemContext
    Overrides  []Override
}

type FilesystemContext struct {
    BaseDir string
    Root    string
}

Load accepts zero or one options value; multiple values are an options error, not a precedence mechanism. The optional argument preserves the shortest load without requiring a builder or an option function per setting. With no options, T starts at its zero value, hooks are absent, and imports/profiles are disabled. Strict YAML policy, decoding, presence, and safe diagnostics are always active; there is no permissive-mode switch.

Definition[T] contains reusable application rules; Options[T] contains the per-load source environment and selection. Applications may reuse a definition for many loads. Treat definitions as read-only after sharing them. Each load invokes the defaults factory afresh, creates new callback contexts, and owns its result. Callbacks must not retain these contexts or configuration pointers; shared closures must be safe for concurrent calls if the application loads concurrently. ConfigKit does not synchronize application-owned captured state.

Source and its constructors are implemented; source.go and the configuration input contract own their current behavior. The loading and filesystem-option behavior below remains target work. Applications must use non-secret source labels because labels appear in diagnostics and source reports. Filesystem context follows the source rules: a file's own directory supplies its base, and an explicit context can set its root. Supplying BaseDir for a file is an options error; bytes/readers require both absolute BaseDir and Root when imports or profiles are used. A base must be inside its root. These options do not change application field path meanings.

NoSource() explicitly selects defaults-only loading. It contributes no document or file source entry and marks no fields explicit by itself; CLI overrides still contribute synthetic sources and explicit values; it still runs schema checking with an absent version, defaults, resolution, validation, and snapshot/inspection. An application requiring an input version can therefore reject this mode. Imports, an active profile, or filesystem context with NoSource() are options errors. A zero Source is invalid and is not an implicit defaults-only request. Empty files and empty byte/reader inputs remain invalid YAML inputs.

The application decides whether its default path may be absent and explicitly selects NoSource() in that case. ConfigKit never suppresses missing-file errors on File; an explicitly selected missing file must still fail. Applications must not treat permission or parse failures as evidence that no file was provided.

A minimal application needs only its type and one load:

type Config struct {
    Enabled bool `yaml:"enabled"`
    Workers int  `yaml:"workers"`
}

func ReadConfig(path string) (Config, error) {
    result, err := configkit.Load[Config](configkit.File(path))
    if err != nil {
        return Config{}, err
    }
    return result.Config, nil
}

Adding defaults and validation remains ordinary application code:

var workerRules = configkit.Definition[Config]{
    Defaults: func() Config { return Config{Enabled: true, Workers: 4} },
    Validate: func(cfg *Config, v *configkit.Validation) {
        if cfg.Workers < 1 {
            v.Error(configkit.Keys("workers"), "must be at least one")
        }
    },
}

func ReadWorkerConfig(path string) (*configkit.Result[Config], error) {
    return configkit.Load(configkit.File(path), configkit.Options[Config]{
        Definition: workerRules,
    })
}

The same application rules support defaults-only operation:

func ReadWorkerDefaults() (*configkit.Result[Config], error) {
    return configkit.Load(configkit.NoSource(), configkit.Options[Config]{
        Definition: workerRules,
    })
}

Paths and metadata

The path API, detached metadata records, and empty/unknown query behavior are now implemented; path.go, records.go, and metadata.go own their current Go contracts. Populating effective presence, origins, and history belongs to the loading stages. Callback path checking and safe historical values remain target behavior below.

Resolution and validation callbacks

func (r *Resolution) IsExplicit(path Path) bool
func (r *Resolution) Record(path Path, kind ChangeKind, reason string, dependencies ...Path)

func Default[V any](r *Resolution, path Path, target *V, value V, reason string, dependencies ...Path)
func Normalize[V any](r *Resolution, path Path, target *V, value V, reason string, dependencies ...Path)

func (r *Resolution) Error(path Path, message string, related ...Path)
func (r *Resolution) Warning(path Path, message string, related ...Path)
func (v *Validation) Error(path Path, message string, related ...Path)
func (v *Validation) Warning(path Path, message string, related ...Path)

ChangeKind distinguishes derived defaults and normalization. Default assigns and records only when the effective input at the path is not explicit; thus it may replace a static baseline. Normalize assigns and records a changed value whether the input was explicit or defaulted. Both helpers accumulate invalid-path or record errors in the resolution context, checked before validation. They do not require applications to construct source locations or duplicate value history. Targets must correspond to the declared path in the final configuration; ConfigKit checks the projected path value against the helper's assigned value and audits all net changes for coverage. This is consistency validation, not proof of the identity of an arbitrary Go pointer supplied by application code.

For a multi-field transformation, the application may mutate normally and call Record for affected paths or a replaced container. The before/final comparison requires coverage of net changes; it does not infer reasons or promise all intermediate mutations. The helper/record design must be exercised on consumer code before stabilization; do not expand bookkeeping merely to make an internal audit easier to implement.

Validation receives the resolved configuration by pointer for ergonomic reuse of application code but must not mutate it, including nested maps or slices. Its context collects independent messages; ConfigKit supplies diagnostic codes and available primary/related locations. A missing field uses the nearest relevant container/source location, or a synthetic location when no input source applies. Application-provided messages and reasons must be safe, value-free prose.

Resolvers and validators can report safe domain messages through Error and Warning, specifying one primary path and optional related paths. ConfigKit supplies phase-specific codes and attaches available locations to all paths. Related paths are useful for duplicates and cross-field constraints; no caller must construct file coordinates. Reported errors fail the load even when a resolver returns nil. Warnings alone do not fail it. Stop before the next phase when a hook has reported errors.

A non-nil resolver error also aborts the load. If the context already contains errors, retain those without adding a redundant generic failure; otherwise emit a safe hook-failure diagnostic. Never forward arbitrary error text by default. Only the explicit reporting methods opt into human-visible application messages; callers must not interpolate secrets or attach raw underlying errors as messages. Validators need no multi-error wrapper. Callback panics are programming failures and are not converted to configuration validation errors. Contexts are valid only during their callback and are not safe for concurrent use within a callback.

A Narratio-like application can retain typed domain logic, including defaults for new map entries. This illustrative example uses strings and time:

type Artifact struct {
    Output  string        `yaml:"output"`
    Timeout time.Duration `yaml:"timeout"`
}

type PipelineConfig struct {
    PollInterval time.Duration       `yaml:"poll_interval"`
    Timeout      time.Duration       `yaml:"timeout"`
    Artifacts    map[string]Artifact `yaml:"artifacts"`
    APIKey       configkit.Secret   `yaml:"api_key"`
}

func resolvePipeline(cfg *PipelineConfig, r *configkit.Resolution) error {
    configkit.Default(r, configkit.Keys("timeout"), &cfg.Timeout,
        cfg.PollInterval/10, "derived from poll interval",
        configkit.Keys("poll_interval"))

    for name, artifact := range cfg.Artifacts {
        path := configkit.Keys("artifacts", name)
        configkit.Default(r, path.Key("timeout"), &artifact.Timeout,
            cfg.Timeout, "inherited pipeline timeout", configkit.Keys("timeout"))
        configkit.Normalize(r, path.Key("output"), &artifact.Output,
            strings.TrimSpace(artifact.Output), "trimmed surrounding whitespace")
        cfg.Artifacts[name] = artifact
    }
    return nil
}

func validatePipeline(cfg *PipelineConfig, v *configkit.Validation) {
    if cfg.PollInterval <= 0 {
        v.Error(configkit.Keys("poll_interval"), "must be positive")
    }
    if cfg.Timeout <= 0 {
        v.Error(configkit.Keys("timeout"), "must be positive")
    }
    for name, artifact := range cfg.Artifacts {
        path := configkit.Keys("artifacts", name)
        if artifact.Timeout <= 0 {
            v.Error(path.Key("timeout"), "must be positive")
        }
        if artifact.Output == "" {
            v.Error(path.Key("output"), "must be nonempty")
        }
    }
}

var pipelineRules = configkit.Definition[PipelineConfig]{
    Defaults: func() PipelineConfig {
        return PipelineConfig{PollInterval: time.Minute}
    },
    Resolve:  resolvePipeline,
    Validate: validatePipeline,
}

func ReadPipeline(path, profile string) (*configkit.Result[PipelineConfig], error) {
    return configkit.Load(configkit.File(path), configkit.Options[PipelineConfig]{
        Definition: pipelineRules,
        Imports:    true,
        Profile:    profile,
    })
}

func ReadPipelineBytes(label string, data []byte) (*configkit.Result[PipelineConfig], error) {
    return configkit.Load(configkit.Bytes(label, data), configkit.Options[PipelineConfig]{
        Definition: pipelineRules,
    })
}

Map entries are copied out and written back using ordinary Go code; helpers audit the final projection. Independent map traversal order must not affect sorted records or diagnostics. Interdependent application transformations must use an application-defined deterministic order. Campaign/session fallback can similarly remain application code; this example does not turn it into ConfigKit composition.

A validator can report both sides of a duplicate without constructing source locations. This example uses sort to make primary-path selection deterministic:

func validateUniqueOutputs(cfg *PipelineConfig, v *configkit.Validation) {
    names := make([]string, 0, len(cfg.Artifacts))
    for name := range cfg.Artifacts {
        names = append(names, name)
    }
    sort.Strings(names)
    seen := make(map[string]configkit.Path)
    for _, name := range names {
        output := cfg.Artifacts[name].Output
        if output == "" {
            continue // The ordinary field validator reports this separately.
        }
        path := configkit.Keys("artifacts", name, "output")
        if first, exists := seen[output]; exists {
            v.Error(path, "output must be unique", first)
        } else {
            seen[output] = path
        }
    }
}

Call this helper from the application's validator when output uniqueness is a domain requirement. The message identifies the relationship without printing potentially sensitive configured values.

Schema checking and secret classification

type VersionRule struct {
    Path  Path
    Check func(Version, *Validation) error
}

type Version struct {
    Present bool
    Null    bool
    Text    string
}

The version checker receives the main document's scalar at the declared path, with absent/null states preserved. Non-string, non-null values are schema-version errors before the hook; the application decides whether absent/null or a string is acceptable. The checker reports actionable safe messages through its validation context, using the same primary/related-path API as other hooks. At this pre-decode phase, locations resolve against the composed input rather than a typed schema; a missing path falls back to the nearest input container. No parser tree is exposed. Reported errors or a non-nil return fail the load. A non-nil return with no reported error produces a generic safe version-check failure at the declared version path; raw error text is not printed. Placement and timing follow schema versioning. A rule with no checker is an options error.

For example, an application can preserve useful migration guidance without embedding input values in its error text:

var schemaVersionPath = configkit.Keys("schema_version")

func checkVersion(version configkit.Version, v *configkit.Validation) error {
    if !version.Present || version.Null {
        v.Error(schemaVersionPath, "schema_version is required")
    } else if version.Text != "example.v1" {
        v.Error(schemaVersionPath, "unsupported schema version; migrate using the application configuration guide")
    }
    return nil
}

var versionRule = configkit.VersionRule{
    Path:  schemaVersionPath,
    Check: checkVersion,
}

Assign &versionRule to a definition's Version field to enable it. Schema versions remain strings under ConfigKit's convention; adopting consumers with integer versions must migrate their syntax rather than enable a second scalar policy.

Definition.Secret is an optional pure path predicate for dynamic entries, combined with secret field tags. It can add sensitivity but cannot override a tag to declassify a value. It is applied to historical as well as final paths. Recorded dependencies propagate sensitivity under the existing redaction rules; applications own secret-producing transformations not described by those records.

Results, errors, and inspection

type Result[T any] struct {
    Config T
    // Private snapshot and metadata storage.
}

func (r *Result[T]) Metadata() Metadata
func (r *Result[T]) Sources() []SourceInfo
func (r *Result[T]) Diagnostics() []Diagnostic
func (r *Result[T]) WriteEffective(output io.Writer) error

type LoadError struct {
    // Private diagnostic and source storage.
}

func (e *LoadError) Error() string
func (e *LoadError) Diagnostics() []Diagnostic
func (e *LoadError) Sources() []SourceInfo
func WriteDiagnostics(output io.Writer, diagnostics []Diagnostic) error

Every configuration/source/options/hook failure returns (nil, *LoadError). Success returns a non-nil result and nil error; its diagnostics contain only nonfatal warnings. Source lists use stable source identifiers and safe labels; diagnostics expose code, severity, path, message, primary location, and related locations as described above. No raw parser error or unrestricted secret-bearing history is exposed. No automatic logging occurs. Inspection methods returning slices provide defensive copies.

Config is caller-owned typed data. Metadata and rendering use an independently captured final snapshot, so later caller mutation cannot rewrite the recorded result. WriteEffective emits deterministic redacted YAML for inspection, with no unredacted switch. It returns output errors rather than a new configuration validation result. Custom projection failures must already have been rejected before a load succeeds. Output can be partial on writer failure.

An application can report errors and inspect a successful load without rebuilding diagnostic formatting. This example uses errors and io and the pipeline loader above:

func InspectPipeline(path, profile string, output io.Writer) error {
    result, err := ReadPipeline(path, profile)
    if err != nil {
        var loadErr *configkit.LoadError
        if errors.As(err, &loadErr) {
            if writeErr := configkit.WriteDiagnostics(output, loadErr.Diagnostics()); writeErr != nil {
                return writeErr
            }
        }
        return err
    }
    if err := configkit.WriteDiagnostics(output, result.Diagnostics()); err != nil {
        return err
    }
    return result.WriteEffective(output)
}

Applications can use Metadata().Lookup, Metadata().History, and Sources() for structured inspection without implementing value rendering or importing a parser. The explanation API below supplies safe values and formatting for application-owned config explain commands.

Historical-value explanation

func (r *Result[T]) Explain(path Path) (Explanation, bool)
func (r *Result[T]) WriteExplanation(output io.Writer, path Path) error

type Explanation struct {
    Path Path
    Effective ValueSnapshot
    Origin Origin
    History []ValueChange
    SecretReference *SecretReferenceInfo
}
type ValueSnapshot struct {
    Present bool
    Redacted bool
    YAML string
}
type ValueChange struct {
    Before, After ValueSnapshot
    Origin Origin
    Reasons []string
    Dependencies []Path
}
type SecretReferenceInfo struct { Name string } // successful resolved lookup

Snapshots contain detached deterministic YAML text, never any, parser objects, or mutable internal trees. Present=false is absence/deletion and has empty YAML; null is present with null YAML. Redacted snapshots use the quoted marker [REDACTED]; Redacted also marks containers containing redacted descendants. Apply the same historical sensitivity/dependency closure as effective rendering before exposing any snapshots. A Secret projects only its environment name; its resolved value is never captured, even privately as history. For classified reference paths omit SecretReference as well as masking the projected name.

Retain private persistent node snapshots or operation deltas with structural sharing; do not copy the entire tree for every changed leaf. Explain a known final or historical path, including removed paths; unknown paths return false, and the writer returns a safe unknown-path error without output. Malformed query paths also fail safely. Render the canonical JSON-pointer path, effective value/source, and chronological history: initial application-default (or Go-zero baseline), base contributions in traversal order, profile, CLI, and recorded domain changes. Include before/after values for changed or explicitly redefined paths, equal-value writes, removals, reasons/dependencies, and composite origins. Inputs overridden before decoding retain policy-validated node values even when they would not fit the final Go type. Defaults/final/domain snapshots use canonical typed projection; label those representations rather than claiming every historical input was typed.

Container queries reconstruct the value at each relevant operation; descendant queries include ancestor replacement/removal and whole-sequence replacements, without implying element-wise sequence merging. Omitted fields/default retention must agree with the presence/default contract; do not fabricate a write for every import that leaves a path alone. Resolution history records the audited net change per path, not every intermediate Go assignment. Lookup of a secret reference annotates its effective name as resolved from the process environment; bootstrap file outcomes come from the separate report and are never guessed.

WriteExplanation is a deterministic human renderer over this safe structured result. Escape control characters in paths, labels, names, and reasons, propagate writer errors/short writes, and permit partial output only on writer failure. No unredacted switch or bundled command is provided. For example an application can call it for /transcription/concurrency to show default 4, an explicitly imported file's 8, then a testing profile's 1, with file line numbers and the profile as effective source. This does not require automatic conf.d discovery.

Module-specific options

Use a canonical object entry with an explicit module identifier and options mapping. Shorthand strings and custom structural decoders are not required to express the domain information. For a known application module catalog, model the available option fields with ordinary typed structs; application validation selects the meaningful fields and rejects incompatible combinations.

Illustrative YAML for an application that supports two modules:

modules:
  - module: chunker
    options:
      window_units: 12
  - module: discussion
    options:
      sections: [short_term]

The corresponding typed configuration and validator need no second parser:

type ModuleOptions struct {
    WindowUnits *int     `yaml:"window_units"`
    Sections    []string `yaml:"sections"`
}

type ModuleSpec struct {
    Module  string        `yaml:"module"`
    Options ModuleOptions `yaml:"options"`
}

type ModuleConfig struct {
    Modules []ModuleSpec `yaml:"modules"`
}

func validateModules(cfg *ModuleConfig, v *configkit.Validation) {
    for index, spec := range cfg.Modules {
        path := configkit.Keys("modules").Index(index)
        switch spec.Module {
        case "chunker":
            if spec.Options.WindowUnits == nil || *spec.Options.WindowUnits < 1 {
                v.Error(path.Key("options").Key("window_units"), "must be a positive integer")
            }
            if spec.Options.Sections != nil {
                v.Error(path.Key("options").Key("sections"), "not supported by this module")
            }
        case "discussion":
            if spec.Options.WindowUnits != nil {
                v.Error(path.Key("options").Key("window_units"), "not supported by this module")
            }
            if len(spec.Options.Sections) == 0 {
                v.Error(path.Key("options").Key("sections"), "must contain at least one section")
            }
        default:
            v.Error(path.Key("module"), "unsupported module")
        }
    }
}

func ReadModuleConfig(path string) (*configkit.Result[ModuleConfig], error) {
    return configkit.Load(configkit.File(path), configkit.Options[ModuleConfig]{
        Definition: configkit.Definition[ModuleConfig]{Validate: validateModules},
    })
}

The pointer and nil-slice states above express optional domain values; this application treats explicit null as unselected for those optional alternatives. ConfigKit rejects unknown fields and invalid scalar types. The application owns which supported options apply to a module and their domain constraints. Larger catalogs may use nested typed option objects rather than one flat options struct. Neither shape requires a library registry, parser hook, or configurable merge rule.

For an application whose option schema is genuinely selected at runtime, map[string]any remains available. ConfigKit owns parsing, paths, and the single dynamic scalar representation; application validation owns the selected schema, including allowed keys and value constraints. Conversion into runtime module arguments is application code. Dynamic integers remain checked int64; existing int assertions must be adapted with range checks when converting to int.

The initial API does not add a generic subtree-conversion helper. Prefer typed configuration where option shapes are known, and direct domain validation where they are dynamic. Do not introduce a second YAML marshal/decode round trip as the blessed pattern. Reconsider a narrow conversion helper only if a concrete dynamic catalog demonstrates substantial duplicated type-binding infrastructure that ordinary models cannot remove. Such a helper would have to retain strictness and source attribution; that is a separate justified extension, not an escape hatch for legacy shorthand. Configuration inspection describes the validated input model, not the application's separately constructed runtime module objects.

Supported types and projection

The supported root T is a struct or string-keyed map. Within it, support ordinary exported scalar fields, pointers, structs, string-keyed maps, slices, fixed-length arrays, dynamic any, and the explicitly supported scalar conversions. Reject unsupported kinds, non-string map keys, cyclic object graphs, and ambiguous field bindings as schema/projection errors. Arrays require matching sequence length. yaml:"-" fields and unexported fields are outside the configuration projection; hooks must not use them to carry runtime resources or configuration needing inspection. Non-configuration runtime state belongs in separate application types.

One field-binding model supplies defaults, decoding, path lookup, change auditing, and rendering. omitempty never removes projected zero values. Preserve nil versus empty maps/slices where represented by the type. Shared pointer identity inside application values is not a configuration semantic; cyclic values are unsupported and independent load/snapshot ownership must still hold.

time.Duration projects to a canonical duration string. A custom scalar's text marshal method supplies its deterministic canonical string; unmarshalling must accept that representation and preserve its configuration meaning. These methods must be side-effect-free with respect to shared state. Custom scalars are atomic for comparison/history; their internal fields are not traversed. Reject types with only a text decoder and no supported inspection representation. Conversion or projection failures produce safe diagnostics. This is a bounded scalar seam, not a custom structural decoder or arbitrary reflection extension framework.

Internal organization

Keep document composition, overlays, and parser machinery internal initially. Applications use the typed lifecycle; application-specific resolution alone does not justify exposing a document-processing toolkit.

Internally separate source access, policy-validated documents, composition, overlay, field binding/decoding, metadata, resolution, and safe rendering. These are responsibilities, not a mandate for public interfaces or a file per concept. Start with one public package and a private YAML adapter; add package boundaries only when dependencies and ownership warrant them.

Use standard-library filesystem, I/O, scalar conversion, reflection, ordering, and synchronization primitives. ConfigKit owns the policy that coordinates them: field binding, presence/default behavior, strict composition, overlays, provenance, resolution auditing, and secret propagation. One descriptor graph and document model support those operations; do not add competing binding or copying models. Convenience functions must not broaden accepted scalar syntax, silently truncate input, lose nested ownership, or substitute Go field promotion for YAML inlining.

The implementation plan owns the selected primitives and their integration checks. Test comparison uses go-cmp only in tests, never as the runtime resolution comparator. Additional dependencies remain case-by-case decisions based on code saved after accounting for adaptation, validation, and recurring maintenance.

YAML dependency and public API isolation

Use the YAML organization's go.yaml.in/yaml/v4 package behind internal/yaml. V4 is upstream's active development line; v3 receives security fixes only. This choice aligns a new library with ongoing upstream maintenance while keeping ConfigKit responsible for its own configuration model. See upstream's version policy. The implementation decisions own the exact selected release-candidate pin and upgrade validation requirements.

The adapter owns all direct upstream imports, including parser plugins, loading, and emission. Its boundary uses ConfigKit-owned document and diagnostic records. No upstream nodes, type aliases, options, plugin interfaces, error types, or generic constraints form part of ConfigKit's public declarations or callbacks. ConfigKit must not return parser-created values hidden inside any, metadata, or an error's unwrap chain. Applications use ordinary Go types, ConfigKit types, and the supported standard-library text interfaces; they need no YAML-package imports or upstream version presets to use ConfigKit.

The existing yaml field-tag spelling is a ConfigKit-owned binding contract, not delegation to upstream reflection or marshal/unmarshal hooks. ConfigKit also owns scalar interpretation, duplicate detection, document count, source-coordinate conventions, diagnostic classifications, and effective-output formatting. Upstream options may help enforce these contracts internally but cannot redefine them.

An upstream breaking API change must be absorbed in the adapter or other private implementation, preserving downstream Go source compatibility and documented configuration behavior. Parser upgrades do not independently authorize changes to ConfigKit's public contracts. If an upstream version cannot preserve those contracts, retain the compatible dependency or adapt the implementation; any intentional ConfigKit contract change follows ConfigKit's own release policy.

This boundary isolates API coupling, not every dependency risk: the YAML module remains a transitive build dependency, and parser defects or behavior changes still require ConfigKit regression checks. It does not require a public parser interface, configurable backends, or simultaneous v3/v4 implementations.

Explicit non-goals

General environment-variable configuration overlays are not on the roadmap. Secret references and explicit bootstrap are the supported environment convention. Literal-secret configuration and implicit secret-directory discovery are unsupported.

Deferred scope

The initial target excludes:

automatic conf.d discovery;
remote imports, arbitrary providers, and additional configuration formats;
custom merge strategies, sequence merging, reset operators, and profile inheritance;
parser-specific or custom structural decoding extension APIs;
generic dynamic-subtree conversion and module-registry APIs;
automatic schema migrations;
hot reload, runtime resource replacement, and remote resource validation;
a bundled CLI framework or commands;
config diffs and fingerprints.

Applications can build their own validate/effective/sources commands on the library. Broadening these boundaries requires concrete consumer evidence and an explicit design change.

77 KiB Raw Blame History