1259 lines
67 KiB
Markdown
1259 lines
67 KiB
Markdown
# ConfigKit — Initial Target Design
|
|
|
|
## Status and ownership
|
|
|
|
This roadmap describes the targeted end-state for ConfigKit's initial release,
|
|
provisionally `v0.1.0`. It is future design, not a claim of implemented behavior
|
|
or a stable v1 compatibility promise. Audience: maintainers and implementation
|
|
agents.
|
|
|
|
This document owns target behavior, boundaries, and deferred scope. The
|
|
[implementation plan](implementation.md) owns stages, dependencies, acceptance
|
|
gates, and progress. Do not add implementation steps or stage status here.
|
|
|
|
The [architecture policy](../policy/architecture.md) owns binding contributor
|
|
constraints. Accepted rationale is recorded in ADRs for
|
|
[composition versus overlay](../adr/0001-separate-composition-and-overlay.md),
|
|
[parser independence](../adr/0002-own-yaml-semantics.md), and
|
|
[presence and resolution history](../adr/0003-preserve-presence-and-resolution-history.md)
|
|
under the [documentation policy](../policy/documentation.md). As features land,
|
|
move their normative behavior to current contracts and replace the corresponding roadmap
|
|
text with links. Retire completed roadmap material during release preparation;
|
|
these files are not permanent duplicate specifications.
|
|
|
|
## Purpose and principles
|
|
|
|
ConfigKit provides a consistent configuration lifecycle for Go applications
|
|
using YAML, including Seriatim, Audita, Narratio, WeatherFeeder, WeatherReporter,
|
|
and Distributor. Its goal is one deterministic, explainable resolution model
|
|
that works for a single file and scales to explicitly imported fragments.
|
|
|
|
- YAML supplies syntax; ConfigKit owns composition, precedence, presence,
|
|
conversion, and error policy.
|
|
- Preserve a source-located tree through composition and overlay. Do not first
|
|
collapse input into `map[string]any` or application structs.
|
|
- Keep application types ordinary. Pointers express application nullability,
|
|
not a universal workaround for distinguishing absent from false or zero.
|
|
- Separate strict composition from intentional overlay.
|
|
- Keep the public API small and the parser private. Avoid a general provider or
|
|
merge-strategy framework.
|
|
|
|
### Success criteria
|
|
|
|
1. **Reduce application infrastructure:** Can applications delete substantial
|
|
configuration infrastructure while retaining clear ownership of their domain
|
|
rules?
|
|
2. **Unify configuration conventions:** Do consumers adopt one consistent
|
|
configuration model, with only the flexibility needed to express genuine
|
|
application requirements?
|
|
|
|
The [architecture policy](../policy/architecture.md#success-criteria) owns these
|
|
coequal criteria and their governing rules. The target must satisfy both: the
|
|
shared lifecycle removes configuration plumbing, while common syntax, presence,
|
|
composition, precedence, and decoding rules establish consistent conventions.
|
|
Application types, interpretation, defaults, and semantic validation retain the
|
|
flexibility needed to describe domain requirements.
|
|
|
|
Consumer migrations are expected, including intentional configuration syntax
|
|
changes. Existing code and tests are extraction foundations and evidence of needs,
|
|
not compatibility obligations for ConfigKit. Prefer canonical object forms over
|
|
consumer-specific shorthand where they express the same information. Do not add
|
|
alternate parsers or merge rules merely to preserve an existing convention.
|
|
|
|
Evaluate an extension by identifying its concrete domain requirement, explaining
|
|
why the existing primitives cannot accommodate it cleanly, and showing how it
|
|
preserves one coherent model. If application code must remain, distinguish domain
|
|
interpretation and validation from duplicated configuration infrastructure. A
|
|
restriction that makes useful information inexpressible is not justified by
|
|
consistency alone.
|
|
|
|
The consumer examples and adoption work must demonstrate both reduced plumbing
|
|
and convergence on the model. Narratio extraction and simpler consumers should
|
|
supply concrete evidence; feature count, unchanged legacy syntax, and fewer lines
|
|
inside ConfigKit are not measures of success. Detailed consumer migrations and
|
|
execution steps belong in the [implementation plan](implementation.md).
|
|
|
|
## Responsibility boundary
|
|
|
|
| ConfigKit owns | Applications own |
|
|
| --- | --- |
|
|
| Local file and labeled bytes/reader inputs | Main-file search paths, bootstrap timing, and CLI parsing/wiring |
|
|
| YAML policy, explicit import traversal, composition, profiles, and explicit CLI overrides | Configuration types and field meanings |
|
|
| Presence, source locations, resolution records, and safe diagnostics | Static defaults, derived defaults, and normalization rules |
|
|
| Secret-reference resolution and explicit directory bootstrap | Credential use and process startup coordination |
|
|
| Strict typed decoding and lifecycle coordination | Semantic validation and schema-version policy |
|
|
| Source reporting, safe history/explanation, and redacted effective output | Remote acquisition, migrations, runtime initialization, and resource checks |
|
|
|
|
Applications may fetch remote data and pass labeled bytes to ConfigKit. This
|
|
provides useful provenance without making ConfigKit responsible for network
|
|
availability, credentials, or caching. Narratio's pipeline/campaign/session
|
|
fallback rules remain application semantics; the extraction boundary is described
|
|
below.
|
|
|
|
## Existing implementation foundation
|
|
|
|
Narratio is the primary extraction source and intended first substantial consumer
|
|
of ConfigKit. Prefer copying and generalizing applicable existing code and tests
|
|
over independently rebuilding equivalent mechanisms. Reuse includes the behavior
|
|
captured by Narratio's tests, not just production code. ConfigKit's target
|
|
contracts govern the result; reuse does not require preserving incidental
|
|
application structure or incompatible behavior.
|
|
|
|
The references below were inspected on 2026-09-06 at Narratio commit
|
|
`b68435128be1e24b94fc7ec259d6e9093669fd99`. Paths refer to the sibling checkout
|
|
`../narratio` relative to the ConfigKit repository root; the links require that
|
|
checkout layout. They are extraction references, not a runtime or build dependency.
|
|
Recheck them against the source revision actually used for extraction.
|
|
|
|
| Foundation | Source references | Intended treatment |
|
|
| --- | --- | --- |
|
|
| File and labeled reader/bytes loading; strict field checking | [load.go](../../../narratio/internal/config/load.go): `decodeStrictYAML`, `decodeStrictYAMLFromReader`, `LoadSessionBytesWithOptions` | Extract reusable input handling and generalize decoding behind ConfigKit's private parser boundary |
|
|
| Node parsing, strict composition, overlays, and canonical rendering | [composition.go](../../../narratio/internal/config/composition.go): `parseCompositionDocument`, `buildCompositionNode`, `mergeAdditiveCompositions`, `mergeOverlayComposition`, `canonicalYAML` | Extract algorithms and tests; replace upstream node kinds, path spelling, scalar semantics, and incompatible merge rules |
|
|
| Explicit imports and profile selection | [pipeline_composition.go](../../../narratio/internal/config/pipeline_composition.go): `loadPipelineCompositionSources`, `loadPipelineImports`, `loadPipelineProfileOverlays`, `selectPipelineProfile` | Reuse traversal/selection mechanics with ConfigKit's recursive graph, containment, directive syntax, and fixed selected-profile convention |
|
|
| Effective value/source inspection | [effective_pipeline.go](../../../narratio/internal/config/effective_pipeline.go): `EffectivePipelineValues`, `EffectivePipelineSources`, `MarshalEffectivePipeline`; [config_commands.go](../../../narratio/internal/app/config_commands.go) | Reuse deterministic projection, ownership, and side-effect-free inspection cases; retain application commands and generated-domain attribution in Narratio |
|
|
| Explicit field presence | [normalize.go](../../../narratio/internal/config/normalize.go): `NormalizeConfig.UnmarshalYAML` | Generalize the requirement into library metadata; replace the application-specific parser hook |
|
|
| Default application | [load.go](../../../narratio/internal/config/load.go): `applyPipelineDefaults` and related helpers; [defaults.go](../../../narratio/internal/config/defaults.go) | Reuse applicable mechanics and behavioral cases while keeping actual default values and domain rules application-owned |
|
|
| Resolution and source attribution | [load.go](../../../narratio/internal/config/load.go): `Resolve`, `mergeCampaignSession`, `selectStableInput`; [config.go](../../../narratio/internal/config/config.go): `ResolvedInputFile`, `SessionSource` | Adapt source tracking into generic provenance; retain campaign/session fallback and domain types in Narratio |
|
|
| Application-supplied remote bytes | [config_loader.go](../../../narratio/internal/app/config_loader.go) | Use as evidence for labeled input support; keep fetching, storage clients, and remote metadata acquisition outside ConfigKit |
|
|
| Existing behavioral coverage | [normalize_test.go](../../../narratio/internal/config/normalize_test.go), [load_validate_test.go](../../../narratio/internal/config/load_validate_test.go), [campaign_config_test.go](../../../narratio/internal/config/campaign_config_test.go) | Extract relevant cases and minimal fixtures; preserve domain-specific assertions in consumer tests |
|
|
|
|
Additional extraction coverage includes [composition tests](../../../narratio/internal/config/composition_test.go),
|
|
[import tests](../../../narratio/internal/config/pipeline_composition_test.go),
|
|
[profile tests](../../../narratio/internal/config/pipeline_profiles_test.go),
|
|
[strict-loader tests](../../../narratio/internal/config/loader_strict_test.go), and
|
|
[inspection tests](../../../narratio/internal/app/config_commands_test.go).
|
|
These are reusable behavior evidence, not an instruction to preserve every assertion.
|
|
|
|
### Compatibility and adaptation
|
|
|
|
Extraction must distinguish preserved behavior, intentionally changed behavior,
|
|
and application-only behavior. Record the source revision and relevant symbols
|
|
with the extraction work so reviewers can compare the implementation and tests.
|
|
Generalized code must not import Narratio's internal packages or copy its domain
|
|
model into ConfigKit's public API.
|
|
|
|
Known adaptation points include:
|
|
|
|
- Some Narratio defaulting helpers use an empty string to trigger a default.
|
|
Adapt these to ConfigKit's [presence and defaults rules](#static-defaults-and-null)
|
|
rather than losing explicitly configured empty values.
|
|
- `decodeStrictYAMLFromReader` now rejects trailing documents. Preserve and
|
|
generalize its strict-loader regressions under ConfigKit's single-document policy.
|
|
- The custom normalization decoder and pointer-based presence conventions need
|
|
to give way to generic metadata while preserving genuine domain nullability.
|
|
- Existing composition nodes and effective-field ownership are substantial
|
|
foundations, but do not retain the full line/column and historical-value records
|
|
ConfigKit requires. Keep application-generated family/campaign attribution as
|
|
domain resolution with recorded dependencies.
|
|
- Narratio rejects overlapping empty mappings, null overlays, and cross-kind
|
|
overlays. ConfigKit's operation table deliberately permits those cases. Adapt
|
|
the implementation and expectations together.
|
|
- Narratio declares import paths and profiles in a root `composition` envelope,
|
|
supports a declared default profile, and loads every declared overlay. ConfigKit
|
|
uses `_configkit.imports`, recursive imports/globs, and only the selected fixed
|
|
profile path; `default` means no overlay. Do not preserve an alternate convention.
|
|
- Narratio uses `os.SameFile` to reject filesystem aliases. ConfigKit uses canonical
|
|
path identity, treats hard links as distinct paths, and enforces root containment.
|
|
- Narratio's owned tree still embeds upstream YAML kinds/tags and dotted/indexed
|
|
paths. Its final binding and effective inspection reserialize/reparse YAML.
|
|
Reuse algorithms without importing those dependencies into ConfigKit's document
|
|
model or replacing direct descriptor-driven decoding and projection.
|
|
- Narratio's configuration diffs and digests are useful consumer evidence, but
|
|
remain outside ConfigKit's initial scope. Their implementation does not supply
|
|
full historical-value explanation or authorize a new public API.
|
|
|
|
Each intentional behavior change needs focused tests and an explicit migration
|
|
account for the affected consumer. Do not copy an old expectation merely because
|
|
it passes in Narratio, or discard a useful case because the implementation moves.
|
|
Use representative Narratio configurations to verify preserved outcomes and
|
|
identify intentional differences. Exercise a simpler consumer as well so the
|
|
public API remains practical for single-document applications.
|
|
|
|
### Capabilities requiring broader implementation
|
|
|
|
The inspected Narratio loader supplies explicit flat imports, strict composition,
|
|
profile overlays, source ownership, and effective inspection in addition to its
|
|
pipeline/campaign/session domain coordination. Prefer extracting those foundations
|
|
rather than rebuilding equivalent mechanisms.
|
|
|
|
ConfigKit still requires new or substantially revised work for recursive import
|
|
graphs and globs, bounded canonical containment, its conservative YAML policy,
|
|
schema-aware defaults/direct decoding, comprehensive location and value history,
|
|
recorded domain resolution, generic secret classification, and safe structured
|
|
reporting. Existing provenance is a starting point, not full explainability.
|
|
|
|
Automatic `conf.d` discovery remains deferred. Existing application relationships
|
|
and declared profile paths do not redefine ConfigKit's import/profile conventions.
|
|
The [implementation plan](implementation.md) owns extraction tasks and ordering.
|
|
|
|
## Consumer fit and intentional migrations
|
|
|
|
These are adoption targets, not promises of unchanged consumer behavior or
|
|
completed integrations. Source links refer to sibling checkouts as in the
|
|
Narratio foundation section. The findings reflect the inspected implementations
|
|
on 2026-09-06 and must be rechecked at migration time.
|
|
|
|
| Consumer and evidence | Useful ConfigKit fit | Intentional migration or retained application responsibility |
|
|
| --- | --- | --- |
|
|
| Narratio: [configuration loader](../../../narratio/internal/config/load.go) | Shared loading, presence, defaults coordination, and source attribution | Replace ad hoc presence/empty-value defaults; retain campaign/session domain resolution |
|
|
| Notarius: [file model](../../../notarius/internal/core/config/file_config.go), [environment handling](../../../notarius/internal/core/config/env.go) | Strict nested models, named pipelines, presence-aware values, and diagnostics | Use canonical object bindings/references and string schema versions; adapt dynamic integer consumers to `int64`; migrate configuration-field environment overrides to files/profiles; keep catalog resolution and runtime invocation choices application-owned |
|
|
| WeatherReporter: [loader](../../../weatherreporter/internal/config/load.go), [module options](../../../weatherreporter/internal/config/reports.go) | Defaults-only loading, durations, report maps, list presence, and typed module options | Replace string shorthand with object entries and private presence flags with metadata; translate explicitly set configuration flags into ConfigKit overrides; reuse secrets-directory loading through explicit ConfigKit bootstrap |
|
|
| Seriatim: [configuration surfaces](../../../seriatim/docs/config.md), [speaker rules](../../../seriatim/internal/speaker/map.go), [autocorrect rules](../../../seriatim/internal/autocorrect/autocorrect.go) | Simple typed loading of ordered YAML rule files, normalization, and cross-rule diagnostics | Adopt strict YAML rejection deliberately; retain matching/replacement algorithms, resource checks, and CLI/environment runtime settings; no central application YAML file is required |
|
|
|
|
WeatherReporter's updated [configuration tests](../../../weatherreporter/internal/config/config_test.go)
|
|
and [external constructed-config tests](../../../weatherreporter/internal/config/overrides_external_test.go)
|
|
cover explicit empty lists, consistent loaded/constructed behavior, and normalization
|
|
without mutating source options. Preserve these domain outcomes through ConfigKit's
|
|
presence, detached ownership, and application validation. ConfigKit permits an empty
|
|
list structurally; a particular consumer may reject it semantically.
|
|
|
|
The initial model standardizes syntax and resolution while preserving application
|
|
field meanings. Module shorthand and integer schema versions are syntax migration
|
|
issues; module-specific option schemas are genuine domain variation addressed
|
|
[below](#module-specific-options). ConfigKit configuration profiles are overlays,
|
|
not substitutes for application-owned LLM or pipeline profile concepts.
|
|
|
|
## External values and invocation state
|
|
|
|
Configuration-field CLI flags use the explicit [override API](#cli-overrides).
|
|
Applications own parsing flags and selecting sources/profiles. Separate invocation
|
|
state, such as an input artifact or execution lane, remains application-owned.
|
|
General environment-variable configuration overlays are an explicit non-goal,
|
|
not deferred work. Environment access is reserved for named secret references.
|
|
|
|
Do not inject hidden overrides through captured environment/CLI values in default
|
|
factories, resolver hooks, or validators. Domain resolution follows the declared
|
|
input layers. Mutating `Result.Config` cannot update a validated snapshot.
|
|
|
|
## Secret references and explicit bootstrap
|
|
|
|
Secret-bearing configuration fields use ConfigKit's opaque `Secret` type. Their
|
|
YAML value is an environment-variable name, for example `api_key: OPENROUTER_API_KEY`.
|
|
Names are public configuration; resolved credentials are not configuration-tree
|
|
values. Literal-secret input, interpolation, fallback literal values, dotenv
|
|
parsing, and implicit directory discovery are unsupported. Names must match
|
|
`[A-Za-z_][A-Za-z0-9_]*`. A string that happens to be a valid name is always a
|
|
reference; ConfigKit cannot detect credentials mistakenly written in other fields.
|
|
|
|
```go
|
|
func SecretEnv(name string) Secret
|
|
func (s Secret) EnvName() string
|
|
func (s Secret) Value() string
|
|
func (s Secret) IsResolved() bool
|
|
```
|
|
|
|
`SecretEnv` creates an unresolved reference; invalid names fail the containing
|
|
load. The zero value is an absent reference, never a literal secret. A non-pointer
|
|
`Secret` is required: absent, null, empty/invalid name, unset environment variable,
|
|
or an empty environment value fails with a safe `secret.*` diagnostic. Use
|
|
`*Secret` for optional credentials: nil is allowed, but every non-nil reference
|
|
must resolve to a nonempty value. Applications validate conditional requirements.
|
|
Only typed fields/maps/slices establish secret-reference semantics; dynamic `any`
|
|
strings are ordinary values and must migrate to typed secret slots when needed.
|
|
|
|
Recognize `Secret` before ordinary struct/text binding. Projection, comparison,
|
|
cloning for defaults, and resolution auditing operate on the name only. Resolve
|
|
the surviving references after recorded application resolution and before semantic
|
|
validation, using one `os.LookupEnv` result per unique name per load. A copied
|
|
previous result used as defaults is re-resolved; it cannot carry stale credentials.
|
|
Validation can use `Value()`; resolution works with names. In addition to ordinary
|
|
projection auditing, reject validator changes to a secret's private resolved state
|
|
by comparing it with this load's lookup result. A same-name unresolved replacement
|
|
must not evade the read-only validator contract. `Value()` returns empty
|
|
on unresolved references; `IsResolved()` distinguishes that state. Private credential
|
|
storage is excluded from every tree, historical snapshot, diagnostic, default
|
|
format, JSON/text serialization, and inspection path. Provide safe formatting for
|
|
all fmt verbs and name-only JSON marshaling; expose no credential decoder/codec.
|
|
Only the deliberately named `Value()` accessor returns a credential. Applications
|
|
own its use and must not embed it in hook messages. Secret references themselves
|
|
need no `configkit:"secret"` tag: the existing tag/predicate classifies additional
|
|
sensitive or derived values and can intentionally hide a reference name.
|
|
|
|
The separate, opt-in bootstrap operation below loads a directory into the process
|
|
environment. `Load` never calls it or changes process-global state. Applications
|
|
choose a directory and call bootstrap once during startup before concurrent work.
|
|
Bootstrap paths are native application-selected paths, independent of import roots.
|
|
An empty or missing directory path is an error; an existing empty directory succeeds.
|
|
|
|
```go
|
|
func LoadSecretsDir(directory string) (*SecretsReport, error)
|
|
func (r *SecretsReport) Entries() []SecretFileResult
|
|
func (r *SecretsReport) WriteTo(output io.Writer) (int64, error)
|
|
type SecretsError struct { /* private safe diagnostic storage */ }
|
|
func (e *SecretsError) Error() string
|
|
func (e *SecretsError) Diagnostics() []Diagnostic
|
|
type SecretFileResult struct {
|
|
Name, Path string
|
|
Status SecretFileStatus
|
|
ExistingEmpty bool
|
|
}
|
|
type SecretFileStatus string
|
|
// Constants use values "loaded", "skipped_existing", "not_applied", "failed":
|
|
// SecretFileLoaded, SecretFileSkippedExisting, SecretFileNotApplied, SecretFileFailed.
|
|
```
|
|
|
|
Extract applicable code and tests directly from WeatherReporter's
|
|
[secrets loader](../../../weatherreporter/internal/config/secrets.go), inspected
|
|
at revision `5d306a202471e10fe98923a1df27d0b0fce5c666`. Reuse its filename checks,
|
|
file-type rejection, single terminal LF/CRLF removal, and separation of `loadSecrets`
|
|
from `applySecrets`. WeatherReporter now preloads file contents, validates its
|
|
configuration, then applies environment writes with attempted rollback. ConfigKit
|
|
preserves existing variables, reports partial apply failure without rollback, and
|
|
keeps bootstrap separate from configuration loading. Record extraction symbols,
|
|
revision and adaptations during implementation.
|
|
|
|
Narratio's [secrets loader](../../../narratio/internal/app/secrets_env.go) at the
|
|
revision recorded above is a complementary foundation: `loadSecretsFromConfigWithDependencies`
|
|
preserves existing variables including empty ones, and `readAPIKeyFile` uses
|
|
[bounded reads](../../../narratio/internal/fileops/read_bounded.go). Its
|
|
[tests](../../../narratio/internal/app/secrets_env_test.go) provide reusable failure
|
|
and environment-preservation cases. Adapt its immediate writes and aggregate/log
|
|
reporting to ConfigKit's preflight and per-file structured report. Narratio skips
|
|
invalid names, removes all trailing CR/LF characters, rejects a symlink at the
|
|
selected directory path, and enforces restrictive permission bits. Those policies
|
|
are not inherited: use ConfigKit's name/type/newline/directory rules and fixed
|
|
bounds. ConfigKit does not add a permission-mode gate in the initial API;
|
|
applications own deployment permissions. Extract bounded-read mechanisms without
|
|
copying Narratio's application dependency framework or logger.
|
|
|
|
Process only immediate entries, sorted by filename. Reject invalid names,
|
|
symlinks, directories, and nonregular files, including entries whose names are
|
|
already set. Allow a symlink in the explicitly selected directory path by resolving
|
|
it to a canonical directory first. Validate all entry metadata before mutation.
|
|
`os.LookupEnv` presence wins, including an empty existing variable: skip reading
|
|
that file's contents and report `SkippedExisting` with `ExistingEmpty` set
|
|
accurately. Read eligible files with bounds, remove exactly one terminal CRLF or
|
|
LF, preserve all other whitespace, and reject NUL bytes. Empty file values are
|
|
allowed at bootstrap, but fail if subsequently used by a required reference.
|
|
Preflight all eligible reads before any `os.Setenv`. No writes occur on preflight
|
|
failure. At apply time recheck presence before each write and skip newly set names.
|
|
Concurrent mutation is unsupported; this is not an atomic compare-and-set facility.
|
|
|
|
Return a non-nil report even on failure. Successful writes are `Loaded`, blocked
|
|
files are `SkippedExisting`, the failed entry is `Failed`, and eligible but unwritten
|
|
entries are `NotApplied`. Directory-level failures have a safe error without a
|
|
fabricated entry. Stop on the first deterministic failure; reports include all
|
|
entries enumerated so far, with unprocessed entries `NotApplied`. An apply failure
|
|
preserves earlier successful writes and reports partial progress; do not attempt
|
|
rollback that could overwrite unrelated process changes. Errors are `*SecretsError`
|
|
with safe `secrets.*` diagnostic codes and no raw unwrap chain. Errors and report
|
|
rendering contain names, paths, statuses, and safe classifications, never contents.
|
|
|
|
Report entries and diagnostic slices are defensive copies. Bootstrap diagnostics
|
|
include safely quoted canonical paths in messages with no invented configuration
|
|
source ID or source location.
|
|
Report formatting escapes control characters and propagates short writes/errors.
|
|
|
|
The report is the authoritative bootstrap provenance: a blocked file explicitly
|
|
identifies the existing process variable as the winner, even when empty. Keep it
|
|
alongside the load result and render it before configuration explanation in an
|
|
application command. Secret-resolution metadata records the variable name and
|
|
`resolved`, `unset`, or `empty` lookup outcome (failures in diagnostics). It does
|
|
not claim which external actor populated the process environment. Reports are
|
|
separate snapshots, not accepted as evidence of the environment's later state;
|
|
there is no implicit global report or stale filename attribution in `Load`.
|
|
|
|
## CLI overrides
|
|
|
|
Precedence is **application defaults < composed base < selected profile < explicit
|
|
CLI overrides**. Domain normalization follows those inputs and is recorded.
|
|
|
|
```go
|
|
type Override struct {
|
|
Path Path
|
|
Value any
|
|
Flag string
|
|
}
|
|
// Options[T].Overrides is []Override.
|
|
```
|
|
|
|
Applications provide records only for flags explicitly set by the caller; parser
|
|
defaults are not overrides. `Flag` is a required, non-secret display name, such as
|
|
`--workers`, never the raw argument or value. ConfigKit has no CLI parser dependency.
|
|
Values are detached and converted to ConfigKit-owned nodes before composition:
|
|
accept built-in string, bool, integer and unsigned integer widths, finite floats,
|
|
`nil`, `[]any`, and `map[string]any` recursively. Strings remain strings; do not
|
|
parse YAML fragments or coerce textual booleans/numbers. Durations and secret
|
|
references use strings. Reject other types, cycles, nonfinite numbers, and limits
|
|
violations with safe `override.*` diagnostics.
|
|
|
|
Validate every path/record before applying any. Reject root paths, sequence-index
|
|
traversal, duplicate paths, and ancestor/descendant overlaps within the override
|
|
list. Numeric tokens are mapping keys only here. Mapping-key paths may create
|
|
missing mapping ancestors, but cannot traverse an existing scalar/null/sequence.
|
|
The target path overlays with the same rules as profiles: mappings recursively
|
|
merge; sequences and all other replacements are atomic. Use a whole sequence
|
|
value to replace a list. Preserve supplied list order; mapping entries use canonical
|
|
key order. Unknown struct fields and effective type mismatches fail normal strict
|
|
decoding. Never allow `_configkit` directives or writes to the configured schema
|
|
version path, including ancestor replacement. Overrides also work with `NoSource()`.
|
|
|
|
Each record contributes a synthetic CLI source after file/profile sources, in
|
|
record order, labeled with `Flag`, with no file coordinates. Explicit false, zero,
|
|
empty string, null, and empty list remain present. Snapshot caller-owned values
|
|
before applying them. History includes replaced inputs even for equal-value writes.
|
|
Neither callers nor hooks may mutate supplied data concurrently with loading.
|
|
|
|
## Resolution lifecycle
|
|
|
|
```text
|
|
application selects source and loading options
|
|
→ parse and enforce YAML policy for every source
|
|
→ discover explicit imports and strictly compose the base
|
|
→ application schema-version check, when configured
|
|
→ apply the selected profile
|
|
→ apply explicit CLI overrides to the node tree
|
|
→ apply static defaults and strictly decode
|
|
→ apply recorded application resolution and normalization
|
|
→ resolve surviving secret references from the environment
|
|
→ collect application semantic validation diagnostics
|
|
→ return independently owned configuration and metadata
|
|
```
|
|
|
|
Value precedence is defaults < composed base < selected profile < CLI overrides. Resolution is
|
|
an application transformation after those layers, not another user override
|
|
source. Future migration support belongs after base composition/version
|
|
inspection and before profile application and typed decoding; migrations are
|
|
not part of the initial release.
|
|
|
|
YAML policy applies to every document even when a later layer would replace its
|
|
values. Schema/type checks apply to the effective input tree; application
|
|
validation runs only after successful decoding and resolution. An invalid
|
|
configuration is never returned as a successful, usable result.
|
|
|
|
## YAML and scalar policy
|
|
|
|
Source ownership, the supported YAML subset, scalar grammar, fixed input limits,
|
|
and safe failure behavior are implemented. The [current configuration input
|
|
contract](../config.md) now owns those definitions, including typed scalar
|
|
conversion through the current loading lifecycle.
|
|
|
|
## Paths and presence
|
|
|
|
Semantic path construction, JSON Pointer parsing, and layered metadata population
|
|
are implemented. Their current contracts are owned by the Go
|
|
documentation in [`path.go`](../../path.go), [`metadata.go`](../../metadata.go),
|
|
and the [current metadata contract](../config.md#layered-metadata), including
|
|
presence, composite origins, parent replacements, removed-path history, and
|
|
synthetic resolution operations.
|
|
|
|
## Composition and overlay
|
|
|
|
Strict base composition and selected-profile overlay are implemented and owned by
|
|
the [current composition contract](../config.md#strict-document-composition) and
|
|
[profile contract](../config.md#selected-profiles).
|
|
|
|
## Explicit imports and source identity
|
|
|
|
Explicit path and glob imports, recursive traversal, source identity, containment,
|
|
filesystem context, and budgets are implemented and now owned by the
|
|
[current configuration contract](../config.md#explicit-imports).
|
|
|
|
## Profiles
|
|
|
|
Selected profiles are implemented and owned by the
|
|
[current profile contract](../config.md#selected-profiles). Applications continue
|
|
to own the flags and commands that choose a profile name.
|
|
|
|
## Static defaults and null
|
|
|
|
The defaults merge and nullability model is implemented privately. The
|
|
[current defaults and typed-decoding contract](../config.md#defaults-and-typed-decoding)
|
|
owns recursive mapping, atomic sequence, pointer allocation, explicit-presence,
|
|
null, and detached-baseline behavior. The public defaults factory and result
|
|
lifecycle remain target work in the sections below.
|
|
|
|
## Typed decoding
|
|
|
|
Schema binding and strict document-to-value decoding are implemented privately.
|
|
The [current application schema contract](../config.md#application-schema-binding)
|
|
owns root/type support, field and tag binding, inlining, codec pairing, and
|
|
schema-aware path rules. The [current defaults and typed-decoding contract](../config.md#defaults-and-typed-decoding)
|
|
owns strict conversions, container application, presence, safe failures, and initial
|
|
origins. Secret handling and rendering remain target work below.
|
|
|
|
## Recorded resolution and validation
|
|
|
|
Application-owned recorded resolution is implemented. The
|
|
[current resolution contract](../config.md#recorded-application-resolution) owns
|
|
explicitness queries, helpers and manual records, structural mutation auditing,
|
|
coalescing, diagnostics, origins, history, and snapshot ownership. Layered input
|
|
history is owned by the [current metadata contract](../config.md#layered-metadata);
|
|
redacted explanation and secret dependency propagation remain target work.
|
|
|
|
Canonical typed projection, detached cloning, structural comparison, and publishing
|
|
independently captured resolved results are implemented. The
|
|
[current ownership contract](../config.md#typed-projection-and-ownership) owns those
|
|
mechanics.
|
|
|
|
Application semantic validation is implemented. The
|
|
[current semantic-validation contract](../config.md#semantic-validation) owns
|
|
schema-aware reporting, nearest locations, warning/error behavior, mutation audit,
|
|
callback lifetime, and application/runtime responsibility boundaries.
|
|
|
|
## Provenance, diagnostics, and redaction
|
|
|
|
Safe diagnostic/provenance value records, canonical diagnostic rendering, and
|
|
`LoadError` access are implemented. Their current contracts are owned by
|
|
[`records.go`](../../records.go) and [`diagnostic.go`](../../diagnostic.go).
|
|
Default, base, profile, and resolution records are attached to the loading lifecycle;
|
|
their source ordering, composite/synthetic origins, replacement history, and detached
|
|
queries are owned by the [current metadata contract](../config.md#layered-metadata).
|
|
Safe historical values, redaction, and rendering remain target work below.
|
|
|
|
Diagnostics have stable codes, severity, semantic path when available, a safe
|
|
message, primary location, and related locations. Sort deterministically by
|
|
source traversal order, location, path, and code. Expose structured errors for
|
|
programmatic inspection; human message wording need not be a compatibility
|
|
contract. Fatal parse/graph failures may stop dependent work; collect independent
|
|
schema and semantic errors where continuing is meaningful.
|
|
|
|
On any error, loading returns no usable configuration. A structured error may
|
|
carry safe diagnostics and sources discovered so far. Successful results carry
|
|
configuration, metadata, sources, and nonfatal diagnostics. Avoid ambiguous
|
|
partial-success results or two competing authoritative error lists.
|
|
|
|
Redaction is part of the initial lifecycle, including parse and decode failures.
|
|
Never blindly forward parser/custom-decoder error text or source snippets that
|
|
may contain values. Report safe expected-type and location information instead.
|
|
Applications likewise must supply value-free validation messages and resolution
|
|
reasons. ConfigKit cannot sanitize secrets embedded in arbitrary application
|
|
prose reliably.
|
|
|
|
Use `configkit:"secret"` on fields and an application path-classification hook
|
|
for dynamic entries. A secret container makes descendants secret. Secret
|
|
classification applies conservatively across current and historical definitions
|
|
of a path: replacing a secret does not declassify its history. It propagates from
|
|
recorded secret dependencies to derived outputs. Applications remain responsible
|
|
for marking secret-producing transformations whose dependencies are not recorded.
|
|
|
|
Provide source reporting and redacted effective rendering in the initial target.
|
|
Render from a canonical projection of the final typed snapshot, including zero
|
|
values and null/empty distinctions supported by the schema. Redacted output is
|
|
for inspection and is not promised to be reloadable. ConfigKit renderers must
|
|
not expose raw secrets; returning typed configuration to the application is an
|
|
intentional trusted boundary. ConfigKit does not log configuration on its own.
|
|
|
|
Provide full safe historical-value explanation in the initial release, as defined
|
|
[below](#historical-value-explanation). Defer config diffs and fingerprints. A future fingerprint design
|
|
must specify canonicalization, secret handling, and disclosure risks before
|
|
claiming identity or reproducibility guarantees.
|
|
|
|
## Application schema versions
|
|
|
|
Applications may declare a schema-version path (for example `/schema_version`)
|
|
and a pre-decode checker. The main document owns this value; fragments omit it,
|
|
and profiles/CLI overrides cannot supply or change it. Check fragment placement before
|
|
composition so a version supplied only by an import is also rejected. ConfigKit enforces that placement when
|
|
the application enables the mechanism. The application decides whether a version
|
|
is required and which values it accepts. ConfigKit does not require one universal
|
|
version field or interpret an application's version as the library version.
|
|
Automatic migration is deferred.
|
|
|
|
## Consumer API design
|
|
|
|
The core declarations, callback lifecycle, strict composition, explicit imports,
|
|
and selected profiles are implemented; their current contract is owned by
|
|
[configuration input](../config.md) and Go reference comments. Remaining declarations
|
|
and behavior in this section are target design, not implemented Go reference
|
|
documentation or a frozen naming commitment. Declarations omit method bodies and
|
|
private fields.
|
|
Consumer snippets assume the indicated standard-library imports and a `configkit`
|
|
import using the module path selected in the [implementation plan](implementation.md#implementation-decisions).
|
|
They are illustrative target examples, not currently runnable examples. The
|
|
implementation plan owns their conversion into maintained executable examples
|
|
and verification against consumer models.
|
|
|
|
### Entry point and reusable application definition
|
|
|
|
`Load`, `Definition`, `Options`, `FilesystemContext`, `Override`, `VersionRule`,
|
|
`Version`, `Validation`, and `Result` are declared. The current loader supports the
|
|
callback and path-import subset described by the [current contract](../config.md);
|
|
later override, secret, and rendering behavior below remains target work and
|
|
unavailable options fail explicitly rather than being ignored.
|
|
|
|
```go
|
|
func Load[T any](source Source, options ...Options[T]) (*Result[T], error)
|
|
|
|
type Definition[T any] struct {
|
|
Defaults func() T
|
|
Resolve func(*T, *Resolution) error
|
|
Validate func(*T, *Validation)
|
|
Version *VersionRule
|
|
Secret func(Path) bool
|
|
}
|
|
|
|
type Options[T any] struct {
|
|
Definition Definition[T]
|
|
Imports bool
|
|
Profile string
|
|
Filesystem *FilesystemContext
|
|
Overrides []Override
|
|
}
|
|
|
|
type FilesystemContext struct {
|
|
BaseDir string
|
|
Root string
|
|
}
|
|
```
|
|
|
|
`Load` accepts zero or one options value; multiple values are an options error,
|
|
not a precedence mechanism. The optional argument preserves the shortest load
|
|
without requiring a builder or an option function per setting. With no options,
|
|
`T` starts at its zero value, hooks are absent, and imports/profiles are disabled.
|
|
Strict YAML policy, decoding, presence, and safe diagnostics are always active;
|
|
there is no permissive-mode switch.
|
|
|
|
`Definition[T]` contains reusable application rules; `Options[T]` contains the
|
|
per-load source environment and selection. Applications may reuse a definition
|
|
for many loads. Treat definitions as read-only after sharing them. Each load
|
|
invokes the defaults factory afresh, creates new callback contexts, and owns its
|
|
result. Callbacks must not retain these contexts or configuration pointers; shared
|
|
closures must be safe for concurrent calls if the application loads concurrently.
|
|
ConfigKit does not synchronize application-owned captured state.
|
|
|
|
`Source`, its constructors, and import filesystem context are implemented;
|
|
[`source.go`](../../source.go) and the [configuration input contract](../config.md#sources)
|
|
own their current behavior. Applications must use non-secret source labels because
|
|
labels appear in diagnostics and source reports. Filesystem options do not change
|
|
application field path meanings.
|
|
|
|
`NoSource()` explicitly selects defaults-only loading. It contributes no document
|
|
or file source entry and marks no fields explicit by itself; CLI overrides still
|
|
contribute synthetic sources and explicit values; it still runs schema checking with
|
|
an absent version, defaults, resolution, validation, and snapshot/inspection.
|
|
An application requiring an input version can therefore reject this mode.
|
|
Imports, an active profile, or filesystem context with `NoSource()` are options
|
|
errors. A zero `Source` is invalid and is not an implicit defaults-only request.
|
|
Empty files and empty byte/reader inputs remain invalid YAML inputs.
|
|
|
|
The application decides whether its default path may be absent and explicitly
|
|
selects `NoSource()` in that case. ConfigKit never suppresses missing-file errors
|
|
on `File`; an explicitly selected missing file must still fail. Applications must
|
|
not treat permission or parse failures as evidence that no file was provided.
|
|
|
|
A minimal application needs only its type and one load:
|
|
|
|
```go
|
|
type Config struct {
|
|
Enabled bool `yaml:"enabled"`
|
|
Workers int `yaml:"workers"`
|
|
}
|
|
|
|
func ReadConfig(path string) (Config, error) {
|
|
result, err := configkit.Load[Config](configkit.File(path))
|
|
if err != nil {
|
|
return Config{}, err
|
|
}
|
|
return result.Config, nil
|
|
}
|
|
```
|
|
|
|
Adding defaults and validation remains ordinary application code:
|
|
|
|
```go
|
|
var workerRules = configkit.Definition[Config]{
|
|
Defaults: func() Config { return Config{Enabled: true, Workers: 4} },
|
|
Validate: func(cfg *Config, v *configkit.Validation) {
|
|
if cfg.Workers < 1 {
|
|
v.Error(configkit.Keys("workers"), "must be at least one")
|
|
}
|
|
},
|
|
}
|
|
|
|
func ReadWorkerConfig(path string) (*configkit.Result[Config], error) {
|
|
return configkit.Load(configkit.File(path), configkit.Options[Config]{
|
|
Definition: workerRules,
|
|
})
|
|
}
|
|
```
|
|
|
|
The same application rules support defaults-only operation:
|
|
|
|
```go
|
|
func ReadWorkerDefaults() (*configkit.Result[Config], error) {
|
|
return configkit.Load(configkit.NoSource(), configkit.Options[Config]{
|
|
Definition: workerRules,
|
|
})
|
|
}
|
|
```
|
|
|
|
### Paths and metadata
|
|
|
|
The path API, detached metadata records, empty/unknown query behavior, and layered
|
|
population are implemented; [`path.go`](../../path.go),
|
|
[`records.go`](../../records.go), [`metadata.go`](../../metadata.go), and the
|
|
[metadata configuration contract](../config.md#layered-metadata) own current
|
|
behavior. Safe historical values remain target behavior for explanation.
|
|
|
|
### Resolution and validation callbacks
|
|
|
|
The resolution and semantic-validation declarations and behavior below are
|
|
implemented. The [current recorded-resolution contract](../config.md#recorded-application-resolution)
|
|
and [semantic-validation contract](../config.md#semantic-validation) are
|
|
authoritative.
|
|
|
|
```go
|
|
func (r *Resolution) IsExplicit(path Path) bool
|
|
func (r *Resolution) Record(path Path, kind ChangeKind, reason string, dependencies ...Path)
|
|
|
|
func Default[V any](r *Resolution, path Path, target *V, value V, reason string, dependencies ...Path)
|
|
func Normalize[V any](r *Resolution, path Path, target *V, value V, reason string, dependencies ...Path)
|
|
|
|
func (r *Resolution) Error(path Path, message string, related ...Path)
|
|
func (r *Resolution) Warning(path Path, message string, related ...Path)
|
|
func (v *Validation) Error(path Path, message string, related ...Path)
|
|
func (v *Validation) Warning(path Path, message string, related ...Path)
|
|
```
|
|
|
|
`ChangeKind` distinguishes derived defaults and normalization. `Default` assigns
|
|
and records only when the effective input at the path is not explicit; thus it
|
|
may replace a static baseline. `Normalize` assigns and records a changed value
|
|
whether the input was explicit or defaulted. Both helpers accumulate invalid-path
|
|
or record errors in the resolution context, checked before validation. They do
|
|
not require applications to construct source locations or duplicate value history.
|
|
Targets must correspond to the declared path in the final configuration; ConfigKit
|
|
checks the projected path value against the helper's assigned value and audits
|
|
all net changes for coverage. This is consistency validation, not proof of the
|
|
identity of an arbitrary Go pointer supplied by application code.
|
|
|
|
For a multi-field transformation, the application may mutate normally and call
|
|
`Record` for affected paths or a replaced container. The before/final comparison
|
|
requires coverage of net changes; it does not infer reasons or promise all
|
|
intermediate mutations. The helper/record design must be exercised on consumer
|
|
code before stabilization; do not expand bookkeeping merely to make an internal
|
|
audit easier to implement.
|
|
|
|
Validation receives the resolved configuration by pointer for ergonomic reuse of
|
|
application code but must not mutate it, including nested maps or slices. Its
|
|
context collects independent messages; ConfigKit supplies diagnostic codes and
|
|
available primary/related locations. A missing field uses the nearest relevant
|
|
container/source location, or a synthetic location when no input source applies.
|
|
Application-provided messages and reasons must be safe, value-free prose.
|
|
|
|
Resolvers and validators can report safe domain messages through `Error` and
|
|
`Warning`, specifying one primary path and optional related paths. ConfigKit
|
|
supplies phase-specific codes and attaches available locations to all paths.
|
|
Related paths are useful for duplicates and cross-field constraints; no caller
|
|
must construct file coordinates. Reported errors fail the load even when a
|
|
resolver returns nil. Warnings alone do not fail it. Stop before the next phase
|
|
when a hook has reported errors.
|
|
|
|
A non-nil resolver error also aborts the load. If the context already contains
|
|
errors, retain those without adding a redundant generic failure; otherwise emit
|
|
a safe hook-failure diagnostic. Never forward arbitrary error text by default.
|
|
Only the explicit reporting methods opt into human-visible application messages;
|
|
callers must not interpolate secrets or attach raw underlying errors as messages.
|
|
Validators need no multi-error wrapper.
|
|
Callback panics are programming failures and are not converted to configuration
|
|
validation errors. Contexts are valid only during their callback and are not safe
|
|
for concurrent use within a callback.
|
|
|
|
A Narratio-like application can retain typed domain logic, including defaults for
|
|
new map entries. This illustrative example uses `strings` and `time`:
|
|
|
|
```go
|
|
type Artifact struct {
|
|
Output string `yaml:"output"`
|
|
Timeout time.Duration `yaml:"timeout"`
|
|
}
|
|
|
|
type PipelineConfig struct {
|
|
PollInterval time.Duration `yaml:"poll_interval"`
|
|
Timeout time.Duration `yaml:"timeout"`
|
|
Artifacts map[string]Artifact `yaml:"artifacts"`
|
|
APIKey configkit.Secret `yaml:"api_key"`
|
|
}
|
|
|
|
func resolvePipeline(cfg *PipelineConfig, r *configkit.Resolution) error {
|
|
configkit.Default(r, configkit.Keys("timeout"), &cfg.Timeout,
|
|
cfg.PollInterval/10, "derived from poll interval",
|
|
configkit.Keys("poll_interval"))
|
|
|
|
for name, artifact := range cfg.Artifacts {
|
|
path := configkit.Keys("artifacts", name)
|
|
configkit.Default(r, path.Key("timeout"), &artifact.Timeout,
|
|
cfg.Timeout, "inherited pipeline timeout", configkit.Keys("timeout"))
|
|
configkit.Normalize(r, path.Key("output"), &artifact.Output,
|
|
strings.TrimSpace(artifact.Output), "trimmed surrounding whitespace")
|
|
cfg.Artifacts[name] = artifact
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func validatePipeline(cfg *PipelineConfig, v *configkit.Validation) {
|
|
if cfg.PollInterval <= 0 {
|
|
v.Error(configkit.Keys("poll_interval"), "must be positive")
|
|
}
|
|
if cfg.Timeout <= 0 {
|
|
v.Error(configkit.Keys("timeout"), "must be positive")
|
|
}
|
|
for name, artifact := range cfg.Artifacts {
|
|
path := configkit.Keys("artifacts", name)
|
|
if artifact.Timeout <= 0 {
|
|
v.Error(path.Key("timeout"), "must be positive")
|
|
}
|
|
if artifact.Output == "" {
|
|
v.Error(path.Key("output"), "must be nonempty")
|
|
}
|
|
}
|
|
}
|
|
|
|
var pipelineRules = configkit.Definition[PipelineConfig]{
|
|
Defaults: func() PipelineConfig {
|
|
return PipelineConfig{PollInterval: time.Minute}
|
|
},
|
|
Resolve: resolvePipeline,
|
|
Validate: validatePipeline,
|
|
}
|
|
|
|
func ReadPipeline(path, profile string) (*configkit.Result[PipelineConfig], error) {
|
|
return configkit.Load(configkit.File(path), configkit.Options[PipelineConfig]{
|
|
Definition: pipelineRules,
|
|
Imports: true,
|
|
Profile: profile,
|
|
})
|
|
}
|
|
|
|
func ReadPipelineBytes(label string, data []byte) (*configkit.Result[PipelineConfig], error) {
|
|
return configkit.Load(configkit.Bytes(label, data), configkit.Options[PipelineConfig]{
|
|
Definition: pipelineRules,
|
|
})
|
|
}
|
|
```
|
|
|
|
Map entries are copied out and written back using ordinary Go code; helpers audit
|
|
the final projection. Independent map traversal order must not affect sorted
|
|
records or diagnostics. Interdependent application transformations must use an
|
|
application-defined deterministic order. Campaign/session fallback can similarly
|
|
remain application code; this example does not turn it into ConfigKit composition.
|
|
|
|
A validator can report both sides of a duplicate without constructing source
|
|
locations. This example uses `sort` to make primary-path selection deterministic:
|
|
|
|
```go
|
|
func validateUniqueOutputs(cfg *PipelineConfig, v *configkit.Validation) {
|
|
names := make([]string, 0, len(cfg.Artifacts))
|
|
for name := range cfg.Artifacts {
|
|
names = append(names, name)
|
|
}
|
|
sort.Strings(names)
|
|
seen := make(map[string]configkit.Path)
|
|
for _, name := range names {
|
|
output := cfg.Artifacts[name].Output
|
|
if output == "" {
|
|
continue // The ordinary field validator reports this separately.
|
|
}
|
|
path := configkit.Keys("artifacts", name, "output")
|
|
if first, exists := seen[output]; exists {
|
|
v.Error(path, "output must be unique", first)
|
|
} else {
|
|
seen[output] = path
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Call this helper from the application's validator when output uniqueness is a
|
|
domain requirement. The message identifies the relationship without printing
|
|
potentially sensitive configured values.
|
|
|
|
### Schema checking and secret classification
|
|
|
|
`VersionRule`, `Version`, and the version-checking use of `Validation` are
|
|
implemented. The [current version contract](../config.md#application-schema-versions)
|
|
owns absent/null/string handling, report locations, warnings, failure behavior, and
|
|
raw-error suppression. Secret classification and later composed-source placement
|
|
checks remain target work.
|
|
|
|
```go
|
|
type VersionRule struct {
|
|
Path Path
|
|
Check func(Version, *Validation) error
|
|
}
|
|
|
|
type Version struct {
|
|
Present bool
|
|
Null bool
|
|
Text string
|
|
}
|
|
```
|
|
|
|
The version checker receives the main document's scalar at the declared path,
|
|
with absent/null states preserved. Non-string, non-null values are schema-version
|
|
errors before the hook; the application decides whether absent/null or a string
|
|
is acceptable. The checker reports actionable safe messages through its validation
|
|
context, using the same primary/related-path API as other hooks. At this pre-decode
|
|
phase, locations resolve against the composed input rather than a typed schema;
|
|
a missing path falls back to the nearest input container. No parser tree is
|
|
exposed. Reported errors or a non-nil return fail the load. A non-nil return with
|
|
no reported error produces a generic safe version-check failure at the declared
|
|
version path; raw error text is not printed. Placement and timing follow
|
|
[schema versioning](#application-schema-versions). A rule with no checker is an
|
|
options error.
|
|
|
|
For example, an application can preserve useful migration guidance without
|
|
embedding input values in its error text:
|
|
|
|
```go
|
|
var schemaVersionPath = configkit.Keys("schema_version")
|
|
|
|
func checkVersion(version configkit.Version, v *configkit.Validation) error {
|
|
if !version.Present || version.Null {
|
|
v.Error(schemaVersionPath, "schema_version is required")
|
|
} else if version.Text != "example.v1" {
|
|
v.Error(schemaVersionPath, "unsupported schema version; migrate using the application configuration guide")
|
|
}
|
|
return nil
|
|
}
|
|
|
|
var versionRule = configkit.VersionRule{
|
|
Path: schemaVersionPath,
|
|
Check: checkVersion,
|
|
}
|
|
```
|
|
|
|
Assign `&versionRule` to a definition's `Version` field to enable it. Schema
|
|
versions remain strings under ConfigKit's convention; adopting consumers with
|
|
integer versions must migrate their syntax rather than enable a second scalar
|
|
policy.
|
|
|
|
`Definition.Secret` is an optional pure path predicate for dynamic entries,
|
|
combined with secret field tags. It can add sensitivity but cannot override a
|
|
tag to declassify a value. It is applied to historical as well as final paths.
|
|
Recorded dependencies propagate sensitivity under the existing redaction rules;
|
|
applications own secret-producing transformations not described by those records.
|
|
|
|
### Results, errors, and inspection
|
|
|
|
`Result.Config`, `Metadata`, `Sources`, and `Diagnostics`, together with
|
|
`LoadError` and `WriteDiagnostics`, are implemented at the current loading boundary.
|
|
The [current loading contract](../config.md#single-source-loading) and Go reference
|
|
comments own their behavior. Effective rendering and richer layered inspection
|
|
below remain target work.
|
|
|
|
```go
|
|
type Result[T any] struct {
|
|
Config T
|
|
// Private snapshot and metadata storage.
|
|
}
|
|
|
|
func (r *Result[T]) Metadata() Metadata
|
|
func (r *Result[T]) Sources() []SourceInfo
|
|
func (r *Result[T]) Diagnostics() []Diagnostic
|
|
func (r *Result[T]) WriteEffective(output io.Writer) error
|
|
|
|
type LoadError struct {
|
|
// Private diagnostic and source storage.
|
|
}
|
|
|
|
func (e *LoadError) Error() string
|
|
func (e *LoadError) Diagnostics() []Diagnostic
|
|
func (e *LoadError) Sources() []SourceInfo
|
|
func WriteDiagnostics(output io.Writer, diagnostics []Diagnostic) error
|
|
```
|
|
|
|
Every configuration/source/options/hook failure returns `(nil, *LoadError)`.
|
|
Success returns a non-nil result and nil error; its diagnostics contain only
|
|
nonfatal warnings. Source lists use stable source identifiers and safe labels;
|
|
diagnostics expose code, severity, path, message, primary location, and related
|
|
locations as described [above](#provenance-diagnostics-and-redaction). No raw
|
|
parser error or unrestricted secret-bearing history is exposed. No automatic
|
|
logging occurs. Inspection methods returning slices provide defensive copies.
|
|
|
|
`Config` is caller-owned typed data. Metadata and rendering use an independently
|
|
captured final snapshot, so later caller mutation cannot rewrite the recorded
|
|
result. `WriteEffective` emits deterministic redacted YAML for inspection, with
|
|
no unredacted switch. It returns output errors rather than a new configuration validation result.
|
|
Custom projection failures must already have been rejected before a load
|
|
succeeds. Output can be partial on writer failure.
|
|
|
|
An application can report errors and inspect a successful load without rebuilding
|
|
diagnostic formatting. This example uses `errors` and `io` and the pipeline
|
|
loader above:
|
|
|
|
```go
|
|
func InspectPipeline(path, profile string, output io.Writer) error {
|
|
result, err := ReadPipeline(path, profile)
|
|
if err != nil {
|
|
var loadErr *configkit.LoadError
|
|
if errors.As(err, &loadErr) {
|
|
if writeErr := configkit.WriteDiagnostics(output, loadErr.Diagnostics()); writeErr != nil {
|
|
return writeErr
|
|
}
|
|
}
|
|
return err
|
|
}
|
|
if err := configkit.WriteDiagnostics(output, result.Diagnostics()); err != nil {
|
|
return err
|
|
}
|
|
return result.WriteEffective(output)
|
|
}
|
|
```
|
|
|
|
Applications can use `Metadata().Lookup`, `Metadata().History`, and `Sources()`
|
|
for structured inspection without implementing value rendering or importing a
|
|
parser. The explanation API below supplies safe values and formatting for application-owned
|
|
`config explain` commands.
|
|
|
|
### Historical-value explanation
|
|
|
|
```go
|
|
func (r *Result[T]) Explain(path Path) (Explanation, bool)
|
|
func (r *Result[T]) WriteExplanation(output io.Writer, path Path) error
|
|
|
|
type Explanation struct {
|
|
Path Path
|
|
Effective ValueSnapshot
|
|
Origin Origin
|
|
History []ValueChange
|
|
SecretReference *SecretReferenceInfo
|
|
}
|
|
type ValueSnapshot struct {
|
|
Present bool
|
|
Redacted bool
|
|
YAML string
|
|
}
|
|
type ValueChange struct {
|
|
Before, After ValueSnapshot
|
|
Origin Origin
|
|
Reasons []string
|
|
Dependencies []Path
|
|
}
|
|
type SecretReferenceInfo struct { Name string } // successful resolved lookup
|
|
```
|
|
|
|
Snapshots contain detached deterministic YAML text, never `any`, parser objects,
|
|
or mutable internal trees. `Present=false` is absence/deletion and has empty YAML;
|
|
null is present with `null` YAML. Redacted snapshots use the quoted marker
|
|
`[REDACTED]`; `Redacted` also marks containers containing redacted descendants.
|
|
Apply the same historical sensitivity/dependency closure as effective rendering
|
|
before exposing any snapshots. A `Secret` projects only its environment name;
|
|
its resolved value is never captured, even privately as history. For classified
|
|
reference paths omit `SecretReference` as well as masking the projected name.
|
|
|
|
Retain private persistent node snapshots or operation deltas with structural
|
|
sharing; do not copy the entire tree for every changed leaf. Explain a known final
|
|
or historical path, including removed paths; unknown paths return false, and the
|
|
writer returns a safe unknown-path error without output. Malformed query paths
|
|
also fail safely. Render the canonical JSON-pointer path, effective value/source,
|
|
and chronological history: initial application-default (or Go-zero baseline),
|
|
base contributions in traversal order, profile, CLI, and recorded domain changes.
|
|
Include before/after values for changed or explicitly redefined paths, equal-value
|
|
writes, removals, reasons/dependencies, and composite origins. Inputs overridden
|
|
before decoding retain policy-validated node values even when they would not fit
|
|
the final Go type. Defaults/final/domain snapshots use canonical typed projection;
|
|
label those representations rather than claiming every historical input was typed.
|
|
|
|
Container queries reconstruct the value at each relevant operation; descendant
|
|
queries include ancestor replacement/removal and whole-sequence replacements,
|
|
without implying element-wise sequence merging. Omitted fields/default retention
|
|
must agree with the presence/default contract; do not fabricate a write for every
|
|
import that leaves a path alone. Resolution history records the audited net change
|
|
per path, not every intermediate Go assignment. Lookup of a secret reference
|
|
annotates its effective name as resolved from the process environment; bootstrap
|
|
file outcomes come from the separate report and are never guessed.
|
|
|
|
`WriteExplanation` is a deterministic human renderer over this safe structured
|
|
result. Escape control characters in paths, labels, names, and reasons, propagate
|
|
writer errors/short writes, and permit partial output only on writer failure.
|
|
No unredacted switch or bundled command is provided. For example an application
|
|
can call it for `/transcription/concurrency` to show default `4`, an explicitly
|
|
imported file's `8`, then a testing profile's `1`, with file line numbers and the
|
|
profile as effective source. This does not require automatic `conf.d` discovery.
|
|
|
|
### Module-specific options
|
|
|
|
Use a canonical object entry with an explicit module identifier and options
|
|
mapping. Shorthand strings and custom structural decoders are not required to
|
|
express the domain information. For a known application module catalog, model
|
|
the available option fields with ordinary typed structs; application validation
|
|
selects the meaningful fields and rejects incompatible combinations.
|
|
|
|
Illustrative YAML for an application that supports two modules:
|
|
|
|
```yaml
|
|
modules:
|
|
- module: chunker
|
|
options:
|
|
window_units: 12
|
|
- module: discussion
|
|
options:
|
|
sections: [short_term]
|
|
```
|
|
|
|
The corresponding typed configuration and validator need no second parser:
|
|
|
|
```go
|
|
type ModuleOptions struct {
|
|
WindowUnits *int `yaml:"window_units"`
|
|
Sections []string `yaml:"sections"`
|
|
}
|
|
|
|
type ModuleSpec struct {
|
|
Module string `yaml:"module"`
|
|
Options ModuleOptions `yaml:"options"`
|
|
}
|
|
|
|
type ModuleConfig struct {
|
|
Modules []ModuleSpec `yaml:"modules"`
|
|
}
|
|
|
|
func validateModules(cfg *ModuleConfig, v *configkit.Validation) {
|
|
for index, spec := range cfg.Modules {
|
|
path := configkit.Keys("modules").Index(index)
|
|
switch spec.Module {
|
|
case "chunker":
|
|
if spec.Options.WindowUnits == nil || *spec.Options.WindowUnits < 1 {
|
|
v.Error(path.Key("options").Key("window_units"), "must be a positive integer")
|
|
}
|
|
if spec.Options.Sections != nil {
|
|
v.Error(path.Key("options").Key("sections"), "not supported by this module")
|
|
}
|
|
case "discussion":
|
|
if spec.Options.WindowUnits != nil {
|
|
v.Error(path.Key("options").Key("window_units"), "not supported by this module")
|
|
}
|
|
if len(spec.Options.Sections) == 0 {
|
|
v.Error(path.Key("options").Key("sections"), "must contain at least one section")
|
|
}
|
|
default:
|
|
v.Error(path.Key("module"), "unsupported module")
|
|
}
|
|
}
|
|
}
|
|
|
|
func ReadModuleConfig(path string) (*configkit.Result[ModuleConfig], error) {
|
|
return configkit.Load(configkit.File(path), configkit.Options[ModuleConfig]{
|
|
Definition: configkit.Definition[ModuleConfig]{Validate: validateModules},
|
|
})
|
|
}
|
|
```
|
|
|
|
The pointer and nil-slice states above express optional domain values; this
|
|
application treats explicit null as unselected for those optional alternatives.
|
|
ConfigKit rejects unknown fields and invalid scalar types. The application owns
|
|
which supported options apply to a module and their domain constraints. Larger
|
|
catalogs may use nested typed option objects rather than one flat options struct.
|
|
Neither shape requires a library registry, parser hook, or configurable merge rule.
|
|
|
|
For an application whose option schema is genuinely selected at runtime,
|
|
`map[string]any` remains available. ConfigKit owns parsing, paths, and the single
|
|
dynamic scalar representation; application validation owns the selected schema,
|
|
including allowed keys and value constraints. Conversion into runtime module
|
|
arguments is application code. Dynamic integers remain checked `int64`; existing
|
|
`int` assertions must be adapted with range checks when converting to `int`.
|
|
|
|
The initial API does not add a generic subtree-conversion helper. Prefer typed
|
|
configuration where option shapes are known, and direct domain validation where
|
|
they are dynamic. Do not introduce a second YAML marshal/decode round trip as the
|
|
blessed pattern. Reconsider a narrow conversion helper only if a concrete dynamic
|
|
catalog demonstrates substantial duplicated type-binding infrastructure that
|
|
ordinary models cannot remove. Such a helper would have to retain strictness and
|
|
source attribution; that is a separate justified extension, not an escape hatch
|
|
for legacy shorthand. Configuration inspection describes the validated input
|
|
model, not the application's separately constructed runtime module objects.
|
|
|
|
### Supported types and projection
|
|
|
|
Supported schema types and field bindings are implemented and owned by the
|
|
[current application schema contract](../config.md#application-schema-binding).
|
|
Projection must additionally reject cyclic object graphs. Arrays require matching
|
|
sequence length. Ignored and unexported fields stay outside configuration projection;
|
|
hooks must not use them to carry runtime resources or configuration needing
|
|
inspection. Non-configuration runtime state belongs in separate application types.
|
|
|
|
Canonical projection, detached copying, cycle and finite-value enforcement, custom
|
|
scalar reconstruction, and structural comparison are implemented and owned by the
|
|
[current ownership contract](../config.md#typed-projection-and-ownership). The same
|
|
field-binding model must still supply defaults, decoding, change auditing, and
|
|
rendering. Application text methods must remain side-effect-free with respect to
|
|
shared state; this is a bounded scalar seam, not a custom structural decoder.
|
|
|
|
## Internal organization
|
|
|
|
Keep document composition, overlays, and parser machinery internal initially.
|
|
Applications use the typed lifecycle; application-specific resolution alone does
|
|
not justify exposing a document-processing toolkit.
|
|
|
|
Internally separate source access, policy-validated documents, composition,
|
|
overlay, field binding/decoding, metadata, resolution, and safe rendering.
|
|
These are responsibilities, not a mandate for public interfaces or a file per
|
|
concept. Start with one public package and a private YAML adapter; add package
|
|
boundaries only when dependencies and ownership warrant them.
|
|
|
|
Use standard-library filesystem, I/O, scalar conversion, reflection, ordering,
|
|
and synchronization primitives. ConfigKit owns the policy that coordinates them:
|
|
field binding, presence/default behavior, strict composition, overlays, provenance,
|
|
resolution auditing, and secret propagation. One descriptor graph and document
|
|
model support those operations; do not add competing binding or copying models.
|
|
Convenience functions must not broaden accepted scalar syntax, silently truncate
|
|
input, lose nested ownership, or substitute Go field promotion for YAML inlining.
|
|
|
|
The [implementation plan](implementation.md#standard-library-mechanisms-and-custom-policy)
|
|
owns the selected primitives and their integration checks. Test comparison uses
|
|
`go-cmp` only in tests, never as the runtime resolution comparator. Additional
|
|
dependencies remain case-by-case decisions based on code saved after accounting
|
|
for adaptation, validation, and recurring maintenance.
|
|
|
|
### YAML dependency and public API isolation
|
|
|
|
Use the YAML organization's `go.yaml.in/yaml/v4` package behind `internal/yaml`.
|
|
V4 is upstream's active development line; v3 receives security fixes only.
|
|
This choice aligns a new library with ongoing upstream maintenance while keeping
|
|
ConfigKit responsible for its own configuration model.
|
|
See [upstream's version policy](https://github.com/yaml/go-yaml#version-intentions).
|
|
The [implementation decisions](implementation.md#implementation-decisions) own
|
|
the exact selected release-candidate pin and upgrade validation requirements.
|
|
|
|
The adapter owns all direct upstream imports, including parser plugins, loading,
|
|
and emission. Its boundary uses ConfigKit-owned document and diagnostic records.
|
|
No upstream nodes, type aliases, options, plugin interfaces, error types, or
|
|
generic constraints form part of ConfigKit's public declarations or callbacks.
|
|
ConfigKit must not return parser-created values hidden inside `any`, metadata,
|
|
or an error's unwrap chain. Applications use ordinary Go types, ConfigKit types,
|
|
and the supported standard-library text interfaces; they need no YAML-package
|
|
imports or upstream version presets to use ConfigKit.
|
|
|
|
The existing `yaml` field-tag spelling is a ConfigKit-owned binding contract,
|
|
not delegation to upstream reflection or marshal/unmarshal hooks. ConfigKit also
|
|
owns scalar interpretation, duplicate detection, document count, source-coordinate
|
|
conventions, diagnostic classifications, and effective-output formatting. Upstream
|
|
options may help enforce these contracts internally but cannot redefine them.
|
|
|
|
An upstream breaking API change must be absorbed in the adapter or other private
|
|
implementation, preserving downstream Go source compatibility and documented
|
|
configuration behavior. Parser upgrades do not independently authorize changes
|
|
to ConfigKit's public contracts. If an upstream version cannot preserve those
|
|
contracts, retain the compatible dependency or adapt the implementation; any
|
|
intentional ConfigKit contract change follows ConfigKit's own release policy.
|
|
|
|
This boundary isolates API coupling, not every dependency risk: the YAML module
|
|
remains a transitive build dependency, and parser defects or behavior changes
|
|
still require ConfigKit regression checks. It does not require a public parser
|
|
interface, configurable backends, or simultaneous v3/v4 implementations.
|
|
|
|
## Explicit non-goals
|
|
|
|
General environment-variable configuration overlays are not on the roadmap.
|
|
Secret references and explicit bootstrap are the supported environment convention.
|
|
Literal-secret configuration and implicit secret-directory discovery are unsupported.
|
|
|
|
## Deferred scope
|
|
|
|
The initial target excludes:
|
|
|
|
- automatic `conf.d` discovery;
|
|
- remote imports, arbitrary providers, and additional configuration formats;
|
|
- custom merge strategies, sequence merging, reset operators, and profile inheritance;
|
|
- parser-specific or custom structural decoding extension APIs;
|
|
- generic dynamic-subtree conversion and module-registry APIs;
|
|
- automatic schema migrations;
|
|
- hot reload, runtime resource replacement, and remote resource validation;
|
|
- a bundled CLI framework or commands;
|
|
- config diffs and fingerprints.
|
|
|
|
Applications can build their own validate/effective/sources commands on the
|
|
library. Broadening these boundaries requires concrete consumer evidence and
|
|
an explicit design change.
|