ConfigKit/docs/roadmap/initial.md

# ConfigKit — Initial Target Design

## Status and ownership

This roadmap describes the targeted end-state for ConfigKit's initial release,
provisionally `v0.1.0`. It is future design, not a claim of implemented behavior
or a stable v1 compatibility promise. Audience: maintainers and implementation
agents.

This document owns target behavior, boundaries, and deferred scope. The
[implementation plan](implementation.md) owns stages, dependencies, acceptance
gates, and progress. Do not add implementation steps or stage status here.

The [architecture policy](../policy/architecture.md) owns binding contributor
constraints. Accepted rationale is recorded in ADRs for
[composition versus overlay](../adr/0001-separate-composition-and-overlay.md),
[parser independence](../adr/0002-own-yaml-semantics.md), and
[presence and resolution history](../adr/0003-preserve-presence-and-resolution-history.md)
under the [documentation policy](../policy/documentation.md). As features land,
move their normative behavior to current contracts and replace the corresponding roadmap
text with links. Retire completed roadmap material during release preparation;
these files are not permanent duplicate specifications.

## Purpose and principles

ConfigKit provides a consistent configuration lifecycle for Go applications
using YAML, including Seriatim, Audita, Narratio, WeatherFeeder, WeatherReporter,
and Distributor. Its goal is one deterministic, explainable resolution model
that works for a single file and scales to explicitly imported fragments.

- YAML supplies syntax; ConfigKit owns composition, precedence, presence,
  conversion, and error policy.
- Preserve a source-located tree through composition and overlay. Do not first
  collapse input into `map[string]any` or application structs.
- Keep application types ordinary. Pointers express application nullability,
  not a universal workaround for distinguishing absent from false or zero.
- Separate strict composition from intentional overlay.
- Keep the public API small and the parser private. Avoid a general provider or
  merge-strategy framework.

### Success criteria

1. **Reduce application infrastructure:** Can applications delete substantial
   configuration infrastructure while retaining clear ownership of their domain
   rules?
2. **Unify configuration conventions:** Do consumers adopt one consistent
   configuration model, with only the flexibility needed to express genuine
   application requirements?

The [architecture policy](../policy/architecture.md#success-criteria) owns these
coequal criteria and their governing rules. The target must satisfy both: the
shared lifecycle removes configuration plumbing, while common syntax, presence,
composition, precedence, and decoding rules establish consistent conventions.
Application types, interpretation, defaults, and semantic validation retain the
flexibility needed to describe domain requirements.

Consumer migrations are expected, including intentional configuration syntax
changes. Existing code and tests are extraction foundations and evidence of needs,
not compatibility obligations for ConfigKit. Prefer canonical object forms over
consumer-specific shorthand where they express the same information. Do not add
alternate parsers or merge rules merely to preserve an existing convention.

Evaluate an extension by identifying its concrete domain requirement, explaining
why the existing primitives cannot accommodate it cleanly, and showing how it
preserves one coherent model. If application code must remain, distinguish domain
interpretation and validation from duplicated configuration infrastructure. A
restriction that makes useful information inexpressible is not justified by
consistency alone.

The consumer examples and adoption work must demonstrate both reduced plumbing
and convergence on the model. Narratio extraction and simpler consumers should
supply concrete evidence; feature count, unchanged legacy syntax, and fewer lines
inside ConfigKit are not measures of success. Detailed consumer migrations and
execution steps belong in the [implementation plan](implementation.md).

## Responsibility boundary

| ConfigKit owns | Applications own |
| --- | --- |
| Local file and labeled bytes/reader inputs | Main-file search paths, environment conventions, and CLI wiring |
| YAML policy, explicit import traversal, composition, and profiles | Configuration types and field meanings |
| Presence, source locations, resolution records, and safe diagnostics | Static defaults, derived defaults, and normalization rules |
| Strict typed decoding and lifecycle coordination | Semantic validation and schema-version policy |
| Source reporting and redacted effective output | Remote acquisition, migrations, runtime initialization, and resource checks |

Applications may fetch remote data and pass labeled bytes to ConfigKit. This
provides useful provenance without making ConfigKit responsible for network
availability, credentials, or caching. Narratio's pipeline/campaign/session
fallback rules remain application semantics; the extraction boundary is described
below.

## Existing implementation foundation

Narratio is the primary extraction source and intended first substantial consumer
of ConfigKit. Prefer copying and generalizing applicable existing code and tests
over independently rebuilding equivalent mechanisms. Reuse includes the behavior
captured by Narratio's tests, not just production code. ConfigKit's target
contracts govern the result; reuse does not require preserving incidental
application structure or incompatible behavior.

The references below were inspected on 2026-09-06 at Narratio commit
`a7ec1955874fda61f3e3f9e3cc33acd345530952`. Paths refer to the sibling checkout
`../narratio` relative to the ConfigKit repository root; the links require that
checkout layout. They are extraction references, not a runtime or build dependency.
Recheck them against the source revision actually used for extraction.

| Foundation | Source references | Intended treatment |
| --- | --- | --- |
| File and labeled reader/bytes loading; strict field checking | [load.go](../../../narratio/internal/config/load.go): `decodeStrictYAML`, `decodeStrictYAMLFromReader`, `LoadSessionBytesWithOptions` | Extract reusable input handling and generalize decoding behind ConfigKit's private parser boundary |
| Explicit field presence | [normalize.go](../../../narratio/internal/config/normalize.go): `NormalizeConfig.UnmarshalYAML` | Generalize the requirement into library metadata; replace the application-specific parser hook |
| Default application | [load.go](../../../narratio/internal/config/load.go): `applyPipelineDefaults` and related helpers; [defaults.go](../../../narratio/internal/config/defaults.go) | Reuse applicable mechanics and behavioral cases while keeping actual default values and domain rules application-owned |
| Resolution and source attribution | [load.go](../../../narratio/internal/config/load.go): `Resolve`, `mergeCampaignSession`, `selectStableInput`; [config.go](../../../narratio/internal/config/config.go): `ResolvedInputFile`, `SessionSource` | Adapt source tracking into generic provenance; retain campaign/session fallback and domain types in Narratio |
| Application-supplied remote bytes | [config_loader.go](../../../narratio/internal/app/config_loader.go) | Use as evidence for labeled input support; keep fetching, storage clients, and remote metadata acquisition outside ConfigKit |
| Existing behavioral coverage | [normalize_test.go](../../../narratio/internal/config/normalize_test.go), [load_validate_test.go](../../../narratio/internal/config/load_validate_test.go), [campaign_config_test.go](../../../narratio/internal/config/campaign_config_test.go) | Extract relevant cases and minimal fixtures; preserve domain-specific assertions in consumer tests |

### Compatibility and adaptation

Extraction must distinguish preserved behavior, intentionally changed behavior,
and application-only behavior. Record the source revision and relevant symbols
with the extraction work so reviewers can compare the implementation and tests.
Generalized code must not import Narratio's internal packages or copy its domain
model into ConfigKit's public API.

Known adaptation points include:

- Some Narratio defaulting helpers use an empty string to trigger a default.
  Adapt these to ConfigKit's [presence and defaults rules](#static-defaults-and-null)
  rather than losing explicitly configured empty values.
- `decodeStrictYAMLFromReader` currently accepts a successfully decoded second
  document. ConfigKit must enforce its [single-document policy](#yaml-and-scalar-policy).
- The custom normalization decoder and pointer-based presence conventions need
  to give way to generic metadata while preserving genuine domain nullability.
- Existing source labels and campaign/session attribution provide useful
  foundations but do not supply full per-path locations, overlay history, or
  recorded post-decode changes.

Each intentional behavior change needs focused tests and an explicit migration
account for the affected consumer. Do not copy an old expectation merely because
it passes in Narratio, or discard a useful case because the implementation moves.
Use representative Narratio configurations to verify preserved outcomes and
identify intentional differences. Exercise a simpler consumer as well so the
public API remains practical for single-document applications.

### Capabilities requiring broader implementation

The inspected loader coordinates specific pipeline, campaign, and session
sources and application fallback rules. It is not the generic import graph and
strict recursive composer described by this target. Extract applicable pieces,
but expect new or substantially revised code for generic composition, profile
overlays, schema-aware defaults/decoding, comprehensive provenance, recorded
resolution, and safe structured diagnostics and rendering.

Automatic `conf.d` discovery remains deferred. The existing application source
relationships do not authorize implicit discovery or define ConfigKit's import
syntax. The [implementation plan](implementation.md) owns concrete extraction
tasks and their ordering.

## Consumer fit and intentional migrations

These are adoption targets, not promises of unchanged consumer behavior or
completed integrations. Source links refer to sibling checkouts as in the
Narratio foundation section. The findings reflect the inspected implementations
on 2026-09-06 and must be rechecked at migration time.

| Consumer and evidence | Useful ConfigKit fit | Intentional migration or retained application responsibility |
| --- | --- | --- |
| Narratio: [configuration loader](../../../narratio/internal/config/load.go) | Shared loading, presence, defaults coordination, and source attribution | Replace ad hoc presence/empty-value defaults; retain campaign/session domain resolution |
| Notarius: [file model](../../../notarius/internal/core/config/file_config.go), [environment handling](../../../notarius/internal/core/config/env.go) | Strict nested models, named pipelines, presence-aware values, and diagnostics | Use canonical object bindings/references and string schema versions; adapt dynamic integer consumers to `int64`; migrate configuration-field environment overrides to files/profiles; keep catalog resolution and runtime invocation choices application-owned |
| WeatherReporter: [loader](../../../weatherreporter/internal/config/load.go), [module options](../../../weatherreporter/internal/config/reports.go) | Defaults-only loading, durations, report maps, list presence, and typed module options | Replace string shorthand with object entries and private presence flags with metadata; migrate configuration-field CLI overrides to files/profiles; keep secrets-directory loading and process-environment mutation outside ConfigKit |
| Seriatim: [configuration surfaces](../../../seriatim/docs/config.md), [speaker rules](../../../seriatim/internal/speaker/map.go), [autocorrect rules](../../../seriatim/internal/autocorrect/autocorrect.go) | Simple typed loading of ordered YAML rule files, normalization, and cross-rule diagnostics | Adopt strict YAML rejection deliberately; retain matching/replacement algorithms, resource checks, and CLI/environment runtime settings; no central application YAML file is required |

The initial model standardizes syntax and resolution while preserving application
field meanings. Module shorthand and integer schema versions are syntax migration
issues; module-specific option schemas are genuine domain variation addressed
[below](#module-specific-options). ConfigKit configuration profiles are overlays,
not substitutes for application-owned LLM or pipeline profile concepts.

## External values and invocation state

Environment and CLI configuration-field overlays remain outside the initial
release. Consumers adopting the complete ConfigKit lifecycle must move such
settings into base files or supported profile overlays, or defer migration of
that configuration surface. Do not silently discard supported consumer flags;
consumer migration must document their replacement or retirement.

Application CLI arguments may select a configuration source/profile or describe
separate invocation state, such as an input artifact, output destination, or
selected execution lane. Environment variables may supply runtime credentials
outside the configuration snapshot. Such state remains application-owned and
must not be reported as part of ConfigKit's effective configuration.

Do not inject configuration overrides through captured environment/CLI values in
default factories, resolver hooks, or validators. Resolution is for domain
derivation and normalization, not an undocumented precedence layer. Mutating
`Result.Config` after loading cannot create a newly validated ConfigKit snapshot
or update its provenance. A future override mechanism would need an explicit
precedence, presence, validation, and origin contract before adoption.

## Resolution lifecycle

```text
application selects source and loading options
  → parse and enforce YAML policy for every source
  → discover explicit imports and strictly compose the base
  → application schema-version check, when configured
  → apply the selected profile
  → apply static defaults and strictly decode
  → apply recorded application resolution and normalization
  → collect application semantic validation diagnostics
  → return independently owned configuration and metadata
```

Value precedence is defaults < composed base < selected profile. Resolution is
an application transformation after those layers, not another user override
source. Future migration support belongs after base composition/version
inspection and before profile application and typed decoding; migrations are
not part of the initial release.

YAML policy applies to every document even when a later layer would replace its
values. Schema/type checks apply to the effective input tree; application
validation runs only after successful decoding and resolution. An invalid
configuration is never returned as a successful, usable result.

## YAML and scalar policy

The supported model is mappings with string keys, sequences, scalars, and null,
with YAML comments and block/flow syntax. Require exactly one document with a
mapping root. Reject empty input, additional documents (including empty ones),
duplicate mapping keys, explicit tags, anchors, aliases, and merge keys (`<<`).
Mapping keys must resolve to strings under this scalar policy; quote keys
that would otherwise be booleans, numbers, or null. Compare decoded string keys
without case folding or Unicode normalization. A quoted `"<<"` is also reserved and rejected.

Use a deliberately limited scalar vocabulary independent of parser defaults:

- Quoted and block scalars are strings.
- Plain `true` and `false` are booleans; plain `null`, `~`, and an empty value
  are null. Other case variants of these reserved literals are rejected when used as
  plain values.
- Numbers use JSON decimal syntax: optional minus, integer part with no leading
  zero except zero itself, optional fraction, and optional decimal exponent.
- Reject YAML numeric extensions such as hexadecimal, octal, digit separators,
  leading plus, and non-finite values. Reject legacy boolean words such as
  `yes`, `no`, `on`, and `off` unless quoted (case-insensitively).
- Date/time-looking plain text remains a string, never an implicit timestamp.
  Other ordinary plain text is a string.

Unsupported scalar forms must produce a policy error rather than inherit a
parser conversion. Preserve numeric lexical information until decoding so
large integers are not rounded through an intermediate floating-point value.

## Paths and presence

Use structured key/index segments internally. Public textual paths use
JSON Pointer escaping (`~0` for tilde and `~1` for slash), with the empty string
for the root. For example, `/transcription/concurrency` names a nested field;
`/outputs/a.b` addresses the literal key `a.b`. Numeric segments are interpreted
using the containing node's kind, distinguishing mapping keys from list indices.
Use YAML field names, not Go field names.

Distinguish absent, explicitly null, explicitly zero/false/empty, and other
explicit values. An explicit value equal to its default remains explicit.
Presence queries describe the surviving input contribution after overlays;
separate history queries describe earlier appearances. A parent mapping's
presence does not make all of its children explicit. Replacing a subtree removes
its descendants from effective presence and origin queries while retaining
historical replacement records.

Application normalization preserves the fact that a surviving input was
explicit. A derived value for an absent input remains non-explicit. Its final
origin is a resolution operation, with input origins retained separately.

## Composition and overlay

Composition combines the main file and explicitly imported fragments as peers.
Repeated semantic definitions are errors, even when their values are equal.
Both conflicting locations must be reported. Prefer mappings keyed by stable
identifiers when entries need to be contributed independently.

The following table is exhaustive by node category. “Value” includes scalar,
sequence, mapping, and null. Absent means no node, not null.

| Existing/lower node | Incoming/higher node | Strict composition | Overlay |
| --- | --- | --- | --- |
| Absent | Absent | Absent | Absent |
| Absent | Value | Incoming | Incoming |
| Value | Absent | Existing | Existing |
| Mapping | Mapping | Recurse by key | Recurse by key |
| Any other pair of present nodes | Any other pair of present nodes | Collision | Replace entire existing node |

Two empty mappings compose successfully. An empty mapping contributes no child
changes to an existing mapping; it does not clear that mapping. A mapping can
replace a lower scalar, sequence, or null during overlay, subject to the final
schema. Sequences are atomic: two definitions collide in composition and the
higher sequence replaces the entire lower sequence in an overlay. `[]` clears a
sequence. There is no append, union, index merge, or mapping-clear operator.

Replacing a parent must not leave effective metadata for removed children.
Sequence histories describe whole-sequence replacement; index origins refer to
the surviving sequence, not an attempted correspondence with previous indices.

## Explicit imports and source identity

Imports are opt-in through the loading API. Reserve the root `_configkit` mapping
for library directives and remove it before application decoding. Initially its
only permitted field is `imports`, a sequence of declarations. Illustrative
fragment:

```yaml
_configkit:
  imports:
    - path: fragments/server.yml
    - glob: fragments/outputs/*.yml
      optional: true
```

Each declaration has exactly one nonempty `path` or `glob`; `optional` is a bool
and defaults to false. Unknown directive fields are errors. When imports are
disabled, the presence of `_configkit` is an error, not an ignored directive.

- Relative paths resolve from the declaring file's directory. Absolute import
  paths and network URLs are rejected.
- Expand declarations in list order, sorting each glob's matches
  lexicographically by slash-separated relative path, with no locale sorting.
  Use segment glob syntax (`*`, `?`, character classes), with no recursive `**`.
- Traverse depth-first: process a file's own contribution, then its declarations
  in order. Ordering stabilizes source lists and diagnostics; it never permits
  duplicate definitions to override one another.
- A required missing path or unmatched glob is an error. Optional suppresses
  only a nonexistent target or zero matches. Invalid patterns, permission
  failures, directories, and invalid matched documents remain errors.
- The configuration root defaults to the main file's directory; callers may
  explicitly supply another root. Resolve symlinks and cleaned absolute paths
  for identity and containment. Imports and profiles must stay inside that
  canonical root. Retain declared paths and import-site locations for messages.
- An active-stack repeat is a cycle; report the complete cycle and import sites.
  A previously visited canonical file outside the active stack is a repeated
  import error, including diamond graphs and overlapping glob matches. Report
  both inclusion routes. Hard links at distinct canonical paths remain distinct
  sources; this is path identity, not inode deduplication.
- Read each accepted source once per load. Do not promise an atomic snapshot of
  a filesystem being modified concurrently. Bound input bytes, node nesting,
  import depth, and source count with documented limits and structured errors.

Bytes/reader sources require a diagnostic label. Imports and profiles for these
sources require an explicit filesystem base directory and root; never infer a
filesystem location from a label such as an S3 URI. Without that context they
remain unavailable. ConfigKit does not fetch remote imports.

Debian-style automatic `conf.d` discovery is deferred. A caller may explicitly
import a file or glob inside a directory named `conf.d`, but that name has no
special behavior. Do not scan adjacent directories implicitly.

## Profiles

Load only the composed base unless the application selects a profile. The API
accepts a name matching `[a-zA-Z0-9_-]+`, not an arbitrary path. The empty name
and reserved lowercase name `default` both mean no overlay; a literal
`profiles/default.yml` is never loaded. Other names resolve exactly to
`<configuration-root>/profiles/<name>.yml`, without extension fallback or case
normalization. A selected missing profile is an error.

A profile is one policy-validated YAML mapping applied using the overlay table.
It must not contain `_configkit`, introduce imports, or alter the application's
declared schema-version path. Reject these directives even if their values
would be unchanged. There is no profile inheritance or multiple-profile stack.
Applications own flags and commands that select this behavior.

## Static defaults and null

Applications supply a defaults factory that creates a fresh configuration per
load. Mutable maps, slices, and pointers must not be shared between factory
results. ConfigKit must not retain mutable aliases between returned loads or
between the application result and its internal metadata snapshots.

Defaults obey the same recursive mapping/atomic replacement rules as a lower
layer, rather than incidental decoder behavior:

| Configured input | Effect on defaults |
| --- | --- |
| Absent field | Keep the default |
| Explicit false, zero, or empty string | Replace the default |
| Partial mapping, including a string-keyed Go map | Retain omitted default children/entries |
| `{}` over an existing mapping | Retain its default children |
| Sequence, including `[]` | Replace the entire default sequence |
| Partial object represented by a pointer | Retain children of a non-nil default object; otherwise allocate and use ordinary zero values for omitted children |
| Allowed null | Replace with nil; suppress lower defaults |

The factory supplies actual defaults; ConfigKit does not invent per-entry
defaults for newly introduced map entries or sequence elements. Application
resolution may supply those. Go zero values are part of the initial typed
baseline; do not imply that the application explicitly declared every zero as
a domain default. Avoid serialization through `omitempty` that would erase
baseline values or presence information.

Null is valid for pointers, maps, slices, and dynamic `any` values. It is invalid
for ordinary scalar fields, structs, and arrays. Thus `workers: null` cannot
silently leave an integer default intact. Applications can further reject null
through validation. Null never means reset to the application default.

## Typed decoding

Strictness must be enforced by ConfigKit across the effective tree:

- Unknown struct fields, type mismatches, overflow, and invalid null are errors.
- Support ordinary exported fields and `yaml` names/ignore tags. Untagged field
  names use the lowercase Go field name. Embedded structs follow the same rule
  unless explicitly tagged `yaml:",inline"`; inline struct fields contribute to
  the containing mapping. Permit at most one inline string-keyed map per object,
  which receives otherwise unmatched keys. Duplicate declared field names and
  default inline-map entries colliding with declared fields are schema errors.
  Without an inline map, unmatched keys remain errors. Ignore `omitempty` for
  decoding, defaults, and effective inspection.
- String-keyed maps accept application-defined keys. Struct values inside them
  remain strict. `map[string]any` permits arbitrary keys and recursively accepts
  the supported scalar/container model; application validation owns its schema.
- Do not coerce numbers or booleans to strings, or quoted numbers to numeric
  fields. Integer targets require integer syntax and range checks. Float targets
  accept finite decimal numbers subject to representable range checks.
- Dynamic integer values use checked `int64`, and fractional/exponent values use
  checked finite `float64`; values outside those ranges require a typed schema
  or a quoted string rather than silent rounding of integer nodes.
- Support `time.Duration` from a duration string. Custom scalar types must
  implement both `encoding.TextUnmarshaler` and `encoding.TextMarshaler` so
  decoding and inspection have an agreed representation. Duration integers are
  rejected. See the [type and projection contract](#supported-types-and-projection).
- Parser-specific unmarshal hooks are not a public extension mechanism. Do not
  let them bypass key, null, or type policy. Custom structural decoders are
  deferred; unsupported type shapes fail with a clear schema diagnostic.

Keep parser nodes and errors internal. Exact package choice and Go API spelling
are implementation decisions, to be exercised against consumer types before
stabilization. The field-binding model must also support defaults, provenance,
secret classification, and rendering consistently.

## Recorded resolution and validation

Application-owned resolution runs after decoding and may derive absent values
or normalize explicitly configured ones. Provide a narrow resolution context
that exposes metadata and records each changed semantic path, operation kind
(derived default or normalization), and a safe reason. Dependency paths may be
recorded when a value is derived from other fields.

A resolver may mutate the typed configuration, but ConfigKit compares its
before/after configuration projection and requires records for changed paths.
Unrecorded changes are resolution errors. Container replacement records cover
changed descendants. Capture previous/final values internally from the actual
configuration rather than trusting separately supplied values. Application
resolvers own meaningful reasons; a diff alone cannot explain intent. Reasons
must not embed secrets. The proposed [resolution API](#resolution-and-validation-callbacks)
provides helpers to combine assignments and records. Snapshot comparison verifies
net changes during the hook; it does not reconstruct intermediate assignments.
A record describes the hook's before/final transition and declared intent, not an
automatically recovered event log. Equal-valued derived-default assignments may
still have a record identifying their intent. Multiple records at the same path
are coalesced for that hook, retaining declared reasons and dependencies.

The final configuration, effective output, presence queries, and explanation
records must agree. Do not render the pre-resolution YAML tree as the final
configuration. ConfigKit promises an independently owned snapshot it does not
subsequently mutate, not language-enforced immutability of exported Go values.
Callers should treat returned configuration as read-only; later caller mutations
are outside the captured metadata snapshot.

Application semantic validation runs after successful resolution and collects
independent errors in a standard diagnostic form. Validators are read-only and
own domain requirements; checking live S3 buckets or starting runtime clients
is outside this lifecycle. Do not run later hooks on an invalid typed value.

## Provenance, diagnostics, and redaction

Retain input sources, layer, line/column where available, import sites, effective
origins, overridden definition records, and resolution operations. Container
origins may be composite; never invent one file location for a mapping assembled
from several sources. Defaults and resolution have synthetic origins rather
than fabricated file coordinates. Preserve definition history without exposing
mutable parser nodes or unrestricted raw value references through the API.

Diagnostics have stable codes, severity, semantic path when available, a safe
message, primary location, and related locations. Sort deterministically by
source traversal order, location, path, and code. Expose structured errors for
programmatic inspection; human message wording need not be a compatibility
contract. Fatal parse/graph failures may stop dependent work; collect independent
schema and semantic errors where continuing is meaningful.

On any error, loading returns no usable configuration. A structured error may
carry safe diagnostics and sources discovered so far. Successful results carry
configuration, metadata, sources, and nonfatal diagnostics. Avoid ambiguous
partial-success results or two competing authoritative error lists.

Redaction is part of the initial lifecycle, including parse and decode failures.
Never blindly forward parser/custom-decoder error text or source snippets that
may contain values. Report safe expected-type and location information instead.
Applications likewise must supply value-free validation messages and resolution
reasons. ConfigKit cannot sanitize secrets embedded in arbitrary application
prose reliably.

Use `configkit:"secret"` on fields and an application path-classification hook
for dynamic entries. A secret container makes descendants secret. Secret
classification applies conservatively across current and historical definitions
of a path: replacing a secret does not declassify its history. It propagates from
recorded secret dependencies to derived outputs. Applications remain responsible
for marking secret-producing transformations whose dependencies are not recorded.

Provide source reporting and redacted effective rendering in the initial target.
Render from a canonical projection of the final typed snapshot, including zero
values and null/empty distinctions supported by the schema. Redacted output is
for inspection and is not promised to be reloadable. ConfigKit renderers must
not expose raw secrets; returning typed configuration to the application is an
intentional trusted boundary. ConfigKit does not log configuration on its own.

Retain sufficient history for later full `explain` tooling, but defer historical
value rendering, config diffs, and fingerprints. A future fingerprint design
must specify canonicalization, secret handling, and disclosure risks before
claiming identity or reproducibility guarantees.

## Application schema versions

Applications may declare a schema-version path (for example `/schema_version`)
and a pre-decode checker. The main document owns this value; fragments omit it,
and profiles cannot supply or change it. Check fragment placement before
composition so a version supplied only by an import is also rejected. ConfigKit enforces that placement when
the application enables the mechanism. The application decides whether a version
is required and which values it accepts. ConfigKit does not require one universal
version field or interpret an application's version as the library version.
Automatic migration is deferred.

## Consumer API design

The following is a proposed public API, not implemented Go reference documentation
or a frozen naming commitment. Declarations omit method bodies and private fields.
Consumer snippets assume the indicated standard-library imports and a `configkit`
import using the module path selected in the [implementation plan](implementation.md#implementation-decisions).
They are illustrative target examples, not currently runnable examples. The
implementation plan owns their conversion into maintained executable examples
and verification against consumer models.

### Entry point and reusable application definition

```go
func Load[T any](source Source, options ...Options[T]) (*Result[T], error)

type Definition[T any] struct {
    Defaults func() T
    Resolve  func(*T, *Resolution) error
    Validate func(*T, *Validation)
    Version  *VersionRule
    Secret   func(Path) bool
}

type Options[T any] struct {
    Definition Definition[T]
    Imports    bool
    Profile    string
    Filesystem *FilesystemContext
}

type FilesystemContext struct {
    BaseDir string
    Root    string
}

func File(path string) Source
func Bytes(label string, data []byte) Source
func Reader(label string, input io.Reader) Source
func NoSource() Source
```

`Load` accepts zero or one options value; multiple values are an options error,
not a precedence mechanism. The optional argument preserves the shortest load
without requiring a builder or an option function per setting. With no options,
`T` starts at its zero value, hooks are absent, and imports/profiles are disabled.
Strict YAML policy, decoding, presence, and safe diagnostics are always active;
there is no permissive-mode switch.

`Definition[T]` contains reusable application rules; `Options[T]` contains the
per-load source environment and selection. Applications may reuse a definition
for many loads. Treat definitions as read-only after sharing them. Each load
invokes the defaults factory afresh, creates new callback contexts, and owns its
result. Callbacks must not retain these contexts or configuration pointers; shared
closures must be safe for concurrent calls if the application loads concurrently.
ConfigKit does not synchronize application-owned captured state.

`Source` is opaque with the constructors above, not an arbitrary provider
interface. `Bytes` snapshots its input; `Reader` is consumed during the load and
is not closed by ConfigKit. A reader source is single-use. File sources are read
on each load. Constructor argument errors surface as load diagnostics. Applications must use
non-secret source labels because labels appear in diagnostics and source reports.
Filesystem context follows the [source rules](#explicit-imports-and-source-identity):
a file's own directory supplies its base, and an explicit context can set its
root. Supplying `BaseDir` for a file is an options error; bytes/readers require
both absolute `BaseDir` and `Root` when imports or profiles are used. A base must
be inside its root. These options do not change application field path meanings.

`NoSource()` explicitly selects defaults-only loading. It contributes no document
or source entry and marks no fields explicit; it still runs schema checking with
an absent version, defaults, resolution, validation, and snapshot/inspection.
An application requiring an input version can therefore reject this mode.
Imports, an active profile, or filesystem context with `NoSource()` are options
errors. A zero `Source` is invalid and is not an implicit defaults-only request.
Empty files and empty byte/reader inputs remain invalid YAML inputs.

The application decides whether its default path may be absent and explicitly
selects `NoSource()` in that case. ConfigKit never suppresses missing-file errors
on `File`; an explicitly selected missing file must still fail. Applications must
not treat permission or parse failures as evidence that no file was provided.

A minimal application needs only its type and one load:

```go
type Config struct {
    Enabled bool `yaml:"enabled"`
    Workers int  `yaml:"workers"`
}

func ReadConfig(path string) (Config, error) {
    result, err := configkit.Load[Config](configkit.File(path))
    if err != nil {
        return Config{}, err
    }
    return result.Config, nil
}
```

Adding defaults and validation remains ordinary application code:

```go
var workerRules = configkit.Definition[Config]{
    Defaults: func() Config { return Config{Enabled: true, Workers: 4} },
    Validate: func(cfg *Config, v *configkit.Validation) {
        if cfg.Workers < 1 {
            v.Error(configkit.Keys("workers"), "must be at least one")
        }
    },
}

func ReadWorkerConfig(path string) (*configkit.Result[Config], error) {
    return configkit.Load(configkit.File(path), configkit.Options[Config]{
        Definition: workerRules,
    })
}
```

The same application rules support defaults-only operation:

```go
func ReadWorkerDefaults() (*configkit.Result[Config], error) {
    return configkit.Load(configkit.NoSource(), configkit.Options[Config]{
        Definition: workerRules,
    })
}
```

### Paths and metadata

```go
func Keys(keys ...string) Path
func ParsePath(pointer string) (Path, error)
func (p Path) Key(key string) Path
func (p Path) Index(index int) Path
func (p Path) String() string

func (m Metadata) Lookup(path Path) (FieldInfo, bool)
func (m Metadata) IsExplicit(path Path) bool
func (m Metadata) History(path Path) []Change
```

`Path` is an immutable key/index representation with the textual form specified
in [paths and presence](#paths-and-presence). `Keys()` is the root. Key/index
builders escape dynamic names correctly; applications should not concatenate
pointer strings. Invalid indices and paths supplied to hooks become structured
errors. Path spelling uses YAML names throughout.

`FieldInfo` reports explicitness, null state, and effective `Origin`. An origin
identifies defaults, input, or application resolution and associated source
locations; it may be composite. `Change` reports layer/operation, prior origins,
safe reasons, dependency paths, and related locations, without raw historical
values. These are library-owned metadata types, never parser nodes. Queries
return copies or immutable views. Unknown paths yield no lookup/history and
false explicitness. Callback mutation/validation paths are checked more strictly:
misspelled struct fields are errors, while valid missing map entries are allowed.

### Resolution and validation callbacks

```go
func (r *Resolution) IsExplicit(path Path) bool
func (r *Resolution) Record(path Path, kind ChangeKind, reason string, dependencies ...Path)

func Default[V any](r *Resolution, path Path, target *V, value V, reason string, dependencies ...Path)
func Normalize[V any](r *Resolution, path Path, target *V, value V, reason string, dependencies ...Path)

func (r *Resolution) Error(path Path, message string, related ...Path)
func (r *Resolution) Warning(path Path, message string, related ...Path)
func (v *Validation) Error(path Path, message string, related ...Path)
func (v *Validation) Warning(path Path, message string, related ...Path)
```

`ChangeKind` distinguishes derived defaults and normalization. `Default` assigns
and records only when the effective input at the path is not explicit; thus it
may replace a static baseline. `Normalize` assigns and records a changed value
whether the input was explicit or defaulted. Both helpers accumulate invalid-path
or record errors in the resolution context, checked before validation. They do
not require applications to construct source locations or duplicate value history.
Targets must correspond to the declared path in the final configuration; ConfigKit
checks the projected path value against the helper's assigned value and audits
all net changes for coverage. This is consistency validation, not proof of the
identity of an arbitrary Go pointer supplied by application code.

For a multi-field transformation, the application may mutate normally and call
`Record` for affected paths or a replaced container. The before/final comparison
requires coverage of net changes; it does not infer reasons or promise all
intermediate mutations. The helper/record design must be exercised on consumer
code before stabilization; do not expand bookkeeping merely to make an internal
audit easier to implement.

Validation receives the resolved configuration by pointer for ergonomic reuse of
application code but must not mutate it, including nested maps or slices. Its
context collects independent messages; ConfigKit supplies diagnostic codes and
available primary/related locations. A missing field uses the nearest relevant
container/source location, or a synthetic location when no input source applies.
Application-provided messages and reasons must be safe, value-free prose.

Resolvers and validators can report safe domain messages through `Error` and
`Warning`, specifying one primary path and optional related paths. ConfigKit
supplies phase-specific codes and attaches available locations to all paths.
Related paths are useful for duplicates and cross-field constraints; no caller
must construct file coordinates. Reported errors fail the load even when a
resolver returns nil. Warnings alone do not fail it. Stop before the next phase
when a hook has reported errors.

A non-nil resolver error also aborts the load. If the context already contains
errors, retain those without adding a redundant generic failure; otherwise emit
a safe hook-failure diagnostic. Never forward arbitrary error text by default.
Only the explicit reporting methods opt into human-visible application messages;
callers must not interpolate secrets or attach raw underlying errors as messages.
Validators need no multi-error wrapper.
Callback panics are programming failures and are not converted to configuration
validation errors. Contexts are valid only during their callback and are not safe
for concurrent use within a callback.

A Narratio-like application can retain typed domain logic, including defaults for
new map entries. This illustrative example uses `strings` and `time`:

```go
type Artifact struct {
    Output  string        `yaml:"output"`
    Timeout time.Duration `yaml:"timeout"`
}

type PipelineConfig struct {
    PollInterval time.Duration       `yaml:"poll_interval"`
    Timeout      time.Duration       `yaml:"timeout"`
    Artifacts    map[string]Artifact `yaml:"artifacts"`
    APIKey       string             `yaml:"api_key" configkit:"secret"`
}

func resolvePipeline(cfg *PipelineConfig, r *configkit.Resolution) error {
    configkit.Default(r, configkit.Keys("timeout"), &cfg.Timeout,
        cfg.PollInterval/10, "derived from poll interval",
        configkit.Keys("poll_interval"))

    for name, artifact := range cfg.Artifacts {
        path := configkit.Keys("artifacts", name)
        configkit.Default(r, path.Key("timeout"), &artifact.Timeout,
            cfg.Timeout, "inherited pipeline timeout", configkit.Keys("timeout"))
        configkit.Normalize(r, path.Key("output"), &artifact.Output,
            strings.TrimSpace(artifact.Output), "trimmed surrounding whitespace")
        cfg.Artifacts[name] = artifact
    }
    return nil
}

func validatePipeline(cfg *PipelineConfig, v *configkit.Validation) {
    if cfg.PollInterval <= 0 {
        v.Error(configkit.Keys("poll_interval"), "must be positive")
    }
    if cfg.Timeout <= 0 {
        v.Error(configkit.Keys("timeout"), "must be positive")
    }
    for name, artifact := range cfg.Artifacts {
        path := configkit.Keys("artifacts", name)
        if artifact.Timeout <= 0 {
            v.Error(path.Key("timeout"), "must be positive")
        }
        if artifact.Output == "" {
            v.Error(path.Key("output"), "must be nonempty")
        }
    }
}

var pipelineRules = configkit.Definition[PipelineConfig]{
    Defaults: func() PipelineConfig {
        return PipelineConfig{PollInterval: time.Minute}
    },
    Resolve:  resolvePipeline,
    Validate: validatePipeline,
}

func ReadPipeline(path, profile string) (*configkit.Result[PipelineConfig], error) {
    return configkit.Load(configkit.File(path), configkit.Options[PipelineConfig]{
        Definition: pipelineRules,
        Imports:    true,
        Profile:    profile,
    })
}

func ReadPipelineBytes(label string, data []byte) (*configkit.Result[PipelineConfig], error) {
    return configkit.Load(configkit.Bytes(label, data), configkit.Options[PipelineConfig]{
        Definition: pipelineRules,
    })
}
```

Map entries are copied out and written back using ordinary Go code; helpers audit
the final projection. Independent map traversal order must not affect sorted
records or diagnostics. Interdependent application transformations must use an
application-defined deterministic order. Campaign/session fallback can similarly
remain application code; this example does not turn it into ConfigKit composition.

A validator can report both sides of a duplicate without constructing source
locations. This example uses `sort` to make primary-path selection deterministic:

```go
func validateUniqueOutputs(cfg *PipelineConfig, v *configkit.Validation) {
    names := make([]string, 0, len(cfg.Artifacts))
    for name := range cfg.Artifacts {
        names = append(names, name)
    }
    sort.Strings(names)
    seen := make(map[string]configkit.Path)
    for _, name := range names {
        output := cfg.Artifacts[name].Output
        if output == "" {
            continue // The ordinary field validator reports this separately.
        }
        path := configkit.Keys("artifacts", name, "output")
        if first, exists := seen[output]; exists {
            v.Error(path, "output must be unique", first)
        } else {
            seen[output] = path
        }
    }
}
```

Call this helper from the application's validator when output uniqueness is a
domain requirement. The message identifies the relationship without printing
potentially sensitive configured values.

### Schema checking and secret classification

```go
type VersionRule struct {
    Path  Path
    Check func(Version, *Validation) error
}

type Version struct {
    Present bool
    Null    bool
    Text    string
}
```

The version checker receives the main document's scalar at the declared path,
with absent/null states preserved. Non-string, non-null values are schema-version
errors before the hook; the application decides whether absent/null or a string
is acceptable. The checker reports actionable safe messages through its validation
context, using the same primary/related-path API as other hooks. At this pre-decode
phase, locations resolve against the composed input rather than a typed schema;
a missing path falls back to the nearest input container. No parser tree is
exposed. Reported errors or a non-nil return fail the load. A non-nil return with
no reported error produces a generic safe version-check failure at the declared
version path; raw error text is not printed. Placement and timing follow
[schema versioning](#application-schema-versions). A rule with no checker is an
options error.

For example, an application can preserve useful migration guidance without
embedding input values in its error text:

```go
var schemaVersionPath = configkit.Keys("schema_version")

func checkVersion(version configkit.Version, v *configkit.Validation) error {
    if !version.Present || version.Null {
        v.Error(schemaVersionPath, "schema_version is required")
    } else if version.Text != "example.v1" {
        v.Error(schemaVersionPath, "unsupported schema version; migrate using the application configuration guide")
    }
    return nil
}

var versionRule = configkit.VersionRule{
    Path:  schemaVersionPath,
    Check: checkVersion,
}
```

Assign `&versionRule` to a definition's `Version` field to enable it. Schema
versions remain strings under ConfigKit's convention; adopting consumers with
integer versions must migrate their syntax rather than enable a second scalar
policy.

`Definition.Secret` is an optional pure path predicate for dynamic entries,
combined with secret field tags. It can add sensitivity but cannot override a
tag to declassify a value. It is applied to historical as well as final paths.
Recorded dependencies propagate sensitivity under the existing redaction rules;
applications own secret-producing transformations not described by those records.

### Results, errors, and inspection

```go
type Result[T any] struct {
    Config T
    // Private snapshot and metadata storage.
}

func (r *Result[T]) Metadata() Metadata
func (r *Result[T]) Sources() []SourceInfo
func (r *Result[T]) Diagnostics() []Diagnostic
func (r *Result[T]) WriteEffective(output io.Writer) error

type LoadError struct {
    // Private diagnostic and source storage.
}

func (e *LoadError) Error() string
func (e *LoadError) Diagnostics() []Diagnostic
func (e *LoadError) Sources() []SourceInfo
func WriteDiagnostics(output io.Writer, diagnostics []Diagnostic) error
```

Every configuration/source/options/hook failure returns `(nil, *LoadError)`.
Success returns a non-nil result and nil error; its diagnostics contain only
nonfatal warnings. Source lists use stable source identifiers and safe labels;
diagnostics expose code, severity, path, message, primary location, and related
locations as described [above](#provenance-diagnostics-and-redaction). No raw
parser error or unrestricted secret-bearing history is exposed. No automatic
logging occurs. Inspection methods returning slices provide defensive copies.

`Config` is caller-owned typed data. Metadata and rendering use an independently
captured final snapshot, so later caller mutation cannot rewrite the recorded
result. `WriteEffective` emits deterministic redacted YAML for inspection, with
no unredacted switch. It returns output errors rather than a new configuration validation result.
Custom projection failures must already have been rejected before a load
succeeds. Output can be partial on writer failure.

An application can report errors and inspect a successful load without rebuilding
diagnostic formatting. This example uses `errors` and `io` and the pipeline
loader above:

```go
func InspectPipeline(path, profile string, output io.Writer) error {
    result, err := ReadPipeline(path, profile)
    if err != nil {
        var loadErr *configkit.LoadError
        if errors.As(err, &loadErr) {
            if writeErr := configkit.WriteDiagnostics(output, loadErr.Diagnostics()); writeErr != nil {
                return writeErr
            }
        }
        return err
    }
    if err := configkit.WriteDiagnostics(output, result.Diagnostics()); err != nil {
        return err
    }
    return result.WriteEffective(output)
}
```

Applications can use `Metadata().Lookup`, `Metadata().History`, and `Sources()`
for structured inspection without implementing value rendering or importing a
parser. A complete historical-value `explain` command remains deferred.

### Module-specific options

Use a canonical object entry with an explicit module identifier and options
mapping. Shorthand strings and custom structural decoders are not required to
express the domain information. For a known application module catalog, model
the available option fields with ordinary typed structs; application validation
selects the meaningful fields and rejects incompatible combinations.

Illustrative YAML for an application that supports two modules:

```yaml
modules:
  - module: chunker
    options:
      window_units: 12
  - module: discussion
    options:
      sections: [short_term]
```

The corresponding typed configuration and validator need no second parser:

```go
type ModuleOptions struct {
    WindowUnits *int     `yaml:"window_units"`
    Sections    []string `yaml:"sections"`
}

type ModuleSpec struct {
    Module  string        `yaml:"module"`
    Options ModuleOptions `yaml:"options"`
}

type ModuleConfig struct {
    Modules []ModuleSpec `yaml:"modules"`
}

func validateModules(cfg *ModuleConfig, v *configkit.Validation) {
    for index, spec := range cfg.Modules {
        path := configkit.Keys("modules").Index(index)
        switch spec.Module {
        case "chunker":
            if spec.Options.WindowUnits == nil || *spec.Options.WindowUnits < 1 {
                v.Error(path.Key("options").Key("window_units"), "must be a positive integer")
            }
            if spec.Options.Sections != nil {
                v.Error(path.Key("options").Key("sections"), "not supported by this module")
            }
        case "discussion":
            if spec.Options.WindowUnits != nil {
                v.Error(path.Key("options").Key("window_units"), "not supported by this module")
            }
            if len(spec.Options.Sections) == 0 {
                v.Error(path.Key("options").Key("sections"), "must contain at least one section")
            }
        default:
            v.Error(path.Key("module"), "unsupported module")
        }
    }
}

func ReadModuleConfig(path string) (*configkit.Result[ModuleConfig], error) {
    return configkit.Load(configkit.File(path), configkit.Options[ModuleConfig]{
        Definition: configkit.Definition[ModuleConfig]{Validate: validateModules},
    })
}
```

The pointer and nil-slice states above express optional domain values; this
application treats explicit null as unselected for those optional alternatives.
ConfigKit rejects unknown fields and invalid scalar types. The application owns
which supported options apply to a module and their domain constraints. Larger
catalogs may use nested typed option objects rather than one flat options struct.
Neither shape requires a library registry, parser hook, or configurable merge rule.

For an application whose option schema is genuinely selected at runtime,
`map[string]any` remains available. ConfigKit owns parsing, paths, and the single
dynamic scalar representation; application validation owns the selected schema,
including allowed keys and value constraints. Conversion into runtime module
arguments is application code. Dynamic integers remain checked `int64`; existing
`int` assertions must be adapted with range checks when converting to `int`.

The initial API does not add a generic subtree-conversion helper. Prefer typed
configuration where option shapes are known, and direct domain validation where
they are dynamic. Do not introduce a second YAML marshal/decode round trip as the
blessed pattern. Reconsider a narrow conversion helper only if a concrete dynamic
catalog demonstrates substantial duplicated type-binding infrastructure that
ordinary models cannot remove. Such a helper would have to retain strictness and
source attribution; that is a separate justified extension, not an escape hatch
for legacy shorthand. Configuration inspection describes the validated input
model, not the application's separately constructed runtime module objects.

### Supported types and projection

The supported root `T` is a struct or string-keyed map. Within it, support ordinary
exported scalar fields, pointers, structs, string-keyed maps, slices, fixed-length
arrays, dynamic `any`, and the explicitly supported scalar conversions. Reject
unsupported kinds, non-string map keys, cyclic object graphs, and ambiguous field
bindings as schema/projection errors. Arrays require matching sequence length.
`yaml:"-"` fields and unexported fields are outside the configuration projection;
hooks must not use them to carry runtime resources or configuration needing
inspection. Non-configuration runtime state belongs in separate application types.

One field-binding model supplies defaults, decoding, path lookup, change auditing,
and rendering. `omitempty` never removes projected zero values. Preserve nil
versus empty maps/slices where represented by the type. Shared pointer identity
inside application values is not a configuration semantic; cyclic values are
unsupported and independent load/snapshot ownership must still hold.

`time.Duration` projects to a canonical duration string. A custom scalar's text
marshal method supplies its deterministic canonical string; unmarshalling must
accept that representation and preserve its configuration meaning. These methods
must be side-effect-free with respect to shared state. Custom scalars are atomic
for comparison/history; their internal fields are not traversed. Reject types
with only a text decoder and no supported inspection representation. Conversion
or projection failures produce safe diagnostics. This is a bounded scalar seam,
not a custom structural decoder or arbitrary reflection extension framework.

## Internal organization

Keep document composition, overlays, and parser machinery internal initially.
Applications use the typed lifecycle; application-specific resolution alone does
not justify exposing a document-processing toolkit.

Internally separate source access, policy-validated documents, composition,
overlay, field binding/decoding, metadata, resolution, and safe rendering.
These are responsibilities, not a mandate for public interfaces or a file per
concept. Start with one public package and a private YAML adapter; add package
boundaries only when dependencies and ownership warrant them.

## Deferred scope

The initial target excludes:

- automatic `conf.d` discovery;
- remote imports, arbitrary providers, and additional configuration formats;
- environment or CLI field overlays;
- custom merge strategies, sequence merging, reset operators, and profile inheritance;
- parser-specific or custom structural decoding extension APIs;
- generic dynamic-subtree conversion and module-registry APIs;
- automatic schema migrations;
- hot reload, runtime resource replacement, and remote resource validation;
- a bundled CLI framework or commands;
- full historical value rendering, config diffs, and fingerprints.

Applications can build their own validate/effective/sources commands on the
library. Broadening these boundaries requires concrete consumer evidence and
an explicit design change.