Finalize the ConfigKit roadmap and implementation plan

README.md

@@ -6,4 +6,5 @@ ConfigKit is in pre-implementation. This repository currently contains its desig
 and contributor policies; no usable Go library has been implemented yet.
 
 The [initial target design](docs/roadmap/initial.md) describes planned behavior.
+The [implementation plan](docs/roadmap/implementation.md) owns the pending stages.
 See the [development guide](docs/development.md) for repository orientation.

docs/adr/0004-secret-references-and-explicit-bootstrap.md

@@ -0,0 +1,43 @@
+# ADR 0004: Use secret references and explicit environment bootstrap
+
+Status: accepted
+
+Date: 2026-09-06
+
+## Context
+
+Consumers need credentials without storing them in YAML and need a consistent
+way to load locally provisioned secret files. WeatherReporter already implements
+useful directory-loading code. Ordinary configuration loading must remain free
+of implicit process mutation, and operators must understand blocked secret files.
+
+## Decision
+
+Represent secrets as ConfigKit-owned typed references to environment-variable
+names. Resolve only surviving references; retain names, never credentials, in
+configuration trees and inspection. Offer directory bootstrap as a separate,
+explicit startup operation, reusing WeatherReporter code where applicable.
+Existing environment variables win even when empty. Report skipped files and
+partial failures without values. Keep bootstrap provenance separate from later
+lookup provenance rather than imply an unchanged environment between calls.
+General environment-variable configuration overlays are not on the roadmap.
+The [feature roadmap](../roadmap/initial.md#secret-references-and-explicit-bootstrap)
+owns the unimplemented contract; stages remain in the implementation plan.
+
+## Alternatives considered
+
+- Literal secrets or interpolated YAML: familiar but allows credentials into
+  configuration artifacts and expands conventions and redaction obligations.
+- Application-only directory loaders: keeps process mutation outside the library
+  but preserves duplicated infrastructure and inconsistent skip/error semantics.
+- Implicit bootstrap during loading: convenient but hides process mutation and
+  makes repeated or concurrent loads difficult to reason about.
+- Directory values overriding existing environment: simple, but defeats explicit
+  deployment environment choices and obscures why a credential was selected.
+
+## Consequences
+
+Applications migrate secret fields, choose startup ordering, and retain bootstrap
+reports. ConfigKit owns bounded loading, reference validation, safe reporting and
+inspection; applications own credential use. No multi-variable transaction or
+protection against concurrent process-environment mutation is promised.

docs/adr/0005-explicit-overrides-and-safe-explanation.md

@@ -0,0 +1,41 @@
+# ADR 0005: Record explicit CLI overrides and safe value explanation
+
+Status: accepted
+
+Date: 2026-09-06
+
+## Context
+
+Consumers need deliberate per-invocation configuration changes and an explanation
+of effective values. Presence and origin records alone cannot show which values
+were replaced. Adding CLI parsing or arbitrary environment overlays would broaden
+the library beyond its consistent configuration model.
+
+## Decision
+
+Use parser-neutral explicit path/value/flag records above selected profiles.
+Defaults remain below strictly composed base input. Apply overrides to the owned
+node tree before typed decoding, retaining the existing parser adapter boundary.
+Recorded domain resolution follows input selection. Provide safe structured
+historical values and an explanation renderer in the initial release, building
+on ADR 0003's presence and audited resolution decision. Applications own commands.
+The [feature roadmap](../roadmap/initial.md#cli-overrides) owns the exact override
+contract and [explanation contract](../roadmap/initial.md#historical-value-explanation).
+
+## Alternatives considered
+
+- Retire configuration flags in favor of profiles: smaller API, but removes useful
+  explicit invocation choices that fit a single fixed precedence model.
+- Mutate structs after loading: easy locally, but bypasses validation and leaves
+  metadata and effective inspection inconsistent.
+- Defer historical values: reduces initial implementation scope but cannot answer
+  why an effective value differs from defaults or an earlier file.
+- Publish raw history trees: flexible but couples consumers to internals and risks
+  leaking sensitive values. Detached, redacted snapshots preserve the boundary.
+
+## Consequences
+
+The library owns override validation and source attribution without choosing a
+CLI framework. History requires bounded retained state and tests for replacement,
+deletion, composite origins and sensitivity. Credentials never enter history;
+secret reference names remain explainable unless explicitly classified sensitive.

docs/development.md

@@ -29,8 +29,12 @@ Accepted decisions:
 - [Own YAML semantics behind a private parser boundary](adr/0002-own-yaml-semantics.md)
 - [Preserve presence and record application resolution](adr/0003-preserve-presence-and-resolution-history.md)
 
-For cross-cutting changes, follow every applicable row. The implementation plan
-remains deferred; `docs/roadmap/implementation.md` has not been created.
+- [Secret references and explicit bootstrap](adr/0004-secret-references-and-explicit-bootstrap.md)
+- [Explicit overrides and safe explanation](adr/0005-explicit-overrides-and-safe-explanation.md)
+
+For cross-cutting changes, follow every applicable row. The
+[implementation plan](roadmap/implementation.md) owns the ordered stages and their
+acceptance checks; all remain pending until implementation evidence is recorded.
 
 ## Baseline validation
 

docs/policy/architecture.md

@@ -61,7 +61,10 @@ remote acquisition, runtime initialization, and resource checks.
 
 Library code must not depend on consuming applications or initialize their
 runtime resources. It must not perform implicit network access, discover
-application-specific paths, or change process-global state to load configuration.
+application-specific paths, or change process-global state during ordinary `Load`.
+The explicit secrets-directory bootstrap is a narrow, opt-in exception: applications
+invoke it during startup, and it reports writes, skips and partial failures. It
+must never run implicitly as part of configuration loading.
 Remote data can enter through application-supplied input without introducing a
 network provider into the library.
 
@@ -90,10 +93,20 @@ parser defaults. Processing order and diagnostics must be deterministic for the
 same inputs and options. Invalid input must not produce a successful usable
 configuration. Do not run application hooks on invalid typed configuration.
 
-Preserve presence and source information through processing. Ordinary Go zero
+User-controlled precedence is application defaults, composed base, selected
+profile, then explicitly supplied CLI overrides. Applications own flag parsing;
+ConfigKit owns overlay semantics and provenance. Domain resolution follows these
+inputs and must remain recorded, not become a hidden override channel.
+
+Preserve node-based composition until the effective input is ready for typed
+decoding. Preserve presence and source information through processing. Ordinary Go zero
 values must not be used to infer whether input was absent. Final configuration,
 metadata, and effective inspection must agree, including application resolution
 changes; see [ADR 0003](../adr/0003-preserve-presence-and-resolution-history.md).
+Safe historical values and an explanation renderer belong to the initial target,
+including superseded/deleted paths and effective source attribution. Inspection
+must agree with the final typed snapshot and apply historical sensitivity rules.
+See [ADR 0005](../adr/0005-explicit-overrides-and-safe-explanation.md).
 Detailed syntax and operation tables belong to the target roadmap until their
 current configuration contract is implemented.
 
@@ -104,6 +117,14 @@ ConfigKit must not mutate a result after returning it. Ordinary exported Go
 values are not language-enforced immutable objects; document caller ownership
 and the lifetime of captured metadata without promising otherwise.
 
+Secret values enter configuration through typed environment-name references;
+literal credentials are not a supported configuration convention. Only an explicit
+accessor exposes a resolved credential to application code. Credential bytes must
+never enter composition trees, history, or inspection. General environment-variable
+configuration overlays are an explicit non-goal. Bootstrap must preserve already
+set variables, including empty ones, and report blocked files without contents.
+See [ADR 0004](../adr/0004-secret-references-and-explicit-bootstrap.md).
+
 Secrets must be considered throughout error handling and inspection. Library
 renderers must apply secret classification, and parser errors must not be
 forwarded blindly when they can include raw input. Do not log configuration

docs/policy/documentation.md

@@ -29,8 +29,7 @@ semantics, defaults, paths, diagnostics, or compatibility guarantees.
 - Roadmaps own unimplemented targets, deferred work, and execution status.
 
 The initial target and implementation plan have separate owners. Do not put
-staged steps or progress in `docs/roadmap/initial.md`. The future
-`docs/roadmap/implementation.md` will own those details when it is created.
+staged steps or progress in `docs/roadmap/initial.md`. `docs/roadmap/implementation.md` owns those details.
 As behavior lands, move its normative definition into the current contract and
 replace the roadmap definition with a link. Retire completed roadmap material
 as part of release preparation; ADRs preserve significant decision history.
@@ -65,13 +64,13 @@ handling, not secret values.
 | Release rules | `docs/policy/release.md` | Compatibility, integrity, gates, immutability, and exceptions |
 | Release procedure | `docs/release.md`, when operational | Exact maintainer commands, prerequisites, supported targets, and verification |
 | Exported Go API | Go documentation beside declarations | Symbol contracts, errors, ownership, and lifecycle API behavior |
-| YAML and resolution semantics | `docs/config.md`, as behavior lands | Syntax, composition, precedence, conversion, and metadata semantics; Go reference docs link here for shared rules |
+| YAML and resolution semantics | `docs/config.md`, as behavior lands | Syntax, composition, precedence, conversion, secret references/bootstrap, and explanation semantics; Go reference docs link here for shared rules |
 | Consumer walkthroughs | `docs/consumers/`, when useful | Task-oriented usage and consumer responsibilities, not duplicate API reference |
 | Component inventory | `docs/internal/overview.md` | Implemented components and links to their owners |
 | Component mechanics | Other `docs/internal/` files, when needed | Detailed implementation behavior and change recipes |
 | Architectural decisions | `docs/adr/` | Context, decisions, alternatives, consequences, and supersession history |
 | Initial target | `docs/roadmap/initial.md` | Unimplemented end-state and deferred feature scope |
-| Implementation plan | Future `docs/roadmap/implementation.md` | Stages, dependencies, acceptance gates, and progress |
+| Implementation plan | `docs/roadmap/implementation.md` | Stages, dependencies, acceptance gates, and progress |
 | Release readiness gaps | `docs/roadmap/release-readiness.md` | Outstanding release prerequisites and closure evidence |
 | Historical release notes | `docs/releases/` | Version-specific historical summaries required by release policy |
 | Complete examples | `examples/` and executable Go examples | Maintained artifacts; prose links instead of duplicating full examples |

docs/policy/testing.md

@@ -29,6 +29,15 @@ Use focused fuzzing for parser boundaries and semantic/filesystem path handling;
 keep deterministic seed regressions in the ordinary suite. Extended fuzz runs
 need a documented bounded invocation when introduced.
 
+Secret bootstrap tests must cover existing variables including empty values,
+preflight without writes, partial apply failures, and safe blocked-file reporting.
+Use controlled environment names and restore state; process-mutating tests must
+not run in parallel. Keep real filesystem coverage and use narrow private seams
+for deterministic environment/write failures. Verify full input precedence and
+safe historical values, including removed and replaced secrets, at their owning
+boundaries. No credential bytes may appear in inspection or failure fixtures'
+expected output; use disposable canaries to prove non-disclosure.
+
 Baseline commands belong in the [development guide](../development.md). Release
 gates belong in the [release policy](release.md) and its eventual runbook. This
 policy does not imply that those tools or tests already exist.

docs/roadmap/implementation.md

@@ -25,8 +25,8 @@ releases.
    Preserve unrelated work. This repository initially has documentation only.
 2. Read the linked feature sections for the requested stage. Follow links to
    current contracts if earlier work has moved a definition out of the roadmap.
-3. Extract applicable Narratio code and cases before writing equivalent new
-   mechanisms. Record source revision, symbols, adaptations, and retained notices.
+3. Extract applicable Narratio and WeatherReporter code and cases before writing equivalent new
+   mechanisms. Record source revision, symbols, and adaptations.
    Do not import sibling applications or their internal packages into this module.
 4. Implement only the stage boundary. Partial internals may be tested directly;
    do not export placeholder APIs, silently ignore unsupported options, or claim
@@ -63,9 +63,9 @@ Promote their externally observable parts into current contracts when implemente
 | --- | --- |
 | Module and package | Module `gitea.maximumdirect.net/eric/ConfigKit`, matching the existing origin's case; root package `configkit`. No `cmd` application. |
 | Go baseline | `go 1.25.0`; use a currently supported patched toolchain at or above that minimum for development, and verify minimum-version compatibility separately. The existing Narratio/Seriatim baselines and local 1.25.5 toolchain support this language baseline. No toolchain directive forcing a newer consumer toolchain. |
-| Parser | Pin `go.yaml.in/yaml/v3 v3.0.5` initially, inside `internal/yaml`. It preserves the v3 node-oriented extraction seam while using the YAML organization's maintained module. Do not inherit its scalar coercions. The [upstream v3 reference](https://pkg.go.dev/go.yaml.in/yaml/v3@v3.0.5) documents this version and provenance. |
+| Parser | Pin `go.yaml.in/yaml/v4 v4.0.0-rc.6` initially, inside `internal/yaml`. Use the v4 streaming loader and dumper with explicit internal options. This selects upstream's active development line while preserving the node-oriented extraction approach. Do not inherit its scalar coercions. The [exact RC reference](https://pkg.go.dev/go.yaml.in/yaml/v4@v4.0.0-rc.6) documents the selected dependency. |
 | Internal representation | A ConfigKit-owned tree in `internal/document`: ordered mapping entries, sequences, typed scalar kind plus original numeric spelling, null, key/value locations, and source ID. It imports neither the public package nor the YAML parser. The YAML adapter translates into it. |
-| Dependency direction | Public package coordinates document operations and reflection binding. Private parser imports `internal/document`; document code never imports parser or public package. Keep composition, binding, provenance, hooks, and rendering coordination in root-package private files until a demonstrated boundary needs another package. |
+| Dependency direction | Public package coordinates document operations and reflection binding. Only `internal/yaml` and its adapter tests import upstream YAML packages/plugins. The adapter imports `internal/document` and translates parser errors into ConfigKit-owned internal records; document code never imports parser or public package. Adapter signatures expose no upstream types to other ConfigKit packages. Keep composition, binding, provenance, hooks, and rendering coordination in root-package private files until a demonstrated boundary needs another package. |
 | Public spelling | Implement the feature roadmap's `Load`, `Definition`, `Options`, sources, path builders, metadata queries, hook helpers, version rule, result methods, and diagnostic methods as written. No functional-options framework or separate builder. |
 | Input limits | Fixed initial ceilings: 8 MiB per source, 64 MiB total loaded bytes, depth 256 for input/projected trees, 64 active import edges, and 1,024 accepted sources. Allow at most 1,000,000 input nodes in aggregate and 1,000,000 nodes in each composed/projected tree; copying a snapshot does not consume the input budget again. Bound glob directory-entry visits separately at 1,000,000 per load. Include the selected profile in source/byte counts. Root depth is zero. No public limit-tuning options initially. Test limit mechanisms with smaller private test parameters. |
 | Filesystem support | Initial supported targets: Linux and macOS on amd64/arm64. Test real symlink semantics on those systems. No Windows path-compatibility promise in this release. Inputs use slash-separated relative import syntax and native paths for selected file/root/base options. |
@@ -74,11 +74,129 @@ Promote their externally observable parts into current contracts when implemente
 | Error boundary | Safe library diagnostics with phase-specific codes; no raw parser/custom-hook error string or raw-value wrapper chain. `LoadError` supports `errors.As`; there is no initial `Unwrap` promise. I/O classifications are exposed as diagnostic codes. |
 | Consumer integration scope | Build copied, sanitized consumer-shaped models/fixtures and executable examples in ConfigKit. Actual changes to Narratio, Notarius, WeatherReporter, and Seriatim are separate consumer migration tasks. Do not claim those apps are migrated when these local examples pass. |
 | Publication scope | Prepare library source and a release runbook/checker. Do not tag, push, publish, or invent a completed release during these stages. Release-candidate validation/publication follows the adopted release procedure as a separate maintainer action. |
+| Test comparisons | Use `github.com/google/go-cmp/cmp`, initially module `github.com/google/go-cmp v0.7.0`, for useful structured test comparisons. Add the requirement when first imported by tests. No production imports or runtime use; do not equate nil and empty or reorder sequences where those differences are contractual. |
 
-The parser dependency choice is based on its own published documentation, not a
-promise that v3.0.5 will always be the newest patch. A later verified dependency
-fix can change the pin with focused YAML regression checks and normal review.
-Do not silently switch parser families or use `@latest` during implementation.
+The selected version is a prerelease, deliberately accepted for initial
+development. Do not use `@latest`, silently change the pin, or depend on untagged
+upstream changes. Reassess the available v4 release before ConfigKit publication;
+a newer RC or final release requires an explicit dependency update and rerunning
+the adapter, YAML-policy, diagnostics, limits, and rendering regressions. A final
+upstream version is preferred when it passes those checks; if the RC is retained,
+record that decision in release readiness and compatibility review.
+
+Follow the feature's [upstream isolation contract](initial.md#yaml-dependency-and-public-api-isolation).
+Translate upstream API changes internally without changing ConfigKit declarations
+or consumer requirements. Never re-export upstream aliases/options/plugins, expose
+parser errors through wrapping, or pass parser-created objects through `any`.
+Keep ConfigKit's semantic checks independent of upstream convenience options.
+The module requirement records the tested dependency; it does not lock a consuming
+application's entire module graph. Do not promise compatibility with arbitrary
+unverified upstream versions or add a selectable-parser framework.
+
+### Updated extraction map
+
+Use the source revisions and adaptation policies owned by the feature's
+[foundation](initial.md#existing-implementation-foundation) and
+[secret bootstrap](initial.md#secret-references-and-explicit-bootstrap) sections.
+Paths below are relative to the named sibling repository. Reinspect them in
+Stage 1 and record the exact revision actually copied. The maintainer is the sole
+author of the relevant application code and has authorized its reuse and relicensing
+as needed for ConfigKit; no separate license check is required for this extraction.
+Keep source provenance for technical traceability. This map assigns reuse
+work to existing stages; it does not change stage ordering or public contracts.
+
+| Stages | Source code and tests | Extraction instruction |
+| --- | --- | --- |
+| 3 | Narratio `internal/config/composition.go`, `load.go`, `loader_strict_test.go`, `composition_test.go` | Extract source parsing and trailing-document regressions. Translate to the private v4 adapter and ConfigKit YAML policy; do not copy raw parser error propagation. |
+| 10 | Narratio `composition.go`, `composition_test.go` | Adapt `mergeAdditiveCompositions`, claim validation and recursive merging. Preserve collision/source tests, replacing empty-mapping rejection with ConfigKit's operation table. |
+| 11–12 | Narratio `pipeline_composition.go`, `pipeline_composition_test.go` | Reuse explicit import processing and duplicate/source cases. Build recursive graph/glob support and bounded containment; replace root-only directives, extension restrictions, and `os.SameFile` identity rules. |
+| 13 | Narratio `mergeOverlayComposition`, `mergeOverlayNodes`, `selectPipelineProfile`, `loadPipelineProfileOverlays`, `pipeline_profiles_test.go` | Adapt overlay mechanics and independence tests. Replace null/kind rejection and declared/all-profile loading with ConfigKit's fixed selected-profile contract. |
+| 14–15, 19 | Narratio `effective_pipeline.go`, composition semantic records, `pipeline_profiles_test.go`, `internal/app/config_commands_test.go` | Extract deterministic effective values, composite ownership and safe inspection cases. Add locations, retained history and generic redaction; remove YAML round trips and domain-specific projection. Do not copy diff/digest APIs. |
+| 16 | WeatherReporter `internal/config/secrets.go`, secret cases in `config_test.go`; Narratio `internal/app/secrets_env.go`, `secrets_env_test.go`, `internal/fileops/read_bounded.go` | Combine WeatherReporter's read/apply separation and one-newline handling with Narratio's preserve-existing and bounded-read mechanisms. Adapt both to the agreed structured-report state machine. |
+| 20 | Narratio inspection/composition/profile tests; WeatherReporter `config_test.go`, `overrides_external_test.go` | Prove side-effect-free inspection, explicit-empty domain validation, loaded/constructed consistency, and nonmutating option normalization with sanitized consumer-shaped fixtures. |
+
+In every owning stage, record copied symbols/cases and intentional expectation
+changes in the extraction inventory. Retain domain algorithms in consumer examples;
+do not copy application frameworks, CLI commands, logging, digests, or provider
+registries into the public library. Existing upstream types, YAML reserialization,
+and partial provenance are adaptation work, not reasons to weaken the target.
+
+### Standard-library mechanisms and custom policy
+
+These are implementation choices for the existing contracts, not additional
+public extension APIs. Apply each row in its owning stages. The Go 1.25 baseline
+supplies the listed primitives; no additional runtime dependency is needed.
+
+| Work and stages | Selected mechanism | ConfigKit work and limits |
+| --- | --- | --- |
+| Source reads (3, 11) | `os.Open`, `io.LimitReader`, `io.ReadAll` | Read at most `min(perSourceLimit, remainingTotalBudget) + 1` bytes; reject excess before parsing. File size is an optional early check, never the sole enforcement. Preserve read failures rather than accept partial input. |
+| Source ownership (3) | `bytes.Clone`, `bytes.NewReader`, `strings.NewReader` | Reject oversized byte arguments before cloning, retaining the constructor error for `Load`. Close only ConfigKit-owned files. Keep a shared private reader-state pointer with `atomic.Bool.CompareAndSwap` to claim a reader before consumption; copying `Source` must not copy an in-use atomic or create another claim. |
+| Identity and containment (11–13) | `filepath.Abs`, `EvalSymlinks`, `Rel`; `os.OpenRoot`/`Root.Open` | Canonicalize root and targets, reject a relative result that is absolute, `..`, or begins with a parent component, and open the canonical root-relative target through `Root`. Preserve declared spelling separately. Do not use string-prefix checks, `os.SameFile`, or `os.DirFS` as a confinement guarantee. |
+| Glob matching (12) | `path.Match` | Use the feature's matching grammar and a small syntax check for forbidden recursive `**`, respecting escapes and character classes. Do not write another wildcard matcher or treat backslash as a native separator. Validate malformed patterns even when directories are empty or absent. |
+| Glob expansion (12) | `os.File.ReadDir(n)` with positive bounded batches | Write a pattern-directed walker with explicit error propagation, canonical containment, and a budget counting every examined entry. Process returned entries and errors correctly, including EOF. Sort bounded candidates before recursion and final accepted relative matches before source traversal. Avoid `filepath.Glob`/`fs.Glob` error suppression and unbounded whole-directory reads. |
+| Numeric conversion (3, 6) | `strconv.ParseInt`/`ParseUint` with base 10, `ParseFloat`, `math.IsNaN`/`IsInf` | Check ConfigKit lexical syntax first. Use the destination bit width, including `int`/`uint`; parse floats directly at 32/64 bits. Do not route wide integers through floats. Keep boolean/null recognition in simple switches; `strconv.ParseBool` accepts forms outside our policy. |
+| Duration/text scalars (5–6) | `time.ParseDuration`, `Duration.String`, `encoding.TextMarshaler`/`TextUnmarshaler` | Enforce string-only input and paired codecs, canonical projection, detached reconstruction, and safe diagnostics. Date-looking text remains text. |
+| Field binding (4) | `reflect.Type`, `StructField`, `StructTag.Lookup`, ordinary reflection accessors | Build our descriptor graph by inspecting declared fields and explicit inline tags. Do not use `reflect.VisibleFields` as the binding algorithm: Go promotion can hide conflicts and does not express our inline rules. No `unsafe` access to private fields. |
+| Detached copies and comparison (5, 8–9) | `maps.Clone`, `slices.Clone`, `bytes.Clone` where shallow copies suffice | Recursively clone supported configuration with the descriptors; shallow clones do not detach nested objects. Compare canonical trees with a focused changed-path traversal, preserving atomic sequences and custom scalars. Neither `reflect.DeepEqual` alone nor runtime `go-cmp` supplies this audit. |
+| Semantic paths (2) | `strings`, `strconv` for validated index tokens | Implement the small strict JSON Pointer codec and key/index representation. Do not use filesystem normalization or URL escaping/percent decoding. |
+| Ordering and import graphs (2, 10–14) | `slices.Sort`/`SortFunc`, `cmp.Compare`, ordinary maps and slices | Define total ordering keys for observable output. Implement bounded DFS with separate active/visited maps and inclusion routes. Keep per-load state local; no global cache, `sync.Map`, or parallel import scheduler initially. |
+| Safe diagnostics/output (2–3, 6, 15) | `errors.Is`/`As`, `fs.ErrNotExist`/`ErrPermission`, `fmt`, `io.WriteString`, `strconv.Quote` | Classify underlying failures and emit ConfigKit-owned records. Never forward `strconv.NumError`, duration-parser errors, or other value-bearing error text/unwrap chains. Quoting escapes controls but does not redact secrets. Check writer errors and short writes; flush buffered output if used. |
+| Test support (2 onward, 21) | `testing`, `t.TempDir`, native fuzzing, `testing/iotest` | Use standard fragmented/error readers and real filesystem fixtures. Add small custom close-observers or failing writers only where needed. Use test-only `go-cmp` for structured expectations, not a second test framework. |
+| Import-boundary checks (3, 20, 22) | `go/parser`, `go/ast`, `strconv.Unquote` | Inspect Go imports, including test files and platform-specific source, rather than regex-match source text. Permit YAML imports only in the adapter and its tests, and `go-cmp` imports only in test files. |
+
+For confined access, canonicalization before `Root.Open` preserves contained
+absolute symlink targets that `Root.Open` would reject if passed their original
+symlink path. This is an implementation of the feature's stable-input contract,
+not a promise of atomic resolution against hostile concurrent replacement.
+Retain regular-file checks and safe error classification. Do not introduce custom
+syscall-level confinement, path discovery, or a public filesystem-provider API.
+
+### Secret, override, and history implementation bounds
+
+Follow the feature contracts for [secrets](initial.md#secret-references-and-explicit-bootstrap),
+[CLI overrides](initial.md#cli-overrides), and
+[explanation](initial.md#historical-value-explanation). Introduce these public APIs
+only in Stages 16–19. Earlier stages cover ordinary types and file/profile layers;
+they must not expose an unresolved secret as a completed load or silently accept
+an override option. Extend the shared descriptor/projection graph for `Secret`
+in Stage 17, before ordinary struct/text handling; do not add a parallel binder.
+
+- Bootstrap uses `os.LookupEnv`/`os.Setenv`, `os.Open` and bounded `ReadDir`/reads,
+  `filepath` canonicalization, and `strings.TrimSuffix` for exactly one newline.
+  Reuse WeatherReporter's actual code and tests, adapting policy rather than
+  importing its package. Validate names with a small ASCII check or its existing
+  regular expression. Keep a private environment/file-access seam for failure tests;
+  do not expose a general environment provider or public mock filesystem.
+- Bootstrap ceilings: 1,024 entries, 1 MiB per eligible file, 8 MiB total eligible
+  bytes. Bound enumeration before sorting, read at most remaining budget plus one,
+  and reject excess before writes. Skipped existing files consume entry budget,
+  not content budget, and are not opened. These budgets are independent of YAML
+  inputs. Errors use `SecretsError` with `Error()` and `Diagnostics() []Diagnostic`;
+  no raw unwrap chain. `LoadSecretsDir` always returns its detached report alongside
+  an error. Diagnostics include safely quoted canonical file paths in messages; no config source
+  ID is fabricated for bootstrap. Reports expose no credential bytes or hashes.
+- Stage 16's preflight/apply state machine owns report status assignment and
+  `ExistingEmpty`. On preflight failure, no `Loaded` entries are possible. On apply
+  failure, stop with earlier loaded/skipped statuses intact. Never log implicitly.
+- Stage 17 reads each referenced name once per load, including failures. Do not
+  cache across loads, mutate the environment, or retain it in metadata. Application
+  hooks cannot inject a resolved credential through a public constructor. Store
+  names in projected nodes and resolved strings only in private `Secret` state.
+- Override input shares the depth/node ceilings with document input; count its
+  nodes toward the aggregate budget. Bound string bytes to 8 MiB per record and
+  64 MiB combined with source bytes; at most 1,024 records, also counting their
+  synthetic sources toward the source ceiling. Validate and detach before applying;
+  no second parser or `flag` framework. Use `reflect` only to reject unsupported
+  values/detect cycles and standard numeric formatting to create typed scalar nodes.
+- Retain structurally shared immutable document roots and audited domain deltas
+  for history. Cap retained unique history nodes at 4,000,000 per load, including
+  projected baseline and resolution snapshots; fail safely if exceeded. Explain
+  projects only the queried subtree for each relevant operation. Do not eagerly
+  store every rendered path/history combination or expose private roots.
+- Test bootstrap real environment integration serially with `t.Setenv`; use the
+  private seam for unset names, preflight errors, write failures and deterministic
+  races between preflight and apply. Test load lookup using controlled names and
+  restore environment; no tests depend on real credentials. `go-cmp` remains test-only.
 
 ### Metadata and diagnostics layout
 
@@ -92,9 +210,9 @@ Expose detached value records for inspection, with these initial fields:
 | `Diagnostic` | `Code Code`, `Severity Severity`, `Path Path`, `Message string`, `Primary *Location`, `Related []Location`; absent primary means no known source |
 | `Origin` | `Layer Layer`, `Locations []Location`, `Operation ChangeKind`; multiple locations represent a composite origin |
 | `FieldInfo` | `Explicit bool`, `Null bool`, `Origin Origin` |
-| `Change` | `Path Path`, `Origin Origin`, `Previous []Origin`, `Reasons []string`, `Dependencies []Path`; no public historical values |
+| `Change` | `Path Path`, `Origin Origin`, `Previous []Origin`, `Reasons []string`, `Dependencies []Path`; safe values available separately through `Explanation` |
 
-`Layer` is a string enum: `default`, `base`, `profile`, `resolution`. A Go zero
+`Layer` is a string enum: `default`, `base`, `profile`, `cli`, `resolution`. A Go zero
 baseline uses layer `default` without claiming an application deliberately chose
 every zero value. `ChangeKind` has the empty value for input/default origins,
 `derived_default`, and `normalization`. `Severity` is `error` or `warning`.
@@ -103,14 +221,16 @@ Define exported constants with the type prefix (for example `LayerBase`,
 
 Use stable string codes grouped by `options.*`, `source.*`, `yaml.*`, `schema.*`,
 `decode.*`, `compose.*`, `import.*`, `profile.*`, `version.*`, `resolve.*`, and
-`validate.*`. Include distinct missing/permission/read, duplicate, invalid-path,
+`validate.*`, `secret.*`, and `override.*`. Bootstrap uses `secrets.*` codes
+on a separate `SecretsError`. Include distinct missing/permission/read, duplicate, invalid-path,
 limit-exceeded, hook-failed, and unrecorded-change classifications where relevant.
 Application reports use their phase's `error`/`warning` code, not arbitrary code
 registration. Define exact exported code constants with their owning feature;
 private helpers do not need new public codes for every branch.
 
 Sources get deterministic per-load IDs `source-1`, `source-2`, and so on in
-traversal order. The selected profile is last. `NoSource` creates no source ID.
+traversal order. File traversal is followed by the selected profile, then synthetic CLI sources.
+`NoSource` itself creates no source ID; its overrides can create CLI sources.
 Public query slices and nested slices/pointers are detached. Default fmt/error
 rendering must not reveal hidden value histories.
 
@@ -125,7 +245,7 @@ rendering must not reveal hidden value histories.
   diagnostics permit schema-valid missing fields/map entries; malformed paths or
   unknown struct fields produce a hook diagnostic. Version hooks use the untyped
   composed tree and nearest known container because decoding has not run yet.
-- Defaults apply to the surviving composed/profile input; intermediate overridden
+- Defaults apply to the surviving composed/profile/CLI input; intermediate overridden
   nodes do not create reset markers. An effective null suppresses defaults at
   its path. Missing children of an effective mapping can retain defaults.
 - Applying nonempty input to a nil pointer-to-struct creates the object and uses
@@ -172,16 +292,17 @@ Tasks:
 1. Create the module and root package documentation with the decisions above.
    Keep the existing repository license. Add no application command, runtime
    dependencies, framework, or placeholder exported loader.
-2. Reinspect Narratio's referenced revision and current worktree. Create
+2. Reinspect Narratio's and WeatherReporter's referenced revisions and worktrees. Create
    `docs/internal/extraction.md` recording source revisions/symbols and the exact
-   loading/presence/default/diagnostic cases to extract. Include applicable notices
-   with copied material. Copy only minimal sanitized fixtures needed by subsequent
+   parsing/composition/import/profile/inspection/secret and default/presence cases
+   assigned by the extraction map. Record preserved, adapted and application-only
+   behavior, including current tests replacing obsolete baseline assumptions.
+   Copy only minimal sanitized fixtures needed by subsequent
    stages; no audio, transcripts, secrets, or entire application package trees.
 3. Document the planned parser pin here; add its module requirement only when the
    real adapter imports it in Stage 3 so tidy does not leave unused dependencies.
 4. Update the developer guide to runnable Go baseline commands and prerequisites.
-   Correct its stale claim that this plan is deferred. Update documentation-policy
-   navigation to this now-existing plan without changing ownership rules. Record
+   Preserve roadmap/current-contract ownership and navigation. Record
    the implemented root package in the overview and retain pre-feature status.
 
 Acceptance:
@@ -209,11 +330,15 @@ Tasks:
 3. Render control characters in labels/messages/paths as escaped text so source
    labels cannot corrupt terminal output. Never include values or parser errors.
 4. Document the implemented path and diagnostic API next to declarations.
+5. Apply the standard-library path, ordering, and diagnostic mechanisms above.
+   Introduce test-only `go-cmp` when comparing structured records; simple scalar
+   assertions can remain ordinary Go checks.
 
 Acceptance:
 
 - Round-trip escaped slash/tilde/empty/dotted/numeric keys and indexed paths;
-  distinguish a malformed path from a valid missing path.
+  distinguish a malformed path from a valid missing path. Preserve literal `..`
+  and percent-containing keys; reject malformed tilde escapes.
 - `errors.As` finds `LoadError`; mutation of returned nested records cannot alter
   later results. Safe writer failures propagate, including partial output.
 - Reordered independent diagnostic submissions render identically; no blanket
@@ -226,21 +351,35 @@ Status: pending.
 Read: feature [YAML policy](initial.md#yaml-and-scalar-policy),
 [source constructors](initial.md#entry-point-and-reusable-application-definition).
 
+Start with the extraction map's parser and strict-loader cases; the old
+second-document acceptance defect is already fixed in Narratio.
+
 Tasks:
 
 1. Adapt Narratio's file/reader loading helpers into the opaque source constructors
    and bounded private reader. Add `NoSource`, byte snapshotting, source labels,
    regular-file checks, deterministic I/O classification, and reader single-use
    state shared by source copies. File loads reopen each time; readers are not closed.
-2. Add the pinned YAML adapter and ConfigKit-owned document tree. Decode into parser
-   nodes only; translate key/value locations and scalar spelling before discarding
-   parser-specific nodes. Do not use parser typed decode for application structs.
+   Use the bounded-read and source-ownership recipes above, including the extra-byte
+   oversize check and an atomic reader claim shared by source copies.
+2. Add the pinned v4 YAML adapter and ConfigKit-owned document tree. Use
+   `yaml.NewLoader` and load each document into `yaml.Node`; translate key/value
+   locations and scalar spelling before discarding parser-specific nodes. Keep
+   upstream imports and error translation inside the adapter. Do not use parser
+   typed decode for application structs or return upstream types from the adapter.
 3. Enforce exactly one mapping document, duplicate keys, tag/alias/anchor/merge-key
    rejection, and ConfigKit's own scalar grammar. A second successful decode is
    an error, including an empty document; do not preserve Narratio's trailing-doc bug.
-4. Enforce byte ceilings before parsing and tree depth/node ceilings during adapter
-   translation. The parser can reject stricter malformed/nesting inputs first, but
-   none may panic or expose input text. Add private small-limit injection for tests.
+   Leave `WithSingleDocument` disabled: in rc.6 it stops the streaming loader after
+   the first document instead of rejecting trailing documents. Check explicitly
+   for another document. Validate duplicate keys in the ConfigKit tree; rc.6 node
+   loading preserves duplicates despite upstream typed-decoding uniqueness options.
+4. Enforce byte ceilings before parsing. Configure the v4 limit plugin privately
+   to bound parser nesting, and enforce ConfigKit tree depth/node ceilings during
+   adapter translation. Calibrate parser-depth counting against ConfigKit's root-zero
+   limit with block/flow boundary cases rather than assume identical counters.
+   Parser failures must not panic or expose input text. Add private small-limit
+   injection for tests. Alias limits do not replace ConfigKit's alias prohibition.
 5. Create `docs/config.md` for implemented syntax/input behavior and link from
    package documentation. Move the implemented definitions out of the feature
    roadmap, preserving navigation for this plan.
@@ -249,11 +388,18 @@ Acceptance:
 
 - Real file and reader tests cover missing/permission/read failures, non-regular
   targets, byte snapshots, second reader use, and no caller-reader close.
+  Cover exact-limit and one-byte-over input, fragmented reads, data returned with
+  an error, and competing loads of copies of the same reader source.
 - Policy tables cover duplicate keys, all forbidden YAML constructs, null spellings,
   quoted literals, JSON numeric syntax, legacy booleans/numbers, timestamp-like
   strings, multiple/empty documents, and root type. Numeric nodes never round via float.
 - Small-limit boundary tests and malformed input tests return safe diagnostics with
   available coordinates. This stage does not promise a typed public `Load` yet.
+- Retain regressions for duplicate node keys, trailing/empty second documents,
+  lexical numbers, explicit-tag visibility, and parser-depth enforcement. Add a
+  focused import-boundary check that rejects direct upstream YAML imports outside
+  the adapter and its tests and production `go-cmp` imports, using the AST recipe
+  above; extend it as packages/examples appear.
 
 ## Stage 4 — Build the reusable schema binding graph
 
@@ -266,6 +412,8 @@ Tasks:
 
 1. Implement reflection descriptors for supported root structs/maps, scalar kinds,
    pointers, nested structs, string-keyed maps, slices, fixed arrays, and `any`.
+   Use declared-field reflection and tag lookup as specified above, not Go field
+   promotion as a substitute for ConfigKit binding.
 2. Bind exported field names, tags, inline structs/pointers and the optional inline
    map. Detect ambiguous names, multiple inline maps, unsupported tag options,
    conflicting defaults-map names (checked against runtime defaults in Stage 6),
@@ -296,6 +444,7 @@ Tasks:
 2. Clone ordinary configuration into independently owned values. Reconstruct paired
    text scalars through canonical text into fresh instances; preserve type identity
    and supported dynamic concrete values without retaining mutable input aliases.
+   Apply the shallow-versus-recursive copy distinction above and reuse descriptors.
 3. Detect cyclic values, non-finite floats, invalid dynamic types, projection limit
    violations, and custom codec errors safely. Do not reveal the codec error text.
 4. Add structural comparison used later by resolution audit and validator-mutation
@@ -324,6 +473,8 @@ Tasks:
 2. Enforce numeric syntax/range and nullability. Support dynamic `int64`/`float64`,
    finite floats, exact-duration string parsing, named scalar types, paired text
    codecs, arrays with exact length, and strict nested map-of-struct decoding.
+   Use the standard-library conversion recipes above after lexical validation;
+   translate failures without retaining value-bearing conversion errors publicly.
 3. Reject unknown fields, tag binding conflicts, and invalid defaults consistently.
    Collect independent decode errors while leaving the failed working object private.
 4. Return a private decoded configuration plus its surviving input presence index,
@@ -390,14 +541,15 @@ Tasks:
    derived-default intent using the clarification rules above.
 3. Verify final helper values against declared paths. Reject invalid paths, invalid
    kinds, invalid projected values, missing records, or use after context closure.
-   Store reasons/dependencies without raw public history values.
+   Store reasons/dependencies and private before/after snapshots for later safe explanation.
 4. Attach phase diagnostics and stop on reported errors even with nil return. A
    non-nil return adds generic safe failure only if no error was already reported.
    Leave callback panics as programming failures. Remove the resolver options gate.
 
 Acceptance:
 
-- Execute the feature's timeout and dynamic artifact examples against real code.
+- Execute the feature's timeout and dynamic artifact examples against real code,
+  using their non-secret fields here; add the typed credential field in Stage 17.
 - Test explicit-null/zero preservation, baseline replacement for absent input,
   normalization of explicit input, map writeback, unrelated pointer/path mismatch,
   unrecorded mutation, equal-value intent, repeated helper writes, and subtree removal.
@@ -438,6 +590,9 @@ Status: pending.
 
 Read: feature [composition table](initial.md#composition-and-overlay).
 
+Extract Narratio's additive composer and conflict tests before writing equivalent
+code. Change empty-map expectations to the feature table and retain exact locations.
+
 Tasks:
 
 1. Implement a private strict composer over ConfigKit document trees. Recurse only
@@ -463,6 +618,9 @@ Status: pending.
 
 Read: feature [import graph](initial.md#explicit-imports-and-source-identity).
 
+Extract the flat import loader and tests, then generalize its graph, identity and
+containment rules. Do not treat its root-only declarations as ConfigKit's contract.
+
 Tasks:
 
 1. Parse the reserved root directive and `path` declarations strictly; preserve
@@ -471,6 +629,8 @@ Tasks:
 2. Resolve paths from their declaring source, canonicalize symlinks, enforce root
    containment, and perform deterministic depth-first traversal. Keep active-stack
    and visited indexes distinct. Track both routes for cycles and repeated imports.
+   Use the canonicalization/`os.Root` recipe above, including ConfigKit-owned root
+   handle cleanup; preserve contained relative and absolute symlink targets.
 3. Read accepted sources once; enforce count/depth/aggregate budgets. Optional paths
    suppress only target nonexistence. Invalid directives, directories, unreadable
    files, canonicalization failures, and out-of-root targets remain errors.
@@ -494,11 +654,15 @@ Status: pending.
 
 Read: feature import syntax and source-context API.
 
+Reuse import failure fixtures where applicable; Narratio's flat loader does not
+supply generic glob traversal or its bounded filesystem behavior.
+
 Tasks:
 
-1. Implement segment glob matching with sorted slash-relative results and no `**`.
-   Use an error-reporting directory walker: do not use a convenience glob API that
-   silently swallows permission/I/O errors. Reuse Stage 11 identity and traversal.
+1. Use `path.Match` with the constrained grammar and bounded `File.ReadDir(n)`
+   expansion recipe above. Return sorted slash-relative matches; no recursive `**`.
+   Keep directory expansion pattern-directed and error-reporting. Reuse Stage 11
+   identity and traversal; do not add another glob engine or use `filepath.Glob`.
 2. Reject invalid patterns before walking; canonicalize and contain visited
    directories/targets, including symlinked directories. Optional zero matches
    must not conceal a walk or permission failure. Bound accepted candidates before
@@ -515,15 +679,20 @@ Acceptance:
 - Fixtures cover lexical order, multiple declarations, overlapping matches, matches
   that are directories, hard links as separate paths, symlink escapes, unmatched
   optional/required patterns, invalid patterns, and visible permission failures.
+  Cover escaped metacharacters, character classes, empty-directory pattern errors,
+  contained absolute symlinks, and budget exhaustion from nonmatching entries.
 - Repeated runs produce identical source IDs, composition, and diagnostic ordering.
   Full import contract and baseline pass.
 
-## Stage 13 — Implement profile overlays and wire final precedence
+## Stage 13 — Implement profile overlays
 
 Status: pending.
 
 Read: feature [profiles](initial.md#profiles), composition/overlay table.
 
+Extract Narratio's overlay mechanics and profile tests; preserve the intentional
+null/kind and profile-selection differences listed in the feature roadmap.
+
 Tasks:
 
 1. Implement private recursive overlay with atomic scalar/sequence/null and
@@ -554,6 +723,9 @@ Status: pending.
 Read: feature [presence](initial.md#paths-and-presence),
 [history](initial.md#provenance-diagnostics-and-redaction), metadata API.
 
+Use Narratio's effective ownership and inspection tests as extraction evidence.
+Its source-only records need locations and complete historical operations.
+
 Tasks:
 
 1. Integrate default, base, profile, and resolution records into a coherent query
@@ -581,6 +753,9 @@ Status: pending.
 Read: feature [redaction](initial.md#provenance-diagnostics-and-redaction),
 [secret predicate](initial.md#schema-checking-and-secret-classification), result API.
 
+Reuse Narratio's deterministic effective-output and non-disclosure cases. Render
+from the owned typed projection, without its marshal/parse reconstruction.
+
 Tasks:
 
 1. Combine descriptor secret tags with the application path predicate over current
@@ -594,9 +769,13 @@ Tasks:
    by decoded string, preserve sequence order, use two-space indentation and a final
    newline, and substitute the quoted string `[REDACTED]` at secret roots. Preserve
    supported nil/empty distinctions. No unredacted option or implicit logging.
-4. Marshal only an adapter tree made from the safe projection; never call application
-   YAML marshal hooks. Finish all text-codec projection work before returning a
-   successful load. Writer errors may leave partial redacted output.
+4. Emit only an adapter tree made from the safe projection through the v4 dumper;
+   never call application YAML marshal hooks. Set indentation, sequence indentation,
+   line breaks, and document-marker behavior explicitly inside the adapter rather
+   than inherit a version preset. Use two-space indentation, noncompact sequences,
+   LF line breaks, and no document start/end markers for the single output document.
+   Finish all text-codec projection work before returning a successful load. Writer
+   errors may leave partial redacted output; translate upstream dump errors safely.
 
 Acceptance:
 
@@ -610,12 +789,157 @@ Acceptance:
   responsibility and that redacted output is not promised reloadable.
 - Render actual resolved values including defaults/normalization; baseline passes.
 
-## Stage 16 — Prove consumer ergonomics and migration contracts
+## Stage 16 — Extract explicit secrets-directory bootstrap
+
+Status: pending.
+
+Read: [secret bootstrap contract](initial.md#secret-references-and-explicit-bootstrap),
+architecture responsibility boundary, testing policy, and WeatherReporter's
+`internal/config/secrets.go` and corresponding `config_test.go` cases; also
+Narratio's `internal/app/secrets_env.go`, its tests, and bounded-read helper.
+
+Tasks:
+
+1. Record both applications' source revisions and reused symbols/tests in
+   the extraction inventory. Reuse WeatherReporter's `loadSecrets`/`applySecrets`
+   separation and Narratio's bounded reads/preserve-existing behavior. Replace
+   overwrite/rollback, immediate writes, aggregate logging, and incompatible
+   name/newline/permission/directory policies with ConfigKit's contract. Keep this
+   operation separate from `Load` and do not add a directory loading option to YAML.
+2. Implement bounded, deterministic enumeration and metadata validation; reject
+   symlinks/nonregular entries, validate names, and preserve existing variables
+   including empty ones without reading their contents.
+3. Implement eligible-file preflight, newline/NUL checks and the apply state
+   machine, including the presence recheck and partial-failure report. Add the
+   public report, status enum, safe typed error and report renderer as specified.
+4. Document startup-only process mutation, non-atomicity, and how applications
+   report blocked files and retain the report alongside configuration results.
+
+Acceptance:
+
+- Adapt WeatherReporter's preflight-failure tests and Narratio's existing-empty
+  environment cases. Replace overwrite/rollback expectations with skip/partial-report
+  checks; do not drop the underlying failure coverage. Keep bootstrap explicit,
+  independent of subsequent configuration validation.
+- Real temporary-directory tests cover newline preservation, invalid names/types,
+  bounded reads/enumeration, empty files, and canonical report paths/order.
+- Existing nonempty and empty variables win and are distinguishable in reports;
+  skipped contents are not read. Preflight failures perform no writes. Controlled
+  apply failure and newly set variable tests prove accurate partial statuses.
+- Canary contents are absent from errors, fmt/JSON reports and rendered output.
+  Writer failures propagate. Run the stage baseline; no actual credentials are used.
+
+## Stage 17 — Add environment-backed typed secret references
+
+Status: pending.
+
+Read: [secret reference contract](initial.md#secret-references-and-explicit-bootstrap),
+shared descriptor/projection rules and lifecycle, Stage 16 report boundary.
+
+Tasks:
+
+1. Implement opaque `Secret`, `SecretEnv`, name/value/resolution accessors, safe
+   fmt behavior and name-only JSON output. Extend existing binding, cloning,
+   decoding, projection, comparison and historical sensitivity machinery.
+2. Resolve effective names after the resolver and before validation. Cache lookup
+   results only for this load. Handle required versus optional references, invalid
+   names, absent/null/empty references, unset and empty variables with safe errors.
+3. Keep credentials out of every document/history/metadata representation. Rebuild
+   references from names when cloning defaults, even if they came from a prior
+   resolved result. Add effective reference-name inspection without claiming a
+   bootstrap file populated the current environment.
+4. Update current contracts and the pipeline example to use typed references;
+   retain tag/predicate redaction for sensitive derived data. Earlier lifecycle
+   examples using `Secret` become runnable only at this stage.
+
+Acceptance:
+
+- Test required and optional references in structs, maps, pointers and sequences;
+  only surviving names are read, once per unique name. Resolution can change a
+  name and validation sees its resolved value. Reject validator replacement with
+  a same-name unresolved reference, using a private-state audit against the lookup
+  cache in addition to projection. Later loads observe changed env.
+- Bootstrap plus load tests show a blocked empty variable in the report and an
+  empty-reference failure at its config path; no fallback to directory contents.
+- Secret values appear only through `Value()`, never safe fmt/JSON, diagnostics,
+  projected trees or history; previously resolved defaults cannot retain stale
+  values. `Load` never mutates environment. Run the baseline.
+
+## Stage 18 — Apply explicit CLI overrides above profiles
+
+Status: pending.
+
+Read: [override contract](initial.md#cli-overrides), overlay/presence/version rules,
+source ordering, and the secret-reference boundary.
+
+Tasks:
+
+1. Add `Override` and `Options.Overrides`. Validate paths, labels, allowed value
+   types, overlap/version/directive restrictions and budgets; detach input into
+   private nodes without YAML reparsing or scalar inference.
+2. Apply records after profiles and before the existing final typed decode using
+   the shared overlay primitive. Create synthetic CLI sources and record explicit
+   presence and before/after operations, including equal-value replacements.
+3. Support defaults-only loads with overrides and add a parser-neutral executable
+   example translating only explicitly set flags. Document general environment
+   overlays as an explicit non-goal, not a future option.
+
+Acceptance:
+
+- A lifecycle test proves defaults < base/imports < profile < CLI, followed by
+  recorded normalization and normal secret lookup/validation. Zero, false, empty
+  string/list and null remain intentional values.
+- Test map overlay and whole-list replacement, invalid ancestors/index paths,
+  duplicate/overlapping paths, type/cycle/limit errors, unknown fields and version
+  protection. Input mutation after load cannot change result or history.
+- CLI diagnostics and provenance identify flag names without raw arguments;
+  CLI secret input is an environment name. Run the baseline.
+
+## Stage 19 — Expose safe historical-value explanation
+
+Status: pending.
+
+Read: [explanation contract](initial.md#historical-value-explanation), redaction
+rules, and history bounds above.
+
+Generalize Narratio's semantic-record and inspection tests. They cover effective
+values and source ownership, not the full chronological history required here.
+
+Tasks:
+
+1. Extend retained operation snapshots where necessary to reconstruct known final
+   and removed paths through defaults, base contributions, profile, CLI, and net
+   domain changes. Use structural sharing and the history budget, not full copies
+   per field. Preserve original policy-validated overridden nodes.
+2. Implement detached `ValueSnapshot`, `ValueChange`, `Explanation`, and reference
+   metadata; apply historical secret/dependency closure before any public value
+   projection. Add `Result.Explain` with absent/null/deleted distinctions.
+3. Implement `WriteExplanation` over the safe result, with canonical pointer paths,
+   composite origins, ordered before/after history, reasons/dependencies and final
+   source. Keep bootstrap reports separate and show how commands present both.
+
+Acceptance:
+
+- Show the requested concurrency history 4 → 8 → 1 and effective profile file/line;
+  extend one test with CLI and normalization. Test equal writes, removed paths,
+  parent replacement, composite maps, atomic sequences and unknown paths.
+- Historical input with an overridden invalid destination type stays explainable;
+  output never claims it was successfully typed. Default null/empty rules agree
+  with final decoding and metadata. Retained history obeys bounded allocation.
+- Canary secrets remain absent even after replacement/deletion/dependency cycles;
+  returned snapshots cannot mutate internal history. Test safe fmt/JSON of public
+  records, escaped labels, deterministic output and failing/short writers.
+- Run baseline and a representative end-to-end bootstrap/load/explain test.
+
+## Stage 20 — Prove consumer ergonomics and migration contracts
 
 Status: pending.
 
 Read: feature consumer-fit table, module options, external values, and both criteria.
 
+Include the updated consumer tests listed in the extraction map. Preserve domain
+rejection of explicit empty lists without making empty lists a library schema error.
+
 Tasks:
 
 1. Create maintained executable examples and minimal synthetic fixtures for four
@@ -623,12 +947,17 @@ Tasks:
    object bindings/string versions/dynamic options; WeatherReporter-like defaults-only
    and report/module options; Seriatim-like ordered speaker/autocorrect rule files.
 2. Generalize the relevant copied Narratio tests and helpers, recording source
-   revision and notices. Use actual consumer field patterns but no private data or
+   revision and adaptations. Use actual consumer field patterns but no private data or
    dependencies on sibling application modules. Keep runtime algorithms in example
    application functions rather than adding library APIs for them.
-3. Compare preserved outcomes against consumer evidence and list intentional changes:
+3. Add a WeatherReporter-shaped startup example that renders bootstrap outcomes,
+   loads typed references, and explains configuration with explicitly set CLI flags.
+   Demonstrate both a blocked empty variable and successful lookup without real
+   credentials or implicit process mutation in `Load`.
+   Compare preserved outcomes against consumer evidence and list intentional changes:
    explicit empty/null behavior, canonical objects, versions, integer representation,
-   strict YAML, and retired environment/CLI configuration overrides. Verify imports
+   strict YAML, retired general environment overrides, explicit CLI records, and
+   secret-reference migration. Verify imports
    and profiles do not absorb application LLM/pipeline-profile semantics.
 4. Convert every feature-roadmap API example into a real compiled/run example or a
    focused behavioral test; document the happy path in README and `docs/consumers/`.
@@ -643,10 +972,14 @@ Acceptance:
   with stable classifications and useful paths, not copied raw parser text.
 - Simple consumers need no imports/profiles and no custom parser hooks. Complex
   examples preserve domain meaning under the same conventions.
-- No external configuration reads, environment mutation, LLM calls, remote services,
+- No uncontrolled external configuration reads, LLM calls, remote services,
   or sibling checkout requirements occur in default tests. Baseline passes.
+- Compile consumer examples using only ConfigKit and standard-library imports.
+  Review exported signatures, reachable record fields, aliases, generic constraints,
+  callbacks, and error chains against the upstream isolation contract. Consumers
+  must not name or configure the YAML dependency to exercise any supported API.
 
-## Stage 17 — Harden boundaries and verify independent loads
+## Stage 21 — Harden boundaries and verify independent loads
 
 Status: pending.
 
@@ -677,7 +1010,7 @@ Acceptance:
 - A fresh offline baseline after dependency acquisition passes without reading any
   developer-specific configuration path.
 
-## Stage 18 — Prepare release validation and reconcile documentation
+## Stage 22 — Prepare release validation and reconcile documentation
 
 Status: pending.
 
@@ -710,6 +1043,9 @@ Tasks:
    Replace implemented feature-roadmap definitions with current contract links;
    repair ADR navigation without changing accepted rationale. Keep this execution
    plan until release preparation retires completed roadmap material.
+   Reassess the v4 pin under the dependency-update rule above. Run the import-boundary
+   check and review the public API for upstream leakage; preserve ConfigKit's
+   behavioral expectations when adapting to any upstream changes.
 6. Update release-readiness evidence for the actual runbook/checker. Keep any
    unvalidated release-candidate/publication gap explicit and retain its bootstrap
    guard until the maintainer closes it. Do not fabricate release notes for an
@@ -735,20 +1071,24 @@ working loader. The linked feature sections own semantics; stages own work.
 
 | Feature area | Owning stages |
 | --- | --- |
-| Extraction, ordinary consumer types, package boundaries | 1, 4–6, 16 |
+| Extraction, ordinary consumer types, package boundaries | 1, 4–6, 20 |
 | YAML subset, precise numeric syntax, safe source failures | 2–3, 6 |
 | Paths, presence, origins, definition history | 2, 6–8, 10, 13–14 |
-| Static defaults, null, container rules, snapshot ownership | 5–7, 14, 17 |
+| Static defaults, null, container rules, snapshot ownership | 5–7, 14, 21 |
 | File/bytes/reader/defaults-only inputs and filesystem context | 3, 7, 11–12 |
 | Version hook and graph-placement restrictions | 7, 11, 13 |
-| Strict imports, glob ordering, identity, limits, collisions | 10–12, 17 |
+| Strict imports, glob ordering, identity, limits, collisions | 10–12, 21 |
 | Named profiles, precedence, atomic replacement | 13–14 |
 | Recorded normalization/derived defaults and hook errors | 8–9, 14 |
 | Semantic validation, related diagnostics, safe messages | 2, 7–9 |
-| Secret tags/predicate/dependencies and effective inspection | 5, 15, 17 |
-| Module-specific options, consistency and intentional migrations | 16 |
-| Runnable docs, examples, source-release checks and runbook | 1, 16, 18 |
-| Deferred-feature boundaries | Every stage; final audit in 18 |
+| Secret tags/predicate/dependencies and effective inspection | 5, 15, 21 |
+| Module-specific options, consistency and intentional migrations | 20 |
+| Runnable docs, examples, source-release checks and runbook | 1, 20, 22 |
+| Secret-directory bootstrap and blocked-file provenance | 16–17 |
+| Environment-backed secret references and safe lookup | 17, 19–21 |
+| Explicit CLI overrides and complete precedence | 18–21 |
+| Safe historical values and explanation renderer | 14–15, 19–21 |
+| Deferred-feature boundaries | Every stage; final audit in 22 |
 
 At final handoff, report completed stages, exact checks, current documentation
 owners, source-extraction evidence, and remaining release/consumer migration work.
@@ -767,7 +1107,7 @@ are not authorization to silently weaken acceptance:
 
 1. **Is a native Linux runner available for the declared support matrix?**
    Recommended: execute filesystem/race tests on native Linux and macOS before
-   release; cross-builds alone cannot verify symlink/permission behavior. Stage 18
+   release; cross-builds alone cannot verify symlink/permission behavior. Stage 22
    can prepare the checker on the available host while keeping missing native
    evidence as an explicit release gate. Alternative: formally narrow the initial
    supported platforms to those with execution evidence, with a documented scope

docs/roadmap/initial.md

@@ -77,11 +77,12 @@ execution steps belong in the [implementation plan](implementation.md).
 
 | ConfigKit owns | Applications own |
 | --- | --- |
-| Local file and labeled bytes/reader inputs | Main-file search paths, environment conventions, and CLI wiring |
-| YAML policy, explicit import traversal, composition, and profiles | Configuration types and field meanings |
+| Local file and labeled bytes/reader inputs | Main-file search paths, bootstrap timing, and CLI parsing/wiring |
+| YAML policy, explicit import traversal, composition, profiles, and explicit CLI overrides | Configuration types and field meanings |
 | Presence, source locations, resolution records, and safe diagnostics | Static defaults, derived defaults, and normalization rules |
+| Secret-reference resolution and explicit directory bootstrap | Credential use and process startup coordination |
 | Strict typed decoding and lifecycle coordination | Semantic validation and schema-version policy |
-| Source reporting and redacted effective output | Remote acquisition, migrations, runtime initialization, and resource checks |
+| Source reporting, safe history/explanation, and redacted effective output | Remote acquisition, migrations, runtime initialization, and resource checks |
 
 Applications may fetch remote data and pass labeled bytes to ConfigKit. This
 provides useful provenance without making ConfigKit responsible for network
@@ -99,7 +100,7 @@ contracts govern the result; reuse does not require preserving incidental
 application structure or incompatible behavior.
 
 The references below were inspected on 2026-09-06 at Narratio commit
-`a7ec1955874fda61f3e3f9e3cc33acd345530952`. Paths refer to the sibling checkout
+`b68435128be1e24b94fc7ec259d6e9093669fd99`. Paths refer to the sibling checkout
 `../narratio` relative to the ConfigKit repository root; the links require that
 checkout layout. They are extraction references, not a runtime or build dependency.
 Recheck them against the source revision actually used for extraction.
@@ -107,12 +108,22 @@ Recheck them against the source revision actually used for extraction.
 | Foundation | Source references | Intended treatment |
 | --- | --- | --- |
 | File and labeled reader/bytes loading; strict field checking | [load.go](../../../narratio/internal/config/load.go): `decodeStrictYAML`, `decodeStrictYAMLFromReader`, `LoadSessionBytesWithOptions` | Extract reusable input handling and generalize decoding behind ConfigKit's private parser boundary |
+| Node parsing, strict composition, overlays, and canonical rendering | [composition.go](../../../narratio/internal/config/composition.go): `parseCompositionDocument`, `buildCompositionNode`, `mergeAdditiveCompositions`, `mergeOverlayComposition`, `canonicalYAML` | Extract algorithms and tests; replace upstream node kinds, path spelling, scalar semantics, and incompatible merge rules |
+| Explicit imports and profile selection | [pipeline_composition.go](../../../narratio/internal/config/pipeline_composition.go): `loadPipelineCompositionSources`, `loadPipelineImports`, `loadPipelineProfileOverlays`, `selectPipelineProfile` | Reuse traversal/selection mechanics with ConfigKit's recursive graph, containment, directive syntax, and fixed selected-profile convention |
+| Effective value/source inspection | [effective_pipeline.go](../../../narratio/internal/config/effective_pipeline.go): `EffectivePipelineValues`, `EffectivePipelineSources`, `MarshalEffectivePipeline`; [config_commands.go](../../../narratio/internal/app/config_commands.go) | Reuse deterministic projection, ownership, and side-effect-free inspection cases; retain application commands and generated-domain attribution in Narratio |
 | Explicit field presence | [normalize.go](../../../narratio/internal/config/normalize.go): `NormalizeConfig.UnmarshalYAML` | Generalize the requirement into library metadata; replace the application-specific parser hook |
 | Default application | [load.go](../../../narratio/internal/config/load.go): `applyPipelineDefaults` and related helpers; [defaults.go](../../../narratio/internal/config/defaults.go) | Reuse applicable mechanics and behavioral cases while keeping actual default values and domain rules application-owned |
 | Resolution and source attribution | [load.go](../../../narratio/internal/config/load.go): `Resolve`, `mergeCampaignSession`, `selectStableInput`; [config.go](../../../narratio/internal/config/config.go): `ResolvedInputFile`, `SessionSource` | Adapt source tracking into generic provenance; retain campaign/session fallback and domain types in Narratio |
 | Application-supplied remote bytes | [config_loader.go](../../../narratio/internal/app/config_loader.go) | Use as evidence for labeled input support; keep fetching, storage clients, and remote metadata acquisition outside ConfigKit |
 | Existing behavioral coverage | [normalize_test.go](../../../narratio/internal/config/normalize_test.go), [load_validate_test.go](../../../narratio/internal/config/load_validate_test.go), [campaign_config_test.go](../../../narratio/internal/config/campaign_config_test.go) | Extract relevant cases and minimal fixtures; preserve domain-specific assertions in consumer tests |
 
+Additional extraction coverage includes [composition tests](../../../narratio/internal/config/composition_test.go),
+[import tests](../../../narratio/internal/config/pipeline_composition_test.go),
+[profile tests](../../../narratio/internal/config/pipeline_profiles_test.go),
+[strict-loader tests](../../../narratio/internal/config/loader_strict_test.go), and
+[inspection tests](../../../narratio/internal/app/config_commands_test.go).
+These are reusable behavior evidence, not an instruction to preserve every assertion.
+
 ### Compatibility and adaptation
 
 Extraction must distinguish preserved behavior, intentionally changed behavior,
@@ -126,13 +137,30 @@ Known adaptation points include:
 - Some Narratio defaulting helpers use an empty string to trigger a default.
   Adapt these to ConfigKit's [presence and defaults rules](#static-defaults-and-null)
   rather than losing explicitly configured empty values.
-- `decodeStrictYAMLFromReader` currently accepts a successfully decoded second
-  document. ConfigKit must enforce its [single-document policy](#yaml-and-scalar-policy).
+- `decodeStrictYAMLFromReader` now rejects trailing documents. Preserve and
+  generalize its strict-loader regressions under ConfigKit's single-document policy.
 - The custom normalization decoder and pointer-based presence conventions need
   to give way to generic metadata while preserving genuine domain nullability.
-- Existing source labels and campaign/session attribution provide useful
-  foundations but do not supply full per-path locations, overlay history, or
-  recorded post-decode changes.
+- Existing composition nodes and effective-field ownership are substantial
+  foundations, but do not retain the full line/column and historical-value records
+  ConfigKit requires. Keep application-generated family/campaign attribution as
+  domain resolution with recorded dependencies.
+- Narratio rejects overlapping empty mappings, null overlays, and cross-kind
+  overlays. ConfigKit's operation table deliberately permits those cases. Adapt
+  the implementation and expectations together.
+- Narratio declares import paths and profiles in a root `composition` envelope,
+  supports a declared default profile, and loads every declared overlay. ConfigKit
+  uses `_configkit.imports`, recursive imports/globs, and only the selected fixed
+  profile path; `default` means no overlay. Do not preserve an alternate convention.
+- Narratio uses `os.SameFile` to reject filesystem aliases. ConfigKit uses canonical
+  path identity, treats hard links as distinct paths, and enforces root containment.
+- Narratio's owned tree still embeds upstream YAML kinds/tags and dotted/indexed
+  paths. Its final binding and effective inspection reserialize/reparse YAML.
+  Reuse algorithms without importing those dependencies into ConfigKit's document
+  model or replacing direct descriptor-driven decoding and projection.
+- Narratio's configuration diffs and digests are useful consumer evidence, but
+  remain outside ConfigKit's initial scope. Their implementation does not supply
+  full historical-value explanation or authorize a new public API.
 
 Each intentional behavior change needs focused tests and an explicit migration
 account for the affected consumer. Do not copy an old expectation merely because
@@ -143,17 +171,20 @@ public API remains practical for single-document applications.
 
 ### Capabilities requiring broader implementation
 
-The inspected loader coordinates specific pipeline, campaign, and session
-sources and application fallback rules. It is not the generic import graph and
-strict recursive composer described by this target. Extract applicable pieces,
-but expect new or substantially revised code for generic composition, profile
-overlays, schema-aware defaults/decoding, comprehensive provenance, recorded
-resolution, and safe structured diagnostics and rendering.
+The inspected Narratio loader supplies explicit flat imports, strict composition,
+profile overlays, source ownership, and effective inspection in addition to its
+pipeline/campaign/session domain coordination. Prefer extracting those foundations
+rather than rebuilding equivalent mechanisms.
 
-Automatic `conf.d` discovery remains deferred. The existing application source
-relationships do not authorize implicit discovery or define ConfigKit's import
-syntax. The [implementation plan](implementation.md) owns concrete extraction
-tasks and their ordering.
+ConfigKit still requires new or substantially revised work for recursive import
+graphs and globs, bounded canonical containment, its conservative YAML policy,
+schema-aware defaults/direct decoding, comprehensive location and value history,
+recorded domain resolution, generic secret classification, and safe structured
+reporting. Existing provenance is a starting point, not full explainability.
+
+Automatic `conf.d` discovery remains deferred. Existing application relationships
+and declared profile paths do not redefine ConfigKit's import/profile conventions.
+The [implementation plan](implementation.md) owns extraction tasks and ordering.
 
 ## Consumer fit and intentional migrations
 
@@ -166,9 +197,16 @@ on 2026-09-06 and must be rechecked at migration time.
 | --- | --- | --- |
 | Narratio: [configuration loader](../../../narratio/internal/config/load.go) | Shared loading, presence, defaults coordination, and source attribution | Replace ad hoc presence/empty-value defaults; retain campaign/session domain resolution |
 | Notarius: [file model](../../../notarius/internal/core/config/file_config.go), [environment handling](../../../notarius/internal/core/config/env.go) | Strict nested models, named pipelines, presence-aware values, and diagnostics | Use canonical object bindings/references and string schema versions; adapt dynamic integer consumers to `int64`; migrate configuration-field environment overrides to files/profiles; keep catalog resolution and runtime invocation choices application-owned |
-| WeatherReporter: [loader](../../../weatherreporter/internal/config/load.go), [module options](../../../weatherreporter/internal/config/reports.go) | Defaults-only loading, durations, report maps, list presence, and typed module options | Replace string shorthand with object entries and private presence flags with metadata; migrate configuration-field CLI overrides to files/profiles; keep secrets-directory loading and process-environment mutation outside ConfigKit |
+| WeatherReporter: [loader](../../../weatherreporter/internal/config/load.go), [module options](../../../weatherreporter/internal/config/reports.go) | Defaults-only loading, durations, report maps, list presence, and typed module options | Replace string shorthand with object entries and private presence flags with metadata; translate explicitly set configuration flags into ConfigKit overrides; reuse secrets-directory loading through explicit ConfigKit bootstrap |
 | Seriatim: [configuration surfaces](../../../seriatim/docs/config.md), [speaker rules](../../../seriatim/internal/speaker/map.go), [autocorrect rules](../../../seriatim/internal/autocorrect/autocorrect.go) | Simple typed loading of ordered YAML rule files, normalization, and cross-rule diagnostics | Adopt strict YAML rejection deliberately; retain matching/replacement algorithms, resource checks, and CLI/environment runtime settings; no central application YAML file is required |
 
+WeatherReporter's updated [configuration tests](../../../weatherreporter/internal/config/config_test.go)
+and [external constructed-config tests](../../../weatherreporter/internal/config/overrides_external_test.go)
+cover explicit empty lists, consistent loaded/constructed behavior, and normalization
+without mutating source options. Preserve these domain outcomes through ConfigKit's
+presence, detached ownership, and application validation. ConfigKit permits an empty
+list structurally; a particular consumer may reject it semantically.
+
 The initial model standardizes syntax and resolution while preserving application
 field meanings. Module shorthand and integer schema versions are syntax migration
 issues; module-specific option schemas are genuine domain variation addressed
@@ -177,24 +215,184 @@ not substitutes for application-owned LLM or pipeline profile concepts.
 
 ## External values and invocation state
 
-Environment and CLI configuration-field overlays remain outside the initial
-release. Consumers adopting the complete ConfigKit lifecycle must move such
-settings into base files or supported profile overlays, or defer migration of
-that configuration surface. Do not silently discard supported consumer flags;
-consumer migration must document their replacement or retirement.
+Configuration-field CLI flags use the explicit [override API](#cli-overrides).
+Applications own parsing flags and selecting sources/profiles. Separate invocation
+state, such as an input artifact or execution lane, remains application-owned.
+General environment-variable configuration overlays are an explicit non-goal,
+not deferred work. Environment access is reserved for named secret references.
 
-Application CLI arguments may select a configuration source/profile or describe
-separate invocation state, such as an input artifact, output destination, or
-selected execution lane. Environment variables may supply runtime credentials
-outside the configuration snapshot. Such state remains application-owned and
-must not be reported as part of ConfigKit's effective configuration.
+Do not inject hidden overrides through captured environment/CLI values in default
+factories, resolver hooks, or validators. Domain resolution follows the declared
+input layers. Mutating `Result.Config` cannot update a validated snapshot.
 
-Do not inject configuration overrides through captured environment/CLI values in
-default factories, resolver hooks, or validators. Resolution is for domain
-derivation and normalization, not an undocumented precedence layer. Mutating
-`Result.Config` after loading cannot create a newly validated ConfigKit snapshot
-or update its provenance. A future override mechanism would need an explicit
-precedence, presence, validation, and origin contract before adoption.
+## Secret references and explicit bootstrap
+
+Secret-bearing configuration fields use ConfigKit's opaque `Secret` type. Their
+YAML value is an environment-variable name, for example `api_key: OPENROUTER_API_KEY`.
+Names are public configuration; resolved credentials are not configuration-tree
+values. Literal-secret input, interpolation, fallback literal values, dotenv
+parsing, and implicit directory discovery are unsupported. Names must match
+`[A-Za-z_][A-Za-z0-9_]*`. A string that happens to be a valid name is always a
+reference; ConfigKit cannot detect credentials mistakenly written in other fields.
+
+```go
+func SecretEnv(name string) Secret
+func (s Secret) EnvName() string
+func (s Secret) Value() string
+func (s Secret) IsResolved() bool
+```
+
+`SecretEnv` creates an unresolved reference; invalid names fail the containing
+load. The zero value is an absent reference, never a literal secret. A non-pointer
+`Secret` is required: absent, null, empty/invalid name, unset environment variable,
+or an empty environment value fails with a safe `secret.*` diagnostic. Use
+`*Secret` for optional credentials: nil is allowed, but every non-nil reference
+must resolve to a nonempty value. Applications validate conditional requirements.
+Only typed fields/maps/slices establish secret-reference semantics; dynamic `any`
+strings are ordinary values and must migrate to typed secret slots when needed.
+
+Recognize `Secret` before ordinary struct/text binding. Projection, comparison,
+cloning for defaults, and resolution auditing operate on the name only. Resolve
+the surviving references after recorded application resolution and before semantic
+validation, using one `os.LookupEnv` result per unique name per load. A copied
+previous result used as defaults is re-resolved; it cannot carry stale credentials.
+Validation can use `Value()`; resolution works with names. In addition to ordinary
+projection auditing, reject validator changes to a secret's private resolved state
+by comparing it with this load's lookup result. A same-name unresolved replacement
+must not evade the read-only validator contract. `Value()` returns empty
+on unresolved references; `IsResolved()` distinguishes that state. Private credential
+storage is excluded from every tree, historical snapshot, diagnostic, default
+format, JSON/text serialization, and inspection path. Provide safe formatting for
+all fmt verbs and name-only JSON marshaling; expose no credential decoder/codec.
+Only the deliberately named `Value()` accessor returns a credential. Applications
+own its use and must not embed it in hook messages. Secret references themselves
+need no `configkit:"secret"` tag: the existing tag/predicate classifies additional
+sensitive or derived values and can intentionally hide a reference name.
+
+The separate, opt-in bootstrap operation below loads a directory into the process
+environment. `Load` never calls it or changes process-global state. Applications
+choose a directory and call bootstrap once during startup before concurrent work.
+Bootstrap paths are native application-selected paths, independent of import roots.
+An empty or missing directory path is an error; an existing empty directory succeeds.
+
+```go
+func LoadSecretsDir(directory string) (*SecretsReport, error)
+func (r *SecretsReport) Entries() []SecretFileResult
+func (r *SecretsReport) WriteTo(output io.Writer) (int64, error)
+type SecretsError struct { /* private safe diagnostic storage */ }
+func (e *SecretsError) Error() string
+func (e *SecretsError) Diagnostics() []Diagnostic
+type SecretFileResult struct {
+    Name, Path string
+    Status SecretFileStatus
+    ExistingEmpty bool
+}
+type SecretFileStatus string
+// Constants use values "loaded", "skipped_existing", "not_applied", "failed":
+// SecretFileLoaded, SecretFileSkippedExisting, SecretFileNotApplied, SecretFileFailed.
+```
+
+Extract applicable code and tests directly from WeatherReporter's
+[secrets loader](../../../weatherreporter/internal/config/secrets.go), inspected
+at revision `5d306a202471e10fe98923a1df27d0b0fce5c666`. Reuse its filename checks,
+file-type rejection, single terminal LF/CRLF removal, and separation of `loadSecrets`
+from `applySecrets`. WeatherReporter now preloads file contents, validates its
+configuration, then applies environment writes with attempted rollback. ConfigKit
+preserves existing variables, reports partial apply failure without rollback, and
+keeps bootstrap separate from configuration loading. Record extraction symbols,
+revision and adaptations during implementation.
+
+Narratio's [secrets loader](../../../narratio/internal/app/secrets_env.go) at the
+revision recorded above is a complementary foundation: `loadSecretsFromConfigWithDependencies`
+preserves existing variables including empty ones, and `readAPIKeyFile` uses
+[bounded reads](../../../narratio/internal/fileops/read_bounded.go). Its
+[tests](../../../narratio/internal/app/secrets_env_test.go) provide reusable failure
+and environment-preservation cases. Adapt its immediate writes and aggregate/log
+reporting to ConfigKit's preflight and per-file structured report. Narratio skips
+invalid names, removes all trailing CR/LF characters, rejects a symlink at the
+selected directory path, and enforces restrictive permission bits. Those policies
+are not inherited: use ConfigKit's name/type/newline/directory rules and fixed
+bounds. ConfigKit does not add a permission-mode gate in the initial API;
+applications own deployment permissions. Extract bounded-read mechanisms without
+copying Narratio's application dependency framework or logger.
+
+Process only immediate entries, sorted by filename. Reject invalid names,
+symlinks, directories, and nonregular files, including entries whose names are
+already set. Allow a symlink in the explicitly selected directory path by resolving
+it to a canonical directory first. Validate all entry metadata before mutation.
+`os.LookupEnv` presence wins, including an empty existing variable: skip reading
+that file's contents and report `SkippedExisting` with `ExistingEmpty` set
+accurately. Read eligible files with bounds, remove exactly one terminal CRLF or
+LF, preserve all other whitespace, and reject NUL bytes. Empty file values are
+allowed at bootstrap, but fail if subsequently used by a required reference.
+Preflight all eligible reads before any `os.Setenv`. No writes occur on preflight
+failure. At apply time recheck presence before each write and skip newly set names.
+Concurrent mutation is unsupported; this is not an atomic compare-and-set facility.
+
+Return a non-nil report even on failure. Successful writes are `Loaded`, blocked
+files are `SkippedExisting`, the failed entry is `Failed`, and eligible but unwritten
+entries are `NotApplied`. Directory-level failures have a safe error without a
+fabricated entry. Stop on the first deterministic failure; reports include all
+entries enumerated so far, with unprocessed entries `NotApplied`. An apply failure
+preserves earlier successful writes and reports partial progress; do not attempt
+rollback that could overwrite unrelated process changes. Errors are `*SecretsError`
+with safe `secrets.*` diagnostic codes and no raw unwrap chain. Errors and report
+rendering contain names, paths, statuses, and safe classifications, never contents.
+
+Report entries and diagnostic slices are defensive copies. Bootstrap diagnostics
+include safely quoted canonical paths in messages with no invented configuration
+source ID or source location.
+Report formatting escapes control characters and propagates short writes/errors.
+
+The report is the authoritative bootstrap provenance: a blocked file explicitly
+identifies the existing process variable as the winner, even when empty. Keep it
+alongside the load result and render it before configuration explanation in an
+application command. Secret-resolution metadata records the variable name and
+`resolved`, `unset`, or `empty` lookup outcome (failures in diagnostics). It does
+not claim which external actor populated the process environment. Reports are
+separate snapshots, not accepted as evidence of the environment's later state;
+there is no implicit global report or stale filename attribution in `Load`.
+
+## CLI overrides
+
+Precedence is **application defaults < composed base < selected profile < explicit
+CLI overrides**. Domain normalization follows those inputs and is recorded.
+
+```go
+type Override struct {
+    Path Path
+    Value any
+    Flag string
+}
+// Options[T].Overrides is []Override.
+```
+
+Applications provide records only for flags explicitly set by the caller; parser
+defaults are not overrides. `Flag` is a required, non-secret display name, such as
+`--workers`, never the raw argument or value. ConfigKit has no CLI parser dependency.
+Values are detached and converted to ConfigKit-owned nodes before composition:
+accept built-in string, bool, integer and unsigned integer widths, finite floats,
+`nil`, `[]any`, and `map[string]any` recursively. Strings remain strings; do not
+parse YAML fragments or coerce textual booleans/numbers. Durations and secret
+references use strings. Reject other types, cycles, nonfinite numbers, and limits
+violations with safe `override.*` diagnostics.
+
+Validate every path/record before applying any. Reject root paths, sequence-index
+traversal, duplicate paths, and ancestor/descendant overlaps within the override
+list. Numeric tokens are mapping keys only here. Mapping-key paths may create
+missing mapping ancestors, but cannot traverse an existing scalar/null/sequence.
+The target path overlays with the same rules as profiles: mappings recursively
+merge; sequences and all other replacements are atomic. Use a whole sequence
+value to replace a list. Preserve supplied list order; mapping entries use canonical
+key order. Unknown struct fields and effective type mismatches fail normal strict
+decoding. Never allow `_configkit` directives or writes to the configured schema
+version path, including ancestor replacement. Overrides also work with `NoSource()`.
+
+Each record contributes a synthetic CLI source after file/profile sources, in
+record order, labeled with `Flag`, with no file coordinates. Explicit false, zero,
+empty string, null, and empty list remain present. Snapshot caller-owned values
+before applying them. History includes replaced inputs even for equal-value writes.
+Neither callers nor hooks may mutate supplied data concurrently with loading.
 
 ## Resolution lifecycle
 
@@ -204,13 +402,15 @@ application selects source and loading options
   → discover explicit imports and strictly compose the base
   → application schema-version check, when configured
   → apply the selected profile
+  → apply explicit CLI overrides to the node tree
   → apply static defaults and strictly decode
   → apply recorded application resolution and normalization
+  → resolve surviving secret references from the environment
   → collect application semantic validation diagnostics
   → return independently owned configuration and metadata
 ```
 
-Value precedence is defaults < composed base < selected profile. Resolution is
+Value precedence is defaults < composed base < selected profile < CLI overrides. Resolution is
 an application transformation after those layers, not another user override
 source. Future migration support belongs after base composition/version
 inspection and before profile application and typed decoding; migrations are
@@ -258,6 +458,10 @@ for the root. For example, `/transcription/concurrency` names a nested field;
 using the containing node's kind, distinguishing mapping keys from list indices.
 Use YAML field names, not Go field names.
 
+Semantic paths are not filesystem paths or URLs. Empty keys, literal `.`/`..`
+keys, and percent sequences retain their meaning; do not clean path components
+or percent-decode them. Only JSON Pointer escape processing applies.
+
 Distinguish absent, explicitly null, explicitly zero/false/empty, and other
 explicit values. An explicit value equal to its default remains explicit.
 Presence queries describe the surviving input contribution after overlays;
@@ -322,7 +526,9 @@ disabled, the presence of `_configkit` is an error, not an ignored directive.
   paths and network URLs are rejected.
 - Expand declarations in list order, sorting each glob's matches
   lexicographically by slash-separated relative path, with no locale sorting.
-  Use segment glob syntax (`*`, `?`, character classes), with no recursive `**`.
+  Use the slash-based [Go `path.Match` grammar](https://pkg.go.dev/path#Match)
+  (`*`, `?`, character classes, and backslash escapes), with recursive `**`
+  syntax forbidden. Backslashes escape pattern characters, not directory separators.
 - Traverse depth-first: process a file's own contribution, then its declarations
   in order. Ordering stabilizes source lists and diagnostics; it never permits
   duplicate definitions to override one another.
@@ -333,6 +539,8 @@ disabled, the presence of `_configkit` is an error, not an ignored directive.
   explicitly supply another root. Resolve symlinks and cleaned absolute paths
   for identity and containment. Imports and profiles must stay inside that
   canonical root. Retain declared paths and import-site locations for messages.
+  Relative and absolute symlink targets are permitted when their canonical target
+  is contained; the spelling of a symlink target alone does not determine containment.
 - An active-stack repeat is a cycle; report the complete cycle and import sites.
   A previously visited canonical file outside the active stack is a repeated
   import error, including diamond graphs and overlapping glob matches. Report
@@ -341,6 +549,9 @@ disabled, the presence of `_configkit` is an error, not an ignored directive.
 - Read each accepted source once per load. Do not promise an atomic snapshot of
   a filesystem being modified concurrently. Bound input bytes, node nesting,
   import depth, and source count with documented limits and structured errors.
+  Excess input is an error, never an implicitly truncated document. Glob traversal
+  counts examined directory entries, including nonmatches, against its work budget;
+  only accepted matches contribute to the sorted source order.
 
 Bytes/reader sources require a diagnostic label. Imports and profiles for these
 sources require an explicit filesystem base directory and root; never infer a
@@ -428,10 +639,10 @@ Strictness must be enforced by ConfigKit across the effective tree:
   let them bypass key, null, or type policy. Custom structural decoders are
   deferred; unsupported type shapes fail with a clear schema diagnostic.
 
-Keep parser nodes and errors internal. Exact package choice and Go API spelling
-are implementation decisions, to be exercised against consumer types before
-stabilization. The field-binding model must also support defaults, provenance,
-secret classification, and rendering consistently.
+Keep parser nodes and errors internal under the
+[upstream isolation contract](#yaml-dependency-and-public-api-isolation).
+The field-binding model must also support defaults, provenance, secret
+classification, and rendering consistently.
 
 ## Recorded resolution and validation
 
@@ -509,8 +720,8 @@ for inspection and is not promised to be reloadable. ConfigKit renderers must
 not expose raw secrets; returning typed configuration to the application is an
 intentional trusted boundary. ConfigKit does not log configuration on its own.
 
-Retain sufficient history for later full `explain` tooling, but defer historical
-value rendering, config diffs, and fingerprints. A future fingerprint design
+Provide full safe historical-value explanation in the initial release, as defined
+[below](#historical-value-explanation). Defer config diffs and fingerprints. A future fingerprint design
 must specify canonicalization, secret handling, and disclosure risks before
 claiming identity or reproducibility guarantees.
 
@@ -518,7 +729,7 @@ claiming identity or reproducibility guarantees.
 
 Applications may declare a schema-version path (for example `/schema_version`)
 and a pre-decode checker. The main document owns this value; fragments omit it,
-and profiles cannot supply or change it. Check fragment placement before
+and profiles/CLI overrides cannot supply or change it. Check fragment placement before
 composition so a version supplied only by an import is also rejected. ConfigKit enforces that placement when
 the application enables the mechanism. The application decides whether a version
 is required and which values it accepts. ConfigKit does not require one universal
@@ -553,6 +764,7 @@ type Options[T any] struct {
     Imports    bool
     Profile    string
     Filesystem *FilesystemContext
+    Overrides  []Override
 }
 
 type FilesystemContext struct {
@@ -593,7 +805,8 @@ both absolute `BaseDir` and `Root` when imports or profiles are used. A base mus
 be inside its root. These options do not change application field path meanings.
 
 `NoSource()` explicitly selects defaults-only loading. It contributes no document
-or source entry and marks no fields explicit; it still runs schema checking with
+or file source entry and marks no fields explicit by itself; CLI overrides still
+contribute synthetic sources and explicit values; it still runs schema checking with
 an absent version, defaults, resolution, validation, and snapshot/inspection.
 An application requiring an input version can therefore reject this mode.
 Imports, an active profile, or filesystem context with `NoSource()` are options
@@ -674,8 +887,8 @@ errors. Path spelling uses YAML names throughout.
 `FieldInfo` reports explicitness, null state, and effective `Origin`. An origin
 identifies defaults, input, or application resolution and associated source
 locations; it may be composite. `Change` reports layer/operation, prior origins,
-safe reasons, dependency paths, and related locations, without raw historical
-values. These are library-owned metadata types, never parser nodes. Queries
+safe reasons, dependency paths, and related locations. Safe historical values
+are available through `Explain`, rather than unrestricted raw-value references. These are library-owned metadata types, never parser nodes. Queries
 return copies or immutable views. Unknown paths yield no lookup/history and
 false explicitness. Callback mutation/validation paths are checked more strictly:
 misspelled struct fields are errors, while valid missing map entries are allowed.
@@ -751,7 +964,7 @@ type PipelineConfig struct {
     PollInterval time.Duration       `yaml:"poll_interval"`
     Timeout      time.Duration       `yaml:"timeout"`
     Artifacts    map[string]Artifact `yaml:"artifacts"`
-    APIKey       string             `yaml:"api_key" configkit:"secret"`
+    APIKey       configkit.Secret   `yaml:"api_key"`
 }
 
 func resolvePipeline(cfg *PipelineConfig, r *configkit.Resolution) error {
@@ -970,7 +1183,74 @@ func InspectPipeline(path, profile string, output io.Writer) error {
 
 Applications can use `Metadata().Lookup`, `Metadata().History`, and `Sources()`
 for structured inspection without implementing value rendering or importing a
-parser. A complete historical-value `explain` command remains deferred.
+parser. The explanation API below supplies safe values and formatting for application-owned
+`config explain` commands.
+
+### Historical-value explanation
+
+```go
+func (r *Result[T]) Explain(path Path) (Explanation, bool)
+func (r *Result[T]) WriteExplanation(output io.Writer, path Path) error
+
+type Explanation struct {
+    Path Path
+    Effective ValueSnapshot
+    Origin Origin
+    History []ValueChange
+    SecretReference *SecretReferenceInfo
+}
+type ValueSnapshot struct {
+    Present bool
+    Redacted bool
+    YAML string
+}
+type ValueChange struct {
+    Before, After ValueSnapshot
+    Origin Origin
+    Reasons []string
+    Dependencies []Path
+}
+type SecretReferenceInfo struct { Name string } // successful resolved lookup
+```
+
+Snapshots contain detached deterministic YAML text, never `any`, parser objects,
+or mutable internal trees. `Present=false` is absence/deletion and has empty YAML;
+null is present with `null` YAML. Redacted snapshots use the quoted marker
+`[REDACTED]`; `Redacted` also marks containers containing redacted descendants.
+Apply the same historical sensitivity/dependency closure as effective rendering
+before exposing any snapshots. A `Secret` projects only its environment name;
+its resolved value is never captured, even privately as history. For classified
+reference paths omit `SecretReference` as well as masking the projected name.
+
+Retain private persistent node snapshots or operation deltas with structural
+sharing; do not copy the entire tree for every changed leaf. Explain a known final
+or historical path, including removed paths; unknown paths return false, and the
+writer returns a safe unknown-path error without output. Malformed query paths
+also fail safely. Render the canonical JSON-pointer path, effective value/source,
+and chronological history: initial application-default (or Go-zero baseline),
+base contributions in traversal order, profile, CLI, and recorded domain changes.
+Include before/after values for changed or explicitly redefined paths, equal-value
+writes, removals, reasons/dependencies, and composite origins. Inputs overridden
+before decoding retain policy-validated node values even when they would not fit
+the final Go type. Defaults/final/domain snapshots use canonical typed projection;
+label those representations rather than claiming every historical input was typed.
+
+Container queries reconstruct the value at each relevant operation; descendant
+queries include ancestor replacement/removal and whole-sequence replacements,
+without implying element-wise sequence merging. Omitted fields/default retention
+must agree with the presence/default contract; do not fabricate a write for every
+import that leaves a path alone. Resolution history records the audited net change
+per path, not every intermediate Go assignment. Lookup of a secret reference
+annotates its effective name as resolved from the process environment; bootstrap
+file outcomes come from the separate report and are never guessed.
+
+`WriteExplanation` is a deterministic human renderer over this safe structured
+result. Escape control characters in paths, labels, names, and reasons, propagate
+writer errors/short writes, and permit partial output only on writer failure.
+No unredacted switch or bundled command is provided. For example an application
+can call it for `/transcription/concurrency` to show default `4`, an explicitly
+imported file's `8`, then a testing profile's `1`, with file line numbers and the
+profile as effective source. This does not require automatic `conf.d` discovery.
 
 ### Module-specific options
 
@@ -1102,20 +1382,76 @@ These are responsibilities, not a mandate for public interfaces or a file per
 concept. Start with one public package and a private YAML adapter; add package
 boundaries only when dependencies and ownership warrant them.
 
+Use standard-library filesystem, I/O, scalar conversion, reflection, ordering,
+and synchronization primitives. ConfigKit owns the policy that coordinates them:
+field binding, presence/default behavior, strict composition, overlays, provenance,
+resolution auditing, and secret propagation. One descriptor graph and document
+model support those operations; do not add competing binding or copying models.
+Convenience functions must not broaden accepted scalar syntax, silently truncate
+input, lose nested ownership, or substitute Go field promotion for YAML inlining.
+
+The [implementation plan](implementation.md#standard-library-mechanisms-and-custom-policy)
+owns the selected primitives and their integration checks. Test comparison uses
+`go-cmp` only in tests, never as the runtime resolution comparator. Additional
+dependencies remain case-by-case decisions based on code saved after accounting
+for adaptation, validation, and recurring maintenance.
+
+### YAML dependency and public API isolation
+
+Use the YAML organization's `go.yaml.in/yaml/v4` package behind `internal/yaml`.
+V4 is upstream's active development line; v3 receives security fixes only.
+This choice aligns a new library with ongoing upstream maintenance while keeping
+ConfigKit responsible for its own configuration model.
+See [upstream's version policy](https://github.com/yaml/go-yaml#version-intentions).
+The [implementation decisions](implementation.md#implementation-decisions) own
+the exact selected release-candidate pin and upgrade validation requirements.
+
+The adapter owns all direct upstream imports, including parser plugins, loading,
+and emission. Its boundary uses ConfigKit-owned document and diagnostic records.
+No upstream nodes, type aliases, options, plugin interfaces, error types, or
+generic constraints form part of ConfigKit's public declarations or callbacks.
+ConfigKit must not return parser-created values hidden inside `any`, metadata,
+or an error's unwrap chain. Applications use ordinary Go types, ConfigKit types,
+and the supported standard-library text interfaces; they need no YAML-package
+imports or upstream version presets to use ConfigKit.
+
+The existing `yaml` field-tag spelling is a ConfigKit-owned binding contract,
+not delegation to upstream reflection or marshal/unmarshal hooks. ConfigKit also
+owns scalar interpretation, duplicate detection, document count, source-coordinate
+conventions, diagnostic classifications, and effective-output formatting. Upstream
+options may help enforce these contracts internally but cannot redefine them.
+
+An upstream breaking API change must be absorbed in the adapter or other private
+implementation, preserving downstream Go source compatibility and documented
+configuration behavior. Parser upgrades do not independently authorize changes
+to ConfigKit's public contracts. If an upstream version cannot preserve those
+contracts, retain the compatible dependency or adapt the implementation; any
+intentional ConfigKit contract change follows ConfigKit's own release policy.
+
+This boundary isolates API coupling, not every dependency risk: the YAML module
+remains a transitive build dependency, and parser defects or behavior changes
+still require ConfigKit regression checks. It does not require a public parser
+interface, configurable backends, or simultaneous v3/v4 implementations.
+
+## Explicit non-goals
+
+General environment-variable configuration overlays are not on the roadmap.
+Secret references and explicit bootstrap are the supported environment convention.
+Literal-secret configuration and implicit secret-directory discovery are unsupported.
+
 ## Deferred scope
 
 The initial target excludes:
 
 - automatic `conf.d` discovery;
 - remote imports, arbitrary providers, and additional configuration formats;
-- environment or CLI field overlays;
 - custom merge strategies, sequence merging, reset operators, and profile inheritance;
 - parser-specific or custom structural decoding extension APIs;
 - generic dynamic-subtree conversion and module-registry APIs;
 - automatic schema migrations;
 - hot reload, runtime resource replacement, and remote resource validation;
 - a bundled CLI framework or commands;
-- full historical value rendering, config diffs, and fingerprints.
+- config diffs and fingerprints.
 
 Applications can build their own validate/effective/sources commands on the
 library. Broadening these boundaries requires concrete consumer evidence and

docs/roadmap/release-readiness.md

@@ -15,6 +15,7 @@ stages. The maintainer preparing the first release owns closure.
 | No release runbook | `docs/release.md` exists with the prerequisites, commands, publication guards, and verification required by release policy |
 | No complete release checker | A documented command exists and exercises every applicable automated candidate gate; manual gates and inapplicable checks are explicit |
 | No consumer verification procedure | Runbook verifies retrieval of the exact released module version and a representative consumer build without local replacement directives |
+| No verified secret/precedence/explanation contracts | Current contracts and passing bootstrap skip/partial-failure, secret isolation, CLI precedence, and safe historical-value regressions; representative consumer examples |
 | No validated release candidate | Version-matched release note, current contracts/examples, and complete checks against the exact committed candidate |
 
 No release can proceed while these prerequisites remain open. Close items only