eric/ConfigKit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Retire completed plans and outline named inheritance

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-29 20:11:42 +00:00
parent b604aa167b
commit 0a4b774bfe
9 changed files with 301 additions and 1318 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/adr/0007-source-backed-underlays.md
View File

@@ -2,9 +2,8 @@
## Status
Proposed draft for review. The
[filesystem sources and underlays roadmap](../roadmap/sources-and-layering.md)
records the target and implementation progress; the
Proposed draft for review. Completed planning and remediation records are
preserved in Git history at `b604aa1` under `docs/roadmap/`; the
[configuration contract](../config.md#layered-loading) owns current behavior.
This extends [ADR 0001](0001-separate-composition-and-overlay.md) and preserves

6
docs/development.md
View File

@@ -80,6 +80,12 @@ and current facade ownership belongs to the [component overview](internal/overvi
## Baseline validation
The completed filesystem-source and underlay roadmap, original feature request,
implementation plan, and resolved remediation record are preserved in Git history
at `b604aa1` under `docs/roadmap/`. Current behavior belongs to the
[configuration contract](config.md#layered-loading). Named-entry inheritance is
separate proposed work in its [roadmap](roadmap/named-inheritance.md).
The completed overlay rename plan and release-workflow simplification record are
preserved in Git history at `73ce6e9` under `docs/roadmap/`. Current overlay
behavior belongs to the [configuration contract](config.md#selected-overlays),

8
docs/roadmap/deferred.md
View File

@@ -16,8 +16,7 @@ The following remain deliberately excluded:
- automatic `conf.d` discovery;
- remote imports, arbitrary providers, and additional configuration formats;
- custom merge strategies, sequence merging, and reset operators;
- named overlay inheritance, tracked separately in the
[PromptKit inheritance request](feature-request-promptkit-inheritance.md);
- named overlay inheritance;
- parser-specific or custom structural-decoding extension APIs;
- generic dynamic-subtree conversion and module-registry APIs;
- automatic schema migrations;
@@ -28,3 +27,8 @@ The following remain deliberately excluded:
Applications may build their own validate, effective-value, source, bootstrap, and
explanation commands around the library. Broadening these boundaries requires
concrete consumer evidence, compatibility review, and an explicit design change.
Opt-in inheritance between named configuration entries is proposed in the
[named-inheritance roadmap](named-inheritance.md), based on the
[downstream request](feature-request-promptkit-inheritance.md). It is not yet
implemented and is distinct from inheritance between selected overlay files.

9
docs/roadmap/feature-request-promptkit-inheritance.md
View File

@@ -24,9 +24,12 @@ contract, if any, would be appropriate.
## Distinction From Layered Loading
The separate [filesystem and layered-loading request](feature-request-promptkit.md)
concerns acquiring configuration from different sources and combining it under
explicit composition or precedence rules.
The completed filesystem and layered-loading request is preserved in Git history
at `b604aa1:docs/roadmap/feature-request-promptkit.md`. The implemented
[layered-loading contract](../config.md#layered-loading) covers acquiring
configuration from different sources and combining it under explicit composition
or precedence rules. The proposed ConfigKit design for this remaining request
is in the [named-inheritance roadmap](named-inheritance.md).
Named inheritance concerns relationships between configuration entries: a child
identifies a base, which may itself identify another base. The relationship

147
docs/roadmap/feature-request-promptkit.md
View File

@@ -1,147 +0,0 @@
# PromptKit Feature Request: Filesystem Sources And Layered Loading
Status: downstream request for consideration, not an approved ConfigKit design
or implementation plan.
## Consumer Context
PromptKit is a Go library for defining, rendering, and executing prompts against
LLM backends. Its consumers supply prompt and profile definitions through local
files, embedded application defaults, maintained backend catalogs, and
programmatic values.
PromptKit intends to adopt ConfigKit to standardize configuration semantics and
remove custom loading machinery. Applications using both libraries should not
need to learn two different sets of generic YAML, presence, and diagnostic rules.
PromptKit is pre-1.0 and can change its definition formats and construction API
to accommodate a coherent shared configuration model, provided its substantive
prompting and execution capabilities remain available.
PromptKit has selected the following two capabilities as prerequisites for that
refactor. They are requested together because embedded defaults and explicit
layering form one common consumer workflow. This document describes downstream
needs only; ConfigKit's public API design and internal implementation remain for
its maintainers to determine.
## 1. Native `fs.FS` Configuration Sources
### Need
Allow a library or application to supply configuration from an `fs.FS`, including
`embed.FS`, without extracting files to disk or treating each file as an isolated
reader disconnected from its filesystem context.
PromptKit packages maintained profile catalogs and supports application-bundled
fallback profiles. Consumers also embed prompt definitions and associated text
assets so deployed programs do not require a separate configuration installation.
These sources should participate in ConfigKit loading as naturally as local files.
### Desired Consumer Outcomes
- Load configuration from an explicitly supplied filesystem and location within
it, with no dependency on a particular host directory layout.
- Retain meaningful source identity and locations in diagnostics and origins,
even when the source has no operating-system filename.
- When configuration uses supported imports or overlays, resolve relative
configuration references in the supplied filesystem context rather than
unexpectedly looking on the host filesystem.
- Make enough source context available for a consuming library to resolve its
own relative resources. For PromptKit, those include referenced message text;
understanding or rendering that text remains PromptKit's responsibility.
- Use the same configuration semantics for embedded and ordinary file-backed
definitions, rather than maintaining a separate embedded-loading path.
### Rationale
Bundled defaults are useful to many Go libraries, not just PromptKit. Native
filesystem support lets the owner package those defaults while callers choose
where their configuration comes from. ConfigKit's existing reader source can
accept file bytes, but supplying bytes alone does not satisfy the filesystem-
relative workflow described here.
## 2. Public In-Memory Layered Loading With Origins
### Need
Allow callers to explicitly supply configuration layers and obtain their effective
configuration with meaningful origins, without temporary files or conversion of
programmatic values to YAML and back.
PromptKit needs to combine library-maintained defaults, application-bundled
defaults, operator-selected configuration, and programmatic definitions. These
inputs do not necessarily live in one filesystem or follow an application-owned
overlay directory convention.
### Desired Consumer Outcomes
- Combine explicitly supplied embedded/file/byte-backed configuration and
programmatic values, with caller-selected precedence where overriding is intended.
- Distinguish combining independent contributions from intentionally overriding
earlier values. Accidental conflicts should not become implicit precedence
merely because sources happened to be discovered in a particular order.
- Preserve the distinction between an omitted field and an explicit zero, false,
empty value, or null so downstream domain rules can make deliberate choices.
Programmatic input needs a usable way to convey this intent as well as file input.
- Explain which source supplied an effective setting and identify relevant
sources when reporting a conflict or invalid setting. Origins must remain useful
when embedded and programmatic inputs have labels rather than host file paths.
- Allow loading through the public library interface without requiring a generated
root configuration file, temporary overlay directory, or serialization round trip.
### PromptKit-Specific Context
PromptKit's intended source priority, highest first, is programmatic profiles,
operator profiles, application fallback profiles, PromptKit-owned profiles, and
external catalog profiles. A higher-priority definition replaces the lower
definition with the same identity. This is distinct from profile inheritance:
a selected profile may name a base profile and override selected settings.
For inherited settings, PromptKit intends omission-based inheritance and explicit
value replacement, with a supplied provider-parameter map replacing the whole
inherited map. These examples explain why generic recursive mapping merge alone
does not cover every downstream operation. They do not request configurable
per-field merge strategies, a PromptKit-specific profile registry, or LLM rules
inside ConfigKit. The eventual public contract should make clear which generic
layering needs it serves and what domain interpretation remains with the consumer.
### Rationale
Many libraries provide defaults that applications then customize. A public
layered-loading capability would let those libraries share ConfigKit's presence,
validation, and origin semantics rather than each maintaining its own generic
composition and diagnostic machinery. Applications using ConfigKit directly and
indirectly through PromptKit would benefit from a consistent explanation of how
their effective configuration was obtained.
## Combined Workflow And Efficiency Needs
A representative consumer bundles default profiles, reads operator configuration,
adds a few programmatic settings, and constructs PromptKit once. Loading produces
an immutable set of definitions; subsequent prompt executions use that set without
reloading configuration. Errors need to be actionable at construction, before any
provider request. A new construction can capture later configuration changes.
Because PromptKit is a shared library, loading overhead and retained memory matter
across applications and catalog sizes. The requested capabilities should support
efficient processing of many definitions and layers without requiring repeated
reads, decoding, serialization, or copying of entire accumulated collections for
each added layer. Efficiency should be assessed against total input and produced
data, including retained provenance; this is not a request for an unconditional
O(n) promise or a fixed timing threshold.
## Scope And Follow-Up
This request does not seek automatic discovery, hot reload, live provider checks,
prompt rendering, model selection, or credential resolution specific to LLM calls.
PromptKit retains its domain relationships and execution responsibilities.
The follow-up design discussion should clarify supported layer inputs, how callers
express programmatic presence and precedence, filesystem-relative behavior, and
the source information consumers can inspect. PromptKit can adapt its APIs and
formats to those decisions; historical loading order is not a compatibility demand.
The current [configuration contract](../config.md) and
[deferred scope](deferred.md) describe ConfigKit's existing behavior and exclusions.
This request proposes expanding the relevant scope, not treating the desired
features as already available. PromptKit plans to consume released ConfigKit
support before implementing the dependent refactor.

565
docs/roadmap/implementation.md
View File

@@ -1,565 +0,0 @@
# Filesystem Sources and Underlays Implementation Plan
## Purpose and baseline
Implement the [feature roadmap](sources-and-layering.md) in the numbered order
below. Each stage is sized as one focused implementation assignment for a
gpt-5.6-terra coding agent. The baseline is
`36172ed20e832b371506d6aceb02dc6c8f50f71b`, plus the documentation changes committed
with this plan. No feature implementation is part of this planning pass.
Read `docs/development.md` and the policies it selects for the stage. The roadmap
owns target semantics; [ADR 0007](../adr/0007-source-backed-underlays.md) and
[ADR 0008](../adr/0008-complete-on-return-loading.md) record the proposed rationale.
Their draft status is independent of implementation progress: do not mark them
accepted automatically. The [configuration contract](../config.md) is the
compatibility baseline. Preserve the root-owned facade and private engine/tree
boundary.
At each stage, preserve unrelated work, add focused behavioral coverage with the
change, update declaration documentation for public additions, and run the checks
specified below. Keep intermediate commits building and tested. Do not expose
placeholder public APIs that silently ignore options. Internal scaffolding may
precede the public feature. Use plain-English commits under the agent policy.
## Settled implementation decisions
### Public input surface
Use these names and shapes; bodies and private fields belong to implementation:
```go
// Root-owned opaque filesystem namespace, copying preserves namespace identity.
type Filesystem struct { /* private engine state */ }
func NewFilesystem(label string, filesystem fs.FS) Filesystem
func (f Filesystem) File(name string) Source
func Mapping(label string, values map[string]any) Source
// Extend the existing record; nil FS selects host-filesystem context.
type FilesystemContext struct {
BaseDir string
Root string
FS *Filesystem
}
type Input struct {
Source Source
Imports bool
Filesystem *FilesystemContext
}
type Underlay struct { Inputs []Input }
type LayeredOptions[T any] struct {
Definition Definition[T]
Underlays []Underlay
Overlay string
OverlayFilesystem *FilesystemContext
Overrides []Override
}
func LoadLayered[T any](main Input, supplied ...LayeredOptions[T]) (*Result[T], error)
type ResourceContext struct {
FS fs.FS
BaseDir string
}
func (r *Result[T]) SourceContext(id string) (ResourceContext, bool)
func (r *Resolution) SourceContext(id string) (ResourceContext, bool)
```
`fs` above means the standard `io/fs` package. All named public types are defined
at the root, not aliases of engine types. Existing `Load` and `Options` stay
source-compatible apart from the additive field in `FilesystemContext`. Avoid a
parallel lifecycle implementation: adapt both entry points to one private load
request. No new dependency or minimum-Go-version change is needed.
- Like `Load`, `LoadLayered` accepts zero or one options value. A zero `Source`
stays invalid. Represent omitted main input as `Input{Source: NoSource()}`.
`NoSource` contributes no document, source ID, or context; preserve its existing
invalid import/context combinations. An empty underlay group is a no-op.
- Each underlay contains strictly composed inputs, including enabled imports.
Groups are ordered lowest to highest. Repeated explicit inputs in different
tiers are allowed subject to each source's reuse contract. Within a tier,
repeated file identities remain import/repetition errors. Equal scalar claims
still conflict. Independent synthetic sources are distinguished by occurrence.
- `Input.Imports` opts in per input and its import graph. The main source and its
imports form the main tier. There is no programmatic high-priority escape hatch:
a mapping source can be main input or any underlay contribution; CLI overrides
retain their explicit-flag contract.
- Active named overlays in `LoadLayered` require `OverlayFilesystem` explicitly.
Their names, root-relative lookup, directive prohibition, and version protection
follow existing rules. An inactive overlay needs no context; an unused explicit
context has no effect. Ordinary `Load` retains implicit selection of its source's
filesystem context and existing `NoSource` restrictions.
### Filesystem namespaces, paths, and acquisition
`NewFilesystem` allocates a reusable private namespace token and retains the
caller-owned filesystem without opening it. Copies share the token; separate
constructors are separate namespaces even for the same underlying filesystem.
Never compare or hash arbitrary `fs.FS` values. Validate nonempty safe labels and
nil filesystem arguments through deferred source diagnostics, following existing
source-constructor conventions. The zero `Filesystem` yields an invalid source.
`Filesystem.File(name)` uses a non-root `fs.ValidPath` file name relative to that
namespace. The default root is `.` and the base is the file's containing directory.
An optional filesystem context can narrow the root; file sources continue to
derive their own base directory. An explicit FS must identify the same namespace
as the file source; context overrides for native file sources supply that FS.
A host `File` cannot be rebound to a virtual namespace.
For byte, reader, and mapping inputs, an explicit context supplies their namespace
and base: with FS set, require valid slash-relative Root and BaseDir (use `.` for
the namespace root), with BaseDir inside Root. With FS nil, preserve the existing
absolute host-directory requirements. Labels never imply a context.
Resolve import paths relative to the declaring document, normalizing joined
paths and rejecting escape from the configured root. Preserve the existing
relative-path/glob language, including contained parent references, optional
missing behavior, sorted matches, bounded traversal, and cycle diagnostics.
Virtual paths use `path`, not host `filepath` rules. Imports stay in their namespace;
generic filesystems provide their own symlink behavior. Preserve canonicalization,
regular-file checks, and `os.Root` containment for host acquisition. Use ordinary
`fs.FS` operations and `fs.ReadDirFile` where available; support the standard
`fs.ReadDir` fallback without claiming control over its internal allocation.
Configuration reads close each file ConfigKit opens, including read/stat/parse
failures. Caller readers and caller filesystems are never closed by ConfigKit.
Namespace tokens live with source objects; deterministic load-local IDs are
assigned by acquisition, not addresses or global counters.
### Programmatic representation and diagnostics
`Mapping` snapshots its input at construction, matching `Bytes`. Convert directly
to the private document tree using the CLI converter's existing built-in scalar,
`[]any`, and `map[string]any` repertoire. Preserve integer precision, finite-float
validation, sorted keys, sequence order, explicit null, and cycle rejection.
Typed nil maps/slices mean null, as in existing overrides. A nil root mapping is
invalid because document roots are mappings; `{}` is a valid empty mapping.
Constructor failures are stored and surfaced on load. Detach once at construction;
do not retain caller maps or serialize them. Concurrent caller mutation during
construction is outside the supported ownership contract.
Generic conversion returns private issues; adapt them to existing `override.*`
codes for CLI input, and `source.invalid` or `source.limit_exceeded` for mapping
input. Do not leak raw values or conversion errors. Source/path locations use
the label and semantic path, with zero line/column for synthetic input. Apply
reserved `configkit` directive rules consistently: with imports enabled, a
mapping can contain the same directive structure as YAML. Without context,
relative imports fail under existing context rules.
### Ordering, lifecycle, budgets, and metadata
Acquire and strictly compose main input first, construct the schema, and check
the optional version. Only the main source owns the version; main imports and
all underlay documents must satisfy fragment-style version-path protections.
With `NoSource` main input, pass an absent version to the checker. Do not run the
checker once per tier or validate the final overlaid version instead.
Then acquire/compose underlays in listed order and combine them recursively,
apply the composed main tier, selected overlay, and CLI overrides. Execute typed
defaulting/decoding, recorded resolution, secret lookup, validation, and final
publication once. Syntax/composition/version failures cannot be masked by higher
precedence; effective typed validation sees the final assembled input only.
Keep source-record order separate from precedence: main depth-first sources,
underlay tiers in order with each input traversed depth-first, selected overlay,
then CLI. IDs remain `source-N`. Existing ordinary loads retain their source order.
The history application order follows precedence, not acquisition order.
Extend `Layer` with `LayerUnderlay = "underlay"`. Add to `SourceInfo` a `Kind
SourceKind` and `UnderlayIndex int` (one-based original tier position for underlay
sources, zero otherwise). Define `SourceKindFile`, `SourceKindFilesystem`,
`SourceKindBytes`, `SourceKindReader`, `SourceKindMapping`, and `SourceKindCLI`
with string values `file`, `filesystem`, `bytes`, `reader`, `mapping`, and `cli`.
Imported documents inherit tier role/index and have their actual acquisition kind.
Preserve existing host Path meaning; filesystem-source Path is namespace-relative;
synthetic Path stays empty. Labels for native files combine namespace label and
relative path for display, while identity uses the token and path.
Origins continue to refer to source IDs through locations. Composite origins
preserve all contributors; equal-value overrides and displaced descendants stay
inspectable. Carry source context separately from serializable source records.
Keep existing inspection schema and renderer structure, adding the new layer/kind
where appropriate rather than changing existing output gratuitously.
Use one load-wide accounting object for layered loads: existing 64 MiB aggregate
input bytes, 1,024 acquired sources, 1,000,000 input nodes, and glob-entry budget;
existing per-source 8 MiB, depth 256, active import depth 64, and retained-history
limit remain. Mapping key/string bytes use the existing CLI converter's byte
accounting. Charge each accepted occurrence once, including CLI contributions;
do not charge intermediate copies as new input. Explicit empty groups and
`NoSource` consume no source allowance. Ordinary loads retain their established
limit behavior. Use existing source/import/overlay/override limit codes according
to the acquisition that exceeds a budget. Add `CodeUnderlayVersion =
"underlay.version"` for version protection in underlay roots; imported fragments
retain `import.version`. Avoid a separate arbitrary tier-count limit.
### Passive resource contexts
`SourceContext` returns false for unknown IDs or sources without a context, and
performs no I/O. The result method is nil-safe; the resolution method follows
existing callback lifetime guards. Return a record copy containing a borrowed
filesystem and slash-relative source BaseDir within its configured root.
Consumers obtain source IDs from ordinary origins; there is no single-context
query that guesses among composite or derived origins. Dependencies continue to
identify the sources of application-derived values through existing metadata.
For native inputs, use a namespace-rooted view over the caller's filesystem and
retain its lifetime semantics. For host input, retain the canonical root path,
not an open `os.Root`. A private `fs.FS` adapter validates a relative resource
name, resolves it using existing host containment semantics, opens a fresh root,
and opens the resource. Return a private `fs.File` wrapper whose Close releases
both file and root, including failed-open cleanup. Expose `fs.ReadDirFile` when
the underlying directory supports it. Resources themselves are application data:
do not apply configuration YAML parsing or configuration-byte budgets to them.
Helpers such as `fs.ReadFile` then close the returned handle normally. Errors
follow filesystem access conventions, separately from load diagnostics.
Neither queries nor retained results keep library-owned filesystem handles open.
Files/resources can change or disappear after loading; configuration/metadata
remain usable. No background work, hidden reopening on metadata queries,
filesystem snapshot, universal sandbox, or transaction guarantee is added.
## Stage 1: Index mapping assembly and establish scaling baselines
**Completed 2026-09-29.** Private per-assembly mapping indexes now serve strict
composition and recursive overlay lookup while ordered entries and diagnostics
remain stable; conflict groups are checked once. Focused Compose/Overlay tests
and the everyday checker passed. On Linux amd64, the 512-entry benchmark
fixtures measured approximately 4.0 ms/op before and 3.3 ms/op after for wide
composition, and 12.9 ms/op before and 12.0 ms/op after for recursive overlay.
These are comparative evidence, not timing gates; provenance work still
dominates the latter and belongs to the next stage.
Scope: `internal/document` and engine composition/overlay lookup, with existing
behavior only.
1. Add parameterized benchmarks for wide disjoint composition and recursive
overlay, recording runtime and allocations at increasing sizes. Keep fixtures
within existing budgets and record the baseline in the stage handoff.
2. Replace repeated linear mapping-key lookup during bulk assembly with a private
indexed builder alongside ordered entries. Build each target index once per
assembly session and update it for appended keys. Keep tree traversal and
diagnostic order stable; avoid mutable indexes exposed through public APIs.
3. Check each strict-composition claim group once while preserving first-conflict
ordering and all relevant locations. Preserve original input ownership.
4. Extend existing composition tests only for uncovered ordering/conflict risks.
Acceptance: existing Compose/Overlay tests pass unchanged; broad keyed input
no longer incurs a full key scan for each insertion. Run the focused composition
tests and the everyday checker. Benchmarks are evidence, not timing gates.
## Stage 2: Prepare load-owned assembly and incremental provenance
**Completed 2026-09-29.** Ordinary loads now apply selected overlays and CLI
overrides to one owned working document, while copy-preserving helpers remain
available for direct use. Acquired history trees stay detached; composite
contributors survive overlay application. Origin deletion visits only replaced
subtrees, mapping contributors append without copying growing lists, and history
replay shares indexed lookup across applications. Focused behavior tests, the
everyday checker, and the engine race suite passed. On Linux amd64 at 128 small
applications, owned assembly measured about 0.30 ms/op versus 6.0 ms/op for
repeated copy-preserving application; alternating subtree/scalar replacement
measured about 0.39 versus 7.3 ms/op. These benchmarks are evidence, not gates.
Scope: `compose.go`, `overlay.go`, origin indexes, override application, and
history replay in `internal/engine`.
1. Introduce a private load-owned accumulator with explicit ownership. Existing
copy-preserving helpers may wrap it, but sequential assembly must mutate one
owned working tree rather than clone all accumulated state per tier.
2. Retain immutable acquired inputs for history, detached from mutable working
nodes. Parameterize internal overlay application by incoming layer and origins
so a composed tier's imported contributors are preserved.
3. Remove displaced origins by visiting the affected subtree. Accumulate mapping
contributors without copying an ever-growing ancestor list on each addition;
materialize defensive public records at the existing publication boundary.
4. Preserve equal-value replacement events, descendant displacement, redaction,
and typed-decoding reconciliation. Extend benchmarks for many small applications
and repeated subtree/scalar replacement. Do not add a persistent-tree framework.
Acceptance: existing override/history/rendering/ownership tests pass; benchmark
growth avoids accumulated full-tree copying. Run the everyday checker and
`GOPROXY=off go test -race ./internal/engine` for ownership changes.
## Stage 3: Unify private filesystem acquisition without public changes
**Completed 2026-09-29.** Imports, glob traversal, and selected-overlay loading
now use a private access boundary with the existing contained host backend and
a native `fs.FS` backend. Namespace tokens and occurrence identities are separate
from labels and paths; repetition tracking is composition-local and input/glob
accounting can be shared. Native tests cover slash-relative imports, sorted
globs, optional missing paths, root escape, overlays, repeated/cyclic imports,
non-comparable filesystems, and handle closure on success and representative
failures. Existing host tests, focused native tests, and the everyday checker
passed. No public constructor or context field was added.
Scope: engine source/file, filesystem, import, glob, and selected-overlay code.
1. Extract a small private namespace/access boundary used by imports and overlays.
Keep the host implementation's canonicalization, containment, regular-file
checks, safe diagnostics, deterministic traversal, and close behavior.
2. Separate namespace identity from source labels and resource paths. Make
repeated/cycle tracking belong to an input composition group, and accounting
capable of being shared across groups.
3. Add a native `fs.FS` backend internally. Resolve paths using slash semantics,
preserve glob grammar and optional-missing behavior, and avoid host fallback.
4. Use real host tests plus `fstest.MapFS` and a small tracking/failing filesystem
to cover imports, missing paths, directory/read failures, and handle closure.
Include a filesystem with a non-comparable dynamic value.
Acceptance: all existing host source/import/glob/overlay tests remain green;
native backend tests establish namespace isolation and supported paths. Run the
everyday checker. No public constructors land until the next stage.
## Stage 4: Expose native filesystem sources through the facade
**Completed 2026-09-29.** The root API now exposes reusable native filesystem
namespaces, namespace-aware contexts, and source acquisition kinds. Ordinary
loads support native files, opt-in imports, selected overlays, and byte/reader
contexts while retaining host semantics. Public tests cover embedded fixtures,
`fstest.MapFS`, same-label namespace isolation, wrapper copies and reuse,
constructor/context diagnostics, and host/CLI source kinds. The everyday checker
passed, including public-boundary and documentation checks.
1. Add `Filesystem`, `NewFilesystem`, `Filesystem.File`, and the additive FS
context field at root and engine boundaries. Constructors follow deferred
diagnostics; copying the wrapper preserves namespace identity.
2. Integrate native sources with ordinary `Load`, including explicitly enabled
imports and named overlays. Enforce the path/context rules settled above.
3. Add `SourceKind` and populate it for every current/native acquisition. Preserve
host paths and use namespace-relative virtual paths with accurate locations.
4. Add facade-level tests using embedded fixtures, `fstest.MapFS`, byte/reader
context, namespace-label collisions, and source reuse. Keep backend error cases
at their Stage 3 owner. Update current source/API docs to this implemented scope.
Acceptance: native `Load` works through the public API and existing sources retain
their contracts. Run the everyday checker, including public-boundary tests.
## Stage 5: Add snapshotted mapping sources
**Completed 2026-09-29.** `Mapping` now snapshots supported built-in values at
construction, retaining safe deferred diagnostics for invalid roots, values,
cycles, nonfinite floats, and limits. A shared private converter preserves CLI
override behavior through a separate adapter. Mapping inputs participate in
ordinary directives, imports, version checks, decoding, and inspection with
their own source kind and synthetic locations. Public tests cover YAML
equivalence, reuse after caller mutation, null/empty/zero distinctions, exact
integers, explicit filesystem context, and safe failures. The everyday checker
and engine race suite passed.
1. Extract reusable value conversion from `override_value.go`; keep CLI-specific
path, label, and diagnostic handling in its adapter.
2. Implement `Mapping` at root and engine boundaries with constructor-time
snapshotting and deferred safe failures. Source-ID assignment at acquisition
must not mutate a reusable stored tree shared by another load.
3. Route converted documents through the same directive processing, strict
composition, version checking, decode, and inspection path as YAML sources.
4. Test omission/zero/false/null, nil versus empty collections/root, exact integers,
invalid/cyclic values, imports with explicit context, and caller mutation after
construction. Verify mapping input does not gain CLI origins or restrictions.
Acceptance: equivalent byte/mapping inputs have equivalent semantic results and
origin meaning, with genuine location differences allowed. Existing override
diagnostics remain stable. Run the everyday checker and engine race tests.
## Stage 6: Implement private grouped-underlay orchestration
**Completed 2026-09-29.** Ordinary and grouped engine loads now use one private
request lifecycle. Group-local strict composition and repetition tracking,
load-wide source/byte/node/glob accounting, main-owned version checking,
underlay layer/index metadata, and precedence-ordered assembly feed one final
decode, resolution, validation, and publication. Engine tests cover two tiers,
multiple contributions, source/history order, absent main/defaults-only loads,
main/import and group conflicts, version protection, shared limits, reusable
files across tiers, and replacement of invalid intermediate typed input. The
everyday checker and engine race suite passed. No root layered API is exposed yet.
1. Add private request/input/group structures matching the settled facade design.
Adapt ordinary `Load` to the shared core while preserving its options and
source-order behavior. Keep its existing validation order where observable.
2. Strictly compose each group with group-scoped repetition tracking and shared
aggregate accounting. Apply the lifecycle order defined above, including the
single main-owned version check before underlay precedence assembly.
3. Add underlay layer/index metadata and version diagnostics. Keep history in
precedence order while source IDs follow acquisition order. Apply composed
main origins over underlays without relabeling main imports as overlays.
4. Reuse Stage 2 assembly and the existing decode/resolution/secret/publication
pipeline. Empty groups, omitted main, defaults-only loads, and underlay-only
loads must follow the chosen semantics.
5. Add engine coverage for two underlay tiers, multiple strict contributions,
main/import conflicts, version checks/absent main, limits across groups, and
one final decode/resolve/validate lifecycle. Test an invalid intermediate typed
value replaced by valid higher input, alongside unmaskable syntax conflicts.
Acceptance: engine layered requests produce correct values and full provenance;
ordinary-load tests remain green. Run the everyday checker and engine race tests.
## Stage 7: Expose layered loading and complete inspection integration
**Completed 2026-09-29.** The root facade now offers `Input`, `Underlay`,
`LayeredOptions`, and `LoadLayered` with copied options/context records. Explicit
host or native selected-overlay contexts work independently of the main source;
repetition uses native namespace identity rather than a display path alone.
Root layer/source conversions and explanation rendering include underlays.
Public tests cover mixed host/native/mapping tiers, recursive precedence,
empty/zero/false/null and sequence replacement, source order and tier indexes,
composite and displaced origins, resolution, redaction, and zero-underlay
equivalence. The everyday checker and full race suite passed.
1. Add `Input`, `Underlay`, `LayeredOptions`, and `LoadLayered` root wrappers and
conversions. Copy option slices/context records into private load state while
preserving reusable namespace identities and single-consumption readers.
2. Wire explicit selected-overlay context for layered loads, including host and
native namespaces. The selected file may not duplicate an acquired file in
that namespace, preserving selected-overlay repetition protection.
3. Finish root Layer/SourceInfo conversions, history/explanation rendering,
composite mapping attribution, deleted descendants, default and derived
origins, and secret redaction. New records must remain detached.
4. Add representative public tests for full precedence and same-name recursive
mapping combination; retain zero/false/empty/null distinctions and atomic
sequence replacement. Test ordinary versus zero-underlay layered equivalence
using equivalent explicit overlay context when required.
Acceptance: callers can load native, host, and mapping tiers in one public call;
configuration, metadata, and safe rendering agree. Run the everyday checker and
`GOPROXY=off go test -race ./...`.
## Stage 8: Add handle-free source-context queries
**Completed 2026-09-29.** Results and active resolution callbacks now expose
passive per-source `ResourceContext` records. Native views borrow caller-owned
filesystems under their configured roots; host views retain only canonical root
paths and open/close fresh roots with each resource file. Main, imported,
selected-overlay, and explicitly contextual synthetic sources have their own
base directories; contextless and CLI sources do not. Public tests verify
origin-based resource access, no I/O on queries, later mutation/deletion,
borrowed-filesystem closure, and callback guards. Tracking tests cover resource
Open failure cleanup, returned file/directory ownership, and loading-handle
closure across success and representative failures. The everyday checker and
full race suite passed.
1. Retain context descriptors keyed by acquired source ID, including imported
sources, in private inspection and live resolution state. Keep descriptors
separate from serializable metadata and preserve callback guards.
2. Implement the root query methods and `ResourceContext`. Native contexts expose
rooted borrowed access; host contexts use the passive adapter described above.
Include host main files even when imports are disabled, and explicit contexts
on byte/reader/mapping sources. Sources without contexts return false.
3. Verify each origin's BaseDir points to its actual source. Composite origins
require callers to follow individual locations; derived dependencies retain
their existing meaning. No arbitrary last-source context is selected.
4. Test that queries cause no filesystem operations; loaded values and metadata
survive source deletion; later resource access observes mutation/missing files;
and caller filesystem closure affects only later resource operations.
5. Verify owned files/root handles close on successful loading and representative
acquisition, parse, composition, version, and validation failures. Test resource
Open failure cleanup and returned-file Close ownership separately. Prefer
narrow tracking seams over timing-based goroutine/descriptor-count assertions.
Acceptance: a retained underlay resource can be opened relative to its true origin
after load, with no library-owned loading handles retained. Run the everyday
checker and full race suite.
## Stage 9: Validate cross-feature consistency and scaling
**Completed 2026-09-29.** A compact public matrix now checks equivalent host,
native, byte, reader, and mapping inputs; tier-local repetition versus cross-tier
reuse; reader consumption; and safe diagnostics/effective/history output after
secret replacement. Engine tests verify exact occurrence accounting across
imports, shared glob/source limits, and the mixed-source CLI ceiling. Rendering
now indexes source labels once per explanation instead of scanning all records
for every location. Loading and passive context ownership were reviewed against
ADR 0008; no handles or background work are retained by queries.
On Linux amd64 with `-benchmem` and 100 ms samples, the 512-entry assembly-only
composition measured about 3.5 ms/op (the earlier post-index baseline was about
3.3 ms), and recursive overlay about 2.2 ms/op (about 12 ms at the first-stage
handoff before subsequent provenance work). The 128-application owned assembly
remained about 0.30 ms/op versus 6.35 ms/op for copy-preserving application.
End-to-end layered measurements were about 7.15 ms/op for 512 broad catalog
entries, 2.59 ms/op for 128 small tiers, 3.27 ms/op for 128 alternating subtree
replacements, 1.86 ms/op for 64 imported native defaults, and 1.36 ms/op for
64 retained-history applications. Allocations grew roughly with these fixture
sizes. I/O, sorting, decoding, callbacks, and retained output contribute to
full-path cost; these observations are not timing gates or an unconditional
complexity guarantee. The everyday checker and full race suite passed.
1. Consolidate a small semantic equivalence matrix across host, native, bytes,
readers, and mappings, using equivalent logical inputs and contexts. Cover
strict conflict versus intentional precedence and version-path preservation;
avoid duplicating every lower-level case at every source permutation.
2. Exercise load-wide limits across mixed sources/CLI, repeated explicit tiers,
shared namespace imports, and source IDs/history order. Protect against double
charging and reader re-consumption. Confirm secret canaries stay absent from
diagnostics and historical/effective output after replacements.
3. Run parameterized benchmarks for broad catalogs, many small tiers, subtree
replacement, imported native defaults, and retained history. Compare Stage 1
baselines with the final path. Address demonstrated avoidable quadratic scans
or accumulated copying within this feature's scope. Account for sorting,
filesystem operations, callbacks, and output/history size explicitly.
4. Review source/context ownership and success/failure cleanup against ADR 0008.
Keep performance evidence in the stage handoff, not permanent timing gates.
Acceptance: no semantic discrepancy remains among equivalent operations; scaling
results substantiate the engineering target without an unconditional O(n) claim.
Run the everyday checker and full race suite, reusing successful checks for an
unchanged final tree. Benchmarks run with `-benchmem` and parameterized sizes.
## Stage 10: Complete consumer examples and architecture documentation
**Completed 2026-09-29.** Maintained executable examples now cover a simple
embedded filesystem source and a layered catalog with imported embedded defaults,
a mapping underlay, an independent host main file, explicit false/zero values,
domain validation, and resource access through the retained origin's source ID.
The configuration contract, public declaration comments, consumer/development
guides, README, component overview, architecture/deferred text, roadmap status,
and draft ADR status links reflect implemented behavior. Named inheritance remains
separate and deferred; the ADRs remain drafts. The everyday checker and full race
suite passed. No release preparation or publication was performed.
1. Add a maintained simple native-filesystem example and a combined example with
embedded defaults, two underlay tiers, independently located main input,
programmatic contributions, and a relative resource from a retained origin.
Keep fixtures secret-free and offline; include zero/false presence and domain
validation. The consumer should perform no generic post-load merge/defaulting
or provenance reconstruction. Reuse existing example/test conventions.
2. Update `docs/config.md`, Go declarations, `docs/consumers/`, README,
`docs/internal/overview.md`, and applicable architecture/deferred-scope text
to describe implemented support. Preserve strict-within-tier and recursive-
between-tier terminology, distinguish source kinds from precedence roles,
and document context lifetimes and the explicit source-ID query workflow.
3. Add the relevant tests/commands to the development guide's existing task map.
Review ADR links and statuses, preserving draft status pending acceptance and
accepted ADR decision bodies. Keep named inheritance separately deferred.
4. Mark implementation progress accurately in this plan and the roadmap. Do not
retire plans, choose/tag a version, prepare release gates, or publish as part
of implementation; those belong to separate release preparation.
Acceptance: examples compile and execute, documentation describes only completed
capabilities, and simple `Load` usage remains simple. Run the everyday checker
(which includes examples, links, and boundary checks); run the full race suite
if code changed since its last successful run. Report completed stages, checks,
benchmark conclusions, and any remaining limitations honestly.
## Validation commands
Use the development guide's focused commands during each stage. The normal
stage-end check is `./scripts/check.sh`; it already includes tests, vet, build,
formatting, and documentation links. Ownership stages additionally use the race
commands above. Benchmarks may use:
```sh
GOPROXY=off go test ./internal/engine -run '^$' -bench 'Compose|Overlay|Underlay' -benchmem
```
Name the new benchmarks to match that expression. Select representative sizes
under existing limits. No CI wait, human macOS validation, external service,
credentials, or new release gate is part of this plan.
## Open Questions
None required to implement this plan. The remaining roadmap recommendations
have been selected above: labeled snapshotted mappings and separate source-context
queries. Private helper names and package-local factoring remain implementation
choices within these contracts. If implementation uncovers a genuine conflict
with existing behavior, report it rather than silently adding a restriction,
weakening compatibility, or broadening a guarantee.

281
docs/roadmap/named-inheritance.md Normal file
View File

@@ -0,0 +1,281 @@
# Opt-In Named Configuration Inheritance
Status: proposed feature design for agreement; not implemented. This document
defines the intended end state and the remaining architectural decisions. A
separate implementation plan will define execution steps after design agreement.
## Purpose and priorities
Allow an application to designate a collection of named configuration entries
whose members can inherit settings from one parent in that collection. ConfigKit
resolves the relationships during loading and returns final typed configuration
with inspectable provenance. Applications retain ownership of field meanings,
domain validation, and execution.
The [downstream request](feature-request-promptkit-inheritance.md) supplies a
concrete need for reusable named defaults and chains of specialization. ConfigKit
adopts a shared structural model, using its existing recursive combination and
presence conventions. The design serves named presets, connection settings, or
similar configuration entries without interpreting their domain.
Priorities are consistency and predictability, accurate presence/provenance, and
reduced consumer configuration infrastructure. Computational efficiency matters
within those goals: resolve shared ancestors once and account for the size of
materialized results and retained history. Avoid an unconditional O(n) promise.
## Current baseline
Baseline: `b604aa167b684525f0d5d806d9c4267824151ee4`, after the filesystem-source
and underlay implementation and its remediation.
- The [configuration contract](../config.md) supports host/native filesystem,
byte, reader, and snapshotted mapping sources through a shared engine.
- Independent contributions compose strictly within a tier; explicitly ordered
tiers combine using recursive overlay. Typed defaults sit below source input;
selected overlays and explicit CLI overrides retain their documented roles.
- Named application entries can already be represented as string-keyed mappings.
ConfigKit currently treats relationships between their keys as application data.
- Recorded application resolution runs after typed decoding and preserves
dependencies and provenance. Consumers currently own relationship resolution;
there is no built-in pre-decode inheritance phase.
- Source contexts support later application-owned resource access. Loading is
synchronous and complete on return, with library-owned loading handles closed.
This feature is a deliberate extension of structural input processing. It follows
the [architecture goals](../policy/architecture.md#success-criteria), the
composition distinction in [ADR 0001](../adr/0001-separate-composition-and-overlay.md),
the presence/provenance principles in
[ADR 0003](../adr/0003-preserve-presence-and-resolution-history.md), and the
complete-on-return design in [draft ADR 0008](../adr/0008-complete-on-return-loading.md).
## Target configuration model
### Explicit collection declarations
Applications opt in particular string-keyed collections through their reusable
configuration definition, consistently for ordinary and layered loading. Each
mapping key is the entry's identity. References use exact names within the
declared collection, independent of filesystem namespaces or declaration order.
One entry can name one parent; chains and shared parents are supported.
The declaration identifies structural metadata rather than enabling callbacks
that choose arbitrary lookup or merge strategies. Ordinary fields remain ordinary
data outside declared collections. The declaration syntax and treatment of the
parent-reference field are addressed in Open Decisions below.
Mappings with no parent remain independent entries. Explicit null or scalar
entries retain their ordinary data meaning and are subject to normal final
schema validation; an entry used as a parent must resolve to a mapping so its
settings can combine with the child. Missing collections contribute no entries.
A present collection must have the mapping shape required by its declaration.
Identity comes from the collection key, not a copied field in the parent payload.
The child keeps its key. Application fields such as `name` or `id` have ordinary
combination semantics unless consumed as explicitly declared structural metadata.
### Source precedence first, inheritance second
First assemble the effective input under the existing source rules: strict
composition within tiers and recursive overlay across underlays, main input,
selected overlay, and explicit CLI overrides. Strict conflicts remain errors.
Then resolve inheritance against the resulting effective collection.
Every parent reference sees that parent's effective definition, including source
overrides. For each entry, resolve the parent's chain, then recursively overlay
the child's own effective payload. Each child obtains independent effective
settings; resolving a child leaves its parent and siblings unchanged.
Source precedence and ancestry answer different questions. Source precedence
selects each entry's own settings and parent reference. Ancestry supplies settings
that entry omits. For example, if a child's own timeout is `10`, changing its
parent's timeout to `30` through higher-priority source input leaves the child's
explicit `10` intact. To change that explicit child value, override the child.
This applies equally to CLI overrides at those respective paths.
### One combination contract
Parent-to-child combination uses the existing recursive mapping overlay:
- Omitted children retain inherited values.
- Present zero, false, and empty strings replace inherited values.
- Mappings combine recursively; an empty mapping retains inherited children.
- Sequences replace as a unit, including empty sequences.
- Explicit null replaces the inherited value, subject to final schema acceptance.
- Cross-kind replacement follows the same overlay rules and final schema checks.
The parent's inheritance-control metadata is not inherited as a payload setting.
The child supplies its own relationship. Inherited payload values can themselves
contain ordinary strings or fields with similar names without acquiring reference
semantics outside the declared position.
These rules intentionally differ from the request's suggested null-as-unspecified
and whole-parameter-map replacement. Consumers adopting the feature adopt
ConfigKit's shared rules. Multiple inheritance, sequence merging, per-field
strategies, deletion operators, cross-collection references, and named-overlay
file inheritance are outside this feature's initial scope.
## Loading and validation
The proposed lifecycle is: existing acquisition/composition and main-owned version
checking; finish source precedence assembly; resolve named inheritance; apply
typed defaults and decode; recorded application resolution; secret lookup;
semantic validation; publish the final configuration and metadata.
Resolve inheritance in the private input representation while omission is still
explicit. Ordinary and layered loading use the same implementation. Typed defaults
remain the final decoder's baseline and are not converted into parent declarations
or recursively copied from parent Go values. Domain resolution receives already
expanded settings and can apply recorded normalization/defaulting under its
existing contract.
Preserve the existing version owner's declaration and checking point. Inheritance
must not supply, replace, or remove the protected version path or bypass ancestor
protections. Identify attempted protected-path changes explicitly during design
of declarations rather than moving the checker to a different lifecycle point.
All declared relationships resolve during the load, including entries the
application might not select for execution. Missing parents, non-mapping parents,
invalid reference values, self-reference, and longer cycles fail the load with
safe structured diagnostics and relevant source locations. Traverse deterministically;
diagnostics should identify the relationship and chain rather than expose payload
values. Synthetic inputs retain labels and semantic paths without fabricated file
coordinates. Normal syntax and strict-composition errors remain unmaskable.
Partial base entries may omit domain-required settings. Omission is not by itself
a structural error. All returned entries still undergo ordinary typed decoding;
application validation determines which entries need to be independently executable
and which are reusable partial definitions. This adds no alternate schema,
automatic abstract-entry marker, or general exemption from validation.
Successful loading yields the completed configuration and metadata. Inheritance
performs no background resolution and retains no filesystem handles. Applications
may load again explicitly using the existing source-reuse contracts.
## Presence, provenance, and confidentiality
Consumers must distinguish direct child input, inherited input, typed defaults,
and recorded application-derived values. Explicit zero/false/empty/null settings
remain distinguishable from omission. The representation of inherited presence
in the existing query surface is an open decision below; Go zero-value inference
is not an acceptable substitute.
Each inherited value retains its originating source and source path, together
with the relationship that placed it at the child's destination path. Chains
remain explainable, including a child's explicit replacement of an inherited
value. Source-precedence history remains inspectable alongside inheritance
derivation; inheritance must not masquerade as another acquired file or source
tier. Effective metadata, resolver-entry queries, history, explanation, and
redacted rendering must agree.
A relative resource inherited from a parent keeps the context of the source
that supplied that resource value. It does not acquire the child's directory.
Composite origins remain composite and derived dependencies remain explicit.
Inherit secret references as unresolved configuration input; resolve typed
credentials only at the existing lifecycle point. Preserve source and destination
sensitivity through inheritance dependencies so copying a classified value cannot
make it public. A child's independent replacement has its own provenance; inherited
or displaced historical secrets remain safely explainable and redacted. No raw
credential values belong in inheritance trees, metadata, or diagnostics.
## Efficiency and ownership
Model each collection as entries and single-parent edges. Use indexed name lookup,
cycle detection, and memoized parent resolution to avoid repeated ancestor work.
Materialize independent public results while allowing carefully owned immutable
private structures to share work. Do not introduce global caches or reload sources
for individual entries.
Assess work against input size, relationship edges, materialized effective values,
and retained provenance/history. Many children can legitimately produce much more
output than the source declarations. Use representative chains, shared-base trees,
wide collections, and replacement cases to evaluate growth. Performance evidence
is not a new release timing gate.
Existing acquired-input accounting remains in force. The eventual implementation
plan must define how expanded output and retained derivation fit existing decoded
tree/history limits, including where expansion is checked. Avoid silently charging
one inherited value as a newly acquired source or inventing arbitrary entry/chain
caps. Any necessary new public limit requires an explicit design decision.
## Work needed from the baseline
The feature requires root-owned opt-in declarations, a private relationship graph
and expansion phase, and lifecycle integration before typed decoding. Presence,
derivation history, diagnostics, sensitivity propagation, and source-relative
resource origins must extend together. The public parser-independent facade
remains the only consumer boundary.
Representative examples and behavioral tests should establish source/inheritance
ordering, chains and shared parents, failure diagnostics, partial bases, explicit
values versus omission, unchanged sequence/null behavior, origin/resource
preservation, result independence, and safe secret inspection. Both loading entry
points and supported source representations should share those semantics.
A new ADR should record the selected inheritance architecture after the open
decisions are resolved. Architecture/deferred-scope documentation must recognize
the new structural responsibility; public API, configuration, consumer, and
internal documentation should describe implemented behavior when it lands.
Implementation sequencing belongs in a separate future plan. Existing callers
that do not opt in retain their behavior; an additive minor release is the target,
with release preparation and publication handled separately.
## Completion criteria
A consumer can declare a named collection, load mixed-source configuration once,
and receive resolved independent entries with one set of combination rules.
It can inspect whether each setting was direct, inherited, defaulted, or derived,
follow inherited resource origins, and validate final domain meaning. Missing or
cyclic relationships fail before a usable result is returned. The consumer needs
no separate graph resolver, generic merge pass, or provenance reconstruction.
## Open Decisions
### 1. Declaring collections and parent references
**Recommendation:** a root-owned declaration in `Definition` identifies a
collection path and one direct parent-reference field name within each entry.
Consume that field as structural metadata during expansion and retain its source
location for inspection. An omitted reference means no parent; explicit null
removes a lower-tier relationship. A string names an exact key in the effective
collection, including an empty string if such an entry exists. Other shapes are
reference errors. Outside the opted-in position, the field remains ordinary data.
This integrates naturally with existing source precedence and avoids a globally
reserved `extends` field or duplicated entry identifiers.
**Alternative:** declare a separate name-to-parent table at a designated input
path. Payload fields never overlap control metadata and application structs need
no inheritance field, but settings and relationships become separated and must
be kept consistent. It also introduces a second keyed collection into diagnostics
and source composition for each inheritance relationship set.
### 2. Meaning of inherited presence
**Recommendation:** an inherited value counts as present input for effective
`IsExplicit`/defaulting behavior, with separate derivation metadata distinguishing
it from direct child input. This treats inheritance as input expansion, keeps
typed and resolver defaults below inherited settings, and avoids consumers
reimplementing presence logic. The API must clearly explain that effective
presence differs from direct declaration at the destination path.
**Alternative:** inherited values remain non-explicit at the child path, with
a new effective-input-presence query/defaulting rule. This preserves a narrow
direct-declaration meaning of explicitness, but requires two presence concepts
through the resolver lifecycle and risks defaults overwriting inherited settings
unless callers adopt the new query consistently.
### 3. Nested or overlapping collection declarations
**Recommendation:** initially support multiple independent, non-overlapping
declared collection paths. Report overlapping declarations as an options error.
An outer inheritance expansion can copy or replace an inner collection, so
overlap otherwise needs a selected rule for whether inner inheritance resolves
before copying or after arrival at each destination. This restriction makes that
unresolved interaction explicit rather than allowing declaration order to choose
semantics. It does not restrict ordinary nested payload mappings.
**Alternative:** define destination-based outer-to-inner expansion, resolving each
inner collection after the enclosing entry is materialized. This supports nested
presets, but creates additional namespace, dependency, and provenance semantics
and can multiply graph instances. It is viable if a concrete consumer need
justifies that additional model; it should not emerge accidentally from traversal.

141
docs/roadmap/remediation.md
View File

@@ -1,141 +0,0 @@
# Filesystem Sources and Underlays Review Findings
Reviewed implementation: `cbf031a`, covering the implementation commits after
`27df838`. These findings concern the agreed source, precedence, and provenance
contracts in the [roadmap](sources-and-layering.md) and
[implementation plan](implementation.md). No implementation fixes are included
in this review.
## Resolution status
All three findings were addressed on 2026-09-29:
- Composition now fills sequence-descendant origins separately from atomic
conflict claims, so initial underlay attribution reaches every surviving node.
- History reconciliation checks contributing origins and recognizes ancestor
replacement events without duplicating their descendant contributions.
- Native contexts use `fs.ValidPath`; host contexts use `filepath.Localize`
before the existing canonicalization and containment checks.
`facade_underlay_origins_test.go` covers imported sequence descendants in first
and later tiers, resolver/result/explanation agreement, and later mapping
contributors. `TestResourceContextsPreserveLiteralBackslashes` covers native
and host resource reads and directory listing. The new tests reproduced all
three failures before the fixes. The findings below retain the original review
evidence and acceptance requirements.
Post-fix validation passed: `./scripts/check.sh` and
`GOPROXY=off go test -race ./...`. Existing sequence replacement, provenance,
history, and resource-containment tests remain passing.
## 1. Preserve underlay attribution for sequence descendants (P2)
**Owners:** `internal/engine/load_request.go:174`,
`internal/engine/compose.go:113`, and `internal/engine/load_result.go:39`.
The first nonempty underlay seeds the accumulator using a composed document.
Composition claims intentionally stop at sequences because sequences are atomic
for strict conflict checking. Consequently its origin index contains the sequence
itself but not sequence elements or nested fields inside them. Relabeling that
index as `LayerUnderlay` leaves those descendants with the decoder's default
`LayerBase` attribution. Later overlay insertion recursively collects origins,
so attribution also depends on whether the same input is the initial tier.
**Reproduction:** load `NoSource()` as main and an underlay mapping containing
`items: [{name: first}]`. The load succeeds. `Metadata.Lookup` reports
`LayerUnderlay` for `/items`, but `LayerBase` for `/items/0` and `/items/0/name`,
all referring to the same underlay source ID. `Metadata.History` repeats the
incorrect base attribution for the descendants.
**Required correction:** cover every surviving input node when assigning origins,
including descendants of atomic sequence values. Keep strict-composition claims
and sequence replacement semantics unchanged. Ensure resolver entry metadata,
returned metadata, and explanation agree with each acquired source's role.
**Regression coverage:** a first-tier sequence containing nested mappings,
including an imported contribution, retained beneath main input; compare with
equivalent later-tier insertion. Assert source ID and underlay role for elements
and their fields as well as the sequence container.
## 2. Preserve later same-role mapping contributions in history (P2)
**Owner:** `internal/engine/load_metadata.go:128` (`applyOverlayMetadata`).
The final-history reconciliation skips an effective origin when its layer and
operation match the last recorded event. Multiple underlay tiers share
`LayerUnderlay`, so those two fields do not establish that the recorded event
contains the effective source contributors. The comparison cannot distinguish
contributions from repeated tiers of the same role.
**Reproduction:** with no main document, supply three ordered underlays:
```yaml
# Tier 1
x: 1
# Tier 2
x: {}
# Tier 3
x: {y: 2}
```
The final `/x` origin correctly includes sources 2 and 3. Its metadata history
contains only source 1 and the replacement by source 2; source 3's contribution
is omitted. The returned effective provenance and chronological history disagree.
**Required correction:** determine whether the effective provenance is already
represented using contributing origins, not merely layer/operation equality.
Preserve meaningful same-tier-role contributions without duplicating events
already represented. Keep existing replacement, equal-value, and displaced-path
history behavior.
**Regression coverage:** scalar-to-mapping replacement followed by a recursive
mapping contribution from a later underlay; assert the final contributor appears
in history and that metadata and explanation describe consistent source origins.
## 3. Honor standard filesystem names in resource contexts (P2)
**Owner:** `internal/engine/resource_context.go:38` and `:77`.
Both resource adapters reject any name containing a backslash, beyond
`fs.ValidPath` validation. A backslash is a legal literal filename character in
an `io/fs` namespace and on the Linux host filesystem. Native source acquisition
accepts such paths, but resource-context access rejects them. This introduces an
unrequested path restriction and applies host-style separator assumptions to a
generic filesystem.
**Reproduction:** supply `fstest.MapFS` with `config.yml` and a resource named
`asset\name`. Loading `config.yml` succeeds; the resource name passes
`fs.ValidPath` and can be read directly from the supplied filesystem.
The equivalent read through the returned context fails with `fs.ErrInvalid`:
```go
context, ok := result.SourceContext(sourceID) // ok is true
_, err := fs.ReadFile(context.FS, name) // rejects the valid name
```
**Required correction:** use standard slash-path validation for native contexts
and preserve literal characters accepted by the supplied filesystem. For host
contexts, use platform-appropriate conversion while preserving the existing
contained-root behavior; native namespace semantics must not inherit Windows
separator restrictions. This is not a request to broaden root traversal or
change symlink guarantees.
**Regression coverage:** access a literal-backslash resource through both native
Open/ReadDir paths where applicable and a Linux host context; retain invalid-path
and outside-root coverage. Avoid rejecting valid native names on other platforms
merely because the corresponding host path would be invalid there.
## Review validation
- `./scripts/check.sh` passed (tests, vet, build, formatting, and whitespace).
- `GOPROXY=off go test -race ./...` passed.
- Broad-catalog and many-small-tier benchmarks ran with `-benchmem` and 50 ms
samples. Allocated bytes scaled approximately with tested input sizes; these
measurements are review evidence, not timing gates.
- A temporary external Go program reproduced all three findings through the
public API; no diagnostic/test code was added to the repository.
The intended facade, shared synchronous lifecycle, strict within-tier composition,
recursive precedence, mapping snapshotting, and passive resource-context design
are present. The findings above are correctness gaps within that design, not a
recommendation to expand its scope or change its selected combination semantics.

457
docs/roadmap/sources-and-layering.md
View File

@@ -1,457 +0,0 @@
# Filesystem Sources and Source-Backed Underlays
Status: implemented; draft ADRs remain pending review. This document preserves
the design target and rationale, including its historical baseline. The
[configuration contract](../config.md) owns current behavior, and the
[implementation plan](implementation.md) records sequential work and validation.
## Purpose and priorities
Address the filesystem and layered-input needs in the
[PromptKit request](feature-request-promptkit.md) as one coherent extension:
native `fs.FS` sources and explicit source-backed underlays, including
programmatic input. ConfigKit's recursive combination model takes precedence
over PromptKit's requested whole-definition replacement. Neither capability
should require temporary files, a generated root YAML document, or serialization
through YAML merely to transfer already available values.
Design priorities, in order:
1. Clear, standardized behavior consistent with ConfigKit's existing conventions.
PromptKit may adapt its formats and APIs; its historical conventions are not
new ConfigKit compatibility obligations.
2. Efficient loading and retained state, aiming for linear or near-linear work
in actual input and output size. Prefer clarity and maintainability where a
more elaborate optimization would compromise the first priority.
3. A small public surface with cohesive private implementation, usable by simple
applications without requiring them to learn the layered-loading machinery.
Consumers load once and receive a complete configuration and its inspection
metadata. They should not need a second generic merge/defaulting pass, a separate
presence tracker, or reconstructed provenance. Domain-specific resolution remains
consumer-owned, but fits into the recorded loading lifecycle before the result
is validated and returned. Subject to consistent semantics and correct presence
and provenance, minimizing consumer integration code is an explicit design goal.
The [named-inheritance request](feature-request-promptkit-inheritance.md) remains
separate and optional. This roadmap neither implements it nor adds speculative
infrastructure for it.
## Planning baseline (historical)
Planning baseline: `ec26591b275bd74f89120bdea69c5f6459dfcafb`, following v0.4.2.
Current behavior is owned by the [configuration contract](../config.md), not by
these proposals.
- `Load[T]` takes one `File`, `Bytes`, `Reader`, or `NoSource` input. Imports
strictly compose documents; one named overlay uses a root-relative directory
convention; explicit CLI overrides follow it.
- Filesystem context uses host directories and an `os.Root`-based implementation.
Byte and reader labels do not imply filesystem paths. There is no native
`fs.FS` source or public batch of heterogeneous layers.
- Composition rejects non-mapping overlaps, including equal values. Overlay
recursively merges mappings and replaces other values. Empty mappings do not
clear existing children; null is an explicit value, not omission.
- An explicitly configured schema-version path currently belongs to the selected
source. Its checker runs before overlay/CLI input, which cannot alter that path;
imported fragments cannot contribute it. This roadmap preserves those ownership
and timing rules.
- CLI overrides already convert supported programmatic scalars, lists, and
string-keyed maps directly to the private document representation. They have
CLI-specific paths, labels, restrictions, and origins, not a general source API.
- Source records and resolution queries expose acquired-source identity and
provenance, but do not provide an `fs.FS` resource context.
- Root-owned public wrappers delegate to `internal/engine`; parser-independent
trees belong to `internal/document`, and YAML parsing stays in `internal/yaml`.
- Current mapping assembly uses repeated linear key searches. Overlay application
clones the accumulated document and provenance; subtree-origin removal scans
the whole origin index. Straightforward repetition across many layers would
amplify these costs.
Existing source/lifecycle contracts, ownership, secret handling, and diagnostics
remain the compatibility baseline. This work is not permission to silently
reinterpret current `Load`, imports, selected overlays, or CLI overrides.
Redesigning schema-version policy is outside this roadmap's scope.
## Target source model
The extension preserves the
[synchronous, complete-on-return lifecycle](../config.md#completion-and-resource-lifetime).
Underlay assembly and the final configuration/metadata snapshot complete within
the load call. Resource-context retention is passive; resource access and any
subsequent loading require explicit application calls. The proposed architectural
rationale is recorded in [ADR 0008](../adr/0008-complete-on-return-loading.md).
The handle-ownership requirements below apply to the new source/context APIs as
they are implemented.
### Native filesystem input
A caller can supply a filesystem, an explicit file location within it, and a
meaningful diagnostic label/context. `embed.FS` is an ordinary supported example,
not a separate parsing mode. Use standard slash-separated `io/fs` path conventions;
do not apply host drive-letter or separator rules to virtual paths.
The source retains the filesystem context needed to acquire its imports, glob
matches, and selected overlay. Relative imports resolve from the importing
document; named overlay lookup remains relative to that configuration context's
root. References must never fall back to the host filesystem simply because a
virtual path is missing. Imports/globs retain their explicit opt-in, deterministic
ordering, composition semantics, and actionable errors.
Filesystem namespace identity and a normalized path identify a resource; labels
are presentation, not uniqueness keys. Identical names in different filesystems
must not collide. Use a reusable load-local context identity, not comparison or
hashing of arbitrary `fs.FS` interface values. Repeated-import and cycle checks
operate within the relevant input composition graph; selecting explicit layers
is distinct from accidentally importing a file twice.
Keep the host-file backend's established canonicalization and containment behavior.
For generic `fs.FS`, describe namespace-relative resolution and the supplied
filesystem's actual behavior. `fs.FS` does not universally provide symlink
canonicalization or a security sandbox; do not invent such guarantees or reject
all implementations that cannot provide host-specific operations. The caller
chooses the filesystem, its acquisition behavior, and its lifetime. ConfigKit
closes handles it opens, not a caller-owned filesystem as a whole.
### Origins and downstream resources
Locations remain meaningful for host files, filesystem-relative files, labeled
bytes, and programmatic values. Preserve deterministic source IDs and traversal
order while distinguishing source identity, input kind, and precedence role.
Do not invent host filenames or line/column coordinates for synthetic inputs.
A consumer can follow an effective field's origin to its contributing source and
obtain the context needed to resolve a relative resource. For a field inherited
through layering, that is the contributing source's context, not simply the last
layer's directory. Composite origins remain composite; ConfigKit must not choose
one contributing directory arbitrarily.
Resource-context access is explicit and separate from serializable diagnostic
records. Querying metadata itself does not read resources. PromptKit owns opening
and interpreting message text, rendering, and handling resource errors. Context
retention must not imply that external resources are snapshotted with the loaded
configuration or remain valid after the caller closes their filesystem.
The context contract covers caller-supplied filesystems and host-file sources.
ConfigKit closes every library-owned filesystem handle before loading returns,
on success or failure, including input files and directory/root handles. Returned
configuration, provenance, and resource-context records retain no such handles
and require no filesystem cleanup by the consumer. Configuration and provenance
remain usable independently of subsequent filesystem access.
Caller-supplied filesystems and readers remain caller-owned. A supplied filesystem
may itself depend on handles whose lifetime the caller manages; retaining its
context does not extend that lifetime or transfer ownership to ConfigKit.
Host-file resource contexts retain the information needed to open resources later,
preserving the host backend's established containment semantics. Later resource
access acquires its own handles: helpers close handles they use internally, and
callers close any resource handles explicitly returned to them. Host contexts
use a passive path-backed filesystem adapter that opens a fresh contained root
for each resource open and releases it with the returned file. Resource
opens observe the backing filesystem at access time and can fail independently
of an earlier successful configuration load. Sources without filesystem context
report its absence explicitly; composite origins expose their contributing
contexts rather than choosing a directory. These rules keep configuration
snapshot ownership separate from resource-access lifetime.
## Target layering model
### Explicit operations, not discovery order
Extend public loading with explicitly supplied lower-precedence configuration
sources, using the same typed definition/result lifecycle as ordinary `Load`.
Support native filesystem, host-file, byte, reader, and programmatic inputs.
The central workflow is bundled defaults overridden by application configuration,
not an unrestricted sequence of user-selectable merge algorithms. Underlays can
form explicitly ordered default tiers; ConfigKit does not discover priorities or
know about PromptKit's catalog tiers. Source kind does not confer priority:
programmatic data can supply the appropriate supported input role without being
misrepresented as CLI flags.
The precedence model, from lowest to highest, is:
**Typed application defaults < source underlays < strictly composed main
configuration < selected overlay < explicit CLI overrides.**
Typed defaults remain the existing Go-value baseline. Source underlays add
configuration-backed defaults with imports, filesystem context, and source
provenance; they are not a replacement for the typed-defaults facility.
An underlay tier is an explicitly supplied group of source contributions. Each
group is strictly composed, including enabled imports, before it participates
in precedence assembly. Consumers supply zero or more tiers in lowest-to-highest
precedence order; later tiers overlay earlier tiers recursively. A source's
position within a group does not authorize replacement. The main input is
optional: an underlay-only load uses the same lifecycle, and a load with neither
main input nor underlays retains defaults-only behavior with supported explicit
overrides. Selecting a named overlay still requires its explicit lookup context.
The public representation exposes grouping and ordering directly. Empty groups
contribute nothing; explicit defaults-only sources also contribute no document.
The selected operation determines how values combine, not their source format
or loading entry point. Equivalent file, embedded, byte/reader, and programmatic
input uses the same presence and combination rules. Strictly composing independent
contributions and intentionally overlaying precedence tiers express different
intentions; merely changing the input medium must not switch between them.
This applies to both ordinary `Load` and
the new layered-loading interface wherever they offer the same operation.
Consistency applies to combination, presence, and origin meaning. Source-specific
acquisition and lifetime rules remain explicit: a reader is single-consumption,
bytes are snapshotted, and filesystem input can observe backing-store changes.
CLI records retain their existing path restrictions. Source equivalence does not
promise identical reuse, locations, resource access, or diagnostic labels.
The existing opt-in version-path ownership rule remains an explicit metadata
constraint, independent of whether its owning source is a file or programmatic
input; it is not a source-format-dependent merge rule.
Keep strict composition and intentional overriding visibly distinct:
| Operation | Desired semantics |
| --- | --- |
| Compose independent contributions | Existing strict composition: recurse through mappings and report non-mapping conflicts with relevant source locations. |
| Overlay settings | Existing recursive mapping overlay: omitted children retain lower input; scalar, sequence, and null values replace. |
Underlay is a precedence role, not another combination algorithm. Assemble the
main document and all its imports by strict composition before applying that
result over the underlays using recursive overlay. A conflicting main/import
definition still fails, even if it would overwrite an underlay or be overwritten
by the selected overlay. Never flatten main and imported documents into a
last-wins list.
An underlay source may bring its explicitly enabled imports and source context.
Independent contributions within an underlay tier compose strictly; intentional
precedence between explicitly ordered tiers uses recursive overlay. Discovery
order does not establish tiers or authorize overriding. Syntax, source, and
composition failures cannot be hidden by higher-priority input.
There is no whole-definition replacement mode, per-layer recursive/atomic switch,
collection-specific identity matching, implicit sequence merge, profile registry,
or merge callback in this feature. Existing atomic scalar, sequence, null, and
cross-kind replacement under overlay remains unchanged.
Preserve the main configuration's version check after its strict composition and
schema construction, before precedence assembly with underlays, selected overlay,
and CLI contributions. Assemble the effective input before typed defaults/decoding,
application resolution, secret resolution, semantic validation, and final
snapshotting. Each lifecycle phase runs once per load, not once per layer.
Higher layers may replace values that would not fit the final typed schema;
effective typed validation is not validation of each intermediate snapshot.
Input syntax, source access, and composition remain validated at acquisition.
Recorded application resolution follows input assembly and decoding. Named-overlay
lookup has an unambiguous filesystem context, separate from version ownership: ordinary
loading retains its source/root convention; a multi-context load explicitly
identifies the lookup context when that convenience is used. Other sources retain
their own import and resource contexts. Callers do not manufacture a primary
document merely to combine layers, and no overlay lookup context is needed when
no named overlay is selected.
Higher-priority mappings retain omitted lower-input children. Empty mappings do
not clear those children; sequences, including empty sequences, replace as a
unit, and null remains explicit subject to the final schema. Typed defaults
continue to fill fields absent from the assembled input under existing rules.
No reset/delete operator is added.
For example, a bundled profile containing `{model: example-model, timeout: 30}`
combined with application input `{model: another-model}` retains `timeout: 30`
from the bundle. Its model origin points to the application and its timeout
origin points to the underlay. PromptKit must validate that final profile rather
than assume the application entry discarded all bundled settings. This knowingly
differs from its requested identity-based whole-definition replacement. Consumer
examples should adopt the shared semantics, not implement a hidden replacement
pass after loading. A replacement operation could be considered separately if
future concrete needs justify it, but is not an open choice in this roadmap.
### Preserve existing schema-version semantics
An application opts into version checking by configuring a version path. A key
spelled `version` has no automatic special status. When checking is configured,
one designated input owns that declaration; fragments cannot contribute it and
underlays, overlays, or CLI overrides cannot supply or alter it, even to an equal
value. Preserve the current version-path and ancestor protections when integrating
the new input role.
Keep the checker at its existing lifecycle point: after base composition and
schema construction, before underlay precedence assembly, selected overlays,
CLI overrides, defaults, and typed decoding. Preserve supported scalar forms,
absent/null distinction,
diagnostics, failure propagation, and version-path integer-to-string decoding.
An absent declaration remains absent for the checker; an invalid version type
fails without invoking it. Later input cannot mask a version-check failure.
Filesystem sources and layered loading do not require a different version
policy. The designated main input owns the version when a `VersionRule` is
configured, extending the existing selected-source role. It may use any supported
source kind. With no main input, the checker receives an absent declaration,
consistent with existing `NoSource` behavior; underlays cannot supply that
declaration. Without version checking, no version owner or synthetic primary
document is required.
Named-overlay lookup context is a separate concern, not a reason to impose
version ownership on unversioned loads.
Consumers continue to define version meaning and acceptance. Effective-version
last-wins validation, independent per-source version checks, schema selection,
and migration are not added by this feature. A future concrete need can motivate
a separate version-policy proposal; preserving the current contract here does
not settle that broader question permanently.
### Programmatic presence
Programmatic input has a non-secret source label and uses explicit structure to
represent presence. Omitted mapping keys are absent; present zero, false, empty
string, empty collection, and nil/null values remain explicit. Use ConfigKit's
existing canonical scalar and collection conventions, not YAML round trips or
Go zero-value inference. The initial representation is a labeled string-keyed
mapping using the scalar/list/map value set supported by explicit overrides.
This reuses established conventions and keeps presence directly expressible.
The source snapshots the mapping at construction, like byte sources. A nil root
mapping represents null and fails the existing mapping-root requirement; an empty
non-nil mapping is a valid empty document.
Do not label generic programmatic sources as CLI input or misuse synthetic flag
names to fit the existing override API. Preserve parser-independent conversion,
safe errors, deterministic map ordering, and existing secret-reference semantics;
this is not a new route for embedding resolved credentials in document trees.
Caller-owned mutable input is detached at a clearly documented ingestion
boundary. Results and inspection remain independently owned under existing
contracts. This does not promise language-enforced immutability of Go values or
an immutable filesystem.
A final Go value alone need not encode presence: an omitted boolean and an
explicit `false` can both decode to `false`. The returned configuration together
with its presence/provenance metadata preserves that distinction through
defaulting, layering, and recorded resolution. Null, empty values, zero, and
absence must not be collapsed by source conversion or merging.
### Provenance and operational boundaries
Origins, presence, history, explanation, and effective rendering agree with the
final typed result. Overridden and replaced definitions remain explainable without
retaining stale effective descendants. Replacements identify their scope and
displaced sources; equal-value explicit overrides are still meaningful history.
Include typed defaults and application-derived values, not only source-supplied
values, in final-value provenance. Retain redaction and callback-lifetime behavior.
Apply existing resource-accounting principles across the whole load rather than
resetting allowances for every layer. Audit how bytes, programmatic nodes,
accepted source occurrences, and glob work are counted without accidental double
charging. Do not add arbitrary new budgets or infer that efficiency work waives
existing ones; any needed adjustment must be explicit in the later design.
## Efficiency target
Measure cost against total acquired bytes/nodes, reference and path data, final
materialized values, and retained provenance/history. Layer count alone is not
a useful size measure. Large history or output can legitimately dominate cost.
Aim for O(N + H + R) processing and storage where N is total input/visited-path
work, H is retained history/provenance, and R is produced result data. Account
separately for deterministic sorting, filesystem traversal, schema processing,
and caller callbacks. This is an engineering target, not an unconditional public
complexity guarantee or a fixed performance gate.
The target implementation:
- Acquires/parses each accepted source occurrence once and directly converts
programmatic input. It does not decode or serialize accumulated configurations
between layers. Respect single-consumption readers and mutable file sources;
no implicit cross-load cache is needed.
- Uses indexed lookup alongside ordered mapping storage, retaining deterministic
iteration and diagnostics without repeatedly scanning growing mappings.
- Builds one load-owned working assembly instead of cloning the accumulated
configuration and history for every added layer. Detach at meaningful ownership
boundaries; safely share immutable private history where appropriate.
- Updates affected subtrees and their provenance rather than scanning all fields
for every replacement. Avoid repeatedly copying ever-growing ancestor-origin
lists or rechecking the same conflict group for every member.
- Reuses one schema/lifecycle evaluation per load and freezes the final result
once. It does not require global caches, copy-on-write public APIs, or a general
persistent-tree framework merely to meet an asymptotic target.
Use representative benchmarks and allocation measurements for broad keyed
catalogs, many small underlay tiers, repeated scalar/sequence replacement,
imported filesystem defaults, and retained history. Compare growth as inputs
scale, including small common
loads. Favor changes addressing demonstrated scaling problems; preserve stable
diagnostic order and ownership rather than trading them for a superficial O(n)
claim. No timing threshold or additional release gate is requested.
## Work needed from the baseline
The target requires changes in the following responsibility areas; the linked
implementation plan owns their execution order and concrete validation:
- Extend the root-owned source/context facade and private acquisition boundary
for `fs.FS`; adapt import, glob, overlay, identity, and resource-context handling
without exporting parser or engine types.
- Add source-backed underlays and programmatic input, sharing one engine path
with ordinary loading rather than building a parallel configuration system.
- Extend source records, histories, diagnostics, and rendering consistently
across input kinds.
- Integrate version-owner selection into layered loading while preserving the
existing version lifecycle and protections. Keep filesystem-context selection
independent of version checking.
- Refactor mapping assembly and provenance updates where current algorithms
would scale poorly under layered workloads. Keep optimization scope tied to
the new workflow rather than rewriting unrelated library subsystems.
- Add focused behavioral coverage for mixed sources, relative resources,
namespace collisions, import failures, composition conflicts, explicit empty
and null values, caller/result independence, and source-correct redaction and
explanation. Add comparative benchmarks without making performance a brittle
pass/fail contract.
- Compare equivalent logical inputs across supported source kinds and loading
interfaces. Assert consistent values, presence, operation outcomes, and origin
meaning while allowing genuine location/label differences. Cover defaulted and
derived values, recursively retained underlay fields, unchanged atomic-value
replacement, typed defaults, strict main/import conflicts, and preservation of
version ownership/check timing across both loading interfaces. Verify that
unversioned layered loads need no synthetic version-owning root.
- Provide maintained simple and PromptKit-shaped examples: embedded defaults,
operator configuration from another context, and labeled programmatic input.
Show explicitly that a same-name profile recursively combines, rather than
replacing its whole lower definition.
Demonstrate exact operations and resource origins without provider calls or
recreating PromptKit itself. The consumer supplies sources, explicit operations,
types, and domain rules; it does not implement generic merging, presence
tracking, or origin reconstruction after loading.
- Update Go declaration documentation, the configuration contract, consumer
guidance, component overview, and relevant architecture/deferred-scope records
as behavior lands. Record significant boundary changes in an ADR without
rewriting accepted ADR history. Keep named inheritance explicitly separate.
The capability is additive and intended for a minor release, preserving current
version semantics. Release preparation and publication remain separate tasks.
## Completion criteria
A consumer can load embedded configuration with relative imports, combine it
with independently located operator configuration and explicit programmatic
values, and obtain one typed result with accurate presence, sources, history,
and resource context. No temporary configuration files or serialization round
trip or subsequent generic configuration processing is needed. Simple `Load`
usage retains its current behavior, including version ownership and validation
timing. Layered loading uses the same version policy. Final-value provenance covers input,
defaults, and recorded domain resolution, and metadata distinguishes explicit
zero/false from absence even when the typed values are equal. Benchmarks
demonstrate that adding inputs does not entail avoidable full-collection copying
or quadratic key lookup. Public contracts and examples clearly distinguish
strict within-tier composition, recursive cross-tier overlay, and downstream domain
resolution. Equivalent-input tests and the maintained consumer example establish
semantic consistency and reduced integration code, not merely API availability.
## Selected resource-context interface
A separate source-ID-based query exposes standard filesystem access and a
source-relative base location through root-owned API types. Ordinary source
records remain descriptive. The query reports absent context explicitly and
supports imported sources as well as initial inputs. Consumers follow each
contributing origin independently for composite values. This avoids a mandatory
downstream filesystem registry and preserves provenance through recursive
combination.
Caller-supplied filesystems are borrowed capabilities; host contexts are passive
path-backed adapters. Neither is a cloned or immutable filesystem. Queries perform
no I/O, and later resource opens follow the ownership rules above. Configuration
results retain no library-owned open handles.