Add explicit path import traversal

This commit is contained in:
2026-09-06 20:41:40 +00:00
parent 3257b2124c
commit 9f04c09be3
15 changed files with 1233 additions and 118 deletions

View File

@@ -20,10 +20,11 @@ The zero `Source` is invalid. Constructor argument failures are retained inside
opaque source and become safe structured diagnostics when the source is consumed.
File sources can observe file changes between consumptions; byte sources cannot.
Each source is limited to 8 MiB. A load will be limited to 64 MiB across all source
and override input when those layers are implemented. Reads request at most the
remaining allowance plus one byte so excess data is rejected rather than silently
truncated. Data returned together with a read error is not accepted as partial input.
Each source is limited to 8 MiB. An imported load is limited to 64 MiB across all
accepted sources, 1,024 accepted sources, 64 active import edges, and 1,000,000
input nodes in aggregate. Reads request at most the remaining allowance plus one byte
so excess data is rejected rather than silently truncated. Data returned together
with a read error is not accepted as partial input.
## Single-source loading
@@ -36,21 +37,22 @@ schema-version checking. Its result is detached before input is applied.
The current lifecycle is:
```text
validate options → parse one source → reject reserved directives → build schema
→ check the optional application version → create and clone defaults
validate options → parse the selected source → traverse/compose enabled imports
→ build schema → check the optional application version → create and clone defaults
→ strictly decode → recorded application resolution → semantic validation
→ snapshot configuration and metadata
```
`File`, `Bytes`, and `Reader` each contribute one base source with the deterministic
ID `source-1`. `NoSource` skips parsing, supplies an absent version to a configured
version checker, contributes no source, and continues through defaults, decode, and
projection. A missing file remains an error; it is never interpreted as no input.
`File`, `Bytes`, and `Reader` each contribute the first base source with deterministic
ID `source-1`; enabled imports continue in depth-first order as `source-2` onward.
`NoSource` skips parsing, supplies an absent version to a configured version checker,
contributes no source, and continues through defaults, decode, and projection. A
missing file remains an error; it is never interpreted as no input.
The current `Options` boundary accepts a defaults factory, `VersionRule`, resolver,
and semantic validator. Imports, active profiles, filesystem context, overrides,
and dynamic secret classification return an explicit
`options.unavailable` error because their lifecycle stages are not implemented.
semantic validator, explicit path imports, and import filesystem context. Active
profiles, standalone filesystem context, overrides, and dynamic secret classification
return `options.unavailable` because their lifecycle stages are not implemented.
The reserved profile spelling `default` selects no active profile. `NoSource`
additionally rejects imports, active profiles, and filesystem context as invalid
combinations. Input containing a reserved `_configkit` mapping key is rejected while
@@ -152,10 +154,8 @@ live services as a substitute for configuration validation.
## Strict document composition
ConfigKit's private document layer can strictly compose parsed base documents. The
operation is available for file-graph integration but is not yet reachable through
the public single-source `Load` lifecycle. Application defaults do not participate
in this operation.
ConfigKit strictly composes parsed base documents when `Options.Imports` is true.
Application defaults do not participate in this operation.
| Existing node | Incoming node | Result |
| --- | --- | --- |
@@ -180,6 +180,44 @@ traversal order. Traversal order never authorizes replacement. Composition clone
the resulting tree and its ordered claim records rather than mutating or aliasing an
input document.
## Explicit path imports
Imports are opt-in. With `Options.Imports` enabled, a document may reserve its root
`_configkit` mapping for an `imports` sequence. ConfigKit removes that mapping before
application decoding. Each sequence item is a mapping with exactly one nonempty
string `path` or `glob` plus an optional boolean `optional`; all other shapes and
fields are `import.invalid`. `glob` is recognized but temporarily returns
`import.unsupported` until bounded glob expansion is implemented. A `_configkit`
key below the root is invalid. With imports disabled, any occurrence remains
`import.disabled`.
Path declarations use slash-separated relative paths. Absolute paths and network
URLs are invalid. Each path resolves from its declaring file's canonical directory,
and files are traversed depth-first in declaration order: a file contributes its own
body before its imports. Order stabilizes source records and diagnostics but never
permits replacement. Required missing paths fail; `optional: true` suppresses only
target nonexistence, not invalid paths, containment failures, directories,
permissions, parse errors, or composition errors.
The configuration root defaults to a selected file's canonical directory. A file
source may supply `FilesystemContext.Root` to widen that root but may not supply
`BaseDir`. Byte and reader sources must supply both `BaseDir` and `Root` as absolute
existing directories when imports are enabled; their diagnostic labels, including
URI-looking labels, never imply filesystem paths. The base and selected file must be
inside the canonical root. ConfigKit canonicalizes symlinks, checks containment with
relative paths, and opens the canonical root-relative target through a library-owned
root handle. Contained relative or absolute symlink targets are accepted; escapes
are rejected. Canonical paths define identity, so hard links at distinct paths remain
distinct sources.
An active-stack repeat is `import.cycle` and reports the complete cycle's import
sites. A previously visited canonical path is `import.repeated` and reports both
inclusion routes, including diamonds and aliases. Each accepted file is read once.
Imported fragments may contain their own root directives, but may not claim the
application's schema-version path or replace one of its ancestors; the version must
come from the selected source. No directory is scanned implicitly, including one
named `conf.d`, and ConfigKit performs no remote fetching.
## YAML documents
The private YAML boundary currently accepts exactly one nonempty document whose root

View File

@@ -2,8 +2,9 @@
This is the contributor entry point for ConfigKit. Read the task-specific owners
below before making changes. The repository is in early implementation: its public
loader currently covers one strict source, typed defaults, schema-version checks,
recorded application resolution, semantic validation, and metadata. The
loader currently covers strict selected sources, explicit path-import graphs,
composition, typed defaults, schema-version checks, recorded application resolution,
semantic validation, and metadata. The
[component overview](internal/overview.md) owns the implementation inventory.
## Initial orientation

View File

@@ -112,6 +112,18 @@ Assigned to explicit import traversal and glob handling:
traversal, canonical containment, cycle/diamond routes, and fixed source IDs.
Canonical paths—not `os.SameFile`—define identity, so hard links remain distinct.
Path-import implementation record (2026-09-06): `splitImportDirectives`,
`parseImportDeclaration`, `importFilesystem`, `importTraversal.visitPath`, and
`loadImportedDocuments` in `imports.go` adapt the assigned envelope, loading, source,
and failure mechanics. The corresponding ConfigKit tests adapt the assigned
declaration-order, disjoint composition, keyed-entry, all-source conflict, unsafe
target, alias, imported parse/schema, and relative-resolution cases. ConfigKit adds
recursive bounded DFS, `os.OpenRoot` confinement, canonical symlink identity,
cycle/diamond route locations, optional-path semantics, aggregate budgets, and
fragment version-placement enforcement. The planned glob syntax is parsed but
remains explicitly unavailable for its next owning work; hard links intentionally
replace Narratio's same-file rejection with path identity.
### Profiles and overlay
Assigned to selected-profile overlay:

View File

@@ -20,18 +20,18 @@ module and package documentation boundary. Its implemented public primitives are
- strict schema-driven decoding over detached defaults in `decode.go`, including
mapping/sequence semantics, null and numeric policy, safe codec errors, presence,
baseline/final trees, and initial default/input origins; and
- public single-source lifecycle coordination in `load.go`, version-report contexts
in `validation.go`, and detached base/default metadata assembly in
`load_metadata.go`; and
- public lifecycle coordination in `load.go`, bounded canonical path-import DFS in
`imports.go`, version-report contexts in `validation.go`, and detached
base/default metadata assembly in `load_metadata.go`; and
- recorded application defaulting and normalization in `resolution.go`, including
explicitness queries, helper verification, structural mutation audit, coalesced
intent, private before/after snapshots, and resolution metadata integration; and
- post-resolution schema-aware report collection and validator mutation detection in
`validation.go`, completing the public single-source callback lifecycle.
`validation.go`, completing the public callback lifecycle after composed input.
The [configuration contract](../config.md) owns implemented input, schema, decoding,
single-source lifecycle, version checking, recorded resolution, semantic validation,
metadata, and ownership behavior.
loading lifecycle, path imports, composition, version checking, recorded resolution,
semantic validation, metadata, and ownership behavior.
The source-extraction inventory records the application foundations assigned to
future implementation work:

View File

@@ -850,7 +850,38 @@ Acceptance:
## Stage 11 — Implement explicit file-import traversal
Status: pending.
Status: complete.
Completion evidence (2026-09-06):
- Added strict recursive `_configkit.imports` parsing with exactly one `path` or
recognized `glob`, optional booleans, unknown-shape rejection, root-only placement,
and detached directive removal. Glob declarations return the deliberate temporary
`import.unsupported` gate for the next owning work.
- Implemented declaration-order depth-first path traversal with canonical symlink
identity, relative containment checks, and canonical root-relative access through a
ConfigKit-owned `os.Root`. File inputs infer their canonical base/root; labeled
byte and reader inputs require explicit absolute base/root context and never infer
paths from URI-like labels.
- Kept active and visited indexes separate, reporting complete cycle import sites and
both diamond/alias inclusion routes. Canonical aliases repeat, while hard links at
distinct paths remain separate sources. Required missing paths, directories,
unreadable targets, escapes, and canonicalization failures remain errors; optional
suppresses only nonexistence.
- Assigned deterministic source IDs while reading each accepted source once, enforced
per-source and aggregate bytes, aggregate input nodes, source count, and active-edge
depth, and retained import-site/source records on failures. Imported fragments are
rejected when they claim the application version path or a non-mapping ancestor,
before strict composition.
- Wired composed input and multi-location origins through `Load`, version checking,
strict decoding, resolution, validation, and base metadata. Temporary-directory
tests cover nested/list-order traversal, composition conflicts, version placement,
directives, optional/required paths, explicit roots, cycles, diamonds, symlinks,
hard links, parse/decode attribution, file types, context rules, and reduced limit
boundaries. The [current path-import contract](../config.md#explicit-path-imports)
owns implemented behavior; glob expansion remains pending. Verified `go test ./...`,
`go test -race ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and
`git diff --check` with Go 1.26.7 on linux/amd64.
Read: feature [import graph](initial.md#explicit-imports-and-source-identity).

View File

@@ -426,14 +426,14 @@ configuration is never returned as a successful, usable result.
Source ownership, the supported YAML subset, scalar grammar, fixed input limits,
and safe failure behavior are implemented. The [current configuration input
contract](../config.md) now owns those definitions, including typed scalar
conversion through the current single-source lifecycle.
conversion through the current loading lifecycle.
## Paths and presence
Semantic path construction, JSON Pointer parsing, and base/default metadata
population are implemented. Their current contracts are owned by the Go
documentation in [`path.go`](../../path.go), [`metadata.go`](../../metadata.go),
and the [single-source configuration contract](../config.md#base-default-metadata).
and the [current configuration contract](../config.md#base-default-metadata).
Layered replacement and resolution behavior below remains target lifecycle work.
Distinguish absent, explicitly null, explicitly zero/false/empty, and other
@@ -475,62 +475,30 @@ the surviving sequence, not an attempted correspondence with previous indices.
## Explicit imports and source identity
Imports are opt-in through the loading API. Reserve the root `_configkit` mapping
for library directives and remove it before application decoding. Initially its
only permitted field is `imports`, a sequence of declarations. Illustrative
fragment:
Explicit path-import traversal, source identity, containment, budgets, recursive
directives, and the temporary glob gate are implemented and owned by the
[current configuration contract](../config.md#explicit-path-imports). Bounded glob
expansion remains target behavior. Illustrative future fragment:
```yaml
_configkit:
imports:
- path: fragments/server.yml
- glob: fragments/outputs/*.yml
optional: true
```
Each declaration has exactly one nonempty `path` or `glob`; `optional` is a bool
and defaults to false. Unknown directive fields are errors. When imports are
disabled, the presence of `_configkit` is an error, not an ignored directive.
- Relative paths resolve from the declaring file's directory. Absolute import
paths and network URLs are rejected.
- Expand declarations in list order, sorting each glob's matches
lexicographically by slash-separated relative path, with no locale sorting.
Use the slash-based [Go `path.Match` grammar](https://pkg.go.dev/path#Match)
(`*`, `?`, character classes, and backslash escapes), with recursive `**`
syntax forbidden. Backslashes escape pattern characters, not directory separators.
- Traverse depth-first: process a file's own contribution, then its declarations
in order. Ordering stabilizes source lists and diagnostics; it never permits
duplicate definitions to override one another.
- A required missing path or unmatched glob is an error. Optional suppresses
only a nonexistent target or zero matches. Invalid patterns, permission
- A required unmatched glob is an error. Optional suppresses only zero matches.
Invalid patterns, permission
failures, directories, and invalid matched documents remain errors.
- The configuration root defaults to the main file's directory; callers may
explicitly supply another root. Resolve symlinks and cleaned absolute paths
for identity and containment. Imports and profiles must stay inside that
canonical root. Retain declared paths and import-site locations for messages.
Relative and absolute symlink targets are permitted when their canonical target
is contained; the spelling of a symlink target alone does not determine containment.
- An active-stack repeat is a cycle; report the complete cycle and import sites.
A previously visited canonical file outside the active stack is a repeated
import error, including diamond graphs and overlapping glob matches. Report
both inclusion routes. Hard links at distinct canonical paths remain distinct
sources; this is path identity, not inode deduplication.
- Read each accepted source once per load. Do not promise an atomic snapshot of
a filesystem being modified concurrently. Bound input bytes, node nesting,
import depth, and source count with documented limits and structured errors.
Excess input is an error, never an implicitly truncated document. Glob traversal
counts examined directory entries, including nonmatches, against its work budget;
only accepted matches contribute to the sorted source order.
Bytes/reader sources require a diagnostic label. Imports and profiles for these
sources require an explicit filesystem base directory and root; never infer a
filesystem location from a label such as an S3 URI. Without that context they
remain unavailable. ConfigKit does not fetch remote imports.
Debian-style automatic `conf.d` discovery is deferred. A caller may explicitly
import a file or glob inside a directory named `conf.d`, but that name has no
special behavior. Do not scan adjacent directories implicitly.
- Glob traversal counts examined directory entries, including nonmatches, against
its separate work budget; only accepted matches contribute to sorted source order.
Expansion remains pattern-directed and must preserve the current path-import
identity, containment, repetition, and depth-first traversal rules.
## Profiles
@@ -650,11 +618,12 @@ Automatic migration is deferred.
## Consumer API design
The core declarations and single-source loading subset are implemented; their
current contract is owned by [configuration input](../config.md#single-source-loading)
and Go reference comments. Remaining declarations and behavior in this section are
target design, not implemented Go reference documentation or a frozen naming
commitment. Declarations omit method bodies and private fields.
The core declarations, callback lifecycle, strict composition, and explicit path
imports are implemented; their current contract is owned by
[configuration input](../config.md) and Go reference comments. Remaining declarations
and behavior in this section are target design, not implemented Go reference
documentation or a frozen naming commitment. Declarations omit method bodies and
private fields.
Consumer snippets assume the indicated standard-library imports and a `configkit`
import using the module path selected in the [implementation plan](implementation.md#implementation-decisions).
They are illustrative target examples, not currently runnable examples. The
@@ -664,11 +633,10 @@ and verification against consumer models.
### Entry point and reusable application definition
`Load`, `Definition`, `Options`, `FilesystemContext`, `Override`, `VersionRule`,
`Version`, `Validation`, and `Result` are declared. The current loader accepts only
the defaults and version-checking subset described by the
[single-source contract](../config.md#single-source-loading); later-feature fields
fail explicitly rather than being ignored. The complete lifecycle and option
behavior below remains the target as its owning stages land.
`Version`, `Validation`, and `Result` are declared. The current loader supports the
callback and path-import subset described by the [current contract](../config.md);
later profile, override, secret, and rendering behavior below remains target work and
unavailable options fail explicitly rather than being ignored.
```go
func Load[T any](source Source, options ...Options[T]) (*Result[T], error)
@@ -710,16 +678,11 @@ result. Callbacks must not retain these contexts or configuration pointers; shar
closures must be safe for concurrent calls if the application loads concurrently.
ConfigKit does not synchronize application-owned captured state.
`Source` and its constructors are implemented; [`source.go`](../../source.go) and
the [configuration input contract](../config.md#sources) own their current behavior.
The loading and filesystem-option behavior below remains target work.
Applications must use non-secret source labels because labels appear in diagnostics
and source reports.
Filesystem context follows the [source rules](#explicit-imports-and-source-identity):
a file's own directory supplies its base, and an explicit context can set its
root. Supplying `BaseDir` for a file is an options error; bytes/readers require
both absolute `BaseDir` and `Root` when imports or profiles are used. A base must
be inside its root. These options do not change application field path meanings.
`Source`, its constructors, and import filesystem context are implemented;
[`source.go`](../../source.go) and the [configuration input contract](../config.md#sources)
own their current behavior. Applications must use non-secret source labels because
labels appear in diagnostics and source reports. Profile-related filesystem behavior
remains target work. Filesystem options do not change application field path meanings.
`NoSource()` explicitly selects defaults-only loading. It contributes no document
or file source entry and marks no fields explicit by itself; CLI overrides still
@@ -1030,7 +993,7 @@ applications own secret-producing transformations not described by those records
### Results, errors, and inspection
`Result.Config`, `Metadata`, `Sources`, and `Diagnostics`, together with
`LoadError` and `WriteDiagnostics`, are implemented at the single-source boundary.
`LoadError` and `WriteDiagnostics`, are implemented at the current loading boundary.
The [current loading contract](../config.md#single-source-loading) and Go reference
comments own their behavior. Effective rendering and richer layered inspection
below remain target work.