Add explicit path import traversal
This commit is contained in:
@@ -20,10 +20,11 @@ The zero `Source` is invalid. Constructor argument failures are retained inside
|
||||
opaque source and become safe structured diagnostics when the source is consumed.
|
||||
File sources can observe file changes between consumptions; byte sources cannot.
|
||||
|
||||
Each source is limited to 8 MiB. A load will be limited to 64 MiB across all source
|
||||
and override input when those layers are implemented. Reads request at most the
|
||||
remaining allowance plus one byte so excess data is rejected rather than silently
|
||||
truncated. Data returned together with a read error is not accepted as partial input.
|
||||
Each source is limited to 8 MiB. An imported load is limited to 64 MiB across all
|
||||
accepted sources, 1,024 accepted sources, 64 active import edges, and 1,000,000
|
||||
input nodes in aggregate. Reads request at most the remaining allowance plus one byte
|
||||
so excess data is rejected rather than silently truncated. Data returned together
|
||||
with a read error is not accepted as partial input.
|
||||
|
||||
## Single-source loading
|
||||
|
||||
@@ -36,21 +37,22 @@ schema-version checking. Its result is detached before input is applied.
|
||||
The current lifecycle is:
|
||||
|
||||
```text
|
||||
validate options → parse one source → reject reserved directives → build schema
|
||||
→ check the optional application version → create and clone defaults
|
||||
validate options → parse the selected source → traverse/compose enabled imports
|
||||
→ build schema → check the optional application version → create and clone defaults
|
||||
→ strictly decode → recorded application resolution → semantic validation
|
||||
→ snapshot configuration and metadata
|
||||
```
|
||||
|
||||
`File`, `Bytes`, and `Reader` each contribute one base source with the deterministic
|
||||
ID `source-1`. `NoSource` skips parsing, supplies an absent version to a configured
|
||||
version checker, contributes no source, and continues through defaults, decode, and
|
||||
projection. A missing file remains an error; it is never interpreted as no input.
|
||||
`File`, `Bytes`, and `Reader` each contribute the first base source with deterministic
|
||||
ID `source-1`; enabled imports continue in depth-first order as `source-2` onward.
|
||||
`NoSource` skips parsing, supplies an absent version to a configured version checker,
|
||||
contributes no source, and continues through defaults, decode, and projection. A
|
||||
missing file remains an error; it is never interpreted as no input.
|
||||
|
||||
The current `Options` boundary accepts a defaults factory, `VersionRule`, resolver,
|
||||
and semantic validator. Imports, active profiles, filesystem context, overrides,
|
||||
and dynamic secret classification return an explicit
|
||||
`options.unavailable` error because their lifecycle stages are not implemented.
|
||||
semantic validator, explicit path imports, and import filesystem context. Active
|
||||
profiles, standalone filesystem context, overrides, and dynamic secret classification
|
||||
return `options.unavailable` because their lifecycle stages are not implemented.
|
||||
The reserved profile spelling `default` selects no active profile. `NoSource`
|
||||
additionally rejects imports, active profiles, and filesystem context as invalid
|
||||
combinations. Input containing a reserved `_configkit` mapping key is rejected while
|
||||
@@ -152,10 +154,8 @@ live services as a substitute for configuration validation.
|
||||
|
||||
## Strict document composition
|
||||
|
||||
ConfigKit's private document layer can strictly compose parsed base documents. The
|
||||
operation is available for file-graph integration but is not yet reachable through
|
||||
the public single-source `Load` lifecycle. Application defaults do not participate
|
||||
in this operation.
|
||||
ConfigKit strictly composes parsed base documents when `Options.Imports` is true.
|
||||
Application defaults do not participate in this operation.
|
||||
|
||||
| Existing node | Incoming node | Result |
|
||||
| --- | --- | --- |
|
||||
@@ -180,6 +180,44 @@ traversal order. Traversal order never authorizes replacement. Composition clone
|
||||
the resulting tree and its ordered claim records rather than mutating or aliasing an
|
||||
input document.
|
||||
|
||||
## Explicit path imports
|
||||
|
||||
Imports are opt-in. With `Options.Imports` enabled, a document may reserve its root
|
||||
`_configkit` mapping for an `imports` sequence. ConfigKit removes that mapping before
|
||||
application decoding. Each sequence item is a mapping with exactly one nonempty
|
||||
string `path` or `glob` plus an optional boolean `optional`; all other shapes and
|
||||
fields are `import.invalid`. `glob` is recognized but temporarily returns
|
||||
`import.unsupported` until bounded glob expansion is implemented. A `_configkit`
|
||||
key below the root is invalid. With imports disabled, any occurrence remains
|
||||
`import.disabled`.
|
||||
|
||||
Path declarations use slash-separated relative paths. Absolute paths and network
|
||||
URLs are invalid. Each path resolves from its declaring file's canonical directory,
|
||||
and files are traversed depth-first in declaration order: a file contributes its own
|
||||
body before its imports. Order stabilizes source records and diagnostics but never
|
||||
permits replacement. Required missing paths fail; `optional: true` suppresses only
|
||||
target nonexistence, not invalid paths, containment failures, directories,
|
||||
permissions, parse errors, or composition errors.
|
||||
|
||||
The configuration root defaults to a selected file's canonical directory. A file
|
||||
source may supply `FilesystemContext.Root` to widen that root but may not supply
|
||||
`BaseDir`. Byte and reader sources must supply both `BaseDir` and `Root` as absolute
|
||||
existing directories when imports are enabled; their diagnostic labels, including
|
||||
URI-looking labels, never imply filesystem paths. The base and selected file must be
|
||||
inside the canonical root. ConfigKit canonicalizes symlinks, checks containment with
|
||||
relative paths, and opens the canonical root-relative target through a library-owned
|
||||
root handle. Contained relative or absolute symlink targets are accepted; escapes
|
||||
are rejected. Canonical paths define identity, so hard links at distinct paths remain
|
||||
distinct sources.
|
||||
|
||||
An active-stack repeat is `import.cycle` and reports the complete cycle's import
|
||||
sites. A previously visited canonical path is `import.repeated` and reports both
|
||||
inclusion routes, including diamonds and aliases. Each accepted file is read once.
|
||||
Imported fragments may contain their own root directives, but may not claim the
|
||||
application's schema-version path or replace one of its ancestors; the version must
|
||||
come from the selected source. No directory is scanned implicitly, including one
|
||||
named `conf.d`, and ConfigKit performs no remote fetching.
|
||||
|
||||
## YAML documents
|
||||
|
||||
The private YAML boundary currently accepts exactly one nonempty document whose root
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
|
||||
This is the contributor entry point for ConfigKit. Read the task-specific owners
|
||||
below before making changes. The repository is in early implementation: its public
|
||||
loader currently covers one strict source, typed defaults, schema-version checks,
|
||||
recorded application resolution, semantic validation, and metadata. The
|
||||
loader currently covers strict selected sources, explicit path-import graphs,
|
||||
composition, typed defaults, schema-version checks, recorded application resolution,
|
||||
semantic validation, and metadata. The
|
||||
[component overview](internal/overview.md) owns the implementation inventory.
|
||||
|
||||
## Initial orientation
|
||||
|
||||
@@ -112,6 +112,18 @@ Assigned to explicit import traversal and glob handling:
|
||||
traversal, canonical containment, cycle/diamond routes, and fixed source IDs.
|
||||
Canonical paths—not `os.SameFile`—define identity, so hard links remain distinct.
|
||||
|
||||
Path-import implementation record (2026-09-06): `splitImportDirectives`,
|
||||
`parseImportDeclaration`, `importFilesystem`, `importTraversal.visitPath`, and
|
||||
`loadImportedDocuments` in `imports.go` adapt the assigned envelope, loading, source,
|
||||
and failure mechanics. The corresponding ConfigKit tests adapt the assigned
|
||||
declaration-order, disjoint composition, keyed-entry, all-source conflict, unsafe
|
||||
target, alias, imported parse/schema, and relative-resolution cases. ConfigKit adds
|
||||
recursive bounded DFS, `os.OpenRoot` confinement, canonical symlink identity,
|
||||
cycle/diamond route locations, optional-path semantics, aggregate budgets, and
|
||||
fragment version-placement enforcement. The planned glob syntax is parsed but
|
||||
remains explicitly unavailable for its next owning work; hard links intentionally
|
||||
replace Narratio's same-file rejection with path identity.
|
||||
|
||||
### Profiles and overlay
|
||||
|
||||
Assigned to selected-profile overlay:
|
||||
|
||||
@@ -20,18 +20,18 @@ module and package documentation boundary. Its implemented public primitives are
|
||||
- strict schema-driven decoding over detached defaults in `decode.go`, including
|
||||
mapping/sequence semantics, null and numeric policy, safe codec errors, presence,
|
||||
baseline/final trees, and initial default/input origins; and
|
||||
- public single-source lifecycle coordination in `load.go`, version-report contexts
|
||||
in `validation.go`, and detached base/default metadata assembly in
|
||||
`load_metadata.go`; and
|
||||
- public lifecycle coordination in `load.go`, bounded canonical path-import DFS in
|
||||
`imports.go`, version-report contexts in `validation.go`, and detached
|
||||
base/default metadata assembly in `load_metadata.go`; and
|
||||
- recorded application defaulting and normalization in `resolution.go`, including
|
||||
explicitness queries, helper verification, structural mutation audit, coalesced
|
||||
intent, private before/after snapshots, and resolution metadata integration; and
|
||||
- post-resolution schema-aware report collection and validator mutation detection in
|
||||
`validation.go`, completing the public single-source callback lifecycle.
|
||||
`validation.go`, completing the public callback lifecycle after composed input.
|
||||
|
||||
The [configuration contract](../config.md) owns implemented input, schema, decoding,
|
||||
single-source lifecycle, version checking, recorded resolution, semantic validation,
|
||||
metadata, and ownership behavior.
|
||||
loading lifecycle, path imports, composition, version checking, recorded resolution,
|
||||
semantic validation, metadata, and ownership behavior.
|
||||
|
||||
The source-extraction inventory records the application foundations assigned to
|
||||
future implementation work:
|
||||
|
||||
@@ -850,7 +850,38 @@ Acceptance:
|
||||
|
||||
## Stage 11 — Implement explicit file-import traversal
|
||||
|
||||
Status: pending.
|
||||
Status: complete.
|
||||
|
||||
Completion evidence (2026-09-06):
|
||||
|
||||
- Added strict recursive `_configkit.imports` parsing with exactly one `path` or
|
||||
recognized `glob`, optional booleans, unknown-shape rejection, root-only placement,
|
||||
and detached directive removal. Glob declarations return the deliberate temporary
|
||||
`import.unsupported` gate for the next owning work.
|
||||
- Implemented declaration-order depth-first path traversal with canonical symlink
|
||||
identity, relative containment checks, and canonical root-relative access through a
|
||||
ConfigKit-owned `os.Root`. File inputs infer their canonical base/root; labeled
|
||||
byte and reader inputs require explicit absolute base/root context and never infer
|
||||
paths from URI-like labels.
|
||||
- Kept active and visited indexes separate, reporting complete cycle import sites and
|
||||
both diamond/alias inclusion routes. Canonical aliases repeat, while hard links at
|
||||
distinct paths remain separate sources. Required missing paths, directories,
|
||||
unreadable targets, escapes, and canonicalization failures remain errors; optional
|
||||
suppresses only nonexistence.
|
||||
- Assigned deterministic source IDs while reading each accepted source once, enforced
|
||||
per-source and aggregate bytes, aggregate input nodes, source count, and active-edge
|
||||
depth, and retained import-site/source records on failures. Imported fragments are
|
||||
rejected when they claim the application version path or a non-mapping ancestor,
|
||||
before strict composition.
|
||||
- Wired composed input and multi-location origins through `Load`, version checking,
|
||||
strict decoding, resolution, validation, and base metadata. Temporary-directory
|
||||
tests cover nested/list-order traversal, composition conflicts, version placement,
|
||||
directives, optional/required paths, explicit roots, cycles, diamonds, symlinks,
|
||||
hard links, parse/decode attribution, file types, context rules, and reduced limit
|
||||
boundaries. The [current path-import contract](../config.md#explicit-path-imports)
|
||||
owns implemented behavior; glob expansion remains pending. Verified `go test ./...`,
|
||||
`go test -race ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and
|
||||
`git diff --check` with Go 1.26.7 on linux/amd64.
|
||||
|
||||
Read: feature [import graph](initial.md#explicit-imports-and-source-identity).
|
||||
|
||||
|
||||
@@ -426,14 +426,14 @@ configuration is never returned as a successful, usable result.
|
||||
Source ownership, the supported YAML subset, scalar grammar, fixed input limits,
|
||||
and safe failure behavior are implemented. The [current configuration input
|
||||
contract](../config.md) now owns those definitions, including typed scalar
|
||||
conversion through the current single-source lifecycle.
|
||||
conversion through the current loading lifecycle.
|
||||
|
||||
## Paths and presence
|
||||
|
||||
Semantic path construction, JSON Pointer parsing, and base/default metadata
|
||||
population are implemented. Their current contracts are owned by the Go
|
||||
documentation in [`path.go`](../../path.go), [`metadata.go`](../../metadata.go),
|
||||
and the [single-source configuration contract](../config.md#base-default-metadata).
|
||||
and the [current configuration contract](../config.md#base-default-metadata).
|
||||
Layered replacement and resolution behavior below remains target lifecycle work.
|
||||
|
||||
Distinguish absent, explicitly null, explicitly zero/false/empty, and other
|
||||
@@ -475,62 +475,30 @@ the surviving sequence, not an attempted correspondence with previous indices.
|
||||
|
||||
## Explicit imports and source identity
|
||||
|
||||
Imports are opt-in through the loading API. Reserve the root `_configkit` mapping
|
||||
for library directives and remove it before application decoding. Initially its
|
||||
only permitted field is `imports`, a sequence of declarations. Illustrative
|
||||
fragment:
|
||||
Explicit path-import traversal, source identity, containment, budgets, recursive
|
||||
directives, and the temporary glob gate are implemented and owned by the
|
||||
[current configuration contract](../config.md#explicit-path-imports). Bounded glob
|
||||
expansion remains target behavior. Illustrative future fragment:
|
||||
|
||||
```yaml
|
||||
_configkit:
|
||||
imports:
|
||||
- path: fragments/server.yml
|
||||
- glob: fragments/outputs/*.yml
|
||||
optional: true
|
||||
```
|
||||
|
||||
Each declaration has exactly one nonempty `path` or `glob`; `optional` is a bool
|
||||
and defaults to false. Unknown directive fields are errors. When imports are
|
||||
disabled, the presence of `_configkit` is an error, not an ignored directive.
|
||||
|
||||
- Relative paths resolve from the declaring file's directory. Absolute import
|
||||
paths and network URLs are rejected.
|
||||
- Expand declarations in list order, sorting each glob's matches
|
||||
lexicographically by slash-separated relative path, with no locale sorting.
|
||||
Use the slash-based [Go `path.Match` grammar](https://pkg.go.dev/path#Match)
|
||||
(`*`, `?`, character classes, and backslash escapes), with recursive `**`
|
||||
syntax forbidden. Backslashes escape pattern characters, not directory separators.
|
||||
- Traverse depth-first: process a file's own contribution, then its declarations
|
||||
in order. Ordering stabilizes source lists and diagnostics; it never permits
|
||||
duplicate definitions to override one another.
|
||||
- A required missing path or unmatched glob is an error. Optional suppresses
|
||||
only a nonexistent target or zero matches. Invalid patterns, permission
|
||||
- A required unmatched glob is an error. Optional suppresses only zero matches.
|
||||
Invalid patterns, permission
|
||||
failures, directories, and invalid matched documents remain errors.
|
||||
- The configuration root defaults to the main file's directory; callers may
|
||||
explicitly supply another root. Resolve symlinks and cleaned absolute paths
|
||||
for identity and containment. Imports and profiles must stay inside that
|
||||
canonical root. Retain declared paths and import-site locations for messages.
|
||||
Relative and absolute symlink targets are permitted when their canonical target
|
||||
is contained; the spelling of a symlink target alone does not determine containment.
|
||||
- An active-stack repeat is a cycle; report the complete cycle and import sites.
|
||||
A previously visited canonical file outside the active stack is a repeated
|
||||
import error, including diamond graphs and overlapping glob matches. Report
|
||||
both inclusion routes. Hard links at distinct canonical paths remain distinct
|
||||
sources; this is path identity, not inode deduplication.
|
||||
- Read each accepted source once per load. Do not promise an atomic snapshot of
|
||||
a filesystem being modified concurrently. Bound input bytes, node nesting,
|
||||
import depth, and source count with documented limits and structured errors.
|
||||
Excess input is an error, never an implicitly truncated document. Glob traversal
|
||||
counts examined directory entries, including nonmatches, against its work budget;
|
||||
only accepted matches contribute to the sorted source order.
|
||||
|
||||
Bytes/reader sources require a diagnostic label. Imports and profiles for these
|
||||
sources require an explicit filesystem base directory and root; never infer a
|
||||
filesystem location from a label such as an S3 URI. Without that context they
|
||||
remain unavailable. ConfigKit does not fetch remote imports.
|
||||
|
||||
Debian-style automatic `conf.d` discovery is deferred. A caller may explicitly
|
||||
import a file or glob inside a directory named `conf.d`, but that name has no
|
||||
special behavior. Do not scan adjacent directories implicitly.
|
||||
- Glob traversal counts examined directory entries, including nonmatches, against
|
||||
its separate work budget; only accepted matches contribute to sorted source order.
|
||||
Expansion remains pattern-directed and must preserve the current path-import
|
||||
identity, containment, repetition, and depth-first traversal rules.
|
||||
|
||||
## Profiles
|
||||
|
||||
@@ -650,11 +618,12 @@ Automatic migration is deferred.
|
||||
|
||||
## Consumer API design
|
||||
|
||||
The core declarations and single-source loading subset are implemented; their
|
||||
current contract is owned by [configuration input](../config.md#single-source-loading)
|
||||
and Go reference comments. Remaining declarations and behavior in this section are
|
||||
target design, not implemented Go reference documentation or a frozen naming
|
||||
commitment. Declarations omit method bodies and private fields.
|
||||
The core declarations, callback lifecycle, strict composition, and explicit path
|
||||
imports are implemented; their current contract is owned by
|
||||
[configuration input](../config.md) and Go reference comments. Remaining declarations
|
||||
and behavior in this section are target design, not implemented Go reference
|
||||
documentation or a frozen naming commitment. Declarations omit method bodies and
|
||||
private fields.
|
||||
Consumer snippets assume the indicated standard-library imports and a `configkit`
|
||||
import using the module path selected in the [implementation plan](implementation.md#implementation-decisions).
|
||||
They are illustrative target examples, not currently runnable examples. The
|
||||
@@ -664,11 +633,10 @@ and verification against consumer models.
|
||||
### Entry point and reusable application definition
|
||||
|
||||
`Load`, `Definition`, `Options`, `FilesystemContext`, `Override`, `VersionRule`,
|
||||
`Version`, `Validation`, and `Result` are declared. The current loader accepts only
|
||||
the defaults and version-checking subset described by the
|
||||
[single-source contract](../config.md#single-source-loading); later-feature fields
|
||||
fail explicitly rather than being ignored. The complete lifecycle and option
|
||||
behavior below remains the target as its owning stages land.
|
||||
`Version`, `Validation`, and `Result` are declared. The current loader supports the
|
||||
callback and path-import subset described by the [current contract](../config.md);
|
||||
later profile, override, secret, and rendering behavior below remains target work and
|
||||
unavailable options fail explicitly rather than being ignored.
|
||||
|
||||
```go
|
||||
func Load[T any](source Source, options ...Options[T]) (*Result[T], error)
|
||||
@@ -710,16 +678,11 @@ result. Callbacks must not retain these contexts or configuration pointers; shar
|
||||
closures must be safe for concurrent calls if the application loads concurrently.
|
||||
ConfigKit does not synchronize application-owned captured state.
|
||||
|
||||
`Source` and its constructors are implemented; [`source.go`](../../source.go) and
|
||||
the [configuration input contract](../config.md#sources) own their current behavior.
|
||||
The loading and filesystem-option behavior below remains target work.
|
||||
Applications must use non-secret source labels because labels appear in diagnostics
|
||||
and source reports.
|
||||
Filesystem context follows the [source rules](#explicit-imports-and-source-identity):
|
||||
a file's own directory supplies its base, and an explicit context can set its
|
||||
root. Supplying `BaseDir` for a file is an options error; bytes/readers require
|
||||
both absolute `BaseDir` and `Root` when imports or profiles are used. A base must
|
||||
be inside its root. These options do not change application field path meanings.
|
||||
`Source`, its constructors, and import filesystem context are implemented;
|
||||
[`source.go`](../../source.go) and the [configuration input contract](../config.md#sources)
|
||||
own their current behavior. Applications must use non-secret source labels because
|
||||
labels appear in diagnostics and source reports. Profile-related filesystem behavior
|
||||
remains target work. Filesystem options do not change application field path meanings.
|
||||
|
||||
`NoSource()` explicitly selects defaults-only loading. It contributes no document
|
||||
or file source entry and marks no fields explicit by itself; CLI overrides still
|
||||
@@ -1030,7 +993,7 @@ applications own secret-producing transformations not described by those records
|
||||
### Results, errors, and inspection
|
||||
|
||||
`Result.Config`, `Metadata`, `Sources`, and `Diagnostics`, together with
|
||||
`LoadError` and `WriteDiagnostics`, are implemented at the single-source boundary.
|
||||
`LoadError` and `WriteDiagnostics`, are implemented at the current loading boundary.
|
||||
The [current loading contract](../config.md#single-source-loading) and Go reference
|
||||
comments own their behavior. Effective rendering and richer layered inspection
|
||||
below remain target work.
|
||||
|
||||
Reference in New Issue
Block a user