ConfigKit/docs/internal/imports.md

Import Discovery and Scheduling Internals

The configuration contract owns selector syntax, defaults, requiredness, placement, inspection evidence, and observable ordering. This note records implementation flow and private identities for maintainers.

Discovery and acquisition

Each source's root directive is parsed into presence-aware declarations and resolved against load-wide defaults. acquisition.go owns shared acquisition policy, load-wide accounting, and tier-scoped repetition/namespace state; import_acquisition.go owns reading candidate bytes, declaration traversal, and import admission. Candidate selection in import_selection.go preserves a normalized namespace path separately from canonical access identity: overlapping roots coalesce the same candidate spelling in that namespace, while aliases reached through distinct spellings retain separate inclusion routes for admission. Directory searches walk each distinct normalized root once, apply all filename patterns to discovered names, and form a set union of matching candidates across overlapping roots. Exclusions are applied as set difference after discovery and candidate normalization. They do not suppress discovery or resource errors.

Directory traversal tracks canonical directories on the active traversal path. It adds an identity when descending and removes it when the frame is popped, so only a back-edge to an active directory is a traversal cycle; a later alias is not mistaken for one. This is separate from import dependency-cycle detection. For host sources, discovery uses rooted Lstat and canonical containment. filesystem_context.go owns resolved access facts and live access setup; the acquisition operation closes its live handles before composition. Native sources use fs.ReadLinkFS inspection when supplied; otherwise traversal follows the capabilities and behavior of the provided fs.FS, without claiming a general sandbox guarantee. source_context.go derives passive resource contexts from resolved sources for later caller-owned reads; those records do not retain live acquisition handles.

Host and native link bindings share one iterative link-frame evaluator. It tracks active links for cycle detection and reuses successfully completed link targets while resolving the remaining path components; each binding retains its own namespace and target-containment rules. Directory reads distinguish an initial-open failure from failures after opening: only an initial missing directory can be treated as optional, while enumeration and close failures remain selection errors, including errors from the standard fs.ReadDir fallback.

Glob selection keeps traversal coordination in selectGlobImportCandidates: it advances literal and patterned segments, reads directories, unions and sorts candidates, validates final targets, then applies exclusions and requiredness. For each patterned directory entry, prepareGlobEntry handles matching and hidden filtering, containment, target and link inspection, intermediate-directory eligibility, host/native access-path selection, and candidate identity and label. It returns a candidate, an ordinary skip, or a diagnostic without mutating the selector's candidate set or traversal state. Directory entry preparation remains separate because its matching, absence, recursion, and traversal-state rules differ.

Acquisition remains authored: declarations are visited in source order and each selected candidate set in namespace-path lexical order. This preserves source records, IDs, source-local version checks, and callback ordering. The acquired inclusion graph is retained separately from composition scopes so cycles that cross item boundaries remain detectable. Source discovery does not itself choose sequence assembly order.

Stable keys and scheduling

The shared scheduler compares immutable tuples component by component using Go's bytewise string order; if one tuple is a prefix of another, the shorter tuple sorts first. The import key is (namespace-kind, namespace-anchor, resource-kind, resource-path):

• Namespace kind is host or native. Host resource paths are canonical absolute paths rendered with slash separators; native paths retain their namespace-relative identity.
• Anchors come from explicit input positions, not discovery order: main is (0,0) and underlay/input positions are (group+1,input+1), each encoded as fixed-width decimal. Host accesses in one scheduled tier share its host anchor. A native filesystem namespace uses its earliest explicit selecting position in that tier. Filesystem tokens distinguish namespace equality, not sort order, so separately constructed filesystems with the same label remain distinct.
• Resource kind is file or synthetic. A synthetic root uses its explicit input anchor and an empty resource path rather than a display label or token address.

Inheritance keys are ("definition", scope-pointer, definition-pointer), where both pointers use canonical escaped semantic path spelling. Source IDs are not part of the key.

Both integrations use the same Kahn scheduler: it deduplicates dependency edges, keeps currently ready nodes in a min-heap, and selects the smallest ready key after every completion. Its scheduling cost is O(E + V log V) time and O(V + E) additional memory, excluding key construction, tuple copying, and tuple comparisons. On a stalled graph, deterministic DFS over the residual graph returns a closed cycle rotated to its smallest key; blocked dependents are not reported as cycle members.

Import edge preparation builds an importer-to-target adjacency index once for an acquisition's outer and item-local scheduling, and once for grouped scheduling. Each scope gathers edges from its participating identities and retains only edges with both endpoints in its local slots. The retained inclusion graph still owns cycle diagnostics and source relationships.

Import composition schedules a main tier or one underlay group as a single strict-scope graph. Applicable collection declarations are gathered for the scope before peer composition, including identity inherited from lower tiers. Identified peer sequences then combine distinct records in assembled source and list order; after the strict scope completes, layered assembly applies the resolved collection operation against the lower value, as defined by the configuration contract. Item contributions own nested local strict-mapping scopes; their helpers complete before the item is placed in its outer scope. Inheritance uses the same ready-node rule inside each sibling mapping scope, while preserving the authored mapping order in its output. These scopes do not flatten into one global graph, and independent precedence tiers remain separate.

The owners are acquisition.go, import_selection.go, import_acquisition.go, directory_walk.go, import_graph.go, import_projection.go, dependency_order.go, and inheritance_schedule.go. Their focused behavior is covered by import-selection, acquisition, scheduler, and scheduling integration tests under internal/engine; see the development test map.