126 lines
7.5 KiB
Markdown
126 lines
7.5 KiB
Markdown
# Import Discovery and Scheduling Internals
|
|
|
|
The [configuration contract](../config.md#explicit-imports) owns selector syntax,
|
|
defaults, requiredness, placement, inspection evidence, and observable ordering.
|
|
This note records implementation flow and private identities for maintainers.
|
|
|
|
## Discovery and acquisition
|
|
|
|
Each source's root directive is parsed into presence-aware declarations and
|
|
resolved against load-wide defaults. `acquisition.go` owns shared acquisition
|
|
policy, load-wide accounting, and tier-scoped repetition/namespace state;
|
|
`import_acquisition.go` owns reading candidate bytes, declaration traversal,
|
|
and import admission. Candidate
|
|
selection in `import_selection.go` preserves a normalized
|
|
namespace path separately from canonical access identity: overlapping roots
|
|
coalesce the same candidate spelling in that namespace, while aliases reached
|
|
through distinct spellings retain separate inclusion routes for admission.
|
|
Directory searches walk each distinct normalized root once, apply all filename
|
|
patterns to discovered names, and form a set union of matching candidates across
|
|
overlapping roots. Exclusions are applied as set difference after discovery and
|
|
candidate normalization. They do not suppress discovery or resource errors.
|
|
|
|
Directory traversal tracks canonical directories on the active traversal path.
|
|
It adds an identity when descending and removes it when the frame is popped, so
|
|
only a back-edge to an active directory is a traversal cycle; a later alias is
|
|
not mistaken for one. This is separate from import dependency-cycle detection.
|
|
For host sources, discovery uses rooted `Lstat` and canonical containment.
|
|
`filesystem_context.go` owns resolved access facts and live access setup; the
|
|
acquisition operation closes its live handles before composition. Native sources
|
|
use `fs.ReadLinkFS` inspection when supplied; otherwise traversal follows the
|
|
capabilities and behavior of the provided `fs.FS`, without claiming a general
|
|
sandbox guarantee. `source_context.go` derives passive resource contexts from
|
|
resolved sources for later caller-owned reads; those records do not retain live
|
|
acquisition handles.
|
|
|
|
Host and native link bindings share one iterative link-frame evaluator. It
|
|
tracks active links for cycle detection and reuses successfully completed link
|
|
targets while resolving the remaining path components; each binding retains its
|
|
own namespace and target-containment rules. Directory reads distinguish an
|
|
initial-open failure from failures after opening: only an initial missing
|
|
directory can be treated as optional, while enumeration and close failures
|
|
remain selection errors, including errors from the standard `fs.ReadDir`
|
|
fallback.
|
|
|
|
Glob selection keeps traversal coordination in `selectGlobImportCandidates`:
|
|
it advances literal and patterned segments, reads directories, unions and sorts
|
|
candidates, validates final targets, then applies exclusions and requiredness.
|
|
For each patterned directory entry, `prepareGlobEntry` handles matching and
|
|
hidden filtering, containment, target and link inspection, intermediate-directory
|
|
eligibility, host/native access-path selection, and candidate identity and label.
|
|
It returns a candidate, an ordinary skip, or a diagnostic without mutating the
|
|
selector's candidate set or traversal state. Directory entry preparation remains
|
|
separate because its matching, absence, recursion, and traversal-state rules
|
|
differ.
|
|
|
|
Acquisition remains authored: declarations are visited in source order and each
|
|
selected candidate set in namespace-path lexical order. This preserves source
|
|
records, IDs, source-local version checks, and callback ordering. The acquired
|
|
inclusion graph is retained separately from composition scopes so cycles that
|
|
cross item boundaries remain detectable. Source discovery does not itself choose
|
|
sequence assembly order.
|
|
|
|
## Stable keys and scheduling
|
|
|
|
The shared scheduler compares immutable tuples component by component using Go's
|
|
bytewise string order; if one tuple is a prefix of another, the shorter tuple
|
|
sorts first. The import key is
|
|
`(namespace-kind, namespace-anchor, resource-kind, resource-path)`:
|
|
|
|
- Namespace kind is `host` or `native`. Host resource paths are canonical
|
|
absolute paths rendered with slash separators; native paths retain their
|
|
namespace-relative identity.
|
|
- Anchors come from explicit input positions, not discovery order: main is
|
|
`(0,0)` and underlay/input positions are `(group+1,input+1)`, each encoded as
|
|
fixed-width decimal. Host accesses in one scheduled tier share its host anchor.
|
|
A native filesystem namespace uses its earliest explicit selecting position
|
|
in that tier. Filesystem tokens distinguish namespace equality, not sort order,
|
|
so separately constructed filesystems with the same label remain distinct.
|
|
- Resource kind is `file` or `synthetic`. A synthetic root uses its explicit
|
|
input anchor and an empty resource path rather than a display label or token
|
|
address.
|
|
|
|
Inheritance keys are
|
|
`("definition", scope-pointer, definition-pointer)`, where both pointers use
|
|
canonical escaped semantic path spelling. Source IDs are not part of the key.
|
|
|
|
Both integrations use the same Kahn scheduler: it deduplicates dependency edges,
|
|
keeps currently ready nodes in a min-heap, and selects the smallest ready key
|
|
after every completion. Its scheduling cost is `O(E + V log V)` time and
|
|
`O(V + E)` additional memory, excluding key construction, tuple copying, and
|
|
tuple comparisons. On a stalled graph, deterministic DFS over the residual graph
|
|
returns a closed cycle rotated to its smallest key; blocked dependents are not
|
|
reported as cycle members.
|
|
|
|
Import edge preparation builds an importer-to-target adjacency index once for
|
|
an acquisition's outer and item-local scheduling, and once for grouped
|
|
scheduling. Each scope gathers edges from its participating identities and
|
|
retains only edges with both endpoints in its local slots. The retained
|
|
inclusion graph still owns cycle diagnostics and source relationships.
|
|
|
|
Import composition schedules a main tier or one underlay group as a single
|
|
strict-scope graph. Applicable collection declarations are gathered for the
|
|
scope before peer composition, including identity inherited from lower tiers.
|
|
Identified peer sequences then combine distinct records in assembled source and
|
|
list order; after the strict scope completes, layered assembly applies the
|
|
resolved collection operation against the lower value, as defined by the
|
|
[configuration contract](../config.md#strict-document-composition). Item
|
|
contributions own nested local strict-mapping scopes; their helpers complete
|
|
before the item is placed in its outer scope. Inheritance uses the same
|
|
ready-node rule inside each sibling mapping scope, while preserving the authored
|
|
mapping order in its output.
|
|
These scopes do not flatten into one global graph, and independent precedence
|
|
tiers remain separate.
|
|
|
|
The owners are [`acquisition.go`](../../internal/engine/acquisition.go),
|
|
[`import_selection.go`](../../internal/engine/import_selection.go),
|
|
[`import_acquisition.go`](../../internal/engine/import_acquisition.go),
|
|
[`directory_walk.go`](../../internal/engine/directory_walk.go),
|
|
[`import_graph.go`](../../internal/engine/import_graph.go),
|
|
[`import_projection.go`](../../internal/engine/import_projection.go),
|
|
[`dependency_order.go`](../../internal/engine/dependency_order.go), and
|
|
[`inheritance_schedule.go`](../../internal/engine/inheritance_schedule.go).
|
|
Their focused behavior is covered by import-selection, acquisition, scheduler,
|
|
and scheduling integration tests under `internal/engine`; see the
|
|
[development test map](../development.md#task-to-test-map).
|