diff --git a/docs/roadmap/imports.md b/docs/roadmap/imports.md index 4448057..083f7d0 100644 --- a/docs/roadmap/imports.md +++ b/docs/roadmap/imports.md @@ -46,9 +46,16 @@ Glob matching uses slash-based Go `path.Match` syntax. Recursive `**` syntax is not supported; authors can match particular depths with patterns such as `prompts/*.yaml` and `prompts/*/*.yaml`. -The proposed changes concern naming, discovery, and absence handling. Import -destinations, composition, precedence, and resource identity remain separate -concerns. +Current imports expand depth-first in declaration order, with each ordinary +source contributing its body before its imports. The existing +[sibling inheritance resolver](../config.md#sibling-inheritance) instead follows +dependency chains before resolving dependents, using sorted sibling names and +an explicit stack. Neither implements the shared ordering selected below. + +The proposed changes concern naming, discovery, absence handling, and uniform +dependency ordering for imports and `extends`. Import destinations, strict +composition, inheritance overlay semantics, tier precedence, and resource +identity retain their distinct roles. ## Proposed Import Vocabulary @@ -107,6 +114,9 @@ interfaces, without adopting all their spellings, defaults, or matching modes. and filesystem contracts remain in force. - **Include hidden entries by default, with an opt-out.** Provide `hidden: false` to exclude hidden files and prune hidden directories from discovery. +- **One dependency-ordering algorithm for imports and `extends`.** Both use the + shared dependency-ready scheduler defined below. This is settled independently + of the existing implementations and includes refactoring both resolvers. The defaults and availability of these controls are settled. Their exact scope at explicitly named roots, the definition of hidden names across filesystems, @@ -158,6 +168,91 @@ The accepted `recursive` spelling is retained. The formerly tentative `max_depth` spelling becomes `depth`. Resource budgets remain separate from depth, and the current limits still apply to an unlimited-depth search. +## Shared Dependency-Ready Ordering + +**Status: settled for imports and `extends`; not yet implemented.** Use +deterministic dependency-ready ordering: Kahn's topological-sort algorithm with +a min-heap of eligible nodes. Both resolvers must use the same algorithm in the +same way, through one shared internal scheduling implementation, common +tie-breaking rules, and common cycle-path logic. + +### Scheduling Rule + +A dependency edge means that the dependent cannot complete until its dependency +has completed. For each dependency graph: + +1. Count each node's outstanding dependencies and identify initially eligible + nodes: those with none. +2. Choose the eligible node with the smallest stable ordering key. +3. Append it to the schedule, decrement its dependents' outstanding counts, and + immediately make newly eligible nodes available for the next selection. +4. Continue until all nodes are scheduled. If unfinished nodes remain with no + eligible node, reject the graph and identify an actual dependency cycle. + +Use stable source or definition identities to construct ordering keys, with +the same deterministic comparison rule in both resolvers. Map iteration order +and discovery-assigned source IDs must not supply the tie-breaker. Specify key +construction for source namespaces and logical definition paths during +implementation planning; it must distinguish nodes within each scheduled graph. + +Construct and validate a complete schedule for the available graph before +applying its resolution operations. A blocked dependent is not necessarily a +cycle member: diagnostics should identify the actual cycle using shared, +deterministic cycle-path logic. Source discovery and parsing may already have +occurred; this rule is not a promise to avoid reading invalid configurations. + +### Scope And Integration + +Discovery and resolution are separate operations. An importing source must be +read to discover its imports, but its contribution need not be assembled first. +Discovery order is not a substitute for the selected resolution schedule. + +Both resolvers construct dependency graphs and apply resolver-specific operations +through the shared scheduler. Imports retain strict composition; `extends` +retains inheritance overlay behavior. Scheduling does not authorize replacement +within a strict composition tier or change precedence between tiers. Shared +dependencies can be scheduled once where valid; this does not relax the existing +prohibition on repeated source inclusion for imports. + +Preserve item-local helper scopes and enclosing-before-nested inheritance +requirements. An imported item completes its helper contributions before being +placed as an item. Enclosing inheritance establishes the context in which nested +relationships are discovered and resolved. Use the same scheduler at those +scope boundaries, with the actual prerequisites represented; do not flatten raw +files or mappings in a way that loses their scope. Inheritance still resolves +after source assembly and precedence have established the effective document. + +The integration must account for resulting sequence positions, provenance, +history, and diagnostic ordering. It must distinguish source acquisition records +from assembly order rather than assuming they are the same. Review both +resolvers' contracts and behavioral tests when implementing the new schedule; +preserving the old traversal is not a design objective. + +### Rationale And Validation + +Dependency-ready ordering selects the smallest currently eligible node without +adding branch preference, batch barriers, or distance-from-root priority. With +a fixed total ordering key, it gives the lexicographically smallest valid +dependency-first schedule. It can interleave independent branches; branch +contiguity is not part of the chosen ordering contract. + +The comparison considered deterministic depth-first postorder, dependency +rounds, and descending longest-path depth as well. All can satisfy dependency +correctness, but their additional ordering policies did not justify choosing +them over the uniform eligibility rule. This decision gives no preference to +the existing implementation or to minimizing pre-1.0 refactoring. + +The scheduler requires O(E + V log V) time and O(V + E) memory for V nodes and +E dependency edges, assuming constant-time key comparisons. Source access, +parsing, merging, and provenance work are separate costs. + +Validate independent branches, unequal chains, branches with multiple +dependencies, shared dependencies at equal and unequal depths, cycles with +blocked non-cycle dependents, and nested scopes. Equivalent valid graphs and +ordering keys must produce identical schedules through the import and +inheritance integrations. Verify independence from graph insertion order and +preserve each resolver's domain-specific admissibility and combination rules. + ## Discovery Semantics ### Filename Matching @@ -355,7 +450,8 @@ scaffolding. No promise is made to preserve every old declaration or use case. The recommendations below are not settled decisions. The new principles resolve transition support, omitted filename filtering, and the defaults and opt-out -availability for symlinks and hidden entries. Seven decision areas remain; +availability for symlinks and hidden entries. Shared dependency-ready ordering +for imports and `extends` is also settled. Six decision areas remain; closely related validation details are grouped together without treating them as implicitly approved. Each has at least one viable alternative. @@ -426,8 +522,8 @@ That eases generated configuration but can turn an erroneous bound into a wider search or make a direct-file import appear recursive. These relaxations can be chosen independently; they do not require accepting ambiguous selectors. Another viable selector shape is to allow multiple selectors as an explicit -union, but it needs ordering and overlap semantics that one-selector declarations -avoid. +union, but it needs rules for constructing the graph and handling overlaps that +one-selector declarations avoid. ### 4. Discovery Control Boundaries And Cycles @@ -467,23 +563,7 @@ but can hide a layout mistake and needs clearly defined inspection evidence. It would not justify suppressing repeated-source errors for files actually selected through distinct routes. -### 5. Match Ordering - -**Why still open:** Inclusion defaults do not determine order. `include_vars` -sorts directories and filenames; `win_find` documents path-sorted results; -POSIX `find` does not promise the same global ordering. - -**Recommendation:** Sort selected slash-relative paths lexicographically within -each declaration, then use existing depth-first import expansion. This provides -one familiar order for discovery, sequence assembly, and diagnostics. It -requires collecting bounded candidates before expansion. - -**Alternative:** Load direct files first, then visit sorted subdirectories -recursively. This offers directory-oriented traversal and can begin loading -sooner, but gives different sequence item positions. Neither choice grants -replacement precedence within strict composition. - -### 6. Application And Operator Policy Precedence +### 5. Application And Operator Policy Precedence **Why still open:** The library defaults and authored opt-outs are settled. Whether consumers can set different defaults or enforce restrictions over @@ -505,7 +585,7 @@ needs clear conflict diagnostics and scope, particularly for explicit sources. A smaller alternative is declaration-level controls only initially: simpler API, but consumers cannot choose common defaults without changing declarations. -### 7. Evidence For Absent Optional Imports +### 6. Evidence For Absent Optional Imports **Why still open:** Permitting absence does not settle how it should be exposed for inspection. Ansible's file lookup, variable loader, and find modules return