From a7ca2c59a2ef15943f0c51d916f9883150fff26e Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 19 Oct 2026 01:54:01 +0000 Subject: [PATCH] Settle import defaults, discovery boundaries, and inspection decisions --- docs/roadmap/imports.md | 244 ++++++++++++++++------------------------ 1 file changed, 100 insertions(+), 144 deletions(-) diff --git a/docs/roadmap/imports.md b/docs/roadmap/imports.md index 083f7d0..83f77a9 100644 --- a/docs/roadmap/imports.md +++ b/docs/roadmap/imports.md @@ -87,7 +87,7 @@ The fields have distinct jobs: | `path` | Import one named file, relative to the declaring file | | `dir` | Search a directory, relative to the declaring file | | `pattern` | Filter candidate files by filename within the searched directories | -| `recursive` | Include descendant directories when true | +| `recursive` | Default false; include descendant directories when true | | `depth` | Bound directory-search depth; 1 means direct files, 0 or omission means unlimited when recursive | | `follow` | Follow symlinks by default; false disables following within the discovery scope defined below | | `hidden` | Include hidden entries by default; false excludes hidden files and prunes hidden directories within the discovery scope defined below | @@ -118,10 +118,10 @@ interfaces, without adopting all their spellings, defaults, or matching modes. shared dependency-ready scheduler defined below. This is settled independently of the existing implementations and includes refactoring both resolvers. -The defaults and availability of these controls are settled. Their exact scope -at explicitly named roots, the definition of hidden names across filesystems, -and application-level policy precedence are described under Open Decisions. -`follow` and `hidden` are the proposed field spellings. +OD-1 (recursion default), OD-4 (discovery boundaries and cycles), OD-5 (policy +precedence), and OD-6 (absent-import evidence) are settled in the sections below. +OD-2 and OD-3 remain open. `follow` and `hidden` are the proposed field spellings; +their applicability to a retained `glob` selector remains part of OD-3. ## Decisions Resolved From Ansible @@ -278,7 +278,11 @@ change resource identity or the effective configuration destination. The selected model uses an explicit search root and recursion flag, rather than introducing recursive `**` syntax or inferring a root from a wildcard prefix. -The default when `recursive` is omitted remains open; examples specify it. + +**OD-1: settled.** Omitting `recursive` means false: search only direct files. +Authors explicitly opt into descendants. This makes the search scope visible +while keeping recursion independent of the permissive defaults for absence, +symlinks, and hidden entries. The settled depth convention is: @@ -289,7 +293,7 @@ The settled depth convention is: - `recursive: false` selects only direct files regardless of a valid depth value. - Depth controls discovery only, not namespaces, identities, or destinations. -Remaining validation and traversal choices are listed below. Recursive discovery +Remaining directive validation choices are listed below. Recursive discovery must fit the existing filesystem containment, repeated-source, and load-budget contracts rather than silently changing them. In particular, examined entries, including nonmatches, still consume the visit budget; a small number of matches @@ -298,11 +302,29 @@ constraint, not a new Ansible-derived decision. ### Symlinks And Hidden Entries -Directory searches default to `follow: true` and `hidden: true`. Authors may -disable either behavior independently of recursion, depth, matching, or absence -handling. `hidden: false` excludes hidden files and prunes hidden directories; -`follow: false` disables following symlinks within the discovery scope. The -precise boundary for explicit roots and file links remains open below. +**OD-4: settled.** Discovery controls are local to the declaration. Directory +searches default to `follow: true` and `hidden: true`, subject to the application +defaults described below. Authors may disable either behavior independently of +recursion, depth, matching, or absence handling. + +Define hidden entries by a leading dot in the portable source namespace. +`hidden: false` skips discovered hidden files and prunes discovered hidden +directories. `follow: false` skips discovered symlink files and prevents descent +through discovered symlink directories. Explicitly named roots and separate +direct-file imports retain their existing source-access semantics. The controls +do not implicitly propagate to declarations inside imported files. + +These are discovery-selection controls, not a load-wide access prohibition. +If OD-2 retains `glob` and OD-3 enables these controls on it, filtering applies +to wildcard-selected entries; explicitly named path components remain explicit +access. This conditional scope rule does not settle either open decision. + +When following links, detect a canonical directory repeated on the active +traversal path and fail with a cycle diagnostic. Do not silently prune the loop +and continue. Existing dependency-cycle and repeated-source errors remain in +force; discovered aliases do not authorize silently deduplicating imported +sources. Directory-cycle handling retains the existing containment and +native-filesystem capabilities. These controls narrow selection rather than authorize ignoring errors in a selected source. `required: true` applies after discovery filters: if the search @@ -313,8 +335,9 @@ not expose it. ## Illustrative Examples These examples describe the proposed interface, not configuration accepted by -the current release. They assume the consumer has enabled imports. File paths -and directory roots resolve from the declaring file; destinations use the +the current release. They assume the consumer has enabled imports and has not +changed the library's policy defaults. File paths and directory roots resolve +from the declaring file; destinations use the existing relative-placement rules. ### Import One File @@ -360,8 +383,8 @@ search, the declaration can also include: hidden: false ``` -The exact treatment of an explicitly named root is an open scope decision, not -an additional change to the settled defaults. +These controls narrow discovery below the explicitly named root; the root +continues to resolve under the existing source-access rules. ### Require A Recursive Prompt Collection @@ -397,8 +420,8 @@ directory searches, with `required: true` opting into failure on absence. This deliberately supersedes the earlier suggestion to retain required-by-default imports. -The table assumes the library's default absence policy. A consumer-selected -default or enforcement option remains open below. +The table assumes the library's default absence policy. Applications may select +a different default using the settled precedence rules below. | Situation | Proposed behavior | | --- | --- | @@ -423,15 +446,36 @@ contribution. ## Who Chooses Strictness? -The general philosophy permits operators and consuming applications to choose -stricter behavior. Import authors can express local requirements through -`required: true`. This does not yet specify an application-wide override API. +**OD-5: settled.** Provide load-level defaults for `required`, `follow`, and +`hidden`. Resolve each applicable field independently using this precedence: -Keep the distinction between choosing defaults and overriding explicit intent. -If an operator or application should be able to strengthen or relax a requirement -declared in an imported document, define that capability and its precedence -separately. Optional-by-default alone does not imply that an authored requirement -can be overridden, or that requirements propagate to transitive imports. +1. Explicit value on the import declaration. +2. Application-selected default, when supplied. +3. Library default: false for `required`, true for `follow` and `hidden`. + +Apply this rule to transitive declarations too. A declaration does not inherit +these field values from its importing parent. Applications may expose the +defaults to operators through their own interfaces; simple consumers need no +additional setup. Exact Go API spelling belongs to implementation planning. + +Explicit declarations remain authoritative, permitting intentional exceptions +to stricter application defaults. Defaults are not enforcement controls. Do not +add options that override explicit declarations as part of this work without a +separate concrete requirement and scope decision. The fields' applicability to +each selector remains part of OD-3. + +## Evidence For Absent Optional Imports + +**OD-6: settled.** Record minimal structured inspection evidence when an +optional declaration selects nothing: its declaring source/location, selector, +and absent outcome. Emit no warning by default and do not fabricate a +loaded-source record. This keeps optional contributions quiet during ordinary +use while allowing applications to explain omissions during troubleshooting. + +Decide the exact inspection record shape during implementation planning, +reusing existing structures where appropriate and retaining existing inspection +contracts. Include ownership tests for the new evidence. This addition does +not require applications to reconstruct absent-import provenance themselves. ## Release Boundary @@ -448,38 +492,25 @@ scaffolding. No promise is made to preserve every old declaration or use case. ## Open Decisions -The recommendations below are not settled decisions. The new principles resolve -transition support, omitted filename filtering, and the defaults and opt-out -availability for symlinks and hidden entries. Shared dependency-ready ordering -for imports and `extends` is also settled. Six decision areas remain; -closely related validation details are grouped together without treating them -as implicitly approved. Each has at least one viable alternative. +Only OD-2 and OD-3 remain open. Their identifiers are preserved for continued +discussion. The recommendations below reflect the latest review against the +architecture principles; they are not settled decisions. In particular, +multiple patterns, retaining `glob`, and applying discovery controls to `glob` +remain proposals. OD-1, OD-4, OD-5, and OD-6 are settled above. -### 1. Recursion Default - -**Why still open:** Recursion determines search scope independently of absence -handling and permissiveness. Ansible's `find` and `win_find` default to -nonrecursive searches; `include_vars` recursively loads directories by default. - -**Recommendation:** Default `recursive` to false. A declaration explicitly -chooses whether subdirectories contribute. This makes the search scope visible, -at the cost of one extra field for recursive collections. - -**Alternative:** Default to true, following `include_vars`. Collection authors -could reorganize files into subdirectories without revising declarations, but -adding a subdirectory could also add contributions to an existing import. -Neither choice changes the settled `follow`, `hidden`, or `required` defaults. - -### 2. Matching Language And Selector Expressiveness +### OD-2. Matching Language And Selector Expressiveness **Why still open:** Permissiveness does not select a matching grammar. Ansible uses different wildcard and regex languages across its discovery interfaces. Also, basename filters and path globs express different searches; the absence of compatibility support does not decide which capabilities the final API needs. -**Recommendation:** Use one scalar `pattern` with the existing case-sensitive -Go `path.Match` grammar, applied to basenames. Keep `dir` as a literal root and -retain `glob` as a permanent, separate selector for wildcard directory segments +**Recommendation (not yet accepted):** Use the existing case-sensitive Go +`path.Match` grammar for basename matching. Accept a scalar `pattern` or a list +of patterns; a candidate matching multiple patterns in one declaration is +selected once. This does not suppress repeated-source errors from separate +inclusion routes. Keep `dir` as a literal root and retain `glob` as a permanent, +separate selector for wildcard directory segments and exact-depth path selection. Its justification is useful expressiveness, not old-document compatibility. Apply the new absence policy to every selector; retaining `glob` must not introduce a legacy behavior mode. This supports both @@ -492,29 +523,32 @@ Users needing selected directory names or depths would use multiple explicit imports or application-side source selection. That is a real capability tradeoff, acceptable if those use cases do not justify a separate selector. -**Filter alternative:** Allow a list of patterns unioned within one declaration, -and/or an explicit Go regex mode. This supports suffix unions or richer filters -without overlapping imports, but adds syntax, matching rules, and validation. -Separate scalar declarations are simpler, though overlapping selections remain -subject to source-repeat errors. Regex compatibility with Python or PowerShell -is not implied. None of these choices changes the settled all-filenames default -when a filter is omitted. +**Filter alternatives:** Restrict `pattern` to a scalar, keeping the parser +smaller but requiring separate declarations for suffix unions; overlapping +selections remain subject to source-repeat errors. An explicit Go regex mode +would support richer filters but add another dialect and validation rules. +Recommend deferring regex until a concrete requirement justifies it. Regex +compatibility with Python or PowerShell is not implied. None of these choices +changes the settled all-filenames default when a filter is omitted. -### 3. Directive Validation +### OD-3. Directive Validation **Why still open:** Permissive discovery does not determine whether malformed or inapplicable option values should be accepted. The draft's existing error boundary remains: invalid declarations fail. We still need to define their valid shapes, rather than infer extra restrictions from that boundary. -**Recommendation:** Accept nonnegative YAML integers for `depth`; zero remains -unlimited. Treat negative, null, string, boolean, and fractional values as +**Recommendation (not yet accepted):** Accept nonnegative YAML integers for +`depth`; zero remains unlimited. Treat negative, null, string, boolean, and fractional values as malformed, even when recursion is disabled. Use exactly one selector per -import: `path`, `dir`, or `glob` if selected for the final API. Keep directory -search options (`pattern`, `recursive`, `depth`, `follow`, and `hidden`) on `dir` -and diagnose them on other forms. This makes likely mistakes visible, but -shared templates must omit inapplicable fields. It imposes no arbitrary maximum -depth. The removed `optional` spelling is not accepted under any variant. +import: `path`, `dir`, or `glob` if selected for the final API. Keep +`pattern`, `recursive`, and `depth` specific to `dir`; allow `follow` and `hidden` +on both `dir` and `glob` if the latter is retained. Diagnose unknown fields and +options that have no meaning for the chosen selector. This makes likely mistakes +visible, but shared templates must omit inapplicable fields. It imposes no arbitrary maximum +depth. The settled combination `recursive: false` with a valid larger `depth` +remains accepted and selects only direct files. The removed `optional` spelling +is not accepted under any variant. **Alternative:** Normalize null depth to omission and negative integers to unlimited, and permit unused directory controls on a direct-file declaration. @@ -524,81 +558,3 @@ chosen independently; they do not require accepting ambiguous selectors. Another viable selector shape is to allow multiple selectors as an explicit union, but it needs rules for constructing the graph and handling overlaps that one-selector declarations avoid. - -### 4. Discovery Control Boundaries And Cycles - -**Why still open:** Including hidden entries and following symlinks by default, -with options to disable both, is settled. The principles do not define hidden -names across filesystems, the reach of an opt-out at explicitly named roots, or -the response to recursive directory-link cycles. Ansible does not provide one -consistent set of answers across the inspected interfaces. - -**Recommendation:** Define hidden entries by a leading dot for the portable -source namespace. With `hidden: false`, skip such files and prune such directories -encountered below the search root. With `follow: false`, skip encountered symlink -files and do not descend through encountered symlink directories. An explicitly -named `dir` root still resolves under existing source-access rules, as does a -separate `path` import. The controls affect this declaration's discovery; they -do not propagate to imports declared by selected files. This gives a simple -selection rule and preserves deliberate explicit access, but is not a ban on -all hidden or symlink access in the load. - -When following links, detect a canonical directory repeated on the active -traversal path and fail with a cycle diagnostic. Keep existing errors for -imported sources reached repeatedly through aliases. This makes cycles visible -without silently deduplicating configuration. It adds directory-cycle handling -alongside the existing source-cycle rules; it does not change containment or -native-filesystem capabilities. - -**Alternative:** Make the controls govern the explicit search root and every -component used to reach it too, and classify hidden entries using host attributes -where available. This can serve a broader exclusion policy, but adds filesystem -inspection requirements and differences between host and abstract filesystems. -A consumer-wide enforcement policy would also need explicit precedence rather -than being inferred from one declaration's flags. - -**Cycle alternative:** Prune an edge back to an active ancestor directory and -continue. This permits link-rich trees without failing for an irrelevant loop, -but can hide a layout mistake and needs clearly defined inspection evidence. -It would not justify suppressing repeated-source errors for files actually -selected through distinct routes. - -### 5. Application And Operator Policy Precedence - -**Why still open:** The library defaults and authored opt-outs are settled. -Whether consumers can set different defaults or enforce restrictions over -explicit declarations remains an API choice. Ansible's task controls are not a -direct counterpart to policy over ConfigKit's transitive import graph. - -**Recommendation:** Provide load-level defaults for `required`, `follow`, and -`hidden`, applied independently when a declaration omits the corresponding -field. Library defaults remain false, true, and true respectively. Explicit -per-declaration values win; imports do not inherit these values from their -parent declaration. Applications can expose defaults to operators through their -own interfaces. This enables stricter deployment choices while retaining -intentional exceptions, but does not enforce a universal exclusion rule. - -**Alternative:** Offer enforcement options that override explicit declarations, -for example requiring every import to match or forbidding discovery through -symlinks throughout the load. This supports centrally imposed restrictions but -needs clear conflict diagnostics and scope, particularly for explicit sources. -A smaller alternative is declaration-level controls only initially: simpler -API, but consumers cannot choose common defaults without changing declarations. - -### 6. Evidence For Absent Optional Imports - -**Why still open:** Permitting absence does not settle how it should be exposed -for inspection. Ansible's file lookup, variable loader, and find modules return -different evidence, with no common absent-import model. - -**Recommendation:** Record minimal structured inspection evidence when an -optional declaration selects nothing: declaring source/location, selector, and -absent outcome. Emit no warning by default and do not fabricate a loaded-source -record. This helps distinguish intentional absence from a typo, but requires an -inspection API decision and ownership tests. Decide its exact shape during -implementation planning, reusing existing structures where appropriate. - -**Alternative:** Add no new inspection surface initially; retain loaded-source -evidence and let `required: true` turn absence into a diagnostic when needed. -This keeps the feature smaller, but successful optional omissions are harder to -investigate. Existing inspection contracts remain intact either way.