Settle import defaults, discovery boundaries, and inspection decisions

docs/roadmap/imports.md

@@ -87,7 +87,7 @@ The fields have distinct jobs:
 | `path` | Import one named file, relative to the declaring file |
 | `dir` | Search a directory, relative to the declaring file |
 | `pattern` | Filter candidate files by filename within the searched directories |
-| `recursive` | Include descendant directories when true |
+| `recursive` | Default false; include descendant directories when true |
 | `depth` | Bound directory-search depth; 1 means direct files, 0 or omission means unlimited when recursive |
 | `follow` | Follow symlinks by default; false disables following within the discovery scope defined below |
 | `hidden` | Include hidden entries by default; false excludes hidden files and prunes hidden directories within the discovery scope defined below |
@@ -118,10 +118,10 @@ interfaces, without adopting all their spellings, defaults, or matching modes.
   shared dependency-ready scheduler defined below. This is settled independently
   of the existing implementations and includes refactoring both resolvers.
 
-The defaults and availability of these controls are settled. Their exact scope
+OD-1 (recursion default), OD-4 (discovery boundaries and cycles), OD-5 (policy
-at explicitly named roots, the definition of hidden names across filesystems,
+precedence), and OD-6 (absent-import evidence) are settled in the sections below.
-and application-level policy precedence are described under Open Decisions.
+OD-2 and OD-3 remain open. `follow` and `hidden` are the proposed field spellings;
-`follow` and `hidden` are the proposed field spellings.
+their applicability to a retained `glob` selector remains part of OD-3.
 
 ## Decisions Resolved From Ansible
 
@@ -278,7 +278,11 @@ change resource identity or the effective configuration destination.
 
 The selected model uses an explicit search root and recursion flag, rather than
 introducing recursive `**` syntax or inferring a root from a wildcard prefix.
-The default when `recursive` is omitted remains open; examples specify it.
+
+**OD-1: settled.** Omitting `recursive` means false: search only direct files.
+Authors explicitly opt into descendants. This makes the search scope visible
+while keeping recursion independent of the permissive defaults for absence,
+symlinks, and hidden entries.
 
 The settled depth convention is:
 
@@ -289,7 +293,7 @@ The settled depth convention is:
 - `recursive: false` selects only direct files regardless of a valid depth value.
 - Depth controls discovery only, not namespaces, identities, or destinations.
 
-Remaining validation and traversal choices are listed below. Recursive discovery
+Remaining directive validation choices are listed below. Recursive discovery
 must fit the existing filesystem containment, repeated-source, and load-budget
 contracts rather than silently changing them. In particular, examined entries,
 including nonmatches, still consume the visit budget; a small number of matches
@@ -298,11 +302,29 @@ constraint, not a new Ansible-derived decision.
 
 ### Symlinks And Hidden Entries
 
-Directory searches default to `follow: true` and `hidden: true`. Authors may
+**OD-4: settled.** Discovery controls are local to the declaration. Directory
-disable either behavior independently of recursion, depth, matching, or absence
+searches default to `follow: true` and `hidden: true`, subject to the application
-handling. `hidden: false` excludes hidden files and prunes hidden directories;
+defaults described below. Authors may disable either behavior independently of
-`follow: false` disables following symlinks within the discovery scope. The
+recursion, depth, matching, or absence handling.
-precise boundary for explicit roots and file links remains open below.
+
+Define hidden entries by a leading dot in the portable source namespace.
+`hidden: false` skips discovered hidden files and prunes discovered hidden
+directories. `follow: false` skips discovered symlink files and prevents descent
+through discovered symlink directories. Explicitly named roots and separate
+direct-file imports retain their existing source-access semantics. The controls
+do not implicitly propagate to declarations inside imported files.
+
+These are discovery-selection controls, not a load-wide access prohibition.
+If OD-2 retains `glob` and OD-3 enables these controls on it, filtering applies
+to wildcard-selected entries; explicitly named path components remain explicit
+access. This conditional scope rule does not settle either open decision.
+
+When following links, detect a canonical directory repeated on the active
+traversal path and fail with a cycle diagnostic. Do not silently prune the loop
+and continue. Existing dependency-cycle and repeated-source errors remain in
+force; discovered aliases do not authorize silently deduplicating imported
+sources. Directory-cycle handling retains the existing containment and
+native-filesystem capabilities.
 
 These controls narrow selection rather than authorize ignoring errors in a
 selected source. `required: true` applies after discovery filters: if the search
@@ -313,8 +335,9 @@ not expose it.
 ## Illustrative Examples
 
 These examples describe the proposed interface, not configuration accepted by
-the current release. They assume the consumer has enabled imports. File paths
+the current release. They assume the consumer has enabled imports and has not
-and directory roots resolve from the declaring file; destinations use the
+changed the library's policy defaults. File paths and directory roots resolve
+from the declaring file; destinations use the
 existing relative-placement rules.
 
 ### Import One File
@@ -360,8 +383,8 @@ search, the declaration can also include:
       hidden: false
 ```
 
-The exact treatment of an explicitly named root is an open scope decision, not
+These controls narrow discovery below the explicitly named root; the root
-an additional change to the settled defaults.
+continues to resolve under the existing source-access rules.
 
 ### Require A Recursive Prompt Collection
 
@@ -397,8 +420,8 @@ directory searches, with `required: true` opting into failure on absence. This
 deliberately supersedes the earlier suggestion to retain required-by-default
 imports.
 
-The table assumes the library's default absence policy. A consumer-selected
+The table assumes the library's default absence policy. Applications may select
-default or enforcement option remains open below.
+a different default using the settled precedence rules below.
 
 | Situation | Proposed behavior |
 | --- | --- |
@@ -423,15 +446,36 @@ contribution.
 
 ## Who Chooses Strictness?
 
-The general philosophy permits operators and consuming applications to choose
+**OD-5: settled.** Provide load-level defaults for `required`, `follow`, and
-stricter behavior. Import authors can express local requirements through
+`hidden`. Resolve each applicable field independently using this precedence:
-`required: true`. This does not yet specify an application-wide override API.
 
-Keep the distinction between choosing defaults and overriding explicit intent.
+1. Explicit value on the import declaration.
-If an operator or application should be able to strengthen or relax a requirement
+2. Application-selected default, when supplied.
-declared in an imported document, define that capability and its precedence
+3. Library default: false for `required`, true for `follow` and `hidden`.
-separately. Optional-by-default alone does not imply that an authored requirement
+
-can be overridden, or that requirements propagate to transitive imports.
+Apply this rule to transitive declarations too. A declaration does not inherit
+these field values from its importing parent. Applications may expose the
+defaults to operators through their own interfaces; simple consumers need no
+additional setup. Exact Go API spelling belongs to implementation planning.
+
+Explicit declarations remain authoritative, permitting intentional exceptions
+to stricter application defaults. Defaults are not enforcement controls. Do not
+add options that override explicit declarations as part of this work without a
+separate concrete requirement and scope decision. The fields' applicability to
+each selector remains part of OD-3.
+
+## Evidence For Absent Optional Imports
+
+**OD-6: settled.** Record minimal structured inspection evidence when an
+optional declaration selects nothing: its declaring source/location, selector,
+and absent outcome. Emit no warning by default and do not fabricate a
+loaded-source record. This keeps optional contributions quiet during ordinary
+use while allowing applications to explain omissions during troubleshooting.
+
+Decide the exact inspection record shape during implementation planning,
+reusing existing structures where appropriate and retaining existing inspection
+contracts. Include ownership tests for the new evidence. This addition does
+not require applications to reconstruct absent-import provenance themselves.
 
 ## Release Boundary
 
@@ -448,38 +492,25 @@ scaffolding. No promise is made to preserve every old declaration or use case.
 
 ## Open Decisions
 
-The recommendations below are not settled decisions. The new principles resolve
+Only OD-2 and OD-3 remain open. Their identifiers are preserved for continued
-transition support, omitted filename filtering, and the defaults and opt-out
+discussion. The recommendations below reflect the latest review against the
-availability for symlinks and hidden entries. Shared dependency-ready ordering
+architecture principles; they are not settled decisions. In particular,
-for imports and `extends` is also settled. Six decision areas remain;
+multiple patterns, retaining `glob`, and applying discovery controls to `glob`
-closely related validation details are grouped together without treating them
+remain proposals. OD-1, OD-4, OD-5, and OD-6 are settled above.
-as implicitly approved. Each has at least one viable alternative.
 
-### 1. Recursion Default
+### OD-2. Matching Language And Selector Expressiveness
-
-**Why still open:** Recursion determines search scope independently of absence
-handling and permissiveness. Ansible's `find` and `win_find` default to
-nonrecursive searches; `include_vars` recursively loads directories by default.
-
-**Recommendation:** Default `recursive` to false. A declaration explicitly
-chooses whether subdirectories contribute. This makes the search scope visible,
-at the cost of one extra field for recursive collections.
-
-**Alternative:** Default to true, following `include_vars`. Collection authors
-could reorganize files into subdirectories without revising declarations, but
-adding a subdirectory could also add contributions to an existing import.
-Neither choice changes the settled `follow`, `hidden`, or `required` defaults.
-
-### 2. Matching Language And Selector Expressiveness
 
 **Why still open:** Permissiveness does not select a matching grammar. Ansible
 uses different wildcard and regex languages across its discovery interfaces.
 Also, basename filters and path globs express different searches; the absence
 of compatibility support does not decide which capabilities the final API needs.
 
-**Recommendation:** Use one scalar `pattern` with the existing case-sensitive
+**Recommendation (not yet accepted):** Use the existing case-sensitive Go
-Go `path.Match` grammar, applied to basenames. Keep `dir` as a literal root and
+`path.Match` grammar for basename matching. Accept a scalar `pattern` or a list
-retain `glob` as a permanent, separate selector for wildcard directory segments
+of patterns; a candidate matching multiple patterns in one declaration is
+selected once. This does not suppress repeated-source errors from separate
+inclusion routes. Keep `dir` as a literal root and retain `glob` as a permanent,
+separate selector for wildcard directory segments
 and exact-depth path selection. Its justification is useful expressiveness,
 not old-document compatibility. Apply the new absence policy to every selector;
 retaining `glob` must not introduce a legacy behavior mode. This supports both
@@ -492,29 +523,32 @@ Users needing selected directory names or depths would use multiple explicit
 imports or application-side source selection. That is a real capability
 tradeoff, acceptable if those use cases do not justify a separate selector.
 
-**Filter alternative:** Allow a list of patterns unioned within one declaration,
+**Filter alternatives:** Restrict `pattern` to a scalar, keeping the parser
-and/or an explicit Go regex mode. This supports suffix unions or richer filters
+smaller but requiring separate declarations for suffix unions; overlapping
-without overlapping imports, but adds syntax, matching rules, and validation.
+selections remain subject to source-repeat errors. An explicit Go regex mode
-Separate scalar declarations are simpler, though overlapping selections remain
+would support richer filters but add another dialect and validation rules.
-subject to source-repeat errors. Regex compatibility with Python or PowerShell
+Recommend deferring regex until a concrete requirement justifies it. Regex
-is not implied. None of these choices changes the settled all-filenames default
+compatibility with Python or PowerShell is not implied. None of these choices
-when a filter is omitted.
+changes the settled all-filenames default when a filter is omitted.
 
-### 3. Directive Validation
+### OD-3. Directive Validation
 
 **Why still open:** Permissive discovery does not determine whether malformed or
 inapplicable option values should be accepted. The draft's existing error
 boundary remains: invalid declarations fail. We still need to define their
 valid shapes, rather than infer extra restrictions from that boundary.
 
-**Recommendation:** Accept nonnegative YAML integers for `depth`; zero remains
+**Recommendation (not yet accepted):** Accept nonnegative YAML integers for
-unlimited. Treat negative, null, string, boolean, and fractional values as
+`depth`; zero remains unlimited. Treat negative, null, string, boolean, and fractional values as
 malformed, even when recursion is disabled. Use exactly one selector per
-import: `path`, `dir`, or `glob` if selected for the final API. Keep directory
+import: `path`, `dir`, or `glob` if selected for the final API. Keep
-search options (`pattern`, `recursive`, `depth`, `follow`, and `hidden`) on `dir`
+`pattern`, `recursive`, and `depth` specific to `dir`; allow `follow` and `hidden`
-and diagnose them on other forms. This makes likely mistakes visible, but
+on both `dir` and `glob` if the latter is retained. Diagnose unknown fields and
-shared templates must omit inapplicable fields. It imposes no arbitrary maximum
+options that have no meaning for the chosen selector. This makes likely mistakes
-depth. The removed `optional` spelling is not accepted under any variant.
+visible, but shared templates must omit inapplicable fields. It imposes no arbitrary maximum
+depth. The settled combination `recursive: false` with a valid larger `depth`
+remains accepted and selects only direct files. The removed `optional` spelling
+is not accepted under any variant.
 
 **Alternative:** Normalize null depth to omission and negative integers to
 unlimited, and permit unused directory controls on a direct-file declaration.
@@ -524,81 +558,3 @@ chosen independently; they do not require accepting ambiguous selectors.
 Another viable selector shape is to allow multiple selectors as an explicit
 union, but it needs rules for constructing the graph and handling overlaps that
 one-selector declarations avoid.
-
-### 4. Discovery Control Boundaries And Cycles
-
-**Why still open:** Including hidden entries and following symlinks by default,
-with options to disable both, is settled. The principles do not define hidden
-names across filesystems, the reach of an opt-out at explicitly named roots, or
-the response to recursive directory-link cycles. Ansible does not provide one
-consistent set of answers across the inspected interfaces.
-
-**Recommendation:** Define hidden entries by a leading dot for the portable
-source namespace. With `hidden: false`, skip such files and prune such directories
-encountered below the search root. With `follow: false`, skip encountered symlink
-files and do not descend through encountered symlink directories. An explicitly
-named `dir` root still resolves under existing source-access rules, as does a
-separate `path` import. The controls affect this declaration's discovery; they
-do not propagate to imports declared by selected files. This gives a simple
-selection rule and preserves deliberate explicit access, but is not a ban on
-all hidden or symlink access in the load.
-
-When following links, detect a canonical directory repeated on the active
-traversal path and fail with a cycle diagnostic. Keep existing errors for
-imported sources reached repeatedly through aliases. This makes cycles visible
-without silently deduplicating configuration. It adds directory-cycle handling
-alongside the existing source-cycle rules; it does not change containment or
-native-filesystem capabilities.
-
-**Alternative:** Make the controls govern the explicit search root and every
-component used to reach it too, and classify hidden entries using host attributes
-where available. This can serve a broader exclusion policy, but adds filesystem
-inspection requirements and differences between host and abstract filesystems.
-A consumer-wide enforcement policy would also need explicit precedence rather
-than being inferred from one declaration's flags.
-
-**Cycle alternative:** Prune an edge back to an active ancestor directory and
-continue. This permits link-rich trees without failing for an irrelevant loop,
-but can hide a layout mistake and needs clearly defined inspection evidence.
-It would not justify suppressing repeated-source errors for files actually
-selected through distinct routes.
-
-### 5. Application And Operator Policy Precedence
-
-**Why still open:** The library defaults and authored opt-outs are settled.
-Whether consumers can set different defaults or enforce restrictions over
-explicit declarations remains an API choice. Ansible's task controls are not a
-direct counterpart to policy over ConfigKit's transitive import graph.
-
-**Recommendation:** Provide load-level defaults for `required`, `follow`, and
-`hidden`, applied independently when a declaration omits the corresponding
-field. Library defaults remain false, true, and true respectively. Explicit
-per-declaration values win; imports do not inherit these values from their
-parent declaration. Applications can expose defaults to operators through their
-own interfaces. This enables stricter deployment choices while retaining
-intentional exceptions, but does not enforce a universal exclusion rule.
-
-**Alternative:** Offer enforcement options that override explicit declarations,
-for example requiring every import to match or forbidding discovery through
-symlinks throughout the load. This supports centrally imposed restrictions but
-needs clear conflict diagnostics and scope, particularly for explicit sources.
-A smaller alternative is declaration-level controls only initially: simpler
-API, but consumers cannot choose common defaults without changing declarations.
-
-### 6. Evidence For Absent Optional Imports
-
-**Why still open:** Permitting absence does not settle how it should be exposed
-for inspection. Ansible's file lookup, variable loader, and find modules return
-different evidence, with no common absent-import model.
-
-**Recommendation:** Record minimal structured inspection evidence when an
-optional declaration selects nothing: declaring source/location, selector, and
-absent outcome. Emit no warning by default and do not fabricate a loaded-source
-record. This helps distinguish intentional absence from a typo, but requires an
-inspection API decision and ownership tests. Decide its exact shape during
-implementation planning, reusing existing structures where appropriate.
-
-**Alternative:** Add no new inspection surface initially; retain loaded-source
-evidence and let `required: true` turn absence into a diagnostic when needed.
-This keeps the feature smaller, but successful optional omissions are harder to
-investigate. Existing inspection contracts remain intact either way.