diff --git a/docs/roadmap/imports.md b/docs/roadmap/imports.md index 8d53166..139a5aa 100644 --- a/docs/roadmap/imports.md +++ b/docs/roadmap/imports.md @@ -5,7 +5,7 @@ direction for discussion, not an implemented contract or an implementation plan. The selected direction separates directory searches, filename matching, recursion, and absence handling. Optional-by-default imports are the intended direction. The consumer-directed and Ansible-aligned decisions below are settled; -remaining choices have recommendations and alternatives under Open Decisions. +API spelling and implementation details remain for implementation planning. All new syntax below is illustrative and unimplemented. ## Guiding Philosophy @@ -86,11 +86,11 @@ The fields have distinct jobs: | --- | --- | | `path` | Import a named file or list of named files, relative to the declaring file | | `dir` | Search a literal directory or list of literal directories, relative to the declaring file | -| `pattern` | A filename wildcard or list of wildcards; select files matching any pattern within the searched directories | +| `pattern` | A filename wildcard or list of wildcards; select files matching any effective pattern, using the default precedence below | | `glob` | Select files with one path wildcard, including wildcard directory segments, relative to the declaring file | | `exclude` | A filename wildcard or list of wildcards; remove matching files from any selector's results | -| `recursive` | Default false; include descendant directories when true | -| `depth` | Bound directory-search depth; 1 means direct files, 0 or omission means unlimited when recursive | +| `recursive` | Library default false, overridable by an application default; include descendant directories when effectively true | +| `depth` | Bound directory-search depth; supplying it requires effective recursion to be enabled; 1 means direct files, 0 or omission means unlimited when recursive | | `follow` | Follow symlinks by default; false disables following within the discovery scope defined below | | `hidden` | Include hidden entries by default; false excludes hidden files and prunes hidden directories within the discovery scope defined below | | `required` | For `path`, require every listed file to be present; for `dir` or `glob`, require at least one selected file after exclusions across the complete search | @@ -108,9 +108,10 @@ interfaces, without adopting all their spellings, defaults, or matching modes. it as an alias or normalize old declarations at runtime. - **Optional contributions by default.** `required: true` opts into failure on absence. Supplied invalid configuration continues to fail. -- **No implicit filename restriction.** Omitting `pattern` selects all - filenames. Authors can narrow selection explicitly. There is no default YAML - extension list; extensionless sources remain discoverable. +- **No library-imposed filename restriction.** Omitting `pattern` selects all + filenames unless the application supplies a default pattern. Authors can + override that default explicitly. There is no library default YAML extension + list; extensionless sources remain discoverable. - **Follow symlinks by default, with an opt-out.** Provide `follow: false` for callers who want discovery without symlink following. Applicable containment and filesystem contracts remain in force. @@ -120,11 +121,12 @@ interfaces, without adopting all their spellings, defaults, or matching modes. shared dependency-ready scheduler defined below. This is settled independently of the existing implementations and includes refactoring both resolvers. -OD-1 (recursion default), OD-2 (matching and selector expressiveness), OD-4 -(discovery boundaries and cycles), OD-5 (policy precedence), and OD-6 -(absent-import evidence) are settled in the sections below. Only OD-3 remains -open. `follow` and `hidden` are the proposed field spellings; their applicability -to `glob` remains part of OD-3. Applying `exclude` to `glob` is settled. +OD-1 (recursion default), OD-2 (matching and selector expressiveness), OD-3 +(directive validation), OD-4 (discovery boundaries and cycles), OD-5 (policy +precedence), and OD-6 (absent-import evidence) are settled in the sections below. +Supplying `depth` requires effective recursion to be enabled, including when +enabled through an application default. This supersedes the earlier direction +to accept but ignore depth when recursion is disabled. ## Decisions Resolved From Ansible @@ -153,13 +155,8 @@ The following choices are settled: and [win_find implementation](https://github.com/ansible-collections/ansible.windows/blob/main/plugins/modules/win_find.ps1). The latter two document unlimited omission more clearly than explicit zero; this choice uses source-confirmed behavior, not a claimed universal documented - Ansible guarantee. Negative values remain a separate open decision. -- **Disabling recursion takes precedence over depth.** With `recursive: false`, - search only direct files, even if a valid larger or unlimited depth is supplied. - Depth does not enable recursion. Both - [find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/find_module.html#parameters) - and [win_find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/windows/win_find_module.html#parameters) - explicitly use this rule where a separate recursion switch exists. + Ansible guarantee. Negative values are invalid under the settled directive + validation rules below. - **Directory searches select files, not directory documents.** Directories are traversal containers, even when their names match `pattern`. This agrees with the default file selection in the find modules and the file-oriented @@ -267,9 +264,10 @@ Recursion changes which directories are visited; it does not change what `*` means. A nonrecursive search considers only files directly inside `dir`. **OD-2: settled.** Use the existing case-sensitive Go `path.Match` wildcard -grammar for filename filters. Omitting `pattern` selects all filenames, with no -implicit extension restriction. Do not add regular expressions, automatic -dialect detection, or recursive `**` syntax as part of this work. +grammar for filename filters. When neither the declaration nor the application +supplies `pattern`, select all filenames, with no implicit extension restriction. +Do not add regular expressions, automatic dialect detection, or recursive `**` +syntax as part of this work. Retain `glob` as a permanent, separate selector using the existing slash-based path wildcard grammar. For example, `collections/*/prompts/*.yaml` selects @@ -294,12 +292,15 @@ D × P and take the **union** of the matching candidate files: `S = ⋃ {matches(d, p) | (d, p) ∈ D × P}` Each root uses the declaration's recursion, depth, and applicable discovery -controls. When `pattern` is omitted, select all filenames in the visited -directories instead. This describes selection semantics, not a requirement to +controls. Resolve an omitted `pattern` from the application default first; +if neither supplies patterns, select all filenames in the visited directories +instead. This describes selection semantics, not a requirement to materialize the product or traverse a directory again for each pattern. Overlapping patterns or directory searches within one declaration select the -same candidate once. This local set union is not global source deduplication: +same candidate once. Repeated identical entries in one `path` list likewise +select that candidate once; duplicate patterns or exclusions have no additional +effect. This local deduplication is not global source deduplication: existing repeated-source and alias checks still apply to distinct inclusion routes. In particular, it does not authorize silently merging separate alias paths to an imported source. @@ -334,7 +335,8 @@ prohibit another declaration from explicitly selecting the same filename. ### Recursive Discovery And Depth Limits -Use `recursive: true` for recursive discovery, optionally bounded by `depth`. +Enable recursion explicitly or through an application default for recursive +discovery, optionally bounded by `depth`. The motivation is to let authors organize definitions in subdirectories without updating import declarations for every depth. Directory organization should not change resource identity or the effective configuration destination. @@ -342,10 +344,10 @@ change resource identity or the effective configuration destination. The selected model uses an explicit search root and recursion flag, rather than introducing recursive `**` syntax or inferring a root from a wildcard prefix. -**OD-1: settled.** Omitting `recursive` means false: search only direct files. -Authors explicitly opt into descendants. This makes the search scope visible -while keeping recursion independent of the permissive defaults for absence, -symlinks, and hidden entries. +**OD-1: settled.** The library default for `recursive` is false: search only +direct files. An application may supply a different default; an explicit +declaration value overrides it, including `recursive: false`. Recursion remains +independent of the permissive defaults for absence, symlinks, and hidden entries. The settled depth convention is: @@ -353,11 +355,21 @@ The settled depth convention is: - Each additional directory increases depth by one. - Omitting `depth`, or setting it to 0, imposes no caller-selected depth limit when recursion is enabled; existing loader resource limits still apply. -- `recursive: false` selects only direct files regardless of a valid depth value. +- Supplying `depth` requires effective `recursive: true`. If recursion is false + after resolving declaration, application, and library defaults, any supplied + `depth` is an error, including `0` or `1`. Depth never enables recursion itself. +- With recursion disabled and `depth` omitted, select only direct files. - Depth controls discovery only, not namespaces, identities, or destinations. -Remaining directive validation choices are listed below. Recursive discovery -must fit the existing filesystem containment, repeated-source, and load-budget +Resolve the effective recursion value before checking depth's applicability. +For example, `depth: 2` with no explicit `recursive` is valid if the application +default enables recursion, but invalid with the library default. Explicit +`recursive: false` overrides an application default of true and makes any +supplied depth invalid. Depth must also have the valid type and range defined +under directive validation; enabling recursion does not excuse malformed depth. + +Recursive discovery must fit the existing filesystem containment, +repeated-source, and load-budget contracts rather than silently changing them. In particular, examined entries, including nonmatches, still consume the visit budget; a small number of matches does not authorize an unbounded filesystem scan. This is an existing ConfigKit @@ -378,9 +390,8 @@ direct-file imports retain their existing source-access semantics. The controls do not implicitly propagate to declarations inside imported files. These are discovery-selection controls, not a load-wide access prohibition. -If OD-3 enables these controls on `glob`, filtering applies -to wildcard-selected entries; explicitly named path components remain explicit -access. This conditional scope rule does not settle that open decision. +On `glob`, filtering applies to wildcard-selected entries; explicitly named +path components remain explicit access. When following links, detect a canonical directory repeated on the active traversal path and fail with a cycle diagnostic. Do not silently prune the loop @@ -449,8 +460,8 @@ keys. Missing directories or no matching files contribute nothing. Adding `depth: 2` limits discovery to direct files and files one directory below `services`: it still selects `services/regional/worker.yaml`, but not -`services/regional/east/worker.yaml`. With `recursive: false`, only direct files -are selected, even if `depth: 2` remains present. +`services/regional/east/worker.yaml`. Switching to `recursive: false` requires +omitting `depth`; leaving `depth: 2` present is a declaration error. Hidden entries and contained symlinks participate by default. To narrow the search, the declaration can also include: @@ -550,12 +561,22 @@ contribution. ## Who Chooses Strictness? -**OD-5: settled.** Provide load-level defaults for `required`, `follow`, and -`hidden`. Resolve each applicable field independently using this precedence: +**OD-5: settled.** Provide load-level defaults for `required`, `follow`, `hidden`, +`recursive`, and `pattern`. Resolve each applicable field independently using +this precedence: 1. Explicit value on the import declaration. 2. Application-selected default, when supplied. -3. Library default: false for `required`, true for `follow` and `hidden`. +3. Library default: false for `required` and `recursive`, true for `follow` and + `hidden`, and no filename restriction for `pattern`. + +An explicit `pattern` replaces the application default rather than combining +with it. In particular, `pattern: []` selects nothing, even when the application +supplies patterns. An application can default to `['*.yml', '*.yaml']`; without +either an application default or an explicit pattern, a directory search +attempts to load all files in its search scope. Application defaults apply only +to selectors that support the field; they do not add inapplicable options to +other selectors. Apply this rule to transitive declarations too. A declaration does not inherit these field values from its importing parent. Applications may expose the @@ -566,7 +587,7 @@ Explicit declarations remain authoritative, permitting intentional exceptions to stricter application defaults. Defaults are not enforcement controls. Do not add options that override explicit declarations as part of this work without a separate concrete requirement and scope decision. The fields' applicability to -each selector remains part of OD-3. +each selector is defined under directive validation. ## Evidence For Absent Optional Imports @@ -616,55 +637,75 @@ in its implementation documentation. Do not label implementation-specific matching or cycle-path helpers with an algorithm name until that algorithm is actually chosen. -## Open Decisions +## Directive Validation -Only OD-3 remains open. Its identifier is preserved for continued discussion; -the recommendations below are not settled decisions. OD-1, OD-2, OD-4, OD-5, -and OD-6 are settled above. +**OD-3: settled.** Validate declaration shapes and values independently of +whether discovery selects any files. Invalid declarations fail even when a +search root is missing or a list is empty. This does not require opening or +parsing excluded files or establish a new first-error ordering guarantee. -### OD-3. Directive Validation +### Selectors And Applicable Options -**Why still open:** Permissive discovery does not determine whether malformed or -inapplicable option values should be accepted. The draft's existing error -boundary remains: invalid declarations fail. We still need to define their -valid shapes, rather than infer extra restrictions from that boundary. +Require exactly one selector per declaration: `path`, `dir`, or `glob`. A list +inside `path` or `dir` still counts as one selector. Use separate declarations +when combining different selector kinds. Diagnose unknown fields and options +that have no meaning for the chosen selector. The removed `optional` spelling +is not accepted. -**Recommendation (not yet accepted):** Accept nonnegative YAML integers for -`depth`; zero remains unlimited. Treat negative, null, string, boolean, and -fractional values as malformed, even when recursion is disabled. Use exactly one selector per -import: `path`, `dir`, or `glob`. Keep -`pattern`, `recursive`, and `depth` specific to `dir`; allow `follow` and `hidden` -on both `dir` and `glob`. `exclude` applies to all three selectors as settled -above. Diagnose unknown fields and options that have no meaning for the chosen -selector. This makes likely mistakes visible, but shared templates must omit -inapplicable fields. It imposes no arbitrary maximum depth. The settled -combination `recursive: false` with a valid larger `depth` -remains accepted and selects only direct files. The removed `optional` spelling -is not accepted under any variant. +| Option | `path` | `dir` | `glob` | +| --- | --- | --- | --- | +| `required`, `exclude` | Yes | Yes | Yes | +| `destination`, `placement` | Yes | Yes | Yes | +| `pattern` | No | Yes | No | +| `recursive`, `depth` | No | Yes | No | +| `follow`, `hidden` | No | Yes | Yes | -The accepted list forms also need explicit validation rules for empty lists, -null values, empty strings, and non-string elements. Recommend allowing empty -lists as empty selections (and `exclude: []` as no exclusions), while rejecting -null values and non-string elements rather than silently coercing them. Under -that recommendation, an explicit `pattern: []` matches nothing, unlike omission; -an empty `path` list has no files whose presence is required. Empty-string -semantics should be reviewed against the existing source-path and wildcard -rules rather than introducing a blanket prohibition. These remain OD-3 choices, -not additional settled restrictions. +### Types And Depth Applicability -**Alternative:** Normalize null depth to omission and negative integers to -unlimited, and permit unused directory controls on a direct-file declaration. -That eases generated configuration but can turn an erroneous bound into a wider -search or make a direct-file import appear recursive. These relaxations can be -chosen independently; they do not require accepting ambiguous selectors. -Another viable selector shape is to allow multiple selectors as an explicit -union, but it needs rules for constructing the graph and handling overlaps that -one-selector declarations avoid. +- `path`, `dir`, `pattern`, and `exclude` accept strings or flat lists of strings. +- `glob` accepts a string. +- `required`, `recursive`, `follow`, and `hidden` accept booleans. +- `depth` accepts nonnegative integers under ConfigKit's existing YAML scalar + rules. Zero means unlimited when recursion is enabled; negative, string, + boolean, and fractional values are invalid. Do not introduce an arbitrary + maximum depth. +- Explicit null is invalid for these fields, including null list members. + Do not coerce non-string members to strings or flatten nested lists. +- Existing `destination` and `placement` validation remains unchanged. -For list validation, another viable choice is to reject empty positive-selector -or pattern lists as likely authoring mistakes while allowing `exclude: []`. -That catches accidentally empty generated inputs, but requires conditional -omission of declarations and is less consistent with permissive empty searches. -Normalizing null to omission would ease generated configuration, but for -`pattern` it would broaden selection to all filenames. Neither alternative -changes the settled string-or-list support for nonempty valid values. +A supplied `depth` additionally requires effective recursion to be enabled, +using the default precedence defined above. Valid type and range alone do not +make depth applicable. This rule applies even to empty directory or pattern +lists. With recursion enabled, malformed depth values still fail validation. + +### Empty Lists, Empty Strings, And Duplicates + +Empty lists are valid and retain their set meanings: + +| Declaration | Meaning | Effect of `required: true` | +| --- | --- | --- | +| `path: []` | No named files | Presence requirement satisfied: no listed file is missing | +| `dir: []` | No search roots | Fail: no selected file | +| `dir: fragments` with `pattern: []` | No positive patterns; select nothing | Fail: no selected file | +| `exclude: []` | Exclude nothing | Otherwise unchanged | + +An empty pattern list differs from omission: omission uses the application +default if supplied, otherwise all filenames. An explicit empty list selects +nothing and overrides any application default. + +Empty strings are invalid for `path`, `dir`, and `glob`, including empty entries +in path or directory lists. Use `.` to explicitly name the declaring directory +as a search root. Empty strings in `pattern` and `exclude` are valid wildcard +expressions that match no actual filename. Other source-path and wildcard +validation rules still apply. + +Repeated identical path entries within one declaration select that candidate +once. Repeated patterns or exclusions have no additional effect, and directory +search overlaps follow the settled set-union rule. These rules do not suppress +existing repeated-source errors across separate declarations or collapse +separate symlink aliases. + +Validate every supplied wildcard expression even if it is inactive because +another list is empty or discovery finds no files. Likewise, malformed list +members remain errors even when other entries are valid. Do not silently drop +invalid members or treat malformed filters as absent.