# Import Ergonomics And Permissive Defaults

Status: working draft / brainstorming notes. This document captures a proposed
direction for discussion, not an implemented contract or an implementation plan.
The selected direction separates directory searches, filename matching,
recursion, and absence handling. Optional-by-default imports are the intended
direction. The Ansible-aligned decisions below are settled for this roadmap;
remaining choices have recommendations and alternatives under Open Decisions.
All new syntax below is illustrative and unimplemented.

## Guiding Philosophy

ConfigKit should provide predictable configuration mechanisms and avoid imposing
application policy. Its defaults should permit optional contributions to be
absent, with explicit options for increased strictness when needed. That choice
may come from an operator or the consuming application; authored import
declarations can also express requirements on contributions.

Permissiveness should distinguish absence from invalid supplied configuration:

> Optional contributions may be absent by default. Consumers can explicitly
> require them. Supplied configuration must still be structurally valid and
> interpretable under the selected composition rules.

This is a general design direction worth making more explicit and consistent
across ConfigKit. It is not blanket authorization to weaken existing contracts,
ignore errors, or add a global permissiveness switch. Other defaults should be
evaluated individually against this principle.

## Current Baseline

The [current import contract](../config.md#explicit-imports) uses `path` or
`glob`, with required imports by default and `optional: true` to permit absence.
Glob matching uses slash-based Go `path.Match` syntax. Recursive `**` syntax is
not supported; authors can match particular depths with patterns such as
`prompts/*.yaml` and `prompts/*/*.yaml`.

The proposed changes concern naming, discovery, and absence handling. Import
destinations, composition, precedence, and resource identity remain separate
concerns.

## Proposed Import Vocabulary

Use a direct-file form and a directory-search form. The latter separates the
search root from its filename filter rather than inferring traversal from a
combined path pattern:

```yaml
configkit:
  imports:
    - dir: prompts
      pattern: '*.yaml'
      recursive: true
      destination: [prompts]
      placement: sequence-item
    - path: defaults.yaml
      required: true
```

The first declaration searches `prompts` and its descendants for matching
filenames, placing each source mapping as one prompt item, and permits no
matches. The second explicitly requires its target file. Sequence composition
still requires the consumer's existing registration, as explained below.

The fields have distinct jobs:

| Field | Proposed meaning |
| --- | --- |
| `path` | Import one named file, relative to the declaring file |
| `dir` | Search a directory, relative to the declaring file |
| `pattern` | Filter candidate files by filename within the searched directories |
| `recursive` | Include descendant directories when true |
| `depth` | Bound directory-search depth; 1 means direct files, 0 or omission means unlimited when recursive |
| `required` | Fail when the declaration contributes no matching file |
| `destination`, `placement` | Apply the existing placement rules to each selected source |

This follows the familiar separation of traversal and filtering in Ansible's
[find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/find_module.html)
and [include_vars](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/include_vars_module.html)
interfaces, without adopting all their spellings, defaults, or matching modes.

## Decisions Resolved From Ansible

The comparison covers `ansible.builtin.find`, `ansible.builtin.include_vars`,
`ansible.builtin.fileglob`, and `ansible.windows.win_find`. A consistent approach
means agreement among the relevant interfaces that offer the capability, not
that every Ansible module exposes it. Existing ConfigKit contracts and the
already selected optional-by-default direction remain authoritative; this is
not a general Ansible compatibility mode.

The following choices are settled:

- **Separate traversal from filename filtering.** Directory roots and filename
  filters have separate roles; filters operate on basenames in visited
  directories. A directory failing the filename filter does not prune its
  descendants. This agrees with the directory-search interfaces above.
  See [find's parameters](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/find_module.html#parameters)
  and the [include_vars implementation](https://github.com/ansible/ansible/blob/stable-2.20/lib/ansible/plugins/action/include_vars.py).
- **Include an optional integer `depth` control.** All three directory-search
  interfaces use that spelling. Direct files are at depth 1, and each additional
  directory adds one. Omission permits unlimited depth when recursion is
  enabled. See [include_vars](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/include_vars_module.html#parameters)
  and [win_find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/windows/win_find_module.html#parameters).
- **Treat `depth: 0` as unlimited.** This follows `include_vars` and agrees with
  the current [find implementation](https://github.com/ansible/ansible/blob/stable-2.20/lib/ansible/modules/find.py)
  and [win_find implementation](https://github.com/ansible-collections/ansible.windows/blob/main/plugins/modules/win_find.ps1).
  The latter two document unlimited omission more clearly than explicit zero;
  this choice uses source-confirmed behavior, not a claimed universal documented
  Ansible guarantee. Negative values remain a separate open decision.
- **Disabling recursion takes precedence over depth.** With `recursive: false`,
  search only direct files, even if a valid larger or unlimited depth is supplied.
  Depth does not enable recursion. Both
  [find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/find_module.html#parameters)
  and [win_find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/windows/win_find_module.html#parameters)
  explicitly use this rule where a separate recursion switch exists.
- **Directory searches select files, not directory documents.** Directories are
  traversal containers, even when their names match `pattern`. This agrees with
  the default file selection in the find modules and the file-oriented
  [fileglob lookup](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/fileglob_lookup.html).
  Existing regular-file and symlink rules still govern opening selected sources;
  the legacy `glob` contract is not implicitly changed by this new search form.

The accepted `recursive` spelling is retained. The formerly tentative
`max_depth` spelling becomes `depth`. Resource budgets remain separate from
depth, and the current limits still apply to an unlimited-depth search.

## Discovery Semantics

### Filename Matching

`pattern` applies to a file's basename, not its full relative path. For example,
`*.yaml` matches both `prompts/one.yaml` and
`prompts/weather/daily/summary.yaml` when the latter's directory is visited.
Recursion changes which directories are visited; it does not change what `*`
means. A nonrecursive search considers only files directly inside `dir`.

Recommend retaining the familiar `path.Match` wildcard grammar for filename
filters. The name `pattern` does not by itself imply regular expressions or
automatic dialect detection. Exact filter defaults and the treatment of
existing path-glob use cases remain open below.

### Recursive Discovery And Depth Limits

Use `recursive: true` for recursive discovery, optionally bounded by `depth`.
The motivation is to let authors organize definitions in subdirectories without
updating import declarations for every depth. Directory organization should not
change resource identity or the effective configuration destination.

The selected model uses an explicit search root and recursion flag, rather than
introducing recursive `**` syntax or inferring a root from a wildcard prefix.
The default when `recursive` is omitted remains open; examples specify it.

The settled depth convention is:

- Files directly inside the search root are at depth `1`.
- Each additional directory increases depth by one.
- Omitting `depth`, or setting it to 0, imposes no caller-selected depth limit
  when recursion is enabled; existing loader resource limits still apply.
- `recursive: false` selects only direct files regardless of a valid depth value.
- Depth controls discovery only, not namespaces, identities, or destinations.

Remaining validation and traversal choices are listed below. Recursive discovery
must fit the existing filesystem containment, repeated-source, and load-budget
contracts rather than silently changing them. In particular, examined entries,
including nonmatches, still consume the visit budget; a small number of matches
does not authorize an unbounded filesystem scan. This is an existing ConfigKit
constraint, not a new Ansible-derived decision.

## Illustrative Examples

These examples describe the proposed interface, not configuration accepted by
the current release. They assume the consumer has enabled imports. File paths
and directory roots resolve from the declaring file; destinations use the
existing relative-placement rules.

### Import One File

```yaml
configkit:
  imports:
    - path: defaults.yaml
```

Import `defaults.yaml` into the declaring source's mapping scope. Under the
proposed default, a missing file contributes nothing. Add `required: true` when
its presence is necessary. A present but invalid file still fails loading.

### Recursively Discover Mapping Fragments

```yaml
configkit:
  imports:
    - dir: services
      pattern: '*.yaml'
      recursive: true
      destination: [services]
```

This discovers files such as `services/api.yaml` and
`services/regional/worker.yaml`, but not `services/README.md`. Each file contains
named service entries, for example `api: {port: 8080}` or
`worker: {concurrency: 4}`. They compose beneath `services` under the existing
strict composition rules. Directory names do not create additional mapping
keys. Missing directories or no matching files contribute nothing.

Adding `depth: 2` limits discovery to direct files and files one directory below
`services`: it still selects `services/regional/worker.yaml`, but not
`services/regional/east/worker.yaml`. With `recursive: false`, only direct files
are selected, even if `depth: 2` remains present.

### Require A Recursive Prompt Collection

```yaml
configkit:
  imports:
    - dir: prompts
      pattern: '*.prompt.yaml'
      recursive: true
      required: true
      destination: [prompts]
      placement: sequence-item
```

This selects `prompts/welcome.prompt.yaml` and
`prompts/weather/daily/summary.prompt.yaml`, but not
`prompts/weather/helpers.yaml`. Each selected file contains one prompt mapping,
such as `id: prompt.welcome` and `text: Welcome`. The consumer registers the
`prompts` sequence with identity field `id` through the existing
`Definition.SequenceCompositions` API. Distinct IDs compose into one sequence;
duplicate IDs fail strict composition. Overlays continue replacing sequences.

Here `required: true` means at least one file must match across the entire
search, not that every visited directory must contain a matching file. The
filename convention keeps helper files out of this discovery declaration; it
does not prevent a selected file from explicitly importing a helper. Existing
repeated-source rules still apply if declarations select overlapping sources.

## Optional Imports By Default

The preferred direction is optional-by-default for both explicit files and
directory searches, with `required: true` opting into failure on absence. This
deliberately supersedes the earlier suggestion to retain required-by-default
imports.

The table assumes the library's default absence policy. A consumer-selected
default or enforcement option remains open below.

| Situation | Proposed behavior |
| --- | --- |
| Missing path; `required` omitted or false | Contribute nothing |
| Missing search directory or no matching files; `required` omitted or false | Contribute nothing |
| Missing file, missing search directory, or zero matches with `required: true` | Fail with an import diagnostic |
| Invalid pattern or malformed import declaration | Fail |
| Permission failure, unreadable matched file, or malformed YAML | Fail |
| Composition conflict or other invalid supplied configuration | Fail under the applicable existing rules |

An optional declaration is permission for absence, not permission to ignore a
present contribution that cannot be read or interpreted. Optionality must not
turn processing failures into apparently successful partial configuration.

This default supports installations with different optional components without
requiring ceremony for every potentially absent directory or file. Its tradeoff
is that a misspelled path or pattern can also contribute nothing. Authors and
applications that need an import to exist can state that requirement explicitly;
applications remain responsible for validating their necessary final values.
Final validation does not automatically detect every unintentionally omitted
contribution.

## Who Chooses Strictness?

The general philosophy permits operators and consuming applications to choose
stricter behavior. Import authors can express local requirements through
`required: true`. This does not yet specify an application-wide override API.

Keep the distinction between choosing defaults and overriding explicit intent.
If an operator or application should be able to strengthen or relax a requirement
declared in an imported document, define that capability and its precedence
separately. Optional-by-default alone does not imply that an authored requirement
can be overridden, or that requirements propagate to transitive imports.

## Migration

Changing absence defaults changes behavior even when existing YAML contains no
optionality field. Renaming keys also changes the accepted syntax. Pre-1.0 is a
useful time to reconsider these contracts, but adoption still needs explicit
migration guidance.

To preserve current intent, currently required imports would acquire
`required: true`; existing `optional: true` declarations could omit the new field
or use `required: false`. A simple `glob: prompts/*.yaml` becomes `dir: prompts`
plus `pattern: '*.yaml'` and `recursive: false`, preserving its discovery scope.
This is not a mechanical rename of `glob` to `pattern`: current globs can also
select particular directory names or depths. Their compatibility path must be
settled before removing any existing form.

## Open Decisions

The recommendations below are not settled decisions. Each concerns either
inconsistent Ansible behavior or a ConfigKit-specific policy without a direct
Ansible counterpart. They do not reopen the accepted absence policy, placement,
strict composition, or source-version rules.

### 1. Recursion Default

**Ansible evidence:** `find` and `win_find` default to nonrecursive searches;
`include_vars` recursively loads directories by default. There is no common
default across these interfaces. See their parameter references above.

**Recommendation:** Default `recursive` to false. This keeps discovery local
unless the declaration explicitly requests descendants and makes migration from
simple current globs straightforward. The cost is one extra field for recursive
collections.

**Alternative:** Default to true, following `include_vars`. Collection authors
could reorganize files into subdirectories without revising declarations, but
adding a subdirectory could also add contributions to an existing import.

### 2. Matching Language And Filter Shape

**Ansible evidence:** `find` supports shell patterns and optional regex;
`include_vars.files_matching` is a regex; `win_find` uses PowerShell wildcards or
regex. Their case and matching details differ. There is no single exact grammar.

**Recommendation:** Keep one scalar `pattern` using ConfigKit's existing
case-sensitive Go `path.Match` grammar, applied to basenames. This reuses a
familiar contract, including escapes and character classes, without introducing
regex mode or platform-dependent case rules. It cannot express every regex or
an arbitrary union of suffixes in one declaration; separate declarations remain
available, subject to existing repeated-source rules.

**Alternative:** Add an explicit regex mode and/or a list of patterns whose
matches are unioned within one declaration. This better supports filters such
as both `.yaml` and `.yml` without overlapping imports, but adds syntax, matching
rules, and validation. Regex would need a specified Go grammar rather than a
promise of Python or PowerShell compatibility. Exclusion filters could accompany
that design, but are not assumed in the initial recommendation.

### 3. Omitted Filename Filter

**Ansible evidence:** The find modules permit unrestricted filename selection.
`include_vars` also has an extension policy, defaulting to `json`, `yaml`, and
`yml`, with separate handling of unknown extensions. Its loading behavior is
not equivalent to a general-purpose find operation.

**Recommendation:** Omitted `pattern` means all filenames, with no implicit
extension filter. This keeps selection independent of filename conventions and
supports extensionless configuration. A selected README or other invalid file
still causes a load error; authors of mixed directories should supply a filter.

**Alternative:** Default discovery to `.yaml` and `.yml`, with an explicit way
to select all files or other extensions. This suits typical configuration
directories and avoids incidental README selection, but adds an extension
policy and an override mechanism that explicit `path` imports do not need.

### 4. Invalid Depth Values

**Ansible evidence:** Positive depth, unlimited omission, and disabled-recursion
precedence have useful common semantics. The implementations do not provide a
consistent negative-depth contract; accepting a value is not evidence of a
useful shared meaning. See the implementation links above.

**Recommendation:** Accept nonnegative YAML integers only. Zero means unlimited;
negative, null, string, boolean, or fractional values are malformed declarations,
even when recursion is disabled. This follows ConfigKit's typed directive style
and gives likely mistakes a direct diagnostic without imposing an arbitrary
maximum depth.

**Alternative:** Treat negative integers as unlimited too, and null as omission.
That can ease generated configurations but adds redundant spellings and risks
turning an erroneous bound into a wider search. Neither interpretation is
required by the settled rule that disabled recursion ignores a valid bound.

### 5. Hidden Files And Directories

**Ansible evidence:** The find modules expose hidden-file filtering, while
`include_vars` has no equivalent hidden-entry exclusion in its directory walk.
Also, filtering a hidden entry is not necessarily the same as pruning a hidden
directory's descendants. See
[find](https://github.com/ansible/ansible/blob/stable-2.20/lib/ansible/modules/find.py)
and [include_vars](https://github.com/ansible/ansible/blob/stable-2.20/lib/ansible/plugins/action/include_vars.py).

**Recommendation:** Give dot-prefixed names no special treatment: visit their
directories within the selected depth and select files by `pattern`. This fits
the existing matcher and avoids an implicit exclusion. Broad recursive searches
may include configuration in hidden directories, so the search root and filter
should reflect the intended collection.

**Alternative:** Add `hidden`, defaulting to false, with an explicit definition
covering both hidden-file selection and hidden-directory pruning. This can
exclude incidental metadata directories, but adds policy and a platform choice
between dot prefixes and filesystem hidden attributes. Callers would need to
opt in when their collections intentionally use hidden names.

### 6. Directory Symlinks And Traversal Cycles

**Ansible evidence:** The find modules default to not following directory links;
the inspected `include_vars` implementation walks with link following enabled.
There is no common default to adopt.

**Recommendation:** Follow contained directory symlinks using ConfigKit's
existing canonicalization and containment rules, preserving the current ability
to select through contained symlinks. For the new recursive walk, detect a
canonical directory repeated on the active traversal path and report a cycle.
Do not silently deduplicate imported files reached through aliases; existing
source-repeat errors remain applicable. This supports linked collections but
requires directory-cycle handling in addition to import-cycle handling.

**Alternative:** Add a traversal `follow` option defaulting to false, skipping
directory symlinks encountered below the search root unless enabled. Keep
explicitly named roots and file targets under existing source-opening rules.
This reduces accidental traversal through linked trees, but introduces a new
distinction between discovery and explicit access and requires callers to opt
in for linked collections. Both choices retain native-filesystem semantics;
neither promises symlink capabilities absent from the supplied filesystem.

### 7. Match Ordering

**Ansible evidence:** `include_vars` sorts directories and filenames;
`win_find` documents results sorted by path. POSIX `find` describes top-down
searching and does not promise the same global ordering. These are not identical
ordering contracts.

**Recommendation:** Sort all selected slash-relative paths lexicographically
within each declaration, then use existing depth-first import expansion. This
extends ConfigKit's existing glob order and stabilizes sequence assembly and
diagnostics. It requires collecting bounded candidates before expansion.

**Alternative:** Load direct files first, then visit sorted subdirectories
recursively. This offers a natural directory-oriented traversal and can begin
loading sooner, but differs from existing global match ordering and can change
sequence item positions. Neither choice grants replacement precedence within
strict composition.

### 8. Application And Operator Strictness Controls

**Ansible evidence:** Task failure controls and module-specific options are not
a direct counterpart to defaults for ConfigKit's transitive import graph.
Ansible cannot settle this public API choice.

**Recommendation:** Add a load-level default for whether imports are required,
used only when a declaration omits `required`. The library default stays false;
an application can choose true and expose that choice to operators through its
own interface. Apply the default independently to each declaration, including
transitive imports. Explicit true or false remains authoritative. This enables
stricter deployments while preserving intentionally optional imports; it does
not enforce that every authored optional declaration succeeds.

**Alternative:** Offer an enforcement mode requiring every declaration to
match, including those explicitly marked `required: false`. That provides a
stronger audit policy but overrides authors' intent and needs clear precedence
and diagnostics. A smaller alternative is authored requirements only for now;
it avoids a public option but leaves applications unable to choose a common
default without editing source declarations.

### 9. Evidence For Absent Optional Imports

**Ansible evidence:** `fileglob` returns an empty result for no matches;
`include_vars` reports included filenames; the find modules expose richer search
results. There is no common inspection model for an absent import declaration.

**Recommendation:** Record minimal structured inspection evidence for an
optional declaration that selects nothing: its declaring source/location,
selector, and absent outcome. Emit no warning by default and do not fabricate a
loaded-source record. This helps distinguish intentional absence from a typo,
but requires an inspection API decision and associated ownership tests. Design
the exact record shape during implementation planning, using existing evidence
structures where appropriate.

**Alternative:** Add no new inspection surface initially; retain loaded-source
evidence and let `required: true` turn absence into a diagnostic when needed.
This keeps the feature smaller, but successful optional omissions are harder to
investigate. Existing inspection contracts must remain intact either way.

### 10. Selector Combinations And Directory-Only Options

**Ansible evidence:** `include_vars` separates file-only and directory-only
arguments, while the find modules are directory-search interfaces. This does
not settle how ConfigKit should combine its new form with retained path globs.

**Recommendation:** Exactly one selector per declaration: `path`, `dir`, or
`glob` if retained. Use a literal directory root; `pattern`, `recursive`, and
`depth` apply only to `dir`. Diagnose irrelevant fields rather than silently
ignoring them. This makes declarations unambiguous and keeps the roles of file
selection and traversal visible, at the cost of templates needing to omit
inapplicable fields. This is a proposed validation rule, not an already settled
restriction.

**Alternative:** Permit directory-search controls on a `path` declaration and
ignore them, retaining exclusive selectors. Shared templates become easier,
but a declaration can appear recursive without being so. Wildcard directory
roots are another viable extension: useful for selecting collections, but they
need root-expansion, depth-origin, ordering, and overlap rules. Retaining `glob`
is the simpler way to preserve existing wildcard-directory use cases.

### 11. Compatibility And Migration

**Ansible evidence:** There is no Ansible precedent that determines ConfigKit's
compatibility commitments or migration schedule.

**Recommendation:** Retain `glob` alongside `dir` because it can select directory
segments and exact depths that basename filters cannot express. Introduce
`required` while accepting explicit `optional` as its inverse for one documented
transition release. Reject declarations containing both fields to avoid two
authorities for the same choice. This retains existing expressiveness and eases
adoption, at the cost of temporary parser and documentation complexity.

The intended optional default applies when both fields are absent, including
legacy `path` and `glob` declarations; old spelling alone does not preserve the
old default. The application default, if adopted, also applies to this absent
case. Explicit legacy `optional: false` maps to `required: true`, and explicit
`optional: true` maps to `required: false`. Release notes must call out the
behavior change and show how to preserve existing requirements. Under the
[release policy](../policy/release.md#versioning-and-compatibility), this belongs
in a pre-1.0 minor release, not a patch release.

**Alternative:** Remove `optional` immediately in that minor release while
retaining `glob`. This establishes one vocabulary sooner, but every declaration
using the old field needs migration before it can load. If a transition is
chosen, accepting both fields when they agree is also viable; it helps generated
documents at the cost of extra normalization rules. Removing `glob` too would
need an expressiveness decision and migration for wildcard directory segments;
it cannot be treated as a simple field rename.