Apply permissive defaults and remove import transition support

This commit is contained in:
2026-10-18 16:37:07 +00:00
parent 82b289f4fe
commit 19fe621df1

View File

@@ -4,7 +4,7 @@ Status: working draft / brainstorming notes. This document captures a proposed
direction for discussion, not an implemented contract or an implementation plan.
The selected direction separates directory searches, filename matching,
recursion, and absence handling. Optional-by-default imports are the intended
direction. The Ansible-aligned decisions below are settled for this roadmap;
direction. The consumer-directed and Ansible-aligned decisions below are settled;
remaining choices have recommendations and alternatives under Open Decisions.
All new syntax below is illustrative and unimplemented.
@@ -27,6 +27,17 @@ across ConfigKit. It is not blanket authorization to weaken existing contracts,
ignore errors, or add a global permissiveness switch. Other defaults should be
evaluated individually against this principle.
Discovery should accept hidden entries and follow symlinks by default, with
options to narrow either behavior. An omitted filename filter should not impose
an implicit extension restriction. These defaults do not waive containment,
source validity, composition, or resource-limit rules. Recursion remains a
separate choice about search scope, not a measure of permissiveness.
This is a hard-breaking pre-1.0 change. Do not build temporary aliases,
translation layers, compatibility modes, or staged deprecation support. Evaluate
the final interface on its own merits, without retaining syntax solely to ease
upgrades.
## Current Baseline
The [current import contract](../config.md#explicit-imports) uses `path` or
@@ -71,6 +82,8 @@ The fields have distinct jobs:
| `pattern` | Filter candidate files by filename within the searched directories |
| `recursive` | Include descendant directories when true |
| `depth` | Bound directory-search depth; 1 means direct files, 0 or omission means unlimited when recursive |
| `follow` | Follow symlinks by default; false disables following within the discovery scope defined below |
| `hidden` | Include hidden entries by default; false excludes hidden files and prunes hidden directories within the discovery scope defined below |
| `required` | Fail when the declaration contributes no matching file |
| `destination`, `placement` | Apply the existing placement rules to each selected source |
@@ -79,6 +92,27 @@ This follows the familiar separation of traversal and filtering in Ansible's
and [include_vars](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/include_vars_module.html)
interfaces, without adopting all their spellings, defaults, or matching modes.
## Decisions Resolved By Project Direction
- **No transition support.** Use the final interface immediately in the selected
pre-1.0 minor release. Remove `optional` in favor of `required`; do not accept
it as an alias or normalize old declarations at runtime.
- **Optional contributions by default.** `required: true` opts into failure on
absence. Supplied invalid configuration continues to fail.
- **No implicit filename restriction.** Omitting `pattern` selects all
filenames. Authors can narrow selection explicitly. There is no default YAML
extension list; extensionless sources remain discoverable.
- **Follow symlinks by default, with an opt-out.** Provide `follow: false` for
callers who want discovery without symlink following. Applicable containment
and filesystem contracts remain in force.
- **Include hidden entries by default, with an opt-out.** Provide `hidden: false`
to exclude hidden files and prune hidden directories from discovery.
The defaults and availability of these controls are settled. Their exact scope
at explicitly named roots, the definition of hidden names across filesystems,
and application-level policy precedence are described under Open Decisions.
`follow` and `hidden` are the proposed field spellings.
## Decisions Resolved From Ansible
The comparison covers `ansible.builtin.find`, `ansible.builtin.include_vars`,
@@ -117,8 +151,8 @@ The following choices are settled:
traversal containers, even when their names match `pattern`. This agrees with
the default file selection in the find modules and the file-oriented
[fileglob lookup](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/fileglob_lookup.html).
Existing regular-file and symlink rules still govern opening selected sources;
the legacy `glob` contract is not implicitly changed by this new search form.
Existing regular-file and containment rules still govern opening selected
sources, together with the selected discovery controls.
The accepted `recursive` spelling is retained. The formerly tentative
`max_depth` spelling becomes `depth`. Resource budgets remain separate from
@@ -136,8 +170,9 @@ means. A nonrecursive search considers only files directly inside `dir`.
Recommend retaining the familiar `path.Match` wildcard grammar for filename
filters. The name `pattern` does not by itself imply regular expressions or
automatic dialect detection. Exact filter defaults and the treatment of
existing path-glob use cases remain open below.
automatic dialect detection. Omitting `pattern` selects all filenames, with no
implicit extension restriction. Matching grammar and whether the final API
needs path-glob expressiveness remain open below.
### Recursive Discovery And Depth Limits
@@ -166,6 +201,20 @@ including nonmatches, still consume the visit budget; a small number of matches
does not authorize an unbounded filesystem scan. This is an existing ConfigKit
constraint, not a new Ansible-derived decision.
### Symlinks And Hidden Entries
Directory searches default to `follow: true` and `hidden: true`. Authors may
disable either behavior independently of recursion, depth, matching, or absence
handling. `hidden: false` excludes hidden files and prunes hidden directories;
`follow: false` disables following symlinks within the discovery scope. The
precise boundary for explicit roots and file links remains open below.
These controls narrow selection rather than authorize ignoring errors in a
selected source. `required: true` applies after discovery filters: if the search
selects no file, it fails. Native filesystems retain their existing capabilities;
this feature does not promise link inspection where a supplied filesystem does
not expose it.
## Illustrative Examples
These examples describe the proposed interface, not configuration accepted by
@@ -208,6 +257,17 @@ Adding `depth: 2` limits discovery to direct files and files one directory below
`services/regional/east/worker.yaml`. With `recursive: false`, only direct files
are selected, even if `depth: 2` remains present.
Hidden entries and contained symlinks participate by default. To narrow the
search, the declaration can also include:
```yaml
follow: false
hidden: false
```
The exact treatment of an explicitly named root is an open scope decision, not
an additional change to the settled defaults.
### Require A Recursive Prompt Collection
```yaml
@@ -278,245 +338,187 @@ declared in an imported document, define that capability and its precedence
separately. Optional-by-default alone does not imply that an authored requirement
can be overridden, or that requirements propagate to transitive imports.
## Migration
## Release Boundary
Changing absence defaults changes behavior even when existing YAML contains no
optionality field. Renaming keys also changes the accepted syntax. Pre-1.0 is a
useful time to reconsider these contracts, but adoption still needs explicit
migration guidance.
Ship one final interface with a hard break in a pre-1.0 minor release, following
the [release policy](../policy/release.md#versioning-and-compatibility). There is
no transition release, legacy parser path, compatibility switch, or temporary
`optional` alias. Release notes should describe the changed behavior and final
syntax; this documentation does not imply migration machinery.
To preserve current intent, currently required imports would acquire
`required: true`; existing `optional: true` declarations could omit the new field
or use `required: false`. A simple `glob: prompts/*.yaml` becomes `dir: prompts`
plus `pattern: '*.yaml'` and `recursive: false`, preserving its discovery scope.
This is not a mechanical rename of `glob` to `pattern`: current globs can also
select particular directory names or depths. Their compatibility path must be
settled before removing any existing form.
Whether to support path globs in the final interface remains a feature-scope
decision because they express directory selection that basename filters cannot.
Any retained form must be justified as a permanent capability, not compatibility
scaffolding. No promise is made to preserve every old declaration or use case.
## Open Decisions
The recommendations below are not settled decisions. Each concerns either
inconsistent Ansible behavior or a ConfigKit-specific policy without a direct
Ansible counterpart. They do not reopen the accepted absence policy, placement,
strict composition, or source-version rules.
The recommendations below are not settled decisions. The new principles resolve
transition support, omitted filename filtering, and the defaults and opt-out
availability for symlinks and hidden entries. Seven decision areas remain;
closely related validation details are grouped together without treating them
as implicitly approved. Each has at least one viable alternative.
### 1. Recursion Default
**Ansible evidence:** `find` and `win_find` default to nonrecursive searches;
`include_vars` recursively loads directories by default. There is no common
default across these interfaces. See their parameter references above.
**Why still open:** Recursion determines search scope independently of absence
handling and permissiveness. Ansible's `find` and `win_find` default to
nonrecursive searches; `include_vars` recursively loads directories by default.
**Recommendation:** Default `recursive` to false. This keeps discovery local
unless the declaration explicitly requests descendants and makes migration from
simple current globs straightforward. The cost is one extra field for recursive
collections.
**Recommendation:** Default `recursive` to false. A declaration explicitly
chooses whether subdirectories contribute. This makes the search scope visible,
at the cost of one extra field for recursive collections.
**Alternative:** Default to true, following `include_vars`. Collection authors
could reorganize files into subdirectories without revising declarations, but
adding a subdirectory could also add contributions to an existing import.
Neither choice changes the settled `follow`, `hidden`, or `required` defaults.
### 2. Matching Language And Filter Shape
### 2. Matching Language And Selector Expressiveness
**Ansible evidence:** `find` supports shell patterns and optional regex;
`include_vars.files_matching` is a regex; `win_find` uses PowerShell wildcards or
regex. Their case and matching details differ. There is no single exact grammar.
**Why still open:** Permissiveness does not select a matching grammar. Ansible
uses different wildcard and regex languages across its discovery interfaces.
Also, basename filters and path globs express different searches; the absence
of compatibility support does not decide which capabilities the final API needs.
**Recommendation:** Keep one scalar `pattern` using ConfigKit's existing
case-sensitive Go `path.Match` grammar, applied to basenames. This reuses a
familiar contract, including escapes and character classes, without introducing
regex mode or platform-dependent case rules. It cannot express every regex or
an arbitrary union of suffixes in one declaration; separate declarations remain
available, subject to existing repeated-source rules.
**Recommendation:** Use one scalar `pattern` with the existing case-sensitive
Go `path.Match` grammar, applied to basenames. Keep `dir` as a literal root and
retain `glob` as a permanent, separate selector for wildcard directory segments
and exact-depth path selection. Its justification is useful expressiveness,
not old-document compatibility. Apply the new absence policy to every selector;
retaining `glob` must not introduce a legacy behavior mode. This supports both
simple collection discovery and more selective searches, but leaves two
matching forms to explain and maintain.
**Alternative:** Add an explicit regex mode and/or a list of patterns whose
matches are unioned within one declaration. This better supports filters such
as both `.yaml` and `.yml` without overlapping imports, but adds syntax, matching
rules, and validation. Regex would need a specified Go grammar rather than a
promise of Python or PowerShell compatibility. Exclusion filters could accompany
that design, but are not assumed in the initial recommendation.
**Alternative:** Offer only `path` and `dir` with a basename `pattern`. This is
a smaller final interface and removes `glob` outright, with no alias or rewrite.
Users needing selected directory names or depths would use multiple explicit
imports or application-side source selection. That is a real capability
tradeoff, acceptable if those use cases do not justify a separate selector.
### 3. Omitted Filename Filter
**Filter alternative:** Allow a list of patterns unioned within one declaration,
and/or an explicit Go regex mode. This supports suffix unions or richer filters
without overlapping imports, but adds syntax, matching rules, and validation.
Separate scalar declarations are simpler, though overlapping selections remain
subject to source-repeat errors. Regex compatibility with Python or PowerShell
is not implied. None of these choices changes the settled all-filenames default
when a filter is omitted.
**Ansible evidence:** The find modules permit unrestricted filename selection.
`include_vars` also has an extension policy, defaulting to `json`, `yaml`, and
`yml`, with separate handling of unknown extensions. Its loading behavior is
not equivalent to a general-purpose find operation.
### 3. Directive Validation
**Recommendation:** Omitted `pattern` means all filenames, with no implicit
extension filter. This keeps selection independent of filename conventions and
supports extensionless configuration. A selected README or other invalid file
still causes a load error; authors of mixed directories should supply a filter.
**Why still open:** Permissive discovery does not determine whether malformed or
inapplicable option values should be accepted. The draft's existing error
boundary remains: invalid declarations fail. We still need to define their
valid shapes, rather than infer extra restrictions from that boundary.
**Alternative:** Default discovery to `.yaml` and `.yml`, with an explicit way
to select all files or other extensions. This suits typical configuration
directories and avoids incidental README selection, but adds an extension
policy and an override mechanism that explicit `path` imports do not need.
**Recommendation:** Accept nonnegative YAML integers for `depth`; zero remains
unlimited. Treat negative, null, string, boolean, and fractional values as
malformed, even when recursion is disabled. Use exactly one selector per
import: `path`, `dir`, or `glob` if selected for the final API. Keep directory
search options (`pattern`, `recursive`, `depth`, `follow`, and `hidden`) on `dir`
and diagnose them on other forms. This makes likely mistakes visible, but
shared templates must omit inapplicable fields. It imposes no arbitrary maximum
depth. The removed `optional` spelling is not accepted under any variant.
### 4. Invalid Depth Values
**Alternative:** Normalize null depth to omission and negative integers to
unlimited, and permit unused directory controls on a direct-file declaration.
That eases generated configuration but can turn an erroneous bound into a wider
search or make a direct-file import appear recursive. These relaxations can be
chosen independently; they do not require accepting ambiguous selectors.
Another viable selector shape is to allow multiple selectors as an explicit
union, but it needs ordering and overlap semantics that one-selector declarations
avoid.
**Ansible evidence:** Positive depth, unlimited omission, and disabled-recursion
precedence have useful common semantics. The implementations do not provide a
consistent negative-depth contract; accepting a value is not evidence of a
useful shared meaning. See the implementation links above.
### 4. Discovery Control Boundaries And Cycles
**Recommendation:** Accept nonnegative YAML integers only. Zero means unlimited;
negative, null, string, boolean, or fractional values are malformed declarations,
even when recursion is disabled. This follows ConfigKit's typed directive style
and gives likely mistakes a direct diagnostic without imposing an arbitrary
maximum depth.
**Why still open:** Including hidden entries and following symlinks by default,
with options to disable both, is settled. The principles do not define hidden
names across filesystems, the reach of an opt-out at explicitly named roots, or
the response to recursive directory-link cycles. Ansible does not provide one
consistent set of answers across the inspected interfaces.
**Alternative:** Treat negative integers as unlimited too, and null as omission.
That can ease generated configurations but adds redundant spellings and risks
turning an erroneous bound into a wider search. Neither interpretation is
required by the settled rule that disabled recursion ignores a valid bound.
**Recommendation:** Define hidden entries by a leading dot for the portable
source namespace. With `hidden: false`, skip such files and prune such directories
encountered below the search root. With `follow: false`, skip encountered symlink
files and do not descend through encountered symlink directories. An explicitly
named `dir` root still resolves under existing source-access rules, as does a
separate `path` import. The controls affect this declaration's discovery; they
do not propagate to imports declared by selected files. This gives a simple
selection rule and preserves deliberate explicit access, but is not a ban on
all hidden or symlink access in the load.
### 5. Hidden Files And Directories
When following links, detect a canonical directory repeated on the active
traversal path and fail with a cycle diagnostic. Keep existing errors for
imported sources reached repeatedly through aliases. This makes cycles visible
without silently deduplicating configuration. It adds directory-cycle handling
alongside the existing source-cycle rules; it does not change containment or
native-filesystem capabilities.
**Ansible evidence:** The find modules expose hidden-file filtering, while
`include_vars` has no equivalent hidden-entry exclusion in its directory walk.
Also, filtering a hidden entry is not necessarily the same as pruning a hidden
directory's descendants. See
[find](https://github.com/ansible/ansible/blob/stable-2.20/lib/ansible/modules/find.py)
and [include_vars](https://github.com/ansible/ansible/blob/stable-2.20/lib/ansible/plugins/action/include_vars.py).
**Alternative:** Make the controls govern the explicit search root and every
component used to reach it too, and classify hidden entries using host attributes
where available. This can serve a broader exclusion policy, but adds filesystem
inspection requirements and differences between host and abstract filesystems.
A consumer-wide enforcement policy would also need explicit precedence rather
than being inferred from one declaration's flags.
**Recommendation:** Give dot-prefixed names no special treatment: visit their
directories within the selected depth and select files by `pattern`. This fits
the existing matcher and avoids an implicit exclusion. Broad recursive searches
may include configuration in hidden directories, so the search root and filter
should reflect the intended collection.
**Cycle alternative:** Prune an edge back to an active ancestor directory and
continue. This permits link-rich trees without failing for an irrelevant loop,
but can hide a layout mistake and needs clearly defined inspection evidence.
It would not justify suppressing repeated-source errors for files actually
selected through distinct routes.
**Alternative:** Add `hidden`, defaulting to false, with an explicit definition
covering both hidden-file selection and hidden-directory pruning. This can
exclude incidental metadata directories, but adds policy and a platform choice
between dot prefixes and filesystem hidden attributes. Callers would need to
opt in when their collections intentionally use hidden names.
### 5. Match Ordering
### 6. Directory Symlinks And Traversal Cycles
**Why still open:** Inclusion defaults do not determine order. `include_vars`
sorts directories and filenames; `win_find` documents path-sorted results;
POSIX `find` does not promise the same global ordering.
**Ansible evidence:** The find modules default to not following directory links;
the inspected `include_vars` implementation walks with link following enabled.
There is no common default to adopt.
**Recommendation:** Follow contained directory symlinks using ConfigKit's
existing canonicalization and containment rules, preserving the current ability
to select through contained symlinks. For the new recursive walk, detect a
canonical directory repeated on the active traversal path and report a cycle.
Do not silently deduplicate imported files reached through aliases; existing
source-repeat errors remain applicable. This supports linked collections but
requires directory-cycle handling in addition to import-cycle handling.
**Alternative:** Add a traversal `follow` option defaulting to false, skipping
directory symlinks encountered below the search root unless enabled. Keep
explicitly named roots and file targets under existing source-opening rules.
This reduces accidental traversal through linked trees, but introduces a new
distinction between discovery and explicit access and requires callers to opt
in for linked collections. Both choices retain native-filesystem semantics;
neither promises symlink capabilities absent from the supplied filesystem.
### 7. Match Ordering
**Ansible evidence:** `include_vars` sorts directories and filenames;
`win_find` documents results sorted by path. POSIX `find` describes top-down
searching and does not promise the same global ordering. These are not identical
ordering contracts.
**Recommendation:** Sort all selected slash-relative paths lexicographically
within each declaration, then use existing depth-first import expansion. This
extends ConfigKit's existing glob order and stabilizes sequence assembly and
diagnostics. It requires collecting bounded candidates before expansion.
**Recommendation:** Sort selected slash-relative paths lexicographically within
each declaration, then use existing depth-first import expansion. This provides
one familiar order for discovery, sequence assembly, and diagnostics. It
requires collecting bounded candidates before expansion.
**Alternative:** Load direct files first, then visit sorted subdirectories
recursively. This offers a natural directory-oriented traversal and can begin
loading sooner, but differs from existing global match ordering and can change
sequence item positions. Neither choice grants replacement precedence within
strict composition.
recursively. This offers directory-oriented traversal and can begin loading
sooner, but gives different sequence item positions. Neither choice grants
replacement precedence within strict composition.
### 8. Application And Operator Strictness Controls
### 6. Application And Operator Policy Precedence
**Ansible evidence:** Task failure controls and module-specific options are not
a direct counterpart to defaults for ConfigKit's transitive import graph.
Ansible cannot settle this public API choice.
**Why still open:** The library defaults and authored opt-outs are settled.
Whether consumers can set different defaults or enforce restrictions over
explicit declarations remains an API choice. Ansible's task controls are not a
direct counterpart to policy over ConfigKit's transitive import graph.
**Recommendation:** Add a load-level default for whether imports are required,
used only when a declaration omits `required`. The library default stays false;
an application can choose true and expose that choice to operators through its
own interface. Apply the default independently to each declaration, including
transitive imports. Explicit true or false remains authoritative. This enables
stricter deployments while preserving intentionally optional imports; it does
not enforce that every authored optional declaration succeeds.
**Recommendation:** Provide load-level defaults for `required`, `follow`, and
`hidden`, applied independently when a declaration omits the corresponding
field. Library defaults remain false, true, and true respectively. Explicit
per-declaration values win; imports do not inherit these values from their
parent declaration. Applications can expose defaults to operators through their
own interfaces. This enables stricter deployment choices while retaining
intentional exceptions, but does not enforce a universal exclusion rule.
**Alternative:** Offer an enforcement mode requiring every declaration to
match, including those explicitly marked `required: false`. That provides a
stronger audit policy but overrides authors' intent and needs clear precedence
and diagnostics. A smaller alternative is authored requirements only for now;
it avoids a public option but leaves applications unable to choose a common
default without editing source declarations.
**Alternative:** Offer enforcement options that override explicit declarations,
for example requiring every import to match or forbidding discovery through
symlinks throughout the load. This supports centrally imposed restrictions but
needs clear conflict diagnostics and scope, particularly for explicit sources.
A smaller alternative is declaration-level controls only initially: simpler
API, but consumers cannot choose common defaults without changing declarations.
### 9. Evidence For Absent Optional Imports
### 7. Evidence For Absent Optional Imports
**Ansible evidence:** `fileglob` returns an empty result for no matches;
`include_vars` reports included filenames; the find modules expose richer search
results. There is no common inspection model for an absent import declaration.
**Why still open:** Permitting absence does not settle how it should be exposed
for inspection. Ansible's file lookup, variable loader, and find modules return
different evidence, with no common absent-import model.
**Recommendation:** Record minimal structured inspection evidence for an
optional declaration that selects nothing: its declaring source/location,
selector, and absent outcome. Emit no warning by default and do not fabricate a
loaded-source record. This helps distinguish intentional absence from a typo,
but requires an inspection API decision and associated ownership tests. Design
the exact record shape during implementation planning, using existing evidence
structures where appropriate.
**Recommendation:** Record minimal structured inspection evidence when an
optional declaration selects nothing: declaring source/location, selector, and
absent outcome. Emit no warning by default and do not fabricate a loaded-source
record. This helps distinguish intentional absence from a typo, but requires an
inspection API decision and ownership tests. Decide its exact shape during
implementation planning, reusing existing structures where appropriate.
**Alternative:** Add no new inspection surface initially; retain loaded-source
evidence and let `required: true` turn absence into a diagnostic when needed.
This keeps the feature smaller, but successful optional omissions are harder to
investigate. Existing inspection contracts must remain intact either way.
### 10. Selector Combinations And Directory-Only Options
**Ansible evidence:** `include_vars` separates file-only and directory-only
arguments, while the find modules are directory-search interfaces. This does
not settle how ConfigKit should combine its new form with retained path globs.
**Recommendation:** Exactly one selector per declaration: `path`, `dir`, or
`glob` if retained. Use a literal directory root; `pattern`, `recursive`, and
`depth` apply only to `dir`. Diagnose irrelevant fields rather than silently
ignoring them. This makes declarations unambiguous and keeps the roles of file
selection and traversal visible, at the cost of templates needing to omit
inapplicable fields. This is a proposed validation rule, not an already settled
restriction.
**Alternative:** Permit directory-search controls on a `path` declaration and
ignore them, retaining exclusive selectors. Shared templates become easier,
but a declaration can appear recursive without being so. Wildcard directory
roots are another viable extension: useful for selecting collections, but they
need root-expansion, depth-origin, ordering, and overlap rules. Retaining `glob`
is the simpler way to preserve existing wildcard-directory use cases.
### 11. Compatibility And Migration
**Ansible evidence:** There is no Ansible precedent that determines ConfigKit's
compatibility commitments or migration schedule.
**Recommendation:** Retain `glob` alongside `dir` because it can select directory
segments and exact depths that basename filters cannot express. Introduce
`required` while accepting explicit `optional` as its inverse for one documented
transition release. Reject declarations containing both fields to avoid two
authorities for the same choice. This retains existing expressiveness and eases
adoption, at the cost of temporary parser and documentation complexity.
The intended optional default applies when both fields are absent, including
legacy `path` and `glob` declarations; old spelling alone does not preserve the
old default. The application default, if adopted, also applies to this absent
case. Explicit legacy `optional: false` maps to `required: true`, and explicit
`optional: true` maps to `required: false`. Release notes must call out the
behavior change and show how to preserve existing requirements. Under the
[release policy](../policy/release.md#versioning-and-compatibility), this belongs
in a pre-1.0 minor release, not a patch release.
**Alternative:** Remove `optional` immediately in that minor release while
retaining `glob`. This establishes one vocabulary sooner, but every declaration
using the old field needs migration before it can load. If a transition is
chosen, accepting both fields when they agree is also viable; it helps generated
documents at the cost of extra normalization rules. Removing `glob` too would
need an expressiveness decision and migration for wildcard directory segments;
it cannot be treated as a simple field rename.
investigate. Existing inspection contracts remain intact either way.