Files
ConfigKit/docs/roadmap/imports.md

712 lines
37 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Import Ergonomics And Permissive Defaults
Status: working draft / brainstorming notes. This document captures a proposed
direction for discussion, not an implemented contract or an implementation plan.
The selected direction separates directory searches, filename matching,
recursion, and absence handling. Optional-by-default imports are the intended
direction. The consumer-directed and Ansible-aligned decisions below are settled;
API spelling and implementation details remain for implementation planning.
All new syntax below is illustrative and unimplemented.
## Guiding Philosophy
ConfigKit should provide predictable configuration mechanisms and avoid imposing
application policy. Its defaults should permit optional contributions to be
absent, with explicit options for increased strictness when needed. That choice
may come from an operator or the consuming application; authored import
declarations can also express requirements on contributions.
Permissiveness should distinguish absence from invalid supplied configuration:
> Optional contributions may be absent by default. Consumers can explicitly
> require them. Supplied configuration must still be structurally valid and
> interpretable under the selected composition rules.
This is a general design direction worth making more explicit and consistent
across ConfigKit. It is not blanket authorization to weaken existing contracts,
ignore errors, or add a global permissiveness switch. Other defaults should be
evaluated individually against this principle.
Discovery should accept hidden entries and follow symlinks by default, with
options to narrow either behavior. An omitted filename filter should not impose
an implicit extension restriction. These defaults do not waive containment,
source validity, composition, or resource-limit rules. Recursion remains a
separate choice about search scope, not a measure of permissiveness.
This is a hard-breaking pre-1.0 change. Do not build temporary aliases,
translation layers, compatibility modes, or staged deprecation support. Evaluate
the final interface on its own merits, without retaining syntax solely to ease
upgrades.
## Current Baseline
The [current import contract](../config.md#explicit-imports) uses `path` or
`glob`, with required imports by default and `optional: true` to permit absence.
Glob matching uses slash-based Go `path.Match` syntax. Recursive `**` syntax is
not supported; authors can match particular depths with patterns such as
`prompts/*.yaml` and `prompts/*/*.yaml`.
Current imports expand depth-first in declaration order, with each ordinary
source contributing its body before its imports. The existing
[sibling inheritance resolver](../config.md#sibling-inheritance) instead follows
dependency chains before resolving dependents, using sorted sibling names and
an explicit stack. Neither implements the shared ordering selected below.
The proposed changes concern naming, discovery, absence handling, and uniform
dependency ordering for imports and `extends`. Import destinations, strict
composition, inheritance overlay semantics, tier precedence, and resource
identity retain their distinct roles.
## Proposed Import Vocabulary
Use direct-file, directory-search, and path-glob forms. Directory searches
separate literal search roots from filename filters; path globs additionally
support wildcard directory segments and selection at particular depths:
```yaml
configkit:
imports:
- dir: prompts
pattern: '*.yaml'
recursive: true
destination: [prompts]
placement: sequence-item
- path: defaults.yaml
required: true
```
The first declaration searches `prompts` and its descendants for matching
filenames, placing each source mapping as one prompt item, and permits no
matches. The second explicitly requires its target file. Sequence composition
still requires the consumer's existing registration, as explained below.
The fields have distinct jobs:
| Field | Proposed meaning |
| --- | --- |
| `path` | Import a named file or list of named files, relative to the declaring file |
| `dir` | Search a literal directory or list of literal directories, relative to the declaring file |
| `pattern` | A filename wildcard or list of wildcards; select files matching any effective pattern, using the default precedence below |
| `glob` | Select files with one path wildcard, including wildcard directory segments, relative to the declaring file |
| `exclude` | A filename wildcard or list of wildcards; remove matching files from any selector's results |
| `recursive` | Library default false, overridable by an application default; include descendant directories when effectively true |
| `depth` | Bound directory-search depth; supplying it requires effective recursion to be enabled; 1 means direct files, 0 or omission means unlimited when recursive |
| `follow` | Follow symlinks by default; false disables following within the discovery scope defined below |
| `hidden` | Include hidden entries by default; false excludes hidden files and prunes hidden directories within the discovery scope defined below |
| `required` | For `path`, require every listed file to be present; for `dir` or `glob`, require at least one selected file after exclusions across the complete search |
| `destination`, `placement` | Apply the existing placement rules to each selected source |
This follows the familiar separation of traversal and filtering in Ansible's
[find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/find_module.html)
and [include_vars](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/include_vars_module.html)
interfaces, without adopting all their spellings, defaults, or matching modes.
## Decisions Resolved By Project Direction
- **No transition support.** Use the final interface immediately in the selected
pre-1.0 minor release. Remove `optional` in favor of `required`; do not accept
it as an alias or normalize old declarations at runtime.
- **Optional contributions by default.** `required: true` opts into failure on
absence. Supplied invalid configuration continues to fail.
- **No library-imposed filename restriction.** Omitting `pattern` selects all
filenames unless the application supplies a default pattern. Authors can
override that default explicitly. There is no library default YAML extension
list; extensionless sources remain discoverable.
- **Follow symlinks by default, with an opt-out.** Provide `follow: false` for
callers who want discovery without symlink following. Applicable containment
and filesystem contracts remain in force.
- **Include hidden entries by default, with an opt-out.** Provide `hidden: false`
to exclude hidden files and prune hidden directories from discovery.
- **One dependency-ordering algorithm for imports and `extends`.** Both use the
shared dependency-ready scheduler defined below. This is settled independently
of the existing implementations and includes refactoring both resolvers.
OD-1 (recursion default), OD-2 (matching and selector expressiveness), OD-3
(directive validation), OD-4 (discovery boundaries and cycles), OD-5 (policy
precedence), and OD-6 (absent-import evidence) are settled in the sections below.
Supplying `depth` requires effective recursion to be enabled, including when
enabled through an application default. This supersedes the earlier direction
to accept but ignore depth when recursion is disabled.
## Decisions Resolved From Ansible
The comparison covers `ansible.builtin.find`, `ansible.builtin.include_vars`,
`ansible.builtin.fileglob`, and `ansible.windows.win_find`. A consistent approach
means agreement among the relevant interfaces that offer the capability, not
that every Ansible module exposes it. Existing ConfigKit contracts and the
already selected optional-by-default direction remain authoritative; this is
not a general Ansible compatibility mode.
The following choices are settled:
- **Separate traversal from filename filtering.** Directory roots and filename
filters have separate roles; filters operate on basenames in visited
directories. A directory failing the filename filter does not prune its
descendants. This agrees with the directory-search interfaces above.
See [find's parameters](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/find_module.html#parameters)
and the [include_vars implementation](https://github.com/ansible/ansible/blob/stable-2.20/lib/ansible/plugins/action/include_vars.py).
- **Include an optional integer `depth` control.** All three directory-search
interfaces use that spelling. Direct files are at depth 1, and each additional
directory adds one. Omission permits unlimited depth when recursion is
enabled. See [include_vars](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/include_vars_module.html#parameters)
and [win_find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/windows/win_find_module.html#parameters).
- **Treat `depth: 0` as unlimited.** This follows `include_vars` and agrees with
the current [find implementation](https://github.com/ansible/ansible/blob/stable-2.20/lib/ansible/modules/find.py)
and [win_find implementation](https://github.com/ansible-collections/ansible.windows/blob/main/plugins/modules/win_find.ps1).
The latter two document unlimited omission more clearly than explicit zero;
this choice uses source-confirmed behavior, not a claimed universal documented
Ansible guarantee. Negative values are invalid under the settled directive
validation rules below.
- **Directory searches select files, not directory documents.** Directories are
traversal containers, even when their names match `pattern`. This agrees with
the default file selection in the find modules and the file-oriented
[fileglob lookup](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/fileglob_lookup.html).
Existing regular-file and containment rules still govern opening selected
sources, together with the selected discovery controls.
The accepted `recursive` spelling is retained. The formerly tentative
`max_depth` spelling becomes `depth`. Resource budgets remain separate from
depth, and the current limits still apply to an unlimited-depth search.
## Shared Dependency-Ready Ordering
**Status: settled for imports and `extends`; not yet implemented.** Use
deterministic dependency-ready ordering: Kahn's topological-sort algorithm with
a min-heap of eligible nodes. Both resolvers must use the same algorithm in the
same way, through one shared internal scheduling implementation, common
tie-breaking rules, and common cycle-path logic.
### Scheduling Rule
A dependency edge means that the dependent cannot complete until its dependency
has completed. For each dependency graph:
1. Count each node's outstanding dependencies and identify initially eligible
nodes: those with none.
2. Choose the eligible node with the smallest stable ordering key.
3. Append it to the schedule, decrement its dependents' outstanding counts, and
immediately make newly eligible nodes available for the next selection.
4. Continue until all nodes are scheduled. If unfinished nodes remain with no
eligible node, reject the graph and identify an actual dependency cycle.
Use stable source or definition identities to construct ordering keys, with
the same deterministic comparison rule in both resolvers. Map iteration order
and discovery-assigned source IDs must not supply the tie-breaker. Specify key
construction for source namespaces and logical definition paths during
implementation planning; it must distinguish nodes within each scheduled graph.
Construct and validate a complete schedule for the available graph before
applying its resolution operations. A blocked dependent is not necessarily a
cycle member: diagnostics should identify the actual cycle using shared,
deterministic cycle-path logic. Source discovery and parsing may already have
occurred; this rule is not a promise to avoid reading invalid configurations.
### Scope And Integration
Discovery and resolution are separate operations. An importing source must be
read to discover its imports, but its contribution need not be assembled first.
Discovery order is not a substitute for the selected resolution schedule.
Both resolvers construct dependency graphs and apply resolver-specific operations
through the shared scheduler. Imports retain strict composition; `extends`
retains inheritance overlay behavior. Scheduling does not authorize replacement
within a strict composition tier or change precedence between tiers. Shared
dependencies can be scheduled once where valid; this does not relax the existing
prohibition on repeated source inclusion for imports.
Preserve item-local helper scopes and enclosing-before-nested inheritance
requirements. An imported item completes its helper contributions before being
placed as an item. Enclosing inheritance establishes the context in which nested
relationships are discovered and resolved. Use the same scheduler at those
scope boundaries, with the actual prerequisites represented; do not flatten raw
files or mappings in a way that loses their scope. Inheritance still resolves
after source assembly and precedence have established the effective document.
The integration must account for resulting sequence positions, provenance,
history, and diagnostic ordering. It must distinguish source acquisition records
from assembly order rather than assuming they are the same. Review both
resolvers' contracts and behavioral tests when implementing the new schedule;
preserving the old traversal is not a design objective.
### Rationale And Validation
Dependency-ready ordering selects the smallest currently eligible node without
adding branch preference, batch barriers, or distance-from-root priority. With
a fixed total ordering key, it gives the lexicographically smallest valid
dependency-first schedule. It can interleave independent branches; branch
contiguity is not part of the chosen ordering contract.
The comparison considered deterministic depth-first postorder, dependency
rounds, and descending longest-path depth as well. All can satisfy dependency
correctness, but their additional ordering policies did not justify choosing
them over the uniform eligibility rule. This decision gives no preference to
the existing implementation or to minimizing pre-1.0 refactoring.
The scheduler requires O(E + V log V) time and O(V + E) memory for V nodes and
E dependency edges, assuming constant-time key comparisons. Source access,
parsing, merging, and provenance work are separate costs.
Validate independent branches, unequal chains, branches with multiple
dependencies, shared dependencies at equal and unequal depths, cycles with
blocked non-cycle dependents, and nested scopes. Equivalent valid graphs and
ordering keys must produce identical schedules through the import and
inheritance integrations. Verify independence from graph insertion order and
preserve each resolver's domain-specific admissibility and combination rules.
## Discovery Semantics
### Filename Matching
`pattern` applies to a file's basename, not its full relative path. For example,
`*.yaml` matches both `prompts/one.yaml` and
`prompts/weather/daily/summary.yaml` when the latter's directory is visited.
Recursion changes which directories are visited; it does not change what `*`
means. A nonrecursive search considers only files directly inside `dir`.
**OD-2: settled.** Use the existing case-sensitive Go `path.Match` wildcard
grammar for filename filters. When neither the declaration nor the application
supplies `pattern`, select all filenames, with no implicit extension restriction.
Do not add regular expressions, automatic dialect detection, or recursive `**`
syntax as part of this work.
Retain `glob` as a permanent, separate selector using the existing slash-based
path wildcard grammar. For example, `collections/*/prompts/*.yaml` selects
files in a particular directory layout without selecting all descendants.
Literal `dir` roots and basename filters do not express that same search.
This capability justifies retaining `glob` independently of compatibility;
it uses the same new absence policy as other selectors. `glob` remains a
single string; list support is being added to `path`, `dir`, and the filename
filters.
### Selector Lists And Set Operations
Accept a string or a list of strings for `path`, `dir`, and `pattern`. Normalize
a scalar to a singleton list so both forms use the same selection logic.
Listed paths and directory roots resolve relative to the declaring file.
List order does not introduce precedence or replace the shared dependency-ready
resolution schedule.
For directory roots D and filename patterns P, search the **Cartesian product**
D × P and take the **union** of the matching candidate files:
`S = ⋃ {matches(d, p) | (d, p) ∈ D × P}`
Each root uses the declaration's recursion, depth, and applicable discovery
controls. Resolve an omitted `pattern` from the application default first;
if neither supplies patterns, select all filenames in the visited directories
instead. This describes selection semantics, not a requirement to
materialize the product or traverse a directory again for each pattern.
Overlapping patterns or directory searches within one declaration select the
same candidate once. Repeated identical entries in one `path` list likewise
select that candidate once; duplicate patterns or exclusions have no additional
effect. This local deduplication is not global source deduplication:
existing repeated-source and alias checks still apply to distinct inclusion
routes. In particular, it does not authorize silently merging separate alias
paths to an imported source.
For `path`, `required: true` requires every listed file to be present. For `dir`,
it requires at least one selected file across the complete union after
exclusions; it does not require a match per directory, pattern, or pair. A
missing directory can therefore contribute nothing while another supplies the
required match. The equivalent `glob` requirement is at least one selected file
after exclusions. Invalid supplied configuration still fails under the existing
rules.
### Exclusions
Accept `exclude` as a string or list of strings with the same case-sensitive
basename wildcard grammar as `pattern`. Apply it to results from `path`, `dir`,
and `glob`. A candidate matching any exclusion is removed before its content is
loaded or parsed. An omitted exclusion removes nothing.
This is **set difference**: for candidate set S and its subset X matching any
exclusion, the selected set is `S ∖ X`. Exclusions win over positive matches.
They filter filenames, not directory paths, and do not prune traversal into a
directory whose name happens to match. For example, `exclude: '*.draft.yaml'`
removes matching files at every visited depth, including results of a path glob.
For searches, evaluate `required` on the selected set after exclusions. For an
explicit `path` list, exclusions filter what is loaded but do not waive the
settled requirement that every listed file be present when `required: true`.
Excluding a candidate is not permission to suppress discovery, containment,
or resource-limit errors. Exclusions are local to the declaration; they do not
prohibit another declaration from explicitly selecting the same filename.
### Recursive Discovery And Depth Limits
Enable recursion explicitly or through an application default for recursive
discovery, optionally bounded by `depth`.
The motivation is to let authors organize definitions in subdirectories without
updating import declarations for every depth. Directory organization should not
change resource identity or the effective configuration destination.
The selected model uses an explicit search root and recursion flag, rather than
introducing recursive `**` syntax or inferring a root from a wildcard prefix.
**OD-1: settled.** The library default for `recursive` is false: search only
direct files. An application may supply a different default; an explicit
declaration value overrides it, including `recursive: false`. Recursion remains
independent of the permissive defaults for absence, symlinks, and hidden entries.
The settled depth convention is:
- Files directly inside the search root are at depth `1`.
- Each additional directory increases depth by one.
- Omitting `depth`, or setting it to 0, imposes no caller-selected depth limit
when recursion is enabled; existing loader resource limits still apply.
- Supplying `depth` requires effective `recursive: true`. If recursion is false
after resolving declaration, application, and library defaults, any supplied
`depth` is an error, including `0` or `1`. Depth never enables recursion itself.
- With recursion disabled and `depth` omitted, select only direct files.
- Depth controls discovery only, not namespaces, identities, or destinations.
Resolve the effective recursion value before checking depth's applicability.
For example, `depth: 2` with no explicit `recursive` is valid if the application
default enables recursion, but invalid with the library default. Explicit
`recursive: false` overrides an application default of true and makes any
supplied depth invalid. Depth must also have the valid type and range defined
under directive validation; enabling recursion does not excuse malformed depth.
Recursive discovery must fit the existing filesystem containment,
repeated-source, and load-budget
contracts rather than silently changing them. In particular, examined entries,
including nonmatches, still consume the visit budget; a small number of matches
does not authorize an unbounded filesystem scan. This is an existing ConfigKit
constraint, not a new Ansible-derived decision.
### Symlinks And Hidden Entries
**OD-4: settled.** Discovery controls are local to the declaration. Directory
searches default to `follow: true` and `hidden: true`, subject to the application
defaults described below. Authors may disable either behavior independently of
recursion, depth, matching, or absence handling.
Define hidden entries by a leading dot in the portable source namespace.
`hidden: false` skips discovered hidden files and prunes discovered hidden
directories. `follow: false` skips discovered symlink files and prevents descent
through discovered symlink directories. Explicitly named roots and separate
direct-file imports retain their existing source-access semantics. The controls
do not implicitly propagate to declarations inside imported files.
These are discovery-selection controls, not a load-wide access prohibition.
On `glob`, filtering applies to wildcard-selected entries; explicitly named
path components remain explicit access.
When following links, detect a canonical directory repeated on the active
traversal path and fail with a cycle diagnostic. Do not silently prune the loop
and continue. Existing dependency-cycle and repeated-source errors remain in
force; discovered aliases do not authorize silently deduplicating imported
sources. Directory-cycle handling retains the existing containment and
native-filesystem capabilities.
These controls narrow selection rather than authorize ignoring errors in a
selected source. `required: true` applies after discovery filters: if the search
selects no file, it fails. Native filesystems retain their existing capabilities;
this feature does not promise link inspection where a supplied filesystem does
not expose it.
## Illustrative Examples
These examples describe the proposed interface, not configuration accepted by
the current release. They assume the consumer has enabled imports and has not
changed the library's policy defaults. File paths and directory roots resolve
from the declaring file; destinations use the
existing relative-placement rules.
### Import One File
```yaml
configkit:
imports:
- path: defaults.yaml
```
Import `defaults.yaml` into the declaring source's mapping scope. Under the
proposed default, a missing file contributes nothing. Add `required: true` when
its presence is necessary. A present but invalid file still fails loading.
### Require Several Named Files
```yaml
configkit:
imports:
- path: [defaults.yaml, site.yaml]
required: true
```
Both files must be present. Listing them together shares the declaration's
options; their textual order does not establish overlay precedence. Their
contributions follow the shared dependency-ready schedule and strict
composition rules.
### Recursively Discover Mapping Fragments
```yaml
configkit:
imports:
- dir: services
pattern: '*.yaml'
recursive: true
destination: [services]
```
This discovers files such as `services/api.yaml` and
`services/regional/worker.yaml`, but not `services/README.md`. Each file contains
named service entries, for example `api: {port: 8080}` or
`worker: {concurrency: 4}`. They compose beneath `services` under the existing
strict composition rules. Directory names do not create additional mapping
keys. Missing directories or no matching files contribute nothing.
Adding `depth: 2` limits discovery to direct files and files one directory below
`services`: it still selects `services/regional/worker.yaml`, but not
`services/regional/east/worker.yaml`. Switching to `recursive: false` requires
omitting `depth`; leaving `depth: 2` present is a declaration error.
Hidden entries and contained symlinks participate by default. To narrow the
search, the declaration can also include:
```yaml
follow: false
hidden: false
```
These controls narrow discovery below the explicitly named root; the root
continues to resolve under the existing source-access rules.
### Search Several Prompt Collections And Filename Patterns
```yaml
configkit:
imports:
- dir: [prompts, shared-prompts]
pattern: ['*.prompt.yaml', '*.prompt.yml']
exclude: ['*.draft.prompt.yaml', '*.draft.prompt.yml']
recursive: true
required: true
destination: [prompts]
placement: sequence-item
```
The two directories and two patterns form four Cartesian-product pairs; the
selected candidates are the union of their matches, minus excluded filenames.
This selects `prompts/welcome.prompt.yaml`,
`prompts/weather/daily/summary.prompt.yaml`, and
`shared-prompts/greeting.prompt.yml`, but not `prompts/weather/helpers.yaml`
or `shared-prompts/greeting.draft.prompt.yml`.
Each selected file contains one prompt mapping,
such as `id: prompt.welcome` and `text: Welcome`. The consumer registers the
`prompts` sequence with identity field `id` through the existing
`Definition.SequenceCompositions` API. Distinct IDs compose into one sequence;
duplicate IDs fail strict composition. Overlays continue replacing sequences.
Here `required: true` means at least one file must remain across the entire
search after exclusions, not that every directory or pattern must produce a
match. One missing root is acceptable if the other supplies a selected file. The
filename convention keeps helper files out of this discovery declaration; it
does not prevent a selected file from explicitly importing a helper. Existing
repeated-source rules still apply if declarations select overlapping sources.
### Select A Directory Layout With A Glob And Exclusions
```yaml
configkit:
imports:
- glob: 'collections/*/prompts/*.yaml'
exclude: '*.draft.yaml'
required: true
destination: [prompts]
placement: sequence-item
```
This selects files such as `collections/support/prompts/welcome.yaml`, excluding
`collections/support/prompts/welcome.draft.yaml`. It does not select deeper
files such as `collections/support/prompts/internal/help.yaml`. At least one
file must remain after exclusions. The same prompt-item composition setup as
the preceding example applies. Exclusion patterns match basenames even though
the positive glob matches paths.
## Optional Imports By Default
The selected direction is optional-by-default for explicit files, directory
searches, and globs, with `required: true` opting into failure on absence. This
deliberately supersedes the earlier suggestion to retain required-by-default
imports.
The table assumes the library's default absence policy. Applications may select
a different default using the settled precedence rules below.
| Situation | Proposed behavior |
| --- | --- |
| Missing `path` entry; `required` omitted or false | That entry contributes nothing; other listed files may still contribute |
| Zero selected files across a complete search; `required` omitted or false | Contribute nothing |
| Any listed `path` file missing with `required: true` | Fail with an import diagnostic |
| Zero selected files across a complete `dir` or `glob` search after exclusions with `required: true` | Fail with an import diagnostic |
| One `dir` root missing, but another supplies a selected file with `required: true` | Satisfy the declaration's presence requirement |
| Invalid pattern or malformed import declaration | Fail |
| Permission failure during discovery, unreadable selected file, or malformed YAML in a selected file | Fail |
| Composition conflict or other invalid supplied configuration | Fail under the applicable existing rules |
An optional declaration is permission for absence, not permission to ignore a
selected contribution that cannot be read or interpreted. Optionality must not
turn processing failures into apparently successful partial configuration.
This default supports installations with different optional components without
requiring ceremony for every potentially absent directory or file. Its tradeoff
is that a misspelled path or pattern can also contribute nothing. Authors and
applications that need an import to exist can state that requirement explicitly;
applications remain responsible for validating their necessary final values.
Final validation does not automatically detect every unintentionally omitted
contribution.
## Who Chooses Strictness?
**OD-5: settled.** Provide load-level defaults for `required`, `follow`, `hidden`,
`recursive`, and `pattern`. Resolve each applicable field independently using
this precedence:
1. Explicit value on the import declaration.
2. Application-selected default, when supplied.
3. Library default: false for `required` and `recursive`, true for `follow` and
`hidden`, and no filename restriction for `pattern`.
An explicit `pattern` replaces the application default rather than combining
with it. In particular, `pattern: []` selects nothing, even when the application
supplies patterns. An application can default to `['*.yml', '*.yaml']`; without
either an application default or an explicit pattern, a directory search
attempts to load all files in its search scope. Application defaults apply only
to selectors that support the field; they do not add inapplicable options to
other selectors.
Apply this rule to transitive declarations too. A declaration does not inherit
these field values from its importing parent. Applications may expose the
defaults to operators through their own interfaces; simple consumers need no
additional setup. Exact Go API spelling belongs to implementation planning.
Explicit declarations remain authoritative, permitting intentional exceptions
to stricter application defaults. Defaults are not enforcement controls. Do not
add options that override explicit declarations as part of this work without a
separate concrete requirement and scope decision. The fields' applicability to
each selector is defined under directive validation.
## Evidence For Absent Optional Imports
**OD-6: settled.** Record minimal structured inspection evidence when an
optional declaration selects nothing: its declaring source/location, selector,
and absent outcome. Emit no warning by default and do not fabricate a
loaded-source record. This keeps optional contributions quiet during ordinary
use while allowing applications to explain omissions during troubleshooting.
Decide the exact inspection record shape during implementation planning,
reusing existing structures where appropriate and retaining existing inspection
contracts. Include ownership tests for the new evidence. This addition does
not require applications to reconstruct absent-import provenance themselves.
## Release Boundary
Ship one final interface with a hard break in a pre-1.0 minor release, following
the [release policy](../policy/release.md#versioning-and-compatibility). There is
no transition release, legacy parser path, compatibility switch, or temporary
`optional` alias. Release notes should describe the changed behavior and final
syntax; this documentation does not imply migration machinery.
Retaining `glob` is a settled capability decision, as described under filename
matching. It does not preserve the old required-by-default behavior or promise
compatibility with every old declaration or use case.
## Named Algorithms And Implementation Documentation
Use the common names below in implementation comments and maintained
documentation when the feature lands. Keep them beside the relevant logic and
in the appropriate owning documents so human readers can recognize the design
without reconstructing it from code. This does not call for new public API
terminology or duplicate contracts across documents.
| Operation | Name and intended use |
| --- | --- |
| Shared import and `extends` scheduling | **Kahn's algorithm with a stable ordering key**, using a min-heap for the ready set; the same dependency-ready topological sort serves both resolvers |
| Multiple directory roots and filename patterns | **Cartesian product** of roots and patterns, followed by **set union** of matching candidates within the declaration |
| Exclusion filtering | **Set difference** between positive candidates and those matching any exclusion |
| Directory symlink cycles | **Active-path cycle detection**, using canonical directories on the current traversal path rather than treating every previously visited directory as a cycle |
Scalar-to-singleton normalization lets scalar and list forms share selection
logic. Set operations describe membership, not resolution order; the shared
stable topological schedule supplies that order. Preserve the scheduler's
complexity explanation and distinction between cycles and blocked dependents
in its implementation documentation. Do not label implementation-specific
matching or cycle-path helpers with an algorithm name until that algorithm is
actually chosen.
## Directive Validation
**OD-3: settled.** Validate declaration shapes and values independently of
whether discovery selects any files. Invalid declarations fail even when a
search root is missing or a list is empty. This does not require opening or
parsing excluded files or establish a new first-error ordering guarantee.
### Selectors And Applicable Options
Require exactly one selector per declaration: `path`, `dir`, or `glob`. A list
inside `path` or `dir` still counts as one selector. Use separate declarations
when combining different selector kinds. Diagnose unknown fields and options
that have no meaning for the chosen selector. The removed `optional` spelling
is not accepted.
| Option | `path` | `dir` | `glob` |
| --- | --- | --- | --- |
| `required`, `exclude` | Yes | Yes | Yes |
| `destination`, `placement` | Yes | Yes | Yes |
| `pattern` | No | Yes | No |
| `recursive`, `depth` | No | Yes | No |
| `follow`, `hidden` | No | Yes | Yes |
### Types And Depth Applicability
- `path`, `dir`, `pattern`, and `exclude` accept strings or flat lists of strings.
- `glob` accepts a string.
- `required`, `recursive`, `follow`, and `hidden` accept booleans.
- `depth` accepts nonnegative integers under ConfigKit's existing YAML scalar
rules. Zero means unlimited when recursion is enabled; negative, string,
boolean, and fractional values are invalid. Do not introduce an arbitrary
maximum depth.
- Explicit null is invalid for these fields, including null list members.
Do not coerce non-string members to strings or flatten nested lists.
- Existing `destination` and `placement` validation remains unchanged.
A supplied `depth` additionally requires effective recursion to be enabled,
using the default precedence defined above. Valid type and range alone do not
make depth applicable. This rule applies even to empty directory or pattern
lists. With recursion enabled, malformed depth values still fail validation.
### Empty Lists, Empty Strings, And Duplicates
Empty lists are valid and retain their set meanings:
| Declaration | Meaning | Effect of `required: true` |
| --- | --- | --- |
| `path: []` | No named files | Presence requirement satisfied: no listed file is missing |
| `dir: []` | No search roots | Fail: no selected file |
| `dir: fragments` with `pattern: []` | No positive patterns; select nothing | Fail: no selected file |
| `exclude: []` | Exclude nothing | Otherwise unchanged |
An empty pattern list differs from omission: omission uses the application
default if supplied, otherwise all filenames. An explicit empty list selects
nothing and overrides any application default.
Empty strings are invalid for `path`, `dir`, and `glob`, including empty entries
in path or directory lists. Use `.` to explicitly name the declaring directory
as a search root. Empty strings in `pattern` and `exclude` are valid wildcard
expressions that match no actual filename. Other source-path and wildcard
validation rules still apply.
Repeated identical path entries within one declaration select that candidate
once. Repeated patterns or exclusions have no additional effect, and directory
search overlaps follow the settled set-union rule. These rules do not suppress
existing repeated-source errors across separate declarations or collapse
separate symlink aliases.
Validate every supplied wildcard expression even if it is inactive because
another list is empty or discovery finds no files. Likewise, malformed list
members remain errors even when other entries are valid. Do not silently drop
invalid members or treat malformed filters as absent.