671 lines
36 KiB
Markdown
671 lines
36 KiB
Markdown
# Import Ergonomics And Permissive Defaults
|
||
|
||
Status: working draft / brainstorming notes. This document captures a proposed
|
||
direction for discussion, not an implemented contract or an implementation plan.
|
||
The selected direction separates directory searches, filename matching,
|
||
recursion, and absence handling. Optional-by-default imports are the intended
|
||
direction. The consumer-directed and Ansible-aligned decisions below are settled;
|
||
remaining choices have recommendations and alternatives under Open Decisions.
|
||
All new syntax below is illustrative and unimplemented.
|
||
|
||
## Guiding Philosophy
|
||
|
||
ConfigKit should provide predictable configuration mechanisms and avoid imposing
|
||
application policy. Its defaults should permit optional contributions to be
|
||
absent, with explicit options for increased strictness when needed. That choice
|
||
may come from an operator or the consuming application; authored import
|
||
declarations can also express requirements on contributions.
|
||
|
||
Permissiveness should distinguish absence from invalid supplied configuration:
|
||
|
||
> Optional contributions may be absent by default. Consumers can explicitly
|
||
> require them. Supplied configuration must still be structurally valid and
|
||
> interpretable under the selected composition rules.
|
||
|
||
This is a general design direction worth making more explicit and consistent
|
||
across ConfigKit. It is not blanket authorization to weaken existing contracts,
|
||
ignore errors, or add a global permissiveness switch. Other defaults should be
|
||
evaluated individually against this principle.
|
||
|
||
Discovery should accept hidden entries and follow symlinks by default, with
|
||
options to narrow either behavior. An omitted filename filter should not impose
|
||
an implicit extension restriction. These defaults do not waive containment,
|
||
source validity, composition, or resource-limit rules. Recursion remains a
|
||
separate choice about search scope, not a measure of permissiveness.
|
||
|
||
This is a hard-breaking pre-1.0 change. Do not build temporary aliases,
|
||
translation layers, compatibility modes, or staged deprecation support. Evaluate
|
||
the final interface on its own merits, without retaining syntax solely to ease
|
||
upgrades.
|
||
|
||
## Current Baseline
|
||
|
||
The [current import contract](../config.md#explicit-imports) uses `path` or
|
||
`glob`, with required imports by default and `optional: true` to permit absence.
|
||
Glob matching uses slash-based Go `path.Match` syntax. Recursive `**` syntax is
|
||
not supported; authors can match particular depths with patterns such as
|
||
`prompts/*.yaml` and `prompts/*/*.yaml`.
|
||
|
||
Current imports expand depth-first in declaration order, with each ordinary
|
||
source contributing its body before its imports. The existing
|
||
[sibling inheritance resolver](../config.md#sibling-inheritance) instead follows
|
||
dependency chains before resolving dependents, using sorted sibling names and
|
||
an explicit stack. Neither implements the shared ordering selected below.
|
||
|
||
The proposed changes concern naming, discovery, absence handling, and uniform
|
||
dependency ordering for imports and `extends`. Import destinations, strict
|
||
composition, inheritance overlay semantics, tier precedence, and resource
|
||
identity retain their distinct roles.
|
||
|
||
## Proposed Import Vocabulary
|
||
|
||
Use direct-file, directory-search, and path-glob forms. Directory searches
|
||
separate literal search roots from filename filters; path globs additionally
|
||
support wildcard directory segments and selection at particular depths:
|
||
|
||
```yaml
|
||
configkit:
|
||
imports:
|
||
- dir: prompts
|
||
pattern: '*.yaml'
|
||
recursive: true
|
||
destination: [prompts]
|
||
placement: sequence-item
|
||
- path: defaults.yaml
|
||
required: true
|
||
```
|
||
|
||
The first declaration searches `prompts` and its descendants for matching
|
||
filenames, placing each source mapping as one prompt item, and permits no
|
||
matches. The second explicitly requires its target file. Sequence composition
|
||
still requires the consumer's existing registration, as explained below.
|
||
|
||
The fields have distinct jobs:
|
||
|
||
| Field | Proposed meaning |
|
||
| --- | --- |
|
||
| `path` | Import a named file or list of named files, relative to the declaring file |
|
||
| `dir` | Search a literal directory or list of literal directories, relative to the declaring file |
|
||
| `pattern` | A filename wildcard or list of wildcards; select files matching any pattern within the searched directories |
|
||
| `glob` | Select files with one path wildcard, including wildcard directory segments, relative to the declaring file |
|
||
| `exclude` | A filename wildcard or list of wildcards; remove matching files from any selector's results |
|
||
| `recursive` | Default false; include descendant directories when true |
|
||
| `depth` | Bound directory-search depth; 1 means direct files, 0 or omission means unlimited when recursive |
|
||
| `follow` | Follow symlinks by default; false disables following within the discovery scope defined below |
|
||
| `hidden` | Include hidden entries by default; false excludes hidden files and prunes hidden directories within the discovery scope defined below |
|
||
| `required` | For `path`, require every listed file to be present; for `dir` or `glob`, require at least one selected file after exclusions across the complete search |
|
||
| `destination`, `placement` | Apply the existing placement rules to each selected source |
|
||
|
||
This follows the familiar separation of traversal and filtering in Ansible's
|
||
[find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/find_module.html)
|
||
and [include_vars](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/include_vars_module.html)
|
||
interfaces, without adopting all their spellings, defaults, or matching modes.
|
||
|
||
## Decisions Resolved By Project Direction
|
||
|
||
- **No transition support.** Use the final interface immediately in the selected
|
||
pre-1.0 minor release. Remove `optional` in favor of `required`; do not accept
|
||
it as an alias or normalize old declarations at runtime.
|
||
- **Optional contributions by default.** `required: true` opts into failure on
|
||
absence. Supplied invalid configuration continues to fail.
|
||
- **No implicit filename restriction.** Omitting `pattern` selects all
|
||
filenames. Authors can narrow selection explicitly. There is no default YAML
|
||
extension list; extensionless sources remain discoverable.
|
||
- **Follow symlinks by default, with an opt-out.** Provide `follow: false` for
|
||
callers who want discovery without symlink following. Applicable containment
|
||
and filesystem contracts remain in force.
|
||
- **Include hidden entries by default, with an opt-out.** Provide `hidden: false`
|
||
to exclude hidden files and prune hidden directories from discovery.
|
||
- **One dependency-ordering algorithm for imports and `extends`.** Both use the
|
||
shared dependency-ready scheduler defined below. This is settled independently
|
||
of the existing implementations and includes refactoring both resolvers.
|
||
|
||
OD-1 (recursion default), OD-2 (matching and selector expressiveness), OD-4
|
||
(discovery boundaries and cycles), OD-5 (policy precedence), and OD-6
|
||
(absent-import evidence) are settled in the sections below. Only OD-3 remains
|
||
open. `follow` and `hidden` are the proposed field spellings; their applicability
|
||
to `glob` remains part of OD-3. Applying `exclude` to `glob` is settled.
|
||
|
||
## Decisions Resolved From Ansible
|
||
|
||
The comparison covers `ansible.builtin.find`, `ansible.builtin.include_vars`,
|
||
`ansible.builtin.fileglob`, and `ansible.windows.win_find`. A consistent approach
|
||
means agreement among the relevant interfaces that offer the capability, not
|
||
that every Ansible module exposes it. Existing ConfigKit contracts and the
|
||
already selected optional-by-default direction remain authoritative; this is
|
||
not a general Ansible compatibility mode.
|
||
|
||
The following choices are settled:
|
||
|
||
- **Separate traversal from filename filtering.** Directory roots and filename
|
||
filters have separate roles; filters operate on basenames in visited
|
||
directories. A directory failing the filename filter does not prune its
|
||
descendants. This agrees with the directory-search interfaces above.
|
||
See [find's parameters](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/find_module.html#parameters)
|
||
and the [include_vars implementation](https://github.com/ansible/ansible/blob/stable-2.20/lib/ansible/plugins/action/include_vars.py).
|
||
- **Include an optional integer `depth` control.** All three directory-search
|
||
interfaces use that spelling. Direct files are at depth 1, and each additional
|
||
directory adds one. Omission permits unlimited depth when recursion is
|
||
enabled. See [include_vars](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/include_vars_module.html#parameters)
|
||
and [win_find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/windows/win_find_module.html#parameters).
|
||
- **Treat `depth: 0` as unlimited.** This follows `include_vars` and agrees with
|
||
the current [find implementation](https://github.com/ansible/ansible/blob/stable-2.20/lib/ansible/modules/find.py)
|
||
and [win_find implementation](https://github.com/ansible-collections/ansible.windows/blob/main/plugins/modules/win_find.ps1).
|
||
The latter two document unlimited omission more clearly than explicit zero;
|
||
this choice uses source-confirmed behavior, not a claimed universal documented
|
||
Ansible guarantee. Negative values remain a separate open decision.
|
||
- **Disabling recursion takes precedence over depth.** With `recursive: false`,
|
||
search only direct files, even if a valid larger or unlimited depth is supplied.
|
||
Depth does not enable recursion. Both
|
||
[find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/find_module.html#parameters)
|
||
and [win_find](https://docs.ansible.com/projects/ansible/latest/collections/ansible/windows/win_find_module.html#parameters)
|
||
explicitly use this rule where a separate recursion switch exists.
|
||
- **Directory searches select files, not directory documents.** Directories are
|
||
traversal containers, even when their names match `pattern`. This agrees with
|
||
the default file selection in the find modules and the file-oriented
|
||
[fileglob lookup](https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/fileglob_lookup.html).
|
||
Existing regular-file and containment rules still govern opening selected
|
||
sources, together with the selected discovery controls.
|
||
|
||
The accepted `recursive` spelling is retained. The formerly tentative
|
||
`max_depth` spelling becomes `depth`. Resource budgets remain separate from
|
||
depth, and the current limits still apply to an unlimited-depth search.
|
||
|
||
## Shared Dependency-Ready Ordering
|
||
|
||
**Status: settled for imports and `extends`; not yet implemented.** Use
|
||
deterministic dependency-ready ordering: Kahn's topological-sort algorithm with
|
||
a min-heap of eligible nodes. Both resolvers must use the same algorithm in the
|
||
same way, through one shared internal scheduling implementation, common
|
||
tie-breaking rules, and common cycle-path logic.
|
||
|
||
### Scheduling Rule
|
||
|
||
A dependency edge means that the dependent cannot complete until its dependency
|
||
has completed. For each dependency graph:
|
||
|
||
1. Count each node's outstanding dependencies and identify initially eligible
|
||
nodes: those with none.
|
||
2. Choose the eligible node with the smallest stable ordering key.
|
||
3. Append it to the schedule, decrement its dependents' outstanding counts, and
|
||
immediately make newly eligible nodes available for the next selection.
|
||
4. Continue until all nodes are scheduled. If unfinished nodes remain with no
|
||
eligible node, reject the graph and identify an actual dependency cycle.
|
||
|
||
Use stable source or definition identities to construct ordering keys, with
|
||
the same deterministic comparison rule in both resolvers. Map iteration order
|
||
and discovery-assigned source IDs must not supply the tie-breaker. Specify key
|
||
construction for source namespaces and logical definition paths during
|
||
implementation planning; it must distinguish nodes within each scheduled graph.
|
||
|
||
Construct and validate a complete schedule for the available graph before
|
||
applying its resolution operations. A blocked dependent is not necessarily a
|
||
cycle member: diagnostics should identify the actual cycle using shared,
|
||
deterministic cycle-path logic. Source discovery and parsing may already have
|
||
occurred; this rule is not a promise to avoid reading invalid configurations.
|
||
|
||
### Scope And Integration
|
||
|
||
Discovery and resolution are separate operations. An importing source must be
|
||
read to discover its imports, but its contribution need not be assembled first.
|
||
Discovery order is not a substitute for the selected resolution schedule.
|
||
|
||
Both resolvers construct dependency graphs and apply resolver-specific operations
|
||
through the shared scheduler. Imports retain strict composition; `extends`
|
||
retains inheritance overlay behavior. Scheduling does not authorize replacement
|
||
within a strict composition tier or change precedence between tiers. Shared
|
||
dependencies can be scheduled once where valid; this does not relax the existing
|
||
prohibition on repeated source inclusion for imports.
|
||
|
||
Preserve item-local helper scopes and enclosing-before-nested inheritance
|
||
requirements. An imported item completes its helper contributions before being
|
||
placed as an item. Enclosing inheritance establishes the context in which nested
|
||
relationships are discovered and resolved. Use the same scheduler at those
|
||
scope boundaries, with the actual prerequisites represented; do not flatten raw
|
||
files or mappings in a way that loses their scope. Inheritance still resolves
|
||
after source assembly and precedence have established the effective document.
|
||
|
||
The integration must account for resulting sequence positions, provenance,
|
||
history, and diagnostic ordering. It must distinguish source acquisition records
|
||
from assembly order rather than assuming they are the same. Review both
|
||
resolvers' contracts and behavioral tests when implementing the new schedule;
|
||
preserving the old traversal is not a design objective.
|
||
|
||
### Rationale And Validation
|
||
|
||
Dependency-ready ordering selects the smallest currently eligible node without
|
||
adding branch preference, batch barriers, or distance-from-root priority. With
|
||
a fixed total ordering key, it gives the lexicographically smallest valid
|
||
dependency-first schedule. It can interleave independent branches; branch
|
||
contiguity is not part of the chosen ordering contract.
|
||
|
||
The comparison considered deterministic depth-first postorder, dependency
|
||
rounds, and descending longest-path depth as well. All can satisfy dependency
|
||
correctness, but their additional ordering policies did not justify choosing
|
||
them over the uniform eligibility rule. This decision gives no preference to
|
||
the existing implementation or to minimizing pre-1.0 refactoring.
|
||
|
||
The scheduler requires O(E + V log V) time and O(V + E) memory for V nodes and
|
||
E dependency edges, assuming constant-time key comparisons. Source access,
|
||
parsing, merging, and provenance work are separate costs.
|
||
|
||
Validate independent branches, unequal chains, branches with multiple
|
||
dependencies, shared dependencies at equal and unequal depths, cycles with
|
||
blocked non-cycle dependents, and nested scopes. Equivalent valid graphs and
|
||
ordering keys must produce identical schedules through the import and
|
||
inheritance integrations. Verify independence from graph insertion order and
|
||
preserve each resolver's domain-specific admissibility and combination rules.
|
||
|
||
## Discovery Semantics
|
||
|
||
### Filename Matching
|
||
|
||
`pattern` applies to a file's basename, not its full relative path. For example,
|
||
`*.yaml` matches both `prompts/one.yaml` and
|
||
`prompts/weather/daily/summary.yaml` when the latter's directory is visited.
|
||
Recursion changes which directories are visited; it does not change what `*`
|
||
means. A nonrecursive search considers only files directly inside `dir`.
|
||
|
||
**OD-2: settled.** Use the existing case-sensitive Go `path.Match` wildcard
|
||
grammar for filename filters. Omitting `pattern` selects all filenames, with no
|
||
implicit extension restriction. Do not add regular expressions, automatic
|
||
dialect detection, or recursive `**` syntax as part of this work.
|
||
|
||
Retain `glob` as a permanent, separate selector using the existing slash-based
|
||
path wildcard grammar. For example, `collections/*/prompts/*.yaml` selects
|
||
files in a particular directory layout without selecting all descendants.
|
||
Literal `dir` roots and basename filters do not express that same search.
|
||
This capability justifies retaining `glob` independently of compatibility;
|
||
it uses the same new absence policy as other selectors. `glob` remains a
|
||
single string; list support is being added to `path`, `dir`, and the filename
|
||
filters.
|
||
|
||
### Selector Lists And Set Operations
|
||
|
||
Accept a string or a list of strings for `path`, `dir`, and `pattern`. Normalize
|
||
a scalar to a singleton list so both forms use the same selection logic.
|
||
Listed paths and directory roots resolve relative to the declaring file.
|
||
List order does not introduce precedence or replace the shared dependency-ready
|
||
resolution schedule.
|
||
|
||
For directory roots D and filename patterns P, search the **Cartesian product**
|
||
D × P and take the **union** of the matching candidate files:
|
||
|
||
`S = ⋃ {matches(d, p) | (d, p) ∈ D × P}`
|
||
|
||
Each root uses the declaration's recursion, depth, and applicable discovery
|
||
controls. When `pattern` is omitted, select all filenames in the visited
|
||
directories instead. This describes selection semantics, not a requirement to
|
||
materialize the product or traverse a directory again for each pattern.
|
||
|
||
Overlapping patterns or directory searches within one declaration select the
|
||
same candidate once. This local set union is not global source deduplication:
|
||
existing repeated-source and alias checks still apply to distinct inclusion
|
||
routes. In particular, it does not authorize silently merging separate alias
|
||
paths to an imported source.
|
||
|
||
For `path`, `required: true` requires every listed file to be present. For `dir`,
|
||
it requires at least one selected file across the complete union after
|
||
exclusions; it does not require a match per directory, pattern, or pair. A
|
||
missing directory can therefore contribute nothing while another supplies the
|
||
required match. The equivalent `glob` requirement is at least one selected file
|
||
after exclusions. Invalid supplied configuration still fails under the existing
|
||
rules.
|
||
|
||
### Exclusions
|
||
|
||
Accept `exclude` as a string or list of strings with the same case-sensitive
|
||
basename wildcard grammar as `pattern`. Apply it to results from `path`, `dir`,
|
||
and `glob`. A candidate matching any exclusion is removed before its content is
|
||
loaded or parsed. An omitted exclusion removes nothing.
|
||
|
||
This is **set difference**: for candidate set S and its subset X matching any
|
||
exclusion, the selected set is `S ∖ X`. Exclusions win over positive matches.
|
||
They filter filenames, not directory paths, and do not prune traversal into a
|
||
directory whose name happens to match. For example, `exclude: '*.draft.yaml'`
|
||
removes matching files at every visited depth, including results of a path glob.
|
||
|
||
For searches, evaluate `required` on the selected set after exclusions. For an
|
||
explicit `path` list, exclusions filter what is loaded but do not waive the
|
||
settled requirement that every listed file be present when `required: true`.
|
||
Excluding a candidate is not permission to suppress discovery, containment,
|
||
or resource-limit errors. Exclusions are local to the declaration; they do not
|
||
prohibit another declaration from explicitly selecting the same filename.
|
||
|
||
### Recursive Discovery And Depth Limits
|
||
|
||
Use `recursive: true` for recursive discovery, optionally bounded by `depth`.
|
||
The motivation is to let authors organize definitions in subdirectories without
|
||
updating import declarations for every depth. Directory organization should not
|
||
change resource identity or the effective configuration destination.
|
||
|
||
The selected model uses an explicit search root and recursion flag, rather than
|
||
introducing recursive `**` syntax or inferring a root from a wildcard prefix.
|
||
|
||
**OD-1: settled.** Omitting `recursive` means false: search only direct files.
|
||
Authors explicitly opt into descendants. This makes the search scope visible
|
||
while keeping recursion independent of the permissive defaults for absence,
|
||
symlinks, and hidden entries.
|
||
|
||
The settled depth convention is:
|
||
|
||
- Files directly inside the search root are at depth `1`.
|
||
- Each additional directory increases depth by one.
|
||
- Omitting `depth`, or setting it to 0, imposes no caller-selected depth limit
|
||
when recursion is enabled; existing loader resource limits still apply.
|
||
- `recursive: false` selects only direct files regardless of a valid depth value.
|
||
- Depth controls discovery only, not namespaces, identities, or destinations.
|
||
|
||
Remaining directive validation choices are listed below. Recursive discovery
|
||
must fit the existing filesystem containment, repeated-source, and load-budget
|
||
contracts rather than silently changing them. In particular, examined entries,
|
||
including nonmatches, still consume the visit budget; a small number of matches
|
||
does not authorize an unbounded filesystem scan. This is an existing ConfigKit
|
||
constraint, not a new Ansible-derived decision.
|
||
|
||
### Symlinks And Hidden Entries
|
||
|
||
**OD-4: settled.** Discovery controls are local to the declaration. Directory
|
||
searches default to `follow: true` and `hidden: true`, subject to the application
|
||
defaults described below. Authors may disable either behavior independently of
|
||
recursion, depth, matching, or absence handling.
|
||
|
||
Define hidden entries by a leading dot in the portable source namespace.
|
||
`hidden: false` skips discovered hidden files and prunes discovered hidden
|
||
directories. `follow: false` skips discovered symlink files and prevents descent
|
||
through discovered symlink directories. Explicitly named roots and separate
|
||
direct-file imports retain their existing source-access semantics. The controls
|
||
do not implicitly propagate to declarations inside imported files.
|
||
|
||
These are discovery-selection controls, not a load-wide access prohibition.
|
||
If OD-3 enables these controls on `glob`, filtering applies
|
||
to wildcard-selected entries; explicitly named path components remain explicit
|
||
access. This conditional scope rule does not settle that open decision.
|
||
|
||
When following links, detect a canonical directory repeated on the active
|
||
traversal path and fail with a cycle diagnostic. Do not silently prune the loop
|
||
and continue. Existing dependency-cycle and repeated-source errors remain in
|
||
force; discovered aliases do not authorize silently deduplicating imported
|
||
sources. Directory-cycle handling retains the existing containment and
|
||
native-filesystem capabilities.
|
||
|
||
These controls narrow selection rather than authorize ignoring errors in a
|
||
selected source. `required: true` applies after discovery filters: if the search
|
||
selects no file, it fails. Native filesystems retain their existing capabilities;
|
||
this feature does not promise link inspection where a supplied filesystem does
|
||
not expose it.
|
||
|
||
## Illustrative Examples
|
||
|
||
These examples describe the proposed interface, not configuration accepted by
|
||
the current release. They assume the consumer has enabled imports and has not
|
||
changed the library's policy defaults. File paths and directory roots resolve
|
||
from the declaring file; destinations use the
|
||
existing relative-placement rules.
|
||
|
||
### Import One File
|
||
|
||
```yaml
|
||
configkit:
|
||
imports:
|
||
- path: defaults.yaml
|
||
```
|
||
|
||
Import `defaults.yaml` into the declaring source's mapping scope. Under the
|
||
proposed default, a missing file contributes nothing. Add `required: true` when
|
||
its presence is necessary. A present but invalid file still fails loading.
|
||
|
||
### Require Several Named Files
|
||
|
||
```yaml
|
||
configkit:
|
||
imports:
|
||
- path: [defaults.yaml, site.yaml]
|
||
required: true
|
||
```
|
||
|
||
Both files must be present. Listing them together shares the declaration's
|
||
options; their textual order does not establish overlay precedence. Their
|
||
contributions follow the shared dependency-ready schedule and strict
|
||
composition rules.
|
||
|
||
### Recursively Discover Mapping Fragments
|
||
|
||
```yaml
|
||
configkit:
|
||
imports:
|
||
- dir: services
|
||
pattern: '*.yaml'
|
||
recursive: true
|
||
destination: [services]
|
||
```
|
||
|
||
This discovers files such as `services/api.yaml` and
|
||
`services/regional/worker.yaml`, but not `services/README.md`. Each file contains
|
||
named service entries, for example `api: {port: 8080}` or
|
||
`worker: {concurrency: 4}`. They compose beneath `services` under the existing
|
||
strict composition rules. Directory names do not create additional mapping
|
||
keys. Missing directories or no matching files contribute nothing.
|
||
|
||
Adding `depth: 2` limits discovery to direct files and files one directory below
|
||
`services`: it still selects `services/regional/worker.yaml`, but not
|
||
`services/regional/east/worker.yaml`. With `recursive: false`, only direct files
|
||
are selected, even if `depth: 2` remains present.
|
||
|
||
Hidden entries and contained symlinks participate by default. To narrow the
|
||
search, the declaration can also include:
|
||
|
||
```yaml
|
||
follow: false
|
||
hidden: false
|
||
```
|
||
|
||
These controls narrow discovery below the explicitly named root; the root
|
||
continues to resolve under the existing source-access rules.
|
||
|
||
### Search Several Prompt Collections And Filename Patterns
|
||
|
||
```yaml
|
||
configkit:
|
||
imports:
|
||
- dir: [prompts, shared-prompts]
|
||
pattern: ['*.prompt.yaml', '*.prompt.yml']
|
||
exclude: ['*.draft.prompt.yaml', '*.draft.prompt.yml']
|
||
recursive: true
|
||
required: true
|
||
destination: [prompts]
|
||
placement: sequence-item
|
||
```
|
||
|
||
The two directories and two patterns form four Cartesian-product pairs; the
|
||
selected candidates are the union of their matches, minus excluded filenames.
|
||
This selects `prompts/welcome.prompt.yaml`,
|
||
`prompts/weather/daily/summary.prompt.yaml`, and
|
||
`shared-prompts/greeting.prompt.yml`, but not `prompts/weather/helpers.yaml`
|
||
or `shared-prompts/greeting.draft.prompt.yml`.
|
||
Each selected file contains one prompt mapping,
|
||
such as `id: prompt.welcome` and `text: Welcome`. The consumer registers the
|
||
`prompts` sequence with identity field `id` through the existing
|
||
`Definition.SequenceCompositions` API. Distinct IDs compose into one sequence;
|
||
duplicate IDs fail strict composition. Overlays continue replacing sequences.
|
||
|
||
Here `required: true` means at least one file must remain across the entire
|
||
search after exclusions, not that every directory or pattern must produce a
|
||
match. One missing root is acceptable if the other supplies a selected file. The
|
||
filename convention keeps helper files out of this discovery declaration; it
|
||
does not prevent a selected file from explicitly importing a helper. Existing
|
||
repeated-source rules still apply if declarations select overlapping sources.
|
||
|
||
### Select A Directory Layout With A Glob And Exclusions
|
||
|
||
```yaml
|
||
configkit:
|
||
imports:
|
||
- glob: 'collections/*/prompts/*.yaml'
|
||
exclude: '*.draft.yaml'
|
||
required: true
|
||
destination: [prompts]
|
||
placement: sequence-item
|
||
```
|
||
|
||
This selects files such as `collections/support/prompts/welcome.yaml`, excluding
|
||
`collections/support/prompts/welcome.draft.yaml`. It does not select deeper
|
||
files such as `collections/support/prompts/internal/help.yaml`. At least one
|
||
file must remain after exclusions. The same prompt-item composition setup as
|
||
the preceding example applies. Exclusion patterns match basenames even though
|
||
the positive glob matches paths.
|
||
|
||
## Optional Imports By Default
|
||
|
||
The selected direction is optional-by-default for explicit files, directory
|
||
searches, and globs, with `required: true` opting into failure on absence. This
|
||
deliberately supersedes the earlier suggestion to retain required-by-default
|
||
imports.
|
||
|
||
The table assumes the library's default absence policy. Applications may select
|
||
a different default using the settled precedence rules below.
|
||
|
||
| Situation | Proposed behavior |
|
||
| --- | --- |
|
||
| Missing `path` entry; `required` omitted or false | That entry contributes nothing; other listed files may still contribute |
|
||
| Zero selected files across a complete search; `required` omitted or false | Contribute nothing |
|
||
| Any listed `path` file missing with `required: true` | Fail with an import diagnostic |
|
||
| Zero selected files across a complete `dir` or `glob` search after exclusions with `required: true` | Fail with an import diagnostic |
|
||
| One `dir` root missing, but another supplies a selected file with `required: true` | Satisfy the declaration's presence requirement |
|
||
| Invalid pattern or malformed import declaration | Fail |
|
||
| Permission failure during discovery, unreadable selected file, or malformed YAML in a selected file | Fail |
|
||
| Composition conflict or other invalid supplied configuration | Fail under the applicable existing rules |
|
||
|
||
An optional declaration is permission for absence, not permission to ignore a
|
||
selected contribution that cannot be read or interpreted. Optionality must not
|
||
turn processing failures into apparently successful partial configuration.
|
||
|
||
This default supports installations with different optional components without
|
||
requiring ceremony for every potentially absent directory or file. Its tradeoff
|
||
is that a misspelled path or pattern can also contribute nothing. Authors and
|
||
applications that need an import to exist can state that requirement explicitly;
|
||
applications remain responsible for validating their necessary final values.
|
||
Final validation does not automatically detect every unintentionally omitted
|
||
contribution.
|
||
|
||
## Who Chooses Strictness?
|
||
|
||
**OD-5: settled.** Provide load-level defaults for `required`, `follow`, and
|
||
`hidden`. Resolve each applicable field independently using this precedence:
|
||
|
||
1. Explicit value on the import declaration.
|
||
2. Application-selected default, when supplied.
|
||
3. Library default: false for `required`, true for `follow` and `hidden`.
|
||
|
||
Apply this rule to transitive declarations too. A declaration does not inherit
|
||
these field values from its importing parent. Applications may expose the
|
||
defaults to operators through their own interfaces; simple consumers need no
|
||
additional setup. Exact Go API spelling belongs to implementation planning.
|
||
|
||
Explicit declarations remain authoritative, permitting intentional exceptions
|
||
to stricter application defaults. Defaults are not enforcement controls. Do not
|
||
add options that override explicit declarations as part of this work without a
|
||
separate concrete requirement and scope decision. The fields' applicability to
|
||
each selector remains part of OD-3.
|
||
|
||
## Evidence For Absent Optional Imports
|
||
|
||
**OD-6: settled.** Record minimal structured inspection evidence when an
|
||
optional declaration selects nothing: its declaring source/location, selector,
|
||
and absent outcome. Emit no warning by default and do not fabricate a
|
||
loaded-source record. This keeps optional contributions quiet during ordinary
|
||
use while allowing applications to explain omissions during troubleshooting.
|
||
|
||
Decide the exact inspection record shape during implementation planning,
|
||
reusing existing structures where appropriate and retaining existing inspection
|
||
contracts. Include ownership tests for the new evidence. This addition does
|
||
not require applications to reconstruct absent-import provenance themselves.
|
||
|
||
## Release Boundary
|
||
|
||
Ship one final interface with a hard break in a pre-1.0 minor release, following
|
||
the [release policy](../policy/release.md#versioning-and-compatibility). There is
|
||
no transition release, legacy parser path, compatibility switch, or temporary
|
||
`optional` alias. Release notes should describe the changed behavior and final
|
||
syntax; this documentation does not imply migration machinery.
|
||
|
||
Retaining `glob` is a settled capability decision, as described under filename
|
||
matching. It does not preserve the old required-by-default behavior or promise
|
||
compatibility with every old declaration or use case.
|
||
|
||
## Named Algorithms And Implementation Documentation
|
||
|
||
Use the common names below in implementation comments and maintained
|
||
documentation when the feature lands. Keep them beside the relevant logic and
|
||
in the appropriate owning documents so human readers can recognize the design
|
||
without reconstructing it from code. This does not call for new public API
|
||
terminology or duplicate contracts across documents.
|
||
|
||
| Operation | Name and intended use |
|
||
| --- | --- |
|
||
| Shared import and `extends` scheduling | **Kahn's algorithm with a stable ordering key**, using a min-heap for the ready set; the same dependency-ready topological sort serves both resolvers |
|
||
| Multiple directory roots and filename patterns | **Cartesian product** of roots and patterns, followed by **set union** of matching candidates within the declaration |
|
||
| Exclusion filtering | **Set difference** between positive candidates and those matching any exclusion |
|
||
| Directory symlink cycles | **Active-path cycle detection**, using canonical directories on the current traversal path rather than treating every previously visited directory as a cycle |
|
||
|
||
Scalar-to-singleton normalization lets scalar and list forms share selection
|
||
logic. Set operations describe membership, not resolution order; the shared
|
||
stable topological schedule supplies that order. Preserve the scheduler's
|
||
complexity explanation and distinction between cycles and blocked dependents
|
||
in its implementation documentation. Do not label implementation-specific
|
||
matching or cycle-path helpers with an algorithm name until that algorithm is
|
||
actually chosen.
|
||
|
||
## Open Decisions
|
||
|
||
Only OD-3 remains open. Its identifier is preserved for continued discussion;
|
||
the recommendations below are not settled decisions. OD-1, OD-2, OD-4, OD-5,
|
||
and OD-6 are settled above.
|
||
|
||
### OD-3. Directive Validation
|
||
|
||
**Why still open:** Permissive discovery does not determine whether malformed or
|
||
inapplicable option values should be accepted. The draft's existing error
|
||
boundary remains: invalid declarations fail. We still need to define their
|
||
valid shapes, rather than infer extra restrictions from that boundary.
|
||
|
||
**Recommendation (not yet accepted):** Accept nonnegative YAML integers for
|
||
`depth`; zero remains unlimited. Treat negative, null, string, boolean, and
|
||
fractional values as malformed, even when recursion is disabled. Use exactly one selector per
|
||
import: `path`, `dir`, or `glob`. Keep
|
||
`pattern`, `recursive`, and `depth` specific to `dir`; allow `follow` and `hidden`
|
||
on both `dir` and `glob`. `exclude` applies to all three selectors as settled
|
||
above. Diagnose unknown fields and options that have no meaning for the chosen
|
||
selector. This makes likely mistakes visible, but shared templates must omit
|
||
inapplicable fields. It imposes no arbitrary maximum depth. The settled
|
||
combination `recursive: false` with a valid larger `depth`
|
||
remains accepted and selects only direct files. The removed `optional` spelling
|
||
is not accepted under any variant.
|
||
|
||
The accepted list forms also need explicit validation rules for empty lists,
|
||
null values, empty strings, and non-string elements. Recommend allowing empty
|
||
lists as empty selections (and `exclude: []` as no exclusions), while rejecting
|
||
null values and non-string elements rather than silently coercing them. Under
|
||
that recommendation, an explicit `pattern: []` matches nothing, unlike omission;
|
||
an empty `path` list has no files whose presence is required. Empty-string
|
||
semantics should be reviewed against the existing source-path and wildcard
|
||
rules rather than introducing a blanket prohibition. These remain OD-3 choices,
|
||
not additional settled restrictions.
|
||
|
||
**Alternative:** Normalize null depth to omission and negative integers to
|
||
unlimited, and permit unused directory controls on a direct-file declaration.
|
||
That eases generated configuration but can turn an erroneous bound into a wider
|
||
search or make a direct-file import appear recursive. These relaxations can be
|
||
chosen independently; they do not require accepting ambiguous selectors.
|
||
Another viable selector shape is to allow multiple selectors as an explicit
|
||
union, but it needs rules for constructing the graph and handling overlaps that
|
||
one-selector declarations avoid.
|
||
|
||
For list validation, another viable choice is to reject empty positive-selector
|
||
or pattern lists as likely authoring mistakes while allowing `exclude: []`.
|
||
That catches accidentally empty generated inputs, but requires conditional
|
||
omission of declarations and is less consistent with permissive empty searches.
|
||
Normalizing null to omission would ease generated configuration, but for
|
||
`pattern` it would broaden selection to all filenames. Neither alternative
|
||
changes the settled string-or-list support for nonempty valid values.
|