# Composable Configuration Discovery Roadmap Status: proposal with the agreed direction, filename groups, and search presets captured below. Remaining behavioral choices are under [Open Decisions](#open-decisions). This document describes future work, not an implemented contract. A staged implementation plan will follow once the decisions are settled. ## Purpose And Ownership Make ordinary file-based loading concise for applications that want conventional configuration locations. ConfigKit supplies useful built-in candidate lists and an ordered selection operation. Consumers continue to own main-file search conventions: they choose which lists apply, their order, custom candidates, and whether a configuration file is required. A consumer can select one preset for a simple application or compose several presets and custom path lists. The application supplies an explicit slug where needed to construct conventional paths. ConfigKit does not infer application identity from the executable name or enable discovery for existing explicit sources. Discovery selects one source for the existing loading lifecycle. It does not create additional configuration tiers or merge every matching file. Existing layered loading remains available when consumers intentionally combine sources. ## Current Baseline The [source contract](../config.md#sources) provides explicit host files, native filesystem files, bytes, readers, mappings, and `NoSource()`. A missing explicit file fails. The zero source is invalid. There is no conventional main-file search. [Ordinary loading](../config.md#single-source-loading) infers a selected overlay's filesystem context from the main source. `NoSource()` currently rejects enabled imports, active overlays, and filesystem context. In [layered loading](../config.md#layered-loading), an absent main input can coexist with underlays; a selected overlay requires an explicit `OverlayFilesystem`. These distinctions matter when an optional search finds nothing. The [architecture policy](../policy/architecture.md#responsibility-boundaries) assigns search conventions to consumers and currently excludes implicit main-file discovery. The addition should clarify that consumers may explicitly select library-provided search conventions. Consumer ownership remains unchanged. ## Agreed Direction ### Filename Groups, Directory Searches, And Search Presets Organize discovery around three composable descriptions. These are ordered lists: their order determines precedence. **Filename groups** provide reusable, ordered filename templates. The built-in groups are: | Group | Filenames, in order | | --- | --- | | Named | `.yml`, `.yaml` | | Generic | `config.yml`, `config.yaml`, `main.yml`, `main.yaml` | `` is the application slug supplied by the consumer. **Directory searches** pair a directory with an ordered list of filename groups. The default selection is `[Named, Generic]`, producing the six filenames above in that order. A directory search may select different groups or reverse their precedence, such as `[Generic, Named]`, while retaining each group's internal order. Consumers can also supply custom filenames. **Search presets** bundle ordered directory searches. Consumers can select a complete preset for the simple case or compose presets, custom directory searches, and explicit file candidates in their desired order. Explicit file candidates are literal paths and receive no filename expansion. ### Ordered Expansion And Concatenation The selection model is **ordered expansion and concatenation**, followed by first-match selection: 1. Expand the selected presets into directory searches in their supplied order. 2. For each directory search, concatenate its selected filename groups in order, substitute the application slug where needed, and join each filename to that directory. 3. Concatenate the resulting candidate lists, preserving the positions of custom directory searches and explicit file candidates. 4. Try candidates in order, stopping at the first selected file or an error as described below. Directory priority comes before filename priority: exhaust one directory's filenames before trying the next directory. When several directories share a filename list, their expansion is a Cartesian product traversed directory first. Different directories may select different lists, so ordered expansion and concatenation describes the complete model. Set union does not express its precedence semantics. The eventual documentation should explain this model using the applicable mathematical nomenclature: ordered lists, ordered concatenation, and Cartesian product where a shared filename list applies. Retain these names in relevant implementation comments where they help readers understand the operations. The model describes ordering without requiring eager construction of every candidate. ### Unix System Preset The agreed Unix system preset is: | Directory, in search order | Filename groups, in order | | --- | --- | | `/usr/local/etc/` | `[Named, Generic]` | | `/etc/` | `[Named, Generic]` | | `/etc` | `[Named]` | This produces 14 candidates: six in each application-specific directory, then `/etc/.yml` and `/etc/.yaml`. For example, `/usr/local/etc//main.yaml` precedes `/etc//.yml`. Selecting only `Named` for `/etc` is an ordinary directory-search description; the selection algorithm needs no special exception. ### Additional Presets (OD-1 — Settled) Provide the following independently selectable presets alongside Unix system. Within each preset, search directories in the order shown: | Preset | Directory | Filename groups, in order | | --- | --- | --- | | User configuration | `/` | `[Named, Generic]` | | User configuration | `` | `[Named]` | | Home directory | `/.` | `[Named, Generic]` | | Home directory | `` | `[Named]` | | Working directory | Current working directory | `[Named]` | Resolve the user-configuration base with [`os.UserConfigDir`](https://pkg.go.dev/os#UserConfigDir), the home base with [`os.UserHomeDir`](https://pkg.go.dev/os#UserHomeDir), and the working-directory base with [`os.Getwd`](https://pkg.go.dev/os#Getwd). Adopt the Go helpers' platform-specific directory and environment behavior, including their errors; the settled OD-2 rules below govern how discovery handles resolution errors. The goal is to support familiar conventions through idiomatic Go. This feature does not add a separate XDG compatibility layer or XDG system preset. A selected user-configuration base has no additional hard-coded `/.config` fallback. The Home directory preset is a separate, explicit choice. The working-directory preset searches only that directory and uses application-named files; it does not walk ancestor directories. The consumer chooses precedence between all presets. Selecting one does not automatically select any others. ### Illustrative Consumer API Names below are illustrative and are not implemented Go API: ```go // A simple system-wide application. source := configkit.SearchConfig("myapp", configkit.UnixSystemDirectories()) ``` ```go // A consumer-selected working-directory, user, then system search. source := configkit.SearchConfig( "myapp", configkit.WorkingDirectory(), configkit.UserConfigDirectory(), configkit.UnixSystemDirectories(), ) ``` ```go // Explicit custom candidates followed by a selected conventional fallback. source := configkit.SearchConfig( "myapp", configkit.Paths("/run/myapp/config.yml", "/opt/myapp/config.yaml"), configkit.UnixSystemDirectories(), ) ``` Use the resulting source with ordinary loading or as a layered main input. Custom searches and file lists participate at their specified position; they do not silently append unselected presets. Reordering presets or their directory searches changes search priority. ### Selection And Failure Behavior | Situation | Selected behavior | | --- | --- | | Candidate is absent | Continue to the next candidate. | | A candidate file is selected | Stop searching and load that file. | | Candidate is inaccessible, unreadable, or not an accepted file type | Fail using normal source diagnostics. | | Selected file fails parsing, version checks, composition, decoding, or validation | Fail without trying another candidate. | | Entire search has no match | Absence is permitted by default; continue through applicable defaults and normal validation. The interaction with other inputs is addressed under OD-3. | | Entire search has no match and is required | Fail with a diagnostic explaining the unsuccessful search. | | Caller explicitly uses `File(path)` | Retain existing exclusive-file behavior, including errors for a missing file. | Requiredness applies to the complete search, not to each preset or directory. Permitting an absent file does not waive application version, schema, or domain requirements; the remaining lifecycle may still reject an incomplete configuration. Perform discovery before parsing the chosen main file. Do not implement fallback by repeatedly running complete loads. Defaults, application resolution, secret lookup, and final validation retain their existing lifecycle execution rules. Once a file has been selected, later failure does not restart discovery. ### Construction And Filesystem Behavior Constructors describe the search without filesystem access. Resolve dynamic base directories and access candidate files during loading, so reusable descriptions can observe changes on subsequent loads. Do not change the process working directory or environment. Reuse the host-file acquisition rules for selected files, including regular-file checks, symlink handling, resource ownership, and source limits. The feature does not require a new filesystem traversal engine, recursive directory search, or new file-size limits. Candidate filenames and groups are bounded by their explicit descriptions, rather than by scanning unrelated directories. The exact Go names, constructors, and representation of search-wide options belong in the implementation plan. Keep the public surface root-owned and keep the single-preset case concise. ### Unavailable Bases And Platform Applicability (OD-2 — Settled) Resolve directory bases only as the ordered search reaches them. Once an earlier candidate is selected, later searches do not resolve their bases. Static invalid search arguments still fail ordinary option validation before source acquisition. If a selected directory's base cannot be resolved, stop and report the error. A missing directory or candidate file beneath a resolved base allows the search to continue; a directory-resolution error does not. This distinction also applies to optional searches: permitting no match does not suppress errors encountered while searching. Preserve the agreed errors for inaccessible or unreadable candidates. Treat using the Unix system preset on an unsupported platform as an explicit configuration error when that preset is reached. Specify its supported platforms in the implementation plan. Do not translate the Unix preset into a different platform's convention. User, home, and working-directory presets use their native Go directory helpers. This keeps search priority predictable when directory resolution fails. Consumers select the presets appropriate to their application and environment; this proposal does not add an option to skip unavailable bases or unsupported presets. ### Placement, Provenance, And Existing Features The selected file participates as the actual file source. Preserve its physical identity, effective placement, source origin, and passive resource context. Relative imports and relative application resources use the selected file's normal context. Discovery does not relocate overlay or CLI coordinates. Missing candidates are not acquired sources and do not create source IDs, configuration history, or explicit input presence. An entirely absent search contributes no synthetic empty configuration file. Diagnostics should distinguish an exhausted required search from a failed candidate. Existing selected-source inspection remains the foundation; a new public per-attempt trace is not needed for this proposal. Preserve collection policies, inheritance, source-version rules, CLI ordering, final validation, and frozen inspection after selection. Search-list order does not change precedence among underlays, the main input, an overlay, and CLI values. An explicitly selected overlay is not another main-file fallback candidate. ## Work Needed From The Baseline The subsequent implementation plan should cover: - Define filename-group, directory-search, preset, and search-source representations, slug validation, and a search-wide requiredness option. Keep construction separate from acquisition and the simple preset selection concise. - Implement ordered expansion and concatenation for presets, filename groups, custom searches, and explicit file candidates, followed by first-match selection through existing file access. Resolve each source once per consumption before entering its parse and composition lifecycle. - Integrate selected and absent outcomes with ordinary and layered main inputs, placement, imports, application version checking, and overlay contexts according to OD-3. Preserve ownership and close acquired handles on success and failure. - Preserve real selected-source metadata and safe diagnostics without treating absent candidates as configuration contributions. - Exercise preset, directory, filename-group, and filename ordering, reversed group precedence, the Unix system preset's 14 candidates and named-only `/etc` search, custom-list composition, both filename extensions, no-match behavior, required searches, unavailable preset bases, errors that must stop fallback, repeated loads, and untouched explicit `File` behavior. Cover imports, overlays, underlays, defaults, CLI, source versions, source contexts, and source-list/inspection evidence at their boundaries. - Clarify the architecture policy's consumer-selected discovery boundary. Record any warranted durable ownership decision in an ADR; keep preset filenames and API mechanics in their canonical contract when implemented. Update consumer examples and the configuration contract when the feature lands. Explain the three-part discovery model and its mathematical terminology in the eventual documentation, with simple preset selection and custom composition examples. This roadmap does not change implementation, policy, or ADRs by itself. Coordinate implementation with the ongoing sequence integration audit rather than assuming its in-progress changes are part of a fixed implementation baseline. ## Open Decisions ### OD-3 — No Match, Imports, And Selected Overlays **Recommendation:** use a shared absent-main lifecycle for optional no-match and explicit `NoSource()`. Allow import enablement when the main file is absent; there are no main-file declarations to traverse. Other actual sources retain their normal import behavior. In layered loads, retain underlays and the existing explicit overlay-filesystem requirement. For ordinary loads with an active overlay and no main file, require an explicit filesystem context. Do not infer an overlay root from the first or last searched directory. A missing selected overlay remains an error. Defaults, CLI overrides, resolution, and validation continue normally, and main-version checking receives absence rather than a fabricated document. This would deliberately relax the current `NoSource()` restrictions on imports and on overlays with explicit context. It keeps one meaning for absent main input and lets optional configuration remain optional without silently ignoring a requested overlay. It requires lifecycle work beyond candidate selection. **Alternative:** map no-match to the current `NoSource()` contract and retain its restrictions. This minimizes lifecycle changes, but an otherwise valid optional search with imports enabled can fail solely because no file was found, and ordinary overlay-only loading would remain unavailable. A special absent-search case could avoid changing `NoSource()`, at the cost of two different absence rules. ### OD-4 — Empty Composition And Implicit Preset Selection **Recommendation:** require explicit preset or search selection to activate a convention. An empty search composition has no candidates: optional search contributes no source, and required search fails as exhausted. Empty custom lists, including empty filename-group selections, follow the same rule. This supports programmatic list construction and introduces no hidden system, user, or working-directory fallback. One explicitly selected preset still gives the simple application a short call. **Alternative:** let omission of searches select a documented default preset. This shortens the simplest call, but omission and an intentionally empty computed list would need distinguishable representations. It also requires choosing a default convention across platforms and application styles. Another alternative is to reject an empty composition as invalid; that catches accidental empty lists but adds a special case beyond the search-wide requiredness rule. ### OD-5 — Slugs And Custom Path Interpretation **Recommendation:** require a nonempty single-component slug when a selected preset uses it to construct filenames. Reject path separators, `.` and `..`, and other forms that cannot be used as that component on the target platform. Preserve spelling; do not silently lowercase, trim, or impose a kebab-case naming scheme. A composition made only of explicit custom paths need not require an application slug. Treat custom paths as literal host file paths. Resolve relative paths against the working directory at load time; do not expand shell variables, `~`, or glob syntax. Consumers can construct expanded paths themselves. Selected presets provide the convenience for user and working-directory bases. **Alternative:** require a restricted portable slug grammar, such as lowercase ASCII letters, digits, and hyphens. This is easy to document and portable, but unnecessarily excludes otherwise usable application names. Automatic shell-like custom-path expansion is also viable, but adds escaping and environment semantics that ordinary `File(path)` does not supply. Literal paths keep both interfaces consistent.