Define discovery filename groups and search presets

docs/roadmap/discovery.md

@@ -1,7 +1,8 @@
 # Composable Configuration Discovery Roadmap
 
-Status: proposal with the agreed direction captured below. Preset contents and
+Status: proposal with the agreed direction, filename groups, and Unix system
-the remaining behavioral choices are under [Open Decisions](#open-decisions).
+preset captured below. Remaining preset and behavioral choices are under
+[Open Decisions](#open-decisions).
 This document describes future work, not an implemented contract. A staged
 implementation plan will follow once the decisions are settled.
 
@@ -43,13 +44,76 @@ library-provided search conventions. Consumer ownership remains unchanged.
 
 ## Agreed Direction
 
-### Ordered Candidate Groups
+### Filename Groups, Directory Searches, And Search Presets
 
-Provide reusable descriptions of built-in search lists and custom ordered paths.
+Organize discovery around three composable descriptions. These are ordered lists:
-Consumers select one or more groups in the desired order. Conceptually, concatenate
+their order determines precedence.
-the groups into one ordered candidate sequence, preserving order within each
+
-group. This is ordered concatenation followed by first-match selection, not a
+**Filename groups** provide reusable, ordered filename templates. The built-in
-set union or a precedence merge.
+groups are:
+
+| Group | Filenames, in order |
+| --- | --- |
+| Named | `<appname>.yml`, `<appname>.yaml` |
+| Generic | `config.yml`, `config.yaml`, `main.yml`, `main.yaml` |
+
+`<appname>` is the application slug supplied by the consumer.
+
+**Directory searches** pair a directory with an ordered list of filename groups.
+The default selection is `[Named, Generic]`, producing the six filenames above
+in that order. A directory search may select different groups or reverse their
+precedence, such as `[Generic, Named]`, while retaining each group's internal
+order. Consumers can also supply custom filenames.
+
+**Search presets** bundle ordered directory searches. Consumers can select a
+complete preset for the simple case or compose presets, custom directory searches,
+and explicit file candidates in their desired order. Explicit file candidates
+are literal paths and receive no filename expansion.
+
+### Ordered Expansion And Concatenation
+
+The selection model is **ordered expansion and concatenation**, followed by
+first-match selection:
+
+1. Expand the selected presets into directory searches in their supplied order.
+2. For each directory search, concatenate its selected filename groups in order,
+   substitute the application slug where needed, and join each filename to that
+   directory.
+3. Concatenate the resulting candidate lists, preserving the positions of custom
+   directory searches and explicit file candidates.
+4. Try candidates in order, stopping at the first selected file or an error as
+   described below.
+
+Directory priority comes before filename priority: exhaust one directory's
+filenames before trying the next directory. When several directories share a
+filename list, their expansion is a Cartesian product traversed directory first.
+Different directories may select different lists, so ordered expansion and
+concatenation describes the complete model. Set union does not express its
+precedence semantics.
+
+The eventual documentation should explain this model using the applicable
+mathematical nomenclature: ordered lists, ordered concatenation, and Cartesian
+product where a shared filename list applies. Retain these names in relevant
+implementation comments where they help readers understand the operations. The
+model describes ordering without requiring eager construction of every candidate.
+
+### Unix System Preset
+
+The agreed Unix system preset is:
+
+| Directory, in search order | Filename groups, in order |
+| --- | --- |
+| `/usr/local/etc/<appname>` | `[Named, Generic]` |
+| `/etc/<appname>` | `[Named, Generic]` |
+| `/etc` | `[Named]` |
+
+This produces 14 candidates: six in each application-specific directory, then
+`/etc/<appname>.yml` and `/etc/<appname>.yaml`. For example,
+`/usr/local/etc/<appname>/main.yaml` precedes
+`/etc/<appname>/<appname>.yml`. Selecting only `Named` for `/etc` is an ordinary
+directory-search description; the selection algorithm needs no special exception.
+
+### Illustrative Consumer API
 
 Names below are illustrative and are not implemented Go API:
 
@@ -78,8 +142,9 @@ source := configkit.SearchConfig(
 ```
 
 Use the resulting source with ordinary loading or as a layered main input. Custom
-lists participate at their specified position; they do not silently append
+searches and file lists participate at their specified position; they do not
-unselected built-in defaults. Reordering groups changes search priority.
+silently append unselected presets. Reordering presets or their directory
+searches changes search priority.
 
 ### Selection And Failure Behavior
 
@@ -93,9 +158,9 @@ unselected built-in defaults. Reordering groups changes search priority.
 | Entire search has no match and is required | Fail with a diagnostic explaining the unsuccessful search. |
 | Caller explicitly uses `File(path)` | Retain existing exclusive-file behavior, including errors for a missing file. |
 
-Requiredness applies to the complete search, not to each group. Permitting an
+Requiredness applies to the complete search, not to each preset or directory.
-absent file does not waive application version, schema, or domain requirements;
+Permitting an absent file does not waive application version, schema, or domain
-the remaining lifecycle may still reject an incomplete configuration.
+requirements; the remaining lifecycle may still reject an incomplete configuration.
 
 Perform discovery before parsing the chosen main file. Do not implement fallback
 by repeatedly running complete loads. Defaults, application resolution, secret
@@ -142,25 +207,31 @@ An explicitly selected overlay is not another main-file fallback candidate.
 
 The subsequent implementation plan should cover:
 
-- Define candidate-group and search-source representations, slug validation, and
+- Define filename-group, directory-search, preset, and search-source representations,
-  a search-wide requiredness option. Keep construction separate from acquisition.
+  slug validation, and a search-wide requiredness option. Keep construction
-- Implement the selected preset expansions, ordered custom paths, and first-match
+  separate from acquisition and the simple preset selection concise.
-  selection through existing file access. Resolve each source once per consumption
+- Implement ordered expansion and concatenation for presets, filename groups,
-  before entering its parse and composition lifecycle.
+  custom searches, and explicit file candidates, followed by first-match selection
+  through existing file access. Resolve each source once per consumption before
+  entering its parse and composition lifecycle.
 - Integrate selected and absent outcomes with ordinary and layered main inputs,
   placement, imports, application version checking, and overlay contexts according
   to OD-3. Preserve ownership and close acquired handles on success and failure.
 - Preserve real selected-source metadata and safe diagnostics without treating
   absent candidates as configuration contributions.
-- Exercise candidate and group ordering, custom-list composition, both filename
+- Exercise preset, directory, filename-group, and filename ordering, reversed
-  extensions, no-match behavior, required searches, unavailable preset bases,
+  group precedence, the Unix system preset's 14 candidates and named-only `/etc`
-  errors that must stop fallback, repeated loads, and untouched explicit `File`
+  search, custom-list composition, both filename extensions, no-match behavior,
+  required searches, unavailable preset bases, errors that must stop fallback,
+  repeated loads, and untouched explicit `File`
   behavior. Cover imports, overlays, underlays, defaults, CLI, source versions,
   source contexts, and source-list/inspection evidence at their boundaries.
 - Clarify the architecture policy's consumer-selected discovery boundary. Record
   any warranted durable ownership decision in an ADR; keep preset filenames and
   API mechanics in their canonical contract when implemented. Update consumer
-  examples and the configuration contract when the feature lands.
+  examples and the configuration contract when the feature lands. Explain the
+  three-part discovery model and its mathematical terminology in the eventual
+  documentation, with simple preset selection and custom composition examples.
 
 This roadmap does not change implementation, policy, or ADRs by itself. Coordinate
 implementation with the ongoing sequence integration audit rather than assuming
@@ -168,54 +239,48 @@ its in-progress changes are part of a fixed implementation baseline.
 
 ## Open Decisions
 
-### OD-1 — Preset Contents And Filename Order
+### OD-1 — Additional Preset Locations
 
-**Recommendation:** begin with three explicitly selected presets. Try `.yml`
+The filename groups, default `[Named, Generic]` order, and Unix system preset
-before `.yaml` at each location, completing both alternatives there before moving
+are settled above. The remaining choice is which additional directory conventions
-to the next location:
+to provide as presets.
 
-| Preset | Candidate locations, in order |
+**Recommendation:** also provide user-configuration and working-directory presets:
-| --- | --- |
+
-| Unix system | `/usr/local/etc/<slug>/config.yml`, `/usr/local/etc/<slug>/config.yaml`, `/etc/<slug>/config.yml`, `/etc/<slug>/config.yaml`, `/etc/<slug>.yml`, `/etc/<slug>.yaml` |
+| Preset | Directory | Filename groups, in order |
-| User configuration | `<user-config-dir>/<slug>/config.yml`, then `config.yaml` in the same directory |
+| --- | --- | --- |
-| Working directory | `./<slug>.yml`, then `./<slug>.yaml` |
+| User configuration | `<user-config-dir>/<appname>` | `[Named, Generic]` |
+| Working directory | Current working directory | `[Named, Generic]` |
 
 For the user preset, use Go's platform-specific
 [`os.UserConfigDir`](https://pkg.go.dev/os#UserConfigDir) as the base. This supplies
 a conventional user location without inventing separate home-directory rules.
-The system list retains the originally proposed location order. Application-named
+The working-directory preset searches that directory without walking ancestors.
-working-directory files make the selected application's convention explicit.
+Both reuse the agreed default filename groups.
-No preset walks ancestor directories.
 
-**Alternative:** use `./config.yml` and `./config.yaml` for the working-directory
+**Alternative:** provide a home-directory convention, such as
-preset, or add them after the application-named files. Generic names are concise
+`~/.<appname>/` with `[Named, Generic]`, instead of or alongside the
-for project-local applications, while application-named files distinguish
+user-config-directory preset. It is familiar for some tools, but providing both
-applications sharing a working directory. Adding both is convenient but expands
+adds another convention for consumers to select and order. Prefer the
-the candidate list and makes a generic file a fallback even when the consumer
+user-config-directory preset initially; custom searches can express other layouts.
-intended an application-specific convention.
-
-An additional viable user convention is a home-directory dotfile or directory,
-such as `~/.<slug>.yml` or `~/.<slug>/config.yml`. It is familiar for some tools,
-but would add a second user convention and ordering to explain. Prefer the
-user-config-directory preset initially; custom paths can express other layouts.
 
 ### OD-2 — Unavailable Bases And Platform Applicability
 
-**Recommendation:** resolve group bases as the ordered search reaches them.
+**Recommendation:** resolve directory bases as the ordered search reaches them.
-If a selected group's base cannot be resolved, report an error. Treat using the
+If a selected directory's base cannot be resolved, report an error. Treat using the
 Unix system preset on a platform where it is unsupported as an explicit
-configuration error when that group is reached. Specify the supported platforms
+configuration error when that preset is reached. Specify the supported platforms
 in the implementation plan. Do not silently translate the Unix preset into a
 different platform's convention. User and working-directory presets use their
 platform-native bases.
 
 An unavailable base is different from a candidate file being absent. Failure to
 determine the user's configuration directory should not silently redirect loading
-to a different group. Once an earlier candidate is selected, later groups need
+to a different directory. Once an earlier candidate is selected, later searches need
 not resolve their bases. Static invalid search arguments still fail ordinary
 option validation before source acquisition.
 
-**Alternative:** treat unsupported presets or unavailable bases as empty groups
+**Alternative:** treat unsupported presets or unavailable bases as empty searches
 and continue. This makes one preset composition usable across more environments,
 but can unexpectedly select a lower-priority configuration. A selectable
 skip-unavailable behavior could serve that need later; it adds a distinct policy
@@ -248,14 +313,15 @@ case could avoid changing `NoSource()`, at the cost of two different absence rul
 
 ### OD-4 — Empty Composition And Implicit Preset Selection
 
-**Recommendation:** require explicit group selection to activate a convention.
+**Recommendation:** require explicit preset or search selection to activate a
-An empty group composition has no candidates: optional search contributes no
+convention. An empty search composition has no candidates: optional search contributes no
-source, and required search fails as exhausted. Empty custom lists follow the
+source, and required search fails as exhausted. Empty custom lists, including
-same rule. This supports programmatic list construction and introduces no hidden
+empty filename-group selections, follow the same rule. This supports programmatic
+list construction and introduces no hidden
 system, user, or working-directory fallback. One explicitly selected preset
 still gives the simple application a short call.
 
-**Alternative:** let omission of groups select a documented default preset.
+**Alternative:** let omission of searches select a documented default preset.
 This shortens the simplest call, but omission and an intentionally empty computed
 list would need distinguishable representations. It also requires choosing a
 default convention across platforms and application styles. Another alternative