From af84107905fe485ca8f51441fbc60d237ed2f3b3 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 24 Oct 2026 01:00:08 +0000 Subject: [PATCH] Document composable configuration discovery --- docs/roadmap/discovery.md | 284 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 284 insertions(+) create mode 100644 docs/roadmap/discovery.md diff --git a/docs/roadmap/discovery.md b/docs/roadmap/discovery.md new file mode 100644 index 0000000..0853def --- /dev/null +++ b/docs/roadmap/discovery.md @@ -0,0 +1,284 @@ +# Composable Configuration Discovery Roadmap + +Status: proposal with the agreed direction captured below. Preset contents and +the remaining behavioral choices are under [Open Decisions](#open-decisions). +This document describes future work, not an implemented contract. A staged +implementation plan will follow once the decisions are settled. + +## Purpose And Ownership + +Make ordinary file-based loading concise for applications that want conventional +configuration locations. ConfigKit supplies useful built-in candidate lists and +an ordered selection operation. Consumers continue to own main-file search +conventions: they choose which lists apply, their order, custom candidates, and +whether a configuration file is required. + +A consumer can select one preset for a simple application or compose several +presets and custom path lists. The application supplies an explicit slug where +needed to construct conventional paths. ConfigKit does not infer application +identity from the executable name or enable discovery for existing explicit +sources. + +Discovery selects one source for the existing loading lifecycle. It does not +create additional configuration tiers or merge every matching file. Existing +layered loading remains available when consumers intentionally combine sources. + +## Current Baseline + +The [source contract](../config.md#sources) provides explicit host files, native +filesystem files, bytes, readers, mappings, and `NoSource()`. A missing explicit +file fails. The zero source is invalid. There is no conventional main-file search. + +[Ordinary loading](../config.md#single-source-loading) infers a selected overlay's +filesystem context from the main source. `NoSource()` currently rejects enabled +imports, active overlays, and filesystem context. In +[layered loading](../config.md#layered-loading), an absent main input can coexist +with underlays; a selected overlay requires an explicit `OverlayFilesystem`. +These distinctions matter when an optional search finds nothing. + +The [architecture policy](../policy/architecture.md#responsibility-boundaries) +assigns search conventions to consumers and currently excludes implicit main-file +discovery. The addition should clarify that consumers may explicitly select +library-provided search conventions. Consumer ownership remains unchanged. + +## Agreed Direction + +### Ordered Candidate Groups + +Provide reusable descriptions of built-in search lists and custom ordered paths. +Consumers select one or more groups in the desired order. Conceptually, concatenate +the groups into one ordered candidate sequence, preserving order within each +group. This is ordered concatenation followed by first-match selection, not a +set union or a precedence merge. + +Names below are illustrative and are not implemented Go API: + +```go +// A simple system-wide application. +source := configkit.SearchConfig("myapp", configkit.UnixSystemDirectories()) +``` + +```go +// A consumer-selected working-directory, user, then system search. +source := configkit.SearchConfig( + "myapp", + configkit.WorkingDirectory(), + configkit.UserConfigDirectory(), + configkit.UnixSystemDirectories(), +) +``` + +```go +// Explicit custom candidates followed by a selected conventional fallback. +source := configkit.SearchConfig( + "myapp", + configkit.Paths("/run/myapp/config.yml", "/opt/myapp/config.yaml"), + configkit.UnixSystemDirectories(), +) +``` + +Use the resulting source with ordinary loading or as a layered main input. Custom +lists participate at their specified position; they do not silently append +unselected built-in defaults. Reordering groups changes search priority. + +### Selection And Failure Behavior + +| Situation | Selected behavior | +| --- | --- | +| Candidate is absent | Continue to the next candidate. | +| A candidate file is selected | Stop searching and load that file. | +| Candidate is inaccessible, unreadable, or not an accepted file type | Fail using normal source diagnostics. | +| Selected file fails parsing, version checks, composition, decoding, or validation | Fail without trying another candidate. | +| Entire search has no match | Absence is permitted by default; continue through applicable defaults and normal validation. The interaction with other inputs is addressed under OD-3. | +| Entire search has no match and is required | Fail with a diagnostic explaining the unsuccessful search. | +| Caller explicitly uses `File(path)` | Retain existing exclusive-file behavior, including errors for a missing file. | + +Requiredness applies to the complete search, not to each group. Permitting an +absent file does not waive application version, schema, or domain requirements; +the remaining lifecycle may still reject an incomplete configuration. + +Perform discovery before parsing the chosen main file. Do not implement fallback +by repeatedly running complete loads. Defaults, application resolution, secret +lookup, and final validation retain their existing lifecycle execution rules. +Once a file has been selected, later failure does not restart discovery. + +### Construction And Filesystem Behavior + +Constructors describe the search without filesystem access. Resolve dynamic base +directories and access candidate files during loading, so reusable descriptions +can observe changes on subsequent loads. Do not change the process working +directory or environment. + +Reuse the host-file acquisition rules for selected files, including regular-file +checks, symlink handling, resource ownership, and source limits. The feature does +not require a new filesystem traversal engine, recursive directory search, or +new file-size limits. Candidate filenames and groups are bounded by their +explicit descriptions, rather than by scanning unrelated directories. + +The exact Go names, constructors, and representation of search-wide options +belong in the implementation plan. Keep the public surface root-owned and keep +the single-preset case concise. + +### Placement, Provenance, And Existing Features + +The selected file participates as the actual file source. Preserve its physical +identity, effective placement, source origin, and passive resource context. +Relative imports and relative application resources use the selected file's +normal context. Discovery does not relocate overlay or CLI coordinates. + +Missing candidates are not acquired sources and do not create source IDs, +configuration history, or explicit input presence. An entirely absent search +contributes no synthetic empty configuration file. Diagnostics should distinguish +an exhausted required search from a failed candidate. Existing selected-source +inspection remains the foundation; a new public per-attempt trace is not needed +for this proposal. + +Preserve collection policies, inheritance, source-version rules, CLI ordering, +final validation, and frozen inspection after selection. Search-list order does +not change precedence among underlays, the main input, an overlay, and CLI values. +An explicitly selected overlay is not another main-file fallback candidate. + +## Work Needed From The Baseline + +The subsequent implementation plan should cover: + +- Define candidate-group and search-source representations, slug validation, and + a search-wide requiredness option. Keep construction separate from acquisition. +- Implement the selected preset expansions, ordered custom paths, and first-match + selection through existing file access. Resolve each source once per consumption + before entering its parse and composition lifecycle. +- Integrate selected and absent outcomes with ordinary and layered main inputs, + placement, imports, application version checking, and overlay contexts according + to OD-3. Preserve ownership and close acquired handles on success and failure. +- Preserve real selected-source metadata and safe diagnostics without treating + absent candidates as configuration contributions. +- Exercise candidate and group ordering, custom-list composition, both filename + extensions, no-match behavior, required searches, unavailable preset bases, + errors that must stop fallback, repeated loads, and untouched explicit `File` + behavior. Cover imports, overlays, underlays, defaults, CLI, source versions, + source contexts, and source-list/inspection evidence at their boundaries. +- Clarify the architecture policy's consumer-selected discovery boundary. Record + any warranted durable ownership decision in an ADR; keep preset filenames and + API mechanics in their canonical contract when implemented. Update consumer + examples and the configuration contract when the feature lands. + +This roadmap does not change implementation, policy, or ADRs by itself. Coordinate +implementation with the ongoing sequence integration audit rather than assuming +its in-progress changes are part of a fixed implementation baseline. + +## Open Decisions + +### OD-1 — Preset Contents And Filename Order + +**Recommendation:** begin with three explicitly selected presets. Try `.yml` +before `.yaml` at each location, completing both alternatives there before moving +to the next location: + +| Preset | Candidate locations, in order | +| --- | --- | +| Unix system | `/usr/local/etc//config.yml`, `/usr/local/etc//config.yaml`, `/etc//config.yml`, `/etc//config.yaml`, `/etc/.yml`, `/etc/.yaml` | +| User configuration | `//config.yml`, then `config.yaml` in the same directory | +| Working directory | `./.yml`, then `./.yaml` | + +For the user preset, use Go's platform-specific +[`os.UserConfigDir`](https://pkg.go.dev/os#UserConfigDir) as the base. This supplies +a conventional user location without inventing separate home-directory rules. +The system list retains the originally proposed location order. Application-named +working-directory files make the selected application's convention explicit. +No preset walks ancestor directories. + +**Alternative:** use `./config.yml` and `./config.yaml` for the working-directory +preset, or add them after the application-named files. Generic names are concise +for project-local applications, while application-named files distinguish +applications sharing a working directory. Adding both is convenient but expands +the candidate list and makes a generic file a fallback even when the consumer +intended an application-specific convention. + +An additional viable user convention is a home-directory dotfile or directory, +such as `~/..yml` or `~/./config.yml`. It is familiar for some tools, +but would add a second user convention and ordering to explain. Prefer the +user-config-directory preset initially; custom paths can express other layouts. + +### OD-2 — Unavailable Bases And Platform Applicability + +**Recommendation:** resolve group bases as the ordered search reaches them. +If a selected group's base cannot be resolved, report an error. Treat using the +Unix system preset on a platform where it is unsupported as an explicit +configuration error when that group is reached. Specify the supported platforms +in the implementation plan. Do not silently translate the Unix preset into a +different platform's convention. User and working-directory presets use their +platform-native bases. + +An unavailable base is different from a candidate file being absent. Failure to +determine the user's configuration directory should not silently redirect loading +to a different group. Once an earlier candidate is selected, later groups need +not resolve their bases. Static invalid search arguments still fail ordinary +option validation before source acquisition. + +**Alternative:** treat unsupported presets or unavailable bases as empty groups +and continue. This makes one preset composition usable across more environments, +but can unexpectedly select a lower-priority configuration. A selectable +skip-unavailable behavior could serve that need later; it adds a distinct policy +from the search-wide requirement that at least one file exist. + +### OD-3 — No Match, Imports, And Selected Overlays + +**Recommendation:** use a shared absent-main lifecycle for optional no-match and +explicit `NoSource()`. Allow import enablement when the main file is absent; +there are no main-file declarations to traverse. Other actual sources retain +their normal import behavior. In layered loads, retain underlays and the existing +explicit overlay-filesystem requirement. + +For ordinary loads with an active overlay and no main file, require an explicit +filesystem context. Do not infer an overlay root from the first or last searched +directory. A missing selected overlay remains an error. Defaults, CLI overrides, +resolution, and validation continue normally, and main-version checking receives +absence rather than a fabricated document. + +This would deliberately relax the current `NoSource()` restrictions on imports +and on overlays with explicit context. It keeps one meaning for absent main input +and lets optional configuration remain optional without silently ignoring a +requested overlay. It requires lifecycle work beyond candidate selection. + +**Alternative:** map no-match to the current `NoSource()` contract and retain its +restrictions. This minimizes lifecycle changes, but an otherwise valid optional +search with imports enabled can fail solely because no file was found, and +ordinary overlay-only loading would remain unavailable. A special absent-search +case could avoid changing `NoSource()`, at the cost of two different absence rules. + +### OD-4 — Empty Composition And Implicit Preset Selection + +**Recommendation:** require explicit group selection to activate a convention. +An empty group composition has no candidates: optional search contributes no +source, and required search fails as exhausted. Empty custom lists follow the +same rule. This supports programmatic list construction and introduces no hidden +system, user, or working-directory fallback. One explicitly selected preset +still gives the simple application a short call. + +**Alternative:** let omission of groups select a documented default preset. +This shortens the simplest call, but omission and an intentionally empty computed +list would need distinguishable representations. It also requires choosing a +default convention across platforms and application styles. Another alternative +is to reject an empty composition as invalid; that catches accidental empty lists +but adds a special case beyond the search-wide requiredness rule. + +### OD-5 — Slugs And Custom Path Interpretation + +**Recommendation:** require a nonempty single-component slug when a selected +preset uses it to construct filenames. Reject path separators, `.` and `..`, +and other forms that cannot be used as that component on the target platform. +Preserve spelling; do not silently lowercase, trim, or impose a kebab-case naming +scheme. A composition made only of explicit custom paths need not require an +application slug. + +Treat custom paths as literal host file paths. Resolve relative paths against +the working directory at load time; do not expand shell variables, `~`, or glob +syntax. Consumers can construct expanded paths themselves. Selected presets +provide the convenience for user and working-directory bases. + +**Alternative:** require a restricted portable slug grammar, such as lowercase +ASCII letters, digits, and hyphens. This is easy to document and portable, but +unnecessarily excludes otherwise usable application names. Automatic shell-like +custom-path expansion is also viable, but adds escaping and environment semantics +that ordinary `File(path)` does not supply. Literal paths keep both interfaces +consistent.