148 lines
7.9 KiB
Markdown
148 lines
7.9 KiB
Markdown
# PromptKit Feature Request: Filesystem Sources And Layered Loading
|
|
|
|
Status: downstream request for consideration, not an approved ConfigKit design
|
|
or implementation plan.
|
|
|
|
## Consumer Context
|
|
|
|
PromptKit is a Go library for defining, rendering, and executing prompts against
|
|
LLM backends. Its consumers supply prompt and profile definitions through local
|
|
files, embedded application defaults, maintained backend catalogs, and
|
|
programmatic values.
|
|
|
|
PromptKit intends to adopt ConfigKit to standardize configuration semantics and
|
|
remove custom loading machinery. Applications using both libraries should not
|
|
need to learn two different sets of generic YAML, presence, and diagnostic rules.
|
|
PromptKit is pre-1.0 and can change its definition formats and construction API
|
|
to accommodate a coherent shared configuration model, provided its substantive
|
|
prompting and execution capabilities remain available.
|
|
|
|
PromptKit has selected the following two capabilities as prerequisites for that
|
|
refactor. They are requested together because embedded defaults and explicit
|
|
layering form one common consumer workflow. This document describes downstream
|
|
needs only; ConfigKit's public API design and internal implementation remain for
|
|
its maintainers to determine.
|
|
|
|
## 1. Native `fs.FS` Configuration Sources
|
|
|
|
### Need
|
|
|
|
Allow a library or application to supply configuration from an `fs.FS`, including
|
|
`embed.FS`, without extracting files to disk or treating each file as an isolated
|
|
reader disconnected from its filesystem context.
|
|
|
|
PromptKit packages maintained profile catalogs and supports application-bundled
|
|
fallback profiles. Consumers also embed prompt definitions and associated text
|
|
assets so deployed programs do not require a separate configuration installation.
|
|
These sources should participate in ConfigKit loading as naturally as local files.
|
|
|
|
### Desired Consumer Outcomes
|
|
|
|
- Load configuration from an explicitly supplied filesystem and location within
|
|
it, with no dependency on a particular host directory layout.
|
|
- Retain meaningful source identity and locations in diagnostics and origins,
|
|
even when the source has no operating-system filename.
|
|
- When configuration uses supported imports or overlays, resolve relative
|
|
configuration references in the supplied filesystem context rather than
|
|
unexpectedly looking on the host filesystem.
|
|
- Make enough source context available for a consuming library to resolve its
|
|
own relative resources. For PromptKit, those include referenced message text;
|
|
understanding or rendering that text remains PromptKit's responsibility.
|
|
- Use the same configuration semantics for embedded and ordinary file-backed
|
|
definitions, rather than maintaining a separate embedded-loading path.
|
|
|
|
### Rationale
|
|
|
|
Bundled defaults are useful to many Go libraries, not just PromptKit. Native
|
|
filesystem support lets the owner package those defaults while callers choose
|
|
where their configuration comes from. ConfigKit's existing reader source can
|
|
accept file bytes, but supplying bytes alone does not satisfy the filesystem-
|
|
relative workflow described here.
|
|
|
|
## 2. Public In-Memory Layered Loading With Origins
|
|
|
|
### Need
|
|
|
|
Allow callers to explicitly supply configuration layers and obtain their effective
|
|
configuration with meaningful origins, without temporary files or conversion of
|
|
programmatic values to YAML and back.
|
|
|
|
PromptKit needs to combine library-maintained defaults, application-bundled
|
|
defaults, operator-selected configuration, and programmatic definitions. These
|
|
inputs do not necessarily live in one filesystem or follow an application-owned
|
|
overlay directory convention.
|
|
|
|
### Desired Consumer Outcomes
|
|
|
|
- Combine explicitly supplied embedded/file/byte-backed configuration and
|
|
programmatic values, with caller-selected precedence where overriding is intended.
|
|
- Distinguish combining independent contributions from intentionally overriding
|
|
earlier values. Accidental conflicts should not become implicit precedence
|
|
merely because sources happened to be discovered in a particular order.
|
|
- Preserve the distinction between an omitted field and an explicit zero, false,
|
|
empty value, or null so downstream domain rules can make deliberate choices.
|
|
Programmatic input needs a usable way to convey this intent as well as file input.
|
|
- Explain which source supplied an effective setting and identify relevant
|
|
sources when reporting a conflict or invalid setting. Origins must remain useful
|
|
when embedded and programmatic inputs have labels rather than host file paths.
|
|
- Allow loading through the public library interface without requiring a generated
|
|
root configuration file, temporary overlay directory, or serialization round trip.
|
|
|
|
### PromptKit-Specific Context
|
|
|
|
PromptKit's intended source priority, highest first, is programmatic profiles,
|
|
operator profiles, application fallback profiles, PromptKit-owned profiles, and
|
|
external catalog profiles. A higher-priority definition replaces the lower
|
|
definition with the same identity. This is distinct from profile inheritance:
|
|
a selected profile may name a base profile and override selected settings.
|
|
|
|
For inherited settings, PromptKit intends omission-based inheritance and explicit
|
|
value replacement, with a supplied provider-parameter map replacing the whole
|
|
inherited map. These examples explain why generic recursive mapping merge alone
|
|
does not cover every downstream operation. They do not request configurable
|
|
per-field merge strategies, a PromptKit-specific profile registry, or LLM rules
|
|
inside ConfigKit. The eventual public contract should make clear which generic
|
|
layering needs it serves and what domain interpretation remains with the consumer.
|
|
|
|
### Rationale
|
|
|
|
Many libraries provide defaults that applications then customize. A public
|
|
layered-loading capability would let those libraries share ConfigKit's presence,
|
|
validation, and origin semantics rather than each maintaining its own generic
|
|
composition and diagnostic machinery. Applications using ConfigKit directly and
|
|
indirectly through PromptKit would benefit from a consistent explanation of how
|
|
their effective configuration was obtained.
|
|
|
|
## Combined Workflow And Efficiency Needs
|
|
|
|
A representative consumer bundles default profiles, reads operator configuration,
|
|
adds a few programmatic settings, and constructs PromptKit once. Loading produces
|
|
an immutable set of definitions; subsequent prompt executions use that set without
|
|
reloading configuration. Errors need to be actionable at construction, before any
|
|
provider request. A new construction can capture later configuration changes.
|
|
|
|
Because PromptKit is a shared library, loading overhead and retained memory matter
|
|
across applications and catalog sizes. The requested capabilities should support
|
|
efficient processing of many definitions and layers without requiring repeated
|
|
reads, decoding, serialization, or copying of entire accumulated collections for
|
|
each added layer. Efficiency should be assessed against total input and produced
|
|
data, including retained provenance; this is not a request for an unconditional
|
|
O(n) promise or a fixed timing threshold.
|
|
|
|
## Scope And Follow-Up
|
|
|
|
This request does not seek automatic discovery, hot reload, live provider checks,
|
|
prompt rendering, model selection, or credential resolution specific to LLM calls.
|
|
PromptKit retains its domain relationships and execution responsibilities.
|
|
|
|
The follow-up design discussion should clarify supported layer inputs, how callers
|
|
express programmatic presence and precedence, filesystem-relative behavior, and
|
|
the source information consumers can inspect. PromptKit can adapt its APIs and
|
|
formats to those decisions; historical loading order is not a compatibility demand.
|
|
|
|
The current [configuration contract](../config.md) and
|
|
[deferred scope](deferred.md) describe ConfigKit's existing behavior and exclusions.
|
|
This request proposes expanding the relevant scope, not treating the desired
|
|
features as already available. PromptKit plans to consume released ConfigKit
|
|
support before implementing the dependent refactor.
|