170 lines
7.1 KiB
Markdown
170 lines
7.1 KiB
Markdown
# Promptkit Feature Request: Application Fallback Profiles
|
|
|
|
Status: Implemented upstream in Promptkit v0.5.0.
|
|
|
|
Promptkit v0.5.0 resolved this request with the public
|
|
`WithFallbackProfileFS` engine option and the precedence and error semantics
|
|
specified below. This document is retained as the downstream rationale for
|
|
the capability.
|
|
|
|
## Purpose
|
|
|
|
Promptkit should allow a consuming application to supply an embedded fallback
|
|
profile source that sits below operator-configured profiles and above
|
|
Promptkit's own built-in profile catalog.
|
|
|
|
This capability would let an application publish stable, domain-specific
|
|
profile IDs with useful defaults while preserving Promptkit's existing
|
|
operator-override behavior. The capability must remain application-neutral;
|
|
Promptkit should provide the source layer but should not own downstream profile
|
|
names, model assignments, or configuration policy.
|
|
|
|
## Downstream Use Case
|
|
|
|
Weatherreporter wants to embed profiles such as `weather-light`,
|
|
`weather-balanced`, and `weather-deep`. Report prompts would select those
|
|
logical profiles instead of naming provider- or model-specific Promptkit
|
|
profiles directly.
|
|
|
|
An installation could then place a profile with the same ID in its configured
|
|
profile directory. For example, a local `weather-light` definition could point
|
|
to an OpenAI-compatible endpoint on the deployment network. When no operator
|
|
definition exists, Weatherreporter's embedded definition would keep the
|
|
application usable without additional profile files.
|
|
|
|
This pattern is useful beyond Weatherreporter. Any Promptkit consumer may want
|
|
application-owned execution tiers or workload-specific defaults without
|
|
adding domain-specific profiles to Promptkit's general built-in catalog.
|
|
|
|
## Current Constraint
|
|
|
|
Promptkit currently resolves matching profile IDs in this order:
|
|
|
|
1. in-memory profiles supplied through `WithProfiles`;
|
|
2. one configured profile file, `fs.FS`, or directory source; and
|
|
3. Promptkit's embedded built-in profiles.
|
|
|
|
These layers do not express the desired application-default relationship:
|
|
|
|
- `WithProfiles` has higher precedence than the configured source, so it would
|
|
prevent an operator file from overriding an application profile with the
|
|
same ID.
|
|
- `WithProfileFS` can hold embedded application assets, but it occupies the
|
|
configured-source layer and therefore replaces rather than sits beneath a
|
|
configured profile directory or file.
|
|
- adding downstream profile IDs to Promptkit's built-in catalog would make the
|
|
library own application-specific policy.
|
|
|
|
A downstream application could build its own filesystem overlay, but that
|
|
would duplicate Promptkit's profile discovery, error, and precedence behavior
|
|
at the consumer boundary.
|
|
|
|
## Requested Capability
|
|
|
|
Add one optional application fallback profile source to engine construction.
|
|
When present, matching profile IDs should resolve in this order:
|
|
|
|
1. in-memory profiles supplied through `WithProfiles`;
|
|
2. the ordinary configured profile source selected through a profile option or
|
|
`Config.ProfileDir`;
|
|
3. the application fallback profile source; and
|
|
4. Promptkit's embedded built-in profiles.
|
|
|
|
When no application fallback is configured, existing source precedence and
|
|
behavior must remain unchanged.
|
|
|
|
The minimum useful public surface is an `fs.FS`-backed option because consumers
|
|
can embed YAML profile assets. A possible API shape is:
|
|
|
|
```go
|
|
promptkit.WithFallbackProfileFS(profileFS, ".")
|
|
```
|
|
|
|
The name is illustrative rather than prescriptive. A companion option for
|
|
validated `Profile` values could be added if Promptkit maintainers find it
|
|
generally useful, but it is not required for the Weatherreporter use case.
|
|
|
|
## Required Semantics
|
|
|
|
- A higher-precedence source falls through only when the requested profile ID
|
|
is absent.
|
|
- A malformed, unreadable, duplicate, ambiguous, or otherwise invalid matching
|
|
profile is an error and must not silently fall through.
|
|
- The fallback source uses the existing strict profile YAML format and profile
|
|
validation rules.
|
|
- Profile values are selected as a whole. This feature does not merge,
|
|
inherit, or partially overlay profile definitions.
|
|
- `InspectProfile`, `Prepare`, prepared execution, and ordinary execution use
|
|
the same profile-source precedence.
|
|
- An explicit request profile continues to take precedence over a prompt's
|
|
`default_profile`; this request concerns definition lookup after the profile
|
|
ID has been selected.
|
|
- Repeated fallback-source options should follow Promptkit's documented
|
|
same-category option convention, normally with the last value replacing the
|
|
earlier value.
|
|
- A canceled lookup, invalid fallback asset, or unknown resolved backend should
|
|
continue to cross the public facade through Promptkit's existing public error
|
|
identities.
|
|
- Exact profile inspection must remain side-effect free and must not contact a
|
|
model provider.
|
|
|
|
## Application And Library Boundaries
|
|
|
|
Promptkit should own:
|
|
|
|
- the additional repository layer;
|
|
- deterministic lookup and fallthrough behavior;
|
|
- validation of the supplied source through the existing profile contract;
|
|
- consistent use of the layer across inspection and execution; and
|
|
- public documentation and tests for the added precedence rule.
|
|
|
|
The consuming application should continue to own:
|
|
|
|
- whether it supplies fallback profiles;
|
|
- the profile IDs and their domain meaning;
|
|
- embedded profile contents and model choices;
|
|
- application configuration and override policy;
|
|
- report- or workload-to-profile assignment; and
|
|
- credential checks and operator-facing errors beyond Promptkit's public
|
|
contract.
|
|
|
|
## Non-Goals
|
|
|
|
This request does not ask Promptkit to add:
|
|
|
|
- Weatherreporter-specific profile IDs to its built-in catalog;
|
|
- profile inheritance, aliases, or field-level merging;
|
|
- automatic endpoint discovery or availability probing;
|
|
- provider failover or fallback from a failed selected profile;
|
|
- per-request model benchmarking or tier selection;
|
|
- application configuration discovery; or
|
|
- eager validation of every profile in every source.
|
|
|
|
## Compatibility
|
|
|
|
The feature can be additive. Engines that do not configure an application
|
|
fallback source should retain their current public behavior and precedence.
|
|
Existing uses of `WithProfiles`, `WithProfileFile`, `WithProfileFS`, and
|
|
`Config.ProfileDir` should not change meaning.
|
|
|
|
The application fallback is deliberately lower precedence than every existing
|
|
consumer-configured source. This preserves the established expectation that a
|
|
custom profile definition can override a packaged default with the same ID.
|
|
|
|
## Acceptance Criteria
|
|
|
|
The capability is sufficient for downstream adoption when Promptkit can
|
|
demonstrate that:
|
|
|
|
- a fallback-only profile can be inspected and used for preparation and
|
|
execution;
|
|
- a configured directory, file, or `fs.FS` profile with the same ID overrides
|
|
the fallback profile;
|
|
- an absent configured profile falls through to the application fallback;
|
|
- an invalid configured match fails instead of falling through;
|
|
- an absent application fallback profile continues to resolve from Promptkit's
|
|
built-in catalog;
|
|
- `WithProfiles` retains highest precedence;
|
|
- behavior is identical across inspection, preparation, and execution; and
|
|
- omitting the new option preserves existing tests and public contracts.
|