Files
promptkit/docs/roadmap/profile-policy.md

360 lines
20 KiB
Markdown

# Built-In Profile Compatibility Policy
## Status
Provisional and under active discussion. Nothing in this roadmap is an
accepted policy, an implementation commitment, a release decision, or a
description of current Promptkit behavior. Current code, GoDoc, durable
documentation, published module contracts, and release tags remain
authoritative until this proposal is finalized and implemented.
## Prerequisite And Sequencing
Implementation of this roadmap must begin only after the
[public facade refactor](public-facade.md) is complete and its compatibility
and validation criteria have passed. The refactor preserves existing behavior;
this subsequent work introduces the deliberate profile-policy changes.
Profile-source composition, maintained-profile construction, and catalog
dependency assembly must use the internal owners established by that refactor.
Both scopes may ship in `v0.10.0`, but they remain separately reviewable changes.
This prerequisite does not prevent continued discussion or planning here.
## Purpose
Define a truthful compatibility and release boundary for Promptkit's
maintained built-in profiles. The intended direction is to permit exceptionally
rapid catalog updates without weakening Promptkit's stability commitments for
its engine, public Go API, configuration semantics, backend registry, or other
non-profile behavior.
Model catalogs change faster than the Promptkit runtime. Providers control
their own model catalogs and service lifecycles. Promptkit cannot
reasonably guarantee that a maintained profile's upstream model or endpoint
will remain available, and retaining obsolete profiles can be actively
misleading when their targets no longer provide the expected service.
The release model should treat external catalog profiles as volatile data,
while providing a small set of Promptkit-owned aliases with stable IDs for
applications that prefer a maintained general-purpose selection.
## Proposed Compatibility Boundary
The provisional policy would exclude external catalog profile data from
Promptkit's cross-version compatibility surface. A catalog or Promptkit
release could, without deprecation or advance notice:
- add or remove an external catalog profile;
- rename a profile, represented as removing one ID and adding another;
- change the upstream model selected by a profile;
- change profile defaults such as reasoning effort, timeout, service tier,
sampling controls, token limits, or extra parameters; and
- correct or otherwise revise profile metadata in response to upstream
behavior.
Promptkit would make no cross-version promise about the continued presence,
identity, settings, suitability, or upstream availability of an external
catalog profile. The separately defined Promptkit-owned alias IDs would have
the continuity guarantee below. Promptkit would make no claim that an upstream
provider will continue to serve a cataloged model or endpoint for the lifetime
of a Promptkit or catalog release.
This volatility exception should apply only to external catalog profile
data. It should not silently weaken the ordinary compatibility rules for:
- Promptkit's exported Go API and GoDoc contracts;
- prompt, profile, and schema format semantics;
- request preparation, execution, validation, repair, cancellation, error,
credential, ownership, and concurrency behavior;
- public built-in backend IDs and registry behavior;
- the external catalog packages' exported Go API;
- the catalog manifest and asset schema understood by Promptkit; or
- consumer-defined profiles and backends.
Backend definitions, including endpoint, credential-environment, and capacity
metadata, remain outside the profile-volatility exception and subject to the
ordinary compatibility policy.
## Proposed Stable Promptkit Profiles
Promptkit would maintain a small, explicitly documented set of built-in
profiles using existing profile definitions and inheritance:
| Stable profile ID | Initial definition | Initial purpose |
| --- | --- | --- |
| `promptkit-openrouter-standard` | Inherits from external catalog profile `gemini-flash-latest`. | A maintained selection for general-purpose text generation. |
| `promptkit-openrouter-auto` | Selects backend `openrouter` and model `openrouter/auto`. | Delegates model selection to OpenRouter's automatic routing. |
These definitions and purposes describe the initial selections only and are
not continuing model-selection promises.
The auto profile can initially be a direct profile definition; it does not
require a second catalog alias or a new public API. It inherits connection and credential
metadata from the OpenRouter backend and need not add generation overrides.
Promptkit forwards `openrouter/auto` as the model identifier through its ordinary
client. OpenRouter owns the routing decision, eligible models, and resulting
provider behavior. Promptkit adds no model-selection algorithm or fallback
loop. Its ordinary local validation does not check the upstream availability
of `openrouter/auto`.
Both profiles receive the stable-ID guarantee below. References to aliases
describe inherited definitions where applicable; the same namespace,
precedence, and structural-validation requirements cover direct definitions.
Only these two profiles are in scope. This roadmap imposes no additional
approval process for future profiles.
The only profile-specific continuity promise is that the literal names
`promptkit-openrouter-standard` and `promptkit-openrouter-auto` remain
available, subject to the supported dependency and consumer-override scope
below. Their actual models may change at any time, for any reason, without
notice to anyone. No deprecation, approval, or change-announcement gate applies
to model selection. The names do not promise a permanent model, provider,
routing strategy, or set of defaults. Both initially use OpenRouter, but the
profile names themselves are the only continuity guarantee.
In particular, `promptkit-openrouter-auto` is not required to retain
`openrouter/auto` or automatic routing indefinitely. Neither profile guarantees
unchanged output, cost, latency, capabilities, or upstream availability. While
using upstream automatic routing, the selected model can vary between requests
even with unchanged Promptkit and catalog versions.
Catalog dependency updates must preserve local resolution of the two maintained
profile names. If a referenced catalog profile disappears or is renamed,
adjust the affected alias together with the dependency update and ship both in
the same Promptkit release. If the catalog retains the referenced ID but
changes its model, inheritance picks up the change automatically; no separate
model-selection approval is required. Ordinary code review and structural
validation apply. Upstream deprecation, retirement, and mapping accuracy remain
the catalog library's responsibility and do not create a Promptkit monitoring
or upstream-verification obligation.
Release validation must prove that every maintained alias resolves against
the selected catalog versions. This continuity guarantee applies to the
versions selected and tested by Promptkit and assumes consumers have not
overridden the alias or its base with their own definitions. Independently
upgrading a transitive catalog can remove an alias's base without updating
Promptkit; that untested combination is outside the guarantee.
Aliases would retain the ordinary profile precedence and inheritance
semantics. They would introduce no new public API or automatic model failover.
An already deployed application would retain its compiled alias and catalog
data until upgraded; an upstream withdrawal would not change its selection
automatically.
Adding these profiles does not change existing default-profile selection.
Consumers opt into them through the existing profile-selection mechanisms.
### Ownership, Precedence, And Composition
The maintained alias definitions belong to a small Promptkit-owned profile
source. Profile precedence is consumer sources in their existing order, then
Promptkit-owned aliases, then external catalog profiles. Consumers may override
an alias or its base using the ordinary profile mechanisms.
Reserve the `promptkit-` profile-ID prefix against definitions in external
catalogs. Reject such catalog definitions even if they do not yet collide with
an existing alias. This namespace rule does not prohibit consumer overrides.
Retain existing catalog-local inheritance and backend-ownership validation,
and verify that the combined maintained profiles resolve without relying on
consumer overrides. Do not assert a permanent model or routing selection based
on a profile name. Runtime inheritance must continue to observe the complete
source hierarchy, including consumer overrides of aliases and their bases.
Reuse the existing resolver; exact private assembly and validation sequencing
belongs in the implementation plan. Do not introduce a second inheritance
implementation or freeze alias values in a way that bypasses consumer overrides.
## Local Validation And Existing Safeguards
Profile volatility does not remove existing structural validation or library
safeguards. Preserve the following implemented responsibilities without making
them additional profile-availability or permanent implementation promises:
- catalog assets are immutable build inputs rather than runtime downloads;
- manifests and profiles satisfy the supported schemas;
- IDs are unique and profiles select their owning backend;
- external catalogs do not define IDs in the reserved `promptkit-` namespace;
- secret-bearing fields and prohibited connection metadata are rejected;
- profile values are normalized, copied, resolved, and overlaid through the
existing Promptkit architecture;
- the selected catalogs pass Promptkit's integration suite before a Promptkit
release adopts them; and
- the module graph and checksums provide a reproducible identity for the data
that was tested.
These checks establish local integrity for selected data, together with the
maintained-profile resolution requirement above. They do not establish external
catalog profile continuity across versions or live upstream availability.
Promptkit's checks cover local syntax and structural semantics: supported
fields, duplicate IDs, namespace conflicts, inheritance resolution, backend
references, endpoint URL format, and existing security and value constraints.
They expressly exclude attempts to verify that an endpoint exists, a model ID
is recognized upstream, or a provider serves that model. Catalog libraries own
the accuracy of upstream model and endpoint mappings. Promptkit must not add
network probes, model-list lookups, or paid inference checks to catalog adoption
or release validation. Ordinary execution continues to report upstream errors
through the existing generation-error contract.
## Provisional Release Model
The first release adopting this policy should be a new Promptkit minor release
because it changes a previously documented consumer expectation and may remove
profile IDs on which deployed configuration depends. `v0.10.0` is the current
candidate.
The OpenRouter catalog's published `v1.0.0` contract promised stable profile
IDs and advance handling before removal. A later README cannot retroactively
weaken that promise. The current recommendation is therefore to establish the
new boundary in `promptkit-backend-openrouter` major version 2, using the dated
release convention below, even if its exported Go declarations and manifest
schema do not change.
After that boundary is established, maintained catalog releases could publish
profile additions, deletions, and modifications rapidly under their documented
volatile-data policy. Promptkit could select a reviewed catalog update without
treating profile churn as a compatibility break in its otherwise stable core.
Later catalog updates and corresponding alias adjustments may ship in a
Promptkit patch release, such as `v0.10.1` after `v0.10.0`, provided unrelated
compatibility contracts remain intact. Profile churn alone does not require
`v0.11.0` or another minor release.
Promptkit's integration validation covers the catalog versions it selected
and tested. It does not guarantee upstream correctness for those versions.
Go module version selection may cause an advanced consumer's build
to select a newer compatible catalog module than Promptkit's own requirement;
the resolved module graph, rather than Promptkit's `go.mod` alone, determines
the actual embedded data.
Release notes may identify selected catalog updates and summarize changes for
convenience, but no notice of model-selection changes is required, in advance
or afterward. Ordinary release records and validation still apply; they must
not introduce a profile-change announcement gate.
### Proposed Date-Based Catalog Versions
The proposed catalog version convention is `vMAJOR.YYYYMMDD.N`. This is a
calendar-based convention encoded in Go-compatible semantic-version syntax:
- `MAJOR` identifies the compatibility generation of the exported Go package
and catalog asset format. Profile additions, deletions, renames, and setting
changes do not require a major-version increase under the proposed policy.
- `YYYYMMDD` is the release publication date in UTC. It occupies the numeric
minor-version component and communicates snapshot freshness, not the scale
or compatibility of a profile change.
- `N` starts at zero for each UTC date and increments for every additional
release that day, regardless of the kind of profile change. It occupies the
numeric patch-version component and carries no profile-continuity promise.
For example, `v2.20260922.0` and `v2.20260922.1` would be successive releases
on September 22, 2026, followed by `v2.20260923.0` on the next day. These are
illustrative dates, not scheduled releases. Use the actual publication date;
published versions remain immutable, and corrections receive new versions.
Within a major generation, these versions sort chronologically under Go's
ordinary module-version ordering.
Go requires a matching major-version suffix in module paths for version 2 and
later. The OpenRouter transition would therefore use
`gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2` in the catalog's
module declaration and Promptkit's imports and dependency requirement. A first
release on the example date would be `v2.20260922.0`; a preliminary `v2.0.0`
release is not required. Promptkit consumers would continue importing
Promptkit through its existing module path. See the
[Go module version rules](https://go.dev/ref/mod#versions) and
[major-version suffix rules](https://go.dev/ref/mod#major-version-suffixes).
Both maintained catalog libraries, OpenRouter and Rakestrawhome, would
transition to this policy and dated major-version-2 convention as part of this
work, with matching `/v2` module paths. Thereafter each would release
independently; releases need not share dates or revision numbers.
Promptkit itself would retain its existing version-number
convention and explicitly select and test individual catalog versions.
This proposal replaces the earlier idea of classifying catalog refreshes as
ordinary minor releases and metadata corrections as patches. The date records
when a snapshot was published; release notes may summarize what changed.
Neither establishes live upstream availability or cross-version profile
continuity. The convention remains provisional along with this roadmap.
## Consumer Guidance
The eventual durable documentation should make the tradeoff explicit:
- external catalog profiles optimize for convenient access to a maintained
current model catalog;
- selecting or inheriting directly from an external catalog profile accepts
its volatility;
- selecting either `promptkit-openrouter-standard` or
`promptkit-openrouter-auto` guarantees only continued availability of that
literal profile name within the supported dependency scope; models and
settings may change for any reason without notice;
- `promptkit-openrouter-auto` initially delegates model selection to OpenRouter;
its initial routing behavior is not a permanent contract, and upstream
routing can change independently of a library upgrade;
- pinning the complete resolved Go module graph preserves the exact catalog
assets used by that build, but cannot preserve live upstream availability;
- applications that require their own fixed model selection or settings should
define a complete application-owned profile with an explicit backend, model,
and defaults; and
- an application-owned alias that inherits from an external catalog profile
isolates application configuration from the catalog ID only while that base
exists; it deliberately continues to inherit changes to the base profile.
Promptkit should not promise automatic migration or replacement aliases for
every removed external catalog ID, deprecation windows for catalog profiles,
or runtime fallback from a removed or unavailable model. Its small maintained
alias set provides continuity through coordinated library releases.
## Expected Documentation And Repository Changes
If accepted, implementation would likely require coordinated changes in
Promptkit and each maintained catalog repository:
- revise catalog compatibility and release documentation to distinguish the
stable Go/schema boundary from volatile profile data and document the
`vMAJOR.YYYYMMDD.N` convention;
- update Promptkit's release policy with a narrowly scoped built-in-profile
volatility exception for external catalogs and an explicit continuity
guarantee for Promptkit-owned alias IDs, while retaining its other pre-`v1`
compatibility rules;
- add `promptkit-openrouter-standard` through existing profile inheritance and
`promptkit-openrouter-auto` as a direct `openrouter/auto` profile; document
their distinct purposes and opt-in selection;
- require coordinated catalog and alias updates, with release checks proving
every maintained alias resolves against the selected catalog versions;
- update the architecture, format, source, development, and consumer
documentation at their existing ownership boundaries;
- keep exact external profile inventories in the catalog repositories, linking
to the selected versions from Promptkit rather than duplicating their tables;
keep Promptkit's small alias inventory and runtime semantics in its own
canonical documentation;
- revise tests and compatibility-fixture terminology so they protect local
structural validity and maintained-profile resolution, without asserting
cross-version stability of external profiles; retain an exact selected-version
snapshot only where its review value justifies its maintenance cost;
- publish both revised catalog contracts as dated major-version-2 releases
with matching `/v2` module paths;
- update Promptkit's catalog imports and requirement, current selected-profile
documentation, and release guidance; and
- publish the policy transition and its consumer impact in the eventual
Promptkit minor release note and annotated tag.
No implementation should begin from this document until its provisional
status has been updated through the repository's normal planning process and
the public facade refactor prerequisite has been completed.
## Non-Goals
This proposal does not introduce:
- runtime catalog downloads, discovery, or mutable global registration;
- any upstream endpoint or model-existence verification by Promptkit, or
guarantees based on current provider health;
- automatic replacement aliases or migration tables for every removed catalog
profile, or runtime fallback models;
- weakened validation of catalog structure or content;
- weaker compatibility for Promptkit's public engine or consumer-defined
configuration; or
- a general exception allowing arbitrary dependency updates to break stable
Promptkit behavior.