29 Commits

Author SHA1 Message Date
4b8169da28 Prepare Promptkit v0.12.0 release 2026-10-03 01:26:43 +00:00
62a4a20529 Preserve message resource origins and profile credential lookup names 2026-10-03 01:21:48 +00:00
8c68b28421 Record ConfigKit adoption review findings 2026-10-03 01:10:29 +00:00
66c55b7397 Benchmark configuration workflows and reuse captured content origins 2026-09-30 04:25:48 +00:00
48343300e2 Remove superseded configuration loaders and consolidate runtime tests 2026-09-30 04:18:27 +00:00
f17a38f60b Document keyed configuration and migrated consumer workflows 2026-09-30 03:34:15 +00:00
aba5e960aa Expose captured configuration inspection and redacted reports 2026-09-30 03:29:28 +00:00
d818e0d162 Render captured prompt templates without reloading sources 2026-09-30 03:24:12 +00:00
ba9c972faa Cut over public engine construction to ConfigKit sources 2026-09-30 03:18:49 +00:00
17b6c7bca5 Integrate released catalog sources with effective configuration 2026-09-30 02:46:33 +00:00
2969a83a71 Record published Rakestrawhome catalog migration 2026-09-30 02:40:22 +00:00
7f58261498 Record published OpenRouter catalog migration 2026-09-30 02:37:50 +00:00
defb240b6e Build presence-aware execution targets from effective configuration 2026-09-30 02:32:54 +00:00
b158a8ec2d Capture keyed prompts and source-backed content at load time 2026-09-30 02:19:52 +00:00
2b847ac2ae Add ConfigKit typed configuration loading core 2026-09-30 02:08:20 +00:00
5f7b19403f Accept the ConfigKit configuration boundary and align the implementation plan 2026-09-30 01:59:30 +00:00
80131fa160 Plan ConfigKit adoption and configuration migration 2026-09-30 00:50:46 +00:00
521291d80f Finalize ConfigKit adoption decisions 2026-09-30 00:40:16 +00:00
0f43460f84 Plan ConfigKit adoption for PromptKit 2026-09-30 00:29:09 +00:00
08409dd472 Align the assessment with ConfigKit v0.5.0 2026-09-30 00:25:10 +00:00
34f5980973 Clarify configuration inspection and execution boundaries 2026-09-29 16:56:50 +00:00
7b012033a6 Align the adoption assessment with ConfigKit source tiers 2026-09-29 16:49:18 +00:00
37f8dba6c3 Clarify ConfigKit prerequisites and efficiency goals 2026-09-29 14:52:38 +00:00
c7bdaa076b Assess ConfigKit adoption for PromptKit 2026-09-29 14:11:30 +00:00
12d6d80024 Prepare the v0.11.2 release 2026-09-28 00:02:23 +00:00
d8147cac49 Remove the policy adoption narrative 2026-09-28 00:01:57 +00:00
a16c2a38d6 Align policies with the standard and record their provenance 2026-09-27 21:18:11 +00:00
843d07b636 Plan standard policy alignment 2026-09-27 21:10:12 +00:00
dc6fd2752d Align OSS attribution notices to the house standard format 2026-09-27 13:40:57 +00:00
144 changed files with 5475 additions and 13889 deletions

View File

@@ -1,5 +1,6 @@
Please review `docs/development.md` for initial orientation in this repository
and follow its task-specific reading guide.
Start with `docs/development.md` for orientation and use its policy reading guide
to select the policies applicable to the task. Do not read the entire suite by
default.
Review and follow `docs/policy/agents.md` before recommending, planning, or
implementing any changes to this repository.
Read and follow `docs/policy/agents.md` before recommending, planning, or
implementing repository changes.

View File

@@ -16,4 +16,8 @@ but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Lesser General Public License for more details.
Third-party code and assets are subject to their respective license terms.
Policy documents copied or adapted from Standard Repository Policies remain licensed
under CC-BY-SA-4.0. See docs/oss/attribution.md for the affected material,
upstream credit, and license links.
Third-party code and assets are subject to their respective license terms.

View File

@@ -8,7 +8,8 @@ gitea.maximumdirect.net/eric/promptkit
```
The root `promptkit` package provides the supported public engine. Consumers
can configure filesystem or in-memory prompt, profile, and schema sources,
can configure file, embedded, or programmatic backend, profile, and prompt
definitions alongside filesystem or in-memory schemas,
prepare requests without generation, run requests with the built-in
OpenAI-compatible client, or inject their own model client and artifact reader.
@@ -20,9 +21,12 @@ Run the maintained offline preparation example from the repository root:
go run ./examples/go-library/prepare
```
It loads a repository-local prompt, supplies an in-memory profile and inline
input, and prints deterministic preparation metadata without contacting a
provider or requiring credentials. Read the
It loads a repository-local prompt, supplies a programmatic profile and inline
input, inspects the captured configuration, and prints deterministic
preparation metadata without contacting a provider or requiring credentials.
The [consumer guide](docs/consumers/pkg-promptkit.md) also shows a local backend,
ordered file/embedded sources, programmatic mappings, and how to replace an
engine when source files change. Read the
[example source](examples/go-library/prepare/main.go), the
[Go package consumer guide](docs/consumers/pkg-promptkit.md), and the
[framework format reference](docs/formats.md) to build a consumer workflow.
@@ -33,6 +37,13 @@ boundary and constraints that framework work must preserve.
## Release Guidance
Consumers upgrading to `v0.12.0` should read the
[v0.12.0 release notes](docs/releases/v0.12.0.md) for the intentional ConfigKit
configuration and public construction API migration.
The [v0.11.2 release notes](docs/releases/v0.11.2.md) cover policy alignment
and copied-policy attribution updates.
The [v0.11.1 release notes](docs/releases/v0.11.1.md) cover the MIT-licensed
catalog updates and licensing documentation improvements.

View File

@@ -29,15 +29,7 @@ func TestAppendedMessagesComposeFrozenEffectivePrompt(t *testing.T) {
{Role: promptkit.RoleUser, Content: "corrective request", CacheControl: cacheControl},
}
client := &fakeLLMClient{response: &promptkit.GenerateResponse{Content: "output"}}
engine, err := promptkit.NewEngine(
promptkit.Config{},
promptkit.WithPromptFS(contractPromptFS("prompt", "profile", "configured prompt"), "."),
promptkit.WithProfiles(promptkit.Profile{ID: "profile", Endpoint: "http://example.test/v1", Model: "model"}),
promptkit.WithLLMClient(client),
)
if err != nil {
t.Fatalf("construct engine: %v", err)
}
engine := newSimpleEngine(t, "configured prompt", promptkit.WithLLMClient(client))
request := promptkit.RunRequest{PromptID: "prompt", AppendedMessages: appended}
plain, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "prompt"})
@@ -101,14 +93,9 @@ func TestAppendedMessagesComposeFrozenEffectivePrompt(t *testing.T) {
}
}
func TestInvalidAppendedMessagesFailBeforeSourceOrModelWork(t *testing.T) {
promptSource := &inspectionCountingFS{}
func TestInvalidAppendedMessagesFailBeforeModelWork(t *testing.T) {
var modelCalls atomic.Int64
engine, err := promptkit.NewEngine(
promptkit.Config{},
promptkit.WithPromptFS(promptSource, "."),
promptkit.WithLLMClient(countingLLMClient{calls: &modelCalls}),
)
engine, err := promptkit.NewEngine(promptkit.Config{}, promptkit.WithLLMClient(countingLLMClient{calls: &modelCalls}))
if err != nil {
t.Fatalf("construct engine: %v", err)
}
@@ -136,9 +123,6 @@ func TestInvalidAppendedMessagesFailBeforeSourceOrModelWork(t *testing.T) {
}
})
}
if promptSource.opens.Load() != 0 {
t.Fatalf("invalid appended message opened prompt sources %d times", promptSource.opens.Load())
}
if modelCalls.Load() != 0 {
t.Fatalf("invalid appended message invoked the model %d times", modelCalls.Load())
}

View File

@@ -1,107 +1,69 @@
package promptkit
import (
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit/internal/backend"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
)
// BackendOpenRouter is the reserved ID of Promptkit's built-in OpenRouter
// backend.
// BackendOpenRouter identifies the maintained OpenRouter backend.
const BackendOpenRouter = backend.OpenRouterID
// BackendRakestrawHome is the reserved ID of Promptkit's built-in
// Rakestrawhome backend.
// BackendRakestrawHome identifies the maintained Rakestrawhome backend.
const BackendRakestrawHome = backend.RakestrawHomeID
// BackendLocal is the case-sensitive conventional ID used by [LocalBackend].
// It is not a built-in or reserved backend and must be registered with
// [WithBackend].
// BackendLocal is a conventional ID for a caller-configured local backend.
const BackendLocal = "local"
// Backend configures one engine-scoped OpenAI-compatible backend.
//
// Backend has no stable JSON representation. Use keyed literals so additions
// to this configuration value do not break source compatibility.
type Backend struct {
// ID is the stable, case-sensitive registry key. NewEngine trims it and
// requires a non-blank value. Built-in backend IDs are reserved.
ID string
// Endpoint is the OpenAI-compatible base endpoint. NewEngine trims it and
// requires an absolute HTTP or HTTPS URL with a host and without user
// information, a query string, or a fragment. Paths are allowed.
Endpoint string
// APIKeyEnv optionally names an environment lookup source for an API key.
// NewEngine trims it and requires the portable form [A-Za-z_][A-Za-z0-9_]*.
// A direct RunRequest.APIKey takes precedence. When no usable credential is
// available, the built-in client omits Authorization; injected clients own
// their own credential-resolution behavior. Store only the name, never a
// credential value.
APIKeyEnv string
// ExtraParams contains backend-wide request defaults. Values must be
// JSON-compatible, finite, acyclic, and keyed by non-empty strings. Keys
// must not be model, session_id, messages, temperature, max_tokens, top_p,
// service_tier, reasoning_effort, or response_format. An empty map supplies
// no defaults. NewEngine deeply copies the map and rejects excessively deep
// or large values for safety.
ExtraParams map[string]any
// ConcurrencyLimit is the maximum number of simultaneous model-generation
// calls allowed for this backend within one Engine. Zero leaves the backend
// unlimited. A negative value makes NewEngine fail with ErrInvalidConfig.
ConcurrencyLimit int
// QueueCapacity controls how many additional Run or RunPrepared calls may
// be admitted beyond ConcurrencyLimit. Nil uses 1024 when ConcurrencyLimit
// is positive; a pointer uses its exact value, including zero. The pointed-to
// value must be non-negative, and QueueCapacity must be nil when
// ConcurrencyLimit is zero. Their sum must fit in an int. WithBackend copies
// the value and does not retain the pointer.
// BackendConfig is a presence-aware convenience value for a keyed backend
// definition. Nil pointer fields are omitted, while non-nil pointers preserve
// explicit zero or blank values. A nil ExtraParams map is omitted; a non-nil
// empty map supplies an empty mapping. Use raw ConfigKit Mapping inputs for
// explicit null. BackendConfig has no stable JSON representation.
type BackendConfig struct {
// Endpoint is the required absolute HTTP(S) base URL after all layers merge.
Endpoint *string
// APIKeyEnv optionally names an environment variable, not a credential.
APIKeyEnv *string
// ConcurrencyLimit is zero for unlimited generation; negative is invalid.
ConcurrencyLimit *int
// QueueCapacity defaults to 1024 for positive concurrency when omitted.
// An explicit value, including zero, requires positive concurrency.
QueueCapacity *int
// ExtraParams supplies JSON-compatible provider request defaults.
ExtraParams map[string]any
}
// LocalBackend returns a caller-owned Backend for a conventional local
// OpenAI-compatible endpoint. It sets ID to BackendLocal and copies endpoint
// and concurrencyLimit into Endpoint and ConcurrencyLimit without
// normalization or validation. APIKeyEnv, ExtraParams, and QueueCapacity keep
// their zero values.
//
// LocalBackend does not read environment variables, register the value, or
// mutate engine or package state. Supply the returned value through
// [WithBackend]; [NewEngine] then applies the ordinary backend validation and
// concurrency semantics, including default queue capacity for a positive
// limit, unlimited behavior for zero, and ErrInvalidConfig for a negative
// limit.
func LocalBackend(endpoint string, concurrencyLimit int) Backend {
return Backend{
ID: BackendLocal,
Endpoint: endpoint,
ConcurrencyLimit: concurrencyLimit,
// BackendSource creates a labeled ConfigKit mapping from keyed backend values.
// It encodes fields without validating them; NewEngine validates the final
// effective configuration after all selected source layers are combined.
// Mapping captures a copy of supported input values at creation; unsupported
// values fail when the source is loaded.
func BackendSource(label string, backends map[string]BackendConfig) configkit.Source {
definitions := make(map[string]any, len(backends))
for id, config := range backends {
fields := make(map[string]any)
if config.Endpoint != nil {
fields["endpoint"] = *config.Endpoint
}
if config.APIKeyEnv != nil {
fields["api_key_env"] = *config.APIKeyEnv
}
if config.ConcurrencyLimit != nil {
fields["concurrency_limit"] = *config.ConcurrencyLimit
}
if config.QueueCapacity != nil {
fields["queue_capacity"] = *config.QueueCapacity
}
if config.ExtraParams != nil {
fields["extra_params"] = providerMapForMapping(config.ExtraParams)
}
definitions[id] = fields
}
return configkit.Mapping(label, map[string]any{"backends": definitions})
}
// WithBackend adds one Backend registration to the constructed Engine.
//
// Registrations accumulate in option order. Every normalized ID must be unique
// across consumer registrations and built-ins; a duplicate or invalid
// definition makes NewEngine fail with ErrInvalidConfig. Built-in IDs,
// including [BackendOpenRouter] and [BackendRakestrawHome], cannot be
// replaced. The immutable registration is scoped to the resulting Engine and
// cannot be enumerated, replaced, removed, or mutated after construction.
// WithBackend does not install package-global state.
func WithBackend(backend Backend) Option {
queueCapacity := 0
queueCapacitySet := backend.QueueCapacity != nil
if queueCapacitySet {
queueCapacity = *backend.QueueCapacity
}
return optionFunc(func(options *engineOptions) error {
options.internal.AddBackend(domain.Backend{
ID: backend.ID,
Endpoint: backend.Endpoint,
APIKeyEnv: backend.APIKeyEnv,
ExtraParams: backend.ExtraParams,
ConcurrencyLimit: backend.ConcurrencyLimit,
QueueCapacity: queueCapacity,
QueueCapacitySet: queueCapacitySet,
})
return nil
})
// LocalBackend supplies endpoint and concurrency pointers for a caller-chosen
// backend ID; it does not select an ID or validate either value.
func LocalBackend(endpoint string, concurrencyLimit int) BackendConfig {
return BackendConfig{Endpoint: &endpoint, ConcurrencyLimit: &concurrencyLimit}
}

View File

@@ -7,6 +7,7 @@ import (
"testing"
"time"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit"
)
@@ -150,17 +151,7 @@ func TestUnlimitedBackendsRetainInjectedClientConcurrency(t *testing.T) {
{
name: "endpoint-only profile",
configure: func(t *testing.T, client promptkit.LLMClient) *promptkit.Engine {
engine, err := promptkit.NewEngine(promptkit.Config{},
promptkit.WithPromptFS(contractPromptFS("prompt", "profile", "message"), "."),
promptkit.WithProfiles(promptkit.Profile{
ID: "profile", Endpoint: "http://endpoint.example/v1", Model: "model",
}),
promptkit.WithLLMClient(client),
)
if err != nil {
t.Fatalf("construct endpoint-only engine: %v", err)
}
return engine
return newSimpleEngine(t, "message", promptkit.WithLLMClient(client))
},
},
}
@@ -250,27 +241,35 @@ func newBackendCapacityEngine(
reader promptkit.ArtifactReader,
) *promptkit.Engine {
t.Helper()
promptFS := contractPromptFS("prompt", "profile", "message")
promptID, content := "prompt", "message"
var inputs []any
if reader != nil {
promptFS = contractInputPromptFS()
promptID, content = "input-prompt", `{{input "input"}}`
inputs = []any{map[string]any{"name": "input", "required": true}}
}
options := []promptkit.Option{
promptkit.WithPromptFS(promptFS, "."),
promptkit.WithBackend(promptkit.Backend{
ID: "limited",
Endpoint: "http://backend.example/v1",
ConcurrencyLimit: limit,
QueueCapacity: queueCapacity,
}),
promptkit.WithProfiles(promptkit.Profile{
ID: "profile", BackendID: "limited", Model: "model",
}),
promptkit.WithLLMClient(client),
backend := map[string]any{"endpoint": "http://backend.example/v1", "concurrency_limit": limit}
if queueCapacity != nil {
backend["queue_capacity"] = *queueCapacity
}
definition := map[string]any{
"default_profile": "profile",
"messages": []any{map[string]any{"role": "user", "content": content}},
"output": map[string]any{"format": "text", "validation_mode": "none"},
}
if inputs != nil {
definition["inputs"] = inputs
}
options := []promptkit.Option{promptkit.WithLLMClient(client)}
if reader != nil {
options = append(options, promptkit.WithArtifactReader(reader))
}
engine, err := promptkit.NewEngine(promptkit.Config{}, options...)
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{Main: &configkit.Input{
Source: configkit.Mapping("capacity fixture", map[string]any{
"backends": map[string]any{"limited": backend},
"profiles": map[string]any{"profile": map[string]any{"backend": "limited", "model": "model"}},
"prompts": map[string]any{promptID: map[string]any{"1": definition}},
}),
}}}, options...)
if err != nil {
t.Fatalf("construct capacity engine: %v", err)
}

View File

@@ -0,0 +1,82 @@
package promptkit
import (
"fmt"
"io"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit/internal/configuration"
)
// ConfigurationInspection is a read-only view of one Engine's captured
// effective configuration and source history. It does not expose typed
// configuration values, filesystem contexts, or runtime credentials.
// Report content may include application prompt and provider data, so callers
// choose where and to whom to disclose it.
type ConfigurationInspection struct {
source configuration.Inspection
}
// InspectConfiguration returns the ConfigKit inspection view captured by this
// Engine. It does not reload sources, resolve credentials, or contact a model.
// A nil Engine returns ErrInvalidConfig.
func (e *Engine) InspectConfiguration() (ConfigurationInspection, error) {
if e == nil || e.internal == nil {
return ConfigurationInspection{}, fmt.Errorf("%w: engine is nil", ErrInvalidConfig)
}
return ConfigurationInspection{source: e.internal.InspectConfiguration()}, nil
}
// Metadata returns detached effective presence and history records.
func (v ConfigurationInspection) Metadata() configkit.Metadata {
if v.source == nil {
return configkit.Metadata{}
}
return v.source.Metadata()
}
// Sources returns a detached list of loaded source descriptions.
func (v ConfigurationInspection) Sources() []configkit.SourceInfo {
if v.source == nil {
return nil
}
return v.source.Sources()
}
// Diagnostics returns detached nonfatal load diagnostics.
func (v ConfigurationInspection) Diagnostics() []configkit.Diagnostic {
if v.source == nil {
return nil
}
return v.source.Diagnostics()
}
// Explain returns detached effective and historical values for a known path.
// Dynamic credential-shaped provider keys are redacted by ConfigKit; ordinary
// prompt and provider content is not automatically safe to disclose.
func (v ConfigurationInspection) Explain(path configkit.Path) (configkit.Explanation, bool) {
if v.source == nil {
return configkit.Explanation{}, false
}
return v.source.Explain(path)
}
// WriteEffective writes ConfigKit's redacted effective YAML. Writer errors
// remain available through errors.Is and errors.As. The zero view returns
// ErrInvalidConfig.
func (v ConfigurationInspection) WriteEffective(w io.Writer) error {
if v.source == nil {
return ErrInvalidConfig
}
return v.source.WriteEffective(w)
}
// WriteExplanation writes ConfigKit's redacted history for path. Unknown paths
// produce no output; writer errors remain inspectable. The zero view returns
// ErrInvalidConfig.
func (v ConfigurationInspection) WriteExplanation(w io.Writer, path configkit.Path) error {
if v.source == nil {
return ErrInvalidConfig
}
return v.source.WriteExplanation(w, path)
}

View File

@@ -0,0 +1,200 @@
package promptkit_test
import (
"bytes"
"context"
"errors"
"io"
"strings"
"sync"
"sync/atomic"
"testing"
"testing/fstest"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestConfigurationInspectionReportsLiteralKeysAndRedactedHistory(t *testing.T) {
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Underlays: []configkit.Underlay{
{Inputs: []configkit.Input{{Source: configkit.Mapping("defaults", map[string]any{
"backends": map[string]any{"local": map[string]any{
"endpoint": "https://example.invalid/v1", "api_key_env": "OPTIONAL_KEY",
"extra_params": map[string]any{"token": "private-sentinel", "visible": "public-value"},
}},
"profiles": map[string]any{
"base.v1": map[string]any{"backend": "local", "model": "default-model"},
"literal/id": map[string]any{"extends": "base.v1"},
},
})}}},
{Inputs: []configkit.Input{{Source: configkit.Mapping("operator", map[string]any{
"profiles": map[string]any{"base.v1": map[string]any{"model": "operator-model"}},
})}}},
},
}})
if err != nil {
t.Fatalf("construct layered engine: %v", err)
}
view, err := engine.InspectConfiguration()
if err != nil {
t.Fatalf("inspect configuration: %v", err)
}
modelPath := configkit.Keys("profiles", "base.v1", "model")
model, found := view.Explain(modelPath)
if !found || !strings.Contains(model.Effective.YAML, "operator-model") || len(model.History) < 2 {
t.Fatalf("layered model explanation = (%+v, %t)", model, found)
}
inheritedPath := configkit.Keys("profiles", "literal/id", "model")
inherited, found := view.Explain(inheritedPath)
if !found || !strings.Contains(inherited.Effective.YAML, "operator-model") || len(inherited.History) == 0 || !view.Metadata().IsExplicit(inheritedPath) {
t.Fatalf("literal-key inheritance explanation = (%+v, %t)", inherited, found)
}
if inheritedPath.String() != "/profiles/literal~1id/model" {
t.Fatalf("literal-key path = %q", inheritedPath)
}
secretPath := configkit.Keys("backends", "local", "extra_params", "token")
secret, found := view.Explain(secretPath)
if !found || !secret.Effective.Redacted || strings.Contains(secret.Effective.YAML, "private-sentinel") {
t.Fatalf("credential-shaped provider value was not redacted: (%+v, %t)", secret, found)
}
var effective bytes.Buffer
if err := view.WriteEffective(&effective); err != nil {
t.Fatalf("write effective configuration: %v", err)
}
if strings.Contains(effective.String(), "private-sentinel") || !strings.Contains(effective.String(), "OPTIONAL_KEY") || !strings.Contains(effective.String(), "public-value") {
t.Fatalf("unexpected effective report: %s", effective.String())
}
var explanation bytes.Buffer
if err := view.WriteExplanation(&explanation, secretPath); err != nil || strings.Contains(explanation.String(), "private-sentinel") {
t.Fatalf("secret explanation = (%q, %v)", explanation.String(), err)
}
sources := view.Sources()
if len(sources) == 0 {
t.Fatal("inspection returned no sources")
}
originalLabel := sources[0].Label
sources[0].Label = "changed"
if view.Sources()[0].Label != originalLabel {
t.Fatal("source mutation reached retained inspection")
}
model.History[0].After.YAML = "changed"
if again, ok := view.Explain(modelPath); !ok || again.History[0].After.YAML == "changed" {
t.Fatal("explanation mutation reached retained inspection")
}
history := view.Metadata().History(modelPath)
if len(history) == 0 {
t.Fatal("metadata history is empty")
}
history[0].Reasons = []string{"changed"}
if again := view.Metadata().History(modelPath); len(again) == 0 || (len(again[0].Reasons) > 0 && again[0].Reasons[0] == "changed") {
t.Fatal("metadata mutation reached retained inspection")
}
}
type failingInspectionWriter struct{ err error }
func (writer failingInspectionWriter) Write([]byte) (int, error) { return 0, writer.err }
func TestConfigurationInspectionZeroAndWriterErrors(t *testing.T) {
var engine *promptkit.Engine
if _, err := engine.InspectConfiguration(); !errors.Is(err, promptkit.ErrInvalidConfig) {
t.Fatalf("nil engine inspection = %v", err)
}
var empty promptkit.ConfigurationInspection
path := configkit.Keys("profiles", "missing")
if _, found := empty.Metadata().Lookup(path); found || len(empty.Sources()) != 0 || len(empty.Diagnostics()) != 0 {
t.Fatal("zero inspection reported configuration")
}
if _, found := empty.Explain(path); found {
t.Fatal("zero inspection explained an unknown path")
}
if err := empty.WriteEffective(io.Discard); !errors.Is(err, promptkit.ErrInvalidConfig) {
t.Fatalf("zero effective writer = %v", err)
}
if err := empty.WriteExplanation(io.Discard, path); !errors.Is(err, promptkit.ErrInvalidConfig) {
t.Fatalf("zero explanation writer = %v", err)
}
valid, err := promptkit.NewEngine(promptkit.Config{})
if err != nil {
t.Fatalf("construct engine: %v", err)
}
view, err := valid.InspectConfiguration()
if err != nil {
t.Fatalf("inspect configuration: %v", err)
}
writeErr := errors.New("report writer failed")
writer := failingInspectionWriter{err: writeErr}
if err := view.WriteEffective(writer); !errors.Is(err, writeErr) {
t.Fatalf("effective writer error = %v", err)
}
if err := view.WriteExplanation(writer, configkit.Keys("profiles", "mistral-small-3", "model")); !errors.Is(err, writeErr) {
t.Fatalf("explanation writer error = %v", err)
}
}
func TestConfigurationInspectionUsesCapturedSourcesWithoutProviderCalls(t *testing.T) {
source := &unreadableAfterConstructionFS{FS: fstest.MapFS{
"config/prompt.yaml": &fstest.MapFile{Data: []byte(capturedPromptDocument)},
"config/body.tmpl": &fstest.MapFile{Data: []byte("hello")},
}}
inputs, err := promptkit.FilesystemInputs("inspection source", source, "config")
if err != nil {
t.Fatalf("select inputs: %v", err)
}
var modelCalls atomic.Int64
client := countingLLMClient{calls: &modelCalls}
endpoint, model := "https://example.invalid/v1", "offline-model"
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Underlays: []configkit.Underlay{{Inputs: inputs}, {Inputs: []configkit.Input{{Source: promptkit.ProfileSource("offline profile", map[string]promptkit.ProfileConfig{
"offline": {Endpoint: &endpoint, Model: &model},
})}}}},
}}, promptkit.WithLLMClient(client))
if err != nil {
t.Fatalf("construct engine: %v", err)
}
source.closed.Store(true)
view, err := engine.InspectConfiguration()
if err != nil {
t.Fatalf("inspect configuration: %v", err)
}
if _, found := view.Explain(configkit.Keys("prompts", "captured", "1", "messages").Index(0).Key("content_file")); !found {
t.Fatal("captured prompt content path was not inspectable")
}
var report bytes.Buffer
if err := view.WriteEffective(&report); err != nil || !strings.Contains(report.String(), "captured") {
t.Fatalf("write captured report = (%q, %v)", report.String(), err)
}
if source.reads.Load() != 0 || modelCalls.Load() != 0 {
t.Fatalf("inspection touched source or provider: reads=%d requests=%d", source.reads.Load(), modelCalls.Load())
}
const calls = 16
var wait sync.WaitGroup
errs := make(chan error, calls)
for index := 0; index < calls; index++ {
wait.Add(1)
go func(index int) {
defer wait.Done()
if index%2 == 0 {
_, err := engine.InspectConfiguration()
errs <- err
return
}
_, err := engine.Run(context.Background(), promptkit.RunRequest{PromptID: "captured", SessionID: "direct"})
errs <- err
}(index)
}
wait.Wait()
close(errs)
for err := range errs {
if err != nil {
t.Fatal(err)
}
}
if source.reads.Load() != 0 || modelCalls.Load() != calls/2 {
t.Fatalf("concurrent inspection/execution = source reads %d, model calls %d", source.reads.Load(), modelCalls.Load())
}
}

83
configuration_sources.go Normal file
View File

@@ -0,0 +1,83 @@
package promptkit
import (
"context"
"fmt"
"io/fs"
"os"
"path/filepath"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit/internal/filecatalog"
)
// DirectoryInputs selects the YAML files beneath an explicitly chosen host
// directory as one strict-composition group. The returned order is deterministic;
// it does not assign precedence among those files. Discovery is recursive over
// .yaml and .yml files. An unavailable root fails; an empty directory yields
// no inputs. Imports are disabled unless callers enable them on the inputs.
func DirectoryInputs(directory string) ([]configkit.Input, error) {
root, err := filepath.Abs(directory)
if err != nil {
return nil, fmt.Errorf("resolve configuration directory: %w", err)
}
info, err := os.Stat(root)
if err != nil {
return nil, fmt.Errorf("inspect configuration directory: %w", err)
}
if !info.IsDir() {
return nil, fmt.Errorf("configuration root is not a directory: %s", root)
}
root, err = filepath.EvalSymlinks(root)
if err != nil {
return nil, fmt.Errorf("resolve configuration directory: %w", err)
}
files, err := filecatalog.FindYAMLFiles(context.Background(), root)
if err != nil {
return nil, fmt.Errorf("discover configuration files: %w", err)
}
inputs := make([]configkit.Input, 0, len(files))
for _, file := range files {
inputs = append(inputs, configkit.Input{
Source: configkit.File(file),
Filesystem: &configkit.FilesystemContext{Root: root},
})
}
return inputs, nil
}
// FilesystemInputs selects the YAML files beneath root in a caller-owned
// filesystem as one strict-composition group. Discovery is recursive over
// .yaml and .yml files, with deterministic ordering and no precedence among
// files. The caller retains ownership of the filesystem; an unavailable root
// fails and an empty directory yields no inputs. Imports remain disabled unless
// callers enable them on inputs.
func FilesystemInputs(label string, filesystem fs.FS, root string) ([]configkit.Input, error) {
if filesystem == nil {
return nil, fmt.Errorf("configuration filesystem is nil")
}
root = filecatalog.CleanFSRoot(root)
if !fs.ValidPath(root) {
return nil, fmt.Errorf("configuration root is not a valid filesystem path: %s", root)
}
info, err := fs.Stat(filesystem, root)
if err != nil {
return nil, fmt.Errorf("inspect configuration root: %w", err)
}
if !info.IsDir() {
return nil, fmt.Errorf("configuration root is not a directory: %s", root)
}
files, err := filecatalog.FindFSYAMLFiles(context.Background(), filesystem, root)
if err != nil {
return nil, fmt.Errorf("discover configuration files: %w", err)
}
namespace := configkit.NewFilesystem(label, filesystem)
inputs := make([]configkit.Input, 0, len(files))
for _, file := range files {
inputs = append(inputs, configkit.Input{
Source: namespace.File(file),
Filesystem: &configkit.FilesystemContext{FS: &namespace, Root: root},
})
}
return inputs, nil
}

View File

@@ -0,0 +1,83 @@
package promptkit_test
import (
"io/fs"
"os"
"path/filepath"
"testing"
"testing/fstest"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit"
)
type sourceDocument struct {
First string `yaml:"first"`
Second string `yaml:"second"`
}
func TestDirectoryInputsSelectsSortedFilesWithRootContext(t *testing.T) {
root := t.TempDir()
if err := os.Mkdir(filepath.Join(root, "nested"), 0o755); err != nil {
t.Fatal(err)
}
for name, data := range map[string]string{
"a.yaml": "first: one\n", "nested/b.yml": "second: two\n", "nested/body.md": "message",
} {
if err := os.WriteFile(filepath.Join(root, name), []byte(data), 0o600); err != nil {
t.Fatal(err)
}
}
inputs, err := promptkit.DirectoryInputs(root)
if err != nil || len(inputs) != 2 || inputs[0].Imports || inputs[1].Imports {
t.Fatalf("directory inputs: %+v, %v", inputs, err)
}
assertSourceInputs(t, inputs, "message")
if _, err := promptkit.DirectoryInputs(filepath.Join(root, "missing")); err == nil {
t.Fatal("missing selected directory succeeded")
}
if empty, err := promptkit.DirectoryInputs(t.TempDir()); err != nil || len(empty) != 0 {
t.Fatalf("empty directory: %+v, %v", empty, err)
}
}
func TestFilesystemInputsSelectsEmbeddedFilesWithRootContext(t *testing.T) {
files := fstest.MapFS{
"settings/a.yaml": {Data: []byte("first: one\n")},
"settings/nested/b.yml": {Data: []byte("second: two\n")},
"settings/nested/body.md": {Data: []byte("message")},
}
inputs, err := promptkit.FilesystemInputs("embedded", files, "settings")
if err != nil || len(inputs) != 2 || inputs[0].Imports || inputs[1].Imports {
t.Fatalf("filesystem inputs: %+v, %v", inputs, err)
}
assertSourceInputs(t, inputs, "message")
if _, err := promptkit.FilesystemInputs("embedded", files, "missing"); err == nil {
t.Fatal("missing selected root succeeded")
}
}
func assertSourceInputs(t *testing.T, inputs []configkit.Input, expected string) {
t.Helper()
result, err := configkit.LoadLayered(configkit.Input{Source: configkit.NoSource()}, configkit.LayeredOptions[sourceDocument]{
Underlays: []configkit.Underlay{{Inputs: inputs}},
})
if err != nil {
t.Fatal(err)
}
if result.Config.First != "one" || result.Config.Second != "two" {
t.Fatalf("configuration files did not compose: %+v", result.Config)
}
field, found := result.Metadata().Lookup(configkit.Keys("second"))
if !found || len(field.Origin.Locations) != 1 {
t.Fatalf("missing second-field origin: %+v", field)
}
context, found := result.SourceContext(field.Origin.Locations[0].SourceID)
if !found || context.BaseDir != "nested" {
t.Fatalf("unexpected rooted source context: %+v", context)
}
data, err := fs.ReadFile(context.FS, "nested/body.md")
if err != nil || string(data) != expected {
t.Fatalf("rooted resource read: %q, %v", data, err)
}
}

View File

@@ -0,0 +1,231 @@
package promptkit_test
import (
"context"
"errors"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"testing"
"testing/fstest"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit"
)
func TestProfileEnvironmentLookupAcceptsNonShellNames(t *testing.T) {
const name = "promptkit.test-api-key"
for _, test := range []struct {
name, value, direct, want string
unset, clear bool
}{
{name: "set", value: " synthetic-key ", want: "Bearer synthetic-key"},
{name: "empty"},
{name: "whitespace", value: " "},
{name: "unset", unset: true},
{name: "direct precedence", value: "synthetic-key", direct: "direct-key", want: "Bearer direct-key"},
{name: "explicit clear", value: "synthetic-key", clear: true},
} {
t.Run(test.name, func(t *testing.T) {
t.Setenv(name, test.value)
if test.unset {
if err := os.Unsetenv(name); err != nil {
t.Fatal(err)
}
}
cfg := simpleConfig("hello")
cfg.Sources.Underlays = []configkit.Underlay{{Inputs: []configkit.Input{*cfg.Sources.Main}}}
lowerName := name
cfg.Sources.Underlays = append(cfg.Sources.Underlays, configkit.Underlay{Inputs: []configkit.Input{{
Source: promptkit.ProfileSource("default credentials", map[string]promptkit.ProfileConfig{
"profile": {APIKeyEnv: &lowerName},
}),
}}})
env := " " + name + " "
if test.clear {
env = ""
}
cfg.Sources.Main = &configkit.Input{Source: promptkit.ProfileSource("credentials", map[string]promptkit.ProfileConfig{
"profile": {APIKeyEnv: &env},
})}
var calls int
var auth string
cfg.HTTPClient = &http.Client{Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
calls++
auth = request.Header.Get("Authorization")
return &http.Response{StatusCode: http.StatusOK, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(`{"choices":[{"message":{"content":"ok"}}]}`))}, nil
})}
engine, err := promptkit.NewEngine(cfg)
if err != nil {
t.Fatal(err)
}
_, err = engine.Run(context.Background(), promptkit.RunRequest{PromptID: "prompt", APIKey: test.direct})
if err != nil || calls != 1 || auth != test.want {
t.Fatalf("run: err=%v calls=%d authorization=%q, want %q", err, calls, auth, test.want)
}
})
}
}
func TestPublicConstructionWithLocalBackendAndProgrammaticProfile(t *testing.T) {
backendID, model := "workstation", "offline-model"
prompt := configkit.Input{Source: configkit.Mapping("prompt", map[string]any{
"prompts": map[string]any{"hello": map[string]any{"1": map[string]any{
"default_profile": "offline",
"messages": []any{map[string]any{"role": "user", "content": "Hello"}},
"output": map[string]any{"format": "text", "validation_mode": "none"},
}}},
})}
client := &fakeLLMClient{response: &promptkit.GenerateResponse{Content: "reply"}}
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{
{Source: promptkit.BackendSource("local backend", map[string]promptkit.BackendConfig{
backendID: promptkit.LocalBackend("http://localhost:8000/v1", 2),
})},
{Source: promptkit.ProfileSource("profile", map[string]promptkit.ProfileConfig{
"offline": {Backend: &backendID, Model: &model},
})},
}}},
Main: &prompt,
}}, promptkit.WithLLMClient(client))
if err != nil {
t.Fatalf("construct engine: %v", err)
}
result, err := engine.Run(context.Background(), promptkit.RunRequest{PromptID: "hello"})
if err != nil {
t.Fatalf("run prompt: %v", err)
}
if result.RawOutput != "reply" || result.SelectedBackendID != backendID || result.EffectiveModelParams.Model != model {
t.Fatalf("unexpected result: %+v", result)
}
if len(client.requests) != 1 || client.requests[0].Target.Endpoint != "http://localhost:8000/v1" {
t.Fatalf("unexpected model request: %+v", client.requests)
}
}
func TestPublicConstructionCombinesEmbeddedOperatorAndProgrammaticInputs(t *testing.T) {
embedded, err := promptkit.FilesystemInputs("embedded defaults", fstest.MapFS{
"config/default.yaml": &fstest.MapFile{Data: []byte(`
profiles:
application:
endpoint: https://example.invalid/v1
model: default-model
prompts:
hello:
"1":
default_profile: application
messages:
- role: user
content: Hello
output:
format: text
validation_mode: none
`)},
}, "config")
if err != nil {
t.Fatalf("select embedded defaults: %v", err)
}
operatorDirectory := t.TempDir()
if err := os.WriteFile(filepath.Join(operatorDirectory, "override.yaml"), []byte("profiles:\n application:\n model: operator-model\n"), 0o644); err != nil {
t.Fatalf("write operator input: %v", err)
}
operator, err := promptkit.DirectoryInputs(operatorDirectory)
if err != nil {
t.Fatalf("select operator inputs: %v", err)
}
programmaticModel := "programmatic-model"
programmatic := configkit.Input{Source: promptkit.ProfileSource("programmatic settings", map[string]promptkit.ProfileConfig{
"application": {Model: &programmaticModel},
})}
for _, test := range []struct {
name string
apply bool
want string
}{
{name: "embedded", want: "default-model"},
{name: "operator", apply: true, want: "operator-model"},
{name: "programmatic", apply: true, want: "programmatic-model"},
} {
t.Run(test.name, func(t *testing.T) {
underlays := []configkit.Underlay{{Inputs: embedded}}
if test.apply {
underlays = append(underlays, configkit.Underlay{Inputs: operator})
}
if test.name == "programmatic" {
underlays = append(underlays, configkit.Underlay{Inputs: []configkit.Input{programmatic}})
}
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{Underlays: underlays}})
if err != nil {
t.Fatalf("construct engine: %v", err)
}
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "hello"})
if err != nil {
t.Fatalf("prepare prompt: %v", err)
}
if prepared.EffectiveModelParams.Model != test.want {
t.Fatalf("model = %q, want %q", prepared.EffectiveModelParams.Model, test.want)
}
})
}
}
func TestPublicConstructionPreservesConfigKitLoadError(t *testing.T) {
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{Main: &configkit.Input{
Source: configkit.Mapping("invalid", map[string]any{"profiles": map[string]any{"broken": map[string]any{"unknown": true}}}),
}}})
var loadErr *configkit.LoadError
if engine != nil || !errors.Is(err, promptkit.ErrInvalidConfig) || !errors.As(err, &loadErr) {
t.Fatalf("construction = (%#v, %v), want invalid config preserving LoadError", engine, err)
}
}
func TestPublicConstructionRejectsSuppliedInvalidMainInput(t *testing.T) {
main := configkit.Input{}
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{Main: &main}})
if engine != nil || !errors.Is(err, promptkit.ErrInvalidConfig) {
t.Fatalf("construction = (%#v, %v), want invalid supplied main input", engine, err)
}
}
func TestPublicConstructionCanOverlayMaintainedBackend(t *testing.T) {
endpoint := "http://local-openrouter.example/v1"
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{Source: promptkit.BackendSource("operator backend", map[string]promptkit.BackendConfig{
promptkit.BackendOpenRouter: {Endpoint: &endpoint},
})}}}},
}})
if err != nil {
t.Fatalf("construct engine with backend overlay: %v", err)
}
inspection, err := engine.InspectProfile(context.Background(), "mistral-small-3")
if err != nil {
t.Fatalf("inspect catalog profile: %v", err)
}
if inspection.EffectiveModelParams.Endpoint != endpoint {
t.Fatalf("effective endpoint = %q, want %q", inspection.EffectiveModelParams.Endpoint, endpoint)
}
}
func TestPreparationRejectsAmbiguousPromptVersion(t *testing.T) {
definition := func(content string) map[string]any {
return map[string]any{
"messages": []any{map[string]any{"role": "user", "content": content}},
"output": map[string]any{"format": "text", "validation_mode": "none"},
}
}
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Main: &configkit.Input{Source: configkit.Mapping("versions", map[string]any{
"prompts": map[string]any{"hello": map[string]any{
"1": definition("first"), "2": definition("second"),
}},
})},
}})
if err != nil {
t.Fatalf("construct engine: %v", err)
}
if result, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "hello"}); result != nil || !errors.Is(err, promptkit.ErrPromptLoad) {
t.Fatalf("ambiguous preparation = (%#v, %v), want ErrPromptLoad", result, err)
}
}

View File

@@ -0,0 +1,73 @@
package promptkit_test
import (
"context"
"fmt"
"net/http"
"sync/atomic"
"testing"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit"
)
type fakeLLMClient struct {
response *promptkit.GenerateResponse
responses []*promptkit.GenerateResponse
err error
requests []promptkit.GenerateRequest
}
func (f *fakeLLMClient) Generate(_ context.Context, request promptkit.GenerateRequest) (*promptkit.GenerateResponse, error) {
f.requests = append(f.requests, request)
if f.err != nil {
return nil, f.err
}
if len(f.responses) > 0 {
index := len(f.requests) - 1
if index >= len(f.responses) {
return nil, fmt.Errorf("no response configured for generation %d", index+1)
}
return f.responses[index], nil
}
return f.response, nil
}
type countingLLMClient struct{ calls *atomic.Int64 }
func (client countingLLMClient) Generate(context.Context, promptkit.GenerateRequest) (*promptkit.GenerateResponse, error) {
if client.calls != nil {
client.calls.Add(1)
}
return &promptkit.GenerateResponse{Content: "ok"}, nil
}
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(request *http.Request) (*http.Response, error) { return f(request) }
// newSimpleEngine exercises the public constructor with a complete keyed
// document. Tests that need layering or resources select their own inputs.
func newSimpleEngine(t *testing.T, content string, options ...promptkit.Option) *promptkit.Engine {
t.Helper()
engine, err := promptkit.NewEngine(simpleConfig(content), options...)
if err != nil {
t.Fatalf("construct consumer test engine: %v", err)
}
return engine
}
func simpleConfig(content string) promptkit.Config {
return promptkit.Config{Sources: promptkit.ConfigurationSources{
Main: &configkit.Input{Source: configkit.Mapping("consumer test", map[string]any{
"profiles": map[string]any{"profile": map[string]any{
"endpoint": "http://example.test/v1", "model": "model",
}},
"prompts": map[string]any{"prompt": map[string]any{"1": map[string]any{
"default_profile": "profile",
"messages": []any{map[string]any{"role": "user", "content": content}},
"output": map[string]any{"format": "text", "validation_mode": "none"},
}}},
})},
}}
}

18
doc.go
View File

@@ -1,23 +1,26 @@
// Package promptkit provides an embeddable engine for preparing and executing
// prompt-defined LLM workflows.
//
// Applications construct an [Engine] with [NewEngine], select filesystem or
// in-memory sources and optional engine-scoped [Backend] registrations, and
// call [Engine.InspectPrompt], [Engine.InspectProfile], [Engine.Prepare],
// Applications construct an [Engine] with [NewEngine], select ConfigKit
// sources for prompts, profiles, and backends, and
// call [Engine.InspectConfiguration], [Engine.InspectPrompt],
// [Engine.InspectProfile], [Engine.Prepare],
// [Engine.PrepareExecution], [Engine.Run], or [Engine.RunPrepared]. Concrete
// registries, repositories, validators, and the built-in OpenAI-compatible
// client remain internal implementation details.
//
// # Concurrency and ownership
//
// An Engine supports concurrent InspectPrompt, InspectProfile, Prepare,
// An Engine supports concurrent InspectConfiguration, InspectPrompt,
// InspectProfile, Prepare,
// PrepareExecution, Run, and RunPrepared calls. Engine-local backend policies
// bound admitted Run and RunPrepared calls and model generations where
// configured, while different backend pools and unlimited backends continue
// independently. An injected [LLMClient] or [ArtifactReader] can therefore
// still receive concurrent calls and must be safe for that use.
//
// NewEngine copies in-memory profiles and backend definitions. Prepare,
// ConfigKit captures the selected configuration when NewEngine constructs an
// Engine. Prepare,
// PrepareExecution, and Run copy request maps, slices, pointer values, and
// JSON-compatible extra parameters before using them. InspectPrompt and
// InspectProfile return copied inspection values. Returned values and values
@@ -52,8 +55,9 @@
// used by those values.
//
// Construction, inspection, handle, and error values, including [Config],
// [Backend], [RunRequest], [ArtifactRef], [ExecutionTargetOverride], [Profile],
// [OpenAICompatibleProfileConfig], [ProfileInspection],
// [ConfigurationSources], [ConfigurationInspection], [BackendConfig],
// [ProfileConfig], [RunRequest],
// [ArtifactRef], [ExecutionTargetOverride], [ProfileInspection],
// [PromptInputDefinition], [PromptInspection], [PreparedExecution],
// [CapacityError], [GenerationError], [RetryExhaustedError], and
// [ExecutionEvent], do not have stable JSON

View File

@@ -0,0 +1,60 @@
# 0001: Configuration Ownership At The ConfigKit Handoff
## Status
Accepted.
## Date
2026-09-30
## Context
PromptKit is adopting ConfigKit to standardize configuration semantics and
simplify its loading machinery. Configuration includes prompts, profiles, and
backend definitions contributed by maintained catalogs and consuming applications.
ConfigKit supplies loading, composition, precedence, inheritance, and inspection
capabilities. PromptKit supplies LLM-specific interpretation and execution.
A clear handoff assigns each library responsibility for the configuration it
owns and keeps validation aligned with the values an application actually uses.
## Decision
ConfigKit owns configuration processing up to the handoff of the final effective
configuration. PromptKit takes ownership at that handoff and applies its domain
interpretation and validation to the final configuration.
The handoff includes effective presence/null information and resource context
supplied by ConfigKit. PromptKit uses this information to interpret effective
settings and resolve referenced resources.
PromptKit passes through ConfigKit's inspection capabilities for explicit
operator use. Configuration history and source explanations serve that
inspection purpose; domain validation operates on the final effective values.
This boundary applies uniformly to application configuration and maintained
catalog configuration, including backend definitions.
## Alternatives Considered
- **Independent source or catalog preflight validation:** preserves checks on
individual contributions, but duplicates configuration processing and evaluates
values that may be replaced before handoff. Raw-source and intermediate-state
validation are outside PromptKit's scope, including equivalent test-only checks.
- **History-based source auditing:** can identify overwritten contributions, but
couples domain enforcement to configuration history. Historical inspection is
an operator capability rather than a validation mechanism.
## Consequences
PromptKit's configuration logic and tests focus on the final effective values,
their domain meaning, and resulting execution behavior. ConfigKit owns failures
arising during its processing, which PromptKit preserves through its error
boundary. Catalog maintainers own their repositories' publishing checks.
The adoption work routes backend configuration through the same handoff and
retires PromptKit's source-level and intermediate-configuration validation.
Effective resource interpretation and explicit operator inspection remain
supported. This concentrates configuration semantics in ConfigKit and reduces
duplicated machinery and maintenance in PromptKit.

View File

@@ -21,54 +21,114 @@ programs.
## Construct An Engine
Create an engine with
[`NewEngine`](../../engine.go). A directory-backed setup supplies a prompt
directory and may supply profile and schema directories:
Create an engine with [`NewEngine`](../../engine.go). Select each directory
explicitly and place its inputs in an ordered ConfigKit underlay. A group
strictly composes its files; later groups can overlay earlier ones:
```go
promptInputs, err := promptkit.DirectoryInputs("prompts")
if err != nil { return err }
profileInputs, err := promptkit.DirectoryInputs("profiles")
if err != nil { return err }
engine, err := promptkit.NewEngine(promptkit.Config{
PromptDir: "prompts",
ProfileDir: "profiles",
SchemaDir: "schemas",
Sources: promptkit.ConfigurationSources{Underlays: []configkit.Underlay{
{Inputs: promptInputs},
{Inputs: profileInputs},
}},
SchemaDir: "schemas",
})
```
Options support single-file or `fs.FS` sources, in-memory profiles,
engine-scoped backends, and injected artifact or model clients. Consult the
[constructor and option GoDoc](../../engine.go) for composition, precedence,
validation, and default transport behavior. Source discovery, format
validation, and profile precedence are defined by the
[framework format reference](../formats.md).
For a single highest-priority document, set `Sources.Main` to a
`*configkit.Input`; omit it for ordinary layered construction. A single file
can use `configkit.File` directly. `ProfileSource`, `BackendSource`, and raw
`configkit.Mapping` supply programmatic definitions. Schema and runtime
collaborator options remain available. Construction validates the final
effective configuration and captures referenced prompt content; construct a
new engine to observe configuration edits. See the [constructor GoDoc](../../engine.go)
for the exact public contract and the [framework format reference](../formats.md)
for definition fields.
## Supply Embedded Application Defaults
Use `WithFallbackProfileFS` when an application packages profile definitions
that should apply unless an operator provides an ordinary configured profile
with the same ID. For example, an application can embed its defaults while
continuing to use `ProfileDir` for operator overrides:
Use `FilesystemInputs` for embedded defaults and `DirectoryInputs` for an
explicitly chosen operator directory. A later programmatic group can override
both. No directory name has built-in precedence:
```go
//go:embed profiles/*.yaml
var applicationProfiles embed.FS
engine, err := promptkit.NewEngine(promptkit.Config{
PromptDir: "prompts",
ProfileDir: operatorProfileDir,
},
promptkit.WithFallbackProfileFS(applicationProfiles, "profiles"),
)
defaults, err := promptkit.FilesystemInputs("embedded defaults", applicationFS, "config")
if err != nil { return err }
operator, err := promptkit.DirectoryInputs(operatorDirectory)
if err != nil { return err }
profile := promptkit.ProfileSource("application choices", map[string]promptkit.ProfileConfig{
"summary": {Model: &selectedModel},
})
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Underlays: []configkit.Underlay{
{Inputs: defaults},
{Inputs: operator},
{Inputs: []configkit.Input{{Source: profile}}},
},
}})
```
Keep application-owned profile IDs and definitions in the embedded source.
Use the ordinary configured profile source for operator overrides. Leave
`operatorProfileDir` empty when the operator did not configure an override
directory; a non-empty path names an authoritative higher-precedence source,
so an unavailable or unreadable directory is an error rather than a reason to
fall back. The
[framework format reference](../formats.md#source-and-profile-precedence)
owns the exact profile format and lookup order; the
[`WithFallbackProfileFS` GoDoc](../../engine.go) owns its option contract and
validation rules.
Only add the operator group when the application selects that directory; an
explicitly selected missing or unreadable root is an error. Each helper finds
`.yaml` and `.yml` files recursively and disables imports by default. The
caller may enable imports on individual inputs. A nil `Sources.Main` is an
empty main source, not another directory lookup.
For a programmatic document with explicit null or a prompt definition, use a
labeled mapping as an input. A mapping owns a snapshot of supported values:
```go
document := configkit.Mapping("application configuration", map[string]any{
"profiles": map[string]any{
"local-summary": map[string]any{
"backend": "local", "model": "example-model",
"temperature": nil, // Suppress an inherited provider control.
},
},
})
input := configkit.Input{Source: document}
// Add input to a later underlay, or set Sources.Main = &input.
```
`ProfileSource` and `BackendSource` are smaller typed conveniences when null
is not needed. A caller underlay can overlay catalog backend fields just as it
can overlay profiles; there is no registration-only backend namespace.
When configuration or message files change, build and validate a new engine,
then switch the application to that engine for subsequent work. Existing
engines and prepared handles retain their captured definitions; schema and
request artifact resources still follow their runtime lifecycle. Inspection
does not watch or reload sources automatically.
## Inspect The Captured Configuration
Use [`Engine.InspectConfiguration`](../../configuration_inspection.go) for an
operator-selected report of effective configuration, source descriptions,
diagnostics, and path history. For a literal ID containing a slash, construct
the semantic path with `configkit.Keys`:
```go
view, err := engine.InspectConfiguration()
if err != nil { return err }
path := configkit.Keys("profiles", "team/summary", "model")
explanation, found := view.Explain(path)
if found && explanation.Effective.Redacted {
// Apply the application's disclosure policy before sharing the report.
}
```
`WriteEffective` and `WriteExplanation` produce ConfigKit-redacted reports;
the application decides whether those reports may be disclosed. Credential-
shaped provider keys are redacted, but arbitrary prompt and provider content
is not necessarily safe to share. Inspection uses the engine's captured result
and does not reopen sources or resolve credentials. `InspectProfile` instead
shows backend-completed execution settings for one profile, and `Prepare` or
`PreparedExecution.Details` shows request-specific rendered messages, inputs,
and validation resources. None of those views is a substitute for the others.
## Inspect A Prompt Before Preparation
@@ -268,8 +328,8 @@ result, err := engine.Run(ctx, request)
These messages are already rendered: Promptkit does not template or resolve
files in them, and they can contain sensitive model output or application
feedback. Promptkit remains stateless; every `Run` call re-resolves its current
sources and the application owns any semantic retry budget. When a
feedback. Every `Run` call uses the engine's captured configuration; the
application owns any semantic retry budget. When a
pre-execution equality check is required, use `PrepareExecution` and compare
its opaque rendered-prompt hash before invoking `RunPrepared`.
@@ -282,12 +342,44 @@ replace execution settings or the complete output contract.
The [public value GoDoc](../../types.go) defines nil, empty, zero, replacement,
copy, and credential behavior. The
[framework format reference](../formats.md) defines how those request values
interact with prompt definitions, file-backed and application fallback
profiles, built-ins, schemas, and framework defaults.
interact with prompt definitions, layered profiles, built-ins, schemas, and
framework defaults.
For programmatic profiles,
[`OpenAICompatibleProfile`](../../profiles.go) converts ordinary
OpenAI-compatible settings into a value accepted by `WithProfiles`.
For programmatic profiles, use [`ProfileSource`](../../profiles.go) with keyed
`ProfileConfig` values. Nil fields are omitted; pointers preserve explicit
zero, false, and blank values. Raw `configkit.Mapping` expresses explicit null.
### Migrate From The Previous Configuration API
Replace `Config.PromptDir` and `Config.ProfileDir` with selected ConfigKit
inputs in `Config.Sources`. The removed `WithPromptFS`, `WithPromptFile`,
`WithProfileFS`, `WithProfileFile`, `WithFallbackProfileFS`, `WithProfiles`,
and `WithBackend` options have no compatibility wrappers. Use
`DirectoryInputs` or `FilesystemInputs` for a strict-composition file group,
ConfigKit file inputs for individual files, and `ProfileSource`,
`BackendSource`, or `configkit.Mapping` for programmatic values. Backend IDs
are keys in configuration, not registrations. Choose caller underlays in
fallback-to-operator-to-programmatic order; use `Sources.Main` for one final
highest-priority input.
Move embedded `id` and `version` fields into keyed `profiles` and `prompts`
maps; convert `base_profile` to ConfigKit `extends`. IDs and references are
exact literal keys, without trimming. A supplied zero, false, or empty string
is now distinguishable from omission; null can clear or suppress an inherited
field according to the [field table](../formats.md#defaults-and-overrides).
Maps merge recursively across inheritance and layers. `{}` does not clear an
inherited map: use null to clear the whole map. At request time, a literal
`extra_params["extends"]` is ordinary provider data, not an inheritance
instruction; a nonempty request ExtraParams map replaces configured params.
Changing a model or backend does not clear unrelated explicit fields, though
backend-derived defaults are applied after inheritance to the final backend.
Invalid final definitions, including unused ones, now fail `NewEngine`; only
request-dependent failures remain at preparation or execution. Prompt text
and static templates are captured at construction, not automatically reloaded
between runs. Replace the engine to pick up changed definitions or message
files. The [format reference](../formats.md) owns exact field and presence
rules; the maintained examples show complete runnable workflows.
### Choose A Profile Ownership Model
@@ -319,18 +411,19 @@ the application ID while inheriting the built-in target. The child can override
only the setting it owns:
```go
promptkit.WithProfiles(promptkit.Profile{
ID: "weather-light",
BaseProfileID: "promptkit-openrouter-standard",
ReasoningEffort: "high",
profile := promptkit.ProfileSource("application profiles", map[string]promptkit.ProfileConfig{
"weather-light": {
Extends: &standardID,
ReasoningEffort: &high,
},
})
```
Select `weather-light` in a prompt or `RunRequest.ProfileID`; it remains the
reported selected profile. See the [profile inheritance format
reference](../formats.md#profile-inheritance) and the
[`Profile` GoDoc](../../types.go) for exact lookup, merging, and validation
behavior. If the base is an external profile, apply the volatility guidance
Place `profile` in a ConfigKit underlay, with `standardID` set to
`"promptkit-openrouter-standard"` and `high` to `"high"`. Select
`weather-light` in a prompt or `RunRequest.ProfileID`; it remains the
reported selected profile. Definition IDs and inheritance references are exact
keys. If the base is an external profile, apply the volatility guidance
above when deciding whether to use an application-owned complete definition
instead.
@@ -355,9 +448,8 @@ if prepared.SelectedBackendID != promptkit.BackendRakestrawHome {
}
```
Do not register `rakestrawhome` manually. When adopting this built-in, remove
an existing `WithBackend` registration with that exact ID; retaining it causes
the intentional duplicate-ID configuration error. Direct request credentials
The maintained catalog supplies this backend, and callers may overlay its
effective configuration through a later ConfigKit source. Direct request credentials
and runtime endpoint overrides remain supported under their ordinary GoDoc and
format contracts.
@@ -424,19 +516,15 @@ Choose the smallest configuration that fits how the endpoint will be reused.
#### Use An Endpoint-Only Profile
Put the endpoint directly on an in-memory profile when only that profile needs
Put the endpoint directly on a programmatic profile when only that profile needs
it and shared backend identity or capacity policy is unnecessary:
```go
engine, err := promptkit.NewEngine(promptkit.Config{
PromptDir: "prompts",
},
promptkit.WithProfiles(promptkit.Profile{
ID: "local-summary",
Endpoint: "http://localhost:8000/v1",
Model: "example-model",
}),
)
endpoint, model := "http://localhost:8000/v1", "example-model"
profile := promptkit.ProfileSource("local profile", map[string]promptkit.ProfileConfig{
"local-summary": {Endpoint: &endpoint, Model: &model},
})
// Add configkit.Input{Source: profile} to a caller underlay.
```
Endpoint-only profiles have an empty backend ID and remain unrestricted by
@@ -448,59 +536,44 @@ Use `LocalBackend` when profiles should share the conventional `local`
identity, endpoint, and concurrency limit:
```go
engine, err := promptkit.NewEngine(promptkit.Config{
PromptDir: "prompts",
},
promptkit.WithBackend(
promptkit.LocalBackend("http://localhost:8000/v1", 2),
),
promptkit.WithProfiles(promptkit.Profile{
ID: "local-summary",
BackendID: promptkit.BackendLocal,
Model: "example-model",
}),
)
backendID, model := promptkit.BackendLocal, "example-model"
backend := promptkit.BackendSource("local backend", map[string]promptkit.BackendConfig{
backendID: promptkit.LocalBackend("http://localhost:8000/v1", 2),
})
profile := promptkit.ProfileSource("local profile", map[string]promptkit.ProfileConfig{
"local-summary": {Backend: &backendID, Model: &model},
})
// Add both sources as inputs in a caller underlay.
```
The helper is explicit: it does not pre-register a backend or read environment
variables. Supplying a positive limit leaves queue capacity omitted, so normal
backend registration selects the existing default waiting capacity of 1024.
The returned value still enters the engine through `WithBackend`.
The helper does not choose an ID or read environment variables. A positive
concurrency limit with omitted queue capacity selects the default waiting
capacity of 1024.
#### Configure A Complete Backend
Use a keyed `Backend` value for authentication, extra request parameters, an
Use a keyed `BackendConfig` value for authentication, extra request parameters, an
explicit queue capacity, a custom ID, or multiple local endpoints:
```go
noWaiting := 0
engine, err := promptkit.NewEngine(promptkit.Config{
PromptDir: "prompts",
},
promptkit.WithBackend(promptkit.Backend{
ID: "local-gpu",
Endpoint: "http://gpu-host:8000/v1",
APIKeyEnv: "LOCAL_GPU_API_KEY",
ExtraParams: map[string]any{"provider_option": "enabled"},
ConcurrencyLimit: 2,
QueueCapacity: &noWaiting,
}),
promptkit.WithProfiles(promptkit.Profile{
ID: "gpu-summary",
BackendID: "local-gpu",
Model: "example-model",
}),
)
endpoint, keyName, limit, noWaiting := "http://gpu-host:8000/v1", "LOCAL_GPU_API_KEY", 2, 0
backend := promptkit.BackendSource("GPU backend", map[string]promptkit.BackendConfig{
"local-gpu": {
Endpoint: &endpoint, APIKeyEnv: &keyName,
ConcurrencyLimit: &limit, QueueCapacity: &noWaiting,
ExtraParams: map[string]any{"provider_option": "enabled"},
},
})
```
Use distinct custom IDs when registering multiple local endpoints.
Registrations belong to one engine and custom IDs cannot replace built-ins.
The [`Backend`, `LocalBackend`, and `WithBackend` GoDoc](../../backends.go)
defines exact construction, validation, copying, uniqueness, concurrency, and
request-default behavior.
Place `backend` in a caller underlay and select `"local-gpu"` from a
`ProfileConfig.Backend` pointer or a keyed file profile. All backend
contributions use the same ConfigKit layering rules, including overlays of
maintained definitions. The [`BackendConfig` and `LocalBackend` GoDoc](../../backends.go)
defines exact omission and construction behavior.
Both file-backed and in-memory profiles select a registration through
`backend` or `Profile.BackendID`. Profile and request endpoint overrides retain
Both file-backed and programmatic profiles select a backend through
`backend` or `ProfileConfig.Backend`. Profile and request endpoint overrides retain
that routing and capacity identity. `PreparedRun.SelectedBackendID`,
`RunResult.SelectedBackendID`, and the effective `ExecutionTarget.BackendID`
expose it to consumers and injected model clients. Endpoint-only profiles
@@ -508,7 +581,7 @@ remain supported and expose an empty backend ID.
### Limit Backend Concurrency
Set `Backend.ConcurrencyLimit` when a backend needs protection from too many
Set `BackendConfig.ConcurrencyLimit` when a backend needs protection from too many
simultaneous model calls. Leaving `QueueCapacity` nil, as in the local-backend
example above, selects the default waiting capacity of 1024.
@@ -517,24 +590,23 @@ zero:
```go
noWaiting := 0
backend := promptkit.Backend{
ID: "local-gpu",
Endpoint: "http://gpu-host:8000/v1",
ConcurrencyLimit: 2,
QueueCapacity: &noWaiting,
backend := promptkit.BackendConfig{
Endpoint: &endpoint,
ConcurrencyLimit: &limit,
QueueCapacity: &noWaiting,
}
```
The pointer distinguishes an explicit zero from omission. Keep using keyed
`Backend` literals so additive configuration fields remain source-compatible.
The pointer distinguishes an explicit zero from omission. Use keyed
`BackendConfig` literals inside `BackendSource` for programmatic definitions.
Capacity belongs to one engine and the selected backend ID; endpoint-only
profiles and custom backends without a configured limit remain unrestricted.
Exact validation, defaulting, ownership, and concurrency semantics belong to
the [`Backend` GoDoc](../../backends.go).
the [`BackendConfig` GoDoc](../../backends.go).
## Credentials
File-backed profiles name an environment variable; in-memory profiles can
File-backed and programmatic profiles can name an environment variable or
require a direct request key. Direct keys are request-scoped and are excluded
from supported JSON values and the package's `String` and `GoString`
summaries. The exact precedence and redaction guarantees belong to
@@ -568,12 +640,13 @@ the application policy that motivated the injection.
Use `errors.Is` with the
[public error sentinels and operation GoDoc](../../engine.go). The declarations
distinguish invalid construction, invalid requests, absent sources,
source-loading failures, collaborator failures, and operational validation
distinguish invalid construction, invalid requests, absent selections,
collaborator failures, and operational validation
failures. Specific request conditions may also match the broader
`ErrInvalidRequest`, and injected collaborator identities are preserved where
documented. Invalid or duplicate backend registrations match
`ErrInvalidConfig`; selecting an unknown backend matches `ErrProfileLoad`.
documented. Invalid effective definitions, including unknown backend
references, match `ErrInvalidConfig` during construction. ConfigKit load
errors remain available through `errors.As`.
When a limited backend has admitted all active and waiting calls, handle
`ErrCapacityExceeded` separately from request errors and provider failures:

View File

@@ -1,9 +1,7 @@
# Development
This is the contributor entry point for Promptkit, a reusable Go library. All
contributors must read the
[architecture policy](policy/architecture.md) before making changes. Agents must
also follow the [agent policy](policy/agents.md) before recommendations or work.
This is the contributor entry point for Promptkit, a reusable Go library.
Use the Policy Reading Guide below to select policies applicable to the task.
## Initial Orientation
@@ -16,18 +14,27 @@ Before starting work:
4. keep documentation limited to implemented behavior unless an accepted
decision or temporary roadmap explicitly owns future work.
Start with:
## Policy Reading Guide
- the [architecture policy](policy/architecture.md) for library boundaries,
dependency direction, invariants, and non-goals;
- the [internal component overview](internal/overview.md) for the current
package and component inventory;
- the [documentation policy](policy/documentation.md) before changing
documentation;
- the [testing policy](policy/testing.md) before adding, rewriting, or deleting
tests; and
- the [release policy](policy/release.md) and [release procedure](release.md)
for version and publication work.
| Policy | Read before… |
| --- | --- |
| [Policy set provenance](policy/meta.md) | Adopting policies, checking upstream revisions, or updating the adopted policy set. |
| [Agents](policy/agents.md) | Recommending, planning, or implementing repository changes. |
| [Architecture](policy/architecture.md) | Planning or changing library code or its structure. |
| [Testing](policy/testing.md) | Selecting validation for code changes, or adding, changing, removing, or evaluating tests. |
| [Documentation](policy/documentation.md) | Writing, revising, or reorganizing documentation. |
| [OSS](policy/oss.md) | Adding or updating dependencies or incorporated third-party material, or changing licensing or attribution. |
| [Release](policy/release.md) | Preparing or publishing a release, changing release procedures, or making versioning and compatibility decisions. |
- Read applicable policies before making decisions they govern; multiple policies
may apply.
- Select policies using the task triggers above. Follow cross-references to
relevant sections when needed to resolve a concrete question.
- Reuse policies already available in context; reread when changed or no longer
available.
- Expand reading when the task's scope expands.
- When revising a policy itself, read that policy and any provisions elsewhere
affected by the revision.
## Task-Specific Reading Guide

View File

@@ -2,394 +2,309 @@
## Purpose
This document is the canonical contract for Promptkit prompt-definition,
profile, and schema files. The [Go package consumer guide](consumers/pkg-promptkit.md)
explains how to select these sources and invoke the engine. The
This is the canonical reference for Promptkit's backend, profile, prompt, and
schema definitions. The [Go consumer guide](consumers/pkg-promptkit.md) shows
how to select sources and use an engine; the
[OpenAI-compatible integration contract](integrations/openai-compatible-chat.md)
owns the resulting outbound wire behavior.
owns outbound wire behavior. [ConfigKit](https://gitea.maximumdirect.net/eric/ConfigKit)
owns the generic YAML grammar, imports, layering, inheritance, and diagnostics.
This reference describes what Promptkit does with the final effective values.
Prompt and profile sources recursively discover files ending in `.yaml` or
`.yml`. Each prompt-definition and profile file contains exactly one YAML
document; comments and trailing whitespace are allowed. YAML decoding is
strict: unknown fields are errors for the selected definition. Definitions are
selected by their YAML `id`, not their file name or directory.
A configuration document has keyed top-level `backends`, `profiles`, and
`prompts` mappings. Their keys are literal IDs, not filename-derived identities.
Prompt versions are a second keyed mapping. Keys and references must be
nonblank and remain exact: Promptkit does not trim or re-key them. Old embedded
`id`, `version`, and `base_profile` fields are invalid. Unknown fields are
errors.
```yaml
backends:
local:
endpoint: http://localhost:8000/v1
concurrency_limit: 2
profiles:
local-summary:
backend: local
model: example-model
prompts:
meeting.summary:
"1.0.0":
default_profile: local-summary
messages:
- role: user
content: Summarize this note.
output:
format: text
validation_mode: none
```
Every effective backend, profile, and prompt definition is validated when
`NewEngine` constructs the engine, even if it is never selected. A profile may
be partial: an endpoint and model need only be complete after request overrides
are applied. A prompt's default profile similarly need not be runnable without
an overriding request. Missing request data, artifacts, schemas, credentials,
and provider work remain runtime concerns.
The engine captures effective definitions, referenced message-file contents,
and static template syntax at construction. It does not reread configuration or
message files on ordinary runs; construct a new engine to observe edits.
Schema and request artifact reads still happen during preparation. Use
`InspectConfiguration` for an explicit effective-configuration or provenance
report, `InspectProfile` for backend-completed profile settings, and preparation
for request-specific rendered details.
## Prompt Definitions
A prompt definition describes inputs, Go-template messages, an optional
default profile, and an output contract.
The `prompts` mapping selects a literal prompt ID, then a string version key:
```yaml
id: meeting.summary
version: "1.0.0"
default_profile: local-summary
description: Summarize a synthetic meeting note.
session_id: '{{.session}}'
inputs:
- name: note
required: true
content_type: text/plain
description: Meeting note to summarize.
messages:
- role: system
content: Return a concise summary.
cache_control:
type: ephemeral
ttl: 1h
- role: user
content_file: ./summary.user.md
output:
format: markdown
validation_mode: basic
repair_attempts: 0
prompts:
meeting.summary:
"1.0.0":
default_profile: local-summary
description: Summarize a synthetic meeting note.
session_id: '{{.session}}'
inputs:
- name: note
required: true
content_type: text/plain
description: Meeting note to summarize.
messages:
- role: system
content: Return a concise summary.
cache_control:
type: ephemeral
ttl: 1h
- role: user
content_file: ./summary.user.md
output:
format: markdown
validation_mode: basic
repair_attempts: 0
```
| Field | Required | Meaning |
| --- | --- | --- |
| `id` | yes | Non-empty prompt identifier used by `RunRequest.PromptID`. |
| `version` | yes | Non-empty version selected by an optional `RunRequest.PromptVersion`. |
| `default_profile` | no | Non-empty profile ID used when the request omits `ProfileID`. |
| `description` | no | Human-readable description. |
| `session_id` | no | Go template rendered from request variables and input helpers. |
| `inputs` | no | Declared input metadata. |
| `messages` | yes | One or more chat-message templates. |
| `output` | yes | Output format and validation settings. |
When a request omits a version, the selected prompt ID must identify exactly
one definition. When it supplies a version, the ID and version pair must be
unique.
Exact prompt inspection uses this same configured source, strict decoding,
referenced content-file resolution, and ID/version selection. It reports the
selected definition's declared metadata without changing the prompt format or
executing the definition.
`messages` must be nonempty and `output` must be a valid mapping. Optional
`default_profile`, `description`, `session_id`, `inputs`, and cache-control
fields may be absent or cleared with null where appropriate. A null whole
definition, required role, output format, or validation mode is invalid.
`required` and `repair_attempts` default to false and zero, respectively, and
are not nullable. A request that omits a prompt version succeeds only if that
prompt ID has exactly one effective version. `InspectPrompt` selects from the
same captured definitions without rendering or resolving a profile.
### Inputs
Each `inputs` item has these fields:
| Field | Required | Meaning |
| --- | --- | --- |
| `name` | yes | Non-empty name used by the request input map and `input` template helper. Names must be unique. |
| `required` | no | When true, preparation fails if the request omits the input. The default is false. |
| `content_type` | no | Expected media-type metadata. |
| `description` | no | Human-readable input description. |
Requests supply inputs as inline or file-backed `ArtifactRef` values. Declared
required inputs must be present. A template reference also requires the named
input to exist, whether or not it was declared. Extra request inputs are
allowed.
Each `inputs` item may contain `name`, `required`, `content_type`, and
`description`. `name` is required, nonblank, and unique within the prompt.
Omitted `required` means false. Null `inputs` means no declarations. Requests
supply inline or file-backed `ArtifactRef` values; declared required inputs
must be present. A template reference also needs its named input, even when
undeclared. Extra request inputs are allowed.
### Messages And Templates
Each message has a `role` that Promptkit trims and lowercases. It must then be
exactly one of `developer`, `system`, `user`, or `assistant`; blank, custom,
`tool`, and `function` roles are invalid. This intentionally tightens the
previous nonblank-string rule. Consumers migrating to the next minor release
must update any nonstandard prompt-definition roles before upgrading.
Each message needs a role and exactly one nonnull `content` or `content_file`.
An explicitly empty inline `content` is valid. Null clears either content
selection. Roles are trimmed and lowercased, then must be `developer`,
`system`, `user`, or `assistant`. Static template syntax is checked during
construction; missing variables and named inputs fail when rendered.
Request variables are template data (`{{.audience}}`), and
`{{input "note"}}` renders an input body.
Each message also has exactly one of:
`content_file` uses the effective field's own source context, including when
inherited or imported. Its relative path is resolved against that source's
base directory within its rooted filesystem. Absolute paths and lexical
escapes from the selected root fail. Host-file sources also prevent symlink
escapes. Promptkit preserves the path text, including meaningful whitespace,
after checking that it is not blank. A source without a unique usable resource
context must supply one explicitly or override the field. Content is captured
at construction, not reopened for each request.
- `content`, containing an inline Go template; or
- `content_file`, naming a file whose contents are the Go template.
`content_file` must be a relative path. It resolves from the directory that
contains the prompt file and must remain within the configured prompt source
root; parent components are allowed only when the resolved target remains
inside that root. Absolute paths and paths that escape the root are rejected.
Operating-system directory and single-file sources also reject symlink targets
outside the root, while injected `fs.FS` sources apply containment in that
filesystem's relative path namespace. For `WithPromptFile`, the source root is
the directory containing the selected prompt file. Promptkit uses the parsed
path text exactly after checking separately that it is not blank, so leading
and trailing whitespace can name real filesystem entries.
Request variables are the template data, so a variable named `audience` is
referenced as `{{.audience}}`. The `{{input "note"}}` helper renders the body
of a named input. Missing variables and input references are errors.
The optional `session_id` uses the same template data and input helper. Its
rendered value is trimmed, omitted when empty, and limited to 256 Unicode code
points. A nonblank direct request session ID bypasses this template completely;
a blank direct value leaves the template behavior unchanged.
Optional `session_id` uses the same template variables and input helper. Its
rendered value is trimmed, omitted if empty, and limited to 256 Unicode code
points. A nonblank direct request session ID bypasses the template.
### Cache Control
`cache_control` is optional and has these fields:
| Field | Required | Values |
| --- | --- | --- |
| `type` | yes | `ephemeral` |
| `ttl` | no | Empty or `1h` |
Promptkit preserves cache-control metadata on the rendered message. The
outbound integration determines its wire representation.
Optional `cache_control` has required `type: ephemeral` and optional `ttl`,
which may be empty or `1h`. Promptkit retains this metadata on rendered
messages; the integration contract defines its wire representation.
### Output Contract
| Field | Required | Values or behavior |
| --- | --- | --- |
| `format` | yes | `text`, `markdown`, or `json`. |
| `validation_mode` | yes | `none`, `basic`, `json`, or `json_schema`. |
| `schema_path` | for `json_schema` | Path to a schema in the configured schema source. |
| `repair_attempts` | no | Integer from zero through three; omitted means zero. A positive value requires `basic`, `json`, or `json_schema` validation. |
| `format` | yes | `text`, `markdown`, or `json` |
| `validation_mode` | yes | `none`, `basic`, `json`, or `json_schema` |
| `schema_path` | for `json_schema` | Path in the selected schema source |
| `repair_attempts` | no | Integer 0–3; default 0. Positive values need `basic`, `json`, or `json_schema` |
The validation modes behave as follows:
- `none` skips content validation;
- `basic` requires non-empty generated content;
- `json` requires valid JSON; and
- `json_schema` requires valid JSON that satisfies the selected schema.
`format` controls output artifact metadata. JSON Schema mode also supplies the
schema to compatible model clients as structured-output metadata. Plain `json`
validation accepts every valid JSON value and does not request a provider-native
JSON-object constraint.
`repair_attempts` counts additional generation calls after a failed validation.
Zero is single-pass. With a positive eligible budget, Promptkit stops at the
first valid candidate. If the budget is exhausted, it returns the final
candidate and its complete failed validation result; generation and operational
validation failures remain errors. `none` never permits repair.
A request-level `OutputContract` replaces the complete prompt output contract.
It does not merge individual fields. If its format is empty, Promptkit uses
`text`.
`none` skips content validation; `basic` requires nonempty content; `json`
accepts any valid JSON value; `json_schema` checks the selected schema.
`format` determines output artifact metadata. JSON Schema mode also supplies
structured-output metadata to compatible clients. Plain `json` does not
request a provider-native JSON-object constraint. A positive repair budget
counts additional generations after failed validation; exhaustion returns the
last candidate and its failed validation result. Operational errors remain
errors. A request-level `OutputContract` replaces the whole configured one,
not individual fields; an empty request format defaults to `text`.
## Built-In Backends
Every engine provides these reserved OpenAI-compatible backend IDs. Consumers
must not register either ID with `WithBackend`; exact registration and
reservation behavior belongs to the [`Backend` GoDoc](../backends.go).
The selected catalog modules contribute these backend definitions as the
earliest underlay. Their settings can be overlaid by caller sources; these IDs
are not reserved against configuration changes.
| ID | Base endpoint | API-key environment variable | Active generation limit | Default queue capacity |
| --- | --- | --- | ---: | ---: |
| `openrouter` | `https://openrouter.ai/api/v1` | `OPENROUTER_API_KEY` | 16 | 1024 |
| `rakestrawhome` | `https://inference.ai.rakestrawhome.com/v1` | `RAKESTRAWHOME_INFERENCE_API_KEY` | 4 | 1024 |
| ID | Catalog endpoint | API-key environment name | Active generation limit |
| --- | --- | --- | ---: |
| `openrouter` | `https://openrouter.ai/api/v1` | `OPENROUTER_API_KEY` | 16 |
| `rakestrawhome` | `https://inference.ai.rakestrawhome.com/v1` | `RAKESTRAWHOME_INFERENCE_API_KEY` | 4 |
A backend definition may have `endpoint`, `api_key_env`,
`concurrency_limit`, `queue_capacity`, and `extra_params`. The final endpoint
is required and must be an absolute HTTP(S) URL with a host, no user
information, query, or fragment. An omitted or null API-key environment name
means no lookup. Omitted/null concurrency is zero (unlimited); a supplied
negative value is invalid. For positive concurrency, omitted/null queue
capacity defaults to 1024. An explicit queue capacity, including zero,
requires positive concurrency and must be nonnegative. Unlimited concurrency
has no queue. Provider maps contain JSON-compatible values; null entries are
ordinary JSON null. Unknown fields, including a raw `api_key`, fail.
## Profile Definitions
A profile supplies model execution settings:
Profiles are keyed by literal ID and may inherit a named parent with
`extends`:
```yaml
id: local-summary
backend: openrouter
model: example-model
temperature: 0.2
max_tokens: 500
top_p: 0.95
timeout_seconds: 90
service_tier: flex
reasoning_effort: medium
extra_params:
provider_option: enabled
profiles:
local-summary:
backend: local
model: example-model
temperature: 0.2
max_tokens: 500
timeout_seconds: 90
extra_params:
provider_option: enabled
local-summary-fast:
extends: local-summary
timeout_seconds: 30
reasoning_effort: low
```
A derived profile can use a named base and override only the settings it owns:
```yaml
id: local-summary-fast
base_profile: local-summary
timeout_seconds: 30
reasoning_effort: low
```
| Field | Required | Meaning |
| --- | --- | --- |
| `id` | yes | Profile identifier, trimmed before selection and publication. It must be non-empty after trimming and unique within one source after normalization. |
| `base_profile` | no | One optional parent profile ID. A derived profile may inherit target fields from it. |
| `backend` | unless `endpoint` is present | Backend registry ID. It is trimmed and registry membership is checked when the profile is prepared or inspected. |
| `endpoint` | unless `backend` is present | OpenAI-compatible base URL, including an API version path when required. A nonempty value is trimmed and must be absolute HTTP or HTTPS with a host and without user information, a query, or a fragment. When both connection fields are present, this overrides the backend endpoint without changing backend identity. |
| `model` | yes | Non-empty provider model name. |
| `temperature` | no | Number from 0 through 2. |
| `max_tokens` | no | Integer zero or greater. |
| `top_p` | no | Number from 0 through 1. |
| `timeout_seconds` | no | Per-generation deadline in whole seconds; integer zero or greater. |
| `service_tier` | no | Provider-specific request tier. |
| `reasoning_effort` | no | Provider-specific reasoning setting. |
| `api_key_env` | no | Optional environment-variable lookup source for an API key. |
| `extra_params` | no | JSON-compatible provider-specific outbound fields. |
Raw `api_key` is prohibited in profile YAML. Store only an environment
variable name in `api_key_env`.
A standalone profile must provide a model and at least one of `backend` or
`endpoint`. A derived profile may omit those target fields because its selected
base chain can provide them. Local parsing still validates a derived profile's
own ID, supplied endpoint, execution-setting bounds, and `extra_params`.
Promptkit does not infer a backend from a model or endpoint. Endpoint-only
profiles remain supported and have no effective backend ID.
The engine always provides the built-in `openrouter` and `rakestrawhome` IDs.
Consumers can add engine-scoped IDs with
[`WithBackend`](../backends.go); exact registration validation belongs to its
GoDoc.
`extra_params` accepts null, booleans, finite numbers, strings, arrays, and
objects with string keys. Keys must be non-empty. With the built-in client,
they also cannot collide with the standard fields listed in the
[outbound request contract](integrations/openai-compatible-chat.md#request-body).
Excessively deep or large JSON-shaped values are rejected for safety.
Profiles may also supply `endpoint`, `top_p`, `service_tier`, `api_key_env`,
and `api_key_required`. Promptkit does not infer a backend from an endpoint or
model. A profile with only an endpoint has no backend capacity identity.
Nonblank endpoints follow the backend URL rules; temperature is 0–2, top_p
is 0–1, max_tokens and timeout_seconds are nonnegative. Provider maps accept
JSON-compatible values with nonempty keys and cannot collide with the built-in
client's [standard request fields](integrations/openai-compatible-chat.md#request-body).
Credential values do not belong in profiles: raw `api_key` is invalid.
### Defaults And Overrides
Execution settings resolve in this order:
After effective profile inheritance, Promptkit applies the selected backend's
defaults once, then request `ExecutionTargetOverride` values. Backend-derived
settings are not inheritance input: changing a child profile's backend does
not carry the parent's derived endpoint or credential source into it. Changing
only a model or backend does not clear unrelated explicitly inherited fields.
1. the framework timeout baseline;
2. the selected backend, when the profile names one;
3. the selected profile; and
4. request `ExecutionTargetOverride` values.
| Profile field | Absent | Explicit value | Explicit null |
| --- | --- | --- | --- |
| `backend` | No backend | Exact backend ID; blank means none | No backend |
| `endpoint` | Selected backend endpoint | Valid nonblank URL; empty clears | Clear, without backend fallback |
| `model` | Unspecified | Nonblank model; empty clears | Unspecified |
| `temperature`, `top_p`, `max_tokens` | Unspecified | Valid value, including outbound zero | Suppress outbound control |
| `service_tier`, `reasoning_effort` | Unspecified | Normalized value; blank suppresses | Suppress outbound control |
| `timeout_seconds` | 600 seconds | Nonnegative; zero disables generation deadline | Reset to 600 seconds |
| `api_key_env` | Backend lookup name, if any | Trimmed name; blank disables | Disable backend lookup |
| `api_key_required` | False | Boolean, including false | False |
| `extra_params` | Backend defaults | Recursively overlay profile map on backend map | Suppress configured provider map |
The framework baseline is:
| Setting | Default |
| --- | --- |
| `temperature` | Unspecified and omitted from compatible provider requests unless a profile or runtime override selects it. |
| `max_tokens` | Unspecified and omitted from compatible provider requests unless a profile or runtime override selects it. |
| `top_p` | Unspecified and omitted from compatible provider requests unless a profile or runtime override selects it. |
| `timeout_seconds` | `600` |
Numeric zero in a file or in-memory profile does not select a numeric value.
For `temperature`, `max_tokens`, and `top_p`, it leaves the provider control
unspecified. For `timeout_seconds`, it retains the framework deadline. Numeric
request overrides use pointers, so an explicit zero is retained and sent to
compatible providers. In particular, an explicit request `timeout_seconds` of
zero disables the per-generation deadline while leaving the caller context and
transport timeout intact.
Non-empty profile strings replace backend defaults, and non-empty request
strings replace both. Request reasoning is the exception: a nil
`ReasoningEffort` pointer inherits the profile, a pointer to a nonblank string
trims and replaces it, and a pointer to a blank string clears it. Backend
identity is retained when either layer overrides the endpoint, so the override
also retains any engine-local capacity policy configured for that backend.
Capacity configuration belongs to the Go
[`Backend` API](../backends.go), not prompt or profile YAML. A non-empty
`extra_params` map at each layer replaces the entire lower-precedence map
rather than merging keys.
The [outbound integration contract](integrations/openai-compatible-chat.md)
defines how the effective settings are serialized.
An empty mapping does not clear inherited map entries; null clears the whole
map. Nested maps merge recursively, while scalar and array values replace.
Runtime nonempty `ExtraParams` overrides replace the entire configured map.
Request overrides otherwise follow the [public value GoDoc](../types.go).
For example, a runtime `extra_params["extends"]` key is ordinary provider
data, not a configuration inheritance instruction. An explicit zero timeout
does not disable the caller context or HTTP transport timeout.
### Source And Profile Precedence
An explicit request profile ID takes precedence over the prompt's
`default_profile`. If neither is present, preparation fails. Exact profile
inspection instead takes one explicit profile ID and does not use a prompt
default.
`NewEngine` composes catalog definitions first, Promptkit-owned definitions
next, then caller underlays in their given order, with an optional main input
last. Later tiers overlay earlier ones. Inputs within one underlay group are
strictly composed: conflicting contributions are errors, not priority
signals. The directory helpers discover `.yaml` and `.yml` files recursively
in sorted order, but that order does not give one file higher precedence.
An explicitly selected missing/unreadable root fails; an empty root contributes
nothing. Imports are disabled by these helpers unless the caller enables them
on an input. Single files can use ConfigKit inputs directly.
Profile sources resolve matching IDs in this order:
1. in-memory profiles supplied with `WithProfiles`;
2. the ordinary configured source selected by a profile file, `fs.FS`, or
configured profile directory;
3. application fallback profiles supplied with `WithFallbackProfileFS`; and
4. Promptkit-owned profiles; and
5. maintained external catalog profiles.
A profile source supplies a complete definition; definitions and their fields
are not merged across sources. A higher-precedence source falls back only when
the requested profile ID is absent. An invalid matching profile is an error and
does not fall back. In-memory `Profile` values follow the same ranges as YAML
profiles. They use `APIKeyRequired` for request-scoped credentials instead of
`api_key_env`. Preparation and exact profile inspection use this same source
precedence.
When a selected definition names `base_profile`, every profile ID in that
chain is looked up through this same precedence order. A higher-precedence
definition therefore shadows a lower-precedence definition of the same base
ID, including a built-in. References are not source-qualified.
Backends, profiles, and prompts all follow these same source rules. A later
profile or backend can change individual fields of a lower definition. A
request `ProfileID` takes precedence over a prompt `default_profile`; if
neither is selected, preparation fails. Exact profile inspection selects one
explicit ID and does not consult a prompt default.
### Profile Inheritance
Promptkit resolves one linear base chain of at most 32 profiles, including the
selected profile. It merges settings from the root base to the selected leaf.
The leaf's `id` remains the selected profile identity. Nonblank string fields
(`backend`, `endpoint`, `model`, `service_tier`, `reasoning_effort`, and
`api_key_env`) and nonzero numeric fields replace inherited values. A nonempty
`extra_params` map replaces the complete inherited map rather than merging
keys, and `APIKeyRequired: true` remains true through the chain. Backend and
endpoint are independent: replacing one does not clear the other.
There is no profile-level clearing syntax. Blank strings, zero numbers, false,
and empty maps remain unspecified and inherit from a base. Use existing
presence-aware request overrides where an execution needs an explicit zero or
empty reasoning setting.
An absent directly selected profile reports the ordinary not-found error. Once
the selected profile exists, a missing base, cycle, overlong chain, or
incomplete resolved target is a profile-load failure. Ordinary operations
resolve chains afresh; prepared execution retains the fully resolved target.
ConfigKit resolves `extends` on effective profile definitions. One profile
may inherit another keyed profile; presence, explicit null, and recursively
merged maps determine the final fields. References are literal IDs, including
IDs containing punctuation. There is no Promptkit-specific depth limit or
second base-chain resolver. Missing references and cycles fail at construction.
For generic inheritance and layer mechanics, consult
[ConfigKit](https://gitea.maximumdirect.net/eric/ConfigKit).
## Built-In Profile Catalog
Every built-in profile selects one maintained built-in backend and inherits
that backend's connection and credential metadata. Profile files do not repeat
those values. A configured, application fallback, or in-memory profile with
the same profile ID takes precedence.
Promptkit owns two opt-in profiles. The following definitions are current
selections, not automatic defaults:
Promptkit contributes two opt-in definitions; neither is selected by default:
| ID | Current definition |
| --- | --- |
| `promptkit-openrouter-standard` | Inherits `gemini-flash-latest`. |
| `promptkit-openrouter-standard` | Extends the selected OpenRouter catalog's `gemini-flash-latest` profile. |
| `promptkit-openrouter-auto` | Selects backend `openrouter` and model `openrouter/auto`. |
Select either name through an existing request `profile_id` or prompt
`default_profile`. Consumers can replace either definition or its inherited
base through the ordinary profile-source precedence above. The
[release policy](policy/release.md#catalog-profile-compatibility) defines the
cross-version guarantee for these names.
Select one through request `ProfileID` or prompt `default_profile`.
Caller layers may overlay these definitions or their catalog bases. The
[release policy](policy/release.md#catalog-profile-compatibility) owns their
cross-version compatibility boundary.
## External Profile Inventories
The external profile inventory is defined by the selected catalog modules, not
duplicated here. Browse the exact data selected by this Promptkit version:
The selected catalog modules, not this document, own their profile inventories:
- [OpenRouter catalog v2.20260926.0 profiles](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/tag/v2.20260926.0/catalog/profiles)
- [Rakestrawhome catalog v2.20260926.0 profiles](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/tag/v2.20260926.0/catalog/profiles)
- [OpenRouter catalog v3.20260930.0 profiles](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/tag/v3.20260930.0/catalog/profiles)
- [Rakestrawhome catalog v3.20260930.0 profiles](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/tag/v3.20260930.0/catalog/profiles)
External profile entries are volatile across catalog releases. The
[release policy](policy/release.md#catalog-profile-compatibility) owns the precise
compatibility boundary; a selected module version identifies tested local data,
not live provider availability.
These are tested local data, not a promise of live provider availability.
The [release policy](policy/release.md#catalog-profile-compatibility) owns
the compatibility boundary.
## Schemas
Schemas are JSON documents selected by a prompt or request
`schema_path`. For a directory or `fs.FS` source, paths resolve within the
configured source root. Referenced nested schemas resolve relative to the
owning schema document. `WithSchemaFile` exposes one schema, addressed by its
base name.
An unreadable, invalid, or unresolvable schema produces an operational
validation error. Generated content that is valid JSON but does not satisfy the
schema produces a failed validation result.
Schemas are JSON documents selected by a prompt or request `schema_path`.
Directory and `fs.FS` schema paths resolve within the configured root;
referenced nested schemas resolve relative to the owning schema document.
`WithSchemaFile` exposes one schema under its base name. An unreadable,
invalid, or unresolvable schema causes an operational validation error.
Valid JSON that fails the schema instead yields a failed validation result.
## Credentials
Credential values belong at the request or environment boundary, never in
prompt, profile, schema, or example files:
Only environment-variable names belong in backend/profile definitions, never
credential values. A request may supply a direct `APIKey` or an `APIKeyEnv`
override. A direct key takes precedence, followed by a request environment
source, an explicit profile source, and an inherited backend source. A true
profile `api_key_required` suppresses the backend-derived environment source,
but an explicit profile or request environment source remains usable. False
restores ordinary optional-credential behavior. Missing or blank optional
environment values leave the built-in client's Authorization header absent;
required availability is checked at preparation and again during prepared
execution. Injected clients own their credential-resolution behavior.
- a backend or file profile can name an optional environment lookup source
with `APIKeyEnv` or `api_key_env`;
- an in-memory profile may set `APIKeyRequired` as an explicit local
requirement;
- a request can provide a direct `APIKey` or override the optional `APIKeyEnv`
source; and
- a direct request key takes precedence over environment lookup.
After a direct request key, the credential-source precedence is request
`APIKeyEnv`, profile `api_key_env`, then the backend default. An in-memory
profile with `APIKeyRequired` clears an inherited backend environment name and
requires a direct key unless the request explicitly supplies `APIKeyEnv`.
Named environment sources are optional: when the selected source is absent,
empty, or whitespace-only, the built-in client omits the `Authorization`
header and handles the provider response normally. `APIKeyRequired` is the
only explicit local availability requirement. Promptkit validates required
credential availability during preparation and rechecks it when a prepared
execution runs. Injected clients receive resolved source metadata but define
their own credential-resolution behavior.
Direct keys are excluded from JSON results and redacted by public string
formatters. Environment-variable names may appear in prepared metadata, but
their values do not.
Credential values are excluded from JSON results and redacted by public
string formatters. ConfigKit inspection redacts credential-shaped dynamic
provider keys, but does not make arbitrary prompt/provider content safe to
disclose. Applications own report disclosure and secret handling.

View File

@@ -11,27 +11,28 @@ contributor workflow and validation.
| Component | Implemented responsibility | References |
| --- | --- | --- |
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, profile-inspection, and ordinary-run observation values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, and typed capacity and generation error mapping. | [Package GoDoc](../../doc.go), [observations](../../observations.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine facade](../../engine.go) |
| `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
| Root `promptkit` package | Provides the supported engine facade, ConfigKit source helpers, profile and backend mapping conveniences, injection options, public requests and results, explicit inspection, observation values, opaque prepared-execution handles, value conversion, redacted formatting, and typed capacity and generation error mapping. | [Package GoDoc](../../doc.go), [observations](../../observations.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine facade](../../engine.go) |
| `examples/go-library/prepare` | Demonstrates offline preparation and explicit configuration inspection with a prompt file, programmatic profile, and inline input. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, programmatic profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
| `examples/go-library/observe` | Demonstrates offline built-in HTTP retry and synchronous metadata observation across structured proposal and assessment runs using embedded sources, two profiles on one finite backend, and an application decision. It is not a public library package. | [Example program](../../examples/go-library/observe/main.go) |
| `internal/backend` | Constructs each engine's immutable registry from maintained definitions and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |
| `internal/catalog` | Strictly validates imported immutable maintained backend and profile catalog assets before engine assembly uses them. | [Catalog adapter](../../internal/catalog/catalog.go), [internal sources](sources.md#profiles-and-built-ins) |
| `internal/backend` | Holds a defensive, immutable lookup and capacity-policy index for final backend definitions validated by the configuration adapter. | [Backend registry](../../internal/backend/registry.go) |
| `internal/catalog` | Selects released keyed catalog assets and PromptKit-owned profile definitions as ConfigKit underlays; effective validation belongs to the configuration loader. | [Catalog adapter](../../internal/catalog/catalog.go), [internal sources](sources.md#profiles-and-built-ins) |
| `internal/configuration` | Loads keyed backend, profile, and prompt configuration through ConfigKit; records effective field presence, validates final values, captures indexed prompt definitions and referenced message content, constructs backend and presence-aware profile snapshots, and retains an explicit read-only inspection view. | [Typed loader](../../internal/configuration/load.go), [internal sources](sources.md#typed-configuration-loading) |
| `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) |
| `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, profiles, execution targets, rendering, generation, and validation, and owns source-neutral invariants for shared execution settings, OpenAI-compatible base endpoints, session identifiers, and output contracts. Source parsing, required fields, other source-specific normalization, defaulting, and boundary-specific error classification remain with their callers. | [Domain declarations](../../internal/domain/domain.go), [endpoint invariant](../../internal/domain/endpoint.go) |
| `internal/engine` | Assembles external catalogs, Promptkit-owned profiles, repositories, registry, capacity, collaborators, repairer, and the one use-case runner for each engine instance. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#profiles-and-built-ins), [internal runner](runner.md) |
| `internal/engine` | Assembles selected catalog and caller configuration into one snapshot, then constructs capacity, rendering, validation, generation, repair, and the use-case runner for each engine. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#profiles-and-built-ins), [internal runner](runner.md) |
| `internal/generation` | Coordinates frozen built-in HTTP attempts and one-shot injected calls through active backend permits, including bounded status retries, cancellable backoff, and attempt observations. | [Internal generation coordination](generation.md) |
| `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) |
| `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) |
| `internal/jsonvalue` | Validates and deeply copies bounded JSON-compatible extra-parameter and prepared-schema trees while preserving supported concrete value types and rejecting cycles or excessive depth and work. | [JSON values](../../internal/jsonvalue/jsonvalue.go) |
| `internal/promptdef` | Loads strictly decoded, validated prompt definitions from filesystem and `fs.FS` sources, including version selection and contained file-backed message content. | [Framework formats](../formats.md), [prompt-definition repository](../../internal/promptdef/filesystem_repository.go) |
| `internal/profile` | Owns immutable in-memory and filesystem profile repositories, local profile validation, raw-source overlays with error-preserving fallback, and inherited-profile resolution. | [Framework formats](../formats.md), [profile repositories](../../internal/profile/filesystem_repository.go), [internal sources](sources.md#profiles-and-built-ins) |
| `internal/promptdef` | Provides a narrow captured-prompt lookup contract, not-found identity, and execution-ready definition hash. | [Prompt lookup](../../internal/promptdef/repository.go), [framework formats](../formats.md) |
| `internal/profile` | Provides a narrow captured-profile lookup contract and not-found identity. | [Profile lookup](../../internal/profile/repository.go), [framework formats](../formats.md) |
| `internal/prompt` | Renders prompt messages from Go templates with artifact, variable, session, and cache-control data. | [Go-template renderer](../../internal/prompt/go_renderer.go) |
| `internal/artifact` | Resolves ordinary inline and unrestricted caller-selected file references into copied artifacts with metadata and hashes. | [Internal sources and validation](sources.md) |
| `internal/validate` | Validates basic, JSON, and JSON Schema output using operating-system filesystem or `fs.FS` schema sources and creates operation-local validation plans with canonical contained schema resources. | [Framework formats](../formats.md#schemas), [internal sources and validation](sources.md) |
| `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible chat requests from resolved execution targets, including bounded structured non-success response decoding, successful-response decoding, authentication, deadline handling, and ownership of the OpenAI-compatible reserved request-field policy. | [Internal model client](llm.md) |
| `internal/observation` | Owns invocation-local run IDs, presence-aware usage and phase accounting, and synchronous metadata event delivery for ordinary and claimed prepared runs and generation attempts. | [Internal execution observation](observation.md) |
| `internal/usecase` | Resolves prompt definitions and hashes, profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary and single-claim prepared execution across internal sources, rendering, artifact loading, operation-local validation plans, generation, capacity, and bounded repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) |
| `internal/usecase` | Selects captured prompts and profiles, applies request overrides and completion checks, and coordinates ordinary and single-claim prepared execution across rendering, artifact loading, operation-local validation plans, generation, capacity, and bounded repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) |
The internal engine assembles these internal components without exposing their
representations. Consumers depend only on the root facade.

View File

@@ -4,7 +4,7 @@
This document describes Promptkit's implemented internal orchestration. The
[architecture policy](../policy/architecture.md) owns dependency and consumer
boundaries. The [source and validation document](sources.md) owns repository,
boundaries. The [source and validation document](sources.md) owns configuration,
artifact, rendering, and validation behavior, while the
[model-client document](llm.md) owns generation behavior and failure
categories.
@@ -16,11 +16,10 @@ and override semantics consumed by the runner.
## Collaborators
`Runner` coordinates narrow internal interfaces for prompt definitions,
profiles, backend resolution, artifacts, rendering, model generation, and
validation. The internal engine supplies one immutable registry containing the
built-in backend and validated consumer additions, one engine-local run
admitter, and a generation coordinator using the same capacity manager. Validation
`Runner` coordinates narrow internal interfaces for captured prompt definitions
and resolved profiles, artifacts, rendering, model generation, and validation.
The internal engine supplies one engine-local run admitter and a generation
coordinator using the same capacity manager. Validation
plans and provider-facing schema metadata come from the validator's preparation
interface.
The internal engine supplies one default output repairer through the explicit
@@ -35,11 +34,11 @@ The runner has no durable run or session store.
The runner uses one prompt-selection and hashing boundary for ordinary
preparation and exact prompt inspection. Preparation retains its early
request-ID check before direct-session normalization; both operations then use
the configured prompt repository to select one definition, load referenced
message content, and calculate the same prompt hash.
request-ID check before direct-session normalization; both operations select
one captured definition and its construction-time hash. Referenced message
content and static template syntax were captured when the engine was built.
Inspection stops after that structural lookup. It does not parse templates or
Inspection stops after that structural lookup. It does not render templates or
touch profile, artifact, schema, renderer, validator, admission, or model
collaborators. The root [`Engine.InspectPrompt`](../../engine.go) GoDoc owns
the public operation's exact contract.
@@ -49,12 +48,11 @@ the public operation's exact contract.
The runner uses one profile-selection and target-resolution boundary for
ordinary preparation and exact profile inspection. Preparation first selects a
request profile or a prompt default; inspection begins with its required
explicit profile ID. Both then apply the ordinary source precedence, resolve a
named backend, and construct the effective target from framework, backend, and
profile values.
Inspection stops after the resulting endpoint and model are structurally
validated. It does not check credential availability or perform prompt,
explicit profile ID. Both use the captured, presence-aware target constructed
from the final effective profile and backend configuration. Request overrides
apply afterward; preparation checks endpoint and model completeness only after
those overrides. Inspection can report a partial profile without requiring a
runnable target. Neither path checks credential availability or performs prompt,
artifact, schema, rendering, admission, or model-client work. The root
[`Engine.InspectProfile`](../../engine.go) GoDoc owns the public operation's
exact contract.
@@ -66,11 +64,10 @@ where a run can be assigned to its selected backend pool. The resolution phase
performs only the work needed to validate routing and admission:
1. validate the required prompt selection and normalize any direct session ID;
2. load the prompt definition and hash the original definition;
2. select the captured prompt definition and hash;
3. select the request profile or the prompt's default profile;
4. resolve the profile's backend ID, when present;
5. resolve application-neutral defaults, backend defaults, profile values,
and explicit request overrides in that order;
4. retain the profile's selected backend identity, when present;
5. apply explicit request overrides to its captured backend-completed target;
6. validate endpoint, model, numeric overrides, and credential requirements;
7. resolve and validate the effective output contract without loading its
schema; and
@@ -102,26 +99,26 @@ Pointer-based numeric overrides preserve an explicit zero. Invalid negative or
out-of-range values fail as invalid requests. Endpoint overrides do not change
the selected backend identity. Non-empty extra-parameter maps replace whole
lower-precedence maps. A direct API key takes precedence over environment
lookup; otherwise request, profile, and backend environment-variable names
apply in that order. A profile requiring a direct key clears an inherited
lookup; otherwise request, explicit profile, and backend environment-variable
names apply in that order. A profile requiring a direct key suppresses an inherited
backend environment name unless the request supplies its own name. Secret
values remain excluded from serialized metadata.
Reasoning overrides are tri-state: nil inherits the profile, a pointer to a
nonblank string trims and replaces it, and a pointer to a blank string clears
it. A nonblank direct session is normalized before source loading, bypasses
it. A nonblank direct session is normalized before prompt selection, bypasses
the prompt session template, and is applied after ordinary message rendering.
A blank direct value retains prompt-template behavior. The runner clears the
template only on a value copy of the definition, so the definition hash always
describes the original source while the rendered-prompt hash includes the
describes captured effective content while the rendered-prompt hash includes the
effective direct or rendered session and the complete effective message
sequence. The rendered-prompt hash uses a versioned, length-framed SHA-256
encoding of that session, every role and content value, and cache-control
presence and values; its hexadecimal value is opaque.
The registry is read-only after engine construction. Concurrent `Prepare` and
`Run` calls resolve independent defensive backend values and keep all
invocation state local.
The configuration snapshot is read-only after engine construction. Concurrent
`Prepare` and `Run` calls copy their resolved targets and keep all invocation
state local.
## Run Flow
@@ -215,7 +212,7 @@ generation, validation, repair, and cancellation failures.
Other context cancellation propagates through the invoked collaborator and is
classified by the owning operation. In particular, cancellation observed by
validation retains the context identity through the validation error category.
An overlong direct session is an invalid request before source loading, while
An overlong direct session is an invalid request before prompt selection, while
an invalid or overlong prompt session template remains a prompt-render failure.
An unknown selected backend, or a selected backend with no configured resolver,
is classified as a profile-load failure.

View File

@@ -2,117 +2,73 @@
## Purpose
This document describes Promptkit's implemented internal source, artifact,
This document describes Promptkit's implemented configuration, artifact,
rendering, and output-validation behavior. The
[architecture policy](../policy/architecture.md) owns the library boundary and
dependency rules. None of these internal packages is a supported consumer API,
and the internal engine assembles them behind the public facade's source options and values.
The [framework format reference](../formats.md) owns the exact file fields,
validation modes, built-in catalog, and source precedence.
[architecture policy](../policy/architecture.md) owns dependency rules, while
the [framework format reference](../formats.md) owns fields, defaults, and
source semantics. Internal packages are not consumer APIs.
## Typed Configuration Loading
`internal/configuration` is the single adapter from ConfigKit's final
effective configuration to execution-owned definitions. It performs one
layered load, captures effective presence (including explicit null), records
normalization and defaults through resolution, and validates all effective
backend, profile, and prompt values. Replaced lower-layer domain values are
not separately validated. It builds an immutable backend index and
presence-aware profile targets, applying final backend defaults only after
profile inheritance. It retains ConfigKit's result privately for explicit
inspection, not for per-run resolution. The runtime/domain packages do not
depend on ConfigKit.
The root `DirectoryInputs` and `FilesystemInputs` helpers discover sorted YAML
files under an explicit root, provide resource context, and leave imports
disabled. Each returned slice belongs to one strict-composition group; file
order does not confer priority. The engine prepends selected catalog and
Promptkit-owned definitions, then caller underlays and optional main. The
[configuration tests](../../internal/configuration/load_test.go) exercise the
real loader and final-value handoff.
## Prompt Definitions
`internal/promptdef` uses one source-neutral flow for prompt selection and
normalization. That flow scans normalized YAML ID and version metadata,
requires one strictly decoded document per file, classifies errors for the
selected definition, detects duplicates, and normalizes the exact match.
Small operating-system and `fs.FS` adapters own discovery, byte reads, display
paths, content opening, and root containment. Each lookup remains a
point-in-time scan: definitions and catalogs are not cached, and file-backed
message content is opened only for the exact selected candidate.
The configuration adapter indexes each prompt by exact ID and version,
materializes file-backed message content using the exact effective field's
origin and resource context, and checks static message/session template syntax
at construction. It stores captured content, compiled templates, declared
input metadata, and a definition hash. It closes content-file handles before
returning and retains no source capability for public inspection.
Message roles are normalized through the shared domain owner by trimming
Unicode whitespace and lowercasing. Only `developer`, `system`, `user`, and
`assistant` are published; invalid roles remain selected prompt-definition
failures rather than becoming request errors. Cache-control metadata uses the
same shared domain normalization and defensive-copy rule.
Role and cache-control normalization is applied to execution-owned message
copies. The retained ConfigKit configuration keeps effective input message
metadata and its resource origins: recording a sequence-wide normalization in
ConfigKit v0.5.0 would replace origins of unchanged content-file fields.
Operating-system sources enforce containment against canonical roots and
targets so symlinks cannot escape. Injected `fs.FS` sources enforce containment
in their clean relative path namespace. A single-file source uses the selected
prompt file's containing directory as its root. Every content path must be
relative and is opened from its exact parsed text after a separate blank check;
contained parent components and whitespace-bearing names remain valid.
Exact prompt inspection performs one point-in-time lookup through that same
repository and validates referenced message content before returning declared
metadata. It does not parse templates or read profile, input, or schema
sources, and it does not retain the definition for a later execution.
Its package tests own prompt selection, strict decoding, definition validation,
duplicate detection, and source containment:
[prompt-definition repository tests](../../internal/promptdef/repository_test.go).
The runner selects from this index for preparation and `InspectPrompt`.
Neither path rescans configuration or reopens message files. `InspectPrompt`
returns declared metadata without rendering or resolving profile, input, or
schema resources. Runtime rendering still checks request variables and input
helpers. The [prompt adapter tests](../../internal/configuration/prompts_test.go)
cover resource origins, context containment, capture, and selection.
## Profiles And Built-Ins
`internal/profile` owns immutable in-memory and filesystem repositories, local
validation, overlays, and resolution for execution profiles. In-memory
definitions are normalized and validated once at that repository boundary and
are defensively copied on storage and lookup. A file contains
exactly one YAML document and its trimmed YAML `id` is its only selection
identity; filenames do not confer authority. Each point lookup reads discovered
files once for their metadata and reuses the selected file's bytes for strict
decoding; unrelated profiles are not fully decoded. Strict selected decoding
recognizes `base_profile` and the optional `backend` field, trims their values,
and permits inherited target fields only when a base is named. File-backed
`extra_params` values are validated and defensively copied through the shared
bounded JSON-value owner before a profile is published. OpenAI-compatible
reserved-field policy remains with the model-client and backend-registry owners.
`internal/catalog` selects the released keyed OpenRouter and Rakestrawhome
assets as one ConfigKit underlay and PromptKit-owned profiles as the next.
It discovers YAML inputs and supplies passive filesystem context without
parsing or pre-validating catalog definitions. The configuration loader
validates only the final effective backend and profile values, including
consumer overlays of maintained bases or backends. There is no catalog-only
preflight, reserved source namespace, or replay of backend registration.
`LoadFSRepository` is the eager immutable loading boundary for internal
catalog consumers. It discovers and strictly validates every raw profile once,
preserves safe source metadata including explicitly present YAML fields, and
publishes independently copied values from memory. It does not resolve profile
inheritance. Configured consumer sources continue to use the lazy point lookup
repositories described above.
`internal/catalog` validates the imported immutable OpenRouter and
Rakestrawhome asset modules as one private adapter boundary. It enforces their
manifest, layout, profile ownership, inheritance, and secret-safety rules
before returning raw catalog sources. Internal engine assembly uses those validated
catalogs as the lowest-precedence profile source.
Normalized external catalog profile IDs with the exact `promptkit-` prefix are
rejected at this adapter boundary. That reservation applies only to imported
catalogs; configured consumer sources retain the ordinary profile-ID rules.
The overlay repository consults the next repository only when the
higher-precedence repository reports that a profile is absent. A reliably
selected malformed profile stops fallback, while an unrelated malformed file
does not become authoritative through its filename. Loading does not check
backend registry membership because the available registry belongs to the
assembled engine; the runner checks membership during preparation and exact
profile inspection.
The internal engine assembles one raw composite catalog in precedence order:
in-memory profiles, one ordinary configured source, an application fallback
source, Promptkit-owned profiles, then the maintained external catalog. It
constructs the owned definitions through the in-memory repository and validates
their maintained-only resolution and backend membership before consumer sources
are overlaid. An explicit file or `fs.FS` profile source replaces
`Config.ProfileDir` within the ordinary configured-source category. One outer
resolving repository wraps that complete raw catalog, so each base lookup
observes the same precedence and shadowing rules.
The resolving repository traverses every selected chain afresh, retains no
cache, detects cycles, limits a chain to 32 profiles, merges root-to-leaf into a
new caller-owned value, and validates the final target before publishing it. It
does not check backend registry membership. Exact `base_profile` syntax, merge
rules, and consumer-visible failure behavior belong to the [framework format
reference](../formats.md#profile-inheritance).
Exact profile inspection performs one point-in-time resolved lookup through
those profile sources and checks the final target without reading prompt, input,
or schema sources. It does not retain that lookup for a later execution.
Prepared execution instead freezes the fully resolved target; a later ordinary
operation performs a fresh traversal.
The imported external catalog provides external profiles and matching backend
definitions. The internal engine owns the Promptkit profiles documented in the
[framework format reference](../formats.md#built-in-profile-catalog). Profile
loading and overlay behavior are owned by the [profile repository tests](../../internal/profile/repository_test.go), while
catalog completeness, backend selection, and duplicate IDs are owned by the
[catalog adapter tests](../../internal/catalog/catalog_test.go).
ConfigKit resolves profile inheritance before the handoff. The configuration
adapter validates effective references and field values, then builds target
snapshots with backend defaults and presence/suppression state. A profile can
remain partial until runtime overrides complete it. `InspectProfile` selects
that captured target without prompt, artifact, schema, credential lookup, or
provider work. Preparation applies runtime overrides and then checks target
completeness. The [target tests](../../internal/configuration/targets_test.go)
cover effective target interpretation, while the [catalog tests](../../internal/catalog/catalog_test.go)
cover selected released assets.
## Ordinary Artifacts
@@ -137,14 +93,15 @@ implemented reader behavior and failures.
## Rendering
`internal/prompt` renders definition messages as Go templates using named
artifacts and variables. Within one render, each referenced artifact body is
converted to text lazily and cached by input name for reuse across the session
and every message; the cache is not shared across renders. Conversion uses
bounded chunks and preserves the artifact bytes exactly.
`internal/prompt` renders captured, pre-parsed definition templates using named
artifacts and variables. It clones templates to bind request-specific input
helpers without mutating shared state. Within one render, each referenced
artifact body is converted to text lazily and cached by input name for reuse
across the session and every message; the cache is not shared across renders.
Conversion uses bounded chunks and preserves the artifact bytes exactly.
Session and message parsing and execution remain synchronous. The renderer
checks cancellation before and after each parse and execution boundary,
Template parsing occurs during engine construction. Runtime execution remains
synchronous. The renderer checks cancellation around execution boundaries,
between artifact conversion chunks, around each message, and before publishing
the complete prompt. It cannot interrupt template work already in progress and
never publishes a partial prompt after observing cancellation. It validates and
@@ -181,9 +138,10 @@ not detach dependency work into background goroutines.
`Prepare` discards its validation plan after returning metadata. `Run` retains
its plan for initial and repaired-output validation, then discards it with the
operation. `PrepareExecution` retains the plan in its private frozen payload;
`RunPrepared` uses that plan without reopening prompt, profile, input, or
schema sources or rerendering the request. A later ordinary `Run` always
performs fresh source resolution and preparation.
`RunPrepared` uses that plan without reopening configuration, message, input,
or schema resources or rerendering the request. A later ordinary `Run` uses
the same captured configuration but prepares request artifacts and schemas
afresh.
The [validator tests](../../internal/validate/standard_validator_test.go) own
basic, JSON, JSON Schema, source resolution, schema loading, compilation,

View File

@@ -1,60 +1,75 @@
# Open Source Attribution Notices
These entries identify upstream material used by Promptkit. The
[OSS policy](../policy/oss.md) owns the format and maintenance rules; the
[project grant](../../COPYRIGHT) does not replace third-party terms.
## Standard Repository Policies
The module entries cover the five external modules used by `go list -deps -test
./...` at adoption, including the transitive `golang.org/x/text` dependency.
Other modules present only in dependency module graphs are not thereby
incorporated into Promptkit. No separately copied third-party code or asset
notices were identified in Promptkit's source; adapted policy documents are
attributed separately below. Standard license texts in `licenses/` are shared
reference texts, not additional project grants.
Material: policy documents under `docs/policy/` and the Policy Reading Guide in
`docs/development.md`, adapted for this repository; root `AGENTS.md`, copied
without modification from upstream `agents/AGENTS.md`.
## Promptkit OpenRouter Catalog
Copyright (C) 2026 Eric Rakestraw
Selected module: `gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2`
at `v2.20260926.0`.
SPDX-License-Identifier: CC-BY-SA-4.0
License and notices: [CC BY-SA 4.0](licenses/CC-BY-SA-4.0.txt) |
[COPYRIGHT](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/COPYRIGHT) |
[LICENSE](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/LICENSE).
[Source](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843).
## ConfigKit
Selected module: `gitea.maximumdirect.net/eric/ConfigKit` at `v0.5.0`.
Copyright (C) 2026 Eric Rakestraw
SPDX-License-Identifier: LGPL-3.0-or-later
License and notices: [LGPL 3.0](../../LICENSE) |
[COPYRIGHT](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/COPYRIGHT) |
[LICENSE](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/LICENSE).
[Source](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd).
## Promptkit OpenRouter Catalog v3
Selected module: `gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v3`
at `v3.20260930.0`.
Copyright (C) 2026 Eric Rakestraw
SPDX-License-Identifier: MIT
License and notices: [MIT](licenses/MIT.txt) |
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/4f731d217e0bc39151c010f50fb5912706e7b6b3/COPYRIGHT) |
[upstream LICENSE](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/4f731d217e0bc39151c010f50fb5912706e7b6b3/LICENSE).
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/f17d208cb7ad1fb043cda2266cd3011d11b40aa0/COPYRIGHT) |
[LICENSE](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/f17d208cb7ad1fb043cda2266cd3011d11b40aa0/LICENSE).
Source: [OpenRouter catalog at the imported commit](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/4f731d217e0bc39151c010f50fb5912706e7b6b3).
[Source](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/f17d208cb7ad1fb043cda2266cd3011d11b40aa0).
## Promptkit Rakestrawhome Catalog
## Promptkit Rakestrawhome Catalog v3
Selected module: `gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v2`
at `v2.20260926.0`.
Selected module: `gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v3`
at `v3.20260930.0`.
Copyright (C) 2026 Eric Rakestraw
SPDX-License-Identifier: MIT
License and notices: [MIT](licenses/MIT.txt) |
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/48918a3ad39fd99843cda94a86133b1b3881d596/COPYRIGHT) |
[upstream LICENSE](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/48918a3ad39fd99843cda94a86133b1b3881d596/LICENSE).
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/993443d88c451901f4c917fa00c204fd3b54d28d/COPYRIGHT) |
[LICENSE](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/993443d88c451901f4c917fa00c204fd3b54d28d/LICENSE).
Source: [Rakestrawhome catalog at the imported commit](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/48918a3ad39fd99843cda94a86133b1b3881d596).
[Source](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/993443d88c451901f4c917fa00c204fd3b54d28d).
## jsonschema
Selected module: `github.com/santhosh-tekuri/jsonschema/v6` at `v6.0.2`.
No project-specific copyright statement was supplied in the selected source's
LICENSE, README, or Go source files; none is invented here.
SPDX-License-Identifier: Apache-2.0
License and notices: [Apache 2.0](licenses/Apache-2.0.txt) |
[upstream LICENSE](https://github.com/santhosh-tekuri/jsonschema/blob/29cbed948d24a04700eb94436416b18a07953b71/LICENSE).
[LICENSE](https://github.com/santhosh-tekuri/jsonschema/blob/29cbed948d24a04700eb94436416b18a07953b71/LICENSE).
Source: [jsonschema at the imported commit](https://github.com/santhosh-tekuri/jsonschema/tree/29cbed948d24a04700eb94436416b18a07953b71).
[Source](https://github.com/santhosh-tekuri/jsonschema/tree/29cbed948d24a04700eb94436416b18a07953b71).
## Go Text
@@ -62,20 +77,17 @@ Selected module: `golang.org/x/text` at `v0.14.0` (transitive dependency).
Copyright (c) 2009 The Go Authors. All rights reserved.
Individual source files retain their additional year-specific Go Authors
notices in the linked source tree.
SPDX-License-Identifier: BSD-3-Clause
License and notices: [BSD 3-Clause](licenses/BSD-3-Clause.txt) |
[upstream LICENSE](https://github.com/golang/text/blob/6c97a165dd661335ff7bce6104a008558123c353/LICENSE) |
[additional patent grant](https://github.com/golang/text/blob/6c97a165dd661335ff7bce6104a008558123c353/PATENTS).
[LICENSE](https://github.com/golang/text/blob/6c97a165dd661335ff7bce6104a008558123c353/LICENSE) |
[PATENTS](https://github.com/golang/text/blob/6c97a165dd661335ff7bce6104a008558123c353/PATENTS).
Source: [Go Text at the imported commit](https://github.com/golang/text/tree/6c97a165dd661335ff7bce6104a008558123c353).
[Source](https://github.com/golang/text/tree/6c97a165dd661335ff7bce6104a008558123c353).
## Go YAML v3
Selected module: `gopkg.in/yaml.v3` at `v3.0.1`.
Selected module: `gopkg.in/yaml.v3` at `v3.0.1` (transitive dependency).
Copyright (c) 2006-2010 Kirill Simonov
@@ -87,32 +99,28 @@ Copyright 2011-2016 Canonical Ltd.
SPDX-License-Identifier: MIT AND Apache-2.0
The licenses apply to different files, not an either/or choice. The upstream
LICENSE identifies the libyaml-derived MIT files and the Apache-licensed
remainder; NOTICE supplies the final copyright statement above.
License and notices: [MIT](licenses/MIT.txt) |
[Apache 2.0](licenses/Apache-2.0.txt) |
[LICENSE](https://github.com/go-yaml/yaml/blob/f6f7691b1fdeb513f56608cd2c32c51f8194bf51/LICENSE) |
[NOTICE](https://github.com/go-yaml/yaml/blob/f6f7691b1fdeb513f56608cd2c32c51f8194bf51/NOTICE).
[Source](https://github.com/go-yaml/yaml/tree/f6f7691b1fdeb513f56608cd2c32c51f8194bf51).
## Go YAML v4
Selected module: `go.yaml.in/yaml/v4` at `v4.0.0-rc.6` (transitive dependency).
Copyright 2006-2010 Kirill Simonov
Copyright 2011-2019 Canonical Ltd
Copyright 2025 The go-yaml Project Contributors
SPDX-License-Identifier: MIT AND Apache-2.0
License and notices: [MIT](licenses/MIT.txt) |
[Apache 2.0](licenses/Apache-2.0.txt) |
[upstream LICENSE and file scope](https://github.com/go-yaml/yaml/blob/f6f7691b1fdeb513f56608cd2c32c51f8194bf51/LICENSE) |
[NOTICE](https://github.com/go-yaml/yaml/blob/f6f7691b1fdeb513f56608cd2c32c51f8194bf51/NOTICE).
[LICENSE](https://github.com/yaml/go-yaml/blob/1c17b9cee72bf81ae73a8be50fbd7dd3a9985125/LICENSE) |
[NOTICE](https://github.com/yaml/go-yaml/blob/1c17b9cee72bf81ae73a8be50fbd7dd3a9985125/NOTICE).
Source: [Go YAML at the imported commit](https://github.com/go-yaml/yaml/tree/f6f7691b1fdeb513f56608cd2c32c51f8194bf51).
## Standard Repository Policies
Material: adapted standard policy documents under `docs/policy/`, with
Promptkit-specific ownership, library boundaries, contributor guidance, and
release/OSS conventions. These documents have been modified from the templates.
No project-specific copyright statement is supplied by the standard policy
repository's README or LICENSE; none is invented here.
SPDX-License-Identifier: CC-BY-SA-4.0
The adapted policy documentation retains these upstream terms; this does not
change Promptkit's library grant.
License and notices: [CC BY-SA 4.0](licenses/CC-BY-SA-4.0.txt).
Source: maintainer-provided internal standard policy templates. The source
repository is non-public; no public source or notice URL is available.
[Source](https://github.com/yaml/go-yaml/tree/1c17b9cee72bf81ae73a8be50fbd7dd3a9985125).

View File

@@ -1,19 +1,20 @@
# Agent Policy
**Base Policy Version:** 1.1
**Base Policy Version:** 1.3
**Last Updated:** 2026-09-23
**Last Updated**: 2026-09-27
**Upstream Source:** [agents.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/agents.md)
## Purpose
Agents must follow these house rules before recommending, planning, or
implementing repository changes.
Agents must read and follow these rules.
## Implementation Discipline
Write the simplest idiomatic code that satisfies the user's request.
Simplify the mechanism, not the user's choices. Do not implicitly strengthen
requirements, restrictions, or guarantees.
**KEY PRINCIPLES:** Write the simplest idiomatic code that satisfies the user's
request. Do not implicitly strengthen requirements, restrictions, or guarantees
beyond what the user actually requested.
**Keep implementation machinery proportionate.** Use straightforward,
idiomatic code and introduce only the machinery needed to support the requested
@@ -28,41 +29,48 @@ operator and caller choice where policy has not been established. If an
additional restriction appears advisable, present it as a decision rather than
silently implementing it.
**Do not invent guarantees.** Do not strengthen public contracts, guarantees,
or invariants beyond those expressly requested by the user.
**Use the standard library.** For example: in almost every case, the best
approach to copying files is to use the standard library. Writing custom code to
implement power-loss protection at the application level, or to guard against
bad actors with privileged and concurrent access to the host's filesystem,
is wasteful, unnecessary, and not idiomatic. Stick to the simple, standard, and
idiomatic approach unless another path is expressly requested by the user.
**An implementation detail is not repository policy.** Tests and documentation
must not elevate incidental behavior into a durable guarantee or restriction
unless that behavior was expressly selected as part of the contract.
## Promptkit Contract Authority
The following clarification preserves Promptkit's established distinction
between implementation choices and authorized contracts.
A correctness constraint must follow from the requested behavior or an
established invariant, not from the convenience of the chosen implementation.
Explain the specific failure it prevents and why a less restrictive
implementation cannot satisfy the contract.
**An implementation detail is not repository policy.** Tests and documentation
must not elevate incidental behavior into a durable guarantee or restriction
unless that behavior was expressly selected as part of the contract. Existing
explicit contracts remain authoritative; changing or weakening one requires a
deliberate decision rather than being inferred from this principle.
Agent-authored plans, tests, documentation, and prior implementations are not
evidence that an additional restriction was deliberately approved. Identify the
authorizing requirement or decision; when that authority is unclear, surface
the ambiguity rather than treating repetition as approval.
**Do not invent guarantees.** Do not strengthen public contracts, guarantees,
or invariants beyond those expressly requested by the user.
**Use the standard library.** Prefer simple, idiomatic standard-library
implementations when they satisfy the requested behavior. Do not add custom
power-loss protection or defenses against privileged concurrent filesystem
actors unless expressly required. Preserve existing repository contracts.
Existing explicit contracts remain authoritative; changing or weakening one
requires a deliberate decision. Agent-authored plans, tests, documentation,
and prior implementations are not evidence that an additional restriction was
deliberately approved. Identify the authorizing requirement or decision; when
that authority is unclear, surface the ambiguity rather than treating repetition
as approval.
## Git Commit Policy
By default, create a commit after completing each coherent unit of work, using a
By default, commit each completed, nonempty unit of work using a
concise, plain-English message describing what changed. This policy is standing
authorization to create those local commits; a separate user request is not
required. An explicit user instruction to leave work uncommitted takes precedence.
A unit of work is a focused, reviewable outcome, including its relevant tests and
documentation. Do not commit after every intermediate edit or create empty commits.
documentation.
Complete the appropriate validation before committing. If work is incomplete or
validation is blocked, report that state rather than presenting it as completed.
validation is blocked, report that state accurately.
Before committing:
@@ -70,10 +78,10 @@ Before committing:
changes. Identify changes that predate the current unit of work.
2. Exclude secrets, local environment files, generated junk, and unrelated changes.
3. Stage only the files or hunks belonging to the completed unit. Use `git add .`
only when every included change belongs to that unit. Do not disturb unrelated
staged or unstaged work.
4. Review `git diff --staged` and run appropriate checks for the change. Do not
repeat checks already passed for unchanged content merely to create a commit.
only when every included change belongs to that unit. Preserve unrelated
staged and unstaged work.
4. Review `git diff --staged` and run appropriate checks for the change. Reuse
successful checks for unchanged content.
5. Review `git log --oneline -n 10` and use
`git commit -m "<concise plain-English commit message>"`.
6. Report the commit identifier and any remaining uncommitted work.

View File

@@ -1,15 +1,41 @@
# Architecture Policy
**Base Policy Version:** 1.0
**Base Policy Version:** 1.3
**Last Updated:** 2026-09-23
**Last Updated**: 2026-09-27
## Purpose
**Upstream Source:** [architecture.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/architecture.md)
This document defines Promptkit's current high-level architecture and the
durable boundaries that implementation changes must preserve. The
[internal component overview](../internal/overview.md) inventories concrete
implemented packages without redefining these rules.
## Purpose And Ownership
The adopted policy defines Promptkit's purpose and the architectural
constraints contributors must follow. It owns system boundaries, responsibility
and dependency rules, established invariants, and explicit non-goals.
Describe the concrete package inventory and implementation mechanics in internal
documentation. Link to canonical public contracts for their exact behavior and
to ADRs for decision rationale.
## Architectural Approach
Hexagonal architecture is strongly recommended, including encapsulating external
integrations and APIs within internal adapters. Keep domain and application
logic independent of external integration details, and make dependency direction
clear in the adopted policy.
Implement that separation with proportionate, idiomatic code. Choose interfaces
and package layout to serve the application's actual boundaries. Use adapters
for collaborators with external integration responsibilities.
## Adopted Constraints And Implemented Structure
Architectural constraints govern current and future development. Record
implemented structure accurately, proposals and implementation status in the
roadmap, and significant decisions and rationale in ADRs. Describe a target
structure as implemented when its changes land.
Constraints must reflect deliberately selected requirements and decisions.
Do not promote incidental implementation choices into architectural invariants.
## System Shape
@@ -25,23 +51,22 @@ The [internal component overview](../internal/overview.md) owns the inventory
of implemented packages and their concrete responsibilities. This policy owns
the boundaries and dependency rules those components must preserve.
The internal engine assembles one immutable backend registry, one capacity manager,
the internal repositories, renderer, validator, outbound client, and use-case
runner. The registry contains validated maintained definitions plus engine-scoped
consumer additions. The root facade translates public values and errors at the
library boundary.
The internal engine constructs the capacity manager from the registry's immutable
policy snapshot, wraps the selected built-in or injected model client, and
supplies bounded admission to the runner. The defaults and renderer depend on
the domain model. Prompt-definition and profile repositories use the domain
model, file catalog, and YAML decoder. The catalog adapter validates imported
immutable maintained data, including its reserved namespace, before internal
engine assembly supplies it to the profile package. Artifact reading uses the
domain model and application-neutral defaults. Validation uses the domain model,
file catalog, and JSON Schema implementation. The model client uses the domain
model and application-neutral defaults, and an injected or standard-library HTTP
client. The use-case runner depends on the narrow interfaces owned by each
internal component, including backend lookup and run admission.
The internal engine composes selected catalog and caller ConfigKit inputs once,
then assembles the captured definitions, capacity manager, renderer, validator,
outbound client, and use-case runner. The configuration adapter interprets and
validates only the final effective values, including presence, null, and
resource context. It retains source history for explicit operator inspection,
not for validation or a second resolution pass. The root facade translates
public values and errors at the library boundary.
The engine constructs capacity policy from final backend definitions, wraps
the selected built-in or injected model client, and supplies bounded admission
to the runner. Runtime/domain components consume execution-owned values, not
ConfigKit types. Artifact reading and validation use the domain model and
application-neutral defaults; validation also uses file-catalog helpers and
JSON Schema. The model client uses the domain model, defaults, and an injected
or standard-library HTTP client. The runner coordinates narrow lookup and
collaborator interfaces without rereading configuration.
The current implementation follows this dependency direction:
@@ -58,9 +83,10 @@ downstream consumers, including Scriptorium
narrow injected abstractions
```
The backend registry depends on the domain model and shared JSON-value
validation, has no mutation API after construction, and consumes the
OpenAI-compatible reserved request-field rule owned by the model client. The
The backend index depends on the domain model and shared JSON-value ownership
and has no mutation API after construction. The configuration adapter applies
the OpenAI-compatible reserved request-field rule owned by the model client
when it validates final provider maps. The
capacity component depends on the domain model and the narrow internal
model-client boundary, not on provider transport implementation. The model
client does not depend on registry or capacity configuration. The facade
@@ -155,3 +181,21 @@ state.
Do not list planned packages as implemented components. When implementation
introduces a package, update the internal inventory and the owning contract or
subsystem document in the same change.
## Trust Boundaries And Threat Scope
Where useful, state the trust boundaries and threat scenarios relevant to the
application's adopted requirements, along with scenarios it explicitly excludes.
For example, distinguish handling untrusted application input from defending
against a privileged actor concurrently modifying the host filesystem.
Threat exclusions define the scope of the adopted requirements. Base defenses
and guarantees on those requirements and raise unsettled scope for a deliberate
decision under the [agent policy](agents.md).
Keep this discussion proportionate to the application. A separate threat
assessment or security document is optional when adopting this template.
For this library, the framework format and integration documents own credential
supply and external contracts; internal documents own implementation mechanisms.
Consumer-specific filesystem and security policy remains outside Promptkit's
boundary as described above.

View File

@@ -1,157 +1,139 @@
# Documentation Policy
**Base Policy Version:** 1.5
**Base Policy Version:** 1.11
**Last Updated:** 2026-09-26
**Last Updated**: 2026-09-27
**Upstream Source:** [documentation.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/documentation.md)
## Purpose
This policy assigns each Promptkit documentation topic to one canonical owner.
Its goal is to keep documentation for this reusable Go library accurate,
concise, discoverable, and resistant to drift for consumers, contributors,
maintainers, integrators, and coding agents.
This policy assigns each documentation topic to one canonical owner. Its goal is
to keep this library's documentation accurate, concise, discoverable, and resistant
to drift for users, operators, developers, integrators, and LLM coding agents.
## Core Rules
### One Canonical Owner
Each authoritative fact belongs in one document or source form. A non-owning
document may give a short, stable summary for orientation, but it must link to
the canonical owner instead of repeating exact contracts.
Each authoritative fact belongs in one document. A non-owning document may give
a short, stable summary for orientation, but it must link to the canonical owner
for volatile details.
Volatile details include exported declarations, accepted inputs, defaults,
schemas, file names, paths, error identities, retry behavior, and runtime
guarantees. If readers could reasonably treat a statement as a contract,
maintain its exact definition only in the owning source.
Volatile details include commands, flags, configuration fields and defaults,
module keys, schemas, file names, paths, status codes, retry behavior, and
runtime guarantees. If readers could reasonably treat a statement as a
contract, maintain it only in the owning document.
### Current State, Decisions, And Future Work
### Current And Future Behavior
Outside `docs/roadmap/`, documentation describes implemented behavior only.
Partial features may be described only to their implemented boundary.
An accepted architecture decision may describe an approved direction before it
is implemented, but acceptance is not evidence that the behavior exists.
Current-state documents change when the implementation lands. Temporary
roadmaps own future work, sequencing, and implementation status; they do not
replace durable policies or current contracts.
Architectural constraints may govern future changes. Describe current structure
from implemented behavior.
ADRs may record an accepted architectural decision before implementation. The
roadmap owns implementation status and sequencing until the decision is
implemented. Current architecture, user, operator, integration, and internal
documentation are updated when the behavior lands.
### Audience And Detail
Write for the document's stated audience and include only the detail needed for
its owned topic. Consumer guidance should not expose incidental implementation
detail. Contributor documentation should link to public contracts and durable
policies instead of restating them.
its owned topic. Keep user and operator docs focused on using and operating the
application. Developer docs should link to user-facing and external contracts
for their definitions.
### Links
### Examples
Use descriptive link text and repository-relative links for repository
documents. Link to the canonical owner rather than to a duplicate summary.
Check every added or changed link and repair or remove links when their target
moves or is retired.
Complete copyable files belong in `examples/`. Documentation may use the
smallest illustrative snippet needed to explain its owned topic, but should link
to maintained examples for complete copyable files.
### Examples And Code Fences
Complete copyable files belong in `examples/` when maintained examples exist.
Documentation may use the smallest illustrative snippet needed for its owned
topic, but should link to a maintained example instead of embedding a second
complete copy.
Examples must be valid, secret-free, and tested where practical. Commands,
imports, and Go snippets must match the implemented library. Use a language tag
on fenced code blocks and make clear when a fragment is illustrative rather
than directly runnable.
Examples must be valid, secret-free, and tested where practical. Commands and
configuration used in documentation should match the application.
### Security And Privacy
Documentation and examples must not contain real credentials, private keys,
private environment dumps, sensitive source material, or private
infrastructure details unless intentionally public. Document secret-handling
mechanisms, not secret values.
private environment dumps, sensitive source material, or private infrastructure
details unless intentionally public. Document secret-handling mechanisms, not
secret values.
## Canonical Ownership
| Topic | Canonical owner | Owned content | Content owned elsewhere |
| --- | --- | --- | --- |
| Project orientation | `README.md` | What Promptkit is, its current usability, module identity, license summary, a minimal successful quickstart, and links onward. | Exact API contracts, contributor procedures, architecture detail, and release steps. |
| Contributor workflow | `docs/development.md` | Task-oriented reading guide, local workflow, validation commands, and repository hygiene. | Architecture rules, API semantics, subsystem behavior, and release procedure. |
| Current architecture | `docs/policy/architecture.md` | System shape, normative ownership, dependency direction, package boundaries, invariants, and non-goals. | Concrete component inventory, implementation mechanics, contributor procedures, decision history, and future work. |
| Agent policy | `docs/policy/agents.md` | Agent implementation discipline, proportionality, idiomatic simplicity, local commit authorization and discipline, and rules against inventing restrictions, guarantees, or repository policy. | Public contracts, architecture, testing, and contributor procedures. |
| Documentation organization | `docs/policy/documentation.md` | Documentation ownership, audience boundaries, maintenance rules, and document lifecycle. | Library architecture or runtime behavior. |
| Testing policy | `docs/policy/testing.md` | Test philosophy, risk-based sufficiency, test boundaries, doubles, coverage guidance, regression policy, and test maintenance. | Subsystem behavior, exact public contracts, subsystem-specific test inventories, and implementation plans. |
| Release policy | `docs/policy/release.md` | Versioning, compatibility, release-note requirements, validation obligations, immutability, recovery, and adoption scope. | Executable commands, publication mechanics, and historical release summaries. |
| Licensing and attribution policy | `docs/policy/oss.md` | Project licensing conventions, attribution format, text and notice-link rules, discoverability, maintenance, and concrete exceptions. | Component entries, license texts, release commands, and dependency security review. |
| Project grant | `COPYRIGHT` | Project identity, holders and years, SPDX expression, grant, warranty disclaimer, and third-party statement. | Complete license texts and component attribution. |
| Project license texts | `LICENSE` | Complete project license and incorporated companion texts. | General licensing policy and dependency attribution. |
| Upstream attribution | `docs/oss/attribution.md` | Component names, copyright statements, license identification, and license, notice, and source links. | General policy and complete standard texts. |
| Standard dependency license texts | `docs/oss/licenses/` | Complete, verbatim texts shared by attribution entries. | Project-specific notices, attribution prose, and policy interpretation. |
| Release procedure | `docs/release.md` | Maintainer prerequisites, candidate preparation, concrete publication guards and commands, verification, and recovery steps. | Release-policy requirements, public API semantics, and historical summaries. |
| Product orientation and minimal end-to-end quickstart | `README.md` | Purpose, value, shortest successful invocation, and navigation, including licensing and attribution. | Complete command reference, configuration reference, operational procedures, implementation detail. |
| Contributor entry point | `docs/development.md` | Task-oriented reading guide, contributor orientation, baseline validation commands, and canonical links. | Package inventory, architecture rules, subsystem behavior, detailed change recipes. |
| Application architecture policy | `docs/policy/architecture.md` | System shape, normative ownership, dependency direction, boundaries, invariants, safety properties, and non-goals. | Concrete package inventory, implementation mechanics, contributor procedures, decision history, future work. |
| Policy-set provenance and upstream navigation | `docs/policy/meta.md` | Policy-set origin, local authority, adoption baselines, and upstream adoption and update links. | Task-specific reading triggers, detailed adoption procedures, and formal attribution notices. |
| Agent policy | `docs/policy/agents.md` | Proportionate, idiomatic implementation; rules against inventing requirements, restrictions, guarantees, or policy. | Application contracts, architecture, testing policy, and contributor procedures. |
| Documentation policy | `docs/policy/documentation.md` | Documentation ownership, audience boundaries, maintenance rules, and ADR/document lifecycle. | Application architecture or product behavior. |
| Testing policy | `docs/policy/testing.md` | Test philosophy, risk sufficiency, boundaries, doubles, coverage, regressions, and test maintenance. | Subsystem behavior, application contracts, subsystem-specific test inventories, and implementation plans. |
| Release policy | `docs/policy/release.md` | Versioning, compatibility, release integrity and immutability, release documentation, proportionate gates, publication controls, completion, and required versus optional checks. | Repository-specific commands, product installation guidance, CLI version semantics, historical release summaries, and CI implementation detail. |
| Open source licensing and attribution policy | `docs/policy/oss.md` | License defaults and author choices, COPYRIGHT and attribution formats, license/notice links, discoverability, maintenance, and concrete exceptions. | Component entries, license texts, release commands, and dependency security review. |
| Project copyright and license grant | `COPYRIGHT` | Project identity, copyright holders and years, SPDX expression, grant, warranty disclaimer, and third-party statement. | Complete license texts and upstream component attribution. |
| Project license texts | `LICENSE` | Complete project and incorporated companion license texts, with required project-specific fields. | General licensing policy and upstream component attribution. |
| Upstream component attribution | `docs/oss/attribution.md` | House Format component entries: identity, material scope and modifications where applicable, copyright, license, required notices, and license, notice, and source links. | General attribution policy, exception reasoning, research narratives, compliance commentary, and complete standard license texts. |
| Standard dependency license texts | `docs/oss/licenses/` | Complete, verbatim standard license texts shared by applicable entries. | Project-specific upstream notices, attribution prose, and policy interpretation. |
| Repository-specific release procedure | `docs/release.md` | Maintainer prerequisites, candidate preparation, validation and publication commands and automated guards, asynchronous deliverables, delivery status, optional consumer verification, and recovery. | General release requirements, product installation guidance, historical release summaries, and current compatibility contracts. |
| Consumer guidance for the public Go API (if applicable) | `docs/consumers/` | Task-oriented public Go API usage, minimal examples, and consumer responsibilities. | HTTP wire semantics, external protocol contracts, internal implementation detail. |
| Integration contracts for external components (if applicable) | `docs/integrations/` | External formats and protocols, upstream and downstream contracts, logical output paths and schemas, media types, and compatibility. | Physical runtime placement and lifecycle, internal transformations, CLI syntax, configuration defaults. |
| Inventory of implemented internal packages | `docs/internal/overview.md` | Implemented packages and components, responsibilities, and links to focused internal docs. | Normative architecture, contributor reading policy, external contracts. |
| Behavior and architecture of internal packages | Other files under `docs/internal/` | Implementation flow, collaborators, state transitions, package-local guarantees and failures, and relevant tests. | Global architecture invariants, configuration definitions and defaults, external schemas, operator procedures. |
| Architectural decision records | `docs/adr/` | Significant decisions, context, alternatives, rationale, consequences, and supersession history. | Current behavior reference, implementation status, task sequencing. |
| Future work and implementation status | `docs/roadmap/` | Proposed, accepted, deferred, or rejected work; implementation status, sequencing, and task breakdowns. | Implemented behavior reference and architectural decision rationale. |
| Release-note history | `docs/releases/` | One checked-in summary per source release under the procedure; the note at its immutable tag is the release record. | Current commands, behavior, contracts, and compatibility definitions. |
| Complete copyable artifacts | `examples/` | Maintained configuration, inputs, and other copyable or runnable files. | Field-by-field reference, command reference, prose explanation. |
| Exact exported Go API | Go declarations and GoDoc, as APIs are implemented | Exported names, signatures, types, values, errors, and exact behavioral contracts. | Task-oriented consumer walkthroughs, implementation details, and future API proposals. |
| Framework file formats | `docs/formats.md` | Prompt-definition and profile YAML fields, schema references, defaults, validation modes, built-in profiles, credentials, and file-to-request precedence. | Exported Go declarations, outbound wire behavior, internal parsing mechanics, and application configuration. |
| Consumer guidance | `docs/consumers/`, when consumer workflows require dedicated guidance | Task-oriented use of implemented public APIs, minimal examples, and consumer responsibilities. | Exact exported declarations and internal mechanics. |
| Durable integration contracts | `docs/integrations/`, when integrations exist | External formats and protocols, compatibility behavior, and upstream or downstream responsibilities. | Internal transformations and public Go declarations. |
| Release-note history | `docs/releases/` | Mandatory version-matched historical summaries for releases made under the adopted release policy; the note at the immutable tag records that release. | Current APIs, behavior, formats, compatibility contracts, and release procedures. |
| Implemented component inventory | `docs/internal/overview.md` | Current packages and components, their implemented responsibilities, and links to focused internal documents. | Normative architecture, contributor workflow, external contracts, and proposed components. |
| Internal subsystem behavior | Other files under `docs/internal/`, when a subsystem needs durable detail | Implementation flow, internal collaborators and state transitions, package-local guarantees and failures, and relevant tests. | Global architecture invariants, public API definitions, and future package plans. |
| Architectural decision history | `docs/adr/`, when repository-local decisions require records | Significant decisions, context, alternatives, rationale, consequences, and supersession history. | Current behavior reference, implementation status, and task sequencing. |
| Temporary feature roadmaps | `docs/roadmap/`, while planned work needs coordination | Proposed or accepted scope, sequencing, gates, and implementation status. | Implemented behavior reference and durable decision rationale. |
| Complete copyable artifacts | `examples/` | Valid inputs, Go programs, and other files intended to be copied or run. | Field-by-field reference, exact API declarations, and prose explanation. |
Conditional owners do not require placeholder files or directories. Create a
consumer, integration, release, subsystem, ADR, roadmap, or example document
only when the corresponding implemented interface, release, decision, planned
effort, or maintained artifact exists.
Create supporting documents for existing interfaces and responsibilities when
substantive content is available.
## Boundary Rules
### Orientation
The README owns project orientation. The development guide routes
contributors. Architecture owns normative structure. The internal overview
owns the current concrete component map. These documents may link to one
another but must not maintain parallel package or behavior descriptions.
Link among orientation, architecture, and internal documents; maintain each
package or behavior description in its canonical owner.
### Public Contracts And Implementation
Go declarations and GoDoc own exact exported API contracts once those APIs
exist. Consumer and integration documents explain how to use those contracts
for a task. Internal documents explain how Promptkit implements them. Internal
documentation may identify a public type or external format as a dependency,
but must link to its canonical definition rather than restate it.
Go declarations and GoDoc own exact exported Go API contracts. The
[framework format reference](../formats.md) owns prompt, profile, and schema
formats; integration documents own external wire behavior. Consumer and
internal documents link to those owners for exact definitions.
The [framework format reference](../formats.md) owns exact prompt, profile, and
schema-file contracts. Integration documents own external wire formats.
### Releases
### Release Documentation
The release policy defines requirements and invariants that every release must
satisfy. The repository-specific release procedure defines the commands and
workflow maintainers use to satisfy that policy. Release notes record history
at immutable tags. Maintain current behavior in its canonical documentation,
update it with the corresponding behavior changes, and keep release procedures
consistent with adopted policy.
The [release policy](release.md) owns release requirements; the
[repository release procedure](../release.md) owns their concrete execution.
Version-matched checked-in notes are required for future releases under that
policy. The source tag is the release identity, and the note in its immutable
tree is the historical summary; annotated tag messages complement that record.
### Project Licensing And Upstream Attribution
Release notes do not own current API or behavior contracts. Link to the
canonical owners for details instead of reproducing those contracts. Published
notes are not disposable guidance and must not be replaced under the same
version. Corrections belong in a subsequent release.
README and release documents link to the canonical licensing and attribution
materials under the [OSS policy](oss.md). The release procedure owns URL
construction and publication mechanics.
Adoption is prospective. Previously published tags and historical notes retain
their original records and are not reformatted or rewritten to satisfy the new
requirements.
### Contracts And Implementation
### Project Licensing And Attribution
The [OSS policy](oss.md) owns licensing conventions, the attribution House
Format, and README/release navigation. COPYRIGHT owns the project grant;
LICENSE supplies complete texts. Component-specific information belongs in
attribution entries, not parallel README or release-note notice lists. The
release procedure owns concrete footer URLs and publication mechanics.
Integration and API documents define externally observable shapes and
semantics. Internal documents explain how this application implements or consumes those
contracts. Internal docs may name a field, file, or protocol to identify a
dependency, but must link to its canonical contract for the definition.
### Security Topics
This policy owns what documentation and examples may contain. Architecture owns
library security boundaries and invariants. Public declarations and integration
documents own consumer-visible security contracts. Internal documents own
implementation mechanisms only.
library security boundaries. Public declarations and integration documents own
consumer-visible contracts; internal documents own implementation mechanisms.
## Architecture Decision Records
@@ -166,31 +148,35 @@ Use sequentially numbered ADR filenames such as
6. alternatives considered;
7. consequences.
Use one of these statuses:
Treat the decision content of an accepted ADR as immutable. When a decision
changes, create a new ADR and update the earlier ADR's status to superseded.
Rejected architectural alternatives belong in the ADR; rejected product ideas
belong in the roadmap.
- **Proposed:** the decision is under consideration and may change;
- **Accepted:** the decision is approved, whether or not implementation is
complete;
- **Rejected:** the proposed decision was considered and not adopted;
- **Superseded:** a later accepted ADR replaces the accepted decision.
## Maintenance
A proposed ADR transitions to Accepted or Rejected. An Accepted ADR transitions
to Superseded only when a later Accepted ADR replaces it. An ADR may be created
as Accepted when the decision has already been made.
When behavior changes, update its canonical owner in the same change. If
ownership moves, remove the old definition and replace it with a link where
navigation remains useful.
Treat the decision content of an Accepted ADR as immutable. A changed decision
requires a later ADR rather than a rewrite of the accepted record. A Superseded
ADR must link to its replacement, and the replacement must link back. Rejected
architectural alternatives belong in the ADR; rejected feature ideas belong in
a roadmap when they need to be retained.
Before completing documentation work:
- verify affected behavior and examples;
- check commands, flags, fields, defaults, schemas, and paths against their
implementation;
- keep unimplemented behavior in the roadmap, subject to the ADR exception;
- remove stale references and validate links;
- confirm that non-owning documents summarize and link rather than redefine;
- confirm that no secrets or sensitive private data were added.
## Promptkit Documentation Scope
Promptkit is a Go library, so exported GoDoc and framework formats replace
application CLI, configuration, operations, and HTTP-service owners. Reading
triggers belong in the development guide, not this topic map.
## Document Lifecycle
Create durable current-state documentation with the implementation it
describes. Update its canonical owner in the same change when behavior changes.
If ownership moves, remove the old definition and leave a link where navigation
remains useful.
Roadmaps are temporary coordination documents. When their work is complete,
record completion, move any still-useful decisions or contracts to their
durable owners, update incoming links, and archive or remove the roadmap
@@ -200,13 +186,9 @@ second current-state reference.
Retain release notes as historical records under the release policy. Do not
apply temporary-roadmap retirement rules to published release notes.
Before completing documentation work:
- verify affected behavior and examples;
- check commands, imports, declarations, defaults, schemas, and paths against
their implementation;
- keep unimplemented behavior in a roadmap, subject to the ADR exception;
- validate links and fenced examples;
- confirm non-owning documents summarize and link rather than redefine;
- remove stale or unsupported claims; and
- confirm that no secrets or sensitive private data were added.
For Promptkit ADRs, retain the existing Proposed, Accepted, Rejected, and
Superseded statuses. Proposed records may become Accepted or Rejected;
Accepted records may become Superseded, with links between the earlier record
and its replacement. Records may be created as Accepted when the decision has
already been made. These lifecycle rules preserve existing local document
semantics.

27
docs/policy/meta.md Normal file
View File

@@ -0,0 +1,27 @@
# Policy Set Provenance
**Base Policy Version:** 1.0
**Last Updated**: 2026-09-27
**Upstream Source:** [meta.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/meta.md)
## Origin And Local Authority
This repository's policy set is adapted from
[Standard Repository Policies](https://gitea.maximumdirect.net/eric/policy).
The adopted local policies govern work in this repository. Upstream revisions
become local policy through deliberate adoption, preserving local decisions.
Each policy's `Upstream Source` identifies its adopted baseline.
## Adoption And Updates
For policy adoption, upstream revision reviews, and updates, consult the
[upstream adoption and update guide](https://gitea.maximumdirect.net/eric/policy/src/branch/main/docs/adoption.md).
That guide and the reference example collection are maintained upstream and
accessed through links. Keep discovery links pointed at the current upstream
project and guide; keep adopted-source and example links pinned to commits.
The local [developer guide](../development.md) routes tasks to applicable
policies. Formal credit and licensing notices for the copied policy material
belong in the local [attribution document](../oss/attribution.md).

View File

@@ -1,135 +1,227 @@
# Open Source Licensing And Attribution Policy
**Base Policy Version:** 1.2
**Base Policy Version:** 1.8
**Last Updated:** 2026-09-26
**Last Updated**: 2026-09-27
Adapted from the standard policy; see [provenance and licensing](../oss/attribution.md#standard-repository-policies).
**Upstream Source:** [oss.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/oss.md)
## Purpose And Scope
This policy defines Promptkit's project licensing conventions and House Format
for upstream attribution. It covers relevant incorporated or distributed
components, including transitive dependencies and copied code, documentation,
or assets. Treatment follows the component's terms and actual use, not merely
its direct-dependency status or common ownership. Development tools do not
automatically require attribution.
This policy defines project licensing conventions and a repeatable House Format
for upstream attribution and license notices. It covers the project's own code
and upstream components incorporated into or distributed with the application
or library, including relevant transitive dependencies and copied code,
documentation, or assets.
Determine treatment from the component's terms and actual use.
Promptkit is a Go library with an author-selected LGPL-3.0-or-later grant in
[COPYRIGHT](../../COPYRIGHT). Its public source repository is
`https://gitea.maximumdirect.net/eric/promptkit`; it publishes source-only Go
module releases through annotated tags, not binaries or separate hosted release
pages. The [release procedure](../release.md) owns publication mechanics.
Consulting development guidance or using tooling alone does not require an
attribution entry. Copying or adapting documentation for inclusion in the
repository is different: preserve its applicable attribution and license terms.
A material's role as development guidance does not exempt copied documents.
Apply any express permissions for attribution-free reuse of example formatting.
The House Format does not establish license compatibility or resolve every
obligation arising from use or distribution. Adoption does not relicense
existing material or override third-party terms.
The baseline is a publicly available source repository with releases published
alongside it. The format is independent of language, forge, and build system.
It does not require attribution for every development tool, establish license
compatibility, or resolve every obligation arising from use or distribution.
## Canonical Locations
The [documentation policy](documentation.md) assigns ownership:
- `COPYRIGHT`: project name, holders and years, SPDX expression, grant,
warranty disclaimer, and statement about third-party material.
- `LICENSE`: complete, unmodified project license texts, including incorporated
companion text. Promptkit includes LGPLv3 followed by GPLv3.
- `COPYRIGHT`: project name, copyright statements, SPDX expression, grant,
warranty disclaimer, and third-party statement.
- `LICENSE` contains the complete, unmodified project license text, including
incorporated companion text. Dependency attribution does not relicense the project.
- `docs/oss/attribution.md`: canonical component entries.
- `docs/oss/licenses/`: complete, verbatim standard dependency texts, including
companions where applicable. Shared texts can serve multiple entries.
- `docs/oss/licenses/`: complete, verbatim standard and applicable companion texts.
Keep explanations and project-specific notices outside standard texts. Preserve
distinct versions and exceptions, using recognizable filenames.
Use recognizable license filenames, such as `Apache-2.0.txt` or `LGPL-3.0.txt`.
A shared standard text may serve multiple entries. Keep explanations outside
license texts and preserve distinct versions and exceptions.
## Upstream Reference Examples
Use the commit-pinned upstream examples to create actual project notices.
Retain their links and keep the example collection upstream; keep applicable
license texts locally as specified above.
## Project Licensing
The house defaults for new projects are GPL-3.0-or-later for applications and
LGPL-3.0-or-later for libraries. An author's explicit choice takes precedence
without an exception rationale, extra approval, or compensating controls.
Promptkit retains its selected library grant; do not infer different licenses
for individual packages or relicense existing code by adopting this policy.
Unless the author specifies otherwise, use **GPL version 3 or later**
(`GPL-3.0-or-later`) for applications and **LGPL version 3 or later**
(`LGPL-3.0-or-later`) for libraries. An author's explicit choice takes precedence:
for example, a one-off script may use MIT. Such a choice requires no exception
rationale, additional approval, or compensating controls under this policy.
Keep COPYRIGHT, LICENSE, SPDX expressions, and README descriptions consistent.
Use accurate holders and years; do not assume the maintainer owns every
contribution or advance years automatically. Update grant, disclaimer, and text
references together when the author deliberately changes licensing. Preserve
third-party terms and required existing notices. No blanket per-file header
requirement is introduced.
These defaults guide new license selection. Adopting this policy does not
relicense existing code or override third-party terms. Preserve an existing
license unless the author deliberately changes it. For mixed-purpose projects,
follow the author's selected scope rather than inferring different licenses for
individual packages.
Use the upstream [application COPYRIGHT example](https://gitea.maximumdirect.net/eric/policy/src/commit/a469273adda336625f96b9d769ea27b33999f146/examples/licensing/application/COPYRIGHT)
or [library COPYRIGHT example](https://gitea.maximumdirect.net/eric/policy/src/commit/a469273adda336625f96b9d769ea27b33999f146/examples/licensing/library/COPYRIGHT)
to create the project’s root `COPYRIGHT` file.
Use verified project names, contribution years, and copyright holders.
For another selected license, adapt the grant, SPDX expression, disclaimer, and
text references together to reflect the author's selected grant.
For GPLv3 projects, copy the complete GPLv3 text into `LICENSE`. For LGPLv3
projects, place the complete LGPLv3 text followed by the complete GPLv3 text in
`LICENSE`; the LGPL incorporates the GPL terms. Keep the texts unmodified and
place project-specific statements in `COPYRIGHT`. For the default grants, the
grant and SPDX expression in `COPYRIGHT` must expressly identify `or-later`;
follow the author’s choice if a different grant is selected.
For other licenses, provide their complete text and any required project-specific
fields; the standard reference collection remains unchanged. Keep `COPYRIGHT`,
`LICENSE`, SPDX expressions, and README descriptions consistent with the actual
grant. This convention introduces no blanket requirement for per-file headers;
preserve notices or add them where the applicable terms require them.
Identify separately licensed documentation or assets in the project grant, with
links to their attribution and license information. An application's default
code license does not override the terms of incorporated documentation.
## Upstream Attribution House Format
Each component entry contains five fields:
Begin `docs/oss/attribution.md` with `# Open Source Attribution Notices`,
followed directly by component entries. These comprise the complete document.
1. Project name, normally as the heading.
2. Upstream copyright statements, preserving their wording.
3. Applicable SPDX license identifier or expression.
4. License and notice links.
5. Source repository link at the exact imported commit.
Each entry presents these five fields in order, with blank lines between them:
These are fields, not a line limit. Include multiple statements or links where
needed. Do not invent absent copyright statements or SPDX identifiers. Describe
nonstandard terms through their linked text, preserve alternatives and
exceptions, and record a selected option when relevant. Do not silently resolve
an unsettled licensing choice.
1. **Project name** as a level-two heading. An optional identification paragraph
may follow, such as `Selected module: MODULE at VERSION.` with the module and
version in backticks. For copied documentation, a concise `Material:` sentence
may identify the affected files and whether they were adapted or copied
without modification. A factual qualifier such as `(transitive dependency)`
is permitted. Limit optional identification prose to factual component and
material scope.
2. **Upstream copyright statement or statements**, preserving their wording.
Put multiple statements in separate paragraphs. Include this field when the
selected upstream source supplies an applicable statement; otherwise, omit it.
3. **SPDX license identifier or expression**, when applicable, using
`SPDX-License-Identifier: EXPRESSION`.
4. **License and notice links**, beginning `License and notices:`, separated
by ` | `, and ending with a period. Label local standard texts by license
name, such as `MIT` or `Apache 2.0`. Label upstream files by their actual
filenames, such as `COPYRIGHT`, `LICENSE`, `NOTICE`, or `PATENTS`.
5. **Source repository link**, rendered exactly as:
```markdown
[Source](COMMIT_URL).
```
Use the upstream [attribution example](https://gitea.maximumdirect.net/eric/policy/src/commit/a469273adda336625f96b9d769ea27b33999f146/examples/licensing/attribution.md)
to create the project’s `docs/oss/attribution.md`.
The example illustrates omitted fields and multiple copyrights and licenses.
Replace the uppercase placeholders and illustrative URLs with actual component
information. Local license links are relative to the destination
`docs/oss/attribution.md`; copy the applicable texts into its `licenses/`
directory. Do not copy example entries for components the project does not use.
These are five fields, not a line limit. Omit the SPDX field when no applicable
identifier exists; identify nonstandard terms through the linked text. Preserve
licensing alternatives and exceptions, and record a selected option when
relevant through the factual license identification. Do not invent an identifier
or silently resolve an unsettled licensing choice.
Keep entries factual. Do not add agent-authored explanations of license
expressions, file-level license scope, research performed, retained source
notices, or compliance conclusions. Avoid filler such as "none is invented
here" and embellished link labels such as "upstream LICENSE", "additional
patent grant", or "at the imported commit". Preserve actual required upstream
content; this rule excludes editorial additions, not required notices. Policy
interpretation and exception reasoning belong in the adopted OSS policy or an
appropriate decision record, not the attribution document.
## License Texts And Upstream Links
Link standard texts to local copies. Link project-specific notices to the
upstream copies at the exact commit corresponding to the imported version or
copied material. Moving branches and version tags are not substitutes for
commit-specific component links.
Link standard license texts to their local copies. Link project-specific
notices to their upstream copies. Every upstream repository or notice link must
identify the exact commit corresponding to the
imported version, including the selected source revision for copied material.
Classify material by content, not filename: an upstream LICENSE or COPYING can
mix standard text with project-specific notices or exceptions. Link both when
needed. Make incorporated companion texts directly reachable from each entry.
Classify material according to its contents. An upstream `LICENSE`,
`COPYING`, or similar file may contain project-specific notices or exceptions
in addition to standard wording. Link both the local standard text and the
commit-specific upstream file when needed. Make companion license texts directly
reachable from the entry.
For project-specific license and attribution material, direct upstream links
are accepted as satisfying reproduction, inclusion, or copy requirements under
the adopted house interpretation. Do not add local-copy requirements or request
repeated approval solely because a license uses those terms. This concerns
notice reproduction, not other concrete obligations such as marking modified
files or supplying corresponding source. Central entries do not authorize
stripping required notices from incorporated source.
For project-specific license and attribution material, this policy accepts
direct links to the applicable upstream texts as satisfying reproduction,
inclusion, or copy requirements. This is the adopted house interpretation.
Agents must apply it without adding local-copy requirements or seeking repeated
approval solely because a license uses those terms.
This interpretation concerns notice reproduction. Concrete obligations such as
marking modified files or supplying corresponding source receive their own
applicable treatment. Preserve required notices already present in incorporated
source files; the central attribution entry does not authorize stripping them.
## Discoverability
End README with friendly statements linking to the actual grant in COPYRIGHT
and to the attribution document. Each new checked-in release note ends with
links whose visible text is `COPYRIGHT | LICENSE | ATTRIBUTION`, using absolute
URLs at the release's immutable tag or corresponding immutable source revision.
These release links differ from imported-component links, which use commits.
End the project's README with two friendly statements linking to its actual
grant in `COPYRIGHT` and the attribution document. For example, for a project
with the stated license:
The release procedure owns concrete URL construction. If a hosted download page
is deliberately introduced, reuse the checked-in note and preserve its footer
there. This convention does not require a new hosted page, downloadable license
assets, source bundles, or another packaging format. Do not maintain duplicate
notice lists.
```markdown
PROJECT_NAME is licensed under the [LGPL v3.0 or later](COPYRIGHT).
[Open Source attribution notices](docs/oss/attribution.md) are available.
```
Each new checked-in release document and its hosted download page must end
with links whose visible text is `COPYRIGHT | LICENSE | ATTRIBUTION`. Use absolute
URLs to those files at the release's immutable tag or corresponding immutable source
revision so the links work in both locations. This release navigation is distinct
from upstream component links, which must use the imported commit.
Reuse the checked-in release document as the hosted release body where supported.
The release procedure owns concrete URLs and publication mechanics for delivering
the links through the selected channel. Do not maintain duplicate notice lists.
These conventions do not by themselves require separate downloadable license
assets, source bundles, or a different binary package format.
## Maintenance And Exceptions
Update attribution in the change that adds, removes, upgrades, copies, or changes
distribution of a relevant component. Inspect the selected source's actual
licenses/notices rather than assuming they are unchanged. Update entries,
commit links, and texts together; retain history through Git.
Update project licensing statements together when the author changes the grant
or its scope. Update upstream notices in the change that adds, removes, upgrades,
copies, or changes the distribution of a relevant component. Check the selected source's license
and notices rather than assuming they are unchanged. Update affected entries,
commit links, and texts; retain historical versions through Git.
Review affected statements, standard-text fidelity, and link destinations during
ordinary change review. Future-release URLs can be checked against candidate
paths before the tag exists remotely. Do not add publication-time network checks,
CI polling, recurring legal sign-offs, mandatory scanners, or an SBOM just to
implement this policy. Attribution maintenance is preparation, not a release gate.
Check affected statements, faithful standard-text copies, and link destinations
as part of ordinary change review. For a future release tag, check intended file
paths against the candidate source; the tag need not resolve remotely before
publication. Do not add publication-time network checks, CI polling, recurring
legal sign-offs, mandatory scanners, or a full software bill of materials merely
to implement this policy.
Raise exceptions only for concrete obligations or unresolved rights questions
the House Format does not address, including distribution-specific issues.
Record their selected treatment here or in the affected entry and resolve them
before the affected use/distribution, without reopening settled choices.
Common ownership is not a blanket exception. Adapted standard policy documents
retain their upstream documentation license and identify modifications in their
provenance entry; this does not change Promptkit's library grant.
Use the House Format routinely. Raise an exception only for a concrete obligation
or unresolved rights question it does not address, including any arising from a
particular distribution channel. Record the issue and selected treatment in the
adopted OSS policy or an appropriate decision record. Attribution entries
contain the resulting factual identification and required material, without
repeating that reasoning. Resolve the issue before the affected use or
distribution without reopening settled choices.
Common ownership of projects is neither required nor a blanket exception.
The maintainer-provided standard policy templates are internal and non-public.
Their attribution entry identifies that source in prose and retains the
applicable license locally; do not link to or require publication of the
internal repository.
Apply the formatting convention prospectively. Do not rewrite published tags,
notes, binaries, or checksums to retrofit it. Address actual omissions deliberately
under the applicable terms and the [release correction policy](release.md#immutability-and-failure-recovery).
Ordinary attribution maintenance and release links are preparation work, not new
release approval gates.
Apply conventions prospectively. Do not rewrite published tags, notes, binaries,
or checksums to retrofit formatting. Handle actual omissions deliberately under
applicable terms and the [release correction policy](release.md#immutability-and-failure-recovery).
## Promptkit Adoption
Promptkit retains its LGPL-3.0-or-later library grant in
[COPYRIGHT](../../COPYRIGHT). Adapted policy documentation has the separate
scope identified there and in [attribution](../oss/attribution.md).
Promptkit publishes source-only Go module tags on Gitea, with no binaries or
separate hosted download page. The [release procedure](../release.md) owns
publication and concrete footer URLs. The hosted-page guidance applies if such
a channel is deliberately introduced; it does not require creating one.

View File

@@ -1,30 +1,229 @@
# Release Policy
**Base Policy Version:** 1.2
**Base Policy Version:** 1.4
**Last Updated:** 2026-09-26
**Last Updated**: 2026-09-27
## Purpose And Adoption Scope
**Upstream Source:** [release.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/release.md)
This policy owns Promptkit's release requirements and compatibility rules.
The [release procedure](../release.md) owns the commands used to satisfy them.
This scoped adoption applies to releases prepared after its adoption on
2026-09-23. Published tags and historical notes are not rewritten to retrofit
the new note format.
## Purpose
## Release Model And Compatibility
This policy defines a proportionate workflow for validating and publishing an
intended source revision under a fresh version. Releases should require little
routine intervention, avoid repeated work, and preserve published contents.
Promptkit publishes source-only Go module releases from committed, pushed
`main` candidates through annotated `vMAJOR.MINOR.PATCH` tags. It publishes
no binaries, hosted releases, or other artifacts and uses maintainer-run
validation rather than hosted CI. Versions must be unused locally and on
`origin`; the existing procedure accepts stable version tags only.
The repository-specific commands belong in `docs/release.md`. Historical release
summaries belong in `docs/releases/`. The procedure must agree with the adopted
policy and link to it rather than introduce additional release requirements.
Before v1, minor releases may change the public API and patches preserve
compatibility within their minor line, subject to the catalog-data provision
below. Release notes must identify compatibility impact, public API changes,
and required consumer action. The source tag identifies the exact release;
the checked-in note at that tag records its historical summary.
## Release Model And Completion
A source release identifies one exact committed source revision through an
immutable version tag and a version-matched, checked-in release note. The
primary release branch is `main` unless the adopted policy specifies otherwise.
The procedure must name the authoritative publication remote.
A successful push of the intended tag, with its source commit available on the
authoritative primary branch, completes the release. The source commit may be
pushed before the tag or published together with it through an atomic push.
The procedure must choose and document the supported approach.
Release commands must finish at successful publication. They must not wait for
or poll CI, require successful CI status, inspect downloadable artifacts, or
check consumer installation or Go module availability as completion gates.
A failed or ambiguous push requires resolving the authoritative remote state
before declaring success or retrying; successful pushes do not require a
separate remote read-back gate.
Repositories may publish source only. Additional binaries, images, packages,
checksums, or other declared deliverables may be published asynchronously from
the tagged source. The procedure must identify those deliverables and the
responsibility of their automation. Published deliverables must be traceable
to that source and identify the same version.
Track source-release completion and artifact delivery separately. Report delivery
failures and optional investigations under artifact-delivery status.
## Versioning And Compatibility
Stable releases use Semantic Versioning in the form `vMAJOR.MINOR.PATCH`, with
no leading zeroes. Prereleases require an explicit adopted policy defining
their format and use.
The selected version must be unused locally and on the authoritative remote.
Do not require queries to every configured remote or publication service.
A version known to have already been published must not be reused, even if the
normal guards do not inspect that channel. Adding a distribution channel
requires a deliberate update to the policy and procedure where needed.
For releases at or after `v1.0.0`:
- a major release may make intentional incompatible changes;
- a minor release may add backward-compatible behavior; and
- a patch release may make backward-compatible fixes.
Before `v1.0.0`, a minor release may change a documented public, operational,
configuration, data, integration, or compatibility contract when the release
note explains the impact and required consumer action. A patch release must
not intentionally break those contracts within its minor line.
Additional compatibility promises and deliberate exceptions belong in the
adopted policy or canonical contract documentation. Release notes explain
changes; they do not redefine current compatibility contracts.
## Release Documentation
Every new release must include `docs/releases/<version>.md` in the tagged
source tree. Use the complete version tag as the filename and begin with a
title containing the product name and that exact version.
Include a short summary and any relevant compatibility impact or upgrade
instructions. Use the Promptkit section structure below and keep the note
proportionate to the change.
Keep exact current contracts in their canonical documents and update affected
documentation with the behavior change. Link to those documents from the note
where useful. If annotated tags are used, derive their summary from the
checked-in note.
Include the release footer defined by the [OSS policy](oss.md#discoverability)
in each new release document and preserve it on the hosted download page. The
release procedure defines the concrete URLs and publication mechanics. Prepare
these links with the release note; they do not add a network check or approval
gate.
Handle version selection, release-note content, and retirement of completed
roadmaps during preparation. Preserve planned and deferred work and repair
affected links when retiring roadmaps. Reuse ordinary change review for
unintended files, credentials, and private data. Scope release-content review
to the tagged source.
## Candidate Validation
Provide one publication command that validates the final committed candidate,
checks publication state, and publishes the intended refs. A validation-only
mode or command may be provided for diagnosis, but must not be a required step
before publication repeats the same checks.
Run the complete deterministic, offline default test suite once for the final
candidate, following the [testing policy](testing.md). For repositories that
publish builds, verify the required build targets. Select those targets for
the application's actual support commitments; native testing on each platform
and access to particular hardware are not default release prerequisites.
Additional gates must address a concrete risk that the existing checks do not
adequately cover. Do not automatically require linting, race testing, module
hygiene audits, documentation audits, credential scans, example validation, or
expanded platform matrices solely because they are possible. Retain such
checks when their distinct protection justifies their cost. A one-time migration
check must not silently become a permanent release requirement.
Release tests must not require real credentials, mutable application services,
paid APIs, provider probes, or inference calls. The procedure may describe dependency
preparation separately from offline validation.
Avoid duplicate validation across release phases. Successful checks for the
same final candidate need not be repeated solely to satisfy another phase;
any reuse must clearly apply to that exact candidate. Persistent validation
caches are not required. If the candidate changes, validate the new candidate.
Cheap publication-state checks may be repeated immediately before pushing
because local or remote state may have changed during validation.
Automate mechanical guards. Human review gates are strongly disfavored: add one
only through an explicit project-specific policy decision explaining the
concrete risk and why ordinary change review or automation is insufficient.
Do not add routine confirmations for checks the command can perform itself.
## Publication Guards
Automate the checks needed to publish the intended source under a fresh version:
- the version has the adopted format and is unused locally and on the
authoritative remote;
- the worktree and index are clean on the primary release branch;
- the source being published is the exact candidate that passed validation,
including its matching release note;
- the remote primary branch already identifies that candidate, or an atomic
source/tag push will advance it to that candidate without discarding remote
history; and
- the tag identifies the validated candidate and only the intended refs are
published.
Fetch the authoritative branch and tag state as needed for these checks.
Do not force branch or tag updates to bypass a failed guard. Publish explicit
refs rather than all local tags. Do not require CI success, publication-service
availability checks, or a separate manual confirmation for each guard.
## Immutability And Failure Recovery
Published version tags and published contents are immutable. Do not move,
overwrite, recreate, or delete a published tag, or replace a published artifact,
checksum, signature, provenance statement, or release note under that version.
The checked-in note at the immutable tag is that release's historical record.
If validation fails before publication, correct and commit the candidate and
rerun the publication command. If a push fails or its outcome is ambiguous,
inspect the authoritative remote before retrying. Remove an unpublished local
tag only after confirming it was never published or used by a release channel.
An unavailable remote leaves publication status unknown; do not overwrite a tag
to resolve uncertainty.
Transient network, authentication, CI, or consumer-availability failures do not
by themselves require a new version. Asynchronous delivery may be retried from
the same immutable tag. Missing artifacts may be published later, provided
existing published contents are not replaced and the additions remain consistent
with the tagged source and any already-published checksum manifest. If that
cannot be achieved, publish a new version.
A defect requiring changes to published source or contents requires a corrected
commit, a new semantically appropriate version, a matching release note, and
the release procedure again. Leave the original release intact. A channel's
yank, deprecation, or advisory mechanism may warn consumers away from a defective
release without replacing its contents or reusing its version.
## Repository Release Procedure
Maintain `docs/release.md` as a short, executable maintainer runbook. It must
identify:
1. prerequisites, primary branch, and authoritative remote;
2. version and release-note preparation;
3. one publication command with its selected local checks and automated guards;
4. declared deliverables, required build targets, and automation responsibilities;
5. the successful-push completion point and separately reported delivery status;
and
6. failure handling, ambiguous-push recovery, and asynchronous delivery retries.
Use copyable commands where practical. Keep optional diagnostics and consumer
verification clearly optional. The procedure must not turn preparation work,
ordinary change review, or post-publication investigation into additional
release gates.
## Promptkit Release Configuration
The release-note requirements were adopted on 2026-09-23 and licensing
navigation on 2026-09-26. Apply them prospectively; previously published notes
and tags retain their original records.
Promptkit publishes source-only Go modules from main to origin on Gitea, using
annotated stable SemVer tags. It has no binaries, hosted download pages, or
asynchronous deliverables. The source commit is pushed through main before the
tag. The [release procedure](../release.md) owns commands and recovery.
The existing publication script requires Summary, Compatibility, Upgrade, and
Changes headings beneath a title of `# Promptkit vMAJOR.MINOR.PATCH`. State
“No action required” or “None” where appropriate. This retained local structure
matches the automated release-note guard; changing it is separate work.
The [publication script](../../scripts/release.sh) runs the
[local checks](../../scripts/release-check.sh) once for the final committed
candidate: race-enabled tests, vet, three offline consumer examples, tracked
Go formatting, Markdown links, module/workspace hygiene, whitespace, and a
credential-pattern scan. These protect concurrency, assembled library use,
source distribution, and documentation integrity. A separate ordinary test run
or standalone build is not an additional release gate. Candidate, tag, and
synchronized-main checks remain automated. No CI or module-availability wait
is required. Licensing footer URLs are preparation work, not a network gate.
## Catalog Profile Compatibility
@@ -51,101 +250,3 @@ migrations.
Catalog adoption and release validation remain local and offline. They do not
probe provider endpoints, request model lists, or make inference calls.
## Release Documentation
Every new release must include `docs/releases/<version>.md` in the tagged
source, using the complete tag as the filename. Its title contains Promptkit
and that exact version. Use this structure:
```markdown
# Promptkit vMAJOR.MINOR.PATCH
This release ...
## Summary
## Compatibility
## Upgrade
## Changes
```
State “No action required” or “None” where appropriate rather than omitting a
section. Keep exact current contracts in their canonical documents and link
to them. Update affected canonical documentation in the same candidate.
These notes are historical records, not optional disposable guidance. Published
release-note contents must not be replaced under the same version. Corrections
belong in a subsequent release. Annotated-tag messages complement the checked-in
note: generate the annotation from that note and the validated commit rather
than maintaining a second summary.
## Validation And Publication
New release notes prepared after the 2026-09-26 OSS-policy adoption end with
the [licensing footer](oss.md#discoverability). The release procedure owns
absolute, release-pinned URLs. This is prospective preparation work, not a new
approval or network-validation gate; historical notes and tags are not reformatted.
The [publication script](../../scripts/release.sh) runs the
[local checks](../../scripts/release-check.sh) once for the final candidate.
Retain clean-candidate, synchronized-main, unused-version, module-hygiene,
release-note structure, and tag-target checks. Publish only the selected tag
ref. Changing the candidate requires fresh validation. No provider endpoint
probes or inference calls are release checks.
Release-note content and retirement of completed roadmaps are preparation
responsibilities, not separate human sign-off gates. Preserve planned and
deferred work and repair links. Ordinary change review covers unintended files,
secrets, and private data; do not repeat it as a release-specific approval.
A successful tag push to origin completes the release. Never wait for CI to
finish or succeed. CI results, remote read-back, and Go module availability
are informational follow-ups, not completion gates. An ambiguous push result
requires confirming publication on origin, not waiting for CI.
## Immutability And Failure Recovery
Never move, replace, recreate, or delete a published tag or replace published
release contents. If validation fails before publication, correct the candidate,
rerun the complete checks, and repeat the publication guards. Remove an
unpublished local tag only after confirming it was not published.
If publication or subsequent investigation reveals a defect, leave the
published release intact. Correct the defect on main, select a new patch or
other semantically appropriate version, write its matching release note, and
repeat the complete procedure. If publication status is uncertain, inspect
the authoritative remote before deciding whether a tag is unpublished; do not
overwrite it to retry.
Transient network, authentication, CI, or module-availability failures do not
by themselves warrant a new version. Investigate separately and publish a
corrective release only when published contents actually need changing.
## Policy Exceptions And Deliberate Scope
This adoption follows the maintainer-approved lean workflow rather than every
base-policy guard:
- Collision checks concern origin, the authoritative publication destination,
rather than every configured remote. Local and origin tag checks, synchronized
main, and an explicit selected-ref push protect the actual release channel.
- Automated local validation and publication-state checks replace manual
mechanical inspections and separate human sign-offs. Ordinary change review,
editorial responsibility for release notes, and roadmap housekeeping remain;
repeating them as approvals would add friction without distinct protection.
- Successful tag publication completes this source-only release. CI can be slow
or unreliable, and module availability depends on external services. Neither
is a completion gate. Local candidate validation and explicit tag-target
checks protect publication; ambiguous push results are resolved against origin.
Actual defects are corrected in a new release, never by replacing a tag.
Ignored local files are not prohibited merely because they exist: they are not
included in the tagged commit. The clean-candidate guard and ordinary review of
tracked changes protect the published contents.
The catalog profile provision preserves an already approved compatibility
boundary; it does not extend volatility to backend definitions or runtime
contracts. Further release-governance changes require deliberate approval.

View File

@@ -1,43 +1,32 @@
# Testing Policy
**Base Policy Version:** 1.0
**Base Policy Version:** 1.3
**Last Updated:** 2026-09-23
**Last Updated**: 2026-09-27
**Upstream Source:** [testing.md](https://gitea.maximumdirect.net/eric/policy/src/commit/d84a38ebab26f048cafbe59a2246700c35d29843/docs/policy/testing.md)
## Purpose
Our tests exist to make **incorrect changes expensive and correct changes cheap**.
We do not optimize for test count, line coverage, exhaustive isolation, or the fewest possible tests. We optimize for sufficient confidence in important behavior while imposing as little unnecessary friction as possible on future development.
We optimize for sufficient confidence in important behavior while minimizing friction on future development.
## Every test has a cost
Testing is not an unqualified good. Every test imposes both an immediate cost and a continuing lifetime cost.
A test must be:
- written and reviewed;
- understood by future maintainers and coding agents;
- executed in maintainer-run validation;
- diagnosed when it fails;
- updated when legitimate behavior changes;
- maintained as fixtures, APIs, and dependencies evolve; and
- removed or rewritten when it becomes redundant, brittle, misleading, or obsolete.
Tests also create cognitive and architectural friction. They can constrain refactoring, duplicate policy, slow feedback loops, add noise to failures, and cause harmless implementation changes to require unrelated edits across the suite.
Every test has an immediate and continuing cost: writing, review, execution,
diagnosis, fixture maintenance, and updates when legitimate behavior changes.
Tests can also constrain refactoring, duplicate policy, and add failure noise.
A test is warranted only when the confidence it provides justifies these costs.
Apply this cost-benefit analysis at two levels:
Evaluate that tradeoff at two levels:
1. **Per test:** What realistic defect does this test detect, how consequential would that defect be, and is that protection worth the test's lifetime cost?
2. **Across the suite:** Does this collection provide materially more confidence than a smaller, simpler suite would?
The preferred test suite is a **lean suite that provides sufficient confidence in the risks that matter, without redundant or low-value tests**. We seek sufficient confidence with the least unnecessary testing friction, not the fewest possible tests.
Some friction is intentional. Tests should make dangerous changes—such as breaking compatibility, corrupting data, violating security boundaries, or reintroducing subtle bugs—require deliberate review. Keep ordinary internal changes inexpensive.
Some friction is intentional. Tests should make dangerous changes—such as breaking compatibility, corrupting data, violating security boundaries, or reintroducing subtle bugs—require deliberate review. They should not make ordinary internal changes needlessly expensive.
The cost of a test is not a reason to omit testing by default. Do not cite maintenance cost abstractly. When omitting a plausible test, be able to state why the protected failure is low-risk, already covered, obvious, reversible, or cheaper to detect elsewhere. For consequential, subtle, or difficult-to-observe behavior, the presumption should favor testing.
Evaluate maintenance cost against the specific failure being protected. When omitting a plausible test, be able to state why the protected failure is low-risk, already covered, obvious, reversible, or cheaper to detect elsewhere. For consequential, subtle, or difficult-to-observe behavior, the presumption should favor testing.
## Default testing style
@@ -51,6 +40,12 @@ Use a **classical/Detroit-style** approach:
Examples of appropriate seams include clocks, randomness, subprocesses, remote APIs, object storage, email, and paid LLM calls.
## Promptkit Test Boundaries
Promptkit is a library without a CLI or hosted CI. The execution requirements,
facade test ownership, public error guidance, and offline consumer examples
below tailor the standard to that boundary.
## Test execution requirements
Promptkit currently uses maintainer-run validation rather than hosted CI.
@@ -110,7 +105,7 @@ Use each test type where it protects a distinct risk:
## What deserves tests
Prioritize tests for:
To the extent the following features are present in the application, prioritize tests for:
1. Public and package-level contracts.
2. Domain rules and important invariants.
@@ -118,17 +113,29 @@ Prioritize tests for:
4. Failure handling, cancellation, retries, recovery, and partial success.
5. Serialization, schemas, compatibility, and round trips.
6. Previously observed or plausible regressions.
7. Representative integration and consumer workflows.
7. Representative integration and end-to-end workflows.
A package-level contract is behavior relied upon by another package or major collaborator, not every observable detail of a package implementation.
A package-level contract comprises behavior relied upon by another package or major collaborator.
For behavior involving **data integrity, destructive operations, compatibility, security, concurrency, idempotency, or recovery**, presume that durable tests are required unless the behavior is already credibly protected at another layer.
Choose tests for requested behavior and established contracts. Tests must not
introduce new restrictions, guarantees, or threat scenarios merely because they
could be asserted. Follow the [agent policy](agents.md) when deciding whether a
behavior belongs to the contract; current implementation behavior alone is not
sufficient authority.
For established behavior involving **data integrity, destructive operations,
compatibility, security, concurrency, idempotency, or recovery**, favor durable
tests for realistic failures whose consequences warrant protection. Identify
the failure and the existing contract it would violate. These categories call
for attention, not an automatic test inventory or permission to strengthen the
application's guarantees. Account for protection already provided at other layers.
Do not add tests merely because a function, branch, or line exists. Do not add a test when the same meaningful risk is already adequately protected elsewhere.
## Choose the right test boundary
Test through the narrowest stable boundary that expresses the behavior clearly.
Choose the narrowest stable boundary that expresses the behavior clearly and
provides durable confidence with minimal incidental coupling.
This is often the package API, but it may instead be:
@@ -136,30 +143,15 @@ This is often the package API, but it may instead be:
- a package-level operation when several internal collaborators jointly produce the behavior; or
- a larger integration boundary when correctness emerges from interaction with a real dependency.
Do not force all behavior through oversized consumer-workflow tests. Do not
test every private helper merely because it exists. Choose the boundary that
gives durable confidence with the least incidental coupling.
## Test behavior, not implementation
A test should protect a decision, contract, or invariant—not memorialize the current implementation.
A test should protect a selected decision, contract, or invariant.
Before adding or retaining a test, ask:
A test is suspect when it primarily detects changes to private constants or
helpers, equivalent operation ordering, incidental formatting, equivalent
algorithms, or internal structure that preserves behavior.
> What realistic defect would this test catch?
A test is suspect when its main purpose is to detect that someone:
- changed an internal constant;
- renamed or split a private helper;
- reordered equivalent internal operations;
- changed incidental formatting;
- replaced one correct algorithm with another; or
- refactored internal object structure without changing behavior.
Refactoring should normally require no test edits unless the refactored structure is itself part of the contract.
A test can be factually correct and still have negative value. Accurately describing current behavior is not enough; the protected behavior must be important enough to justify the future friction.
Behavioral tests should normally remain stable through internal refactoring that preserves the contract.
## Expected effects of different changes
@@ -176,13 +168,13 @@ A test failing is not the same as a test needing to be edited. Many tests may co
## Separate mechanism from policy
Configurable thresholds and defaults must not be duplicated throughout the test suite.
Derive expectations from test-controlled configuration and relationships.
The following fragments are illustrative rather than standalone Go programs.
Do not encode an internal concurrency limit indirectly:
For this example, assume the selected contract rejects a request to start more
processes than a configured limit. Do not encode its production default indirectly:
```go
// Production policy:
// Production default:
const maxConcurrency = 4
// Brittle test:
@@ -200,37 +192,40 @@ require.NoError(t, runner.Start(limit))
require.ErrorIs(t, runner.Start(limit+1), ErrTooMuchConcurrency)
```
The test should prove:
For this selected contract, the test should prove:
- the configured limit is accepted; and
- one beyond the configured limit is rejected.
Rejection is not a general requirement for concurrency control. If the selected
contract queues excess work or limits active workers while accepting larger
batches, test that behavior instead. Do not add rejection or a configurable
limit solely to reproduce this example.
The production default should be tested exactly only when its literal value is itself a public, operational, safety, protocol, or compatibility requirement.
Apply the same rule to limits, timeouts, capacities, retry counts, and ranges: test relationships and behavior, not duplicated literals.
Apply the same rule to limits, timeouts, capacities, retry counts, and ranges: test relationships and behavior.
For concurrency limits, test both kinds of behavior when relevant:
For established concurrency limits, distinguish these behaviors where the contract requires them:
1. **Configuration enforcement:** invalid or excessive requested values are handled correctly.
1. **Configuration handling:** values are accepted or rejected according to the selected contract.
2. **Runtime enforcement:** observed peak concurrency never exceeds the configured limit.
Use a test-controlled limit and measure the behavior relative to that limit. Do not merely assert today's default value.
Use a test-controlled limit and measure the behavior relative to that limit.
## Avoid semantic duplication across layers
## Assign each behavior a test owner
Each behavior should have a clear test owner.
Where these layers exist, assign ownership as follows:
- Parser tests own parsing cases.
- Validator tests own validation rules.
- Domain tests own transformations and invariants.
- Boundary tests own external integration behavior.
- Adapter tests own external integration behavior.
- Orchestrator tests own coordination and failure propagation.
- Consumer-workflow tests prove that representative assembled library use
works.
- External-package root tests own the public Go facade contract.
- Consumer-workflow tests prove that representative assembled library use works.
Higher-level tests should not repeat every lower-level case. A single intentional policy change should not require unrelated edits across many test files.
Tests that are individually reasonable may still be collectively redundant. Evaluate the marginal value of each additional test in light of the protection already provided by the rest of the suite.
Give each layer its distinct responsibility and keep expectation updates localized to the contract being changed.
## Use test doubles deliberately
@@ -251,111 +246,78 @@ Mocks are appropriate when the contract includes facts such as:
- an expensive API is called no more than once; or
- a security audit event is emitted.
Do not use mocks merely to isolate every object or reproduce the implementation's call graph.
## Go-specific guidance
Use:
Select techniques by behavior and risk:
- table-driven tests for meaningful behavioral categories and boundaries;
- `t.TempDir()` for real filesystem behavior;
- `httptest.Server` for realistic HTTP interactions;
- fuzz tests for parsers, normalization, path handling, and broad input spaces;
- fuzz tests when exploring broad input spaces, such as parsers or path handling, adds useful protection beyond representative cases;
- golden files only when the complete output is intentionally stable;
- integration tests where correctness depends on component interaction; and
- a small number of representative consumer-workflow tests.
- a small number of representative end-to-end tests when assembled workflows expose meaningful risks not adequately covered at narrower boundaries.
Avoid exact error-string assertions unless the wording is itself contractual. Prefer `errors.Is`, `errors.As`, typed errors, or structured error fields.
Prefer `errors.Is`, `errors.As`, typed errors, or structured error fields. Use exact error-wording assertions only when the wording is contractual.
At public API boundaries, prefer stable error identity, structured values, and
the smallest semantic fragment needed to identify the failure. Do not snapshot
complete diagnostic wording unless it is contractual.
the smallest semantic fragment needed to identify the failure. Use complete
diagnostic snapshots only when the wording is contractual.
Golden-file updates must require an explicit local flag. Ordinary validation
runs must never update golden files automatically, and maintainers must inspect
the semantic diff before accepting an update.
Golden-file updates must require an explicit local flag. Ordinary validation runs must never update golden files automatically, and reviewers must inspect the semantic diff before accepting an update.
Keep tests readable and direct. Test helpers and fixture frameworks must earn their own maintenance cost; do not build elaborate test infrastructure for small or isolated needs.
Keep tests readable and direct. Size test helpers and fixture frameworks to the maintenance benefit they provide.
## Coverage
Coverage is a diagnostic, not a target.
Use coverage to locate untested critical branches and unexpectedly weak packages. Evaluate test quality through meaningful behavioral protection and choose coverage according to risk.
Use it to find untested critical branches and unexpectedly weak packages. Do not write low-value tests solely to increase a percentage, and do not infer test quality from coverage alone.
Pure domain logic will often warrant higher coverage than facade wiring or
external adapters. Uneven coverage is acceptable when it reflects risk.
Increasing coverage is valuable only when the newly covered behavior protects a meaningful risk at an acceptable cost.
Pure domain logic will often warrant higher coverage than facade wiring or external adapters. Uneven coverage is acceptable when it reflects risk.
## Regression tests
A bug fix should normally include a regression test that fails before the fix and passes afterward.
Retain the test when the defect could realistically recur and its consequences justify the ongoing cost. Prefer the narrowest durable test of the violated contract or invariant; do not preserve accidental implementation details from the original bug.
Retain the test when the defect could realistically recur and its consequences justify the ongoing cost. Prefer the narrowest durable test of the violated contract or invariant.
Not every historical bug requires a permanent test. If the underlying design has made recurrence impossible, the test has become redundant, or a stronger invariant test now subsumes it, remove or consolidate it.
Remove or consolidate a test when the underlying design has made recurrence impossible, its protection is redundant, or a stronger invariant test subsumes it.
## Deleting or rewriting tests
Tests are maintained code, not permanent historical artifacts.
Delete or rewrite a test when its maintenance cost exceeds the confidence it provides.
Delete or rewrite tests when maintenance cost exceeds their continuing
protective value.
Strong candidates include tests that:
- require updates after harmless internal changes;
- directly assert private constants without protecting a real contract;
- duplicate the same policy across several layers;
- assert incidental private constants or snapshot large amounts of incidental output;
- verify mock choreography rather than outcomes;
- snapshot large amounts of incidental output;
- test trivial private helpers already exercised through stable package behavior;
- protect risks already covered more effectively elsewhere;
- duplicate policy across layers, retest trivial helpers covered by package
behavior, or protect risks covered more effectively elsewhere;
- are flaky, misleading, obsolete, or disproportionately expensive to diagnose; or
- no longer correspond to a plausible failure mode.
Several brittle tests may encode one genuine requirement. Replace them with one durable behavior-level or invariant test rather than preserving all of them.
Deleting a low-value test can improve the quality of the suite by reducing noise, maintenance burden, and friction around legitimate change.
Several brittle tests may encode one genuine requirement. Consolidate them into one durable behavior-level or invariant test.
## Reviewing a proposed test
Use the following questions when the value, boundary, or durability of a proposed test is not self-evident. Significant test additions should be reviewable against them, but written answers are not required for every routine test.
Use the following questions when a proposed test needs closer evaluation. Significant test additions should be reviewable against them; written answers are optional for routine tests.
1. What realistic defect would it catch?
2. How likely is that defect?
3. How consequential would it be?
4. Is the behavior already protected elsewhere?
5. At which layer should this behavior be owned?
6. Does the test assert a durable contract or an incidental implementation detail?
7. Could the implementation be refactored without changing the behavior and without editing this test?
8. What should cause this test to fail?
9. What legitimate changes should not cause this test to fail?
10. What ongoing maintenance, execution, and diagnostic cost will the test impose?
11. Is there a smaller or more direct test that protects the same risk?
Do not add the test when its expected lifetime cost exceeds its expected protective value.
When deciding not to test plausible behavior, record or be able to explain why the risk is low, already protected, obvious, reversible, or cheaper to detect elsewhere.
1. Which requested behavior or established contract does it protect, and what realistic defect would violate it?
2. How likely and consequential is that defect, and is it already protected elsewhere?
3. What is the narrowest stable boundary that can test it clearly?
4. Would a correct internal refactor or a change to an incidental default require test edits?
5. Does its confidence justify its maintenance, execution, and diagnostic cost, or would a smaller test suffice?
## Definition of sufficient
A test suite is sufficient when:
- important contracts and invariants are protected;
- meaningful boundaries and failure modes are exercised;
- realistic and consequential regressions are credibly protected against silent recurrence;
- behavior involving data integrity, destructive operations, compatibility, security, concurrency, idempotency, and recovery is credibly protected;
- important external boundaries have realistic integration coverage;
- representative complete workflows are tested;
- failures provide useful signal rather than redundant noise;
- legitimate internal changes usually do not require test edits; and
- important requested behavior and established contracts are protected against realistic, consequential failures;
- boundaries and component interactions have coverage where they introduce meaningful risks not protected elsewhere;
- failures provide useful, distinct signals;
- tests usually remain stable through legitimate internal changes; and
- additional tests would mostly repeat existing protection or preserve inconsequential implementation details.
Sufficiency is a risk judgment, not a coverage percentage or test count.
Reassess it as the library, its consumers, and the consequences of failure
evolve.
The governing rule is:
> Test heavily where failure is consequential, subtle, or difficult to detect after the fact. Test lightly where failure is obvious, reversible, and inexpensive—and retain no test whose lifetime cost exceeds the confidence it provides.
Reassess sufficiency as the application, its users, and the consequences of
failure evolve.

38
docs/releases/v0.11.2.md Normal file
View File

@@ -0,0 +1,38 @@
# Promptkit v0.11.2
This patch aligns repository policies with the current standard and clarifies
licensing and attribution for the copied policy documents.
## Summary
The update adds policy provenance, selective contributor reading guidance, and
consistent upstream policy language while preserving Promptkit-specific rules.
## Compatibility
No public API, runtime behavior, dependency, or catalog changes are introduced.
Promptkit remains LGPL-3.0-or-later. The [project grant](../../COPYRIGHT)
identifies the separate CC-BY-SA-4.0 scope for copied or adapted policy material;
the [attribution notices](../oss/attribution.md) identify the affected documents.
The existing release checks and successful-tag-push completion point are unchanged.
## Upgrade
```sh
go get gitea.maximumdirect.net/eric/promptkit@v0.11.2
```
No code or configuration migration is required from v0.11.1.
## Changes
- Align adopted policies with Standard Repository Policies while retaining
library-specific requirements and deliberate local decisions.
- Add per-document upstream source links and the
[policy-set provenance document](../policy/meta.md).
- Add selective policy reading to the [contributor guide](../development.md)
and update agent orientation.
- Standardize factual attribution entries and identify copied-policy licensing.
- Retire the completed policy-alignment roadmap.
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.2/COPYRIGHT) | [LICENSE](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.2/LICENSE) | [ATTRIBUTION](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.11.2/docs/oss/attribution.md)

74
docs/releases/v0.12.0.md Normal file
View File

@@ -0,0 +1,74 @@
# Promptkit v0.12.0
This release replaces Promptkit's configuration loader with ConfigKit and
migrates maintained catalogs to keyed `/v3` assets. It intentionally changes
the public construction API and configuration format.
## Summary
Backends, profiles, and versioned prompts now share one layered configuration
snapshot built when `NewEngine` is called. ConfigKit supplies composition,
inheritance, presence, and source history; Promptkit validates the final
effective definitions and captures prompt content and templates. Consumers can
inspect the captured configuration explicitly without triggering a reload.
## Compatibility
This is an incompatible minor release under Promptkit's pre-1.0
[versioning policy](../policy/release.md#versioning-and-compatibility). The
former source options, `Config.PromptDir`, `Config.ProfileDir`, and the old
profile/backend convenience values have been removed. Configuration documents
now use literal keys beneath `backends`, `profiles`, and `prompts`; embedded
`id`/`version` fields and `base_profile` are no longer accepted. Source groups
compose strictly within a group and overlay in the supplied order. Explicit
zero, false, empty, and null values can now differ from omission; maps overlay
recursively, and an empty map does not clear an inherited map.
All final definitions, including unused ones, are validated during engine
construction. Prompt text and static templates are captured then; an existing
engine does not automatically reload changed configuration or message files.
Request artifacts, schemas, credentials, and provider operations retain their
runtime boundaries. See the [format reference](../formats.md) for the current
configuration contract and the [consumer migration guide](../consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api)
for API and source-selection changes.
Maintained OpenRouter and Rakestrawhome catalogs are selected as `/v3` modules,
both at `v3.20260930.0`. Their asset format changed; consumers that read catalog
assets directly must adopt the keyed format. The maintained Promptkit profile
IDs remain available under the [catalog compatibility policy](../policy/release.md#catalog-profile-compatibility),
which does not promise fixed profile data.
## Upgrade
```sh
go get gitea.maximumdirect.net/eric/promptkit@v0.12.0
```
Replace removed source and backend registration options with
`Config.Sources`, explicitly ordered ConfigKit underlays, and optionally a
highest-priority main input. Use `DirectoryInputs` or `FilesystemInputs` for
selected directories, `ProfileSource` and `BackendSource` for typed
programmatic values, or `configkit.Mapping` for explicit null and combined
documents. Move profile `base_profile` to `extends` and embedded identities to
mapping keys. Review zero/false/null values and any assumptions that replacing
a model or backend discards unrelated explicit settings. Rebuild the engine
when definitions or referenced message files change. The
[consumer guide](../consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api)
provides the migration details and maintained examples.
## Changes
- Add ConfigKit v0.5.0 as the configuration boundary and consume released
OpenRouter and Rakestrawhome `/v3` catalog modules.
- Add keyed backend, profile, and prompt definitions with final-value domain
validation, explicit presence/null behavior, and source-relative content
capture.
- Add `ConfigurationSources`, directory discovery helpers, programmatic
profile/backend source builders, and read-only configuration inspection.
- Preserve execution workflows while rendering captured templates, applying
backend defaults after inheritance, and honoring explicit suppression of
provider settings.
- Remove superseded loader, catalog preflight, and inheritance machinery;
update examples, format guidance, and internal documentation.
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/COPYRIGHT) | [LICENSE](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/LICENSE) | [ATTRIBUTION](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/oss/attribution.md)

View File

@@ -42,20 +42,21 @@ checkpoint equality or when a broader semantic identity design is selected.
### Eager Source Validation
**Reason for deferral:** Exact prompt and profile inspection may already
provide a sufficiently small validation surface. Experience from downstream
adoption should establish whether an engine-wide operation would add enough
value to justify its broader contract.
provide a sufficiently small validation surface alongside construction-time
definition validation. Experience from downstream adoption should establish
whether a separate operation would add enough value to justify its broader
contract.
Promptkit could provide an explicit offline operation that discovers and
structurally validates configured prompt, profile, and schema sources without
model generation. The normal `NewEngine` path would remain lazy.
Promptkit could provide an explicit offline operation that also checks schema
resources without model generation. `NewEngine` already validates effective
definitions and captures prompt content; schema resources remain
request-dependent.
An eager operation would need coherent handling for duplicate prompt IDs and
versions, strict YAML decoding, referenced content files, profile/backend
membership, schema syntax and transitive references, context cancellation,
and source-specific public errors. Credential declarations must remain
separate from credential values; checking current environment availability,
if supported at all, should be an explicit option and must not expose secrets.
Such an operation would need coherent handling for schema syntax and transitive
references, context cancellation, and source-specific public errors. Credential
declarations must remain separate from credential values; checking current
environment availability, if supported at all, should be an explicit option
and must not expose secrets.
Reconsider this idea after downstream use of `InspectPrompt`,
`InspectProfile`, and fixture-based preparation demonstrates a concrete gap.

View File

@@ -38,13 +38,8 @@ consumers.
## Ideas
### Remaining Standard Policy Alignment
Review the internal standard's architecture and testing clarifications about
adopted constraints, threat scope, and tests that must not invent contracts.
Adapt relevant guidance to Promptkit's library boundaries without introducing
new runtime restrictions or release gates. This is separate from the completed
licensing-documentation alignment.
No ideas are currently awaiting selection. Active feature work belongs in its
focused roadmap rather than this catalog.
## Entry Format

208
engine.go
View File

@@ -8,6 +8,7 @@ import (
"net/http"
"time"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
internalengine "gitea.maximumdirect.net/eric/promptkit/internal/engine"
"gitea.maximumdirect.net/eric/promptkit/internal/observation"
@@ -15,7 +16,7 @@ import (
)
// ErrInvalidConfig identifies invalid engine construction, including missing
// required configuration, invalid options or backend registrations, and a nil
// effective configuration, invalid options, and a nil
// Engine receiver.
var ErrInvalidConfig = errors.New("invalid engine configuration")
@@ -34,13 +35,13 @@ var (
// nor the selected prompt's default profile is present. Such an error also
// matches ErrInvalidRequest.
ErrProfileRequired = errors.New("profile selection is required")
// ErrPromptLoad identifies a failure to read, decode, validate, select, or
// hash a prompt definition, except for the not-found case represented by
// ErrPromptNotFound.
// ErrPromptLoad identifies a prompt selection failure other than the
// not-found case represented by ErrPromptNotFound. Invalid definitions
// instead fail construction with ErrInvalidConfig.
ErrPromptLoad = errors.New("failed to load prompt definition")
// ErrProfileLoad identifies a failure to read, decode, validate, or select
// an execution profile or resolve its backend, except for the profile
// not-found case represented by ErrProfileNotFound.
// ErrProfileLoad identifies a profile selection or resolution failure other
// than the not-found case represented by ErrProfileNotFound. Invalid
// definitions instead fail construction with ErrInvalidConfig.
ErrProfileLoad = errors.New("failed to load execution profile")
// ErrAPIKeyEnvMissing identifies an explicitly required APIKeyEnv whose
// environment variable is unset or empty after direct RunRequest.APIKey
@@ -84,7 +85,8 @@ var (
// Engine inspects prompts and profiles and prepares and runs Promptkit prompt
// requests.
//
// An Engine is safe for concurrent calls to [Engine.InspectPrompt],
// An Engine is safe for concurrent calls to [Engine.InspectConfiguration],
// [Engine.InspectPrompt],
// [Engine.InspectProfile], [Engine.Prepare], [Engine.PrepareExecution],
// [Engine.Run], and [Engine.RunPrepared]. Each Engine owns independent
// backend-capacity pools that coordinate Run and RunPrepared admission and
@@ -94,18 +96,21 @@ type Engine struct {
internal *internalengine.Engine
}
// Config selects the directory-backed sources and built-in model-client
// transport used by [NewEngine]. Config has no stable JSON representation.
// ConfigurationSources selects ordered ConfigKit underlays and an optional
// highest-priority main input. Each underlay's inputs strictly compose within
// their group; later underlays overlay earlier ones. A nil Main uses
// ConfigKit's empty source. A non-nil Main is passed through unchanged and
// may fail construction if invalid.
type ConfigurationSources struct {
Underlays []configkit.Underlay
Main *configkit.Input
}
// Config selects configuration sources and built-in model-client transport
// used by [NewEngine]. Config has no stable JSON representation.
type Config struct {
// PromptDir is the directory searched recursively for prompt definitions.
// It is required unless a WithPromptFS or WithPromptFile option supplies the
// prompt source.
PromptDir string
// ProfileDir is an optional ordinary configured source whose profiles take
// precedence over application fallback and maintained catalog profiles. An
// empty value selects the lower-precedence sources unless a profile-source
// option supplies the ordinary source.
ProfileDir string
// Sources combines maintained definitions with caller-selected inputs.
Sources ConfigurationSources
// SchemaDir is the root for JSON Schema files. An empty value uses the
// current directory. WithSchemaFS or WithSchemaFile replaces this source.
SchemaDir string
@@ -128,12 +133,9 @@ type Config struct {
// Option customizes engine construction.
//
// NewEngine applies options in argument order and ignores nil options. Within
// each prompt-source, ordinary-profile-source, fallback-profile-source,
// in-memory-profile, schema-source, model-client, and artifact-reader
// category, the last non-nil valid option replaces earlier options in that
// category. WithBackend is the additive exception: unique registrations
// accumulate, and a repeated backend ID is an error rather than a replacement.
// An invalid option fails construction even if a later option would replace it.
// each schema-source, model-client, and artifact-reader category, the last
// non-nil valid option replaces earlier options. An invalid option fails
// construction even if a later option would replace it.
type Option interface {
apply(*engineOptions) error
}
@@ -181,102 +183,6 @@ func WithArtifactReader(reader ArtifactReader) Option {
})
}
// WithPromptFS loads prompt definitions from fsys under root.
//
// The source uses the same strict prompt YAML rules as configured prompt
// directories, and prompt content_file paths resolve within this source.
// fsys must be non-nil and root must be non-empty; otherwise NewEngine fails
// with ErrInvalidConfig. This option replaces Config.PromptDir and earlier
// prompt-source options.
func WithPromptFS(fsys fs.FS, root string) Option {
return optionFunc(func(options *engineOptions) error {
return options.internal.SetPromptFS(fsys, root)
})
}
// WithPromptFile loads prompt definitions from the single prompt file at path.
//
// Relative prompt content_file paths resolve from the file's directory. path
// must name an existing non-directory file when NewEngine applies the option.
// This option replaces Config.PromptDir and earlier prompt-source options.
func WithPromptFile(path string) Option {
return optionFunc(func(options *engineOptions) error {
return options.internal.SetPromptFile(path)
})
}
// WithProfileFS loads execution profiles from fsys under root.
//
// Profiles from this ordinary configured source take precedence over
// application fallback and built-in profiles. Profile YAML must use api_key_env
// for environment-based credentials; raw API keys are rejected. fsys must be
// non-nil and root must be non-empty; otherwise NewEngine fails with
// ErrInvalidConfig. This option replaces Config.ProfileDir and earlier file or
// FS profile-source options, but remains below WithProfiles in precedence.
func WithProfileFS(fsys fs.FS, root string) Option {
return optionFunc(func(options *engineOptions) error {
return options.internal.SetProfileFS(fsys, root)
})
}
// WithProfileFile loads execution profiles from the single profile file at path.
//
// The profile takes precedence over application fallback and built-in profiles.
// Profile YAML must use api_key_env for environment-based credentials; raw API
// keys are rejected. path must name an existing non-directory file when
// NewEngine applies the option. This option replaces Config.ProfileDir and
// earlier file or FS profile-source options, but remains below WithProfiles in
// precedence.
func WithProfileFile(path string) Option {
return optionFunc(func(options *engineOptions) error {
return options.internal.SetProfileFile(path)
})
}
// WithFallbackProfileFS supplies application-owned fallback profile
// definitions from fsys under root.
//
// Profile lookup checks, in order, profiles supplied by WithProfiles; the
// ordinary configured source selected by WithProfileFile, WithProfileFS, or
// Config.ProfileDir; this fallback source; and Promptkit's maintained catalog
// profiles. Each source supplies a complete profile definition; profile fields
// are not merged between sources. Only an absent profile ID proceeds to the
// next source. A matching read, parse, duplicate, validation, or credential
// format failure stops resolution.
//
// Files use the ordinary strict profile YAML and api_key_env credential rules.
// Loading and validation are lazy: NewEngine validates this option's arguments
// but does not read profile files. fsys must be non-nil and root must be
// nonblank; otherwise NewEngine returns an error matching ErrInvalidConfig.
// Repeating this option replaces the earlier valid fallback source.
//
// This option controls profile-definition lookup, not provider or generation
// failover.
func WithFallbackProfileFS(fsys fs.FS, root string) Option {
return optionFunc(func(options *engineOptions) error {
return options.internal.SetFallbackProfileFS(fsys, root)
})
}
// WithProfiles configures in-memory profiles that take precedence over
// ordinary configured, application fallback, and built-in profiles.
//
// NewEngine locally validates and copies every profile. IDs must be unique
// within one call. An invalid local definition, duplicate ID, or unsupported
// ExtraParams value makes construction fail with ErrInvalidConfig. A derived
// profile's base reference and resolved target completeness are checked when it
// is selected or inspected. Repeating WithProfiles replaces the complete
// earlier in-memory set rather than merging it.
func WithProfiles(profiles ...Profile) Option {
return optionFunc(func(options *engineOptions) error {
definitions := make([]domain.ExecutionProfile, len(profiles))
for index, publicProfile := range profiles {
definitions[index] = toDomainProfile(publicProfile)
}
return options.internal.SetMemoryProfiles(definitions)
})
}
// WithSchemaFS loads JSON Schema documents from fsys under root.
//
// Prompt schema_path values resolve within this source when schema validation
@@ -302,14 +208,15 @@ func WithSchemaFile(path string) Option {
// NewEngine constructs an Engine from configuration and options.
//
// Options are applied in order according to [Option]. PromptDir is required
// unless a prompt-source option is present. Construction validates option
// arguments, in-memory profiles, and backend registrations but defers reading
// and validating prompt, file-backed profile, and schema contents until Prepare
// or Run needs them.
// ConfigKit composes maintained catalog inputs, PromptKit-owned definitions,
// caller underlays, and the optional main input in that order. Construction
// validates the final effective backend, profile, and prompt definitions and
// captures referenced prompt content. A missing prompt collection is allowed.
// Schema documents and request artifacts remain runtime resources.
//
// NewEngine returns an error matching ErrInvalidConfig for invalid
// configuration, options, or backend-capacity policies. Each constructed
// configuration or options, preserving a ConfigKit LoadError when loading
// fails. Each constructed
// Engine has independent backend-capacity pools. Construction does not perform
// model requests or require credentials. A negative Config.TransientRetries
// is invalid.
@@ -320,16 +227,16 @@ func NewEngine(cfg Config, opts ...Option) (*Engine, error) {
continue
}
if err := opt.apply(&options); err != nil {
return nil, fmt.Errorf("%w: %v", ErrInvalidConfig, err)
return nil, fmt.Errorf("%w: %w", ErrInvalidConfig, err)
}
}
assembled, err := internalengine.New(
internalengine.Config{PromptDir: cfg.PromptDir, ProfileDir: cfg.ProfileDir, SchemaDir: cfg.SchemaDir, Timeout: cfg.Timeout, HTTPClient: cfg.HTTPClient, TransientRetries: copyIntPtr(cfg.TransientRetries)},
internalengine.Config{Sources: internalengine.ConfigurationSources{Underlays: cfg.Sources.Underlays, Main: cfg.Sources.Main}, SchemaDir: cfg.SchemaDir, Timeout: cfg.Timeout, HTTPClient: cfg.HTTPClient, TransientRetries: copyIntPtr(cfg.TransientRetries)},
options.internal,
)
if err != nil {
return nil, fmt.Errorf("%w: %v", ErrInvalidConfig, err)
return nil, fmt.Errorf("%w: %w", ErrInvalidConfig, err)
}
return &Engine{internal: assembled}, nil
}
@@ -339,9 +246,9 @@ func NewEngine(cfg Config, opts ...Option) (*Engine, error) {
//
// InspectPrompt requires a nonblank promptID. It passes nonblank promptID and
// promptVersion values unchanged to the engine's ordinary, case-sensitive
// prompt selection. An empty version succeeds only when that source has one
// selected ID; a nonempty version selects one exact ID/version pair. The
// configured prompt source is used without merging, fallback, or enumeration.
// prompt selection. An empty version succeeds only when that ID has one
// version; a nonempty version selects one exact ID/version pair from the
// captured configuration snapshot.
//
// A successful result proves that the selected definition and any referenced
// message content files were structurally loaded. Inputs are returned in
@@ -349,22 +256,22 @@ func NewEngine(cfg Config, opts ...Option) (*Engine, error) {
// resolved. OutputContract is the normalized declared contract, with a JSON
// Schema path when declared but without loading or compiling that schema.
// PromptHash is the same opaque equality value as PreparedRun.PromptHash for
// the selected definition and observed source state; its spelling, length,
// the selected captured definition; its spelling, length,
// encoding, algorithm, and security properties are not contracts.
//
// This method does not return prompt bodies, templates, source paths, schemas,
// rendered messages, or execution settings. It does not resolve a profile or
// credential, read artifacts or schemas, render, validate, admit capacity,
// contact a provider, or generate model output. The returned PromptInspection
// and its input slice are caller-owned. Filesystem-backed inspection is a
// point-in-time lookup and does not freeze a definition for later execution.
// and its input slice are caller-owned. Inspection does not reload configuration
// or content files; construct another Engine to observe source changes.
//
// A nil Engine returns an error matching ErrInvalidConfig. A blank prompt ID
// matches ErrInvalidRequest. An absent exact ID or version matches
// ErrPromptNotFound and not ErrPromptLoad. Malformed, unreadable, duplicate,
// ambiguous, referenced-content, or hashing failures match ErrPromptLoad.
// Cancellation during lookup matches ErrPromptLoad while preserving the
// context error. InspectPrompt returns no partial result on error.
// ErrPromptNotFound and not ErrPromptLoad. An ambiguous omitted version matches
// ErrPromptLoad. Invalid definitions and resource failures reject construction
// instead. Cancellation during lookup preserves the context error. InspectPrompt
// returns no partial result on error.
func (e *Engine) InspectPrompt(
ctx context.Context,
promptID string,
@@ -384,10 +291,9 @@ func (e *Engine) InspectPrompt(
// InspectProfile resolves one explicit profile without selecting a prompt or
// starting execution work.
//
// InspectProfile trims surrounding whitespace from profileID and looks up the
// resulting nonblank ID exactly and case-sensitively through the engine's
// in-memory, ordinary configured-source, application fallback, and built-in
// profile precedence. It applies the framework timeout baseline, selected
// InspectProfile looks up a nonblank profileID exactly and case-sensitively
// in the captured effective configuration. It applies the framework timeout
// baseline, selected
// backend, and then selected profile to EffectiveModelParams without a request
// override. BackendID is empty for an endpoint-only profile.
//
@@ -398,16 +304,15 @@ func (e *Engine) InspectPrompt(
// absent or blank named environment variable is not an error.
//
// The returned ProfileInspection and all nested mutable values are
// caller-owned. Filesystem-backed inspection is a point-in-time lookup and
// does not freeze the profile for a later execution. This method does not load
// caller-owned. Inspection does not reload configuration; construct another
// Engine to observe source changes. This method does not load
// a prompt, render, read artifacts or schemas, admit backend capacity, contact
// a provider, or generate model output.
//
// A nil Engine returns an error matching ErrInvalidConfig. A blank profile ID
// matches ErrInvalidRequest. An absent exact ID matches ErrProfileNotFound and
// not ErrProfileLoad. Malformed or unreadable profile data, an unknown backend,
// or an invalid resolved target matches ErrProfileLoad. Cancellation during
// profile loading matches ErrProfileLoad while preserving the context error.
// not ErrProfileLoad. Invalid definitions or unknown backends reject
// construction instead. Cancellation preserves the context error.
// InspectProfile returns no partial result on error.
func (e *Engine) InspectProfile(ctx context.Context, profileID string) (*ProfileInspection, error) {
if e == nil || e.internal == nil {
@@ -495,8 +400,9 @@ func (e *Engine) PrepareExecution(ctx context.Context, req RunRequest) (*Prepare
}
// Run prepares a request, invokes the configured LLMClient, and validates the
// generated output. Each call resolves current sources and composes a fresh,
// stateless effective prompt with any validated RunRequest.AppendedMessages.
// generated output. Each call uses the Engine's captured configuration and
// composes a fresh effective prompt with any validated
// RunRequest.AppendedMessages.
//
// A content-validation failure is a successful run whose
// RunResult.Validation has Status ValidationFailed. When its output contract

File diff suppressed because it is too large Load Diff

View File

@@ -72,10 +72,6 @@ func publicErrorFor(err error) error {
return ErrPromptLoad
case errors.Is(err, usecase.ErrProfileLoad):
return ErrProfileLoad
case errors.Is(err, promptdef.ErrInvalidYAML), errors.Is(err, promptdef.ErrInvalidPromptDefinition):
return ErrPromptLoad
case isProfileLoadCause(err):
return ErrProfileLoad
case errors.Is(err, usecase.ErrAPIKeyEnvMissing):
return errors.Join(ErrInvalidRequest, ErrAPIKeyEnvMissing)
case errors.Is(err, capacity.ErrCapacityExceeded):
@@ -94,9 +90,3 @@ func publicErrorFor(err error) error {
return nil
}
}
func isProfileLoadCause(err error) bool {
return errors.Is(err, profile.ErrInvalidYAML) ||
errors.Is(err, profile.ErrInvalidProfile) ||
errors.Is(err, profile.ErrRawAPIKeyNotAllowed)
}

View File

@@ -10,6 +10,7 @@ import (
"os"
"strings"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit"
)
@@ -78,14 +79,24 @@ type summary struct {
func run() (summary, error) {
transport := &scriptedTransport{}
engine, err := promptkit.NewEngine(promptkit.Config{HTTPClient: &http.Client{Transport: transport}},
promptkit.WithPromptFS(assets, "prompts"),
promptkit.WithSchemaFS(assets, "schemas"),
promptkit.WithBackend(promptkit.Backend{ID: "offline-backend", Endpoint: "https://example.invalid/v1", ConcurrencyLimit: 1}),
promptkit.WithProfiles(
promptkit.Profile{ID: "proposal-profile", BackendID: "offline-backend", Model: "proposal-model"},
promptkit.Profile{ID: "assessment-profile", BackendID: "offline-backend", Model: "assessment-model"},
))
inputs, err := promptkit.FilesystemInputs("example prompts", assets, "prompts")
if err != nil {
return summary{}, err
}
backendID, proposalModel, assessmentModel := "offline-backend", "proposal-model", "assessment-model"
inputs = append(inputs,
configkit.Input{Source: promptkit.BackendSource("example backend", map[string]promptkit.BackendConfig{
backendID: promptkit.LocalBackend("https://example.invalid/v1", 1),
})},
configkit.Input{Source: promptkit.ProfileSource("example profiles", map[string]promptkit.ProfileConfig{
"proposal-profile": {Backend: &backendID, Model: &proposalModel},
"assessment-profile": {Backend: &backendID, Model: &assessmentModel},
})},
)
engine, err := promptkit.NewEngine(promptkit.Config{
Sources: promptkit.ConfigurationSources{Underlays: []configkit.Underlay{{Inputs: inputs}}},
HTTPClient: &http.Client{Transport: transport},
}, promptkit.WithSchemaFS(assets, "schemas"))
if err != nil {
return summary{}, err
}

View File

@@ -1,17 +1,18 @@
id: example.assessment
version: "1.0.0"
default_profile: assessment-profile
description: Assess a structured proposal independently of schema validity.
inputs:
- name: proposal
required: true
content_type: application/json
messages:
- role: system
content: Return a concise assessment as JSON.
- role: user
content: '{{input "proposal"}}'
output:
format: json
validation_mode: json_schema
schema_path: assessment.schema.json
prompts:
example.assessment:
"1.0.0":
default_profile: assessment-profile
description: Assess a structured proposal independently of schema validity.
inputs:
- name: proposal
required: true
content_type: application/json
messages:
- role: system
content: Return a concise assessment as JSON.
- role: user
content: '{{input "proposal"}}'
output:
format: json
validation_mode: json_schema
schema_path: assessment.schema.json

View File

@@ -1,17 +1,18 @@
id: example.proposal
version: "1.0.0"
default_profile: proposal-profile
description: Propose a bounded improvement from a synthetic idea.
inputs:
- name: idea
required: true
content_type: text/plain
messages:
- role: system
content: Return a concise proposal as JSON.
- role: user
content: '{{input "idea"}}'
output:
format: json
validation_mode: json_schema
schema_path: proposal.schema.json
prompts:
example.proposal:
"1.0.0":
default_profile: proposal-profile
description: Propose a bounded improvement from a synthetic idea.
inputs:
- name: idea
required: true
content_type: text/plain
messages:
- role: system
content: Return a concise proposal as JSON.
- role: user
content: '{{input "idea"}}'
output:
format: json
validation_mode: json_schema
schema_path: proposal.schema.json

View File

@@ -6,6 +6,7 @@ import (
"fmt"
"os"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit"
)
@@ -18,18 +19,23 @@ type summary struct {
}
func main() {
engine, err := promptkit.NewEngine(
promptkit.Config{},
promptkit.WithPromptFile("examples/go-library/prepare/prompt.yaml"),
promptkit.WithProfiles(promptkit.Profile{
ID: "offline-example",
Endpoint: "https://example.invalid/v1",
Model: "offline-model",
}),
)
endpoint, model := "https://example.invalid/v1", "offline-model"
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{Source: promptkit.ProfileSource("example profile", map[string]promptkit.ProfileConfig{
"offline-example": {Endpoint: &endpoint, Model: &model},
})}}}},
Main: &configkit.Input{Source: configkit.File("examples/go-library/prepare/prompt.yaml")},
}})
if err != nil {
exit(err)
}
view, err := engine.InspectConfiguration()
if err != nil {
exit(err)
}
if _, found := view.Explain(configkit.Keys("profiles", "offline-example", "model")); !found {
exit(fmt.Errorf("configured profile model was not found"))
}
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: "example.prepare",

View File

@@ -1,16 +1,17 @@
id: example.prepare
version: "1.0.0"
default_profile: offline-example
description: Prepare a prompt without contacting a model provider.
inputs:
- name: note
required: true
content_type: text/plain
messages:
- role: system
content: Summarize the note in one sentence.
- role: user
content: '{{input "note"}}'
output:
format: text
validation_mode: basic
prompts:
example.prepare:
"1.0.0":
default_profile: offline-example
description: Prepare a prompt without contacting a model provider.
inputs:
- name: note
required: true
content_type: text/plain
messages:
- role: system
content: Summarize the note in one sentence.
- role: user
content: '{{input "note"}}'
output:
format: text
validation_mode: basic

View File

@@ -6,6 +6,7 @@ import (
"fmt"
"os"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit"
)
@@ -38,16 +39,13 @@ type summary struct {
}
func main() {
engine, err := promptkit.NewEngine(
promptkit.Config{},
promptkit.WithPromptFile("examples/go-library/run/prompt.yaml"),
promptkit.WithProfiles(promptkit.Profile{
ID: "offline-example",
Endpoint: "https://example.invalid/v1",
Model: "offline-model",
}),
promptkit.WithLLMClient(deterministicClient{}),
)
endpoint, model := "https://example.invalid/v1", "offline-model"
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{Source: promptkit.ProfileSource("example profile", map[string]promptkit.ProfileConfig{
"offline-example": {Endpoint: &endpoint, Model: &model},
})}}}},
Main: &configkit.Input{Source: configkit.File("examples/go-library/run/prompt.yaml")},
}}, promptkit.WithLLMClient(deterministicClient{}))
if err != nil {
exit(err)
}

View File

@@ -1,16 +1,17 @@
id: example.run
version: "1.0.0"
default_profile: offline-example
description: Run a prompt with a deterministic injected model client.
inputs:
- name: note
required: true
content_type: text/plain
messages:
- role: system
content: Summarize the note in one sentence.
- role: user
content: '{{input "note"}}'
output:
format: text
validation_mode: basic
prompts:
example.run:
"1.0.0":
default_profile: offline-example
description: Run a prompt with a deterministic injected model client.
inputs:
- name: note
required: true
content_type: text/plain
messages:
- role: system
content: Summarize the note in one sentence.
- role: user
content: '{{input "note"}}'
output:
format: text
validation_mode: basic

View File

@@ -0,0 +1,177 @@
package promptkit_test
import (
"context"
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"sync"
"sync/atomic"
"testing"
"testing/fstest"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit"
)
const capturedPromptDocument = `
prompts:
captured:
"1":
default_profile: offline
session_id: '{{.session}}'
messages:
- role: user
content_file: body.tmpl
output:
format: text
validation_mode: none
`
func capturedPromptEngine(t *testing.T, source fs.FS) *promptkit.Engine {
t.Helper()
inputs, err := promptkit.FilesystemInputs("application prompts", source, "config")
if err != nil {
t.Fatalf("select prompt inputs: %v", err)
}
endpoint, model := "https://example.invalid/v1", "offline-model"
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Underlays: []configkit.Underlay{
{Inputs: inputs},
{Inputs: []configkit.Input{{Source: promptkit.ProfileSource("offline profile", map[string]promptkit.ProfileConfig{
"offline": {Endpoint: &endpoint, Model: &model},
})}}},
},
}})
if err != nil {
t.Fatalf("construct engine: %v", err)
}
return engine
}
func TestEngineCapturesPromptContentUntilReconstruction(t *testing.T) {
files := fstest.MapFS{
"config/prompt.yaml": &fstest.MapFile{Data: []byte(capturedPromptDocument)},
"config/body.tmpl": &fstest.MapFile{Data: []byte("before {{.name}}")},
}
first := capturedPromptEngine(t, files)
files["config/body.tmpl"] = &fstest.MapFile{Data: []byte("after {{.name}}")}
request := promptkit.RunRequest{PromptID: "captured", Vars: map[string]string{"name": "Ada", "session": "session-1"}}
oldPrepared, err := first.Prepare(context.Background(), request)
if err != nil {
t.Fatalf("prepare with existing engine: %v", err)
}
second := capturedPromptEngine(t, files)
newPrepared, err := second.Prepare(context.Background(), request)
if err != nil {
t.Fatalf("prepare with rebuilt engine: %v", err)
}
if oldPrepared.Messages[0].Content != "before Ada" || newPrepared.Messages[0].Content != "after Ada" || oldPrepared.PromptHash == newPrepared.PromptHash {
t.Fatalf("captured content or hash did not follow engine lifetime: old=%+v new=%+v", oldPrepared, newPrepared)
}
}
type unreadableAfterConstructionFS struct {
fs.FS
closed atomic.Bool
reads atomic.Int64
}
func (source *unreadableAfterConstructionFS) Open(name string) (fs.File, error) {
if source.closed.Load() {
source.reads.Add(1)
return nil, fs.ErrPermission
}
return source.FS.Open(name)
}
func TestCapturedTemplatesBindInputsIndependentlyAcrossConcurrentPreparations(t *testing.T) {
source := &unreadableAfterConstructionFS{FS: fstest.MapFS{
"config/prompt.yaml": &fstest.MapFile{Data: []byte(capturedPromptDocument)},
"config/body.tmpl": &fstest.MapFile{Data: []byte(`{{.name}}: {{input "note"}}`)},
}}
engine := capturedPromptEngine(t, source)
source.closed.Store(true)
const calls = 24
var wait sync.WaitGroup
errs := make(chan error, calls)
for index := 0; index < calls; index++ {
wait.Add(1)
go func(index int) {
defer wait.Done()
name := fmt.Sprintf("caller-%d", index)
body := fmt.Sprintf("body-%d", index)
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: "captured", SessionID: "direct-session",
Vars: map[string]string{"name": name},
Inputs: map[string]promptkit.ArtifactRef{"note": promptkit.Inline(body)},
})
if err == nil && (prepared.Messages[0].Content != name+": "+body || prepared.SessionID != "direct-session") {
err = fmt.Errorf("unexpected prepared content: %+v", prepared)
}
errs <- err
}(index)
}
wait.Wait()
close(errs)
for err := range errs {
if err != nil {
t.Fatal(err)
}
}
if source.reads.Load() != 0 {
t.Fatalf("preparation reopened captured source %d times", source.reads.Load())
}
_, err := engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: "captured", Vars: map[string]string{"name": "Ada"},
Inputs: map[string]promptkit.ArtifactRef{"note": promptkit.Inline("body")},
})
if !errors.Is(err, promptkit.ErrPromptRender) {
t.Fatalf("missing session variable = %v, want runtime render error", err)
}
_, err = engine.Prepare(context.Background(), promptkit.RunRequest{
PromptID: "captured", SessionID: "direct-session", Vars: map[string]string{"name": "Ada"},
})
if !errors.Is(err, promptkit.ErrPromptRender) {
t.Fatalf("missing input helper artifact = %v, want runtime render error", err)
}
}
func TestPreparationStillReadsRequestArtifactsAndSchemasAfterConstruction(t *testing.T) {
endpoint, model := "https://example.invalid/v1", "offline-model"
schemas := fstest.MapFS{}
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
Underlays: []configkit.Underlay{{Inputs: []configkit.Input{{Source: promptkit.ProfileSource("profile", map[string]promptkit.ProfileConfig{
"offline": {Endpoint: &endpoint, Model: &model},
})}}}},
Main: &configkit.Input{Source: configkit.Mapping("structured prompt", map[string]any{
"prompts": map[string]any{"structured": map[string]any{"1": map[string]any{
"default_profile": "offline",
"inputs": []any{map[string]any{"name": "note", "required": true}},
"messages": []any{map[string]any{"role": "user", "content": `{{input "note"}}`}},
"output": map[string]any{"format": "json", "validation_mode": "json_schema", "schema_path": "schema.json"},
}}},
})},
}}, promptkit.WithSchemaFS(schemas, "."))
if err != nil {
t.Fatalf("construct engine before request resources exist: %v", err)
}
schemas["schema.json"] = &fstest.MapFile{Data: []byte(`{"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object"}`)}
artifactPath := filepath.Join(t.TempDir(), "note.txt")
if err := os.WriteFile(artifactPath, []byte("request artifact"), 0o600); err != nil {
t.Fatalf("write request artifact: %v", err)
}
prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "structured", Inputs: map[string]promptkit.ArtifactRef{
"note": promptkit.File(artifactPath),
}})
if err != nil {
t.Fatalf("prepare with resources supplied after construction: %v", err)
}
if len(prepared.Messages) != 1 || prepared.Messages[0].Content != "request artifact" || prepared.StructuredOutput == nil || prepared.StructuredOutput.JSONSchema == nil {
t.Fatalf("runtime artifact or schema was not prepared: %+v", prepared)
}
}

View File

@@ -21,14 +21,14 @@ func TestBuiltInGenerationError(t *testing.T) {
engine := newBuiltInGenerationErrorEngine(t, http.StatusUnprocessableEntity,
`{"error":{"code":"`+codeMarker+`","type":"`+typeMarker+`","message":"`+messageMarker+`"}}`)
result, err := engine.Run(context.Background(), generationErrorRunRequest())
result, err := engine.Run(context.Background(), simpleGenerationRequest())
if result != nil {
t.Fatalf("Run result = %#v, want nil", result)
}
assertGenerationError(t, err, http.StatusUnprocessableEntity, codeMarker, typeMarker, messageMarker)
preparedEngine := newBuiltInGenerationErrorEngine(t, http.StatusServiceUnavailable, `{"error":{}}`)
prepared, err := preparedEngine.PrepareExecution(context.Background(), generationErrorRunRequest())
prepared, err := preparedEngine.PrepareExecution(context.Background(), simpleGenerationRequest())
if err != nil {
t.Fatalf("PrepareExecution: %v", err)
}
@@ -43,7 +43,7 @@ func TestBuiltInGenerationErrorWithAppendedMessages(t *testing.T) {
const messageMarker = "combined-request-provider-marker"
engine := newBuiltInGenerationErrorEngine(t, http.StatusUnprocessableEntity,
`{"error":{"message":"`+messageMarker+`"}}`)
request := generationErrorRunRequest()
request := simpleGenerationRequest()
request.AppendedMessages = []promptkit.RenderedMessage{
{Role: promptkit.RoleAssistant, Content: "previous response"},
{Role: promptkit.RoleUser, Content: "consumer correction"},
@@ -63,7 +63,7 @@ func TestBuiltInRepairGenerationError(t *testing.T) {
messageMarker = "repair-message-marker"
)
calls := 0
config := contractConfig(frameworkSchemaDir)
config := simpleConfig("message")
config.HTTPClient = &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
calls++
if calls == 1 {
@@ -77,7 +77,7 @@ func TestBuiltInRepairGenerationError(t *testing.T) {
if err != nil {
t.Fatalf("NewEngine: %v", err)
}
req := generationErrorRunRequest()
req := simpleGenerationRequest()
req.Validation = &promptkit.OutputContract{
Format: promptkit.FormatJSON,
ValidationMode: promptkit.ValidationJSON,
@@ -96,7 +96,7 @@ func TestBuiltInRepairGenerationError(t *testing.T) {
func TestBuiltInGenerationFailureCategories(t *testing.T) {
transportCause := errors.New("private transport diagnostic")
config := contractConfig(frameworkSchemaDir)
config := simpleConfig("message")
config.HTTPClient = &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
return nil, transportCause
})}
@@ -104,7 +104,7 @@ func TestBuiltInGenerationFailureCategories(t *testing.T) {
if err != nil {
t.Fatal(err)
}
result, err := engine.Run(context.Background(), generationErrorRunRequest())
result, err := engine.Run(context.Background(), simpleGenerationRequest())
if result != nil || !errors.Is(err, promptkit.ErrLLMGenerate) || !errors.Is(err, promptkit.ErrTransport) || !errors.Is(err, transportCause) {
t.Fatalf("transport result=%#v, error=%v", result, err)
}
@@ -122,7 +122,7 @@ func TestBuiltInGenerationFailureCategories(t *testing.T) {
} {
t.Run(tc.name, func(t *testing.T) {
engine := newBuiltInGenerationErrorEngine(t, http.StatusOK, tc.body)
result, err := engine.Run(context.Background(), generationErrorRunRequest())
result, err := engine.Run(context.Background(), simpleGenerationRequest())
if result != nil || !errors.Is(err, promptkit.ErrLLMGenerate) || !errors.Is(err, promptkit.ErrMalformedResponse) || errors.Is(err, promptkit.ErrTransport) {
t.Fatalf("malformed result=%#v, error=%v", result, err)
}
@@ -139,7 +139,7 @@ func TestBuiltInGenerationPreservesContextIdentity(t *testing.T) {
{name: "deadline", cause: context.DeadlineExceeded},
} {
t.Run(tc.name, func(t *testing.T) {
config := contractConfig(frameworkSchemaDir)
config := simpleConfig("message")
config.HTTPClient = &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
return nil, tc.cause
})}
@@ -147,7 +147,7 @@ func TestBuiltInGenerationPreservesContextIdentity(t *testing.T) {
if err != nil {
t.Fatal(err)
}
result, err := engine.Run(context.Background(), generationErrorRunRequest())
result, err := engine.Run(context.Background(), simpleGenerationRequest())
if result != nil || !errors.Is(err, tc.cause) || !errors.Is(err, promptkit.ErrLLMGenerate) || !errors.Is(err, promptkit.ErrTransport) {
t.Fatalf("context result=%#v, error=%v", result, err)
}
@@ -165,8 +165,8 @@ func TestInjectedGenerationFailureCategories(t *testing.T) {
{name: "arbitrary error", client: &fakeLLMClient{err: errors.New("injected failure")}},
} {
t.Run(tc.name, func(t *testing.T) {
engine := newContractEngineWithOptions(t, frameworkSchemaDir, promptkit.WithLLMClient(tc.client))
result, err := engine.Run(context.Background(), generationErrorRunRequest())
engine := newSimpleEngine(t, "message", promptkit.WithLLMClient(tc.client))
result, err := engine.Run(context.Background(), simpleGenerationRequest())
if result != nil || !errors.Is(err, promptkit.ErrLLMGenerate) || errors.Is(err, promptkit.ErrTransport) || errors.Is(err, promptkit.ErrMalformedResponse) != tc.malformed {
t.Fatalf("injected result=%#v, error=%v", result, err)
}
@@ -207,7 +207,7 @@ func assertGenerationError(t *testing.T, err error, statusCode int, code, provid
func newBuiltInGenerationErrorEngine(t *testing.T, statusCode int, body string) *promptkit.Engine {
t.Helper()
config := contractConfig(frameworkSchemaDir)
config := simpleConfig("message")
noRetries := 0
config.TransientRetries = &noRetries
config.HTTPClient = &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
@@ -224,12 +224,6 @@ func newBuiltInGenerationErrorEngine(t *testing.T, statusCode int, body string)
return engine
}
func generationErrorRunRequest() promptkit.RunRequest {
return promptkit.RunRequest{
PromptID: frameworkMarkdownSummaryPromptID,
Inputs: map[string]promptkit.ArtifactRef{
"transcript": promptkit.Inline("Rin opens the gate."),
"glossary": promptkit.Inline("gate: A guarded passage."),
},
}
func simpleGenerationRequest() promptkit.RunRequest {
return promptkit.RunRequest{PromptID: "prompt"}
}

11
go.mod
View File

@@ -3,10 +3,13 @@ module gitea.maximumdirect.net/eric/promptkit
go 1.25.5
require (
gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2 v2.20260926.0
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v2 v2.20260926.0
gitea.maximumdirect.net/eric/ConfigKit v0.5.0
gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v3 v3.20260930.0
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v3 v3.20260930.0
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2
gopkg.in/yaml.v3 v3.0.1
)
require golang.org/x/text v0.14.0 // indirect
require (
go.yaml.in/yaml/v4 v4.0.0-rc.6 // indirect
golang.org/x/text v0.14.0 // indirect
)

14
go.sum
View File

@@ -1,14 +1,16 @@
gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2 v2.20260926.0 h1:wdab+R+lgMZxzYwhbAuCrW7zdSQ1cW9ovBR6UzLmZR4=
gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2 v2.20260926.0/go.mod h1:a3VBbh9oaGXD6ruxhtwHj+AWW6DY/SSyby1QtMmYSUo=
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v2 v2.20260926.0 h1:baCCIB6Ee5t0S4Zc++NTBIPcJEv5UPS6DlSrpZfykwg=
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v2 v2.20260926.0/go.mod h1:mtgD1E0SGm0e/I8Ffhz+AYX7GN7RKUWJXehNFIKHs4Y=
gitea.maximumdirect.net/eric/ConfigKit v0.5.0 h1:2KwG10+kApWM3avOoYA+p768xcbHUp7tQ2rqizBNpyg=
gitea.maximumdirect.net/eric/ConfigKit v0.5.0/go.mod h1:tYWpsrbLyu05r8+1MapWp5fN6ZrX9I8hU2xEPedYmYk=
gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v3 v3.20260930.0 h1:lzfCDbzJQRGzIlRPRprnAV9hVVL5zUQy80VMQ0kbYa0=
gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v3 v3.20260930.0/go.mod h1:HjS0f+3q0IdHCTEz+VVAEgIwIWJFAZUjuKdAtvtqKdg=
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v3 v3.20260930.0 h1:h12ZTRldAmpLAomj0oQRGteFxHLN1LPE6XDl9zJNbm8=
gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v3 v3.20260930.0/go.mod h1:OEFBeKdjTsLzqJwLmXI2O5gG3r64FktcVJVb3mBrRfg=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
go.yaml.in/yaml/v4 v4.0.0-rc.6 h1:1h7H1ohdUh93/FyE4YaDa1Zh64K6VVbjF4K6WUxMtH4=
go.yaml.in/yaml/v4 v4.0.0-rc.6/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -5,63 +5,40 @@ package backend
import (
"errors"
"fmt"
"regexp"
"sort"
"strings"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/jsonvalue"
"gitea.maximumdirect.net/eric/promptkit/internal/llm"
)
const (
// OpenRouterID is the reserved ID of Promptkit's built-in OpenRouter
// backend.
// OpenRouterID identifies the maintained OpenRouter backend.
OpenRouterID = "openrouter"
// RakestrawHomeID is the reserved ID of Promptkit's built-in Rakestrawhome
// backend.
// RakestrawHomeID identifies the maintained Rakestrawhome backend.
RakestrawHomeID = "rakestrawhome"
defaultQueueCapacity = 1024
)
// ErrBackendNotFound identifies a registry lookup for an unknown backend ID.
var ErrBackendNotFound = errors.New("backend not found")
var environmentVariableName = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
// Registry is an immutable collection of validated backend definitions.
type Registry struct {
backends map[string]domain.Backend
}
// NewRegistry constructs a registry containing maintained definitions followed
// by consumer additions. Every ID must be unique across both groups.
func NewRegistry(maintained, additions []domain.Backend) (*Registry, error) {
registry := &Registry{
backends: make(map[string]domain.Backend, len(maintained)+len(additions)),
}
definitions := make([]domain.Backend, 0, len(maintained)+len(additions))
definitions = append(definitions, maintained...)
definitions = append(definitions, additions...)
for _, definition := range definitions {
definition.ID = strings.TrimSpace(definition.ID)
if definition.ID == "" {
return nil, errors.New("backend ID must not be blank")
}
if _, exists := registry.backends[definition.ID]; exists {
return nil, fmt.Errorf("backend ID %q is already registered", definition.ID)
}
normalized, err := NormalizeDefinition(definition)
// NewEffectiveRegistry stores already-validated effective backend definitions
// under their exact configuration keys. Configuration validation and
// normalization precede this ownership boundary.
func NewEffectiveRegistry(definitions map[string]domain.Backend) (*Registry, error) {
registry := &Registry{backends: make(map[string]domain.Backend, len(definitions))}
for id, definition := range definitions {
params, err := jsonvalue.CopyMap(definition.ExtraParams)
if err != nil {
return nil, err
return nil, fmt.Errorf("backend %q extra parameters: %w", id, err)
}
registry.backends[normalized.ID] = normalized
definition.ID = id
definition.ExtraParams = params
registry.backends[id] = definition
}
return registry, nil
}
@@ -100,80 +77,3 @@ func (r *Registry) CapacityPolicies() map[string]domain.BackendCapacityPolicy {
}
return policies
}
// NormalizeDefinition validates and defensively copies one backend definition.
func NormalizeDefinition(definition domain.Backend) (domain.Backend, error) {
endpoint, err := domain.NormalizeOpenAICompatibleBaseEndpoint(definition.Endpoint)
if err != nil {
return domain.Backend{}, fmt.Errorf("backend %q endpoint: %w", definition.ID, err)
}
definition.Endpoint = endpoint
definition.APIKeyEnv = strings.TrimSpace(definition.APIKeyEnv)
if definition.APIKeyEnv != "" && !environmentVariableName.MatchString(definition.APIKeyEnv) {
return domain.Backend{}, fmt.Errorf(
"backend %q api key environment variable %q is invalid",
definition.ID,
definition.APIKeyEnv,
)
}
if definition.ConcurrencyLimit < 0 {
return domain.Backend{}, fmt.Errorf(
"backend %q concurrency limit must not be negative",
definition.ID,
)
}
if definition.QueueCapacity < 0 {
return domain.Backend{}, fmt.Errorf(
"backend %q queue capacity must not be negative",
definition.ID,
)
}
if definition.ConcurrencyLimit == 0 {
if definition.QueueCapacitySet {
return domain.Backend{}, fmt.Errorf(
"backend %q queue capacity requires a positive concurrency limit",
definition.ID,
)
}
definition.QueueCapacity = 0
} else {
if !definition.QueueCapacitySet {
definition.QueueCapacity = defaultQueueCapacity
definition.QueueCapacitySet = true
}
maxInt := int(^uint(0) >> 1)
if definition.QueueCapacity > maxInt-definition.ConcurrencyLimit {
return domain.Backend{}, fmt.Errorf(
"backend %q total capacity overflows int",
definition.ID,
)
}
}
keys := make([]string, 0, len(definition.ExtraParams))
for key := range definition.ExtraParams {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
if key == "" {
return domain.Backend{}, fmt.Errorf("backend %q extra parameter key must not be empty", definition.ID)
}
if llm.IsReservedOpenAIChatRequestField(key) {
return domain.Backend{}, fmt.Errorf(
"backend %q extra parameter %q collides with a reserved request field",
definition.ID,
key,
)
}
}
extraParams, err := jsonvalue.CopyMap(definition.ExtraParams)
if err != nil {
return domain.Backend{}, fmt.Errorf("backend %q extra parameters: %w", definition.ID, err)
}
definition.ExtraParams = extraParams
return definition, nil
}

View File

@@ -3,374 +3,54 @@ package backend_test
import (
"errors"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/promptkit/internal/backend"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
)
const validEndpoint = "https://backend.example/v1"
func TestRegistryIncludesMaintainedDefinitions(t *testing.T) {
maintained := []domain.Backend{
{ID: backend.OpenRouterID, Endpoint: validEndpoint, ConcurrencyLimit: 2, QueueCapacity: 3, QueueCapacitySet: true},
{ID: backend.RakestrawHomeID, Endpoint: "https://second.example/v1", ConcurrencyLimit: 4, QueueCapacity: 5, QueueCapacitySet: true},
}
registry, err := backend.NewRegistry(maintained, nil)
if err != nil {
t.Fatalf("construct registry: %v", err)
}
for _, expected := range maintained {
t.Run(expected.ID, func(t *testing.T) {
definition, err := registry.GetBackend(expected.ID)
if err != nil {
t.Fatalf("look up maintained definition: %v", err)
}
if !reflect.DeepEqual(definition, expected) {
t.Fatalf("unexpected maintained definition: %#v", definition)
}
})
}
policies := registry.CapacityPolicies()
if len(policies) != 2 ||
policies[backend.OpenRouterID] != (domain.BackendCapacityPolicy{
ConcurrencyLimit: 2,
QueueCapacity: 3,
}) ||
policies[backend.RakestrawHomeID] != (domain.BackendCapacityPolicy{
ConcurrencyLimit: 4,
QueueCapacity: 5,
}) {
t.Fatalf("unexpected built-in capacity policies: %#v", policies)
}
}
func TestRegistryNormalizesUniqueAdditionsAndIsolatesMutations(t *testing.T) {
nested := map[string]int{"limit": 2}
extraParams := map[string]any{
"count": int64(7),
"nested": nested,
}
registry, err := backend.NewRegistry(nil, []domain.Backend{
{
ID: " custom ",
Endpoint: " https://custom.example/openai/v1 ",
APIKeyEnv: " CUSTOM_API_KEY ",
ExtraParams: extraParams,
ConcurrencyLimit: 3,
QueueCapacity: 2,
QueueCapacitySet: true,
},
{
ID: "Custom",
Endpoint: validEndpoint,
func TestEffectiveRegistryOwnsDefinitionsAndCapacityPolicies(t *testing.T) {
params := map[string]any{"nested": map[string]any{"enabled": true}}
registry, err := backend.NewEffectiveRegistry(map[string]domain.Backend{
" local.v1 ": {
Endpoint: "https://backend.example/v1", ExtraParams: params,
ConcurrencyLimit: 2, QueueCapacity: 3, QueueCapacitySet: true,
},
})
if err != nil {
t.Fatalf("construct registry: %v", err)
t.Fatal(err)
}
nested["limit"] = 99
extraParams["added"] = true
got, err := registry.GetBackend("custom")
if err != nil {
t.Fatalf("look up custom backend: %v", err)
params["nested"].(map[string]any)["enabled"] = false
selected, err := registry.GetBackend(" local.v1 ")
if err != nil || selected.ID != " local.v1 " || selected.ExtraParams["nested"].(map[string]any)["enabled"] != true {
t.Fatalf("effective backend was not captured under its literal ID: %+v, %v", selected, err)
}
if got.ID != "custom" ||
got.Endpoint != "https://custom.example/openai/v1" ||
got.APIKeyEnv != "CUSTOM_API_KEY" ||
got.ConcurrencyLimit != 3 ||
got.QueueCapacity != 2 ||
!got.QueueCapacitySet {
t.Fatalf("unexpected normalized definition: %#v", got)
selected.ExtraParams["nested"].(map[string]any)["enabled"] = false
again, err := registry.GetBackend(" local.v1 ")
if err != nil || again.ExtraParams["nested"].(map[string]any)["enabled"] != true {
t.Fatalf("lookup exposed registry values: %+v, %v", again, err)
}
if count, ok := got.ExtraParams["count"].(int64); !ok || count != 7 {
t.Fatalf("integer type or value changed: %#v", got.ExtraParams["count"])
}
gotNested, ok := got.ExtraParams["nested"].(map[string]int)
if !ok || gotNested["limit"] != 2 {
t.Fatalf("container type or value changed: %#v", got.ExtraParams["nested"])
}
if _, exists := got.ExtraParams["added"]; exists {
t.Fatalf("registry retained caller map: %#v", got.ExtraParams)
}
gotNested["limit"] = 100
got.ExtraParams["added"] = true
again, err := registry.GetBackend("custom")
if err != nil {
t.Fatalf("look up custom backend again: %v", err)
}
if again.ExtraParams["nested"].(map[string]int)["limit"] != 2 {
t.Fatalf("lookup exposed registry nested map: %#v", again.ExtraParams)
}
if _, exists := again.ExtraParams["added"]; exists {
t.Fatalf("lookup exposed registry map: %#v", again.ExtraParams)
}
if _, err := registry.GetBackend("Custom"); err != nil {
t.Fatalf("backend IDs should be case-sensitive: %v", err)
}
policies := registry.CapacityPolicies()
if len(policies) != 1 {
t.Fatalf("unexpected capacity policy count: %#v", policies)
}
policies["custom"] = domain.BackendCapacityPolicy{}
delete(policies, "custom")
againPolicies := registry.CapacityPolicies()
if againPolicies["custom"] != (domain.BackendCapacityPolicy{
ConcurrencyLimit: 3,
QueueCapacity: 2,
if !reflect.DeepEqual(policies, map[string]domain.BackendCapacityPolicy{
" local.v1 ": {ConcurrencyLimit: 2, QueueCapacity: 3},
}) {
t.Fatalf("capacity policy map mutated registry state: %#v", againPolicies)
t.Fatalf("capacity policy = %+v", policies)
}
delete(policies, " local.v1 ")
if len(registry.CapacityPolicies()) != 1 {
t.Fatal("capacity policy map exposed registry state")
}
}
func TestNewRegistryNormalizesCapacityPolicy(t *testing.T) {
maxInt := int(^uint(0) >> 1)
tests := []struct {
name string
definition domain.Backend
want domain.BackendCapacityPolicy
wantSet bool
wantError bool
}{
{
name: "unlimited when omitted",
definition: domain.Backend{},
},
{
name: "default queue",
definition: domain.Backend{
ConcurrencyLimit: 2,
},
want: domain.BackendCapacityPolicy{
ConcurrencyLimit: 2,
QueueCapacity: 1024,
},
wantSet: true,
},
{
name: "explicit zero queue",
definition: domain.Backend{
ConcurrencyLimit: 2,
QueueCapacitySet: true,
},
want: domain.BackendCapacityPolicy{
ConcurrencyLimit: 2,
},
wantSet: true,
},
{
name: "negative concurrency limit",
definition: domain.Backend{
ConcurrencyLimit: -1,
},
wantError: true,
},
{
name: "negative queue capacity",
definition: domain.Backend{
ConcurrencyLimit: 1,
QueueCapacity: -1,
QueueCapacitySet: true,
},
wantError: true,
},
{
name: "queue without limit",
definition: domain.Backend{
QueueCapacitySet: true,
},
wantError: true,
},
{
name: "total overflow",
definition: domain.Backend{
ConcurrencyLimit: maxInt,
QueueCapacity: 1,
QueueCapacitySet: true,
},
wantError: true,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
tc.definition.ID = "custom"
tc.definition.Endpoint = validEndpoint
registry, err := backend.NewRegistry(nil, []domain.Backend{tc.definition})
if tc.wantError {
if err == nil {
t.Fatal("expected invalid capacity policy error")
}
return
}
if err != nil {
t.Fatalf("construct registry: %v", err)
}
definition, err := registry.GetBackend("custom")
if err != nil {
t.Fatalf("look up custom backend: %v", err)
}
if definition.ConcurrencyLimit != tc.want.ConcurrencyLimit ||
definition.QueueCapacity != tc.want.QueueCapacity ||
definition.QueueCapacitySet != tc.wantSet {
t.Fatalf("normalized capacity=(%d, %d, %t), want (%d, %d, %t)",
definition.ConcurrencyLimit,
definition.QueueCapacity,
definition.QueueCapacitySet,
tc.want.ConcurrencyLimit,
tc.want.QueueCapacity,
tc.wantSet,
)
}
policies := registry.CapacityPolicies()
got, ok := policies["custom"]
if ok != tc.wantSet || got != tc.want {
t.Fatalf("capacity policy=(%#v, %t), want (%#v, %t)", got, ok, tc.want, tc.wantSet)
}
})
}
}
func TestNewRegistryRejectsDuplicateIDs(t *testing.T) {
tests := []struct {
name string
additions []domain.Backend
wantID string
}{
{
name: "OpenRouter collision after normalization",
additions: []domain.Backend{{ID: " openrouter "}},
wantID: backend.OpenRouterID,
},
{
name: "Rakestrawhome collision after normalization",
additions: []domain.Backend{{ID: " rakestrawhome "}},
wantID: backend.RakestrawHomeID,
},
{
name: "consumer collision after normalization",
additions: []domain.Backend{
{ID: "custom", Endpoint: validEndpoint},
{ID: " custom ", Endpoint: "https://other.example/v1"},
},
wantID: "custom",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := backend.NewRegistry(nil, tc.additions)
if err == nil {
t.Fatal("expected duplicate ID error")
}
if !strings.Contains(err.Error(), tc.wantID) {
t.Fatalf("expected error to identify %q, got %v", tc.wantID, err)
}
})
}
}
func TestNewRegistryValidatesIDs(t *testing.T) {
for _, id := range []string{"", " \t\n "} {
t.Run(id, func(t *testing.T) {
_, err := backend.NewRegistry(nil, []domain.Backend{{
ID: id,
Endpoint: validEndpoint,
}})
if err == nil {
t.Fatal("expected blank ID error")
}
})
}
}
func TestNewRegistryValidatesEndpoints(t *testing.T) {
tests := []struct {
name string
endpoint string
}{
{name: "blank", endpoint: ""},
{name: "relative", endpoint: "/v1"},
{name: "missing host", endpoint: "https:///v1"},
{name: "unsupported scheme", endpoint: "ftp://backend.example/v1"},
{name: "user information", endpoint: "https://user@backend.example/v1"},
{name: "query", endpoint: "https://backend.example/v1?mode=chat"},
{name: "empty query", endpoint: "https://backend.example/v1?"},
{name: "fragment", endpoint: "https://backend.example/v1#chat"},
{name: "empty fragment", endpoint: "https://backend.example/v1#"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := backend.NewRegistry(nil, []domain.Backend{{
ID: "custom",
Endpoint: tc.endpoint,
}})
if err == nil {
t.Fatal("expected invalid endpoint error")
}
})
}
}
func TestNewRegistryValidatesEnvironmentVariableNames(t *testing.T) {
for _, name := range []string{"1API_KEY", "API-KEY", "API KEY", "ÅPI_KEY"} {
t.Run(name, func(t *testing.T) {
_, err := backend.NewRegistry(nil, []domain.Backend{{
ID: "custom",
Endpoint: validEndpoint,
APIKeyEnv: name,
}})
if err == nil {
t.Fatal("expected invalid environment-variable name error")
}
})
}
}
func TestNewRegistryRejectsInvalidAndReservedExtraParameters(t *testing.T) {
tests := []struct {
name string
extraParams map[string]any
}{
{name: "unsupported value", extraParams: map[string]any{"value": make(chan int)}},
{name: "reserved key", extraParams: map[string]any{"model": "override"}},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := backend.NewRegistry(nil, []domain.Backend{{
ID: "custom",
Endpoint: validEndpoint,
ExtraParams: tc.extraParams,
}})
if err == nil {
t.Fatal("expected invalid extra parameters error")
}
})
}
}
func TestRegistryLookupReportsNotFound(t *testing.T) {
registry, err := backend.NewRegistry(nil, nil)
func TestEffectiveRegistryMissingBackend(t *testing.T) {
registry, err := backend.NewEffectiveRegistry(nil)
if err != nil {
t.Fatalf("construct registry: %v", err)
t.Fatal(err)
}
_, err = registry.GetBackend("missing")
if !errors.Is(err, backend.ErrBackendNotFound) {
t.Fatalf("expected ErrBackendNotFound, got %v", err)
if _, err := registry.GetBackend("missing"); !errors.Is(err, backend.ErrBackendNotFound) {
t.Fatalf("missing backend error = %v", err)
}
if !strings.Contains(err.Error(), "missing") {
t.Fatalf("expected error to identify backend, got %v", err)
if _, err := registry.GetBackend("local.v1"); !errors.Is(err, backend.ErrBackendNotFound) {
t.Fatalf("trimmed lookup unexpectedly matched: %v", err)
}
}

View File

@@ -1,249 +1,52 @@
// Package catalog validates immutable maintained backend catalog assets.
// Package catalog selects the maintained configuration assets supplied with
// PromptKit. ConfigKit owns their composition and effective validation.
package catalog
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
"path"
"sort"
"strings"
"gitea.maximumdirect.net/eric/ConfigKit"
openrouter "gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v3"
rakestrawhome "gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v3"
"gitea.maximumdirect.net/eric/promptkit/internal/backend"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/profile"
"gitea.maximumdirect.net/eric/promptkit/internal/filecatalog"
)
// Source identifies one immutable backend catalog asset tree.
type Source struct {
Name string
ExpectedBackendID string
FS fs.FS
Root string
}
// Set is the validated maintained backend and raw profile catalog.
type Set struct {
Backends []domain.Backend
Profiles profile.Repository
profileIDs []string
}
// Load validates and combines immutable catalog sources in source order.
func Load(sources ...Source) (Set, error) {
if len(sources) == 0 {
return Set{}, errors.New("at least one catalog source is required")
}
loaded := Set{Backends: make([]domain.Backend, 0, len(sources))}
names := map[string]bool{}
backendIDs := map[string]bool{}
profileIDs := map[string]bool{}
for _, source := range sources {
if err := validateSource(source, names); err != nil {
return Set{}, err
}
names[source.Name] = true
definition, err := loadBackend(source)
if err != nil {
return Set{}, err
}
if backendIDs[definition.ID] {
return Set{}, fmt.Errorf("catalog %s: backend ID duplicates an earlier catalog", source.Name)
}
backendIDs[definition.ID] = true
repository, metadata, err := profile.LoadFSRepository(context.Background(), source.FS, path.Join(source.Root, "profiles"))
if err != nil {
return Set{}, fmt.Errorf("catalog %s: %w", source.Name, err)
}
if len(metadata) == 0 {
return Set{}, fmt.Errorf("catalog %s: profiles must not be empty", source.Name)
}
resolvingRepository := profile.NewResolvingRepository(repository)
for _, entry := range metadata {
if strings.HasPrefix(entry.ID, "promptkit-") {
return Set{}, fmt.Errorf("catalog %s: %s uses the reserved promptkit- profile ID prefix", source.Name, entry.Path)
}
if profileIDs[entry.ID] {
return Set{}, fmt.Errorf("catalog %s: %s duplicates an earlier profile ID", source.Name, entry.Path)
}
if containsField(entry.ExplicitFields, "endpoint") || containsField(entry.ExplicitFields, "api_key_env") {
return Set{}, fmt.Errorf("catalog %s: %s contains connection metadata", source.Name, entry.Path)
}
value, err := repository.GetProfile(context.Background(), entry.ID)
if err != nil {
return Set{}, fmt.Errorf("catalog %s: %s: %w", source.Name, entry.Path, err)
}
if err := rejectSecretKeys(value.ExtraParams); err != nil {
return Set{}, fmt.Errorf("catalog %s: %s: prohibited extra parameter key", source.Name, entry.Path)
}
resolved, err := resolvingRepository.GetProfile(context.Background(), entry.ID)
if err != nil {
return Set{}, fmt.Errorf("catalog %s: %s has invalid profile inheritance", source.Name, entry.Path)
}
if resolved.BackendID != definition.ID {
return Set{}, fmt.Errorf("catalog %s: %s selects a different backend", source.Name, entry.Path)
}
profileIDs[entry.ID] = true
loaded.profileIDs = append(loaded.profileIDs, entry.ID)
}
if loaded.Profiles == nil {
loaded.Profiles = repository
} else {
loaded.Profiles = profile.NewOverlayRepository(loaded.Profiles, repository)
}
loaded.Backends = append(loaded.Backends, definition)
}
sort.Strings(loaded.profileIDs)
return loaded, nil
}
func validateSource(source Source, names map[string]bool) error {
if strings.TrimSpace(source.Name) == "" || source.Name != strings.TrimSpace(source.Name) {
return errors.New("catalog source name must not be blank")
}
if names[source.Name] {
return fmt.Errorf("duplicate catalog source name %q", source.Name)
}
if source.FS == nil {
return fmt.Errorf("catalog %s: filesystem is nil", source.Name)
}
if source.Root == "." || source.Root == "" || !fs.ValidPath(source.Root) {
return fmt.Errorf("catalog %s: asset root is invalid", source.Name)
}
if strings.TrimSpace(source.ExpectedBackendID) == "" {
return fmt.Errorf("catalog %s: expected backend ID is blank", source.Name)
}
return validateLayout(source)
}
func validateLayout(source Source) error {
manifestPath := path.Join(source.Root, "backend.json")
profilesRoot := path.Join(source.Root, "profiles")
return fs.WalkDir(source.FS, source.Root, func(assetPath string, entry fs.DirEntry, err error) error {
if err != nil {
return err
}
if assetPath == source.Root {
if !entry.IsDir() {
return fmt.Errorf("catalog %s: invalid asset path %s", source.Name, assetPath)
}
return nil
}
if entry.IsDir() {
if assetPath == profilesRoot || strings.HasPrefix(assetPath, profilesRoot+"/") {
return nil
}
return fmt.Errorf("catalog %s: invalid asset path %s", source.Name, assetPath)
}
if assetPath == manifestPath && entry.Type().IsRegular() {
return nil
}
if strings.HasPrefix(assetPath, profilesRoot+"/") && entry.Type().IsRegular() && strings.HasSuffix(assetPath, ".yml") {
return nil
}
return fmt.Errorf("catalog %s: invalid asset path %s", source.Name, assetPath)
})
}
func loadBackend(source Source) (domain.Backend, error) {
data, err := fs.ReadFile(source.FS, path.Join(source.Root, "backend.json"))
// Underlays selects external catalogs followed by PromptKit-owned profiles.
// Neither group is interpreted before the final effective configuration loads.
func Underlays() ([]configkit.Underlay, error) {
openrouterInputs, err := assetInputs("OpenRouter catalog", openrouter.FS(), openrouter.Root)
if err != nil {
return domain.Backend{}, fmt.Errorf("catalog %s: backend.json: %w", source.Name, err)
return nil, err
}
type manifest struct {
SchemaVersion *int `json:"schema_version"`
ID *string `json:"id"`
Endpoint *string `json:"endpoint"`
APIKeyEnv *string `json:"api_key_env"`
ConcurrencyLimit *int `json:"concurrency_limit"`
QueueCapacity *int `json:"queue_capacity"`
ExtraParams json.RawMessage `json:"extra_params"`
}
decoder := json.NewDecoder(bytes.NewReader(data))
decoder.DisallowUnknownFields()
var value manifest
if err := decoder.Decode(&value); err != nil {
return domain.Backend{}, fmt.Errorf("catalog %s: backend.json: invalid manifest", source.Name)
}
var trailing any
if err := decoder.Decode(&trailing); !errors.Is(err, io.EOF) {
return domain.Backend{}, fmt.Errorf("catalog %s: backend.json: invalid manifest", source.Name)
}
if value.SchemaVersion == nil || *value.SchemaVersion != 1 || value.ID == nil || value.Endpoint == nil || value.APIKeyEnv == nil || value.ConcurrencyLimit == nil || value.QueueCapacity == nil || value.ExtraParams == nil {
return domain.Backend{}, fmt.Errorf("catalog %s: backend.json: required field is missing or unsupported", source.Name)
}
if *value.ID != source.ExpectedBackendID {
return domain.Backend{}, fmt.Errorf("catalog %s: backend ID does not match expected ID", source.Name)
}
if strings.TrimSpace(*value.APIKeyEnv) == "" {
return domain.Backend{}, fmt.Errorf("catalog %s: backend.json: api key environment variable must not be blank", source.Name)
}
extraParams, err := decodeExtraParams(value.ExtraParams)
rakestrawhomeInputs, err := assetInputs("Rakestrawhome catalog", rakestrawhome.FS(), rakestrawhome.Root)
if err != nil {
return domain.Backend{}, fmt.Errorf("catalog %s: backend.json: invalid extra parameters", source.Name)
return nil, err
}
if err := rejectSecretKeys(extraParams); err != nil {
return domain.Backend{}, fmt.Errorf("catalog %s: backend.json: prohibited extra parameter key", source.Name)
}
normalized, err := backend.NormalizeDefinition(domain.Backend{ID: *value.ID, Endpoint: *value.Endpoint, APIKeyEnv: *value.APIKeyEnv, ExtraParams: extraParams, ConcurrencyLimit: *value.ConcurrencyLimit, QueueCapacity: *value.QueueCapacity, QueueCapacitySet: true})
catalogInputs := append(openrouterInputs, rakestrawhomeInputs...)
owned := configkit.Input{Source: configkit.Mapping("PromptKit-owned profiles", map[string]any{
"profiles": map[string]any{
"promptkit-openrouter-standard": map[string]any{"extends": "gemini-flash-latest"},
"promptkit-openrouter-auto": map[string]any{"backend": backend.OpenRouterID, "model": "openrouter/auto"},
},
})}
return []configkit.Underlay{{Inputs: catalogInputs}, {Inputs: []configkit.Input{owned}}}, nil
}
func assetInputs(label string, filesystem fs.FS, root string) ([]configkit.Input, error) {
files, err := filecatalog.FindFSYAMLFiles(context.Background(), filesystem, root)
if err != nil {
return domain.Backend{}, fmt.Errorf("catalog %s: backend.json: invalid backend definition", source.Name)
return nil, fmt.Errorf("discover %s configuration: %w", label, err)
}
return normalized, nil
}
func decodeExtraParams(data []byte) (map[string]any, error) {
if bytes.Equal(bytes.TrimSpace(data), []byte("null")) {
return nil, nil
}
decoder := json.NewDecoder(bytes.NewReader(data))
decoder.UseNumber()
var value map[string]any
if err := decoder.Decode(&value); err != nil || value == nil {
return nil, errors.New("extra parameters must be an object or null")
}
var trailing any
if err := decoder.Decode(&trailing); !errors.Is(err, io.EOF) {
return nil, errors.New("extra parameters must contain exactly one JSON value")
}
return value, nil
}
func containsField(fields []string, target string) bool {
for _, field := range fields {
if field == target {
return true
}
}
return false
}
func rejectSecretKeys(value map[string]any) error {
return rejectSecretValue(value)
}
func rejectSecretValue(value any) error {
switch value := value.(type) {
case map[string]any:
for key, child := range value {
switch strings.ToLower(key) {
case "api_key", "apikey", "authorization", "credential", "credentials", "password", "secret", "token", "access_token":
return errors.New("prohibited key")
}
if err := rejectSecretValue(child); err != nil {
return err
}
}
case []any:
for _, child := range value {
if err := rejectSecretValue(child); err != nil {
return err
}
}
}
return nil
namespace := configkit.NewFilesystem(label, filesystem)
inputs := make([]configkit.Input, 0, len(files))
for _, name := range files {
inputs = append(inputs, configkit.Input{
Source: namespace.File(name),
Filesystem: &configkit.FilesystemContext{FS: &namespace, Root: root},
})
}
return inputs, nil
}

View File

@@ -2,355 +2,67 @@ package catalog
import (
"context"
"encoding/json"
"fmt"
"io/fs"
"strings"
"errors"
"testing"
"testing/fstest"
openrouter "gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2"
rakestrawhome "gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v2"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit/internal/configuration"
)
func TestLoadPublishedCatalogs(t *testing.T) {
loaded, err := Load(
Source{Name: "OpenRouter", ExpectedBackendID: "openrouter", FS: openrouter.FS(), Root: openrouter.Root},
Source{Name: "Rakestrawhome", ExpectedBackendID: "rakestrawhome", FS: rakestrawhome.FS(), Root: rakestrawhome.Root},
)
func TestPublishedCatalogsResolveThroughEffectiveConfiguration(t *testing.T) {
underlays, err := Underlays()
if err != nil {
t.Fatalf("load published catalogs: %v", err)
t.Fatal(err)
}
if loaded.Profiles == nil || len(loaded.profileIDs) == 0 {
t.Fatalf("published catalogs have no usable profiles: %#v", loaded)
state, err := configuration.Load(configkit.Input{Source: configkit.NoSource()}, underlays)
if err != nil {
t.Fatal(err)
}
if _, err := loaded.Profiles.GetProfile(context.Background(), "gemini-flash-latest"); err != nil {
t.Fatalf("load maintained standard-profile base: %v", err)
openrouter, err := state.GetBackend("openrouter")
if err != nil || openrouter.Endpoint != "https://openrouter.ai/api/v1" || openrouter.ConcurrencyLimit != 16 || openrouter.QueueCapacity != 1024 {
t.Fatalf("OpenRouter backend: %+v, %v", openrouter, err)
}
expectedBackends := map[string]struct {
endpoint string
apiKeyEnv string
concurrencyLimit int
queueCapacity int
}{
"openrouter": {
endpoint: "https://openrouter.ai/api/v1",
apiKeyEnv: "OPENROUTER_API_KEY",
concurrencyLimit: 16,
queueCapacity: 1024,
},
"rakestrawhome": {
endpoint: "https://inference.ai.rakestrawhome.com/v1",
apiKeyEnv: "RAKESTRAWHOME_INFERENCE_API_KEY",
concurrencyLimit: 4,
queueCapacity: 1024,
},
rakestrawhome, err := state.GetBackend("rakestrawhome")
if err != nil || rakestrawhome.Endpoint != "https://inference.ai.rakestrawhome.com/v1" || rakestrawhome.ConcurrencyLimit != 4 {
t.Fatalf("Rakestrawhome backend: %+v, %v", rakestrawhome, err)
}
if len(loaded.Backends) != len(expectedBackends) {
t.Fatalf("published backend count = %d, want %d", len(loaded.Backends), len(expectedBackends))
}
for _, definition := range loaded.Backends {
expected, found := expectedBackends[definition.ID]
if !found {
t.Fatalf("unexpected published backend %q", definition.ID)
}
if definition.Endpoint != expected.endpoint ||
definition.APIKeyEnv != expected.apiKeyEnv ||
definition.ConcurrencyLimit != expected.concurrencyLimit ||
definition.QueueCapacity != expected.queueCapacity ||
!definition.QueueCapacitySet ||
definition.ExtraParams != nil {
t.Fatalf("unexpected published backend definition: %#v", definition)
}
}
}
func TestLoadRejectsInvalidSources(t *testing.T) {
for name, sources := range map[string][]Source{
"none": nil,
"blank name": {{Name: " ", ExpectedBackendID: "openrouter", FS: openrouter.FS(), Root: openrouter.Root}},
"duplicate name": {
{Name: "same", ExpectedBackendID: "openrouter", FS: openrouter.FS(), Root: openrouter.Root},
{Name: "same", ExpectedBackendID: "rakestrawhome", FS: rakestrawhome.FS(), Root: rakestrawhome.Root},
},
"nil filesystem": {{Name: "missing", ExpectedBackendID: "openrouter", Root: openrouter.Root}},
"invalid root": {{Name: "invalid-root", ExpectedBackendID: "openrouter", FS: openrouter.FS(), Root: "."}},
"blank expected backend": {{Name: "blank-backend", ExpectedBackendID: " ", FS: openrouter.FS(), Root: openrouter.Root}},
for id, want := range map[string]string{
"promptkit-openrouter-standard": "~google/gemini-flash-latest",
"promptkit-openrouter-auto": "openrouter/auto",
"rakestrawhome-gemma-4-31b": "google/gemma-4-31b-it",
} {
t.Run(name, func(t *testing.T) {
if _, err := Load(sources...); err == nil {
t.Fatal("expected error")
}
})
}
}
func TestLoadRejectsInvalidLayouts(t *testing.T) {
tests := map[string]func(fstest.MapFS){
"unexpected file": func(fsys fstest.MapFS) {
fsys["catalog/notes.txt"] = &fstest.MapFile{Data: []byte("unexpected")}
},
"unexpected directory": func(fsys fstest.MapFS) {
fsys["catalog/unexpected"] = &fstest.MapFile{Mode: fs.ModeDir}
},
"nonregular manifest": func(fsys fstest.MapFS) {
fsys["catalog/backend.json"].Mode = fs.ModeSymlink
},
"wrong profile extension": func(fsys fstest.MapFS) {
fsys["catalog/profiles/extra.yaml"] = &fstest.MapFile{Data: []byte(validProfile("extra", "one"))}
},
"nonregular profile": func(fsys fstest.MapFS) {
fsys["catalog/profiles/one-profile.yml"].Mode = fs.ModeSymlink
},
}
for name, mutate := range tests {
t.Run(name, func(t *testing.T) {
fsys := validCatalogFS("one")
mutate(fsys)
if _, err := Load(testSource("one", fsys)); err == nil {
t.Fatal("expected invalid layout error")
}
})
}
}
func TestLoadRejectsInvalidManifests(t *testing.T) {
tests := map[string]string{
"malformed": `{`,
"trailing value": validManifest("one", "TEST_API_KEY", "null") + `{}`,
"missing fields": `{"schema_version":1,"id":"one"}`,
"unsupported version": strings.Replace(validManifest("one", "TEST_API_KEY", "null"), `"schema_version":1`, `"schema_version":2`, 1),
"unknown field": strings.Replace(validManifest("one", "TEST_API_KEY", "null"), `"extra_params":null`, `"extra_params":null,"unknown":true`, 1),
"blank API key env": validManifest("one", " ", "null"),
"invalid API key env": validManifest("one", "LEAK-MARKER", "null"),
"invalid endpoint": strings.Replace(validManifest("one", "TEST_API_KEY", "null"), `https://one.example/v1`, `ftp://leak-marker.invalid/v1`, 1),
"zero concurrency": strings.Replace(validManifest("one", "TEST_API_KEY", "null"), `"concurrency_limit":2`, `"concurrency_limit":0`, 1),
"non-object parameters": validManifest("one", "TEST_API_KEY", `[]`),
"secret parameter": validManifest("one", "TEST_API_KEY", `{"nested":{"token":"leak-marker"}}`),
}
for name, manifest := range tests {
t.Run(name, func(t *testing.T) {
fsys := validCatalogFS("one")
fsys["catalog/backend.json"].Data = []byte(manifest)
_, err := Load(testSource("one", fsys))
if err == nil {
t.Fatal("expected invalid manifest error")
}
if strings.Contains(strings.ToLower(err.Error()), "leak-marker") {
t.Fatalf("catalog error exposed manifest content: %v", err)
}
})
}
}
func TestLoadPreservesManifestJSONNumbers(t *testing.T) {
fsys := validCatalogFS("one")
fsys["catalog/backend.json"].Data = []byte(validManifest(
"one",
"TEST_API_KEY",
`{"large":9007199254740993,"nested":[1.25]}`,
))
loaded, err := Load(testSource("one", fsys))
if err != nil {
t.Fatalf("load catalog: %v", err)
}
if got := loaded.Backends[0].ExtraParams["large"]; got != json.Number("9007199254740993") {
t.Fatalf("large JSON integer = %#v, want preserved json.Number", got)
}
nested := loaded.Backends[0].ExtraParams["nested"].([]any)
if nested[0] != json.Number("1.25") {
t.Fatalf("nested JSON number = %#v, want preserved json.Number", nested[0])
}
}
func TestLoadRejectsInvalidCatalogProfiles(t *testing.T) {
tests := map[string]func(fstest.MapFS){
"empty": func(fsys fstest.MapFS) {
delete(fsys, "catalog/profiles/one-profile.yml")
fsys["catalog/profiles"] = &fstest.MapFile{Mode: fs.ModeDir}
},
"malformed": func(fsys fstest.MapFS) {
fsys["catalog/profiles/one-profile.yml"].Data = []byte("id: [")
},
"raw API key": func(fsys fstest.MapFS) {
fsys["catalog/profiles/one-profile.yml"].Data = []byte(validProfile("one-profile", "one") + "api_key: leak-marker\n")
},
"endpoint field": func(fsys fstest.MapFS) {
fsys["catalog/profiles/one-profile.yml"].Data = []byte(validProfile("one-profile", "one") + "endpoint: ''\n")
},
"API key environment field": func(fsys fstest.MapFS) {
fsys["catalog/profiles/one-profile.yml"].Data = []byte(validProfile("one-profile", "one") + "api_key_env: ''\n")
},
"owner mismatch": func(fsys fstest.MapFS) {
fsys["catalog/profiles/one-profile.yml"].Data = []byte(validProfile("one-profile", "other"))
},
"missing base": func(fsys fstest.MapFS) {
fsys["catalog/profiles/one-profile.yml"].Data = []byte("id: one-profile\nbase_profile: leak-marker\n")
},
"cyclic base": func(fsys fstest.MapFS) {
fsys["catalog/profiles/one-profile.yml"].Data = []byte("id: one-profile\nbase_profile: second\n")
fsys["catalog/profiles/second.yml"] = &fstest.MapFile{Data: []byte("id: second\nbase_profile: one-profile\n")}
},
"secret profile parameter": func(fsys fstest.MapFS) {
fsys["catalog/profiles/one-profile.yml"].Data = []byte(validProfile("one-profile", "one") + "extra_params:\n nested:\n password: leak-marker\n")
},
"unknown field is redacted": func(fsys fstest.MapFS) {
fsys["catalog/profiles/one-profile.yml"].Data = []byte(validProfile("one-profile", "one") + "leak_marker: leak-marker\n")
},
}
for name, mutate := range tests {
t.Run(name, func(t *testing.T) {
fsys := validCatalogFS("one")
mutate(fsys)
_, err := Load(testSource("one", fsys))
if err == nil {
t.Fatal("expected invalid profile error")
}
if strings.Contains(strings.ToLower(err.Error()), "leak-marker") {
t.Fatalf("catalog error exposed profile content: %v", err)
}
})
}
}
func TestLoadReservesPromptkitProfilePrefixForCatalogs(t *testing.T) {
tests := map[string]struct {
profileID string
wantError bool
}{
"maintained name": {
profileID: "promptkit-openrouter-standard",
wantError: true,
},
"other reserved name": {
profileID: "promptkit-example",
wantError: true,
},
"ordinary name": {
profileID: "ordinary-profile",
},
}
for name, test := range tests {
t.Run(name, func(t *testing.T) {
fsys := validCatalogFS("one")
fsys["catalog/profiles/one-profile.yml"].Data = []byte(validProfile(test.profileID, "one"))
_, err := Load(testSource("one", fsys))
if (err != nil) != test.wantError {
t.Fatalf("Load() error = %v, want error %t", err, test.wantError)
}
if err != nil && !strings.Contains(err.Error(), "catalog one: one-profile.yml") {
t.Fatalf("Load() error = %v, want catalog source and path context", err)
}
})
}
}
func TestLoadRejectsCrossCatalogConflicts(t *testing.T) {
t.Run("duplicate backend", func(t *testing.T) {
second := catalogFS("one", map[string]string{
"catalog/profiles/second.yml": validProfile("second", "one"),
}, "null")
_, err := Load(
Source{Name: "first", ExpectedBackendID: "one", FS: validCatalogFS("one"), Root: "catalog"},
Source{Name: "second", ExpectedBackendID: "one", FS: second, Root: "catalog"},
)
if err == nil {
t.Fatal("expected duplicate backend error")
profile, err := state.GetProfile(context.Background(), id)
if err != nil || profile.ResolvedTarget.Model != want {
t.Fatalf("profile %s: %+v, %v", id, profile, err)
}
})
t.Run("duplicate profile", func(t *testing.T) {
first := catalogFS("one", map[string]string{
"catalog/profiles/shared.yml": validProfile("shared", "one"),
}, "null")
second := catalogFS("two", map[string]string{
"catalog/profiles/shared.yml": validProfile("shared", "two"),
}, "null")
_, err := Load(testSource("one", first), testSource("two", second))
if err == nil {
t.Fatal("expected duplicate profile error")
}
})
t.Run("cross-catalog base", func(t *testing.T) {
first := catalogFS("one", map[string]string{
"catalog/profiles/base.yml": validProfile("base", "one"),
}, "null")
second := catalogFS("two", map[string]string{
"catalog/profiles/child.yml": "id: child\nbase_profile: base\n",
}, "null")
_, err := Load(testSource("one", first), testSource("two", second))
if err == nil {
t.Fatal("expected cross-catalog base error")
}
})
}
}
func TestLoadReturnsDefensiveCatalogValues(t *testing.T) {
fsys := catalogFS("one", map[string]string{
"catalog/profiles/one-profile.yml": validProfile("one-profile", "one") + "extra_params:\n nested:\n value: profile\n",
}, `{"nested":{"value":"backend"}}`)
loaded, err := Load(testSource("one", fsys))
func TestConsumerOverlayReplacesCatalogDefaultsBeforeValidation(t *testing.T) {
underlays, err := Underlays()
if err != nil {
t.Fatalf("load catalog: %v", err)
t.Fatal(err)
}
loaded.Backends[0].ExtraParams["nested"].(map[string]any)["value"] = "changed"
profileValue, err := loaded.Profiles.GetProfile(context.Background(), "one-profile")
underlays = append(underlays, configkit.Underlay{Inputs: []configkit.Input{{Source: configkit.Mapping("operator", map[string]any{
"backends": map[string]any{"openrouter": map[string]any{"endpoint": "invalid lower value"}},
})}}})
main := configkit.Input{Source: configkit.Mapping("application", map[string]any{
"backends": map[string]any{"openrouter": map[string]any{"endpoint": "https://alternate.example/v1"}},
"profiles": map[string]any{"gemini-flash-latest": map[string]any{"model": "consumer-model"}},
})}
state, err := configuration.Load(main, underlays)
if err != nil {
t.Fatalf("load profile: %v", err)
t.Fatalf("replaced lower-layer value failed final validation: %v", err)
}
profileValue.ExtraParams["nested"].(map[string]any)["value"] = "changed"
again, err := Load(testSource("one", fsys))
if err != nil {
t.Fatalf("reload catalog: %v", err)
profile, err := state.GetProfile(context.Background(), "promptkit-openrouter-standard")
if err != nil || profile.ResolvedTarget.Model != "consumer-model" || profile.ResolvedTarget.Endpoint != "https://alternate.example/v1" {
t.Fatalf("maintained profile did not observe final configuration: %+v, %v", profile, err)
}
if got := again.Backends[0].ExtraParams["nested"].(map[string]any)["value"]; got != "backend" {
t.Fatalf("backend mutation escaped returned set: %#v", got)
}
againProfile, err := loaded.Profiles.GetProfile(context.Background(), "one-profile")
if err != nil {
t.Fatalf("reload profile: %v", err)
}
if got := againProfile.ExtraParams["nested"].(map[string]any)["value"]; got != "profile" {
t.Fatalf("profile mutation escaped returned value: %#v", got)
_, err = configuration.Load(configkit.Input{Source: configkit.Mapping("invalid final", map[string]any{
"backends": map[string]any{"openrouter": map[string]any{"endpoint": "invalid final value"}},
})}, underlays[:2])
var loadError *configkit.LoadError
if !errors.As(err, &loadError) {
t.Fatalf("invalid final backend did not produce a ConfigKit load error: %v", err)
}
}
func testSource(id string, fsys fs.FS) Source {
return Source{Name: id, ExpectedBackendID: id, FS: fsys, Root: "catalog"}
}
func validCatalogFS(id string) fstest.MapFS {
return catalogFS(id, map[string]string{
"catalog/profiles/" + id + "-profile.yml": validProfile(id+"-profile", id),
}, "null")
}
func catalogFS(id string, profiles map[string]string, extraParams string) fstest.MapFS {
fsys := fstest.MapFS{
"catalog/backend.json": &fstest.MapFile{Data: []byte(validManifest(id, "TEST_API_KEY", extraParams))},
}
for name, content := range profiles {
fsys[name] = &fstest.MapFile{Data: []byte(content)}
}
return fsys
}
func validManifest(id, apiKeyEnv, extraParams string) string {
return fmt.Sprintf(
`{"schema_version":1,"id":%q,"endpoint":%q,"api_key_env":%q,"concurrency_limit":2,"queue_capacity":3,"extra_params":%s}`,
id,
"https://"+id+".example/v1",
apiKeyEnv,
extraParams,
)
}
func validProfile(id, backendID string) string {
return fmt.Sprintf("id: %s\nbackend: %s\nmodel: test-model\n", id, backendID)
}
var _ fs.FS = openrouter.FS()

View File

@@ -0,0 +1,179 @@
package configuration
import (
"context"
"fmt"
"testing"
"testing/fstest"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
)
const benchmarkDefinitionCount = 128
var (
constructionResult *loaded
selectedPrompt *domain.PromptDefinition
selectedProfile *domain.ExecutionProfile
)
type configurationShape struct {
main configkit.Input
underlays []configkit.Underlay
inputEntries int
effectiveEntries int
tiers int
}
func benchmarkPrompt(profileID string) map[string]any {
return map[string]any{
"default_profile": profileID,
"messages": []any{map[string]any{"role": "user", "content": "Hello {{.name}}"}},
"output": map[string]any{"format": "text", "validation_mode": "none"},
}
}
func independentDefinitions(count int) configurationShape {
profiles := make(map[string]any, count)
prompts := make(map[string]any, count)
for index := range count {
profileID := fmt.Sprintf("profile-%03d", index)
promptID := fmt.Sprintf("prompt-%03d", index)
profiles[profileID] = map[string]any{"endpoint": "https://example.test/v1", "model": profileID}
prompts[promptID] = map[string]any{"1": benchmarkPrompt(profileID)}
}
return configurationShape{
main: configkit.Input{Source: configkit.Mapping("independent definitions", map[string]any{"profiles": profiles, "prompts": prompts})},
inputEntries: count * 2,
effectiveEntries: count * 2,
tiers: 1,
}
}
func fileBackedDefinitions(count int) configurationShape {
profiles := make(map[string]any, count)
prompts := make(map[string]any, count)
for index := range count {
profileID := fmt.Sprintf("profile-%03d", index)
promptID := fmt.Sprintf("prompt-%03d", index)
profiles[profileID] = map[string]any{"endpoint": "https://example.test/v1", "model": profileID}
prompt := benchmarkPrompt(profileID)
prompt["messages"] = []any{map[string]any{"role": "user", "content_file": "body.md"}}
prompts[promptID] = map[string]any{"1": prompt}
}
namespace := configkit.NewFilesystem("benchmark content", fstest.MapFS{
"assets/body.md": &fstest.MapFile{Data: []byte("Hello {{.name}}")},
})
return configurationShape{
main: configkit.Input{
Source: configkit.Mapping("file-backed definitions", map[string]any{"profiles": profiles, "prompts": prompts}),
Filesystem: &configkit.FilesystemContext{FS: &namespace, Root: ".", BaseDir: "assets"},
},
inputEntries: count * 2, effectiveEntries: count * 2, tiers: 1,
}
}
func sharedBaseDefinitions(count int) configurationShape {
profiles := make(map[string]any, count+1)
profiles["base"] = map[string]any{
"endpoint": "https://example.test/v1", "model": "shared-model",
"extra_params": map[string]any{"provider": map[string]any{"mode": "shared", "limit": 4}},
}
for index := range count {
profiles[fmt.Sprintf("child-%03d", index)] = map[string]any{"extends": "base", "temperature": float64(index%10) / 10}
}
return configurationShape{
main: configkit.Input{Source: configkit.Mapping("inherited definitions", map[string]any{
"profiles": profiles,
"prompts": map[string]any{"shared": map[string]any{"1": benchmarkPrompt(fmt.Sprintf("child-%03d", count-1))}},
})},
inputEntries: count + 2,
effectiveEntries: count + 2,
tiers: 1,
}
}
func layeredDefinitions(count int) configurationShape {
base := make(map[string]any, count)
operator := make(map[string]any, count)
programmatic := make(map[string]any, count)
final := make(map[string]any, count)
for index := range count {
id := fmt.Sprintf("profile-%03d", index)
base[id] = map[string]any{"endpoint": "https://example.test/v1", "model": "base-model"}
operator[id] = map[string]any{"model": "operator-model"}
programmatic[id] = map[string]any{"extra_params": map[string]any{"provider": map[string]any{"mode": "programmatic"}}}
final[id] = map[string]any{"temperature": float64(index%10) / 10}
}
return configurationShape{
underlays: []configkit.Underlay{
{Inputs: []configkit.Input{{Source: configkit.Mapping("fallback", map[string]any{"profiles": base})}}},
{Inputs: []configkit.Input{{Source: configkit.Mapping("operator", map[string]any{"profiles": operator})}}},
{Inputs: []configkit.Input{{Source: configkit.Mapping("programmatic", map[string]any{"profiles": programmatic})}}},
},
main: configkit.Input{Source: configkit.Mapping("main", map[string]any{
"profiles": final,
"prompts": map[string]any{"layered": map[string]any{"1": benchmarkPrompt(fmt.Sprintf("profile-%03d", count-1))}},
})},
inputEntries: count*4 + 1,
effectiveEntries: count + 1,
tiers: 4,
}
}
func BenchmarkConfigurationConstruction(b *testing.B) {
for _, scenario := range []struct {
name string
shape configurationShape
}{
{name: "independent", shape: independentDefinitions(benchmarkDefinitionCount)},
{name: "file_backed", shape: fileBackedDefinitions(64)},
{name: "shared_base", shape: sharedBaseDefinitions(benchmarkDefinitionCount)},
{name: "layered", shape: layeredDefinitions(benchmarkDefinitionCount)},
} {
b.Run(scenario.name, func(b *testing.B) {
b.ReportAllocs()
for b.Loop() {
result, err := load(scenario.shape.main, scenario.shape.underlays)
if err != nil {
b.Fatal(err)
}
constructionResult = result
}
b.ReportMetric(float64(scenario.shape.inputEntries), "input-entries/op")
b.ReportMetric(float64(scenario.shape.effectiveEntries), "effective-entries/op")
b.ReportMetric(float64(scenario.shape.tiers), "tiers/op")
})
}
}
func BenchmarkConfigurationSelection(b *testing.B) {
shape := independentDefinitions(benchmarkDefinitionCount)
result, err := load(shape.main, nil)
if err != nil {
b.Fatal(err)
}
b.Run("prompt", func(b *testing.B) {
b.ReportAllocs()
for b.Loop() {
definition, _, err := result.prompt("prompt-127", "1")
if err != nil {
b.Fatal(err)
}
selectedPrompt = definition
}
b.ReportMetric(float64(shape.effectiveEntries), "effective-entries/op")
})
b.Run("profile", func(b *testing.B) {
b.ReportAllocs()
for b.Loop() {
profile, err := result.GetProfile(context.Background(), "profile-127")
if err != nil {
b.Fatal(err)
}
selectedProfile = profile
}
b.ReportMetric(float64(shape.effectiveEntries), "effective-entries/op")
})
}

View File

@@ -0,0 +1,45 @@
package configuration
import (
"io"
"strings"
"gitea.maximumdirect.net/eric/ConfigKit"
)
// Inspection is the read-only subset of the retained ConfigKit result used by
// the facade. It does not expose typed configuration or resource contexts.
type Inspection interface {
Metadata() configkit.Metadata
Sources() []configkit.SourceInfo
Diagnostics() []configkit.Diagnostic
Explain(configkit.Path) (configkit.Explanation, bool)
WriteEffective(io.Writer) error
WriteExplanation(io.Writer, configkit.Path) error
}
// Inspection returns only the retained result's inspection operations.
func (state *loaded) Inspection() Inspection {
if state == nil {
return nil
}
return state.result
}
func secretProviderPath(path configkit.Path) bool {
pointer := path.String()
if pointer == "" {
return false
}
parts := strings.Split(strings.TrimPrefix(pointer, "/"), "/")
if len(parts) < 4 || (parts[0] != "backends" && parts[0] != "profiles") || parts[2] != "extra_params" {
return false
}
key := strings.ToLower(strings.ReplaceAll(strings.ReplaceAll(parts[len(parts)-1], "~1", "/"), "~0", "~"))
switch key {
case "api_key", "apikey", "authorization", "credential", "credentials", "password", "secret", "token", "access_token":
return true
default:
return false
}
}

View File

@@ -0,0 +1,317 @@
// Package configuration adapts ConfigKit's effective configuration to
// Promptkit's definition and execution boundaries.
package configuration
import (
"fmt"
"regexp"
"sort"
"strings"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit/internal/backend"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/jsonvalue"
"gitea.maximumdirect.net/eric/promptkit/internal/llm"
)
type document struct {
Backends map[string]backendConfig `yaml:"backends"`
Profiles map[string]profileConfig `yaml:"profiles"`
Prompts map[string]map[string]promptConfig `yaml:"prompts"`
}
type backendConfig struct {
Endpoint *string `yaml:"endpoint"`
APIKeyEnv *string `yaml:"api_key_env"`
ConcurrencyLimit *int `yaml:"concurrency_limit"`
QueueCapacity *int `yaml:"queue_capacity"`
ExtraParams map[string]any `yaml:"extra_params"`
}
type profileConfig struct {
Backend *string `yaml:"backend"`
Endpoint *string `yaml:"endpoint"`
Model *string `yaml:"model"`
Temperature *float64 `yaml:"temperature"`
TopP *float64 `yaml:"top_p"`
MaxTokens *int `yaml:"max_tokens"`
TimeoutSeconds *int `yaml:"timeout_seconds"`
ServiceTier *string `yaml:"service_tier"`
ReasoningEffort *string `yaml:"reasoning_effort"`
APIKeyEnv *string `yaml:"api_key_env"`
APIKeyRequired *bool `yaml:"api_key_required"`
ExtraParams map[string]any `yaml:"extra_params"`
}
type promptConfig struct {
DefaultProfile *string `yaml:"default_profile"`
Description *string `yaml:"description"`
SessionID *string `yaml:"session_id"`
Inputs []promptInputConfig `yaml:"inputs"`
Messages []promptMessageConfig `yaml:"messages"`
Output *promptOutputConfig `yaml:"output"`
}
type promptInputConfig struct {
Name string `yaml:"name"`
Required bool `yaml:"required"`
ContentType string `yaml:"content_type"`
Description string `yaml:"description"`
}
type promptMessageConfig struct {
Role *string `yaml:"role"`
Content *string `yaml:"content"`
ContentFile *string `yaml:"content_file"`
CacheControl *cacheControlConfig `yaml:"cache_control"`
}
type cacheControlConfig struct {
Type *string `yaml:"type"`
TTL *string `yaml:"ttl"`
}
type promptOutputConfig struct {
Format *string `yaml:"format"`
ValidationMode *string `yaml:"validation_mode"`
SchemaPath *string `yaml:"schema_path"`
RepairAttempts int `yaml:"repair_attempts"`
}
// fieldState distinguishes omission, an explicit value, and an explicit null.
// Inherited values are explicit because ConfigKit has already expanded them.
type fieldState uint8
const (
fieldAbsent fieldState = iota
fieldValue
fieldNull
)
type loaded struct {
result *configkit.Result[document]
backends map[string]map[string]fieldState
profiles map[string]map[string]fieldState
prompts map[string]map[string]*promptEntry
ambiguous map[string]bool
registry *backend.Registry
indexedProfiles map[string]domain.ExecutionProfile
}
var envName = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
var backendFields = []string{"endpoint", "api_key_env", "concurrency_limit", "queue_capacity", "extra_params"}
var profileFields = []string{"backend", "endpoint", "model", "temperature", "top_p", "max_tokens", "timeout_seconds", "service_tier", "reasoning_effort", "api_key_env", "api_key_required", "extra_params"}
// Load captures one immutable effective configuration for internal consumers.
func Load(main configkit.Input, underlays []configkit.Underlay) (*loaded, error) {
return load(main, underlays)
}
// load performs one ConfigKit lifecycle over the selected source tiers.
func load(main configkit.Input, underlays []configkit.Underlay) (*loaded, error) {
state := &loaded{
backends: make(map[string]map[string]fieldState),
profiles: make(map[string]map[string]fieldState),
}
result, err := configkit.LoadLayered(main, configkit.LayeredOptions[document]{
Underlays: underlays,
Definition: configkit.Definition[document]{
Secret: secretProviderPath,
Resolve: func(config *document, resolution *configkit.Resolution) error {
captureFields(config.Backends, "backends", backendFields, resolution, state.backends)
captureFields(config.Profiles, "profiles", profileFields, resolution, state.profiles)
resolveBackends(config.Backends, resolution)
resolveProfiles(config.Profiles, resolution)
resolvePrompts(config.Prompts, resolution)
return nil
},
Validate: validateEffective,
},
})
if err != nil {
return nil, fmt.Errorf("load configuration: %w", err)
}
state.result = result
if err := state.materializePrompts(); err != nil {
return nil, fmt.Errorf("load configuration: %w", err)
}
if err := state.buildExecutionProfiles(); err != nil {
return nil, fmt.Errorf("load configuration: %w", err)
}
return state, nil
}
func captureFields[T any](entries map[string]T, collection string, fields []string, resolution *configkit.Resolution, destination map[string]map[string]fieldState) {
for id := range entries {
states := make(map[string]fieldState, len(fields))
for _, field := range fields {
info, exists := resolution.InputInfo(configkit.Keys(collection, id, field))
switch {
case !exists || !info.Explicit:
states[field] = fieldAbsent
case info.Null:
states[field] = fieldNull
default:
states[field] = fieldValue
}
}
destination[id] = states
}
}
func sortedKeys[T any](entries map[string]T) []string {
keys := make([]string, 0, len(entries))
for key := range entries {
keys = append(keys, key)
}
sort.Strings(keys)
return keys
}
func normalizeString(value **string, path configkit.Path, resolution *configkit.Resolution, normalized string) bool {
if value != nil && *value != nil && **value != normalized {
configkit.Normalize(resolution, path, value, &normalized, "normalize configuration value")
return true
}
return false
}
func resolveBackends(backends map[string]backendConfig, resolution *configkit.Resolution) {
for _, id := range sortedKeys(backends) {
backend := backends[id]
base := configkit.Keys("backends", id)
if backend.Endpoint != nil {
endpoint, err := domain.NormalizeOpenAICompatibleBaseEndpoint(*backend.Endpoint)
if err != nil {
resolution.Error(base.Key("endpoint"), err.Error())
} else {
normalizeString(&backend.Endpoint, base.Key("endpoint"), resolution, endpoint)
}
}
if backend.APIKeyEnv != nil {
normalizeString(&backend.APIKeyEnv, base.Key("api_key_env"), resolution, strings.TrimSpace(*backend.APIKeyEnv))
}
if backend.ConcurrencyLimit != nil && *backend.ConcurrencyLimit > 0 && backend.QueueCapacity == nil {
capacity := 1024
if resolution.IsExplicit(base.Key("queue_capacity")) {
configkit.Normalize(resolution, base.Key("queue_capacity"), &backend.QueueCapacity, &capacity, "select default queue capacity")
} else {
configkit.Default(resolution, base.Key("queue_capacity"), &backend.QueueCapacity, &capacity, "select default queue capacity")
}
}
backends[id] = backend
}
}
func resolveProfiles(profiles map[string]profileConfig, resolution *configkit.Resolution) {
for _, id := range sortedKeys(profiles) {
profile := profiles[id]
base := configkit.Keys("profiles", id)
if profile.Endpoint != nil && *profile.Endpoint != "" {
endpoint, err := domain.NormalizeOpenAICompatibleBaseEndpoint(*profile.Endpoint)
if err != nil {
resolution.Error(base.Key("endpoint"), err.Error())
} else {
normalizeString(&profile.Endpoint, base.Key("endpoint"), resolution, endpoint)
}
}
for _, field := range []struct {
name string
value **string
}{
{"service_tier", &profile.ServiceTier},
{"reasoning_effort", &profile.ReasoningEffort},
{"api_key_env", &profile.APIKeyEnv},
} {
if *field.value != nil {
normalizeString(field.value, base.Key(field.name), resolution, strings.TrimSpace(**field.value))
}
}
profiles[id] = profile
}
}
func validateEffective(config *document, validation *configkit.Validation) {
for _, id := range sortedKeys(config.Backends) {
backend := config.Backends[id]
base := configkit.Keys("backends", id)
if strings.TrimSpace(id) == "" {
validation.Error(base, "backend ID must not be blank")
}
if backend.Endpoint == nil || *backend.Endpoint == "" {
validation.Error(base.Key("endpoint"), "endpoint is required")
}
if backend.APIKeyEnv != nil && *backend.APIKeyEnv != "" && !envName.MatchString(*backend.APIKeyEnv) {
validation.Error(base.Key("api_key_env"), "environment variable name is invalid")
}
if backend.ConcurrencyLimit != nil && *backend.ConcurrencyLimit < 0 {
validation.Error(base.Key("concurrency_limit"), "concurrency limit must not be negative")
}
if backend.QueueCapacity != nil {
if *backend.QueueCapacity < 0 {
validation.Error(base.Key("queue_capacity"), "queue capacity must not be negative")
}
if backend.ConcurrencyLimit == nil || *backend.ConcurrencyLimit <= 0 {
validation.Error(base.Key("queue_capacity"), "queue capacity requires a positive concurrency limit")
} else if *backend.QueueCapacity > int(^uint(0)>>1)-*backend.ConcurrencyLimit {
validation.Error(base.Key("queue_capacity"), "total capacity overflows int")
}
}
validateExtraParams(backend.ExtraParams, base.Key("extra_params"), true, validation)
}
for _, id := range sortedKeys(config.Profiles) {
profile := config.Profiles[id]
base := configkit.Keys("profiles", id)
if strings.TrimSpace(id) == "" {
validation.Error(base, "profile ID must not be blank")
}
if profile.Backend != nil && *profile.Backend != "" {
if _, exists := config.Backends[*profile.Backend]; !exists {
validation.Error(base.Key("backend"), "backend is not defined")
}
}
if profile.Model != nil && *profile.Model != "" && strings.TrimSpace(*profile.Model) == "" {
validation.Error(base.Key("model"), "model must not be blank")
}
if profile.Temperature != nil {
if err := domain.ValidateExecutionTargetSettings(domain.ExecutionTarget{Temperature: *profile.Temperature}); err != nil {
validation.Error(base.Key("temperature"), err.Error())
}
}
if profile.TopP != nil {
if err := domain.ValidateExecutionTargetSettings(domain.ExecutionTarget{TopP: *profile.TopP}); err != nil {
validation.Error(base.Key("top_p"), err.Error())
}
}
if profile.MaxTokens != nil {
if err := domain.ValidateExecutionTargetSettings(domain.ExecutionTarget{MaxTokens: *profile.MaxTokens}); err != nil {
validation.Error(base.Key("max_tokens"), err.Error())
}
}
if profile.TimeoutSeconds != nil {
if err := domain.ValidateExecutionTargetSettings(domain.ExecutionTarget{TimeoutSeconds: *profile.TimeoutSeconds}); err != nil {
validation.Error(base.Key("timeout_seconds"), err.Error())
}
}
validateExtraParams(profile.ExtraParams, base.Key("extra_params"), false, validation)
}
}
func validateExtraParams(values map[string]any, path configkit.Path, backend bool, validation *configkit.Validation) {
if values == nil {
return
}
if _, err := jsonvalue.CopyMap(values); err != nil {
validation.Error(path, "extra parameters must be a JSON-compatible map with nonempty keys")
return
}
if backend {
for key := range values {
if llm.IsReservedOpenAIChatRequestField(key) {
validation.Error(path.Key(key), "extra parameter conflicts with a request field")
}
}
}
}

View File

@@ -0,0 +1,138 @@
package configuration
import (
"errors"
"testing"
"gitea.maximumdirect.net/eric/ConfigKit"
)
func mapping(label string, values map[string]any) configkit.Input {
return configkit.Input{Source: configkit.Mapping(label, values)}
}
func TestLoadComposesLiteralKeysAndEffectivePresence(t *testing.T) {
base := mapping("defaults", map[string]any{
"backends": map[string]any{
"local.v1": map[string]any{"endpoint": "http://localhost:8080/v1", "concurrency_limit": 2},
},
"profiles": map[string]any{
"base/one": map[string]any{"backend": "local.v1", "temperature": 0.8, "top_p": 0.9, "api_key_required": true,
"extra_params": map[string]any{"provider": map[string]any{"first": 1, "second": 2}}},
"child.v1": map[string]any{"extends": "base/one"},
},
"prompts": map[string]any{"invoice/summary": map[string]any{"v1.0": map[string]any{
"messages": []any{map[string]any{"role": "user", "content": "Summarize."}},
"output": map[string]any{"format": "text", "validation_mode": "basic"},
}}},
})
overlay := mapping("operator", map[string]any{
"backends": map[string]any{"local.v1": map[string]any{"queue_capacity": 0}},
"profiles": map[string]any{"child.v1": map[string]any{
"temperature": 0, "top_p": nil, "api_key_required": false,
"extra_params": map[string]any{"provider": map[string]any{"second": 3}},
}},
})
loaded, err := load(configkit.Input{Source: configkit.NoSource()}, []configkit.Underlay{
{Inputs: []configkit.Input{base}}, {Inputs: []configkit.Input{overlay}},
})
if err != nil {
t.Fatal(err)
}
profile := loaded.result.Config.Profiles["child.v1"]
if profile.Temperature == nil || *profile.Temperature != 0 || profile.TopP != nil || profile.APIKeyRequired == nil || *profile.APIKeyRequired {
t.Fatalf("effective values were not retained: %+v", profile)
}
if loaded.profiles["child.v1"]["temperature"] != fieldValue || loaded.profiles["child.v1"]["top_p"] != fieldNull || loaded.profiles["child.v1"]["api_key_required"] != fieldValue {
t.Fatalf("effective presence was lost: %+v", loaded.profiles["child.v1"])
}
provider := profile.ExtraParams["provider"].(map[string]any)
if provider["first"] != int64(1) || provider["second"] != int64(3) {
t.Fatalf("recursive overlay failed: %+v", provider)
}
backend := loaded.result.Config.Backends["local.v1"]
if backend.QueueCapacity == nil || *backend.QueueCapacity != 0 || loaded.backends["local.v1"]["queue_capacity"] != fieldValue {
t.Fatalf("explicit zero queue capacity was lost: %+v", backend)
}
if loaded.result.Config.Prompts["invoice/summary"]["v1.0"].Messages[0].Content == nil {
t.Fatal("literal prompt ID and version were not decoded")
}
}
func TestLoadRecordsBackendNormalizationAndNullDefault(t *testing.T) {
loaded, err := load(mapping("backend", map[string]any{"backends": map[string]any{
"local": map[string]any{"endpoint": " HTTP://localhost:8080/v1 ", "api_key_env": " TOKEN_NAME ",
"concurrency_limit": 2, "queue_capacity": nil},
}}), nil)
if err != nil {
t.Fatal(err)
}
backend := loaded.result.Config.Backends["local"]
if backend.Endpoint == nil || *backend.Endpoint != "http://localhost:8080/v1" ||
backend.APIKeyEnv == nil || *backend.APIKeyEnv != "TOKEN_NAME" ||
backend.QueueCapacity == nil || *backend.QueueCapacity != 1024 {
t.Fatalf("backend normalization/default was not applied: %+v", backend)
}
if loaded.backends["local"]["queue_capacity"] != fieldNull {
t.Fatalf("explicit null was not retained: %+v", loaded.backends["local"])
}
}
func TestLoadDefaultsAbsentQueueCapacity(t *testing.T) {
loaded, err := load(mapping("backend", map[string]any{"backends": map[string]any{
"local": map[string]any{"endpoint": "http://localhost:8080/v1", "concurrency_limit": 1},
}}), nil)
if err != nil {
t.Fatal(err)
}
backend := loaded.result.Config.Backends["local"]
if backend.QueueCapacity == nil || *backend.QueueCapacity != 1024 || loaded.backends["local"]["queue_capacity"] != fieldAbsent {
t.Fatalf("absent queue capacity did not receive its default: %+v", backend)
}
}
func TestLoadRejectsStrictGroupConflict(t *testing.T) {
_, err := load(configkit.Input{Source: configkit.NoSource()}, []configkit.Underlay{{Inputs: []configkit.Input{
mapping("first", map[string]any{"profiles": map[string]any{"shared": map[string]any{"model": "one"}}}),
mapping("second", map[string]any{"profiles": map[string]any{"shared": map[string]any{"model": "one"}}}),
}}})
var loadError *configkit.LoadError
if !errors.As(err, &loadError) {
t.Fatalf("expected ConfigKit load error, got %v", err)
}
}
func TestLoadValidatesOnlyFinalDomainValues(t *testing.T) {
underlay := mapping("defaults", map[string]any{
"backends": map[string]any{"local": map[string]any{"endpoint": "not a URL"}},
"profiles": map[string]any{"partial": map[string]any{"backend": "local"}},
})
main := mapping("operator", map[string]any{"backends": map[string]any{"local": map[string]any{"endpoint": "http://localhost:8080/v1"}}})
loaded, err := load(main, []configkit.Underlay{{Inputs: []configkit.Input{underlay}}})
if err != nil {
t.Fatalf("replaced lower-layer domain value should not fail: %v", err)
}
if loaded.result.Config.Profiles["partial"].Model != nil {
t.Fatal("unused partial profile unexpectedly gained a model")
}
_, err = load(configkit.Input{Source: configkit.NoSource()}, []configkit.Underlay{{Inputs: []configkit.Input{
mapping("invalid", map[string]any{"backends": map[string]any{"unused": map[string]any{"endpoint": "not a URL"}}}),
}}})
var loadError *configkit.LoadError
if !errors.As(err, &loadError) {
t.Fatalf("expected path-aware effective backend failure, got %v", err)
}
if diagnostics := loadError.Diagnostics(); len(diagnostics) != 1 || diagnostics[0].Path.String() != "/backends/unused/endpoint" {
t.Fatalf("expected endpoint diagnostic, got %+v", diagnostics)
}
}
func TestLoadKeepsSyntaxErrorsFromLowerTiers(t *testing.T) {
_, err := load(mapping("operator", map[string]any{"backends": map[string]any{"local": map[string]any{"endpoint": "http://localhost:8080"}}}),
[]configkit.Underlay{{Inputs: []configkit.Input{{Source: configkit.Bytes("broken", []byte("backends: ["))}}}})
var loadError *configkit.LoadError
if !errors.As(err, &loadError) {
t.Fatalf("expected lower-tier parsing error, got %v", err)
}
}

View File

@@ -0,0 +1,331 @@
package configuration
import (
"context"
"errors"
"fmt"
"io"
"strings"
"text/template"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/filecatalog"
"gitea.maximumdirect.net/eric/promptkit/internal/prompt"
"gitea.maximumdirect.net/eric/promptkit/internal/promptdef"
)
var (
errPromptNotFound = errors.New("prompt definition not found")
errPromptAmbiguous = errors.New("prompt version is ambiguous")
)
type promptEntry struct {
definition domain.PromptDefinition
inspection domain.PromptInspection
hash string
session *template.Template
messages []*template.Template
}
func resolvePrompts(prompts map[string]map[string]promptConfig, resolution *configkit.Resolution) {
for _, id := range sortedKeys(prompts) {
for _, version := range sortedKeys(prompts[id]) {
definition := prompts[id][version]
base := configkit.Keys("prompts", id, version)
for _, field := range []struct {
name string
value **string
}{
{"description", &definition.Description},
{"session_id", &definition.SessionID},
} {
if *field.value != nil {
normalizeString(field.value, base.Key(field.name), resolution, strings.TrimSpace(**field.value))
}
}
inputsChanged := false
for index := range definition.Inputs {
input := &definition.Inputs[index]
inputPath := base.Key("inputs").Index(index)
for _, field := range []struct {
name string
value *string
}{
{"name", &input.Name},
{"content_type", &input.ContentType},
{"description", &input.Description},
} {
normalized := strings.TrimSpace(*field.value)
if normalized != *field.value {
configkit.Normalize(resolution, inputPath.Key(field.name), field.value, normalized, "normalize input metadata")
inputsChanged = true
}
}
}
if inputsChanged {
resolution.Record(base.Key("inputs"), configkit.ChangeNormalization, "normalize input metadata")
}
// Normalize message metadata on execution-owned copies below. ConfigKit
// records sequence mutations as whole-value replacements, which would
// discard the effective source contexts of unchanged content_file fields.
if definition.Output != nil && definition.Output.SchemaPath != nil {
normalizeString(&definition.Output.SchemaPath, base.Key("output").Key("schema_path"), resolution, strings.TrimSpace(*definition.Output.SchemaPath))
}
prompts[id][version] = definition
}
}
}
func (state *loaded) materializePrompts() error {
state.prompts = make(map[string]map[string]*promptEntry, len(state.result.Config.Prompts))
state.ambiguous = make(map[string]bool)
var metadata *configkit.Metadata
lookupOrigin := func(path configkit.Path) (configkit.FieldInfo, bool) {
if metadata == nil {
captured := state.result.Metadata()
metadata = &captured
}
return metadata.Lookup(path)
}
for _, id := range sortedKeys(state.result.Config.Prompts) {
if strings.TrimSpace(id) == "" {
return fmt.Errorf("%s: prompt ID must not be blank", configkit.Keys("prompts", id))
}
versions := state.result.Config.Prompts[id]
if versions == nil {
return fmt.Errorf("%s: prompt collection must be a mapping", configkit.Keys("prompts", id))
}
state.prompts[id] = make(map[string]*promptEntry, len(versions))
state.ambiguous[id] = len(versions) > 1
for _, version := range sortedKeys(versions) {
if strings.TrimSpace(version) == "" {
return fmt.Errorf("%s: prompt version must not be blank", configkit.Keys("prompts", id, version))
}
entry, err := state.materializePrompt(id, version, versions[version], lookupOrigin)
if err != nil {
return err
}
state.prompts[id][version] = entry
}
}
return nil
}
func (state *loaded) materializePrompt(id, version string, raw promptConfig, lookupOrigin func(configkit.Path) (configkit.FieldInfo, bool)) (*promptEntry, error) {
base := configkit.Keys("prompts", id, version)
if len(raw.Messages) == 0 {
return nil, fmt.Errorf("%s: at least one message is required", base.Key("messages"))
}
definition := domain.PromptDefinition{ID: id, Version: version}
if raw.DefaultProfile != nil {
if strings.TrimSpace(*raw.DefaultProfile) == "" {
return nil, fmt.Errorf("%s: default profile must not be blank", base.Key("default_profile"))
}
definition.DefaultProfile = *raw.DefaultProfile
}
if raw.Description != nil {
definition.Description = *raw.Description
}
if raw.SessionID != nil {
definition.SessionID = *raw.SessionID
}
seenInputs := make(map[string]bool, len(raw.Inputs))
definition.Inputs = make([]domain.PromptInput, 0, len(raw.Inputs))
for index, input := range raw.Inputs {
path := base.Key("inputs").Index(index)
if input.Name == "" {
return nil, fmt.Errorf("%s: input name is required", path.Key("name"))
}
if seenInputs[input.Name] {
return nil, fmt.Errorf("%s: input name is duplicated", path.Key("name"))
}
seenInputs[input.Name] = true
definition.Inputs = append(definition.Inputs, domain.PromptInput{
Name: input.Name, Required: input.Required, ContentType: input.ContentType, Description: input.Description,
})
}
entry := &promptEntry{messages: make([]*template.Template, 0, len(raw.Messages))}
definition.Templates = make([]domain.PromptMessageTemplate, 0, len(raw.Messages))
for index, message := range raw.Messages {
path := base.Key("messages").Index(index)
if message.Role == nil {
return nil, fmt.Errorf("%s: role is required", path.Key("role"))
}
role, err := domain.NormalizeMessageRole(*message.Role)
if err != nil {
return nil, fmt.Errorf("%s: %w", path.Key("role"), err)
}
if (message.Content != nil) == (message.ContentFile != nil) {
return nil, fmt.Errorf("%s: exactly one of content or content_file is required", path)
}
content := ""
if message.Content != nil {
content = *message.Content
} else {
var err error
content, err = state.readContent(path.Key("content_file"), *message.ContentFile, lookupOrigin)
if err != nil {
return nil, err
}
}
var control *domain.CacheControl
if message.CacheControl != nil {
if message.CacheControl.Type == nil {
return nil, fmt.Errorf("%s: type is required", path.Key("cache_control").Key("type"))
}
control = &domain.CacheControl{Type: domain.CacheControlType(*message.CacheControl.Type)}
if message.CacheControl.TTL != nil {
control.TTL = *message.CacheControl.TTL
}
var err error
control, err = domain.NormalizeCacheControl(control)
if err != nil {
return nil, fmt.Errorf("%s: %w", path.Key("cache_control"), err)
}
}
compiled, err := prompt.CompileTemplate(fmt.Sprintf("msg_%d", index), content)
if err != nil {
return nil, fmt.Errorf("%s: invalid template syntax: %w", path, err)
}
entry.messages = append(entry.messages, compiled)
definition.Templates = append(definition.Templates, domain.PromptMessageTemplate{
Role: role, Content: content, CacheControl: control,
})
}
if raw.Output == nil {
return nil, fmt.Errorf("%s: output is required", base.Key("output"))
}
if raw.Output.Format == nil || raw.Output.ValidationMode == nil {
return nil, fmt.Errorf("%s: format and validation_mode are required", base.Key("output"))
}
contract := domain.OutputContract{
Format: domain.OutputFormat(*raw.Output.Format),
ValidationMode: domain.ValidationMode(*raw.Output.ValidationMode),
RepairAttempts: raw.Output.RepairAttempts,
}
if raw.Output.SchemaPath != nil {
contract.SchemaPath = *raw.Output.SchemaPath
}
if err := domain.ValidateOutputContract(contract); err != nil {
return nil, fmt.Errorf("%s: invalid output contract: %w", base.Key("output"), err)
}
definition.OutputFormat = contract.Format
definition.Validation = contract
if raw.SessionID != nil {
var err error
entry.session, err = prompt.CompileTemplate("session_id", definition.SessionID)
if err != nil {
return nil, fmt.Errorf("%s: invalid template syntax: %w", base.Key("session_id"), err)
}
}
hash, err := promptdef.HashDefinition(&definition)
if err != nil {
return nil, fmt.Errorf("%s: hash prompt definition: %w", base, err)
}
entry.definition = definition
entry.hash = hash
entry.inspection = domain.PromptInspection{
PromptID: id, PromptVersion: version, PromptHash: hash,
DefaultProfileID: definition.DefaultProfile, Inputs: append([]domain.PromptInput(nil), definition.Inputs...),
OutputContract: contract,
}
return entry, nil
}
func (state *loaded) readContent(fieldPath configkit.Path, name string, lookupOrigin func(configkit.Path) (configkit.FieldInfo, bool)) (string, error) {
if strings.TrimSpace(name) == "" {
return "", fmt.Errorf("%s: content_file path must not be blank", fieldPath)
}
field, found := lookupOrigin(fieldPath)
if !found || len(field.Origin.Locations) != 1 {
return "", fmt.Errorf("%s: content_file needs one contributing source with a filesystem context", fieldPath)
}
sourceID := field.Origin.Locations[0].SourceID
context, found := state.result.SourceContext(sourceID)
if !found || context.FS == nil {
return "", fmt.Errorf("%s: content_file source %s has no filesystem context", fieldPath, sourceID)
}
resolved, _, err := filecatalog.ResolveFSPath(".", context.BaseDir, name)
if err != nil {
return "", fmt.Errorf("%s: content_file path is outside its source root or invalid: %w", fieldPath, err)
}
file, err := context.FS.Open(resolved)
if err != nil {
return "", fmt.Errorf("%s: open content_file from source %s: %w", fieldPath, sourceID, err)
}
content, readErr := io.ReadAll(file)
closeErr := file.Close()
if readErr != nil {
return "", fmt.Errorf("%s: read content_file from source %s: %w", fieldPath, sourceID, readErr)
}
if closeErr != nil {
return "", fmt.Errorf("%s: close content_file from source %s: %w", fieldPath, sourceID, closeErr)
}
return string(content), nil
}
func (state *loaded) prompt(id, version string) (*domain.PromptDefinition, domain.PromptInspection, error) {
if state == nil || strings.TrimSpace(id) == "" || (version != "" && strings.TrimSpace(version) == "") {
return nil, domain.PromptInspection{}, errPromptNotFound
}
versions := state.prompts[id]
if version == "" {
if state.ambiguous[id] {
return nil, domain.PromptInspection{}, errPromptAmbiguous
}
for selected := range versions {
version = selected
}
}
entry := versions[version]
if entry == nil {
return nil, domain.PromptInspection{}, errPromptNotFound
}
definition := entry.definition
definition.Inputs = append([]domain.PromptInput(nil), definition.Inputs...)
definition.Templates = append([]domain.PromptMessageTemplate(nil), definition.Templates...)
for index := range definition.Templates {
if control := definition.Templates[index].CacheControl; control != nil {
copy := *control
definition.Templates[index].CacheControl = &copy
}
}
inspection := entry.inspection
inspection.Inputs = append([]domain.PromptInput(nil), inspection.Inputs...)
return &definition, inspection, nil
}
// GetPromptSnapshot returns an independently owned captured definition and its
// construction-time hash. Lookup retains literal ID and version keys.
func (state *loaded) GetPromptSnapshot(ctx context.Context, id, version string) (*domain.PromptDefinition, string, error) {
if err := ctx.Err(); err != nil {
return nil, "", err
}
definition, inspection, err := state.prompt(id, version)
if errors.Is(err, errPromptNotFound) {
return nil, "", promptdef.ErrPromptDefinitionNotFound
}
if err != nil {
return nil, "", err
}
return definition, inspection.PromptHash, nil
}
// GetPromptDefinition implements the runner's prompt lookup boundary.
func (state *loaded) GetPromptDefinition(ctx context.Context, id, version string) (*domain.PromptDefinition, error) {
definition, _, err := state.GetPromptSnapshot(ctx, id, version)
return definition, err
}
// GetCompiledTemplates returns construction-time parses for one exact prompt.
func (state *loaded) GetCompiledTemplates(id, version string) (prompt.CompiledTemplates, bool) {
if state == nil {
return prompt.CompiledTemplates{}, false
}
entry := state.prompts[id][version]
if entry == nil {
return prompt.CompiledTemplates{}, false
}
return prompt.CompiledTemplates{Session: entry.session, Messages: entry.messages}, true
}

View File

@@ -0,0 +1,206 @@
package configuration
import (
"errors"
"io/fs"
"os"
"path/filepath"
"strings"
"testing"
"testing/fstest"
"gitea.maximumdirect.net/eric/ConfigKit"
)
func TestLoadCapturesInheritedImportedContent(t *testing.T) {
files := fstest.MapFS{
"app/main.yaml": {Data: []byte("configkit:\n imports:\n - path: ../catalog/base.yaml\nprompts:\n child:\n extends: base\n")},
"catalog/base.yaml": {Data: []byte("prompts:\n base:\n v1:\n messages:\n - role: ' USER '\n cache_control:\n type: ' ephemeral '\n ttl: ' 1h '\n content_file: ./body.md\n output:\n format: text\n validation_mode: basic\n")},
"catalog/body.md": {Data: []byte("Captured {{.name}}")},
}
namespace := configkit.NewFilesystem("embedded", files)
state, err := load(configkit.Input{
Source: namespace.File("app/main.yaml"), Imports: true,
Filesystem: &configkit.FilesystemContext{FS: &namespace, Root: "."},
}, nil)
if err != nil {
var loadError *configkit.LoadError
if errors.As(err, &loadError) {
t.Fatalf("%v: %+v", err, loadError.Diagnostics())
}
t.Fatal(err)
}
definition, inspection, err := state.prompt("child", "")
if err != nil {
t.Fatal(err)
}
if definition.Templates[0].Content != "Captured {{.name}}" || inspection.PromptHash == "" {
t.Fatalf("inherited file content was not captured: %+v, %+v", definition, inspection)
}
message := definition.Templates[0]
if message.Role != "user" || message.CacheControl == nil || message.CacheControl.Type != "ephemeral" || message.CacheControl.TTL != "1h" {
t.Fatalf("message metadata was not normalized: %+v", message)
}
field, ok := state.result.Metadata().Lookup(configkit.Keys("prompts", "child", "v1", "messages").Index(0).Key("content_file"))
if !ok || len(field.Origin.Locations) != 1 {
t.Fatalf("content origin was lost during normalization: %+v", field)
}
var originPath string
for _, source := range state.result.Sources() {
if source.ID == field.Origin.Locations[0].SourceID {
originPath = source.Path
}
}
if originPath != "catalog/base.yaml" {
t.Fatalf("content origin = %q, want catalog/base.yaml", originPath)
}
files["catalog/body.md"].Data = []byte("Changed")
again, againInspection, err := state.prompt("child", "v1")
if err != nil || again.Templates[0].Content != "Captured {{.name}}" || againInspection.PromptHash != inspection.PromptHash {
t.Fatalf("captured definition changed after source mutation: %+v, %+v, %v", again, againInspection, err)
}
definition.Templates[0].Content = "caller mutation"
third, _, err := state.prompt("child", "v1")
if err != nil || third.Templates[0].Content != "Captured {{.name}}" {
t.Fatalf("caller mutation escaped into index: %+v, %v", third, err)
}
}
func TestLoadUsesExplicitMappingContextAndClosesContent(t *testing.T) {
filesystem := &countingFS{FS: fstest.MapFS{
"assets/body.md": {Data: []byte("From mapping")},
}}
namespace := configkit.NewFilesystem("mapping-context", filesystem)
source := configkit.Mapping("application", promptWithFile("body.md"))
state, err := load(configkit.Input{
Source: source,
Filesystem: &configkit.FilesystemContext{FS: &namespace, Root: ".", BaseDir: "assets"},
}, nil)
if err != nil {
t.Fatal(err)
}
definition, _, err := state.prompt("example", "v1")
if err != nil || definition.Templates[0].Content != "From mapping" || filesystem.contentOpened != 1 || filesystem.contentClosed != 1 {
t.Fatalf("mapping context or handle ownership failed: %+v, opens=%d closes=%d, %v", definition, filesystem.contentOpened, filesystem.contentClosed, err)
}
_, err = load(mapping("contextless", promptWithFile("body.md")), nil)
if err == nil || !strings.Contains(err.Error(), "/prompts/example/v1/messages/0/content_file") {
t.Fatalf("expected semantic path for contextless resource, got %v", err)
}
}
func TestLoadCapturesNativeContentFile(t *testing.T) {
root := t.TempDir()
definitionPath := filepath.Join(root, "definition.yaml")
contentPath := filepath.Join(root, "body.md")
if err := os.WriteFile(definitionPath, []byte("prompts:\n example:\n v1:\n messages:\n - role: user\n content_file: body.md\n output:\n format: text\n validation_mode: basic\n"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(contentPath, []byte("Native body"), 0o600); err != nil {
t.Fatal(err)
}
state, err := load(configkit.Input{Source: configkit.File(definitionPath), Filesystem: &configkit.FilesystemContext{Root: root}}, nil)
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(contentPath, []byte("Modified"), 0o600); err != nil {
t.Fatal(err)
}
definition, _, err := state.prompt("example", "v1")
if err != nil || definition.Templates[0].Content != "Native body" {
t.Fatalf("native content was not captured: %+v, %v", definition, err)
}
}
func TestLoadRejectsMalformedStaticSyntaxAndAmbiguousSelection(t *testing.T) {
_, err := load(mapping("bad-template", map[string]any{"prompts": map[string]any{"example": map[string]any{
"v1": map[string]any{"messages": []any{map[string]any{"role": "user", "content": "{{if}}"}},
"output": map[string]any{"format": "text", "validation_mode": "basic"}},
}}}), nil)
if err == nil || !strings.Contains(err.Error(), "invalid template syntax") {
t.Fatalf("expected construction-time syntax error, got %v", err)
}
state, err := load(mapping("versions", map[string]any{"prompts": map[string]any{"example": map[string]any{
"v1": map[string]any{"messages": []any{map[string]any{"role": "user", "content": "one"}},
"output": map[string]any{"format": "text", "validation_mode": "basic"}},
"v2": map[string]any{"messages": []any{map[string]any{"role": "user", "content": "two"}},
"output": map[string]any{"format": "text", "validation_mode": "basic"}},
}}}), nil)
if err != nil {
t.Fatal(err)
}
if _, _, err := state.prompt("example", ""); !errors.Is(err, errPromptAmbiguous) {
t.Fatalf("expected ambiguous version selection, got %v", err)
}
}
func TestLoadAcceptsEmptyInlineContentAndRejectsInvalidPromptFields(t *testing.T) {
state, err := load(mapping("empty-content", map[string]any{"prompts": map[string]any{" exact.id ": map[string]any{"v1": map[string]any{
"inputs": []any{map[string]any{"name": " note ", "required": true}},
"messages": []any{map[string]any{"role": " USER ", "content": "", "content_file": nil,
"cache_control": map[string]any{"type": " ephemeral ", "ttl": "1h "}}},
"output": map[string]any{"format": "text", "validation_mode": "basic"},
}}}}), nil)
if err != nil {
var loadError *configkit.LoadError
if errors.As(err, &loadError) {
t.Fatalf("%v: %+v", err, loadError.Diagnostics())
}
t.Fatal(err)
}
definition, _, err := state.prompt(" exact.id ", "v1")
if err != nil || definition.ID != " exact.id " || definition.Templates[0].Content != "" || definition.Templates[0].Role != "user" ||
definition.Inputs[0].Name != "note" || definition.Templates[0].CacheControl.TTL != "1h" {
t.Fatalf("explicit empty content or literal ID was changed: %+v, %v", definition, err)
}
if _, _, err := state.prompt("exact.id", "v1"); !errors.Is(err, errPromptNotFound) {
t.Fatalf("trimmed prompt identity unexpectedly matched: %v", err)
}
_, err = load(mapping("obsolete-field", map[string]any{"prompts": map[string]any{"example": map[string]any{"v1": map[string]any{
"id": "example", "messages": []any{map[string]any{"role": "user", "content": "ok"}},
"output": map[string]any{"format": "text", "validation_mode": "basic"},
}}}}), nil)
var loadError *configkit.LoadError
if !errors.As(err, &loadError) {
t.Fatalf("obsolete identity field should fail strict decoding: %v", err)
}
_, err = load(mapping("null-prompt", map[string]any{"prompts": map[string]any{"example": nil}}), nil)
if err == nil {
t.Fatal("null prompt collection was accepted")
}
}
func promptWithFile(name string) map[string]any {
return map[string]any{"prompts": map[string]any{"example": map[string]any{"v1": map[string]any{
"messages": []any{map[string]any{"role": "user", "content_file": name}},
"output": map[string]any{"format": "text", "validation_mode": "basic"},
}}}}
}
type countingFS struct {
fs.FS
contentOpened int
contentClosed int
}
func (filesystem *countingFS) Open(name string) (fs.File, error) {
file, err := filesystem.FS.Open(name)
if err != nil {
return nil, err
}
if name == "assets/body.md" {
filesystem.contentOpened++
return &countingFile{File: file, owner: filesystem}, nil
}
return file, nil
}
type countingFile struct {
fs.File
owner *countingFS
}
func (file *countingFile) Close() error {
file.owner.contentClosed++
return file.File.Close()
}

View File

@@ -0,0 +1,187 @@
package configuration
import (
"context"
"fmt"
"gitea.maximumdirect.net/eric/promptkit/internal/backend"
"gitea.maximumdirect.net/eric/promptkit/internal/defaults"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/jsonvalue"
profilepkg "gitea.maximumdirect.net/eric/promptkit/internal/profile"
)
func (state *loaded) buildExecutionProfiles() error {
definitions := make(map[string]domain.Backend, len(state.result.Config.Backends))
for _, id := range sortedKeys(state.result.Config.Backends) {
config := state.result.Config.Backends[id]
params, err := jsonvalue.CopyMap(config.ExtraParams)
if err != nil {
return fmt.Errorf("backend %q extra parameters: %w", id, err)
}
definition := domain.Backend{ID: id, ExtraParams: params}
if config.Endpoint != nil {
definition.Endpoint = *config.Endpoint
}
if config.APIKeyEnv != nil {
definition.APIKeyEnv = *config.APIKeyEnv
}
if config.ConcurrencyLimit != nil {
definition.ConcurrencyLimit = *config.ConcurrencyLimit
}
if config.QueueCapacity != nil {
definition.QueueCapacity = *config.QueueCapacity
definition.QueueCapacitySet = definition.ConcurrencyLimit > 0
}
definitions[id] = definition
}
registry, err := backend.NewEffectiveRegistry(definitions)
if err != nil {
return err
}
state.registry = registry
state.indexedProfiles = make(map[string]domain.ExecutionProfile, len(state.result.Config.Profiles))
for _, id := range sortedKeys(state.result.Config.Profiles) {
config := state.result.Config.Profiles[id]
fields := state.profiles[id]
target := defaults.ExecutionTargetDefault()
if config.Backend != nil && *config.Backend != "" {
selected, err := registry.GetBackend(*config.Backend)
if err != nil {
return fmt.Errorf("profile %q backend: %w", id, err)
}
target.BackendID = selected.ID
target.Endpoint = selected.Endpoint
target.APIKeyEnv = selected.APIKeyEnv
target.ExtraParams = selected.ExtraParams
}
presence := domain.ExecutionTargetPresence{}
if fields["endpoint"] != fieldAbsent {
target.Endpoint = valueOrZero(config.Endpoint)
}
if fields["model"] != fieldAbsent {
target.Model = valueOrZero(config.Model)
}
if fields["temperature"] != fieldAbsent {
target.Temperature = valueOrZero(config.Temperature)
presence.Temperature = fields["temperature"] == fieldValue
}
if fields["top_p"] != fieldAbsent {
target.TopP = valueOrZero(config.TopP)
presence.TopP = fields["top_p"] == fieldValue
}
if fields["max_tokens"] != fieldAbsent {
target.MaxTokens = valueOrZero(config.MaxTokens)
presence.MaxTokens = fields["max_tokens"] == fieldValue
}
if fields["timeout_seconds"] == fieldValue {
target.TimeoutSeconds = valueOrZero(config.TimeoutSeconds)
presence.TimeoutSeconds = true
}
if fields["service_tier"] != fieldAbsent {
target.ServiceTier = valueOrZero(config.ServiceTier)
}
if fields["reasoning_effort"] != fieldAbsent {
target.ReasoningEffort = valueOrZero(config.ReasoningEffort)
}
if fields["api_key_env"] != fieldAbsent {
target.APIKeyEnv = valueOrZero(config.APIKeyEnv)
}
if fields["api_key_required"] != fieldAbsent {
target.APIKeyRequired = valueOrZero(config.APIKeyRequired)
}
if target.APIKeyRequired && fields["api_key_env"] == fieldAbsent {
target.APIKeyEnv = ""
}
switch fields["extra_params"] {
case fieldNull:
target.ExtraParams = nil
case fieldValue:
merged, err := overlayProviderMap(target.ExtraParams, config.ExtraParams)
if err != nil {
return fmt.Errorf("profile %q extra parameters: %w", id, err)
}
target.ExtraParams = merged
}
profile := domain.ExecutionProfile{ID: id, ResolvedTarget: &target, TargetPresence: presence}
state.indexedProfiles[id] = profile
}
return nil
}
func valueOrZero[T any](value *T) T {
if value == nil {
var zero T
return zero
}
return *value
}
// overlayProviderMap applies only the backend-to-profile JSON-object default
// merge. ConfigKit has already resolved source layering and inheritance.
func overlayProviderMap(lower, upper map[string]any) (map[string]any, error) {
merged, err := jsonvalue.CopyMap(lower)
if err != nil {
return nil, err
}
if merged == nil {
merged = make(map[string]any)
}
for key, value := range upper {
if child, ok := value.(map[string]any); ok {
if existing, ok := merged[key].(map[string]any); ok {
combined, err := overlayProviderMap(existing, child)
if err != nil {
return nil, err
}
merged[key] = combined
continue
}
}
copied, err := jsonvalue.Copy(value)
if err != nil {
return nil, err
}
merged[key] = copied
}
return merged, nil
}
// GetProfile returns an independently owned indexed profile for the internal
// runner's lookup boundary.
func (state *loaded) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
if err := ctx.Err(); err != nil {
return nil, err
}
if state == nil {
return nil, fmt.Errorf("%w: %q", profilepkg.ErrProfileNotFound, id)
}
profile, found := state.indexedProfiles[id]
if !found {
return nil, fmt.Errorf("%w: %q", profilepkg.ErrProfileNotFound, id)
}
target := *profile.ResolvedTarget
params, err := jsonvalue.CopyMap(target.ExtraParams)
if err != nil {
return nil, err
}
target.ExtraParams = params
profile.ResolvedTarget = &target
return &profile, nil
}
// GetBackend returns an independently owned effective backend definition.
func (state *loaded) GetBackend(id string) (domain.Backend, error) {
if state == nil {
return domain.Backend{}, fmt.Errorf("backend registry is not configured")
}
return state.registry.GetBackend(id)
}
// CapacityPolicies returns the selected backend admission policies.
func (state *loaded) CapacityPolicies() map[string]domain.BackendCapacityPolicy {
if state == nil {
return nil
}
return state.registry.CapacityPolicies()
}

View File

@@ -0,0 +1,152 @@
package configuration
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"reflect"
"testing"
"gitea.maximumdirect.net/eric/promptkit/internal/defaults"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/llm"
)
func TestLoadBuildsEffectiveTargetsAfterInheritance(t *testing.T) {
state, err := load(mapping("effective-targets", map[string]any{
"backends": map[string]any{
"first": map[string]any{"endpoint": "https://first.example/v1", "api_key_env": "FIRST_KEY",
"extra_params": map[string]any{"source": "first", "nested": map[string]any{"old": true}}},
"second": map[string]any{"endpoint": "https://second.example/v1", "api_key_env": "SECOND_KEY",
"concurrency_limit": 2, "queue_capacity": 0,
"extra_params": map[string]any{"source": "second", "nested": map[string]any{"backend": true}}},
},
"profiles": map[string]any{
"base": map[string]any{"backend": "first", "model": "model", "temperature": 0.8,
"top_p": 0.9, "max_tokens": 100, "timeout_seconds": 30, "api_key_required": true,
"extra_params": map[string]any{"nested": map[string]any{"profile": true}}},
"child": map[string]any{"extends": "base", "backend": "second", "temperature": 0.0,
"top_p": nil, "max_tokens": 0, "timeout_seconds": nil, "api_key_env": nil,
"api_key_required": false,
"extra_params": map[string]any{"nested": map[string]any{"child": nil}}},
"direct-key": map[string]any{"backend": "second", "api_key_required": true},
"named-key": map[string]any{"backend": "second", "api_key_required": true, "api_key_env": "CUSTOM_KEY"},
"no-provider-map": map[string]any{"backend": "second", "extra_params": nil},
"no-timeout": map[string]any{"backend": "second", "timeout_seconds": 0},
"partial": map[string]any{},
"endpoint-only": map[string]any{"endpoint": "https://standalone.example/v1"},
"cleared-endpoint": map[string]any{"extends": "base", "endpoint": nil},
},
}), nil)
if err != nil {
t.Fatal(err)
}
selected, err := state.registry.GetBackend("second")
if err != nil || selected.QueueCapacity != 0 || !selected.QueueCapacitySet {
t.Fatalf("backend capacity was not preserved: %+v, %v", selected, err)
}
child, err := state.GetProfile(context.Background(), "child")
if err != nil {
t.Fatal(err)
}
if _, err := state.GetProfile(context.Background(), " child "); err == nil {
t.Fatal("profile lookup silently changed a literal ID")
}
target := child.ResolvedTarget
if target.BackendID != "second" || target.Endpoint != "https://second.example/v1" || target.Model != "model" ||
target.Temperature != 0 || target.TopP != 0 || target.MaxTokens != 0 ||
target.TimeoutSeconds != defaults.ExecutionDefaultTimeoutSeconds || target.APIKeyEnv != "" || target.APIKeyRequired {
t.Fatalf("effective child target is wrong: %+v", target)
}
if !child.TargetPresence.Temperature || !child.TargetPresence.MaxTokens || child.TargetPresence.TopP || child.TargetPresence.TimeoutSeconds {
t.Fatalf("numeric presence was not retained: %+v", child.TargetPresence)
}
wantParams := map[string]any{"source": "second", "nested": map[string]any{
"backend": true, "profile": true, "child": nil,
}}
if !reflect.DeepEqual(target.ExtraParams, wantParams) {
t.Fatalf("backend/profile provider map overlay failed: %+v", target.ExtraParams)
}
if _, exists := target.ExtraParams["nested"].(map[string]any)["old"]; exists {
t.Fatal("previous backend derived parameter leaked into child")
}
for id, wantEnv := range map[string]string{"direct-key": "", "named-key": "CUSTOM_KEY"} {
profile, err := state.GetProfile(context.Background(), id)
if err != nil || profile.ResolvedTarget.APIKeyEnv != wantEnv {
t.Fatalf("credential source for %s: %+v, %v", id, profile, err)
}
}
noMap, _ := state.GetProfile(context.Background(), "no-provider-map")
if noMap.ResolvedTarget.ExtraParams != nil {
t.Fatalf("explicit null provider map did not suppress backend defaults: %+v", noMap.ResolvedTarget.ExtraParams)
}
noTimeout, _ := state.GetProfile(context.Background(), "no-timeout")
if noTimeout.ResolvedTarget.TimeoutSeconds != 0 || !noTimeout.TargetPresence.TimeoutSeconds {
t.Fatalf("explicit zero timeout was lost: %+v", noTimeout)
}
partial, _ := state.GetProfile(context.Background(), "partial")
if partial.ResolvedTarget.Endpoint != "" || partial.ResolvedTarget.Model != "" {
t.Fatalf("partial profile gained a runnable target: %+v", partial.ResolvedTarget)
}
endpointOnly, _ := state.GetProfile(context.Background(), "endpoint-only")
if endpointOnly.ResolvedTarget.Endpoint != "https://standalone.example/v1" ||
endpointOnly.ResolvedTarget.BackendID != "" || endpointOnly.ResolvedTarget.Model != "" {
t.Fatalf("endpoint-only profile was not retained: %+v", endpointOnly.ResolvedTarget)
}
clearedEndpoint, _ := state.GetProfile(context.Background(), "cleared-endpoint")
if clearedEndpoint.ResolvedTarget.Endpoint != "" || clearedEndpoint.ResolvedTarget.BackendID != "first" {
t.Fatalf("explicit null recovered backend endpoint: %+v", clearedEndpoint.ResolvedTarget)
}
child.ResolvedTarget.ExtraParams["source"] = "mutated"
again, _ := state.GetProfile(context.Background(), "child")
if again.ResolvedTarget.ExtraParams["source"] != "second" {
t.Fatal("profile lookup exposed snapshot ownership")
}
}
func TestLoadedZeroControlsReachBuiltInRequest(t *testing.T) {
payloads := make(chan map[string]any, 1)
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
var payload map[string]any
if err := json.NewDecoder(request.Body).Decode(&payload); err != nil {
writer.WriteHeader(http.StatusBadRequest)
return
}
payloads <- payload
writer.Header().Set("Content-Type", "application/json")
_, _ = writer.Write([]byte(`{"choices":[{"message":{"content":"ok"}}]}`))
}))
defer server.Close()
state, err := load(mapping("wire-controls", map[string]any{
"backends": map[string]any{"local": map[string]any{"endpoint": server.URL + "/v1"}},
"profiles": map[string]any{"direct": map[string]any{
"backend": "local", "model": "example", "temperature": 0.0, "max_tokens": 0, "top_p": nil,
}},
}), nil)
if err != nil {
t.Fatal(err)
}
profile, err := state.GetProfile(context.Background(), "direct")
if err != nil {
t.Fatal(err)
}
client, err := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{})
if err != nil {
t.Fatal(err)
}
_, err = client.Generate(context.Background(), domain.GenerateRequest{
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hello"}}},
Target: *profile.ResolvedTarget, TargetPresence: profile.TargetPresence,
})
if err != nil {
t.Fatal(err)
}
payload := <-payloads
if payload["temperature"] != float64(0) || payload["max_tokens"] != float64(0) {
t.Fatalf("explicit zeros were omitted from outbound request: %+v", payload)
}
if _, exists := payload["top_p"]; exists {
t.Fatalf("explicit null top_p was sent: %+v", payload)
}
}

View File

@@ -192,22 +192,11 @@ type BackendCapacityPolicy struct {
QueueCapacity int
}
// ExecutionProfile describes how and where to execute a model.
// ExecutionProfile holds a captured configuration target for execution.
type ExecutionProfile struct {
ID string `yaml:"id"`
BaseProfileID string `yaml:"base_profile"`
BackendID string `yaml:"backend"`
Endpoint string `yaml:"endpoint"`
Model string `yaml:"model"`
Temperature float64 `yaml:"temperature"`
MaxTokens int `yaml:"max_tokens"`
TopP float64 `yaml:"top_p"`
TimeoutSeconds int `yaml:"timeout_seconds"`
ServiceTier string `yaml:"service_tier"`
ReasoningEffort string `yaml:"reasoning_effort"`
APIKeyEnv string `yaml:"api_key_env"`
APIKeyRequired bool `yaml:"-" json:"-"`
ExtraParams map[string]any `yaml:"extra_params"`
ID string
ResolvedTarget *ExecutionTarget
TargetPresence ExecutionTargetPresence
}
// ExecutionTargetOverride represents per-request runtime setting overrides.
@@ -224,8 +213,9 @@ type ExecutionTargetOverride struct {
ExtraParams map[string]any `json:"extra_params,omitempty"`
}
// ExecutionTargetPresence tracks which effective runtime fields came from an
// explicit request override even when the resolved value is a zero value.
// ExecutionTargetPresence tracks effective numeric values that must be sent
// even when they are zero. Values may come from configuration or a request
// override; explicit configuration null suppresses the corresponding bit.
type ExecutionTargetPresence struct {
Temperature bool
MaxTokens bool

View File

@@ -11,49 +11,42 @@ import (
"strings"
"time"
openrouter "gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v2"
rakestrawhome "gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v2"
"gitea.maximumdirect.net/eric/ConfigKit"
"gitea.maximumdirect.net/eric/promptkit/internal/artifact"
"gitea.maximumdirect.net/eric/promptkit/internal/backend"
"gitea.maximumdirect.net/eric/promptkit/internal/capacity"
"gitea.maximumdirect.net/eric/promptkit/internal/catalog"
"gitea.maximumdirect.net/eric/promptkit/internal/configuration"
"gitea.maximumdirect.net/eric/promptkit/internal/defaults"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/generation"
"gitea.maximumdirect.net/eric/promptkit/internal/llm"
"gitea.maximumdirect.net/eric/promptkit/internal/profile"
"gitea.maximumdirect.net/eric/promptkit/internal/prompt"
"gitea.maximumdirect.net/eric/promptkit/internal/promptdef"
"gitea.maximumdirect.net/eric/promptkit/internal/usecase"
"gitea.maximumdirect.net/eric/promptkit/internal/validate"
)
// Config carries construction settings selected by the public facade.
type Config struct {
PromptDir string
ProfileDir string
Sources ConfigurationSources
SchemaDir string
Timeout time.Duration
HTTPClient *http.Client
TransientRetries *int
}
// ConfigurationSources identifies caller-selected ConfigKit layers.
type ConfigurationSources struct {
Underlays []configkit.Underlay
Main *configkit.Input
}
// Options collects construction choices from the public facade.
type Options struct {
llmClient llm.Client
artifactReader artifact.Reader
promptDefinitions promptdef.Repository
profiles profile.Repository
fallbackProfiles profile.Repository
memoryProfiles profile.Repository
backends []domain.Backend
validator validate.Validator
promptSource bool
profileSource bool
fallbackProfileSource bool
memorySource bool
validatorSource bool
artifactSource bool
llmClient llm.Client
artifactReader artifact.Reader
validator validate.Validator
validatorSource bool
artifactSource bool
}
// SetLLMClient selects the client collaborator for one engine.
@@ -65,72 +58,6 @@ func (o *Options) SetArtifactReader(reader artifact.Reader) {
o.artifactSource = true
}
// SetPromptFS selects a prompt source rooted in fsys.
func (o *Options) SetPromptFS(fsys fs.FS, root string) error {
if err := validateSourceFS(fsys, root); err != nil {
return err
}
o.promptDefinitions = promptdef.NewFSRepository(fsys, root)
o.promptSource = true
return nil
}
// SetPromptFile selects a single prompt file.
func (o *Options) SetPromptFile(path string) error {
fsys, root, directory, err := fileSource(path)
if err != nil {
return err
}
o.promptDefinitions = promptdef.NewFileRepository(fsys, root, directory)
o.promptSource = true
return nil
}
// SetProfileFS selects an ordinary profile source rooted in fsys.
func (o *Options) SetProfileFS(fsys fs.FS, root string) error {
if err := validateSourceFS(fsys, root); err != nil {
return err
}
o.profiles = profile.NewFSRepository(fsys, root)
o.profileSource = true
return nil
}
// SetProfileFile selects a single ordinary profile file.
func (o *Options) SetProfileFile(path string) error {
fsys, root, _, err := fileSource(path)
if err != nil {
return err
}
o.profiles = profile.NewFSRepository(fsys, root)
o.profileSource = true
return nil
}
// SetFallbackProfileFS selects a fallback profile source rooted in fsys.
func (o *Options) SetFallbackProfileFS(fsys fs.FS, root string) error {
if err := validateSourceFS(fsys, root); err != nil {
return err
}
o.fallbackProfiles = profile.NewFSRepository(fsys, root)
o.fallbackProfileSource = true
return nil
}
// SetMemoryProfiles validates and stores application-owned profile definitions.
func (o *Options) SetMemoryProfiles(definitions []domain.ExecutionProfile) error {
repository, err := profile.NewMemoryRepository(definitions)
if err != nil {
return err
}
o.memoryProfiles = repository
o.memorySource = true
return nil
}
// AddBackend appends one backend registration.
func (o *Options) AddBackend(backend domain.Backend) { o.backends = append(o.backends, backend) }
// SetSchemaFS selects a schema source rooted in fsys.
func (o *Options) SetSchemaFS(fsys fs.FS, root string) error {
if err := validateSourceFS(fsys, root); err != nil {
@@ -154,7 +81,8 @@ func (o *Options) SetSchemaFile(path string) error {
// Engine forwards domain operations to the single assembled use-case runner.
type Engine struct {
runner *usecase.Runner
runner *usecase.Runner
inspection configuration.Inspection
}
// New assembles an engine from facade-selected configuration and collaborators.
@@ -163,33 +91,20 @@ func New(cfg Config, options Options) (*Engine, error) {
if err != nil {
return nil, err
}
promptDefinitions := options.promptDefinitions
if !options.promptSource {
if strings.TrimSpace(cfg.PromptDir) == "" {
return nil, fmt.Errorf("prompt directory is required")
}
promptDefinitions = promptdef.NewFilesystemRepository(cfg.PromptDir)
}
maintained, err := catalog.Load(
catalog.Source{Name: "OpenRouter", ExpectedBackendID: backend.OpenRouterID, FS: openrouter.FS(), Root: openrouter.Root},
catalog.Source{Name: "Rakestrawhome", ExpectedBackendID: backend.RakestrawHomeID, FS: rakestrawhome.FS(), Root: rakestrawhome.Root},
)
underlays, err := catalog.Underlays()
if err != nil {
return nil, fmt.Errorf("failed to load maintained catalogs: %w", err)
return nil, fmt.Errorf("select maintained configuration: %w", err)
}
maintainedProfiles, err := newMaintainedProfileRepository(maintained.Profiles, maintained.Backends)
underlays = append(underlays, cfg.Sources.Underlays...)
main := configkit.Input{Source: configkit.NoSource()}
if cfg.Sources.Main != nil {
main = *cfg.Sources.Main
}
snapshot, err := configuration.Load(main, underlays)
if err != nil {
return nil, fmt.Errorf("failed to construct Promptkit-maintained profiles: %w", err)
return nil, err
}
profiles := newProfileRepository(cfg.ProfileDir, options, maintainedProfiles)
registry, err := backend.NewRegistry(maintained.Backends, options.backends)
if err != nil {
return nil, fmt.Errorf("failed to construct backend registry: %w", err)
}
manager, err := capacity.NewManager(registry.CapacityPolicies())
manager, err := capacity.NewManager(snapshot.CapacityPolicies())
if err != nil {
return nil, fmt.Errorf("failed to construct backend capacity manager: %w", err)
}
@@ -217,23 +132,18 @@ func New(cfg Config, options Options) (*Engine, error) {
reader = artifact.NewCompositeReader()
}
return &Engine{runner: usecase.NewRunnerWithRepairer(promptDefinitions, profiles, registry, reader, prompt.NewGoRenderer(), client, validator, usecase.NewDefaultOutputRepairer(client), manager, defaultRetries)}, nil
return &Engine{
runner: usecase.NewRunnerWithRepairer(snapshot, snapshot, reader, prompt.NewCompiledRenderer(snapshot), client, validator, usecase.NewDefaultOutputRepairer(client), manager, defaultRetries),
inspection: snapshot.Inspection(),
}, nil
}
func newProfileRepository(profileDir string, options Options, maintained profile.Repository) profile.Repository {
repository := maintained
if options.fallbackProfileSource {
repository = profile.NewOverlayRepository(options.fallbackProfiles, repository)
// InspectConfiguration returns read-only operations over the captured result.
func (e *Engine) InspectConfiguration() configuration.Inspection {
if e == nil {
return nil
}
if options.profileSource {
repository = profile.NewOverlayRepository(options.profiles, repository)
} else if strings.TrimSpace(profileDir) != "" {
repository = profile.NewOverlayRepository(profile.NewFilesystemRepository(profileDir), repository)
}
if options.memorySource {
repository = profile.NewOverlayRepository(options.memoryProfiles, repository)
}
return profile.NewResolvingRepository(repository)
return e.inspection
}
func validateSourceFS(fsys fs.FS, root string) error {

View File

@@ -1,50 +0,0 @@
package engine
import (
"context"
"fmt"
"gitea.maximumdirect.net/eric/promptkit/internal/backend"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/profile"
)
func maintainedProfileDefinitions() []domain.ExecutionProfile {
return []domain.ExecutionProfile{
{
ID: "promptkit-openrouter-standard",
BaseProfileID: "gemini-flash-latest",
},
{
ID: "promptkit-openrouter-auto",
BackendID: backend.OpenRouterID,
Model: "openrouter/auto",
},
}
}
func newMaintainedProfileRepository(catalogProfiles profile.Repository, maintainedBackends []domain.Backend) (profile.Repository, error) {
definitions := maintainedProfileDefinitions()
owned, err := profile.NewMemoryRepository(definitions)
if err != nil {
return nil, fmt.Errorf("create Promptkit-maintained profiles: %w", err)
}
combined := profile.NewOverlayRepository(owned, catalogProfiles)
resolved := profile.NewResolvingRepository(combined)
registry, err := backend.NewRegistry(maintainedBackends, nil)
if err != nil {
return nil, fmt.Errorf("construct maintained backend registry: %w", err)
}
for _, definition := range definitions {
effective, err := resolved.GetProfile(context.Background(), definition.ID)
if err != nil {
return nil, fmt.Errorf("resolve Promptkit-maintained profile %q: %w", definition.ID, err)
}
if _, err := registry.GetBackend(effective.BackendID); err != nil {
return nil, fmt.Errorf("Promptkit-maintained profile %q selects unavailable backend %q: %w", definition.ID, effective.BackendID, err)
}
}
return combined, nil
}

View File

@@ -1,78 +0,0 @@
package engine
import (
"context"
"testing"
"gitea.maximumdirect.net/eric/promptkit/internal/backend"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/profile"
)
func TestNewMaintainedProfileRepository(t *testing.T) {
newCatalogProfiles := func(t *testing.T, definitions []domain.ExecutionProfile) profile.Repository {
t.Helper()
repository, err := profile.NewMemoryRepository(definitions)
if err != nil {
t.Fatalf("create catalog profiles: %v", err)
}
return repository
}
newBackend := func(id string) domain.Backend {
return domain.Backend{ID: id, Endpoint: "https://" + id + ".example/v1", APIKeyEnv: "TEST_API_KEY"}
}
t.Run("resolves inherited and direct definitions", func(t *testing.T) {
repository, err := newMaintainedProfileRepository(
newCatalogProfiles(t, []domain.ExecutionProfile{{
ID: "gemini-flash-latest",
BackendID: backend.OpenRouterID,
Model: "catalog-selected-model",
}}),
[]domain.Backend{newBackend(backend.OpenRouterID)},
)
if err != nil {
t.Fatalf("create maintained profiles: %v", err)
}
resolved := profile.NewResolvingRepository(repository)
standard, err := resolved.GetProfile(context.Background(), "promptkit-openrouter-standard")
if err != nil {
t.Fatalf("resolve inherited profile: %v", err)
}
if standard.BackendID != backend.OpenRouterID || standard.Model != "catalog-selected-model" {
t.Fatalf("unexpected inherited profile: %#v", standard)
}
auto, err := resolved.GetProfile(context.Background(), "promptkit-openrouter-auto")
if err != nil {
t.Fatalf("resolve direct profile: %v", err)
}
definitions := maintainedProfileDefinitions()
if auto.BackendID != definitions[1].BackendID || auto.Model != definitions[1].Model {
t.Fatalf("unexpected direct profile: %#v", auto)
}
})
t.Run("rejects a missing inherited base", func(t *testing.T) {
if _, err := newMaintainedProfileRepository(
newCatalogProfiles(t, nil),
[]domain.Backend{newBackend(backend.OpenRouterID)},
); err == nil {
t.Fatal("expected missing base error")
}
})
t.Run("rejects an unavailable effective backend", func(t *testing.T) {
if _, err := newMaintainedProfileRepository(
newCatalogProfiles(t, []domain.ExecutionProfile{{
ID: "gemini-flash-latest",
BackendID: "other",
Model: "catalog-selected-model",
}}),
[]domain.Backend{newBackend("other")},
); err == nil {
t.Fatal("expected unavailable backend error")
}
})
}

View File

@@ -1,47 +0,0 @@
package profile
import (
"errors"
"strings"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
)
// NormalizeAndValidateDefinition normalizes and validates one source-local
// profile definition without resolving a base profile.
func NormalizeAndValidateDefinition(profile *domain.ExecutionProfile) error {
if profile == nil {
return errors.New("profile is required")
}
profile.ID = strings.TrimSpace(profile.ID)
profile.BaseProfileID = strings.TrimSpace(profile.BaseProfileID)
profile.BackendID = strings.TrimSpace(profile.BackendID)
profile.Endpoint = strings.TrimSpace(profile.Endpoint)
if profile.ID == "" {
return errors.New("id is required")
}
if profile.Endpoint != "" {
endpoint, err := domain.NormalizeOpenAICompatibleBaseEndpoint(profile.Endpoint)
if err != nil {
return err
}
profile.Endpoint = endpoint
}
if profile.BaseProfileID == "" {
if profile.BackendID == "" && profile.Endpoint == "" {
return errors.New("backend or endpoint is required")
}
if strings.TrimSpace(profile.Model) == "" {
return errors.New("model is required")
}
}
return domain.ValidateExecutionTargetSettings(domain.ExecutionTarget{
Temperature: profile.Temperature,
MaxTokens: profile.MaxTokens,
TopP: profile.TopP,
TimeoutSeconds: profile.TimeoutSeconds,
})
}

View File

@@ -1,98 +0,0 @@
package profile
import (
"context"
"fmt"
"io/fs"
"sort"
"strings"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/filecatalog"
"gitea.maximumdirect.net/eric/promptkit/internal/jsonvalue"
)
// LoadedProfileMetadata identifies one profile accepted by LoadFSRepository.
type LoadedProfileMetadata struct {
// ID is the normalized profile ID.
ID string
// Path is the safe root-relative source path.
Path string
// ExplicitFields lists the sorted top-level YAML fields present in source.
ExplicitFields []string
}
// LoadFSRepository eagerly validates every profile under root and returns an
// immutable raw repository and independently owned source metadata.
func LoadFSRepository(ctx context.Context, fsys fs.FS, root string) (Repository, []LoadedProfileMetadata, error) {
if fsys == nil {
return nil, nil, fmt.Errorf("failed to read profile directory: filesystem is nil")
}
paths, err := filecatalog.FindFSYAMLFiles(ctx, fsys, root)
if err != nil {
return nil, nil, fmt.Errorf("failed to read profile directory: %w", err)
}
repository := &loadedRepository{profiles: make(map[string]domain.ExecutionProfile, len(paths))}
metadata := make([]LoadedProfileMetadata, 0, len(paths))
for _, path := range paths {
if err := ctx.Err(); err != nil {
return nil, nil, err
}
data, err := fs.ReadFile(fsys, path)
if err != nil {
return nil, nil, fmt.Errorf("failed to read profile file %s: %w", filecatalog.DisplayPath(root, path), err)
}
fileMetadata, err := readProfileFileMetadata(data)
if err != nil {
return nil, nil, fmt.Errorf("%w: %s", ErrInvalidYAML, filecatalog.DisplayPath(root, path))
}
if fileMetadata.hasRawAPIKey {
return nil, nil, fmt.Errorf("%w: %s", ErrRawAPIKeyNotAllowed, filecatalog.DisplayPath(root, path))
}
definition, err := decodeProfile(data)
if err != nil {
return nil, nil, fmt.Errorf("%w: %s", ErrInvalidYAML, filecatalog.DisplayPath(root, path))
}
definition.ExtraParams, err = jsonvalue.CopyMap(definition.ExtraParams)
if err != nil {
return nil, nil, fmt.Errorf("%w: %s", ErrInvalidProfile, filecatalog.DisplayPath(root, path))
}
if err := NormalizeAndValidateDefinition(definition); err != nil {
return nil, nil, fmt.Errorf("%w: %s", ErrInvalidProfile, filecatalog.DisplayPath(root, path))
}
if _, exists := repository.profiles[definition.ID]; exists {
return nil, nil, fmt.Errorf("%w: %s: duplicate profile ID", ErrInvalidProfile, filecatalog.DisplayPath(root, path))
}
repository.profiles[definition.ID] = *definition
fields := append([]string(nil), fileMetadata.explicitFields...)
sort.Strings(fields)
metadata = append(metadata, LoadedProfileMetadata{
ID: definition.ID,
Path: filecatalog.DisplayPath(root, path),
ExplicitFields: fields,
})
}
sort.Slice(metadata, func(left, right int) bool { return metadata[left].ID < metadata[right].ID })
return repository, metadata, nil
}
type loadedRepository struct {
profiles map[string]domain.ExecutionProfile
}
func (r *loadedRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
if err := ctx.Err(); err != nil {
return nil, err
}
definition, found := r.profiles[strings.TrimSpace(id)]
if !found {
return nil, ErrProfileNotFound
}
extraParams, err := jsonvalue.CopyMap(definition.ExtraParams)
if err != nil {
return nil, fmt.Errorf("copy loaded profile %q: %w", definition.ID, err)
}
definition.ExtraParams = extraParams
return &definition, nil
}

View File

@@ -1,110 +0,0 @@
package profile
import (
"context"
"errors"
"io/fs"
"reflect"
"testing"
"testing/fstest"
)
func TestLoadFSRepositoryLoadsSortedIndependentProfiles(t *testing.T) {
fsys := fstest.MapFS{
"profiles/z.yml": {Data: []byte("id: z\nbackend: local\nmodel: z-model\nextra_params:\n nested:\n value: one\n")},
"profiles/a.yml": {Data: []byte("id: a\nbackend: local\nmodel: a-model\nendpoint: ''\n")},
}
repository, metadata, err := LoadFSRepository(context.Background(), fsys, "profiles")
if err != nil {
t.Fatalf("load repository: %v", err)
}
if len(metadata) != 2 || metadata[0].ID != "a" || metadata[1].ID != "z" || metadata[0].Path != "a.yml" {
t.Fatalf("unexpected metadata: %#v", metadata)
}
if len(metadata[0].ExplicitFields) != 4 || metadata[0].ExplicitFields[0] != "backend" || metadata[0].ExplicitFields[1] != "endpoint" {
t.Fatalf("expected explicitly empty endpoint metadata, got %#v", metadata[0].ExplicitFields)
}
metadata[1].ExplicitFields[0] = "changed"
first, err := repository.GetProfile(context.Background(), "z")
if err != nil {
t.Fatalf("load profile: %v", err)
}
first.ExtraParams["nested"].(map[string]any)["value"] = "changed"
second, err := repository.GetProfile(context.Background(), "z")
if err != nil {
t.Fatalf("reload profile: %v", err)
}
if second.ExtraParams["nested"].(map[string]any)["value"] != "one" {
t.Fatalf("profile value was not defensively copied: %#v", second)
}
}
func TestLoadFSRepositoryRejectsInvalidProfiles(t *testing.T) {
tests := map[string]fstest.MapFS{
"duplicate IDs": {
"profiles/one.yml": {Data: []byte("id: duplicate\nbackend: local\nmodel: one\n")},
"profiles/two.yml": {Data: []byte("id: duplicate\nbackend: local\nmodel: two\n")},
},
"raw API key": {
"profiles/one.yml": {Data: []byte("id: one\nbackend: local\nmodel: one\napi_key: forbidden\n")},
},
"multiple documents": {
"profiles/one.yml": {Data: []byte("id: one\nbackend: local\nmodel: one\n---\nid: two\n")},
},
}
for name, fsys := range tests {
t.Run(name, func(t *testing.T) {
_, _, err := LoadFSRepository(context.Background(), fsys, "profiles")
if err == nil {
t.Fatal("expected load failure")
}
})
}
}
func TestLoadedRepositoryHonorsCancellationAndMissingProfiles(t *testing.T) {
repository, _, err := LoadFSRepository(context.Background(), fstest.MapFS{
"profiles/one.yml": {Data: []byte("id: one\nbackend: local\nmodel: one\n")},
}, "profiles")
if err != nil {
t.Fatalf("load repository: %v", err)
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
if _, err := repository.GetProfile(ctx, "one"); !errors.Is(err, context.Canceled) {
t.Fatalf("expected cancellation, got %v", err)
}
if _, err := repository.GetProfile(context.Background(), "missing"); !errors.Is(err, ErrProfileNotFound) {
t.Fatalf("expected missing profile, got %v", err)
}
}
func TestLoadFSRepositoryMatchesPointLookupForRawProfiles(t *testing.T) {
fsys := fstest.MapFS{
"profiles/base.yml": {Data: []byte("id: base\nbackend: local\nmodel: base-model\n")},
"profiles/derived.yml": {Data: []byte("id: derived\nbase_profile: base\nreasoning_effort: high\n")},
}
eager, _, err := LoadFSRepository(context.Background(), fsys, "profiles")
if err != nil {
t.Fatalf("load eager repository: %v", err)
}
pointLookup := NewFSRepository(fsys, "profiles")
for _, id := range []string{"base", "derived"} {
t.Run(id, func(t *testing.T) {
got, err := eager.GetProfile(context.Background(), id)
if err != nil {
t.Fatalf("load eager profile: %v", err)
}
want, err := pointLookup.GetProfile(context.Background(), id)
if err != nil {
t.Fatalf("load point-in-time profile: %v", err)
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("raw profiles differ: got %#v, want %#v", got, want)
}
})
}
}
var _ fs.FS = fstest.MapFS{}

View File

@@ -1,259 +0,0 @@
package profile
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"io/fs"
"os"
"strings"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/filecatalog"
"gitea.maximumdirect.net/eric/promptkit/internal/jsonvalue"
"gopkg.in/yaml.v3"
)
var (
ErrProfileNotFound = errors.New("execution profile not found")
ErrInvalidYAML = errors.New("invalid YAML format")
ErrInvalidProfile = errors.New("invalid execution profile configuration")
ErrRawAPIKeyNotAllowed = errors.New("raw api_key is not allowed; use api_key_env")
)
type filesystemRepository struct {
dir string
}
func NewFilesystemRepository(dir string) Repository {
return &filesystemRepository{dir: dir}
}
func (r *filesystemRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
return loadProfile(ctx, os.DirFS(r.dir), ".", id)
}
type fsRepository struct {
fsys fs.FS
root string
}
func NewFSRepository(fsys fs.FS, root string) Repository {
return &fsRepository{fsys: fsys, root: root}
}
func (r *fsRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
return loadProfile(ctx, r.fsys, r.root, id)
}
type overlayRepository struct {
primary Repository
fallback Repository
}
func NewOverlayRepository(primary, fallback Repository) Repository {
return &overlayRepository{primary: primary, fallback: fallback}
}
func (r *overlayRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
if r.primary != nil {
prof, err := r.primary.GetProfile(ctx, id)
if err == nil {
return prof, nil
}
if !errors.Is(err, ErrProfileNotFound) {
return nil, err
}
}
if r.fallback == nil {
return nil, ErrProfileNotFound
}
return r.fallback.GetProfile(ctx, id)
}
func loadProfile(ctx context.Context, fsys fs.FS, root string, id string) (*domain.ExecutionProfile, error) {
id = strings.TrimSpace(id)
if id == "" {
return nil, fmt.Errorf("%w: profile id is required", ErrInvalidProfile)
}
if fsys == nil {
return nil, fmt.Errorf("failed to read profile directory: filesystem is nil")
}
files, err := filecatalog.FindFSYAMLFiles(ctx, fsys, root)
if err != nil {
return nil, fmt.Errorf("failed to read profile directory: %w", err)
}
var matches []profileMatch
for _, fullPath := range files {
select {
case <-ctx.Done():
return nil, ctx.Err()
default:
}
relPath := filecatalog.DisplayPath(root, fullPath)
data, err := fs.ReadFile(fsys, fullPath)
if err != nil {
return nil, fmt.Errorf("failed to read profile file %s: %w", relPath, err)
}
metadata, metadataErr := readProfileFileMetadata(data)
idMatch := metadata.matchesID(id)
if metadataErr != nil {
if idMatch {
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidYAML, relPath, metadataErr)
}
continue
}
if metadata.hasRawAPIKey {
if idMatch {
return nil, fmt.Errorf("%w: %s", ErrRawAPIKeyNotAllowed, relPath)
}
continue
}
if !idMatch {
continue
}
prof, err := decodeProfile(data)
if err != nil {
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidYAML, relPath, err)
}
prof.ID = strings.TrimSpace(prof.ID)
if prof.ID != id {
continue
}
prof.ExtraParams, err = jsonvalue.CopyMap(prof.ExtraParams)
if err != nil {
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidProfile, relPath, err)
}
if err := NormalizeAndValidateDefinition(prof); err != nil {
if errors.Is(err, ErrRawAPIKeyNotAllowed) {
return nil, fmt.Errorf("%w: %s", err, relPath)
}
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidProfile, relPath, err)
}
matches = append(matches, profileMatch{
profile: prof,
path: relPath,
})
}
if len(matches) > 1 {
paths := make([]string, 0, len(matches))
for _, match := range matches {
paths = append(paths, match.path)
}
return nil, fmt.Errorf("%w: duplicate execution profile id %q found in: %s", ErrInvalidProfile, id, strings.Join(paths, ", "))
}
if len(matches) == 1 {
return matches[0].profile, nil
}
return nil, ErrProfileNotFound
}
type profileMatch struct {
profile *domain.ExecutionProfile
path string
}
type profileFileMetadata struct {
ids []string
hasRawAPIKey bool
explicitFields []string
}
func readProfileFileMetadata(data []byte) (profileFileMetadata, error) {
decoder := yaml.NewDecoder(bytes.NewReader(data))
var node yaml.Node
if err := decoder.Decode(&node); err != nil {
return profileFileMetadata{}, err
}
metadata := profileMetadataFromNode(&node)
documentCount := 1
for {
var trailing yaml.Node
err := decoder.Decode(&trailing)
if errors.Is(err, io.EOF) {
if documentCount == 1 {
return metadata, nil
}
return metadata, errors.New("profile file must contain exactly one YAML document")
}
if err != nil {
return metadata, err
}
documentCount++
metadata.merge(profileMetadataFromNode(&trailing))
}
}
func profileMetadataFromNode(node *yaml.Node) profileFileMetadata {
if node.Kind != yaml.DocumentNode || len(node.Content) == 0 {
return profileFileMetadata{}
}
mapping := node.Content[0]
if mapping.Kind != yaml.MappingNode {
return profileFileMetadata{}
}
var metadata profileFileMetadata
for i := 0; i+1 < len(mapping.Content); i += 2 {
key := mapping.Content[i]
value := mapping.Content[i+1]
metadata.explicitFields = append(metadata.explicitFields, key.Value)
switch key.Value {
case "id":
metadata.ids = append(metadata.ids, strings.TrimSpace(value.Value))
case "api_key":
metadata.hasRawAPIKey = true
}
}
return metadata
}
func (m profileFileMetadata) matchesID(id string) bool {
for _, candidate := range m.ids {
if candidate == id {
return true
}
}
return false
}
func (m *profileFileMetadata) merge(other profileFileMetadata) {
m.ids = append(m.ids, other.ids...)
m.hasRawAPIKey = m.hasRawAPIKey || other.hasRawAPIKey
m.explicitFields = append(m.explicitFields, other.explicitFields...)
}
func decodeProfile(data []byte) (*domain.ExecutionProfile, error) {
var prof domain.ExecutionProfile
decoder := yaml.NewDecoder(bytes.NewReader(data))
decoder.KnownFields(true)
if err := decoder.Decode(&prof); err != nil {
return nil, err
}
if err := requireYAMLStreamEnd(decoder); err != nil {
return nil, err
}
return &prof, nil
}
func requireYAMLStreamEnd(decoder *yaml.Decoder) error {
var trailing yaml.Node
err := decoder.Decode(&trailing)
if errors.Is(err, io.EOF) {
return nil
}
if err != nil {
return err
}
return errors.New("profile file must contain exactly one YAML document")
}

View File

@@ -1,50 +0,0 @@
package profile
import (
"context"
"fmt"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/jsonvalue"
)
// NewMemoryRepository constructs an immutable repository from profile
// definitions supplied by an internal caller.
func NewMemoryRepository(definitions []domain.ExecutionProfile) (Repository, error) {
repository := &memoryRepository{profiles: make(map[string]domain.ExecutionProfile, len(definitions))}
for _, definition := range definitions {
extraParams, err := jsonvalue.CopyMap(definition.ExtraParams)
if err != nil {
return nil, err
}
definition.ExtraParams = extraParams
if err := NormalizeAndValidateDefinition(&definition); err != nil {
return nil, err
}
if _, exists := repository.profiles[definition.ID]; exists {
return nil, fmt.Errorf("duplicate profile id %q", definition.ID)
}
repository.profiles[definition.ID] = definition
}
return repository, nil
}
type memoryRepository struct {
profiles map[string]domain.ExecutionProfile
}
func (r *memoryRepository) GetProfile(_ context.Context, id string) (*domain.ExecutionProfile, error) {
if r == nil {
return nil, ErrProfileNotFound
}
definition, found := r.profiles[id]
if !found {
return nil, ErrProfileNotFound
}
extraParams, err := jsonvalue.CopyMap(definition.ExtraParams)
if err != nil {
return nil, fmt.Errorf("copy memory profile %q: %w", definition.ID, err)
}
definition.ExtraParams = extraParams
return &definition, nil
}

View File

@@ -1,91 +0,0 @@
package profile
import (
"context"
"errors"
"testing"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
)
func TestMemoryRepositoryOwnsNormalizedDefinitions(t *testing.T) {
nested := map[string]any{"value": "original"}
definitions := []domain.ExecutionProfile{{
ID: " profile ",
BackendID: " backend ",
Endpoint: " https://example.test/v1/ ",
Model: "model",
ExtraParams: map[string]any{"nested": nested},
}}
repository, err := NewMemoryRepository(definitions)
if err != nil {
t.Fatalf("construct repository: %v", err)
}
nested["value"] = "mutated input"
first, err := repository.GetProfile(context.Background(), "profile")
if err != nil {
t.Fatalf("load profile: %v", err)
}
if first.ID != "profile" || first.BackendID != "backend" || first.Endpoint != "https://example.test/v1/" {
t.Fatalf("normalized definition = %#v", first)
}
if first.ExtraParams["nested"].(map[string]any)["value"] != "original" {
t.Fatalf("repository retained input mutation: %#v", first.ExtraParams)
}
first.ExtraParams["nested"].(map[string]any)["value"] = "mutated result"
second, err := repository.GetProfile(context.Background(), "profile")
if err != nil {
t.Fatalf("reload profile: %v", err)
}
if second.ExtraParams["nested"].(map[string]any)["value"] != "original" {
t.Fatalf("repository retained result mutation: %#v", second.ExtraParams)
}
}
func TestMemoryRepositoryPreservesInheritanceAndRejectsInvalidDefinitions(t *testing.T) {
derived, err := NewMemoryRepository([]domain.ExecutionProfile{{
ID: "derived",
BaseProfileID: "base",
ReasoningEffort: "high",
}})
if err != nil {
t.Fatalf("construct derived repository: %v", err)
}
profile, err := derived.GetProfile(context.Background(), "derived")
if err != nil {
t.Fatalf("load derived profile: %v", err)
}
if profile.BaseProfileID != "base" || profile.ReasoningEffort != "high" {
t.Fatalf("derived profile = %#v", profile)
}
if _, err := derived.GetProfile(context.Background(), "missing"); !errors.Is(err, ErrProfileNotFound) {
t.Fatalf("missing profile error = %v, want ErrProfileNotFound", err)
}
tests := []struct {
name string
definitions []domain.ExecutionProfile
}{
{
name: "normalized duplicate IDs",
definitions: []domain.ExecutionProfile{
{ID: " duplicate ", Endpoint: "https://one.example/v1", Model: "one"},
{ID: "duplicate", Endpoint: "https://two.example/v1", Model: "two"},
},
},
{
name: "invalid extra parameters",
definitions: []domain.ExecutionProfile{{ID: "invalid", Endpoint: "https://example.test/v1", Model: "model", ExtraParams: map[string]any{"invalid": func() {}}}},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if _, err := NewMemoryRepository(test.definitions); err == nil {
t.Fatal("expected repository construction error")
}
})
}
}

View File

@@ -2,11 +2,15 @@ package profile
import (
"context"
"errors"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
)
// Repository loads execution profiles.
// ErrProfileNotFound identifies an absent profile in the captured index.
var ErrProfileNotFound = errors.New("execution profile not found")
// Repository selects execution profiles from a captured index.
type Repository interface {
GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error)
}

View File

@@ -1,58 +0,0 @@
package profile
import (
"context"
"fmt"
"testing"
"testing/fstest"
)
func BenchmarkProfileRepositoryLookup(b *testing.B) {
for _, size := range []int{10, 1000} {
b.Run(fmt.Sprintf("catalog-%d", size), func(b *testing.B) {
files := fstest.MapFS{
"target.yaml": profileMapFile(`
id: target
endpoint: http://localhost:8000/v1
model: target-model
extra_params:
selected: true
`),
}
metadataNames := []string{"target.yaml"}
for i := 1; i < size; i++ {
name := fmt.Sprintf("profile-%04d.yaml", i)
files[name] = profileMapFile(fmt.Sprintf(`
id: profile-%04d
endpoint: http://localhost:8000/v1
model: unrelated-model
temperature: 0.5
max_tokens: 500
extra_params:
provider:
order:
- first
- second
`, i))
metadataNames = append(metadataNames, name)
}
fsys := &recordingProfileFS{FS: files}
repo := NewFSRepository(fsys, ".")
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if _, err := repo.GetProfile(context.Background(), "target"); err != nil {
b.Fatal(err)
}
}
b.StopTimer()
for _, name := range metadataNames {
if got := fsys.openCount(name); got != b.N {
b.Fatalf("metadata %q opens = %d, want %d", name, got, b.N)
}
}
})
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,160 +0,0 @@
package profile
import (
"context"
"errors"
"fmt"
"strings"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gitea.maximumdirect.net/eric/promptkit/internal/jsonvalue"
)
const maximumProfileChainLength = 32
type resolvingRepository struct {
source Repository
}
// NewResolvingRepository resolves inherited profile definitions from source.
func NewResolvingRepository(source Repository) Repository {
return &resolvingRepository{source: source}
}
func (r *resolvingRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
if r == nil || r.source == nil {
return nil, fmt.Errorf("%w: profile repository is required", ErrInvalidProfile)
}
requestedID := strings.TrimSpace(id)
if requestedID == "" {
return nil, fmt.Errorf("%w: profile id is required", ErrInvalidProfile)
}
if err := ctx.Err(); err != nil {
return nil, err
}
profile, err := r.getRawProfile(ctx, requestedID)
if err != nil {
return nil, err
}
if profile == nil {
return nil, fmt.Errorf("%w: selected profile %q is nil", ErrInvalidProfile, requestedID)
}
chain := []*domain.ExecutionProfile{profile}
chainIDs := []string{requestedID}
visited := map[string]struct{}{requestedID: {}}
current := profile
for {
baseID := strings.TrimSpace(current.BaseProfileID)
if baseID == "" {
break
}
if err := ctx.Err(); err != nil {
return nil, err
}
if _, seen := visited[baseID]; seen {
return nil, fmt.Errorf("%w: profile inheritance cycle %s", ErrInvalidProfile, joinProfileChain(chainIDs, baseID))
}
if len(chain) >= maximumProfileChainLength {
return nil, fmt.Errorf("%w: profile inheritance chain exceeds %d profiles: %s", ErrInvalidProfile, maximumProfileChainLength, joinProfileChain(chainIDs, baseID))
}
base, err := r.getRawProfile(ctx, baseID)
if err != nil {
if errors.Is(err, ErrProfileNotFound) {
return nil, fmt.Errorf("%w: base profile %q is missing in chain %s", ErrInvalidProfile, baseID, joinProfileChain(chainIDs, baseID))
}
return nil, fmt.Errorf("%w: failed to load base profile %q in chain %s: %w", ErrInvalidProfile, baseID, joinProfileChain(chainIDs, baseID), err)
}
if base == nil {
return nil, fmt.Errorf("%w: base profile %q is nil in chain %s", ErrInvalidProfile, baseID, joinProfileChain(chainIDs, baseID))
}
chain = append(chain, base)
chainIDs = append(chainIDs, baseID)
visited[baseID] = struct{}{}
current = base
}
resolved, err := mergeProfileChain(chain)
if err != nil {
return nil, fmt.Errorf("%w: resolved profile chain %s: %w", ErrInvalidProfile, strings.Join(chainIDs, " -> "), err)
}
if err := validateResolvedProfile(resolved); err != nil {
return nil, fmt.Errorf("%w: resolved profile chain %s: %w", ErrInvalidProfile, strings.Join(chainIDs, " -> "), err)
}
return resolved, nil
}
func (r *resolvingRepository) getRawProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
profile, err := r.source.GetProfile(ctx, id)
if err != nil {
return nil, err
}
if err := ctx.Err(); err != nil {
return nil, err
}
return profile, nil
}
func joinProfileChain(chain []string, next string) string {
return strings.Join(append(append([]string(nil), chain...), next), " -> ")
}
func mergeProfileChain(chain []*domain.ExecutionProfile) (*domain.ExecutionProfile, error) {
resolved := &domain.ExecutionProfile{ID: chain[0].ID}
for index := len(chain) - 1; index >= 0; index-- {
definition := chain[index]
if strings.TrimSpace(definition.BackendID) != "" {
resolved.BackendID = definition.BackendID
}
if strings.TrimSpace(definition.Endpoint) != "" {
resolved.Endpoint = definition.Endpoint
}
if strings.TrimSpace(definition.Model) != "" {
resolved.Model = definition.Model
}
if definition.Temperature != 0 {
resolved.Temperature = definition.Temperature
}
if definition.MaxTokens != 0 {
resolved.MaxTokens = definition.MaxTokens
}
if definition.TopP != 0 {
resolved.TopP = definition.TopP
}
if definition.TimeoutSeconds != 0 {
resolved.TimeoutSeconds = definition.TimeoutSeconds
}
if strings.TrimSpace(definition.ServiceTier) != "" {
resolved.ServiceTier = definition.ServiceTier
}
if strings.TrimSpace(definition.ReasoningEffort) != "" {
resolved.ReasoningEffort = definition.ReasoningEffort
}
if strings.TrimSpace(definition.APIKeyEnv) != "" {
resolved.APIKeyEnv = definition.APIKeyEnv
}
resolved.APIKeyRequired = resolved.APIKeyRequired || definition.APIKeyRequired
if len(definition.ExtraParams) != 0 {
extraParams, err := jsonvalue.CopyMap(definition.ExtraParams)
if err != nil {
return nil, err
}
resolved.ExtraParams = extraParams
}
}
resolved.ID = chain[0].ID
resolved.BaseProfileID = ""
return resolved, nil
}
func validateResolvedProfile(profile *domain.ExecutionProfile) error {
if profile == nil {
return errors.New("resolved profile is required")
}
profile.BaseProfileID = ""
return NormalizeAndValidateDefinition(profile)
}

View File

@@ -1,418 +0,0 @@
package profile
import (
"context"
"errors"
"fmt"
"io/fs"
"reflect"
"strings"
"sync"
"testing"
"testing/fstest"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
)
func TestResolvingRepositoryMergesProfileChain(t *testing.T) {
repo := &resolvingTestRepository{profiles: map[string]*domain.ExecutionProfile{
"leaf": {
ID: "leaf",
BaseProfileID: "middle",
BackendID: "leaf-backend",
TopP: 0.8,
TimeoutSeconds: 45,
ReasoningEffort: "high",
},
"middle": {
ID: "middle",
BaseProfileID: "root",
Endpoint: "https://middle.example/v1",
Model: "middle-model",
MaxTokens: 256,
APIKeyEnv: "MIDDLE_API_KEY",
APIKeyRequired: true,
ExtraParams: map[string]any{"middle": map[string]any{"value": "middle"}},
},
"root": {
ID: "root",
BackendID: "root-backend",
Endpoint: "https://root.example/v1",
Model: "root-model",
Temperature: 0.3,
ServiceTier: "priority",
ExtraParams: map[string]any{"root": "value"},
},
}}
got, err := NewResolvingRepository(repo).GetProfile(context.Background(), "leaf")
if err != nil {
t.Fatalf("resolve profile: %v", err)
}
want := &domain.ExecutionProfile{
ID: "leaf",
BackendID: "leaf-backend",
Endpoint: "https://middle.example/v1",
Model: "middle-model",
Temperature: 0.3,
MaxTokens: 256,
TopP: 0.8,
TimeoutSeconds: 45,
ServiceTier: "priority",
ReasoningEffort: "high",
APIKeyEnv: "MIDDLE_API_KEY",
APIKeyRequired: true,
ExtraParams: map[string]any{"middle": map[string]any{"value": "middle"}},
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("resolved profile:\n got %#v\nwant %#v", got, want)
}
}
func TestResolvingRepositoryRejectsMissingSourceAndProfileID(t *testing.T) {
if _, err := NewResolvingRepository(nil).GetProfile(context.Background(), "profile"); !errors.Is(err, ErrInvalidProfile) {
t.Fatalf("nil source error = %v, want ErrInvalidProfile", err)
}
repo := &resolvingTestRepository{profiles: map[string]*domain.ExecutionProfile{}}
if _, err := NewResolvingRepository(repo).GetProfile(context.Background(), " \t "); !errors.Is(err, ErrInvalidProfile) {
t.Fatalf("blank id error = %v, want ErrInvalidProfile", err)
}
if got := repo.callCount(" "); got != 0 {
t.Fatalf("blank id looked up source %d times", got)
}
}
func TestResolvingRepositoryCopiesExtraParams(t *testing.T) {
baseParams := map[string]any{"nested": map[string]any{"value": "base"}}
repo := &resolvingTestRepository{profiles: map[string]*domain.ExecutionProfile{
"child": {ID: "child", BaseProfileID: "base"},
"base": {
ID: "base",
Endpoint: "https://base.example/v1",
Model: "model",
ExtraParams: baseParams,
},
}}
resolver := NewResolvingRepository(repo)
first, err := resolver.GetProfile(context.Background(), "child")
if err != nil {
t.Fatalf("resolve inherited map: %v", err)
}
first.ExtraParams["nested"].(map[string]any)["value"] = "mutated"
second, err := resolver.GetProfile(context.Background(), "child")
if err != nil {
t.Fatalf("resolve inherited map again: %v", err)
}
if got := second.ExtraParams["nested"].(map[string]any)["value"]; got != "base" {
t.Fatalf("later result retained mutation: %v", got)
}
if got := baseParams["nested"].(map[string]any)["value"]; got != "base" {
t.Fatalf("source map retained mutation: %v", got)
}
repo.set("child", &domain.ExecutionProfile{
ID: "child",
BaseProfileID: "base",
ExtraParams: map[string]any{"child": "replacement"},
})
replaced, err := resolver.GetProfile(context.Background(), "child")
if err != nil {
t.Fatalf("resolve replacement map: %v", err)
}
if !reflect.DeepEqual(replaced.ExtraParams, map[string]any{"child": "replacement"}) {
t.Fatalf("extra params = %#v, want complete child replacement", replaced.ExtraParams)
}
}
func TestResolvingRepositoryUsesRawOverlayForEachLookup(t *testing.T) {
leafSource := NewFSRepository(profileTestFS(map[string]string{
"leaf.yaml": "id: leaf\nbase_profile: base\n",
}), ".")
fallback := NewFSRepository(profileTestFS(map[string]string{
"base.yaml": "id: base\nendpoint: https://fallback.example/v1\nmodel: fallback-model\n",
}), ".")
overlay := NewOverlayRepository(leafSource, fallback)
resolver := NewResolvingRepository(overlay)
got, err := resolver.GetProfile(context.Background(), "leaf")
if err != nil {
t.Fatalf("resolve fallback base: %v", err)
}
if got.Model != "fallback-model" {
t.Fatalf("fallback base model = %q", got.Model)
}
shadowing := NewOverlayRepository(NewFSRepository(profileTestFS(map[string]string{
"leaf.yaml": "id: leaf\nbase_profile: base\n",
"base.yaml": "id: base\nendpoint: https://primary.example/v1\nmodel: primary-model\n",
}), "."), fallback)
got, err = NewResolvingRepository(shadowing).GetProfile(context.Background(), "leaf")
if err != nil {
t.Fatalf("resolve shadowed base: %v", err)
}
if got.Model != "primary-model" || got.Endpoint != "https://primary.example/v1" {
t.Fatalf("shadowed base = %+v", got)
}
}
func TestResolvingRepositoryReportsSafetyAndSourceErrors(t *testing.T) {
sourceErr := errors.New("source failure")
tests := []struct {
name string
repo *resolvingTestRepository
id string
want []error
wantNot error
contains []string
}{
{
name: "missing selected profile preserves not found",
repo: &resolvingTestRepository{profiles: map[string]*domain.ExecutionProfile{}},
id: "missing",
want: []error{ErrProfileNotFound},
wantNot: ErrInvalidProfile,
},
{
name: "missing base is invalid but not not found",
repo: &resolvingTestRepository{profiles: map[string]*domain.ExecutionProfile{
"leaf": {ID: "leaf", BaseProfileID: "missing"},
}},
id: "leaf",
want: []error{ErrInvalidProfile},
wantNot: ErrProfileNotFound,
contains: []string{"missing", "leaf -> missing"},
},
{
name: "direct cycle",
repo: &resolvingTestRepository{profiles: map[string]*domain.ExecutionProfile{
"a": {ID: "a", BaseProfileID: "a"},
}},
id: "a",
want: []error{ErrInvalidProfile},
contains: []string{"a -> a"},
},
{
name: "indirect cycle",
repo: &resolvingTestRepository{profiles: map[string]*domain.ExecutionProfile{
"a": {ID: "a", BaseProfileID: "b"},
"b": {ID: "b", BaseProfileID: "c"},
"c": {ID: "c", BaseProfileID: "a"},
}},
id: "a",
want: []error{ErrInvalidProfile},
contains: []string{"a -> b -> c -> a"},
},
{
name: "nil result",
repo: &resolvingTestRepository{profiles: map[string]*domain.ExecutionProfile{
"leaf": nil,
}},
id: "leaf",
want: []error{ErrInvalidProfile},
},
{
name: "incomplete resolved profile",
repo: &resolvingTestRepository{profiles: map[string]*domain.ExecutionProfile{
"leaf": {ID: "leaf", BaseProfileID: "base"},
"base": {ID: "base", Model: "model"},
}},
id: "leaf",
want: []error{ErrInvalidProfile},
},
{
name: "base source error is retained",
repo: &resolvingTestRepository{
profiles: map[string]*domain.ExecutionProfile{"leaf": {ID: "leaf", BaseProfileID: "base"}},
errors: map[string]error{"base": sourceErr},
},
id: "leaf",
want: []error{ErrInvalidProfile, sourceErr},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := NewResolvingRepository(tc.repo).GetProfile(context.Background(), tc.id)
for _, want := range tc.want {
if !errors.Is(err, want) {
t.Fatalf("error = %v, want %v", err, want)
}
}
if tc.wantNot != nil && errors.Is(err, tc.wantNot) {
t.Fatalf("error = %v, must not match %v", err, tc.wantNot)
}
for _, fragment := range tc.contains {
if !strings.Contains(err.Error(), fragment) {
t.Fatalf("error = %v, want %q", err, fragment)
}
}
})
}
}
func TestResolvingRepositoryEnforcesChainLength(t *testing.T) {
for _, count := range []int{maximumProfileChainLength, maximumProfileChainLength + 1} {
t.Run(fmt.Sprintf("%d profiles", count), func(t *testing.T) {
profiles := make(map[string]*domain.ExecutionProfile, count)
for index := 1; index <= count; index++ {
id := fmt.Sprintf("profile-%d", index)
definition := &domain.ExecutionProfile{ID: id}
if index == count {
definition.Endpoint = "https://root.example/v1"
definition.Model = "model"
} else {
definition.BaseProfileID = fmt.Sprintf("profile-%d", index+1)
}
profiles[id] = definition
}
got, err := NewResolvingRepository(&resolvingTestRepository{profiles: profiles}).GetProfile(context.Background(), "profile-1")
if count == maximumProfileChainLength {
if err != nil || got == nil {
t.Fatalf("profile = %+v, error = %v, want accepted chain", got, err)
}
return
}
if !errors.Is(err, ErrInvalidProfile) {
t.Fatalf("error = %v, want ErrInvalidProfile", err)
}
})
}
}
func TestResolvingRepositoryIsFreshAndCancellationAware(t *testing.T) {
repo := &resolvingTestRepository{profiles: map[string]*domain.ExecutionProfile{
"leaf": {ID: "leaf", BaseProfileID: "base"},
"base": {ID: "base", Endpoint: "https://base.example/v1", Model: "first", ExtraParams: map[string]any{"nested": map[string]any{"value": "first"}}},
}}
resolver := NewResolvingRepository(repo)
first, err := resolver.GetProfile(context.Background(), "leaf")
if err != nil || first.Model != "first" {
t.Fatalf("first result=(%+v, %v)", first, err)
}
repo.set("base", &domain.ExecutionProfile{ID: "base", Endpoint: "https://base.example/v1", Model: "second", ExtraParams: map[string]any{"nested": map[string]any{"value": "second"}}})
second, err := resolver.GetProfile(context.Background(), "leaf")
if err != nil || second.Model != "second" {
t.Fatalf("second result=(%+v, %v)", second, err)
}
canceled, cancel := context.WithCancel(context.Background())
cancel()
if _, err := resolver.GetProfile(canceled, "leaf"); !errors.Is(err, context.Canceled) {
t.Fatalf("canceled lookup error = %v", err)
}
if got := repo.callCount("leaf"); got != 2 {
t.Fatalf("calls after canceled lookup = %d, want 2", got)
}
duringTraversal, cancelDuringTraversal := context.WithCancel(context.Background())
repo.afterGet = func(id string) {
if id == "leaf" {
cancelDuringTraversal()
}
}
if _, err := resolver.GetProfile(duringTraversal, "leaf"); !errors.Is(err, context.Canceled) {
t.Fatalf("during traversal error = %v", err)
}
if got := repo.callCount("base"); got != 2 {
t.Fatalf("base calls after cancellation = %d, want 2", got)
}
terminalLookup, cancelTerminalLookup := context.WithCancel(context.Background())
repo.afterGet = func(id string) {
if id == "base" {
cancelTerminalLookup()
}
}
if _, err := resolver.GetProfile(terminalLookup, "leaf"); !errors.Is(err, context.Canceled) {
t.Fatalf("terminal lookup cancellation error = %v", err)
}
if got := repo.callCount("base"); got != 3 {
t.Fatalf("base calls after terminal cancellation = %d, want 3", got)
}
repo.afterGet = nil
var wg sync.WaitGroup
errors := make(chan error, 8)
for index := 0; index < cap(errors); index++ {
wg.Add(1)
go func() {
defer wg.Done()
resolved, err := resolver.GetProfile(context.Background(), "leaf")
if err != nil {
errors <- err
return
}
resolved.ExtraParams["nested"].(map[string]any)["value"] = "mutated"
}()
}
wg.Wait()
close(errors)
for err := range errors {
t.Errorf("concurrent resolution: %v", err)
}
latest, err := resolver.GetProfile(context.Background(), "leaf")
if err != nil || latest.ExtraParams["nested"].(map[string]any)["value"] != "second" {
t.Fatalf("latest result=(%+v, %v)", latest, err)
}
}
type resolvingTestRepository struct {
mu sync.Mutex
profiles map[string]*domain.ExecutionProfile
errors map[string]error
calls map[string]int
afterGet func(string)
}
func (r *resolvingTestRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
if err := ctx.Err(); err != nil {
return nil, err
}
r.mu.Lock()
if r.calls == nil {
r.calls = make(map[string]int)
}
r.calls[id]++
err := r.errors[id]
profile := r.profiles[id]
afterGet := r.afterGet
r.mu.Unlock()
if afterGet != nil {
afterGet(id)
}
if err != nil {
return nil, err
}
if profile == nil {
if _, exists := r.profiles[id]; exists {
return nil, nil
}
return nil, ErrProfileNotFound
}
copy := *profile
return &copy, nil
}
func (r *resolvingTestRepository) set(id string, profile *domain.ExecutionProfile) {
r.mu.Lock()
defer r.mu.Unlock()
r.profiles[id] = profile
}
func (r *resolvingTestRepository) callCount(id string) int {
r.mu.Lock()
defer r.mu.Unlock()
return r.calls[id]
}
func profileTestFS(files map[string]string) fs.FS {
fsys := make(fstest.MapFS, len(files))
for name, content := range files {
fsys[name] = profileMapFile(content)
}
return fsys
}

View File

@@ -1,3 +0,0 @@
id: invalid_yaml
endpoint: http://localhost:8000/v1
model: [broken

View File

@@ -1,2 +0,0 @@
id: missing-endpoint
model: gpt-4o-mini

View File

@@ -1,2 +0,0 @@
endpoint: http://localhost:8000/v1
model: gpt-4o-mini

View File

@@ -1,2 +0,0 @@
id: missing-model
endpoint: http://localhost:8000/v1

View File

@@ -1,4 +0,0 @@
id: raw-api-key
endpoint: http://localhost:8000/v1
model: gpt-4o-mini
api_key: super-secret-should-not-be-here

View File

@@ -1,4 +0,0 @@
id: unknown-field
endpoint: http://localhost:8000/v1
model: gpt-4o-mini
foo: bar

View File

@@ -1,7 +0,0 @@
id: local-default
endpoint: http://localhost:8000/v1
model: gpt-4o-mini
temperature: 0.2
max_tokens: 700
top_p: 1.0
timeout_seconds: 120

View File

@@ -1,8 +0,0 @@
id: local-secure
endpoint: http://localhost:8000/v1
model: gpt-4o-mini
api_key_env: PROMPTKIT_API_KEY
service_tier: priority
reasoning_effort: medium
extra_params:
provider: local

View File

@@ -21,12 +21,20 @@ var (
const artifactTextChunkSize = 64 * 1024
type goRenderer struct{}
type goRenderer struct {
templates TemplateLookup
}
func NewGoRenderer() Renderer {
return &goRenderer{}
}
// NewCompiledRenderer renders definitions using templates captured with the
// configuration snapshot. Per-call clones bind request-specific input helpers.
func NewCompiledRenderer(templates TemplateLookup) Renderer {
return &goRenderer{templates: templates}
}
func (r *goRenderer) Render(ctx context.Context, definition *domain.PromptDefinition, inputs map[string]*domain.Artifact, vars map[string]string) (*domain.RenderedPrompt, error) {
if err := ctx.Err(); err != nil {
return nil, err
@@ -49,11 +57,30 @@ func (r *goRenderer) Render(ctx context.Context, definition *domain.PromptDefini
funcs := template.FuncMap{
"input": resolver.resolve,
}
var compiled CompiledTemplates
if r.templates != nil {
var found bool
compiled, found = r.templates.GetCompiledTemplates(definition.ID, definition.Version)
if !found || len(compiled.Messages) != len(definition.Templates) {
return nil, fmt.Errorf("%w: captured templates are unavailable", ErrRenderFailure)
}
}
if err := ctx.Err(); err != nil {
return nil, err
}
sessionID, err := renderSessionID(ctx, definition.SessionID, funcs, vars)
var sessionID string
var err error
if r.templates != nil {
if definition.SessionID != "" {
if compiled.Session == nil {
return nil, fmt.Errorf("%w: captured session template is unavailable", ErrRenderFailure)
}
sessionID, err = renderCompiledSessionID(ctx, compiled.Session, funcs, vars)
}
} else {
sessionID, err = renderSessionID(ctx, definition.SessionID, funcs, vars)
}
if err != nil {
return nil, err
}
@@ -76,7 +103,19 @@ func (r *goRenderer) Render(ctx context.Context, definition *domain.PromptDefini
if err := ctx.Err(); err != nil {
return nil, err
}
tmpl, parseErr := template.New(fmt.Sprintf("msg_%d", i)).Funcs(funcs).Option("missingkey=error").Parse(tmplMsg.Content)
var tmpl *template.Template
var parseErr error
if r.templates != nil {
if compiled.Messages[i] == nil {
return nil, fmt.Errorf("%w: captured message template is unavailable", ErrRenderFailure)
}
tmpl, parseErr = compiled.Messages[i].Clone()
if parseErr == nil {
tmpl.Funcs(funcs)
}
} else {
tmpl, parseErr = parseTemplate(fmt.Sprintf("msg_%d", i), tmplMsg.Content, funcs)
}
if err := ctx.Err(); err != nil {
return nil, err
}
@@ -167,14 +206,32 @@ func renderSessionID(ctx context.Context, raw string, funcs template.FuncMap, va
if err := ctx.Err(); err != nil {
return "", err
}
tmpl, parseErr := template.New("session_id").Funcs(funcs).Option("missingkey=error").Parse(raw)
tmpl, parseErr := parseTemplate("session_id", raw, funcs)
if err := ctx.Err(); err != nil {
return "", err
}
if parseErr != nil {
return "", fmt.Errorf("%w: session_id: %v", ErrInvalidTemplate, parseErr)
}
return executeSessionTemplate(ctx, tmpl, vars)
}
func renderCompiledSessionID(ctx context.Context, compiled *template.Template, funcs template.FuncMap, vars map[string]string) (string, error) {
if err := ctx.Err(); err != nil {
return "", err
}
tmpl, err := compiled.Clone()
if err := ctx.Err(); err != nil {
return "", err
}
if err != nil {
return "", fmt.Errorf("%w: session_id: %v", ErrRenderFailure, err)
}
tmpl.Funcs(funcs)
return executeSessionTemplate(ctx, tmpl, vars)
}
func executeSessionTemplate(ctx context.Context, tmpl *template.Template, vars map[string]string) (string, error) {
if err := ctx.Err(); err != nil {
return "", err
}

View File

@@ -0,0 +1,27 @@
package prompt
import "text/template"
// CompiledTemplates is the immutable parsed template set for one captured
// prompt. Callers must not mutate its templates after handing it to a renderer.
type CompiledTemplates struct {
Session *template.Template
Messages []*template.Template
}
// TemplateLookup provides parsed templates for a captured prompt identity.
type TemplateLookup interface {
GetCompiledTemplates(id, version string) (CompiledTemplates, bool)
}
// CompileTemplate checks static syntax with the same grammar and function
// names as rendering, without evaluating request inputs or variables.
func CompileTemplate(name, raw string) (*template.Template, error) {
return parseTemplate(name, raw, template.FuncMap{
"input": func(string) (string, error) { return "", nil },
})
}
func parseTemplate(name, raw string, funcs template.FuncMap) (*template.Template, error) {
return template.New(name).Funcs(funcs).Option("missingkey=error").Parse(raw)
}

View File

@@ -1,98 +0,0 @@
package promptdef
import (
"fmt"
"io/fs"
"os"
"path"
"path/filepath"
"strings"
"gitea.maximumdirect.net/eric/promptkit/internal/filecatalog"
)
type contentSourceRoot interface {
readContentFile(sourcePath string, contentFile string) (string, string, error)
}
type osContentSourceRoot struct {
root string
sourcePathsRelative bool
}
func (r osContentSourceRoot) readContentFile(sourcePath string, contentFile string) (string, string, error) {
if strings.TrimSpace(contentFile) == "" {
return "", "", fmt.Errorf("path is required")
}
if filepath.IsAbs(contentFile) {
return "", "", fmt.Errorf("path %q must be relative", contentFile)
}
root, err := filepath.Abs(r.root)
if err != nil {
return "", "", fmt.Errorf("resolve source root %q: %w", r.root, err)
}
canonicalRoot, err := filepath.EvalSymlinks(root)
if err != nil {
return "", "", fmt.Errorf("resolve source root %q: %w", r.root, err)
}
promptPath := sourcePath
if r.sourcePathsRelative && !filepath.IsAbs(promptPath) {
promptPath = filepath.Join(root, filepath.FromSlash(promptPath))
} else {
promptPath, err = filepath.Abs(promptPath)
if err != nil {
return "", "", fmt.Errorf("resolve prompt source %q: %w", sourcePath, err)
}
}
resolvedPath := filepath.Clean(filepath.Join(filepath.Dir(promptPath), contentFile))
if !containsOSPath(root, resolvedPath) {
return "", "", fmt.Errorf("path %q escapes source root %q", contentFile, r.root)
}
canonicalPath, err := filepath.EvalSymlinks(resolvedPath)
if err != nil {
return "", "", err
}
if !containsOSPath(canonicalRoot, canonicalPath) {
return "", "", fmt.Errorf("path %q escapes source root %q", contentFile, r.root)
}
body, err := os.ReadFile(canonicalPath)
if err != nil {
return "", "", err
}
return string(body), resolvedPath, nil
}
type fsContentSourceRoot struct {
fsys fs.FS
root string
}
func (r fsContentSourceRoot) readContentFile(sourcePath string, contentFile string) (string, string, error) {
root := filecatalog.CleanFSRoot(r.root)
cleanSourcePath := path.Clean(sourcePath)
if cleanSourcePath == root {
root = path.Dir(root)
}
resolvedPath, _, err := filecatalog.ResolveFSPath(root, path.Dir(cleanSourcePath), contentFile)
if err != nil {
return "", "", err
}
body, err := fs.ReadFile(r.fsys, resolvedPath)
if err != nil {
return "", "", err
}
return string(body), resolvedPath, nil
}
func containsOSPath(root string, name string) bool {
relative, err := filepath.Rel(root, name)
if err != nil || filepath.IsAbs(relative) {
return false
}
return relative != ".." && !strings.HasPrefix(relative, ".."+string(filepath.Separator))
}

View File

@@ -1,328 +0,0 @@
package promptdef
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"io/fs"
"strings"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
"gopkg.in/yaml.v3"
)
var (
ErrPromptDefinitionNotFound = errors.New("prompt definition not found")
ErrInvalidYAML = errors.New("invalid YAML format")
ErrInvalidPromptDefinition = errors.New("invalid prompt definition configuration")
)
type sourceRepository struct {
source promptDefinitionSource
}
type promptDefinitionFile struct {
ID string `yaml:"id"`
Version string `yaml:"version"`
DefaultProfile *string `yaml:"default_profile"`
Description string `yaml:"description"`
SessionID string `yaml:"session_id"`
Inputs []promptInputFile `yaml:"inputs"`
Messages []promptMessageFile `yaml:"messages"`
Output promptOutputContractFile `yaml:"output"`
}
type promptInputFile struct {
Name string `yaml:"name"`
Required bool `yaml:"required"`
ContentType string `yaml:"content_type"`
Description string `yaml:"description"`
}
type promptMessageFile struct {
Role string `yaml:"role"`
Content string `yaml:"content"`
ContentFile string `yaml:"content_file"`
CacheControl *cacheControlFile `yaml:"cache_control"`
}
type cacheControlFile struct {
Type string `yaml:"type"`
TTL string `yaml:"ttl"`
}
type promptOutputContractFile struct {
Format domain.OutputFormat `yaml:"format"`
ValidationMode domain.ValidationMode `yaml:"validation_mode"`
SchemaPath string `yaml:"schema_path"`
RepairAttempts int `yaml:"repair_attempts"`
}
func NewFilesystemRepository(dir string) Repository {
return &sourceRepository{
source: osPromptSource{
root: dir,
contentRoot: osContentSourceRoot{root: dir},
},
}
}
func NewFSRepository(fsys fs.FS, root string) Repository {
return &sourceRepository{
source: fsPromptSource{
fsys: fsys,
root: root,
contentRoot: fsContentSourceRoot{fsys: fsys, root: root},
},
}
}
// NewFileRepository constructs a repository for one operating-system prompt file.
func NewFileRepository(fsys fs.FS, file string, sourceDir string) Repository {
return &sourceRepository{
source: fsPromptSource{
fsys: fsys,
root: file,
contentRoot: osContentSourceRoot{
root: sourceDir,
sourcePathsRelative: true,
},
},
}
}
func (r *sourceRepository) GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error) {
if strings.TrimSpace(id) == "" {
return nil, fmt.Errorf("%w: prompt id is required", ErrInvalidPromptDefinition)
}
if r == nil || r.source == nil {
return nil, errors.New("failed to read prompt definition directory: source is nil")
}
files, err := r.source.findYAMLFiles(ctx)
if err != nil {
return nil, fmt.Errorf("failed to read prompt definition directory: %w", err)
}
var matches []promptDefinitionMatch
for _, fullPath := range files {
select {
case <-ctx.Done():
return nil, ctx.Err()
default:
}
relPath := r.source.displayPath(fullPath)
data, err := r.source.readDefinition(fullPath)
if err != nil {
return nil, fmt.Errorf("failed to read prompt definition file %s: %w", relPath, err)
}
raw, err := decodePromptDefinition(data)
if err != nil {
if promptDefinitionDataMatches(data, id, version) {
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidYAML, relPath, err)
}
continue
}
if !promptDefinitionMatches(raw, id, version) {
continue
}
matches = append(matches, promptDefinitionMatch{
raw: raw,
sourcePath: fullPath,
path: relPath,
})
}
if len(matches) > 1 {
paths := make([]string, 0, len(matches))
for _, match := range matches {
paths = append(paths, match.path)
}
if version != "" {
return nil, fmt.Errorf("%w: duplicate prompt definition id %q version %q found in: %s", ErrInvalidPromptDefinition, id, version, strings.Join(paths, ", "))
}
return nil, fmt.Errorf("%w: duplicate prompt definition id %q found in: %s", ErrInvalidPromptDefinition, id, strings.Join(paths, ", "))
}
if len(matches) == 1 {
match := matches[0]
def, err := normalizePromptDefinition(match.raw, r.source, match.sourcePath)
if err != nil {
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidPromptDefinition, match.path, err)
}
return def, nil
}
return nil, ErrPromptDefinitionNotFound
}
type promptDefinitionMatch struct {
raw *promptDefinitionFile
sourcePath string
path string
}
func decodePromptDefinition(data []byte) (*promptDefinitionFile, error) {
var raw promptDefinitionFile
decoder := yaml.NewDecoder(bytes.NewReader(data))
decoder.KnownFields(true)
if err := decoder.Decode(&raw); err != nil {
return nil, err
}
var additional yaml.Node
if err := decoder.Decode(&additional); err != io.EOF {
if err != nil {
return nil, err
}
return nil, errors.New("prompt definition file must contain exactly one YAML document")
}
return &raw, nil
}
func promptDefinitionDataMatches(data []byte, id string, version string) bool {
var raw struct {
ID string `yaml:"id"`
Version string `yaml:"version"`
}
if err := yaml.NewDecoder(bytes.NewReader(data)).Decode(&raw); err != nil {
return false
}
return promptSelectorMatches(raw.ID, raw.Version, id, version)
}
func promptDefinitionMatches(raw *promptDefinitionFile, id string, version string) bool {
if raw == nil {
return false
}
return promptSelectorMatches(raw.ID, raw.Version, id, version)
}
func promptSelectorMatches(rawID string, rawVersion string, id string, version string) bool {
if strings.TrimSpace(rawID) != id {
return false
}
return version == "" || strings.TrimSpace(rawVersion) == version
}
func normalizePromptDefinition(raw *promptDefinitionFile, sourceRoot contentSourceRoot, sourcePath string) (*domain.PromptDefinition, error) {
return normalizePromptDefinitionWithContent(raw, func(contentFile string) (string, string, error) {
return sourceRoot.readContentFile(sourcePath, contentFile)
})
}
func normalizePromptDefinitionWithContent(raw *promptDefinitionFile, readContentFile func(string) (string, string, error)) (*domain.PromptDefinition, error) {
if raw == nil {
return nil, errors.New("prompt definition is nil")
}
id := strings.TrimSpace(raw.ID)
if id == "" {
return nil, errors.New("id is required")
}
version := strings.TrimSpace(raw.Version)
if version == "" {
return nil, errors.New("version is required")
}
if len(raw.Messages) == 0 {
return nil, errors.New("at least one message is required")
}
inputs := make([]domain.PromptInput, 0, len(raw.Inputs))
seenInputNames := make(map[string]struct{}, len(raw.Inputs))
for i, in := range raw.Inputs {
name := strings.TrimSpace(in.Name)
if name == "" {
return nil, fmt.Errorf("input %d has empty name", i)
}
if _, exists := seenInputNames[name]; exists {
return nil, fmt.Errorf("duplicate input name %q", name)
}
seenInputNames[name] = struct{}{}
inputs = append(inputs, domain.PromptInput{
Name: name,
Required: in.Required,
ContentType: strings.TrimSpace(in.ContentType),
Description: strings.TrimSpace(in.Description),
})
}
templates := make([]domain.PromptMessageTemplate, 0, len(raw.Messages))
for i, msg := range raw.Messages {
role, err := domain.NormalizeMessageRole(msg.Role)
if err != nil {
return nil, fmt.Errorf("message %d role: %w", i, err)
}
hasContent := strings.TrimSpace(msg.Content) != ""
hasContentFile := strings.TrimSpace(msg.ContentFile) != ""
if hasContent == hasContentFile {
return nil, fmt.Errorf("message %d must set exactly one of content or content_file", i)
}
var rawCacheControl *domain.CacheControl
if msg.CacheControl != nil {
rawCacheControl = &domain.CacheControl{
Type: domain.CacheControlType(msg.CacheControl.Type),
TTL: msg.CacheControl.TTL,
}
}
cacheControl, err := domain.NormalizeCacheControl(rawCacheControl)
if err != nil {
return nil, fmt.Errorf("message %d cache_control: %w", i, err)
}
templateContent := msg.Content
resolvedContentFile := ""
if hasContentFile {
body, resolvedPath, err := readContentFile(msg.ContentFile)
if err != nil {
return nil, fmt.Errorf("prompt %q message %d: failed to read content_file %q: %w", id, i, msg.ContentFile, err)
}
templateContent = body
resolvedContentFile = resolvedPath
}
templates = append(templates, domain.PromptMessageTemplate{
Role: role,
Content: templateContent,
ContentFile: resolvedContentFile,
CacheControl: cacheControl,
})
}
outputContract := domain.OutputContract{
Format: raw.Output.Format,
ValidationMode: raw.Output.ValidationMode,
SchemaPath: strings.TrimSpace(raw.Output.SchemaPath),
RepairAttempts: raw.Output.RepairAttempts,
}
if err := domain.ValidateOutputContract(outputContract); err != nil {
return nil, fmt.Errorf("output: %w", err)
}
defaultProfile := ""
if raw.DefaultProfile != nil {
defaultProfile = strings.TrimSpace(*raw.DefaultProfile)
if defaultProfile == "" {
return nil, errors.New("default_profile must be a non-empty string when set")
}
}
return &domain.PromptDefinition{
ID: id,
Version: version,
DefaultProfile: defaultProfile,
Description: strings.TrimSpace(raw.Description),
SessionID: strings.TrimSpace(raw.SessionID),
Inputs: inputs,
Templates: templates,
OutputFormat: raw.Output.Format,
Validation: outputContract,
}, nil
}

View File

@@ -0,0 +1,19 @@
package promptdef
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
)
// HashDefinition hashes an execution-ready prompt, including captured content.
func HashDefinition(definition *domain.PromptDefinition) (string, error) {
bytes, err := json.Marshal(definition)
if err != nil {
return "", err
}
sum := sha256.Sum256(bytes)
return hex.EncodeToString(sum[:]), nil
}

View File

@@ -2,11 +2,15 @@ package promptdef
import (
"context"
"errors"
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
)
// Repository loads prompt definitions.
// ErrPromptDefinitionNotFound identifies an absent captured prompt definition.
var ErrPromptDefinitionNotFound = errors.New("prompt definition not found")
// Repository selects prompt definitions from a captured index.
type Repository interface {
GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error)
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,63 +0,0 @@
package promptdef
import (
"context"
"errors"
"io/fs"
"os"
"gitea.maximumdirect.net/eric/promptkit/internal/filecatalog"
)
type promptDefinitionSource interface {
contentSourceRoot
findYAMLFiles(context.Context) ([]string, error)
readDefinition(string) ([]byte, error)
displayPath(string) string
}
type osPromptSource struct {
root string
contentRoot osContentSourceRoot
}
func (s osPromptSource) findYAMLFiles(ctx context.Context) ([]string, error) {
return filecatalog.FindYAMLFiles(ctx, s.root)
}
func (s osPromptSource) readDefinition(name string) ([]byte, error) {
return os.ReadFile(name)
}
func (s osPromptSource) displayPath(name string) string {
return filecatalog.RelativePath(s.root, name)
}
func (s osPromptSource) readContentFile(sourcePath string, contentFile string) (string, string, error) {
return s.contentRoot.readContentFile(sourcePath, contentFile)
}
type fsPromptSource struct {
fsys fs.FS
root string
contentRoot contentSourceRoot
}
func (s fsPromptSource) findYAMLFiles(ctx context.Context) ([]string, error) {
if s.fsys == nil {
return nil, errors.New("filesystem is nil")
}
return filecatalog.FindFSYAMLFiles(ctx, s.fsys, s.root)
}
func (s fsPromptSource) readDefinition(name string) ([]byte, error) {
return fs.ReadFile(s.fsys, name)
}
func (s fsPromptSource) displayPath(name string) string {
return filecatalog.DisplayPath(s.root, name)
}
func (s fsPromptSource) readContentFile(sourcePath string, contentFile string) (string, string, error) {
return s.contentRoot.readContentFile(sourcePath, contentFile)
}

View File

@@ -1,10 +0,0 @@
id: both-content-and-content-file
version: "1.0.0"
messages:
- role: user
content: "Hi"
content_file: ./messages/user_prompt.tmpl
output:
format: text
validation_mode: none
repair_attempts: 0

View File

@@ -1,14 +0,0 @@
id: duplicate-input-names
version: "1.0.0"
inputs:
- name: transcript
required: true
- name: transcript
required: false
messages:
- role: user
content: "Hi"
output:
format: text
validation_mode: none
repair_attempts: 0

View File

@@ -1,10 +0,0 @@
id: empty-cache-control-type
version: "1.0.0"
messages:
- role: system
content: "Use cached instructions."
cache_control: {}
output:
format: markdown
validation_mode: basic
repair_attempts: 0

View File

@@ -1,9 +0,0 @@
id: invalid-validation-mode
version: "1.0.0"
messages:
- role: user
content: "Hi"
output:
format: text
validation_mode: nope
repair_attempts: 0

View File

@@ -1,9 +0,0 @@
id: invalid-yaml
version: "1.0.0"
messages:
- role: user
content: [broken
output:
format: text
validation_mode: none
repair_attempts: 0

View File

@@ -1,9 +0,0 @@
id: json-schema-without-schema-path
version: "1.0.0"
messages:
- role: user
content: "Return JSON"
output:
format: json
validation_mode: json_schema
repair_attempts: 0

View File

@@ -1,2 +0,0 @@
Use transcript:
{{input "transcript"}}

View File

@@ -1,9 +0,0 @@
id: missing-content-file
version: "1.0.0"
messages:
- role: user
content_file: ./messages/does_not_exist.tmpl
output:
format: text
validation_mode: none
repair_attempts: 0

View File

@@ -1,8 +0,0 @@
version: "1.0.0"
messages:
- role: user
content: "Hi"
output:
format: text
validation_mode: none
repair_attempts: 0

View File

@@ -1,8 +0,0 @@
id: neither-content-nor-content-file
version: "1.0.0"
messages:
- role: user
output:
format: text
validation_mode: none
repair_attempts: 0

View File

@@ -1,6 +0,0 @@
id: no-messages
version: "1.0.0"
output:
format: text
validation_mode: none
repair_attempts: 0

Some files were not shown because too many files have changed in this diff Show More