diff --git a/docs/consumers/pkg-promptkit.md b/docs/consumers/pkg-promptkit.md index a0ac9f8..41b661c 100644 --- a/docs/consumers/pkg-promptkit.md +++ b/docs/consumers/pkg-promptkit.md @@ -40,6 +40,32 @@ validation, and default transport behavior. Source discovery, format validation, and profile precedence are defined by the [framework format reference](../formats.md). +## Supply Embedded Application Defaults + +Use `WithFallbackProfileFS` when an application packages profile definitions +that should apply unless an operator provides an ordinary configured profile +with the same ID. For example, an application can embed its defaults while +continuing to use `ProfileDir` for operator overrides: + +```go +//go:embed profiles/*.yaml +var applicationProfiles embed.FS + +engine, err := promptkit.NewEngine(promptkit.Config{ + PromptDir: "prompts", + ProfileDir: "profiles", +}, + promptkit.WithFallbackProfileFS(applicationProfiles, "profiles"), +) +``` + +Keep application-owned profile IDs and definitions in the embedded source. +Use the ordinary configured profile source for operator overrides. The +[framework format reference](../formats.md#source-and-profile-precedence) +owns the exact profile format and lookup order; the +[`WithFallbackProfileFS` GoDoc](../../engine.go) owns its option contract and +validation rules. + ## Inspect A Prompt Before Preparation Use [`Engine.InspectPrompt`](../../engine.go) to check one configured prompt's @@ -152,8 +178,8 @@ replace execution settings or the complete output contract. The [public value GoDoc](../../types.go) defines nil, empty, zero, replacement, copy, and credential behavior. The [framework format reference](../formats.md) defines how those request values -interact with prompt definitions, file-backed profiles, built-ins, schemas, -and framework defaults. +interact with prompt definitions, file-backed and application fallback +profiles, built-ins, schemas, and framework defaults. For programmatic profiles, [`OpenAICompatibleProfile`](../../profiles.go) converts ordinary diff --git a/docs/formats.md b/docs/formats.md index 8cffdfd..8aa1775 100644 --- a/docs/formats.md +++ b/docs/formats.md @@ -233,14 +233,18 @@ default. Profile sources resolve matching IDs in this order: 1. in-memory profiles supplied with `WithProfiles`; -2. a profile file, `fs.FS`, or configured profile directory; and -3. embedded built-in profiles. +2. the ordinary configured source selected by a profile file, `fs.FS`, or + configured profile directory; +3. application fallback profiles supplied with `WithFallbackProfileFS`; and +4. embedded built-in profiles. -A higher-precedence source falls back only when the profile is absent. An -invalid matching profile is an error and does not fall back. In-memory -`Profile` values follow the same ranges as YAML profiles. They use -`APIKeyRequired` for request-scoped credentials instead of `api_key_env`. -Preparation and exact profile inspection use this same source precedence. +A profile source supplies a complete definition; definitions and their fields +are not merged across sources. A higher-precedence source falls back only when +the requested profile ID is absent. An invalid matching profile is an error and +does not fall back. In-memory `Profile` values follow the same ranges as YAML +profiles. They use `APIKeyRequired` for request-scoped credentials instead of +`api_key_env`. Preparation and exact profile inspection use this same source +precedence. ## Built-In Profile Catalog @@ -248,8 +252,8 @@ Every built-in selects the `openrouter` backend. The engine's built-in backend registry supplies `https://openrouter.ai/api/v1` and the environment-variable name `OPENROUTER_API_KEY`, so individual profiles contain only model and generation settings. Built-in profile files do not repeat those connection -values. A custom or in-memory profile with the same profile ID takes -precedence. +values. A configured, application fallback, or in-memory profile with the same +profile ID takes precedence. | Provider | ID | Model | | --- | --- | --- | diff --git a/docs/internal/overview.md b/docs/internal/overview.md index 5b0019d..23401a8 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -11,7 +11,7 @@ contributor workflow and validation. | Component | Implemented responsibility | References | | --- | --- | --- | -| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, typed capacity errors, public error mapping, and engine-local profile-source assembly. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) | +| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, typed capacity errors, public error mapping, and engine-local profile-source assembly including application fallbacks. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) | | `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) | | `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) | | `internal/backend` | Constructs each engine's immutable registry from the built-in OpenRouter definition and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) | diff --git a/docs/internal/sources.md b/docs/internal/sources.md index 01e52d1..1b8a049 100644 --- a/docs/internal/sources.md +++ b/docs/internal/sources.md @@ -37,9 +37,9 @@ available registry belongs to the assembled engine; the runner checks membership during preparation and exact profile inspection. The root engine assembles profile repositories in precedence order: in-memory -profiles, one ordinary configured source, then the embedded built-in catalog. -An explicit file or `fs.FS` profile source replaces `Config.ProfileDir` within -the ordinary configured-source category. +profiles, one ordinary configured source, an application fallback source, then +the embedded built-in catalog. An explicit file or `fs.FS` profile source +replaces `Config.ProfileDir` within the ordinary configured-source category. Exact profile inspection performs one point-in-time lookup through those profile sources and checks the resolved target without reading prompt, input, diff --git a/engine.go b/engine.go index f5fcaa7..0040ddc 100644 --- a/engine.go +++ b/engine.go @@ -98,9 +98,10 @@ type Config struct { // It is required unless a WithPromptFS or WithPromptFile option supplies the // prompt source. PromptDir string - // ProfileDir is an optional directory whose profiles take precedence over - // embedded built-in profiles. An empty value selects only built-ins unless - // profile options are also supplied. + // ProfileDir is an optional ordinary configured source whose profiles take + // precedence over application fallback and embedded built-in profiles. An + // empty value selects the lower-precedence sources unless a profile-source + // option supplies the ordinary source. ProfileDir string // SchemaDir is the root for JSON Schema files. An empty value uses the // current directory. WithSchemaFS or WithSchemaFile replaces this source. @@ -119,12 +120,12 @@ type Config struct { // Option customizes engine construction. // // NewEngine applies options in argument order and ignores nil options. Within -// each prompt-source, profile-source, in-memory-profile, schema-source, -// model-client, and artifact-reader category, the last non-nil valid option -// replaces earlier options in that category. WithBackend is the additive -// exception: unique registrations accumulate, and a repeated backend ID is an -// error rather than a replacement. An invalid option fails construction even -// if a later option would replace it. +// each prompt-source, ordinary-profile-source, fallback-profile-source, +// in-memory-profile, schema-source, model-client, and artifact-reader +// category, the last non-nil valid option replaces earlier options in that +// category. WithBackend is the additive exception: unique registrations +// accumulate, and a repeated backend ID is an error rather than a replacement. +// An invalid option fails construction even if a later option would replace it. type Option interface { apply(*engineOptions) error } @@ -136,18 +137,20 @@ func (f optionFunc) apply(options *engineOptions) error { } type engineOptions struct { - llmClient llm.Client - artifactReader artifactadapter.Reader - promptDefs promptdef.Repository - profiles profile.Repository - memoryProfiles profile.Repository - backends []domain.Backend - validator validate.Validator - promptSource bool - profileSource bool - memorySource bool - validatorSource bool - artifactSource bool + llmClient llm.Client + artifactReader artifactadapter.Reader + promptDefs promptdef.Repository + profiles profile.Repository + fallbackProfiles profile.Repository + memoryProfiles profile.Repository + backends []domain.Backend + validator validate.Validator + promptSource bool + profileSource bool + fallbackProfileSource bool + memorySource bool + validatorSource bool + artifactSource bool } // WithLLMClient replaces the built-in model client used by [Engine.Run] and @@ -224,12 +227,12 @@ func WithPromptFile(path string) Option { // WithProfileFS loads execution profiles from fsys under root. // -// Profiles from this source overlay built-in profiles. Profile YAML must use -// api_key_env for environment-based credentials; raw API keys are rejected. -// fsys must be non-nil and root must be non-empty; otherwise NewEngine fails -// with ErrInvalidConfig. This option replaces Config.ProfileDir and earlier -// file or FS profile-source options, but remains below WithProfiles in -// precedence. +// Profiles from this ordinary configured source take precedence over +// application fallback and built-in profiles. Profile YAML must use api_key_env +// for environment-based credentials; raw API keys are rejected. fsys must be +// non-nil and root must be non-empty; otherwise NewEngine fails with +// ErrInvalidConfig. This option replaces Config.ProfileDir and earlier file or +// FS profile-source options, but remains below WithProfiles in precedence. func WithProfileFS(fsys fs.FS, root string) Option { return optionFunc(func(options *engineOptions) error { if fsys == nil { @@ -246,11 +249,12 @@ func WithProfileFS(fsys fs.FS, root string) Option { // WithProfileFile loads execution profiles from the single profile file at path. // -// The profile overlays built-in profiles. Profile YAML must use api_key_env for -// environment-based credentials; raw API keys are rejected. path must name an -// existing non-directory file when NewEngine applies the option. This option -// replaces Config.ProfileDir and earlier file or FS profile-source options, -// but remains below WithProfiles in precedence. +// The profile takes precedence over application fallback and built-in profiles. +// Profile YAML must use api_key_env for environment-based credentials; raw API +// keys are rejected. path must name an existing non-directory file when +// NewEngine applies the option. This option replaces Config.ProfileDir and +// earlier file or FS profile-source options, but remains below WithProfiles in +// precedence. func WithProfileFile(path string) Option { return optionFunc(func(options *engineOptions) error { fsys, root, err := fileSource(path) @@ -263,8 +267,41 @@ func WithProfileFile(path string) Option { }) } +// WithFallbackProfileFS supplies application-owned fallback profile +// definitions from fsys under root. +// +// Profile lookup checks, in order, profiles supplied by WithProfiles; the +// ordinary configured source selected by WithProfileFile, WithProfileFS, or +// Config.ProfileDir; this fallback source; and Promptkit's embedded built-in +// profiles. Each source supplies a complete profile definition; profile fields +// are not merged between sources. Only an absent profile ID proceeds to the +// next source. A matching read, parse, duplicate, validation, or credential +// format failure stops resolution. +// +// Files use the ordinary strict profile YAML and api_key_env credential rules. +// Loading and validation are lazy: NewEngine validates this option's arguments +// but does not read profile files. fsys must be non-nil and root must be +// nonblank; otherwise NewEngine returns an error matching ErrInvalidConfig. +// Repeating this option replaces the earlier valid fallback source. +// +// This option controls profile-definition lookup, not provider or generation +// failover. +func WithFallbackProfileFS(fsys fs.FS, root string) Option { + return optionFunc(func(options *engineOptions) error { + if fsys == nil { + return ErrInvalidConfig + } + if strings.TrimSpace(root) == "" { + return ErrInvalidConfig + } + options.fallbackProfiles = profile.NewFSRepository(fsys, root) + options.fallbackProfileSource = true + return nil + }) +} + // WithProfiles configures in-memory profiles that take precedence over -// configured profile files and built-in profiles. +// ordinary configured, application fallback, and built-in profiles. // // NewEngine validates and copies every profile. IDs must be unique within one // call. An invalid profile, duplicate ID, or unsupported ExtraParams value @@ -406,6 +443,10 @@ func NewEngine(cfg Config, opts ...Option) (*Engine, error) { func newProfileRepository(profileDir string, options engineOptions) profile.Repository { repository := builtin.NewRepository() + if options.fallbackProfileSource { + repository = profile.NewOverlayRepository(options.fallbackProfiles, repository) + } + if options.profileSource { repository = profile.NewOverlayRepository(options.profiles, repository) } else if strings.TrimSpace(profileDir) != "" { @@ -491,10 +532,10 @@ func (e *Engine) InspectPrompt( // // InspectProfile trims surrounding whitespace from profileID and looks up the // resulting nonblank ID exactly and case-sensitively through the engine's -// ordinary in-memory, configured-source, and built-in profile precedence. It -// applies the framework timeout baseline, selected backend, and then selected -// profile to EffectiveModelParams without a request override. BackendID is -// empty for an endpoint-only profile. +// in-memory, ordinary configured-source, application fallback, and built-in +// profile precedence. It applies the framework timeout baseline, selected +// backend, and then selected profile to EffectiveModelParams without a request +// override. BackendID is empty for an endpoint-only profile. // // APIKeyEnv in the returned target is an environment-variable name, never its // value. APIKeyRequired instead reports a direct credential requirement and is diff --git a/engine_test.go b/engine_test.go index 173948f..5d91bca 100644 --- a/engine_test.go +++ b/engine_test.go @@ -2301,6 +2301,8 @@ func TestSourceOptionsRejectInvalidInputs(t *testing.T) { {name: "profile fs nil", opt: promptkit.WithProfileFS(nil, "profiles")}, {name: "profile fs empty root", opt: promptkit.WithProfileFS(fstest.MapFS{}, "")}, {name: "profile file empty", opt: promptkit.WithProfileFile("")}, + {name: "fallback profile fs nil", opt: promptkit.WithFallbackProfileFS(nil, "profiles")}, + {name: "fallback profile fs empty root", opt: promptkit.WithFallbackProfileFS(fstest.MapFS{}, "")}, {name: "schema fs nil", opt: promptkit.WithSchemaFS(nil, "schemas")}, {name: "schema fs empty root", opt: promptkit.WithSchemaFS(fstest.MapFS{}, "")}, {name: "schema file empty", opt: promptkit.WithSchemaFile("")}, diff --git a/public_contract_test.go b/public_contract_test.go index 96d4114..11b47f5 100644 --- a/public_contract_test.go +++ b/public_contract_test.go @@ -973,6 +973,24 @@ func TestRepeatedOptionsUseLastValueInEachCategory(t *testing.T) { } }) + t.Run("fallback profile source", func(t *testing.T) { + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("prompt", "profile", "message"), "."), + promptkit.WithFallbackProfileFS(contractProfileFS("profile", "first-model"), "."), + promptkit.WithFallbackProfileFS(contractProfileFS("profile", "second-model"), "."), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "prompt"}) + if err != nil { + t.Fatalf("prepare from last fallback profile source: %v", err) + } + if prepared.EffectiveModelParams.Model != "second-model" { + t.Fatalf("expected last fallback profile source, got %q", prepared.EffectiveModelParams.Model) + } + }) + t.Run("in-memory profiles", func(t *testing.T) { first := profile first.Model = "first-model" @@ -1057,6 +1075,204 @@ func TestRepeatedOptionsUseLastValueInEachCategory(t *testing.T) { }) } +func TestFallbackProfileSourcePrecedence(t *testing.T) { + const profileID = "application-profile" + + prepareModel := func(t *testing.T, engine *promptkit.Engine, promptID string) string { + t.Helper() + prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: promptID}) + if err != nil { + t.Fatalf("prepare: %v", err) + } + return prepared.EffectiveModelParams.Model + } + + t.Run("in-memory profiles override ordinary and fallback profiles", func(t *testing.T) { + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("prompt", profileID, "message"), "."), + promptkit.WithFallbackProfileFS(contractProfileFS(profileID, "fallback-model"), "."), + promptkit.WithProfileFS(contractProfileFS(profileID, "ordinary-model"), "."), + promptkit.WithProfiles(promptkit.Profile{ID: profileID, Endpoint: "http://example.test/v1", Model: "memory-model"}), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + if model := prepareModel(t, engine, "prompt"); model != "memory-model" { + t.Fatalf("expected in-memory profile, got %q", model) + } + }) + + t.Run("ordinary filesystem source overrides fallback profile", func(t *testing.T) { + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("prompt", profileID, "message"), "."), + promptkit.WithFallbackProfileFS(contractProfileFS(profileID, "fallback-model"), "."), + promptkit.WithProfileFS(contractProfileFS(profileID, "ordinary-model"), "."), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + if model := prepareModel(t, engine, "prompt"); model != "ordinary-model" { + t.Fatalf("expected ordinary profile, got %q", model) + } + }) + + t.Run("configured directory overrides fallback profile", func(t *testing.T) { + profileDir := t.TempDir() + writePublicProfileFile(t, profileDir, profileID, "http://example.test/v1", "directory-model") + engine, err := promptkit.NewEngine(promptkit.Config{ProfileDir: profileDir}, + promptkit.WithPromptFS(contractPromptFS("prompt", profileID, "message"), "."), + promptkit.WithFallbackProfileFS(contractProfileFS(profileID, "fallback-model"), "."), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + if model := prepareModel(t, engine, "prompt"); model != "directory-model" { + t.Fatalf("expected configured directory profile, got %q", model) + } + }) + + t.Run("fallback profile overrides built-in profile", func(t *testing.T) { + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("prompt", "mistral-small-3", "message"), "."), + promptkit.WithFallbackProfileFS(contractProfileFS("mistral-small-3", "fallback-model"), "."), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + if model := prepareModel(t, engine, "prompt"); model != "fallback-model" { + t.Fatalf("expected fallback profile, got %q", model) + } + }) + + t.Run("missing fallback profile uses built-in profile", func(t *testing.T) { + t.Setenv("OPENROUTER_API_KEY", "test-key") + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("prompt", "mistral-small-3", "message"), "."), + promptkit.WithFallbackProfileFS(contractProfileFS(profileID, "fallback-model"), "."), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + if model := prepareModel(t, engine, "prompt"); model != "mistralai/mistral-small-3.2-24b-instruct" { + t.Fatalf("expected built-in profile, got %q", model) + } + }) +} + +func TestFallbackProfileSourcePreservesLazyLoadingAndErrors(t *testing.T) { + const profileID = "application-profile" + + t.Run("construction defers malformed fallback profiles", func(t *testing.T) { + _, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(fstest.MapFS{}, "."), + promptkit.WithFallbackProfileFS(fstest.MapFS{ + "broken.yaml": &fstest.MapFile{Data: []byte("id: broken\nunknown: value\n")}, + }, "."), + ) + if err != nil { + t.Fatalf("construct engine with malformed fallback profile: %v", err) + } + }) + + t.Run("unrelated malformed fallback profile does not block matching definition", func(t *testing.T) { + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("prompt", profileID, "message"), "."), + promptkit.WithFallbackProfileFS(fstest.MapFS{ + "broken.yaml": &fstest.MapFile{Data: []byte("id: unrelated\nunknown: value\n")}, + "valid.yaml": &fstest.MapFile{Data: []byte("id: application-profile\nendpoint: http://example.test/v1\nmodel: fallback-model\n")}, + }, "."), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "prompt"}) + if err != nil { + t.Fatalf("prepare from valid fallback profile: %v", err) + } + if prepared.EffectiveModelParams.Model != "fallback-model" { + t.Fatalf("unexpected fallback profile model: %q", prepared.EffectiveModelParams.Model) + } + }) + + t.Run("matching malformed fallback profile does not reach built-in profile", func(t *testing.T) { + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("prompt", "mistral-small-3", "message"), "."), + promptkit.WithFallbackProfileFS(fstest.MapFS{ + "mistral-small-3.yaml": &fstest.MapFile{Data: []byte("id: mistral-small-3\nendpoint: http://example.test/v1\nmodel: fallback-model\nunknown: value\n")}, + }, "."), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + if _, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "prompt"}); !errors.Is(err, promptkit.ErrProfileLoad) { + t.Fatalf("expected ErrProfileLoad, got %v", err) + } + }) + + t.Run("matching malformed ordinary profile does not reach fallback profile", func(t *testing.T) { + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("prompt", profileID, "message"), "."), + promptkit.WithProfileFS(fstest.MapFS{ + "application-profile.yaml": &fstest.MapFile{Data: []byte("id: application-profile\nendpoint: http://example.test/v1\nmodel: ordinary-model\nunknown: value\n")}, + }, "."), + promptkit.WithFallbackProfileFS(contractProfileFS(profileID, "fallback-model"), "."), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + if _, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "prompt"}); !errors.Is(err, promptkit.ErrProfileLoad) { + t.Fatalf("expected ErrProfileLoad, got %v", err) + } + }) +} + +func TestFallbackProfileSourceWorksAcrossWorkflows(t *testing.T) { + const profileID = "application-profile" + client := &fakeLLMClient{response: &promptkit.GenerateResponse{Content: "ok"}} + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("prompt", profileID, "message"), "."), + promptkit.WithFallbackProfileFS(contractProfileFS(profileID, "fallback-model"), "."), + promptkit.WithLLMClient(client), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + + inspection, err := engine.InspectProfile(context.Background(), profileID) + if err != nil { + t.Fatalf("inspect fallback profile: %v", err) + } + prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "prompt"}) + if err != nil { + t.Fatalf("prepare fallback profile: %v", err) + } + preparedExecution, err := engine.PrepareExecution(context.Background(), promptkit.RunRequest{PromptID: "prompt"}) + if err != nil { + t.Fatalf("prepare execution with fallback profile: %v", err) + } + preparedDetails := preparedExecution.Details() + preparedResult, err := engine.RunPrepared(context.Background(), preparedExecution) + if err != nil { + t.Fatalf("run prepared fallback profile: %v", err) + } + runResult, err := engine.Run(context.Background(), promptkit.RunRequest{PromptID: "prompt"}) + if err != nil { + t.Fatalf("run fallback profile: %v", err) + } + + for name, model := range map[string]string{ + "inspection": inspection.EffectiveModelParams.Model, + "preparation": prepared.EffectiveModelParams.Model, + "prepared execution": preparedDetails.EffectiveModelParams.Model, + "prepared result": preparedResult.EffectiveModelParams.Model, + "run result": runResult.EffectiveModelParams.Model, + } { + if model != "fallback-model" { + t.Fatalf("%s model=%q, want fallback-model", name, model) + } + } +} + func TestEngineSupportsConcurrentPrepareAndRun(t *testing.T) { engine, err := promptkit.NewEngine(promptkit.Config{}, promptkit.WithPromptFS(contractPromptFS("prompt", "profile", "message"), "."),