diff --git a/docs/formats.md b/docs/formats.md index 23c7919..a371303 100644 --- a/docs/formats.md +++ b/docs/formats.md @@ -283,7 +283,8 @@ Profile sources resolve matching IDs in this order: 2. the ordinary configured source selected by a profile file, `fs.FS`, or configured profile directory; 3. application fallback profiles supplied with `WithFallbackProfileFS`; and -4. maintained external catalog profiles. +4. Promptkit-owned profiles; and +5. maintained external catalog profiles. A profile source supplies a complete definition; definitions and their fields are not merged across sources. A higher-precedence source falls back only when @@ -326,6 +327,18 @@ that backend's connection and credential metadata. Profile files do not repeat those values. A configured, application fallback, or in-memory profile with the same profile ID takes precedence. +Promptkit owns two opt-in profiles. Their following definitions are current +selections, not automatic defaults: + +| ID | Current definition | +| --- | --- | +| `promptkit-openrouter-standard` | Inherits `gemini-flash-latest`. | +| `promptkit-openrouter-auto` | Selects backend `openrouter` and model `openrouter/auto`. | + +Select either name through an existing request `profile_id` or prompt +`default_profile`. Consumers can replace either definition or its inherited +base through the ordinary profile-source precedence above. + | Provider | ID | Backend | Model | | --- | --- | --- | --- | | aion-labs | `aion-2` | `openrouter` | `aion-labs/aion-2.0` | diff --git a/docs/internal/overview.md b/docs/internal/overview.md index b0d13a8..74f13fe 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -18,7 +18,7 @@ contributor workflow and validation. | `internal/catalog` | Strictly validates imported immutable maintained backend and profile catalog assets before engine assembly uses them. | [Catalog adapter](../../internal/catalog/catalog.go), [internal sources](sources.md#profiles-and-built-ins) | | `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) | | `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, profiles, execution targets, rendering, generation, and validation, and owns source-neutral invariants for shared execution settings, OpenAI-compatible base endpoints, session identifiers, and output contracts. Source parsing, required fields, other source-specific normalization, defaulting, and boundary-specific error classification remain with their callers. | [Domain declarations](../../internal/domain/domain.go), [endpoint invariant](../../internal/domain/endpoint.go) | -| `internal/engine` | Assembles catalogs, repositories, registry, capacity, collaborators, repairer, and the one use-case runner for each engine instance. | [Engine assembly](../../internal/engine/engine.go), [internal runner](runner.md) | +| `internal/engine` | Assembles external catalogs, Promptkit-owned profiles, repositories, registry, capacity, collaborators, repairer, and the one use-case runner for each engine instance. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#profiles-and-built-ins), [internal runner](runner.md) | | `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) | | `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) | | `internal/jsonvalue` | Validates and deeply copies bounded JSON-compatible extra-parameter and prepared-schema trees while preserving supported concrete value types and rejecting cycles or excessive depth and work. | [JSON values](../../internal/jsonvalue/jsonvalue.go) | diff --git a/docs/internal/sources.md b/docs/internal/sources.md index 4bec302..9ce53ce 100644 --- a/docs/internal/sources.md +++ b/docs/internal/sources.md @@ -70,7 +70,7 @@ repositories described above. Rakestrawhome asset modules as one private adapter boundary. It enforces their manifest, layout, profile ownership, inheritance, and secret-safety rules before returning raw catalog sources. Internal engine assembly uses those validated -catalogs as the maintained lowest-precedence profile source. +catalogs as the lowest-precedence profile source. Normalized external catalog profile IDs with the exact `promptkit-` prefix are rejected at this adapter boundary. That reservation applies only to imported @@ -86,10 +86,13 @@ profile inspection. The internal engine assembles one raw composite catalog in precedence order: in-memory profiles, one ordinary configured source, an application fallback -source, then the maintained external catalog. An explicit file or `fs.FS` profile -source replaces `Config.ProfileDir` within the ordinary configured-source -category. One outer resolving repository wraps that complete raw catalog, so -each base lookup observes the same precedence and shadowing rules. +source, Promptkit-owned profiles, then the maintained external catalog. It +constructs the owned definitions through the in-memory repository and validates +their maintained-only resolution and backend membership before consumer sources +are overlaid. An explicit file or `fs.FS` profile source replaces +`Config.ProfileDir` within the ordinary configured-source category. One outer +resolving repository wraps that complete raw catalog, so each base lookup +observes the same precedence and shadowing rules. The resolving repository traverses every selected chain afresh, retains no cache, detects cycles, limits a chain to 32 profiles, merges root-to-leaf into a @@ -104,9 +107,10 @@ or schema sources. It does not retain that lookup for a later execution. Prepared execution instead freezes the fully resolved target; a later ordinary operation performs a fresh traversal. -The maintained external catalog provides every built-in profile and its -matching backend definition. Profile loading and overlay behavior are owned by the -[profile repository tests](../../internal/profile/repository_test.go), while +The imported external catalog provides external profiles and matching backend +definitions. The internal engine owns the Promptkit profiles documented in the +[framework format reference](../formats.md#built-in-profile-catalog). Profile +loading and overlay behavior are owned by the [profile repository tests](../../internal/profile/repository_test.go), while catalog completeness, backend selection, and duplicate IDs are owned by the [catalog adapter tests](../../internal/catalog/catalog_test.go). diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 0950b5a..3e6ca51 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -151,6 +151,8 @@ consumer source behavior are unchanged. ## Stage 4: Add And Compose Promptkit-Owned Profiles +**Completed:** 2026-09-22 + 1. Add a maintained-profile helper in `internal/engine` with the two raw definitions specified above. Build its memory repository, overlay it above the raw external catalog repository, and supply that raw composition to the diff --git a/internal/engine/engine.go b/internal/engine/engine.go index 1fbfdb3..e36ab08 100644 --- a/internal/engine/engine.go +++ b/internal/engine/engine.go @@ -173,7 +173,12 @@ func New(cfg Config, options Options) (*Engine, error) { return nil, fmt.Errorf("failed to load maintained catalogs: %w", err) } - profiles := newProfileRepository(cfg.ProfileDir, options, maintained.Profiles) + maintainedProfiles, err := newMaintainedProfileRepository(maintained.Profiles, maintained.Backends) + if err != nil { + return nil, fmt.Errorf("failed to construct Promptkit-maintained profiles: %w", err) + } + + profiles := newProfileRepository(cfg.ProfileDir, options, maintainedProfiles) registry, err := backend.NewRegistry(maintained.Backends, options.backends) if err != nil { return nil, fmt.Errorf("failed to construct backend registry: %w", err) diff --git a/internal/engine/maintained_profiles.go b/internal/engine/maintained_profiles.go new file mode 100644 index 0000000..619daa4 --- /dev/null +++ b/internal/engine/maintained_profiles.go @@ -0,0 +1,50 @@ +package engine + +import ( + "context" + "fmt" + + "gitea.maximumdirect.net/eric/promptkit/internal/backend" + "gitea.maximumdirect.net/eric/promptkit/internal/domain" + "gitea.maximumdirect.net/eric/promptkit/internal/profile" +) + +func maintainedProfileDefinitions() []domain.ExecutionProfile { + return []domain.ExecutionProfile{ + { + ID: "promptkit-openrouter-standard", + BaseProfileID: "gemini-flash-latest", + }, + { + ID: "promptkit-openrouter-auto", + BackendID: backend.OpenRouterID, + Model: "openrouter/auto", + }, + } +} + +func newMaintainedProfileRepository(catalogProfiles profile.Repository, maintainedBackends []domain.Backend) (profile.Repository, error) { + definitions := maintainedProfileDefinitions() + owned, err := profile.NewMemoryRepository(definitions) + if err != nil { + return nil, fmt.Errorf("create Promptkit-maintained profiles: %w", err) + } + + combined := profile.NewOverlayRepository(owned, catalogProfiles) + resolved := profile.NewResolvingRepository(combined) + registry, err := backend.NewRegistry(maintainedBackends, nil) + if err != nil { + return nil, fmt.Errorf("construct maintained backend registry: %w", err) + } + for _, definition := range definitions { + effective, err := resolved.GetProfile(context.Background(), definition.ID) + if err != nil { + return nil, fmt.Errorf("resolve Promptkit-maintained profile %q: %w", definition.ID, err) + } + if _, err := registry.GetBackend(effective.BackendID); err != nil { + return nil, fmt.Errorf("Promptkit-maintained profile %q selects unavailable backend %q: %w", definition.ID, effective.BackendID, err) + } + } + + return combined, nil +} diff --git a/internal/engine/maintained_profiles_test.go b/internal/engine/maintained_profiles_test.go new file mode 100644 index 0000000..413b78d --- /dev/null +++ b/internal/engine/maintained_profiles_test.go @@ -0,0 +1,78 @@ +package engine + +import ( + "context" + "testing" + + "gitea.maximumdirect.net/eric/promptkit/internal/backend" + "gitea.maximumdirect.net/eric/promptkit/internal/domain" + "gitea.maximumdirect.net/eric/promptkit/internal/profile" +) + +func TestNewMaintainedProfileRepository(t *testing.T) { + newCatalogProfiles := func(t *testing.T, definitions []domain.ExecutionProfile) profile.Repository { + t.Helper() + repository, err := profile.NewMemoryRepository(definitions) + if err != nil { + t.Fatalf("create catalog profiles: %v", err) + } + return repository + } + newBackend := func(id string) domain.Backend { + return domain.Backend{ID: id, Endpoint: "https://" + id + ".example/v1", APIKeyEnv: "TEST_API_KEY"} + } + + t.Run("resolves inherited and direct definitions", func(t *testing.T) { + repository, err := newMaintainedProfileRepository( + newCatalogProfiles(t, []domain.ExecutionProfile{{ + ID: "gemini-flash-latest", + BackendID: backend.OpenRouterID, + Model: "catalog-selected-model", + }}), + []domain.Backend{newBackend(backend.OpenRouterID)}, + ) + if err != nil { + t.Fatalf("create maintained profiles: %v", err) + } + + resolved := profile.NewResolvingRepository(repository) + standard, err := resolved.GetProfile(context.Background(), "promptkit-openrouter-standard") + if err != nil { + t.Fatalf("resolve inherited profile: %v", err) + } + if standard.BackendID != backend.OpenRouterID || standard.Model != "catalog-selected-model" { + t.Fatalf("unexpected inherited profile: %#v", standard) + } + + auto, err := resolved.GetProfile(context.Background(), "promptkit-openrouter-auto") + if err != nil { + t.Fatalf("resolve direct profile: %v", err) + } + definitions := maintainedProfileDefinitions() + if auto.BackendID != definitions[1].BackendID || auto.Model != definitions[1].Model { + t.Fatalf("unexpected direct profile: %#v", auto) + } + }) + + t.Run("rejects a missing inherited base", func(t *testing.T) { + if _, err := newMaintainedProfileRepository( + newCatalogProfiles(t, nil), + []domain.Backend{newBackend(backend.OpenRouterID)}, + ); err == nil { + t.Fatal("expected missing base error") + } + }) + + t.Run("rejects an unavailable effective backend", func(t *testing.T) { + if _, err := newMaintainedProfileRepository( + newCatalogProfiles(t, []domain.ExecutionProfile{{ + ID: "gemini-flash-latest", + BackendID: "other", + Model: "catalog-selected-model", + }}), + []domain.Backend{newBackend("other")}, + ); err == nil { + t.Fatal("expected unavailable backend error") + } + }) +} diff --git a/public_contract_test.go b/public_contract_test.go index 96499df..ded0a7c 100644 --- a/public_contract_test.go +++ b/public_contract_test.go @@ -1203,6 +1203,105 @@ func TestFallbackProfileSourcePrecedence(t *testing.T) { }) } +func TestPromptkitOwnedProfilesUseExistingProfileWorkflows(t *testing.T) { + const ( + standardID = "promptkit-openrouter-standard" + autoID = "promptkit-openrouter-auto" + ) + + t.Run("inspection and preparation resolve both names", func(t *testing.T) { + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("prompt", standardID, "message"), "."), + ) + if err != nil { + t.Fatalf("construct engine: %v", err) + } + + for _, profileID := range []string{standardID, autoID} { + inspection, err := engine.InspectProfile(context.Background(), profileID) + if err != nil { + t.Fatalf("inspect %q: %v", profileID, err) + } + prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{ + PromptID: "prompt", + ProfileID: profileID, + APIKey: "test-key", + }) + if err != nil { + t.Fatalf("prepare %q: %v", profileID, err) + } + if inspection.ProfileID != profileID || + inspection.EffectiveModelParams.BackendID != promptkit.BackendOpenRouter || + prepared.SelectedProfileID != profileID || + prepared.SelectedBackendID != promptkit.BackendOpenRouter || + !reflect.DeepEqual(prepared.EffectiveModelParams, inspection.EffectiveModelParams) { + t.Fatalf("inspection and preparation differ for %q: inspection=%#v prepared=%#v", profileID, inspection, prepared) + } + } + }) + + t.Run("consumer definitions override an owned profile and its base", func(t *testing.T) { + auto, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("auto", autoID, "message"), "."), + promptkit.WithProfiles(promptkit.Profile{ + ID: autoID, + Endpoint: "http://consumer.example/v1", + Model: "consumer-auto-model", + }), + ) + if err != nil { + t.Fatalf("construct override engine: %v", err) + } + autoPrepared, err := auto.Prepare(context.Background(), promptkit.RunRequest{PromptID: "auto"}) + if err != nil { + t.Fatalf("prepare overridden profile: %v", err) + } + if autoPrepared.EffectiveModelParams.Model != "consumer-auto-model" { + t.Fatalf("owned profile override model = %q", autoPrepared.EffectiveModelParams.Model) + } + + standard, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("standard", standardID, "message"), "."), + promptkit.WithProfiles(promptkit.Profile{ + ID: "gemini-flash-latest", + Endpoint: "http://consumer.example/v1", + Model: "consumer-base-model", + }), + ) + if err != nil { + t.Fatalf("construct inherited override engine: %v", err) + } + standardPrepared, err := standard.Prepare(context.Background(), promptkit.RunRequest{PromptID: "standard"}) + if err != nil { + t.Fatalf("prepare profile with overridden base: %v", err) + } + if standardPrepared.EffectiveModelParams.Model != "consumer-base-model" { + t.Fatalf("owned profile base override model = %q", standardPrepared.EffectiveModelParams.Model) + } + }) + + t.Run("prompt default selection remains explicit", func(t *testing.T) { + engine, err := promptkit.NewEngine(promptkit.Config{}, + promptkit.WithPromptFS(contractPromptFS("default", "application-default", "message"), "."), + promptkit.WithProfiles(promptkit.Profile{ + ID: "application-default", + Endpoint: "http://consumer.example/v1", + Model: "consumer-default-model", + }), + ) + if err != nil { + t.Fatalf("construct default engine: %v", err) + } + prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{PromptID: "default"}) + if err != nil { + t.Fatalf("prepare default profile: %v", err) + } + if prepared.SelectedProfileID != "application-default" { + t.Fatalf("selected default profile = %q", prepared.SelectedProfileID) + } + }) +} + func TestFallbackProfileSourcePreservesLazyLoadingAndErrors(t *testing.T) { const profileID = "application-profile"