diff --git a/README.md b/README.md index 7e9fbac..85cd191 100644 --- a/README.md +++ b/README.md @@ -9,10 +9,12 @@ gitea.maximumdirect.net/eric/promptkit Framework extraction is in progress. The repository now contains the `internal/domain` model, application-neutral `internal/defaults`, and -`internal/filecatalog` helpers that form the implementation foundation. These -internal packages are not a consumer API, and the root package does not yet -provide a usable public framework API, so there is no installation or usage -example at this time. +`internal/filecatalog` helpers that form the implementation foundation. It also +contains internal prompt-definition and profile repositories, the embedded +built-in profile catalog, and Go-template prompt rendering. These internal +packages are not a consumer API, and the root package does not yet provide a +usable public framework API, so there is no installation or usage example at +this time. Contributors should start with the [development guide](docs/development.md). The [architecture policy](docs/policy/architecture.md) defines the library diff --git a/docs/internal/overview.md b/docs/internal/overview.md index 980b946..cc2df9a 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -15,10 +15,14 @@ contributor workflow and validation. | `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, profiles, execution targets, rendering, generation, and validation. | [Domain declarations](../../internal/domain/domain.go) | | `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) | | `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) | +| `internal/promptdef` | Loads strictly decoded, validated prompt definitions from filesystem and `fs.FS` sources, including version selection and contained file-backed message content. | [Prompt-definition repository](../../internal/promptdef/filesystem_repository.go) | +| `internal/profile` | Loads strictly decoded, validated execution profiles from filesystem and `fs.FS` sources and composes repositories with error-preserving fallback. | [Profile repositories](../../internal/profile/filesystem_repository.go) | +| `internal/profile/builtin` | Embeds the built-in execution profile catalog and combines it with an optional primary repository. | [Built-in profile repository](../../internal/profile/builtin/repository.go) | +| `internal/prompt` | Renders prompt messages from Go templates with artifact, variable, session, and cache-control data. | [Go-template renderer](../../internal/prompt/go_renderer.go) | -These packages form the internal extraction foundation. Prompt and profile -sources, rendering, artifact reading, validation, model clients, orchestration, -and a usable public engine are not implemented in Promptkit yet. +These packages provide the internal model, source, and rendering foundation. +Artifact reading, output validation, model clients, orchestration, and a usable +public engine are not implemented in Promptkit yet. ## Maintenance diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index 1c4ad64..09c976d 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -16,19 +16,27 @@ The module root contains package `promptkit`, which is the public facade. It declares the module's public package boundary but does not yet provide a usable exported framework API. -The implemented internal foundation consists of: +The implemented internal components consist of: - `internal/domain`, which owns framework data values shared by later internal components; - `internal/defaults`, which owns application-neutral framework defaults and constructs the default execution target; and - `internal/filecatalog`, which discovers YAML files and provides source-path - helpers for filesystem and `fs.FS` consumers. + helpers for filesystem and `fs.FS` consumers; +- `internal/promptdef`, which loads and validates prompt definitions from + filesystem and `fs.FS` sources; +- `internal/profile`, which loads, validates, and overlays execution profiles + from filesystem and `fs.FS` sources; +- `internal/profile/builtin`, which embeds the built-in execution profile + catalog; and +- `internal/prompt`, which renders prompt messages from Go templates. -`internal/defaults` depends on `internal/domain`; the file catalog is -independent and uses only the standard library. Prompt and profile sources, -rendering, artifact reading, validation, model clients, orchestration, and the -public engine have not yet been extracted. +The defaults and renderer depend on the domain model. Prompt-definition and +profile repositories use the domain model, file catalog, and YAML decoder. The +built-in profile repository supplies an embedded `fs.FS` to the profile +package. Artifact reading, output validation, model clients, orchestration, and +the public engine have not yet been extracted. Future framework extraction must follow this dependency direction: diff --git a/go.mod b/go.mod index 5406445..83032dc 100644 --- a/go.mod +++ b/go.mod @@ -1,3 +1,5 @@ module gitea.maximumdirect.net/eric/promptkit go 1.25.5 + +require gopkg.in/yaml.v3 v3.0.1 diff --git a/go.sum b/go.sum new file mode 100644 index 0000000..a62c313 --- /dev/null +++ b/go.sum @@ -0,0 +1,4 @@ +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/internal/profile/builtin/assets/aion-labs/aion-2.yml b/internal/profile/builtin/assets/aion-labs/aion-2.yml new file mode 100644 index 0000000..785a6fd --- /dev/null +++ b/internal/profile/builtin/assets/aion-labs/aion-2.yml @@ -0,0 +1,9 @@ +id: aion-2 +endpoint: https://openrouter.ai/api/v1 +model: aion-labs/aion-2.0 +temperature: 0.72 +reasoning_effort: high +top_p: 0.95 +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/anthropic/claude-fable-latest.yml b/internal/profile/builtin/assets/anthropic/claude-fable-latest.yml new file mode 100644 index 0000000..1a1e7aa --- /dev/null +++ b/internal/profile/builtin/assets/anthropic/claude-fable-latest.yml @@ -0,0 +1,7 @@ +id: claude-fable-latest +endpoint: https://openrouter.ai/api/v1 +model: "~anthropic/claude-fable-latest" +reasoning_effort: high +timeout_seconds: 600 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/anthropic/claude-haiku-latest.yml b/internal/profile/builtin/assets/anthropic/claude-haiku-latest.yml new file mode 100644 index 0000000..f22bf15 --- /dev/null +++ b/internal/profile/builtin/assets/anthropic/claude-haiku-latest.yml @@ -0,0 +1,7 @@ +id: claude-haiku-latest +endpoint: https://openrouter.ai/api/v1 +model: "~anthropic/claude-haiku-latest" +reasoning_effort: medium +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/anthropic/claude-opus-latest.yml b/internal/profile/builtin/assets/anthropic/claude-opus-latest.yml new file mode 100644 index 0000000..c192c3a --- /dev/null +++ b/internal/profile/builtin/assets/anthropic/claude-opus-latest.yml @@ -0,0 +1,7 @@ +id: claude-opus-latest +endpoint: https://openrouter.ai/api/v1 +model: "~anthropic/claude-opus-latest" +reasoning_effort: high +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/anthropic/claude-sonnet-latest.yml b/internal/profile/builtin/assets/anthropic/claude-sonnet-latest.yml new file mode 100644 index 0000000..c7be449 --- /dev/null +++ b/internal/profile/builtin/assets/anthropic/claude-sonnet-latest.yml @@ -0,0 +1,7 @@ +id: claude-sonnet-latest +endpoint: https://openrouter.ai/api/v1 +model: "~anthropic/claude-sonnet-latest" +reasoning_effort: high +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/deepseek/deepseek-3-2.yml b/internal/profile/builtin/assets/deepseek/deepseek-3-2.yml new file mode 100644 index 0000000..1f27fbc --- /dev/null +++ b/internal/profile/builtin/assets/deepseek/deepseek-3-2.yml @@ -0,0 +1,7 @@ +id: deepseek-3-2 +endpoint: https://openrouter.ai/api/v1 +model: deepseek/deepseek-v3.2 +reasoning_effort: high +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/deepseek/deepseek-4-flash.yml b/internal/profile/builtin/assets/deepseek/deepseek-4-flash.yml new file mode 100644 index 0000000..a2edf84 --- /dev/null +++ b/internal/profile/builtin/assets/deepseek/deepseek-4-flash.yml @@ -0,0 +1,7 @@ +id: deepseek-4-flash +endpoint: https://openrouter.ai/api/v1 +model: deepseek/deepseek-v4-flash +#reasoning_effort: medium +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/deepseek/deepseek-4-pro.yml b/internal/profile/builtin/assets/deepseek/deepseek-4-pro.yml new file mode 100644 index 0000000..c7af1ff --- /dev/null +++ b/internal/profile/builtin/assets/deepseek/deepseek-4-pro.yml @@ -0,0 +1,7 @@ +id: deepseek-4-pro +endpoint: https://openrouter.ai/api/v1 +model: deepseek/deepseek-v4-pro +reasoning_effort: high +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/google/gemini-2-flash-lite.yml b/internal/profile/builtin/assets/google/gemini-2-flash-lite.yml new file mode 100644 index 0000000..e648e70 --- /dev/null +++ b/internal/profile/builtin/assets/google/gemini-2-flash-lite.yml @@ -0,0 +1,9 @@ +id: gemini-2-flash-lite +endpoint: https://openrouter.ai/api/v1 +model: "google/gemini-2.5-flash-lite" +#temperature: 0.15 +reasoning_effort: high +#top_p: 0.98 +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/google/gemini-2-flash.yml b/internal/profile/builtin/assets/google/gemini-2-flash.yml new file mode 100644 index 0000000..3b1267e --- /dev/null +++ b/internal/profile/builtin/assets/google/gemini-2-flash.yml @@ -0,0 +1,9 @@ +id: gemini-2-flash +endpoint: https://openrouter.ai/api/v1 +model: "google/gemini-2.5-flash" +#temperature: 0.15 +reasoning_effort: high +#top_p: 0.98 +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/google/gemini-2-pro.yml b/internal/profile/builtin/assets/google/gemini-2-pro.yml new file mode 100644 index 0000000..b779db6 --- /dev/null +++ b/internal/profile/builtin/assets/google/gemini-2-pro.yml @@ -0,0 +1,9 @@ +id: gemini-2-pro +endpoint: https://openrouter.ai/api/v1 +model: "google/gemini-2.5-pro" +#temperature: 0.15 +reasoning_effort: high +#top_p: 0.98 +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/google/gemini-3-flash-lite.yml b/internal/profile/builtin/assets/google/gemini-3-flash-lite.yml new file mode 100644 index 0000000..ae44f66 --- /dev/null +++ b/internal/profile/builtin/assets/google/gemini-3-flash-lite.yml @@ -0,0 +1,9 @@ +id: gemini-3-flash-lite +endpoint: https://openrouter.ai/api/v1 +model: "google/gemini-3.1-flash-lite" +#temperature: 0.15 +reasoning_effort: high +#top_p: 0.98 +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/google/gemini-flash-latest.yml b/internal/profile/builtin/assets/google/gemini-flash-latest.yml new file mode 100644 index 0000000..2bcda6e --- /dev/null +++ b/internal/profile/builtin/assets/google/gemini-flash-latest.yml @@ -0,0 +1,9 @@ +id: gemini-flash-latest +endpoint: https://openrouter.ai/api/v1 +model: "~google/gemini-flash-latest" +#temperature: 0.15 +reasoning_effort: high +#top_p: 0.98 +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/google/gemini-pro-latest.yml b/internal/profile/builtin/assets/google/gemini-pro-latest.yml new file mode 100644 index 0000000..2e77cbc --- /dev/null +++ b/internal/profile/builtin/assets/google/gemini-pro-latest.yml @@ -0,0 +1,9 @@ +id: gemini-pro-latest +endpoint: https://openrouter.ai/api/v1 +model: "~google/gemini-pro-latest" +#temperature: 0.15 +reasoning_effort: high +#top_p: 0.98 +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/google/gemma-4-31b.yml b/internal/profile/builtin/assets/google/gemma-4-31b.yml new file mode 100644 index 0000000..f8ff113 --- /dev/null +++ b/internal/profile/builtin/assets/google/gemma-4-31b.yml @@ -0,0 +1,9 @@ +id: gemma-4-31b +endpoint: https://openrouter.ai/api/v1 +model: google/gemma-4-31b-it:exacto +temperature: 0.15 +reasoning_effort: high +top_p: 0.98 +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/minimax/minimax-m2.yml b/internal/profile/builtin/assets/minimax/minimax-m2.yml new file mode 100644 index 0000000..6ce7fdf --- /dev/null +++ b/internal/profile/builtin/assets/minimax/minimax-m2.yml @@ -0,0 +1,9 @@ +id: minimax-m2 +endpoint: https://openrouter.ai/api/v1 +model: minimax/minimax-m2.5 +temperature: 0.5 +reasoning_effort: high +top_p: 0.95 +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/minimax/minimax-m3.yml b/internal/profile/builtin/assets/minimax/minimax-m3.yml new file mode 100644 index 0000000..da9a3af --- /dev/null +++ b/internal/profile/builtin/assets/minimax/minimax-m3.yml @@ -0,0 +1,9 @@ +id: minimax-m3 +endpoint: https://openrouter.ai/api/v1 +model: minimax/minimax-m3 +#temperature: 0.5 +reasoning_effort: high +#top_p: 0.95 +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/mistral/mistral-large-2512.yml b/internal/profile/builtin/assets/mistral/mistral-large-2512.yml new file mode 100644 index 0000000..106ea2e --- /dev/null +++ b/internal/profile/builtin/assets/mistral/mistral-large-2512.yml @@ -0,0 +1,7 @@ +id: mistral-large-2512 +endpoint: https://openrouter.ai/api/v1 +model: mistralai/mistral-large-2512 +temperature: 0.15 +top_p: 0.98 +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY diff --git a/internal/profile/builtin/assets/mistral/mistral-medium-3-5.yml b/internal/profile/builtin/assets/mistral/mistral-medium-3-5.yml new file mode 100644 index 0000000..762149f --- /dev/null +++ b/internal/profile/builtin/assets/mistral/mistral-medium-3-5.yml @@ -0,0 +1,8 @@ +id: mistral-medium-3-5 +endpoint: https://openrouter.ai/api/v1 +model: mistralai/mistral-medium-3-5 +temperature: 0.15 +reasoning_effort: high +top_p: 0.98 +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY diff --git a/internal/profile/builtin/assets/mistral/mistral-small-3.yml b/internal/profile/builtin/assets/mistral/mistral-small-3.yml new file mode 100644 index 0000000..d077918 --- /dev/null +++ b/internal/profile/builtin/assets/mistral/mistral-small-3.yml @@ -0,0 +1,7 @@ +id: mistral-small-3 +endpoint: https://openrouter.ai/api/v1 +model: mistralai/mistral-small-3.2-24b-instruct +temperature: 0.05 +top_p: 1.0 +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY diff --git a/internal/profile/builtin/assets/mistral/mistral-small-4.yml b/internal/profile/builtin/assets/mistral/mistral-small-4.yml new file mode 100644 index 0000000..fd0fce6 --- /dev/null +++ b/internal/profile/builtin/assets/mistral/mistral-small-4.yml @@ -0,0 +1,8 @@ +id: mistral-small-4 +endpoint: https://openrouter.ai/api/v1 +model: mistralai/mistral-small-2603 +temperature: 0.1 +reasoning_effort: high +top_p: 0.98 +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY diff --git a/internal/profile/builtin/assets/nvidia/nemotron-3-ultra.yml b/internal/profile/builtin/assets/nvidia/nemotron-3-ultra.yml new file mode 100644 index 0000000..bb55536 --- /dev/null +++ b/internal/profile/builtin/assets/nvidia/nemotron-3-ultra.yml @@ -0,0 +1,7 @@ +id: nemotron-3-ultra +endpoint: https://openrouter.ai/api/v1 +model: nvidia/nemotron-3-ultra-550b-a55b +reasoning_effort: high +timeout_seconds: 180 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/openai/gpt-5-mini.yml b/internal/profile/builtin/assets/openai/gpt-5-mini.yml new file mode 100644 index 0000000..72ea4b7 --- /dev/null +++ b/internal/profile/builtin/assets/openai/gpt-5-mini.yml @@ -0,0 +1,7 @@ +id: gpt-5-mini +endpoint: https://openrouter.ai/api/v1 +model: "openai/gpt-5.4-mini" +reasoning_effort: high +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/assets/openai/gpt-5-nano.yml b/internal/profile/builtin/assets/openai/gpt-5-nano.yml new file mode 100644 index 0000000..069a3d5 --- /dev/null +++ b/internal/profile/builtin/assets/openai/gpt-5-nano.yml @@ -0,0 +1,7 @@ +id: gpt-5-nano +endpoint: https://openrouter.ai/api/v1 +model: "openai/gpt-5.4-nano" +reasoning_effort: high +timeout_seconds: 240 +api_key_env: OPENROUTER_API_KEY +service_tier: flex diff --git a/internal/profile/builtin/repository.go b/internal/profile/builtin/repository.go new file mode 100644 index 0000000..827504f --- /dev/null +++ b/internal/profile/builtin/repository.go @@ -0,0 +1,31 @@ +package builtin + +import ( + "embed" + "strings" + + "gitea.maximumdirect.net/eric/promptkit/internal/profile" +) + +const assetRoot = "assets" + +//go:embed assets/**/*.yml +var assets embed.FS + +func NewRepository() profile.Repository { + return profile.NewFSRepository(assets, assetRoot) +} + +func NewRepositoryWithPrimary(primary profile.Repository) profile.Repository { + if primary == nil { + return NewRepository() + } + return profile.NewOverlayRepository(primary, NewRepository()) +} + +func NewRepositoryWithDirectory(dir string) profile.Repository { + if strings.TrimSpace(dir) == "" { + return NewRepository() + } + return NewRepositoryWithPrimary(profile.NewFilesystemRepository(dir)) +} diff --git a/internal/profile/builtin/repository_test.go b/internal/profile/builtin/repository_test.go new file mode 100644 index 0000000..9c825b9 --- /dev/null +++ b/internal/profile/builtin/repository_test.go @@ -0,0 +1,127 @@ +package builtin + +import ( + "context" + "errors" + "io/fs" + "strings" + "testing" + + "gitea.maximumdirect.net/eric/promptkit/internal/domain" + "gitea.maximumdirect.net/eric/promptkit/internal/profile" + "gopkg.in/yaml.v3" +) + +func TestBuiltInProfilesValidateThroughRepository(t *testing.T) { + repo := NewRepository() + ids := loadBuiltInProfileIDs(t) + if len(ids) == 0 { + t.Fatal("expected built-in profiles") + } + + for id := range ids { + t.Run(id, func(t *testing.T) { + p, err := repo.GetProfile(context.Background(), id) + if err != nil { + t.Fatalf("expected built-in profile %q to load, got %v", id, err) + } + if p.ID != id { + t.Fatalf("expected profile id %q, got %q", id, p.ID) + } + }) + } +} + +func TestBuiltInProfilesDoNotContainDuplicateIDsOrRawAPIKeys(t *testing.T) { + loadBuiltInProfileIDs(t) +} + +func loadBuiltInProfileIDs(t *testing.T) map[string]string { + t.Helper() + + ids := map[string]string{} + err := fs.WalkDir(assets, assetRoot, func(name string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + if d.IsDir() || !strings.HasSuffix(name, ".yml") { + return nil + } + + data, err := assets.ReadFile(name) + if err != nil { + t.Fatalf("failed to read built-in profile %s: %v", name, err) + } + + var raw map[string]any + if err := yaml.Unmarshal(data, &raw); err != nil { + t.Fatalf("failed to decode built-in profile %s: %v", name, err) + } + if _, ok := raw["api_key"]; ok { + t.Fatalf("built-in profile %s contains raw api_key", name) + } + id, ok := raw["id"].(string) + if !ok || strings.TrimSpace(id) == "" { + t.Fatalf("built-in profile %s has missing id", name) + } + if previous, ok := ids[id]; ok { + t.Fatalf("duplicate built-in profile id %q in %s and %s", id, previous, name) + } + ids[id] = name + return nil + }) + if err != nil { + t.Fatalf("failed to walk built-in profiles: %v", err) + } + return ids +} + +func TestRepositoryWithPrimaryUsesPrimaryBeforeBuiltIns(t *testing.T) { + repo := NewRepositoryWithPrimary(staticProfileRepo{ + profiles: map[string]string{"mistral-small-3": "custom-model"}, + }) + + p, err := repo.GetProfile(context.Background(), "mistral-small-3") + if err != nil { + t.Fatalf("expected profile to load, got %v", err) + } + if p.Model != "custom-model" { + t.Fatalf("expected primary profile to override built-in, got %+v", p) + } +} + +func TestRepositoryWithPrimaryFallsBackToBuiltIns(t *testing.T) { + repo := NewRepositoryWithPrimary(staticProfileRepo{}) + + p, err := repo.GetProfile(context.Background(), "mistral-small-3") + if err != nil { + t.Fatalf("expected built-in profile to load, got %v", err) + } + if p.ID != "mistral-small-3" { + t.Fatalf("unexpected profile: %+v", p) + } +} + +func TestRepositoryWithPrimaryDoesNotFallBackAfterPrimaryError(t *testing.T) { + repo := NewRepositoryWithPrimary(staticProfileRepo{err: profile.ErrInvalidProfile}) + + _, err := repo.GetProfile(context.Background(), "mistral-small-3") + if !errors.Is(err, profile.ErrInvalidProfile) { + t.Fatalf("expected primary error, got %v", err) + } +} + +type staticProfileRepo struct { + profiles map[string]string + err error +} + +func (r staticProfileRepo) GetProfile(_ context.Context, id string) (*domain.ExecutionProfile, error) { + if r.err != nil { + return nil, r.err + } + if model, ok := r.profiles[id]; ok { + return &domain.ExecutionProfile{ID: id, Endpoint: "http://primary/v1", Model: model}, nil + } + return nil, profile.ErrProfileNotFound +} diff --git a/internal/profile/filesystem_repository.go b/internal/profile/filesystem_repository.go new file mode 100644 index 0000000..ad3a093 --- /dev/null +++ b/internal/profile/filesystem_repository.go @@ -0,0 +1,213 @@ +package profile + +import ( + "bytes" + "context" + "errors" + "fmt" + "io/fs" + "os" + "path" + "strings" + + "gitea.maximumdirect.net/eric/promptkit/internal/domain" + "gitea.maximumdirect.net/eric/promptkit/internal/filecatalog" + "gopkg.in/yaml.v3" +) + +var ( + ErrProfileNotFound = errors.New("execution profile not found") + ErrInvalidYAML = errors.New("invalid YAML format") + ErrInvalidProfile = errors.New("invalid execution profile configuration") + ErrRawAPIKeyNotAllowed = errors.New("raw api_key is not allowed; use api_key_env") +) + +type filesystemRepository struct { + dir string +} + +func NewFilesystemRepository(dir string) Repository { + return &filesystemRepository{dir: dir} +} + +func (r *filesystemRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) { + return loadProfile(ctx, os.DirFS(r.dir), ".", id) +} + +type fsRepository struct { + fsys fs.FS + root string +} + +func NewFSRepository(fsys fs.FS, root string) Repository { + return &fsRepository{fsys: fsys, root: root} +} + +func (r *fsRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) { + return loadProfile(ctx, r.fsys, r.root, id) +} + +type overlayRepository struct { + primary Repository + fallback Repository +} + +func NewOverlayRepository(primary, fallback Repository) Repository { + return &overlayRepository{primary: primary, fallback: fallback} +} + +func (r *overlayRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) { + if r.primary != nil { + prof, err := r.primary.GetProfile(ctx, id) + if err == nil { + return prof, nil + } + if !errors.Is(err, ErrProfileNotFound) { + return nil, err + } + } + if r.fallback == nil { + return nil, ErrProfileNotFound + } + return r.fallback.GetProfile(ctx, id) +} + +func loadProfile(ctx context.Context, fsys fs.FS, root string, id string) (*domain.ExecutionProfile, error) { + if strings.TrimSpace(id) == "" { + return nil, fmt.Errorf("%w: profile id is required", ErrInvalidProfile) + } + if fsys == nil { + return nil, fmt.Errorf("failed to read profile directory: filesystem is nil") + } + + files, err := filecatalog.FindFSYAMLFiles(ctx, fsys, root) + if err != nil { + return nil, fmt.Errorf("failed to read profile directory: %w", err) + } + + var matches []profileMatch + for _, fullPath := range files { + select { + case <-ctx.Done(): + return nil, ctx.Err() + default: + } + + relPath := filecatalog.DisplayPath(root, fullPath) + fileMatch := filecatalog.Stem(path.Base(fullPath)) == id + data, err := fs.ReadFile(fsys, fullPath) + if err != nil { + return nil, fmt.Errorf("failed to read profile file %s: %w", relPath, err) + } + metadata := readProfileFileMetadata(data) + idMatch := fileMatch || metadata.id == id + if metadata.hasRawAPIKey { + if idMatch { + return nil, fmt.Errorf("%w: %s", ErrRawAPIKeyNotAllowed, relPath) + } + continue + } + + var prof domain.ExecutionProfile + decoder := yaml.NewDecoder(bytes.NewReader(data)) + decoder.KnownFields(true) + if err := decoder.Decode(&prof); err != nil { + if idMatch { + return nil, fmt.Errorf("%w: %s: %v", ErrInvalidYAML, relPath, err) + } + continue + } + + if prof.ID != id { + continue + } + if err := validateProfile(&prof); err != nil { + if errors.Is(err, ErrRawAPIKeyNotAllowed) { + return nil, fmt.Errorf("%w: %s", err, relPath) + } + return nil, fmt.Errorf("%w: %s: %v", ErrInvalidProfile, relPath, err) + } + matches = append(matches, profileMatch{ + profile: &prof, + path: relPath, + }) + } + + if len(matches) > 1 { + paths := make([]string, 0, len(matches)) + for _, match := range matches { + paths = append(paths, match.path) + } + return nil, fmt.Errorf("%w: duplicate execution profile id %q found in: %s", ErrInvalidProfile, id, strings.Join(paths, ", ")) + } + + if len(matches) == 1 { + return matches[0].profile, nil + } + + return nil, ErrProfileNotFound +} + +type profileMatch struct { + profile *domain.ExecutionProfile + path string +} + +type profileFileMetadata struct { + id string + hasRawAPIKey bool +} + +func readProfileFileMetadata(data []byte) profileFileMetadata { + var node yaml.Node + if err := yaml.NewDecoder(bytes.NewReader(data)).Decode(&node); err != nil { + return profileFileMetadata{} + } + if node.Kind != yaml.DocumentNode || len(node.Content) == 0 { + return profileFileMetadata{} + } + mapping := node.Content[0] + if mapping.Kind != yaml.MappingNode { + return profileFileMetadata{} + } + + var metadata profileFileMetadata + for i := 0; i+1 < len(mapping.Content); i += 2 { + key := mapping.Content[i] + value := mapping.Content[i+1] + switch key.Value { + case "id": + metadata.id = strings.TrimSpace(value.Value) + case "api_key": + metadata.hasRawAPIKey = true + } + } + return metadata +} + +func validateProfile(p *domain.ExecutionProfile) error { + if strings.TrimSpace(p.ID) == "" { + return errors.New("id is required") + } + if strings.TrimSpace(p.Endpoint) == "" { + return errors.New("endpoint is required") + } + if strings.TrimSpace(p.Model) == "" { + return errors.New("model is required") + } + + if p.Temperature < 0 || p.Temperature > 2 { + return errors.New("temperature must be between 0 and 2") + } + if p.MaxTokens < 0 { + return errors.New("max_tokens must be greater than or equal to 0") + } + if p.TopP < 0 || p.TopP > 1 { + return errors.New("top_p must be between 0 and 1") + } + if p.TimeoutSeconds < 0 { + return errors.New("timeout_seconds must be greater than or equal to 0") + } + + return nil +} diff --git a/internal/profile/repository.go b/internal/profile/repository.go new file mode 100644 index 0000000..af82744 --- /dev/null +++ b/internal/profile/repository.go @@ -0,0 +1,12 @@ +package profile + +import ( + "context" + + "gitea.maximumdirect.net/eric/promptkit/internal/domain" +) + +// Repository loads execution profiles. +type Repository interface { + GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) +} diff --git a/internal/profile/repository_test.go b/internal/profile/repository_test.go new file mode 100644 index 0000000..7de08a3 --- /dev/null +++ b/internal/profile/repository_test.go @@ -0,0 +1,479 @@ +package profile + +import ( + "context" + "encoding/json" + "errors" + "os" + "path/filepath" + "strings" + "testing" + "testing/fstest" + + "gitea.maximumdirect.net/eric/promptkit/internal/domain" +) + +func TestFilesystemRepository_GetProfile(t *testing.T) { + tmpDir, err := os.MkdirTemp("", "execution_profile_test") + if err != nil { + t.Fatal(err) + } + defer os.RemoveAll(tmpDir) + + files, err := os.ReadDir("testdata") + if err != nil { + t.Fatalf("failed to read testdata: %v", err) + } + for _, f := range files { + src := filepath.Join("testdata", f.Name()) + dst := filepath.Join(tmpDir, f.Name()) + data, err := os.ReadFile(src) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(dst, data, 0644); err != nil { + t.Fatal(err) + } + } + + repo := NewFilesystemRepository(tmpDir) + ctx := context.Background() + + t.Run("valid local profile", func(t *testing.T) { + p, err := repo.GetProfile(ctx, "local-default") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.ID != "local-default" { + t.Fatalf("unexpected id: %q", p.ID) + } + if p.Endpoint == "" || p.Model == "" { + t.Fatalf("expected endpoint/model to be set: %+v", p) + } + }) + + t.Run("valid profile with api_key_env", func(t *testing.T) { + p, err := repo.GetProfile(ctx, "local-secure") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.APIKeyEnv != "SCRIPTORIUM_API_KEY" { + t.Fatalf("unexpected api_key_env: %q", p.APIKeyEnv) + } + if p.ReasoningEffort != "medium" { + t.Fatalf("unexpected reasoning_effort: %q", p.ReasoningEffort) + } + if p.ServiceTier != "priority" { + t.Fatalf("unexpected service_tier: %q", p.ServiceTier) + } + }) + + t.Run("valid nested profile", func(t *testing.T) { + nestedDir := filepath.Join(tmpDir, "local") + if err := os.MkdirAll(nestedDir, 0o755); err != nil { + t.Fatal(err) + } + writeProfileTestFile(t, filepath.Join(nestedDir, "nested-local.yaml"), ` +id: nested-local +endpoint: http://localhost:8000/v1 +model: nested-model +temperature: 0.1 +`) + + p, err := repo.GetProfile(ctx, "nested-local") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.Model != "nested-model" { + t.Fatalf("unexpected model: %q", p.Model) + } + }) + + t.Run("valid profile with JSON-compatible extra params", func(t *testing.T) { + writeProfileTestFile(t, filepath.Join(tmpDir, "json-extra-params.yaml"), ` +id: json-extra-params +endpoint: http://localhost:8000/v1 +model: nested-model +extra_params: + string_value: enabled + number_value: 42 + boolean_value: true + object_value: + nested: value + count: 2 + array_value: + - first + - 3 + - false +`) + + p, err := repo.GetProfile(ctx, "json-extra-params") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + + var got map[string]any + encoded, err := json.Marshal(p.ExtraParams) + if err != nil { + t.Fatalf("expected extra_params to marshal as JSON, got %v", err) + } + if err := json.Unmarshal(encoded, &got); err != nil { + t.Fatalf("expected extra_params JSON to decode, got %v", err) + } + + if got["string_value"] != "enabled" { + t.Fatalf("unexpected string extra param: %#v", got["string_value"]) + } + if got["number_value"] != float64(42) { + t.Fatalf("unexpected number extra param: %#v", got["number_value"]) + } + if got["boolean_value"] != true { + t.Fatalf("unexpected boolean extra param: %#v", got["boolean_value"]) + } + objectValue, ok := got["object_value"].(map[string]any) + if !ok { + t.Fatalf("expected object extra param, got %#v", got["object_value"]) + } + if objectValue["nested"] != "value" || objectValue["count"] != float64(2) { + t.Fatalf("unexpected object extra param: %#v", objectValue) + } + arrayValue, ok := got["array_value"].([]any) + if !ok { + t.Fatalf("expected array extra param, got %#v", got["array_value"]) + } + if len(arrayValue) != 3 || arrayValue[0] != "first" || arrayValue[1] != float64(3) || arrayValue[2] != false { + t.Fatalf("unexpected array extra param: %#v", arrayValue) + } + }) + + t.Run("duplicate profile IDs fail as ambiguous", func(t *testing.T) { + writeProfileTestFile(t, filepath.Join(tmpDir, "duplicate-profile-a.yaml"), ` +id: duplicate-profile +endpoint: http://localhost:8000/v1 +model: first-model +`) + nestedDir := filepath.Join(tmpDir, "duplicates") + if err := os.MkdirAll(nestedDir, 0o755); err != nil { + t.Fatal(err) + } + writeProfileTestFile(t, filepath.Join(nestedDir, "duplicate-profile-b.yaml"), ` +id: duplicate-profile +endpoint: http://localhost:8000/v1 +model: second-model +`) + + _, err := repo.GetProfile(ctx, "duplicate-profile") + if !errors.Is(err, ErrInvalidProfile) { + t.Fatalf("expected duplicate profile to return ErrInvalidProfile, got %v", err) + } + for _, want := range []string{"duplicate execution profile id", "duplicate-profile-a.yaml", filepath.Join("duplicates", "duplicate-profile-b.yaml")} { + if !strings.Contains(err.Error(), want) { + t.Fatalf("expected error to contain %q, got %v", want, err) + } + } + }) + + t.Run("nested raw api_key rejected for likely target file", func(t *testing.T) { + nestedDir := filepath.Join(tmpDir, "secure") + if err := os.MkdirAll(nestedDir, 0o755); err != nil { + t.Fatal(err) + } + writeProfileTestFile(t, filepath.Join(nestedDir, "not_named_like_id.yaml"), ` +id: nested_raw_api_key +endpoint: http://localhost:8000/v1 +model: m +api_key: secret +`) + + _, err := repo.GetProfile(ctx, "nested_raw_api_key") + if !errors.Is(err, ErrRawAPIKeyNotAllowed) { + t.Fatalf("expected ErrRawAPIKeyNotAllowed, got %v", err) + } + if !strings.Contains(err.Error(), filepath.Join("secure", "not_named_like_id.yaml")) { + t.Fatalf("expected nested path in error, got %v", err) + } + }) + + t.Run("raw api_key in non-target profile is ignored", func(t *testing.T) { + writeProfileTestFile(t, filepath.Join(tmpDir, "raw-api-key-non-target.yaml"), ` +id: raw-api-key-non-target +endpoint: http://localhost:8000/v1 +model: m +api_key: secret +`) + + _, err := repo.GetProfile(ctx, "does-not-exist-with-raw-key-nearby") + if !errors.Is(err, ErrProfileNotFound) { + t.Fatalf("expected ErrProfileNotFound for non-target raw api_key file, got %v", err) + } + }) + + t.Run("invalid yaml", func(t *testing.T) { + _, err := repo.GetProfile(ctx, "invalid_yaml") + if !errors.Is(err, ErrInvalidYAML) { + t.Fatalf("expected ErrInvalidYAML, got %v", err) + } + }) + + t.Run("missing id", func(t *testing.T) { + _, err := repo.GetProfile(ctx, "missing_id") + if !errors.Is(err, ErrProfileNotFound) { + t.Fatalf("expected ErrProfileNotFound, got %v", err) + } + }) + + t.Run("missing endpoint", func(t *testing.T) { + _, err := repo.GetProfile(ctx, "missing-endpoint") + if !errors.Is(err, ErrInvalidProfile) { + t.Fatalf("expected ErrInvalidProfile, got %v", err) + } + }) + + t.Run("missing model", func(t *testing.T) { + _, err := repo.GetProfile(ctx, "missing-model") + if !errors.Is(err, ErrInvalidProfile) { + t.Fatalf("expected ErrInvalidProfile, got %v", err) + } + }) + + t.Run("unknown field", func(t *testing.T) { + _, err := repo.GetProfile(ctx, "unknown_field") + if !errors.Is(err, ErrInvalidYAML) { + t.Fatalf("expected ErrInvalidYAML for strict decode unknown field, got %v", err) + } + }) + + t.Run("raw api_key rejected", func(t *testing.T) { + _, err := repo.GetProfile(ctx, "raw_api_key") + if !errors.Is(err, ErrRawAPIKeyNotAllowed) { + t.Fatalf("expected ErrRawAPIKeyNotAllowed, got %v", err) + } + }) + + t.Run("profile not found", func(t *testing.T) { + _, err := repo.GetProfile(ctx, "does-not-exist") + if !errors.Is(err, ErrProfileNotFound) { + t.Fatalf("expected ErrProfileNotFound, got %v", err) + } + }) +} + +func writeProfileTestFile(t *testing.T, path string, content string) { + t.Helper() + if err := os.WriteFile(path, []byte(strings.TrimLeft(content, "\n")), 0o644); err != nil { + t.Fatalf("failed to write profile test file %q: %v", path, err) + } +} + +func TestFSRepository(t *testing.T) { + ctx := context.Background() + + t.Run("loads valid profiles from nested directories", func(t *testing.T) { + repo := NewFSRepository(fstest.MapFS{ + "profiles/provider/nested.yaml": profileMapFile(` +id: nested-profile +endpoint: http://localhost:8000/v1 +model: nested-model +temperature: 0.1 +`), + }, "profiles") + + p, err := repo.GetProfile(ctx, "nested-profile") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.ID != "nested-profile" || p.Model != "nested-model" { + t.Fatalf("unexpected profile: %+v", p) + } + }) + + t.Run("rejects unknown YAML fields", func(t *testing.T) { + repo := NewFSRepository(fstest.MapFS{ + "profiles/unknown.yaml": profileMapFile(` +id: unknown-profile +endpoint: http://localhost:8000/v1 +model: model +unknown: value +`), + }, "profiles") + + _, err := repo.GetProfile(ctx, "unknown-profile") + if !errors.Is(err, ErrInvalidYAML) { + t.Fatalf("expected ErrInvalidYAML, got %v", err) + } + }) + + t.Run("rejects raw api_key in selected profile", func(t *testing.T) { + repo := NewFSRepository(fstest.MapFS{ + "profiles/raw.yaml": profileMapFile(` +id: raw-profile +endpoint: http://localhost:8000/v1 +model: model +api_key: secret +`), + }, "profiles") + + _, err := repo.GetProfile(ctx, "raw-profile") + if !errors.Is(err, ErrRawAPIKeyNotAllowed) { + t.Fatalf("expected ErrRawAPIKeyNotAllowed, got %v", err) + } + }) + + t.Run("ignores raw api_key in non-selected profiles", func(t *testing.T) { + repo := NewFSRepository(fstest.MapFS{ + "profiles/raw.yaml": profileMapFile(` +id: raw-profile +endpoint: http://localhost:8000/v1 +model: model +api_key: secret +`), + "profiles/valid.yaml": profileMapFile(` +id: valid-profile +endpoint: http://localhost:8000/v1 +model: model +`), + }, "profiles") + + p, err := repo.GetProfile(ctx, "valid-profile") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.ID != "valid-profile" { + t.Fatalf("unexpected profile: %+v", p) + } + }) + + t.Run("rejects duplicate IDs within one source", func(t *testing.T) { + repo := NewFSRepository(fstest.MapFS{ + "profiles/a.yaml": profileMapFile(` +id: duplicate-profile +endpoint: http://localhost:8000/v1 +model: first +`), + "profiles/nested/b.yaml": profileMapFile(` +id: duplicate-profile +endpoint: http://localhost:8000/v1 +model: second +`), + }, "profiles") + + _, err := repo.GetProfile(ctx, "duplicate-profile") + if !errors.Is(err, ErrInvalidProfile) { + t.Fatalf("expected ErrInvalidProfile, got %v", err) + } + for _, want := range []string{"duplicate execution profile id", "a.yaml", "nested/b.yaml"} { + if !strings.Contains(err.Error(), want) { + t.Fatalf("expected error to contain %q, got %v", want, err) + } + } + }) +} + +func TestOverlayRepository(t *testing.T) { + ctx := context.Background() + primaryProfile := &domain.ExecutionProfile{ID: "shared", Endpoint: "http://primary", Model: "primary"} + fallbackProfile := &domain.ExecutionProfile{ID: "shared", Endpoint: "http://fallback", Model: "fallback"} + + t.Run("returns primary matches before fallback matches", func(t *testing.T) { + repo := NewOverlayRepository( + staticProfileRepo{profiles: map[string]*domain.ExecutionProfile{"shared": primaryProfile}}, + staticProfileRepo{profiles: map[string]*domain.ExecutionProfile{"shared": fallbackProfile}}, + ) + + p, err := repo.GetProfile(ctx, "shared") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.Model != "primary" { + t.Fatalf("expected primary profile, got %+v", p) + } + }) + + t.Run("falls back on primary not found", func(t *testing.T) { + repo := NewOverlayRepository( + staticProfileRepo{}, + staticProfileRepo{profiles: map[string]*domain.ExecutionProfile{"shared": fallbackProfile}}, + ) + + p, err := repo.GetProfile(ctx, "shared") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.Model != "fallback" { + t.Fatalf("expected fallback profile, got %+v", p) + } + }) + + t.Run("does not fall back after primary load errors", func(t *testing.T) { + for _, tc := range []struct { + name string + err error + }{ + {name: "invalid yaml", err: ErrInvalidYAML}, + {name: "invalid profile", err: ErrInvalidProfile}, + {name: "raw api key", err: ErrRawAPIKeyNotAllowed}, + } { + t.Run(tc.name, func(t *testing.T) { + repo := NewOverlayRepository( + staticProfileRepo{err: tc.err}, + staticProfileRepo{profiles: map[string]*domain.ExecutionProfile{"shared": fallbackProfile}}, + ) + + _, err := repo.GetProfile(ctx, "shared") + if !errors.Is(err, tc.err) { + t.Fatalf("expected %v, got %v", tc.err, err) + } + }) + } + }) + + t.Run("returns not found when both sources miss", func(t *testing.T) { + repo := NewOverlayRepository(staticProfileRepo{}, staticProfileRepo{}) + + _, err := repo.GetProfile(ctx, "missing") + if !errors.Is(err, ErrProfileNotFound) { + t.Fatalf("expected ErrProfileNotFound, got %v", err) + } + }) + + t.Run("nil primary uses fallback", func(t *testing.T) { + repo := NewOverlayRepository(nil, staticProfileRepo{profiles: map[string]*domain.ExecutionProfile{"shared": fallbackProfile}}) + + p, err := repo.GetProfile(ctx, "shared") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.Model != "fallback" { + t.Fatalf("expected fallback profile, got %+v", p) + } + }) + + t.Run("nil fallback returns not found after primary miss", func(t *testing.T) { + repo := NewOverlayRepository(staticProfileRepo{}, nil) + + _, err := repo.GetProfile(ctx, "missing") + if !errors.Is(err, ErrProfileNotFound) { + t.Fatalf("expected ErrProfileNotFound, got %v", err) + } + }) +} + +func profileMapFile(content string) *fstest.MapFile { + return &fstest.MapFile{Data: []byte(strings.TrimLeft(content, "\n"))} +} + +type staticProfileRepo struct { + profiles map[string]*domain.ExecutionProfile + err error +} + +func (r staticProfileRepo) GetProfile(_ context.Context, id string) (*domain.ExecutionProfile, error) { + if r.err != nil { + return nil, r.err + } + if p, ok := r.profiles[id]; ok { + cp := *p + return &cp, nil + } + return nil, ErrProfileNotFound +} diff --git a/internal/profile/testdata/invalid_yaml.yaml b/internal/profile/testdata/invalid_yaml.yaml new file mode 100644 index 0000000..6ba79f8 --- /dev/null +++ b/internal/profile/testdata/invalid_yaml.yaml @@ -0,0 +1,3 @@ +id: invalid_yaml +endpoint: http://localhost:8000/v1 +model: [broken diff --git a/internal/profile/testdata/missing_endpoint.yaml b/internal/profile/testdata/missing_endpoint.yaml new file mode 100644 index 0000000..443c8d6 --- /dev/null +++ b/internal/profile/testdata/missing_endpoint.yaml @@ -0,0 +1,2 @@ +id: missing-endpoint +model: gpt-4o-mini diff --git a/internal/profile/testdata/missing_id.yaml b/internal/profile/testdata/missing_id.yaml new file mode 100644 index 0000000..4c1421d --- /dev/null +++ b/internal/profile/testdata/missing_id.yaml @@ -0,0 +1,2 @@ +endpoint: http://localhost:8000/v1 +model: gpt-4o-mini diff --git a/internal/profile/testdata/missing_model.yaml b/internal/profile/testdata/missing_model.yaml new file mode 100644 index 0000000..838092b --- /dev/null +++ b/internal/profile/testdata/missing_model.yaml @@ -0,0 +1,2 @@ +id: missing-model +endpoint: http://localhost:8000/v1 diff --git a/internal/profile/testdata/raw_api_key.yaml b/internal/profile/testdata/raw_api_key.yaml new file mode 100644 index 0000000..a1b61ea --- /dev/null +++ b/internal/profile/testdata/raw_api_key.yaml @@ -0,0 +1,4 @@ +id: raw-api-key +endpoint: http://localhost:8000/v1 +model: gpt-4o-mini +api_key: super-secret-should-not-be-here diff --git a/internal/profile/testdata/unknown_field.yaml b/internal/profile/testdata/unknown_field.yaml new file mode 100644 index 0000000..9b1916a --- /dev/null +++ b/internal/profile/testdata/unknown_field.yaml @@ -0,0 +1,4 @@ +id: unknown-field +endpoint: http://localhost:8000/v1 +model: gpt-4o-mini +foo: bar diff --git a/internal/profile/testdata/valid_local_profile.yaml b/internal/profile/testdata/valid_local_profile.yaml new file mode 100644 index 0000000..6430a37 --- /dev/null +++ b/internal/profile/testdata/valid_local_profile.yaml @@ -0,0 +1,7 @@ +id: local-default +endpoint: http://localhost:8000/v1 +model: gpt-4o-mini +temperature: 0.2 +max_tokens: 700 +top_p: 1.0 +timeout_seconds: 120 diff --git a/internal/profile/testdata/valid_with_api_key_env.yaml b/internal/profile/testdata/valid_with_api_key_env.yaml new file mode 100644 index 0000000..8059ca0 --- /dev/null +++ b/internal/profile/testdata/valid_with_api_key_env.yaml @@ -0,0 +1,8 @@ +id: local-secure +endpoint: http://localhost:8000/v1 +model: gpt-4o-mini +api_key_env: SCRIPTORIUM_API_KEY +service_tier: priority +reasoning_effort: medium +extra_params: + provider: local diff --git a/internal/prompt/go_renderer.go b/internal/prompt/go_renderer.go new file mode 100644 index 0000000..199c8c8 --- /dev/null +++ b/internal/prompt/go_renderer.go @@ -0,0 +1,125 @@ +package prompt + +import ( + "bytes" + "context" + "errors" + "fmt" + "gitea.maximumdirect.net/eric/promptkit/internal/domain" + "strings" + "text/template" + "unicode/utf8" +) + +var ( + ErrMissingRequiredInput = errors.New("missing required input artifact") + ErrUnknownInput = errors.New("referenced unknown input artifact") + ErrInvalidTemplate = errors.New("invalid prompt template") + ErrRenderFailure = errors.New("prompt render failure") + ErrInvalidMessageRole = errors.New("invalid or empty message role") +) + +type goRenderer struct{} + +func NewGoRenderer() Renderer { + return &goRenderer{} +} + +func (r *goRenderer) Render(ctx context.Context, definition *domain.PromptDefinition, inputs map[string]*domain.Artifact, vars map[string]string) (*domain.RenderedPrompt, error) { + if definition == nil { + return nil, fmt.Errorf("%w: nil prompt definition", ErrRenderFailure) + } + + // 1. Verify required inputs + for _, in := range definition.Inputs { + if !in.Required { + continue + } + art, ok := inputs[in.Name] + if !ok || art == nil { + return nil, fmt.Errorf("%w: %s", ErrMissingRequiredInput, in.Name) + } + } + + // 2. Setup template functions + funcs := template.FuncMap{ + "input": func(name string) (string, error) { + art, ok := inputs[name] + if !ok || art == nil { + return "", fmt.Errorf("%w: %s", ErrUnknownInput, name) + } + return string(art.Body), nil + }, + } + + sessionID, err := renderSessionID(definition.SessionID, funcs, vars) + if err != nil { + return nil, err + } + + var renderedMessages []domain.RenderedMessage + + for i, tmplMsg := range definition.Templates { + select { + case <-ctx.Done(): + return nil, ctx.Err() + default: + } + + if tmplMsg.Role == "" { + return nil, fmt.Errorf("%w: message %d", ErrInvalidMessageRole, i) + } + + // Parse and execute template + tmpl, err := template.New(fmt.Sprintf("msg_%d", i)).Funcs(funcs).Option("missingkey=error").Parse(tmplMsg.Content) + if err != nil { + return nil, fmt.Errorf("%w: message %d: %v", ErrInvalidTemplate, i, err) + } + + var buf bytes.Buffer + if err := tmpl.Execute(&buf, vars); err != nil { + return nil, fmt.Errorf("%w: message %d: %w", ErrRenderFailure, i, err) + } + + renderedMessages = append(renderedMessages, domain.RenderedMessage{ + Role: tmplMsg.Role, + Content: buf.String(), + CacheControl: cloneCacheControl(tmplMsg.CacheControl), + }) + } + + return &domain.RenderedPrompt{ + SessionID: sessionID, + Messages: renderedMessages, + }, nil +} + +func renderSessionID(raw string, funcs template.FuncMap, vars map[string]string) (string, error) { + if strings.TrimSpace(raw) == "" { + return "", nil + } + + tmpl, err := template.New("session_id").Funcs(funcs).Option("missingkey=error").Parse(raw) + if err != nil { + return "", fmt.Errorf("%w: session_id: %v", ErrInvalidTemplate, err) + } + + var buf bytes.Buffer + if err := tmpl.Execute(&buf, vars); err != nil { + return "", fmt.Errorf("%w: session_id: %w", ErrRenderFailure, err) + } + + sessionID := strings.TrimSpace(buf.String()) + if n := utf8.RuneCountInString(sessionID); n > domain.SessionIDMaxLength { + return "", fmt.Errorf("%w: session_id length %d exceeds maximum %d", ErrRenderFailure, n, domain.SessionIDMaxLength) + } + return sessionID, nil +} + +func cloneCacheControl(in *domain.CacheControl) *domain.CacheControl { + if in == nil { + return nil + } + out := *in + return &out +} diff --git a/internal/prompt/renderer.go b/internal/prompt/renderer.go new file mode 100644 index 0000000..d5f09ae --- /dev/null +++ b/internal/prompt/renderer.go @@ -0,0 +1,11 @@ +package prompt + +import ( + "context" + "gitea.maximumdirect.net/eric/promptkit/internal/domain" +) + +// Renderer renders prompt templates using named artifacts and variables. +type Renderer interface { + Render(ctx context.Context, definition *domain.PromptDefinition, inputs map[string]*domain.Artifact, vars map[string]string) (*domain.RenderedPrompt, error) +} diff --git a/internal/prompt/renderer_test.go b/internal/prompt/renderer_test.go new file mode 100644 index 0000000..414fcf4 --- /dev/null +++ b/internal/prompt/renderer_test.go @@ -0,0 +1,345 @@ +package prompt + +import ( + "context" + "errors" + "strings" + "testing" + + "gitea.maximumdirect.net/eric/promptkit/internal/domain" +) + +func TestGoRenderer_Render(t *testing.T) { + renderer := NewGoRenderer() + ctx := context.Background() + + inputs := map[string]*domain.Artifact{ + "transcript": {Body: []byte("The quick brown fox.")}, + } + vars := map[string]string{ + "role": "helpful assistant", + "tone": "concise", + } + + t.Run("rendering inline message content", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "user", Content: "Analyze this: {{input \"transcript\"}}"}, + }, + } + + res, err := renderer.Render(ctx, def, inputs, vars) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if len(res.Messages) != 1 { + t.Fatalf("expected 1 message, got %d", len(res.Messages)) + } + if res.Messages[0].Content != "Analyze this: The quick brown fox." { + t.Fatalf("unexpected rendered content: %q", res.Messages[0].Content) + } + }) + + t.Run("rendering file-backed message content loaded into prompt definition", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "user", Content: "From file: {{input \"transcript\"}}", ContentFile: "/tmp/user.tmpl"}, + }, + } + + res, err := renderer.Render(ctx, def, inputs, vars) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if got := res.Messages[0].Content; got != "From file: The quick brown fox." { + t.Fatalf("unexpected file-backed render result: %q", got) + } + }) + + t.Run("rendering system and user messages", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "system", Content: "You are a {{.role}}."}, + {Role: "user", Content: "Analyze this: {{input \"transcript\"}}"}, + }, + } + + res, err := renderer.Render(ctx, def, inputs, vars) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if len(res.Messages) != 2 { + t.Fatalf("expected 2 messages, got %d", len(res.Messages)) + } + if res.Messages[0].Role != "system" || res.Messages[1].Role != "user" { + t.Fatalf("unexpected roles: %#v", res.Messages) + } + }) + + t.Run("copying cache control to rendered messages", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + { + Role: "system", + Content: "You are concise.", + CacheControl: &domain.CacheControl{ + Type: domain.CacheControlEphemeral, + TTL: "1h", + }, + }, + {Role: "user", Content: "Analyze this: {{input \"transcript\"}}"}, + }, + } + + res, err := renderer.Render(ctx, def, inputs, vars) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if len(res.Messages) != 2 { + t.Fatalf("expected 2 messages, got %d", len(res.Messages)) + } + if res.Messages[0].CacheControl == nil { + t.Fatal("expected rendered cache control") + } + if res.Messages[0].CacheControl.Type != domain.CacheControlEphemeral { + t.Fatalf("unexpected cache control type: %q", res.Messages[0].CacheControl.Type) + } + if res.Messages[0].CacheControl.TTL != "1h" { + t.Fatalf("unexpected cache control ttl: %q", res.Messages[0].CacheControl.TTL) + } + if res.Messages[1].CacheControl != nil { + t.Fatalf("expected no cache control on second message, got %#v", res.Messages[1].CacheControl) + } + }) + + t.Run("rendered cache control does not alias source template", func(t *testing.T) { + source := &domain.CacheControl{Type: domain.CacheControlEphemeral, TTL: "1h"} + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "system", Content: "You are concise.", CacheControl: source}, + }, + } + + res, err := renderer.Render(ctx, def, inputs, vars) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if res.Messages[0].CacheControl == source { + t.Fatal("expected rendered cache control to be cloned") + } + + res.Messages[0].CacheControl.TTL = "" + if source.TTL != "1h" { + t.Fatalf("source cache control was mutated, ttl=%q", source.TTL) + } + }) + + t.Run("accessing vars", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "system", Content: "Speak in a {{.tone}} tone."}, + }, + } + + res, err := renderer.Render(ctx, def, inputs, vars) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if res.Messages[0].Content != "Speak in a concise tone." { + t.Fatalf("unexpected vars rendering: %q", res.Messages[0].Content) + } + }) + + t.Run("rendering session id from vars", func(t *testing.T) { + def := &domain.PromptDefinition{ + SessionID: " {{ .session_id }} ", + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "system", Content: "Speak in a {{.tone}} tone."}, + }, + } + + res, err := renderer.Render(ctx, def, inputs, map[string]string{ + "tone": "concise", + "session_id": "agent-session-123", + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if res.SessionID != "agent-session-123" { + t.Fatalf("unexpected session id: %q", res.SessionID) + } + }) + + t.Run("empty rendered session id is omitted", func(t *testing.T) { + def := &domain.PromptDefinition{ + SessionID: " ", + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "system", Content: "Speak in a {{.tone}} tone."}, + }, + } + + res, err := renderer.Render(ctx, def, inputs, vars) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if res.SessionID != "" { + t.Fatalf("expected empty session id, got %q", res.SessionID) + } + }) + + t.Run("missing session id var fails rendering", func(t *testing.T) { + def := &domain.PromptDefinition{ + SessionID: "{{ .session_id }}", + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "system", Content: "Speak in a {{.tone}} tone."}, + }, + } + + _, err := renderer.Render(ctx, def, inputs, vars) + if !errors.Is(err, ErrRenderFailure) { + t.Fatalf("expected ErrRenderFailure, got %v", err) + } + }) + + t.Run("too long rendered session id fails rendering", func(t *testing.T) { + def := &domain.PromptDefinition{ + SessionID: "{{ .session_id }}", + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "system", Content: "Speak in a {{.tone}} tone."}, + }, + } + + _, err := renderer.Render(ctx, def, inputs, map[string]string{ + "tone": "concise", + "session_id": strings.Repeat("x", domain.SessionIDMaxLength+1), + }) + if !errors.Is(err, ErrRenderFailure) { + t.Fatalf("expected ErrRenderFailure, got %v", err) + } + }) + + t.Run("inserting required input artifact", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "user", Content: "{{input \"transcript\"}}"}, + }, + } + + res, err := renderer.Render(ctx, def, inputs, vars) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if res.Messages[0].Content != "The quick brown fox." { + t.Fatalf("unexpected required input rendering: %q", res.Messages[0].Content) + } + }) + + t.Run("optional input absent and not referenced", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{ + {Name: "transcript", Required: true}, + {Name: "glossary", Required: false}, + }, + Templates: []domain.PromptMessageTemplate{ + {Role: "user", Content: "Transcript: {{input \"transcript\"}}"}, + }, + } + + res, err := renderer.Render(ctx, def, inputs, vars) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if len(res.Messages) != 1 { + t.Fatalf("expected one rendered message, got %d", len(res.Messages)) + } + }) + + t.Run("optional input absent but referenced, expecting failure", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{ + {Name: "transcript", Required: true}, + {Name: "glossary", Required: false}, + }, + Templates: []domain.PromptMessageTemplate{ + {Role: "user", Content: "Glossary: {{input \"glossary\"}}"}, + }, + } + + _, err := renderer.Render(ctx, def, inputs, vars) + if !errors.Is(err, ErrRenderFailure) { + t.Fatalf("expected ErrRenderFailure, got %v", err) + } + if !errors.Is(err, ErrUnknownInput) { + t.Fatalf("expected ErrUnknownInput, got %v", err) + } + }) + + t.Run("required input missing, expecting failure", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "user", Content: "Analyze this: {{input \"transcript\"}}"}, + }, + } + + _, err := renderer.Render(ctx, def, map[string]*domain.Artifact{}, vars) + if !errors.Is(err, ErrMissingRequiredInput) { + t.Fatalf("expected ErrMissingRequiredInput, got %v", err) + } + }) + + t.Run("invalid template syntax", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "user", Content: "Hello {{.unclosed"}, + }, + } + + _, err := renderer.Render(ctx, def, inputs, vars) + if !errors.Is(err, ErrInvalidTemplate) { + t.Fatalf("expected ErrInvalidTemplate, got %v", err) + } + }) + + t.Run("unknown input reference", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "user", Content: "Hello {{input \"ghost\"}}"}, + }, + } + + _, err := renderer.Render(ctx, def, inputs, vars) + if !errors.Is(err, ErrRenderFailure) { + t.Fatalf("expected ErrRenderFailure, got %v", err) + } + if !errors.Is(err, ErrUnknownInput) { + t.Fatalf("expected ErrUnknownInput, got %v", err) + } + }) + + t.Run("empty message role", func(t *testing.T) { + def := &domain.PromptDefinition{ + Inputs: []domain.PromptInput{{Name: "transcript", Required: true}}, + Templates: []domain.PromptMessageTemplate{ + {Role: "", Content: "Hello"}, + }, + } + _, err := renderer.Render(ctx, def, inputs, vars) + if !errors.Is(err, ErrInvalidMessageRole) { + t.Fatalf("expected ErrInvalidMessageRole, got %v", err) + } + }) +} diff --git a/internal/promptdef/filesystem_repository.go b/internal/promptdef/filesystem_repository.go new file mode 100644 index 0000000..ad86817 --- /dev/null +++ b/internal/promptdef/filesystem_repository.go @@ -0,0 +1,484 @@ +package promptdef + +import ( + "bytes" + "context" + "errors" + "fmt" + "io/fs" + "os" + "path" + "path/filepath" + "strings" + + "gitea.maximumdirect.net/eric/promptkit/internal/domain" + "gitea.maximumdirect.net/eric/promptkit/internal/filecatalog" + "gopkg.in/yaml.v3" +) + +var ( + ErrPromptDefinitionNotFound = errors.New("prompt definition not found") + ErrInvalidYAML = errors.New("invalid YAML format") + ErrInvalidPromptDefinition = errors.New("invalid prompt definition configuration") +) + +type filesystemRepository struct { + dir string +} + +type fsRepository struct { + fsys fs.FS + root string +} + +type promptDefinitionFile struct { + ID string `yaml:"id"` + Version string `yaml:"version"` + DefaultProfile *string `yaml:"default_profile"` + Description string `yaml:"description"` + SessionID string `yaml:"session_id"` + Inputs []promptInputFile `yaml:"inputs"` + Messages []promptMessageFile `yaml:"messages"` + Output promptOutputContractFile `yaml:"output"` +} + +type promptInputFile struct { + Name string `yaml:"name"` + Required bool `yaml:"required"` + ContentType string `yaml:"content_type"` + Description string `yaml:"description"` +} + +type promptMessageFile struct { + Role string `yaml:"role"` + Content string `yaml:"content"` + ContentFile string `yaml:"content_file"` + CacheControl *cacheControlFile `yaml:"cache_control"` +} + +type cacheControlFile struct { + Type string `yaml:"type"` + TTL string `yaml:"ttl"` +} + +type promptOutputContractFile struct { + Format domain.OutputFormat `yaml:"format"` + ValidationMode domain.ValidationMode `yaml:"validation_mode"` + SchemaPath string `yaml:"schema_path"` + RepairAttempts int `yaml:"repair_attempts"` +} + +func NewFilesystemRepository(dir string) Repository { + return &filesystemRepository{dir: dir} +} + +func NewFSRepository(fsys fs.FS, root string) Repository { + return &fsRepository{fsys: fsys, root: root} +} + +func (r *filesystemRepository) GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error) { + if strings.TrimSpace(id) == "" { + return nil, fmt.Errorf("%w: prompt id is required", ErrInvalidPromptDefinition) + } + + files, err := filecatalog.FindYAMLFiles(ctx, r.dir) + if err != nil { + return nil, fmt.Errorf("failed to read prompt definition directory: %w", err) + } + + var matches []promptDefinitionMatch + for _, fullPath := range files { + select { + case <-ctx.Done(): + return nil, ctx.Err() + default: + } + + relPath := filecatalog.RelativePath(r.dir, fullPath) + fileMatch := filecatalog.Stem(filepath.Base(fullPath)) == id + + raw, err := loadPromptDefinitionFile(fullPath) + if err != nil { + if fileMatch || promptDefinitionFileHasID(fullPath, id) { + return nil, fmt.Errorf("%w: %s: %v", ErrInvalidYAML, relPath, err) + } + continue + } + + def, err := normalizePromptDefinition(raw, fullPath) + if err != nil { + if fileMatch || strings.TrimSpace(raw.ID) == id { + return nil, fmt.Errorf("%w: %s: %v", ErrInvalidPromptDefinition, relPath, err) + } + continue + } + + if def.ID != id { + continue + } + if version != "" && def.Version != version { + continue + } + matches = append(matches, promptDefinitionMatch{ + def: def, + path: relPath, + }) + } + + if len(matches) > 1 { + paths := make([]string, 0, len(matches)) + for _, match := range matches { + paths = append(paths, match.path) + } + if version != "" { + return nil, fmt.Errorf("%w: duplicate prompt definition id %q version %q found in: %s", ErrInvalidPromptDefinition, id, version, strings.Join(paths, ", ")) + } + return nil, fmt.Errorf("%w: duplicate prompt definition id %q found in: %s", ErrInvalidPromptDefinition, id, strings.Join(paths, ", ")) + } + + if len(matches) == 1 { + return matches[0].def, nil + } + + return nil, ErrPromptDefinitionNotFound +} + +func (r *fsRepository) GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error) { + return loadPromptDefinition(ctx, r.fsys, r.root, id, version) +} + +type promptDefinitionMatch struct { + def *domain.PromptDefinition + path string +} + +func loadPromptDefinitionFile(path string) (*promptDefinitionFile, error) { + data, err := os.ReadFile(path) + if err != nil { + return nil, fmt.Errorf("failed to read prompt definition file: %w", err) + } + + var raw promptDefinitionFile + decoder := yaml.NewDecoder(bytes.NewReader(data)) + decoder.KnownFields(true) + if err := decoder.Decode(&raw); err != nil { + return nil, err + } + return &raw, nil +} + +func promptDefinitionFileHasID(path string, id string) bool { + data, err := os.ReadFile(path) + if err != nil { + return false + } + var raw struct { + ID string `yaml:"id"` + } + if err := yaml.NewDecoder(bytes.NewReader(data)).Decode(&raw); err != nil { + return false + } + return strings.TrimSpace(raw.ID) == id +} + +func loadPromptDefinition(ctx context.Context, fsys fs.FS, root string, id string, version string) (*domain.PromptDefinition, error) { + if strings.TrimSpace(id) == "" { + return nil, fmt.Errorf("%w: prompt id is required", ErrInvalidPromptDefinition) + } + if fsys == nil { + return nil, fmt.Errorf("failed to read prompt definition directory: filesystem is nil") + } + + files, err := filecatalog.FindFSYAMLFiles(ctx, fsys, root) + if err != nil { + return nil, fmt.Errorf("failed to read prompt definition directory: %w", err) + } + cleanRoot := filecatalog.CleanFSRoot(root) + rootInfo, err := fs.Stat(fsys, cleanRoot) + if err != nil { + return nil, fmt.Errorf("failed to read prompt definition directory: %w", err) + } + + var matches []promptDefinitionMatch + for _, fullPath := range files { + select { + case <-ctx.Done(): + return nil, ctx.Err() + default: + } + + relPath := filecatalog.DisplayPath(root, fullPath) + fileMatch := filecatalog.Stem(path.Base(fullPath)) == id + data, err := fs.ReadFile(fsys, fullPath) + if err != nil { + if fileMatch { + return nil, fmt.Errorf("%w: %s: failed to read prompt definition file: %v", ErrInvalidYAML, relPath, err) + } + continue + } + + raw, err := decodePromptDefinition(data) + if err != nil { + if fileMatch || promptDefinitionDataHasID(data, id) { + return nil, fmt.Errorf("%w: %s: %v", ErrInvalidYAML, relPath, err) + } + continue + } + + def, err := normalizePromptDefinitionFromFS(raw, fsys, root, fullPath, rootInfo.IsDir()) + if err != nil { + if fileMatch || strings.TrimSpace(raw.ID) == id { + return nil, fmt.Errorf("%w: %s: %v", ErrInvalidPromptDefinition, relPath, err) + } + continue + } + + if def.ID != id { + continue + } + if version != "" && def.Version != version { + continue + } + matches = append(matches, promptDefinitionMatch{ + def: def, + path: relPath, + }) + } + + if len(matches) > 1 { + paths := make([]string, 0, len(matches)) + for _, match := range matches { + paths = append(paths, match.path) + } + if version != "" { + return nil, fmt.Errorf("%w: duplicate prompt definition id %q version %q found in: %s", ErrInvalidPromptDefinition, id, version, strings.Join(paths, ", ")) + } + return nil, fmt.Errorf("%w: duplicate prompt definition id %q found in: %s", ErrInvalidPromptDefinition, id, strings.Join(paths, ", ")) + } + + if len(matches) == 1 { + return matches[0].def, nil + } + + return nil, ErrPromptDefinitionNotFound +} + +func decodePromptDefinition(data []byte) (*promptDefinitionFile, error) { + var raw promptDefinitionFile + decoder := yaml.NewDecoder(bytes.NewReader(data)) + decoder.KnownFields(true) + if err := decoder.Decode(&raw); err != nil { + return nil, err + } + return &raw, nil +} + +func promptDefinitionDataHasID(data []byte, id string) bool { + var raw struct { + ID string `yaml:"id"` + } + if err := yaml.NewDecoder(bytes.NewReader(data)).Decode(&raw); err != nil { + return false + } + return strings.TrimSpace(raw.ID) == id +} + +func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*domain.PromptDefinition, error) { + promptDir := filepath.Dir(sourcePath) + return normalizePromptDefinitionWithContent(raw, func(contentFile string) (string, string, error) { + resolvedPath := strings.TrimSpace(contentFile) + if !filepath.IsAbs(resolvedPath) { + resolvedPath = filepath.Join(promptDir, resolvedPath) + } + resolvedPath = filepath.Clean(resolvedPath) + + body, err := os.ReadFile(resolvedPath) + if err != nil { + return "", "", err + } + return string(body), resolvedPath, nil + }) +} + +func normalizePromptDefinitionFromFS(raw *promptDefinitionFile, fsys fs.FS, root string, sourcePath string, rootIsDir bool) (*domain.PromptDefinition, error) { + promptDir := path.Dir(sourcePath) + return normalizePromptDefinitionWithContent(raw, func(contentFile string) (string, string, error) { + var resolvedPath string + if rootIsDir { + var err error + resolvedPath, _, err = filecatalog.ResolveFSPath(root, promptDir, contentFile) + if err != nil { + return "", "", err + } + } else { + resolvedPath = strings.TrimSpace(contentFile) + if !path.IsAbs(resolvedPath) { + resolvedPath = path.Join(promptDir, resolvedPath) + } + resolvedPath = strings.TrimPrefix(path.Clean(resolvedPath), "/") + } + + body, err := fs.ReadFile(fsys, resolvedPath) + if err != nil { + return "", "", err + } + return string(body), resolvedPath, nil + }) +} + +func normalizePromptDefinitionWithContent(raw *promptDefinitionFile, readContentFile func(string) (string, string, error)) (*domain.PromptDefinition, error) { + if raw == nil { + return nil, errors.New("prompt definition is nil") + } + + id := strings.TrimSpace(raw.ID) + if id == "" { + return nil, errors.New("id is required") + } + + version := strings.TrimSpace(raw.Version) + if version == "" { + return nil, errors.New("version is required") + } + + if len(raw.Messages) == 0 { + return nil, errors.New("at least one message is required") + } + + inputs := make([]domain.PromptInput, 0, len(raw.Inputs)) + seenInputNames := make(map[string]struct{}, len(raw.Inputs)) + for i, in := range raw.Inputs { + name := strings.TrimSpace(in.Name) + if name == "" { + return nil, fmt.Errorf("input %d has empty name", i) + } + if _, exists := seenInputNames[name]; exists { + return nil, fmt.Errorf("duplicate input name %q", name) + } + seenInputNames[name] = struct{}{} + + inputs = append(inputs, domain.PromptInput{ + Name: name, + Required: in.Required, + ContentType: strings.TrimSpace(in.ContentType), + Description: strings.TrimSpace(in.Description), + }) + } + + templates := make([]domain.PromptMessageTemplate, 0, len(raw.Messages)) + for i, msg := range raw.Messages { + role := strings.TrimSpace(msg.Role) + if role == "" { + return nil, fmt.Errorf("message %d role is required", i) + } + + hasContent := strings.TrimSpace(msg.Content) != "" + hasContentFile := strings.TrimSpace(msg.ContentFile) != "" + if hasContent == hasContentFile { + return nil, fmt.Errorf("message %d (%s) must set exactly one of content or content_file", i, role) + } + + cacheControl, err := normalizeCacheControl(msg.CacheControl) + if err != nil { + return nil, fmt.Errorf("message %d (%s) cache_control: %w", i, role, err) + } + + templateContent := msg.Content + resolvedContentFile := "" + if hasContentFile { + body, resolvedPath, err := readContentFile(msg.ContentFile) + if err != nil { + return nil, fmt.Errorf("prompt %q message %d (%s): failed to read content_file %q: %w", id, i, role, msg.ContentFile, err) + } + templateContent = body + resolvedContentFile = resolvedPath + } + + templates = append(templates, domain.PromptMessageTemplate{ + Role: role, + Content: templateContent, + ContentFile: resolvedContentFile, + CacheControl: cacheControl, + }) + } + + if !isValidOutputFormat(raw.Output.Format) { + return nil, fmt.Errorf("invalid output format: %q", raw.Output.Format) + } + if !isValidValidationMode(raw.Output.ValidationMode) { + return nil, fmt.Errorf("invalid validation mode: %q", raw.Output.ValidationMode) + } + if raw.Output.ValidationMode == domain.ValidationJSONSchema && strings.TrimSpace(raw.Output.SchemaPath) == "" { + return nil, errors.New("output.schema_path is required when output.validation_mode is json_schema") + } + if raw.Output.RepairAttempts < 0 { + return nil, errors.New("output.repair_attempts must be greater than or equal to 0") + } + + defaultProfile := "" + if raw.DefaultProfile != nil { + defaultProfile = strings.TrimSpace(*raw.DefaultProfile) + if defaultProfile == "" { + return nil, errors.New("default_profile must be a non-empty string when set") + } + } + + return &domain.PromptDefinition{ + ID: id, + Version: version, + DefaultProfile: defaultProfile, + Description: strings.TrimSpace(raw.Description), + SessionID: strings.TrimSpace(raw.SessionID), + Inputs: inputs, + Templates: templates, + OutputFormat: raw.Output.Format, + Validation: domain.OutputContract{ + Format: raw.Output.Format, + ValidationMode: raw.Output.ValidationMode, + SchemaPath: strings.TrimSpace(raw.Output.SchemaPath), + RepairAttempts: raw.Output.RepairAttempts, + }, + }, nil +} + +func normalizeCacheControl(raw *cacheControlFile) (*domain.CacheControl, error) { + if raw == nil { + return nil, nil + } + + cacheType := strings.TrimSpace(raw.Type) + if cacheType == "" { + return nil, errors.New("type is required") + } + if domain.CacheControlType(cacheType) != domain.CacheControlEphemeral { + return nil, fmt.Errorf("unsupported type %q", cacheType) + } + + ttl := strings.TrimSpace(raw.TTL) + if ttl != "" && ttl != "1h" { + return nil, fmt.Errorf("unsupported ttl %q", ttl) + } + + return &domain.CacheControl{ + Type: domain.CacheControlType(cacheType), + TTL: ttl, + }, nil +} + +func isValidOutputFormat(f domain.OutputFormat) bool { + switch f { + case domain.FormatText, domain.FormatMarkdown, domain.FormatJSON: + return true + default: + return false + } +} + +func isValidValidationMode(m domain.ValidationMode) bool { + switch m { + case domain.ValidationNone, domain.ValidationBasic, domain.ValidationJSON, domain.ValidationJSONSchema: + return true + default: + return false + } +} diff --git a/internal/promptdef/repository.go b/internal/promptdef/repository.go new file mode 100644 index 0000000..b1241d4 --- /dev/null +++ b/internal/promptdef/repository.go @@ -0,0 +1,12 @@ +package promptdef + +import ( + "context" + + "gitea.maximumdirect.net/eric/promptkit/internal/domain" +) + +// Repository loads prompt definitions. +type Repository interface { + GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error) +} diff --git a/internal/promptdef/repository_test.go b/internal/promptdef/repository_test.go new file mode 100644 index 0000000..d957326 --- /dev/null +++ b/internal/promptdef/repository_test.go @@ -0,0 +1,526 @@ +package promptdef + +import ( + "context" + "errors" + "io/fs" + "os" + "path/filepath" + "strings" + "testing" + "testing/fstest" + + "gitea.maximumdirect.net/eric/promptkit/internal/domain" +) + +func TestFilesystemRepository_GetPromptDefinition(t *testing.T) { + tmpDir := t.TempDir() + if err := copyTree("testdata", tmpDir); err != nil { + t.Fatalf("failed to copy testdata: %v", err) + } + + repo := NewFilesystemRepository(tmpDir) + ctx := context.Background() + + t.Run("valid inline prompt", func(t *testing.T) { + p, err := repo.GetPromptDefinition(ctx, "valid-inline", "") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.ID != "valid-inline" { + t.Fatalf("unexpected id: %q", p.ID) + } + if p.Version != "1.0.0" { + t.Fatalf("unexpected version: %q", p.Version) + } + if p.OutputFormat != domain.FormatMarkdown { + t.Fatalf("unexpected output format: %q", p.OutputFormat) + } + if p.Validation.ValidationMode != domain.ValidationBasic { + t.Fatalf("unexpected validation mode: %q", p.Validation.ValidationMode) + } + if len(p.Templates) != 2 { + t.Fatalf("expected 2 messages, got %d", len(p.Templates)) + } + if len(p.Inputs) != 1 { + t.Fatalf("expected 1 input, got %d", len(p.Inputs)) + } + if p.Inputs[0].ContentType != "text/markdown" { + t.Fatalf("expected input content_type to be preserved, got %q", p.Inputs[0].ContentType) + } + }) + + t.Run("valid file-backed prompt", func(t *testing.T) { + p, err := repo.GetPromptDefinition(ctx, "valid-file-backed", "") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if len(p.Templates) != 2 { + t.Fatalf("expected 2 messages, got %d", len(p.Templates)) + } + if !strings.Contains(p.Templates[1].Content, "{{input \"transcript\"}}") { + t.Fatalf("expected content_file template body to be loaded, got %q", p.Templates[1].Content) + } + if p.Templates[1].ContentFile == "" { + t.Fatal("expected ContentFile source metadata to be preserved") + } + if !filepath.IsAbs(p.Templates[1].ContentFile) { + t.Fatalf("expected resolved content_file path to be absolute, got %q", p.Templates[1].ContentFile) + } + }) + + t.Run("valid cache control with ttl", func(t *testing.T) { + p, err := repo.GetPromptDefinition(ctx, "valid-cache-control-ttl", "") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if len(p.Templates) != 2 { + t.Fatalf("expected 2 messages, got %d", len(p.Templates)) + } + assertCacheControl(t, p.Templates[0].CacheControl, domain.CacheControlEphemeral, "1h") + if p.Templates[1].CacheControl != nil { + t.Fatalf("expected second message cache control to be nil, got %#v", p.Templates[1].CacheControl) + } + }) + + t.Run("valid cache control without ttl", func(t *testing.T) { + p, err := repo.GetPromptDefinition(ctx, "valid-cache-control-without-ttl", "") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if len(p.Templates) != 2 { + t.Fatalf("expected 2 messages, got %d", len(p.Templates)) + } + assertCacheControl(t, p.Templates[0].CacheControl, domain.CacheControlEphemeral, "") + if p.Templates[1].CacheControl != nil { + t.Fatalf("expected second message cache control to be nil, got %#v", p.Templates[1].CacheControl) + } + }) + + t.Run("valid session id template", func(t *testing.T) { + p, err := repo.GetPromptDefinition(ctx, "valid-session-id", "") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.SessionID != "{{ .session_id }}" { + t.Fatalf("expected trimmed session_id template, got %q", p.SessionID) + } + }) + + t.Run("valid nested file-backed prompt resolves content file relative to nested YAML", func(t *testing.T) { + nestedDir := filepath.Join(tmpDir, "dnd", "recap") + if err := os.MkdirAll(nestedDir, 0o755); err != nil { + t.Fatal(err) + } + writePromptTestFile(t, filepath.Join(nestedDir, "nested_recap.yaml"), ` +id: nested-recap +version: "1.0.0" +messages: + - role: user + content_file: ./nested_recap.user.tmpl +output: + format: markdown + validation_mode: basic + repair_attempts: 0 +`) + writePromptTestFile(t, filepath.Join(nestedDir, "nested_recap.user.tmpl"), `Nested recap: {{input "transcript"}}`) + + p, err := repo.GetPromptDefinition(ctx, "nested-recap", "") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if len(p.Templates) != 1 { + t.Fatalf("expected one template, got %d", len(p.Templates)) + } + if !strings.Contains(p.Templates[0].Content, "Nested recap") { + t.Fatalf("expected nested content file body, got %q", p.Templates[0].Content) + } + if !strings.Contains(p.Templates[0].ContentFile, filepath.Join("dnd", "recap", "nested_recap.user.tmpl")) { + t.Fatalf("expected nested content file path, got %q", p.Templates[0].ContentFile) + } + }) + + t.Run("prompt with default_profile", func(t *testing.T) { + p, err := repo.GetPromptDefinition(ctx, "with-default-profile", "") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if p.DefaultProfile != "local-default" { + t.Fatalf("unexpected default profile: %q", p.DefaultProfile) + } + if len(p.Inputs) != 1 { + t.Fatalf("expected one input, got %d", len(p.Inputs)) + } + if p.Inputs[0].ContentType != "" { + t.Fatalf("expected missing content_type to remain empty, got %q", p.Inputs[0].ContentType) + } + }) + + t.Run("duplicate prompt IDs fail as ambiguous", func(t *testing.T) { + writePromptTestFile(t, filepath.Join(tmpDir, "duplicate_a.yaml"), ` +id: duplicate-prompt +version: "1.0.0" +messages: + - role: user + content: First duplicate. +output: + format: markdown + validation_mode: basic + repair_attempts: 0 +`) + nestedDir := filepath.Join(tmpDir, "nested") + if err := os.MkdirAll(nestedDir, 0o755); err != nil { + t.Fatal(err) + } + writePromptTestFile(t, filepath.Join(nestedDir, "duplicate_b.yaml"), ` +id: duplicate-prompt +version: "2.0.0" +messages: + - role: user + content: Second duplicate. +output: + format: markdown + validation_mode: basic + repair_attempts: 0 +`) + + _, err := repo.GetPromptDefinition(ctx, "duplicate-prompt", "") + if !errors.Is(err, ErrInvalidPromptDefinition) { + t.Fatalf("expected duplicate prompt to return ErrInvalidPromptDefinition, got %v", err) + } + for _, want := range []string{"duplicate prompt definition id", "duplicate_a.yaml", filepath.Join("nested", "duplicate_b.yaml")} { + if !strings.Contains(err.Error(), want) { + t.Fatalf("expected error to contain %q, got %v", want, err) + } + } + }) + + t.Run("duplicate prompt ID and requested version fails as ambiguous", func(t *testing.T) { + writePromptTestFile(t, filepath.Join(tmpDir, "version_duplicate_a.yaml"), ` +id: duplicate-version-prompt +version: "1.0.0" +messages: + - role: user + content: First duplicate version. +output: + format: markdown + validation_mode: basic + repair_attempts: 0 +`) + nestedDir := filepath.Join(tmpDir, "versioned") + if err := os.MkdirAll(nestedDir, 0o755); err != nil { + t.Fatal(err) + } + writePromptTestFile(t, filepath.Join(nestedDir, "version_duplicate_b.yaml"), ` +id: duplicate-version-prompt +version: "1.0.0" +messages: + - role: user + content: Second duplicate version. +output: + format: markdown + validation_mode: basic + repair_attempts: 0 +`) + + _, err := repo.GetPromptDefinition(ctx, "duplicate-version-prompt", "1.0.0") + if !errors.Is(err, ErrInvalidPromptDefinition) { + t.Fatalf("expected duplicate prompt version to return ErrInvalidPromptDefinition, got %v", err) + } + for _, want := range []string{"duplicate prompt definition id", "version \"1.0.0\"", "version_duplicate_a.yaml", filepath.Join("versioned", "version_duplicate_b.yaml")} { + if !strings.Contains(err.Error(), want) { + t.Fatalf("expected error to contain %q, got %v", want, err) + } + } + }) + + t.Run("non-matching malformed nested prompt is ignored for not found lookup", func(t *testing.T) { + nestedDir := filepath.Join(tmpDir, "broken") + if err := os.MkdirAll(nestedDir, 0o755); err != nil { + t.Fatal(err) + } + writePromptTestFile(t, filepath.Join(nestedDir, "unrelated.yaml"), "id: [") + + _, err := repo.GetPromptDefinition(ctx, "does-not-exist-even-with-broken-nested-file", "") + if !errors.Is(err, ErrPromptDefinitionNotFound) { + t.Fatalf("expected ErrPromptDefinitionNotFound, got %v", err) + } + }) + + t.Run("strict decode failure in nested prompt matches by YAML ID", func(t *testing.T) { + nestedDir := filepath.Join(tmpDir, "strict") + if err := os.MkdirAll(nestedDir, 0o755); err != nil { + t.Fatal(err) + } + writePromptTestFile(t, filepath.Join(nestedDir, "not_named_like_id.yaml"), ` +id: nested-strict-error +version: "1.0.0" +unknown_field: true +messages: + - role: user + content: Invalid because of unknown field. +output: + format: markdown + validation_mode: basic + repair_attempts: 0 +`) + + _, err := repo.GetPromptDefinition(ctx, "nested-strict-error", "") + if !errors.Is(err, ErrInvalidYAML) { + t.Fatalf("expected ErrInvalidYAML, got %v", err) + } + if !strings.Contains(err.Error(), filepath.Join("strict", "not_named_like_id.yaml")) { + t.Fatalf("expected nested path in error, got %v", err) + } + }) + + t.Run("version lookup", func(t *testing.T) { + _, err := repo.GetPromptDefinition(ctx, "valid-inline", "9.9.9") + if !errors.Is(err, ErrPromptDefinitionNotFound) { + t.Fatalf("expected ErrPromptDefinitionNotFound, got %v", err) + } + }) + + cases := []struct { + name string + id string + targetErr error + errSubstrs []string + }{ + {name: "invalid YAML", id: "invalid_yaml", targetErr: ErrInvalidYAML}, + {name: "missing id", id: "missing_id", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"id is required"}}, + {name: "no messages", id: "no_messages", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"at least one message is required"}}, + {name: "both content and content_file", id: "both_content_and_content_file", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"exactly one"}}, + {name: "neither content nor content_file", id: "neither_content_nor_content_file", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"exactly one"}}, + {name: "missing content_file", id: "missing_content_file", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"failed to read content_file"}}, + {name: "duplicate input names", id: "duplicate_input_names", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"duplicate input name"}}, + {name: "invalid validation mode", id: "invalid_validation_mode", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"invalid validation mode"}}, + {name: "json_schema without schema_path", id: "json_schema_without_schema_path", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"schema_path"}}, + {name: "unknown input field", id: "unknown_input_field", targetErr: ErrInvalidYAML, errSubstrs: []string{"field unknown_input_setting not found"}}, + {name: "empty cache control type", id: "empty_cache_control_type", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"cache_control", "type is required"}}, + {name: "unsupported cache control type", id: "unsupported_cache_control_type", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"cache_control", "unsupported type"}}, + {name: "unsupported cache control ttl", id: "unsupported_cache_control_ttl", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"cache_control", "unsupported ttl"}}, + {name: "unknown cache control field", id: "unknown_cache_control_field", targetErr: ErrInvalidYAML, errSubstrs: []string{"field unexpected not found"}}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + _, err := repo.GetPromptDefinition(ctx, tc.id, "") + if !errors.Is(err, tc.targetErr) { + t.Fatalf("expected %v, got %v", tc.targetErr, err) + } + for _, sub := range tc.errSubstrs { + if !strings.Contains(err.Error(), sub) { + t.Fatalf("expected error to contain %q, got %v", sub, err) + } + } + }) + } + + t.Run("prompt definition not found", func(t *testing.T) { + _, err := repo.GetPromptDefinition(ctx, "does-not-exist", "") + if !errors.Is(err, ErrPromptDefinitionNotFound) { + t.Fatalf("expected ErrPromptDefinitionNotFound, got %v", err) + } + }) +} + +func TestFSRepositoryGetPromptDefinition(t *testing.T) { + repo := NewFSRepository(fstest.MapFS{ + "prompts/nested/prompt.yaml": &fstest.MapFile{Data: []byte(` +id: fs-prompt +version: "1.0.0" +inputs: + - name: transcript + required: true +messages: + - role: user + content_file: ./messages/user.tmpl +output: + format: markdown + validation_mode: basic + repair_attempts: 0 +`)}, + "prompts/nested/messages/user.tmpl": &fstest.MapFile{Data: []byte(`Summarize {{input "transcript"}}.`)}, + }, "prompts") + + got, err := repo.GetPromptDefinition(context.Background(), "fs-prompt", "") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if got.ID != "fs-prompt" { + t.Fatalf("unexpected prompt id: %q", got.ID) + } + if len(got.Templates) != 1 || !strings.Contains(got.Templates[0].Content, `{{input "transcript"}}`) { + t.Fatalf("expected content_file body to be loaded, got %+v", got.Templates) + } + if got.Templates[0].ContentFile != "prompts/nested/messages/user.tmpl" { + t.Fatalf("unexpected content file path: %q", got.Templates[0].ContentFile) + } +} + +func TestFSRepositoryContentFileContainment(t *testing.T) { + t.Run("nested prompt can reference file inside root", func(t *testing.T) { + repo := NewFSRepository(fstest.MapFS{ + "prompts/nested/prompt.yaml": &fstest.MapFile{Data: []byte(` +id: fs-contained-prompt +version: "1.0.0" +messages: + - role: user + content_file: ../shared/user.tmpl +output: + format: markdown + validation_mode: basic + repair_attempts: 0 +`)}, + "prompts/shared/user.tmpl": &fstest.MapFile{Data: []byte(`Inside root.`)}, + }, "prompts") + + got, err := repo.GetPromptDefinition(context.Background(), "fs-contained-prompt", "") + if err != nil { + t.Fatalf("expected no error, got %v", err) + } + if len(got.Templates) != 1 || got.Templates[0].Content != "Inside root." { + t.Fatalf("expected contained content file, got %+v", got.Templates) + } + }) + + tests := []struct { + name string + contentFile string + wantErr string + }{ + {name: "parent escape rejected", contentFile: "../outside.tmpl", wantErr: "escapes source root"}, + {name: "absolute path rejected", contentFile: "/outside.tmpl", wantErr: "must be relative"}, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + repo := NewFSRepository(fstest.MapFS{ + "prompts/prompt.yaml": &fstest.MapFile{Data: []byte(` +id: fs-escaped-prompt +version: "1.0.0" +messages: + - role: user + content_file: ` + tc.contentFile + ` +output: + format: markdown + validation_mode: basic + repair_attempts: 0 +`)}, + "outside.tmpl": &fstest.MapFile{Data: []byte(`Outside root.`)}, + }, "prompts") + + _, err := repo.GetPromptDefinition(context.Background(), "fs-escaped-prompt", "") + if !errors.Is(err, ErrInvalidPromptDefinition) { + t.Fatalf("expected ErrInvalidPromptDefinition, got %v", err) + } + if !strings.Contains(err.Error(), tc.wantErr) { + t.Fatalf("expected error to contain %q, got %v", tc.wantErr, err) + } + }) + } +} + +func TestFSRepositoryRejectsDuplicatePromptIDs(t *testing.T) { + repo := NewFSRepository(fstest.MapFS{ + "one.yaml": &fstest.MapFile{Data: []byte(` +id: duplicate-fs-prompt +version: "1.0.0" +messages: + - role: user + content: First. +output: + format: text + validation_mode: none + repair_attempts: 0 +`)}, + "nested/two.yaml": &fstest.MapFile{Data: []byte(` +id: duplicate-fs-prompt +version: "1.0.0" +messages: + - role: user + content: Second. +output: + format: text + validation_mode: none + repair_attempts: 0 +`)}, + }, ".") + + _, err := repo.GetPromptDefinition(context.Background(), "duplicate-fs-prompt", "") + if !errors.Is(err, ErrInvalidPromptDefinition) { + t.Fatalf("expected ErrInvalidPromptDefinition, got %v", err) + } + if !strings.Contains(err.Error(), "one.yaml") || !strings.Contains(err.Error(), "nested/two.yaml") { + t.Fatalf("expected duplicate paths in error, got %v", err) + } +} + +func TestFSRepositoryRejectsUnknownYAMLFields(t *testing.T) { + repo := NewFSRepository(fstest.MapFS{ + "not_named_like_id.yaml": &fstest.MapFile{Data: []byte(` +id: strict-fs-prompt +version: "1.0.0" +unknown: true +messages: + - role: user + content: Invalid. +output: + format: text + validation_mode: none + repair_attempts: 0 +`)}, + }, ".") + + _, err := repo.GetPromptDefinition(context.Background(), "strict-fs-prompt", "") + if !errors.Is(err, ErrInvalidYAML) { + t.Fatalf("expected ErrInvalidYAML, got %v", err) + } +} + +func assertCacheControl(t *testing.T, got *domain.CacheControl, wantType domain.CacheControlType, wantTTL string) { + t.Helper() + if got == nil { + t.Fatal("expected cache control, got nil") + } + if got.Type != wantType { + t.Fatalf("unexpected cache control type: got %q want %q", got.Type, wantType) + } + if got.TTL != wantTTL { + t.Fatalf("unexpected cache control ttl: got %q want %q", got.TTL, wantTTL) + } +} + +func writePromptTestFile(t *testing.T, path string, content string) { + t.Helper() + if err := os.WriteFile(path, []byte(strings.TrimLeft(content, "\n")), 0o644); err != nil { + t.Fatalf("failed to write prompt test file %q: %v", path, err) + } +} + +func copyTree(src, dst string) error { + return filepath.WalkDir(src, func(path string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + rel, err := filepath.Rel(src, path) + if err != nil { + return err + } + if rel == "." { + return nil + } + + target := filepath.Join(dst, rel) + if d.IsDir() { + return os.MkdirAll(target, 0o755) + } + + data, err := os.ReadFile(path) + if err != nil { + return err + } + return os.WriteFile(target, data, 0o644) + }) +} diff --git a/internal/promptdef/testdata/both_content_and_content_file.yaml b/internal/promptdef/testdata/both_content_and_content_file.yaml new file mode 100644 index 0000000..de11155 --- /dev/null +++ b/internal/promptdef/testdata/both_content_and_content_file.yaml @@ -0,0 +1,10 @@ +id: both-content-and-content-file +version: "1.0.0" +messages: + - role: user + content: "Hi" + content_file: ./messages/user_prompt.tmpl +output: + format: text + validation_mode: none + repair_attempts: 0 diff --git a/internal/promptdef/testdata/duplicate_input_names.yaml b/internal/promptdef/testdata/duplicate_input_names.yaml new file mode 100644 index 0000000..9412e4d --- /dev/null +++ b/internal/promptdef/testdata/duplicate_input_names.yaml @@ -0,0 +1,14 @@ +id: duplicate-input-names +version: "1.0.0" +inputs: + - name: transcript + required: true + - name: transcript + required: false +messages: + - role: user + content: "Hi" +output: + format: text + validation_mode: none + repair_attempts: 0 diff --git a/internal/promptdef/testdata/empty_cache_control_type.yaml b/internal/promptdef/testdata/empty_cache_control_type.yaml new file mode 100644 index 0000000..03330ec --- /dev/null +++ b/internal/promptdef/testdata/empty_cache_control_type.yaml @@ -0,0 +1,10 @@ +id: empty-cache-control-type +version: "1.0.0" +messages: + - role: system + content: "Use cached instructions." + cache_control: {} +output: + format: markdown + validation_mode: basic + repair_attempts: 0 diff --git a/internal/promptdef/testdata/invalid_validation_mode.yaml b/internal/promptdef/testdata/invalid_validation_mode.yaml new file mode 100644 index 0000000..478abc5 --- /dev/null +++ b/internal/promptdef/testdata/invalid_validation_mode.yaml @@ -0,0 +1,9 @@ +id: invalid-validation-mode +version: "1.0.0" +messages: + - role: user + content: "Hi" +output: + format: text + validation_mode: nope + repair_attempts: 0 diff --git a/internal/promptdef/testdata/invalid_yaml.yaml b/internal/promptdef/testdata/invalid_yaml.yaml new file mode 100644 index 0000000..8cc64e8 --- /dev/null +++ b/internal/promptdef/testdata/invalid_yaml.yaml @@ -0,0 +1,9 @@ +id: invalid-yaml +version: "1.0.0" +messages: + - role: user + content: [broken +output: + format: text + validation_mode: none + repair_attempts: 0 diff --git a/internal/promptdef/testdata/json_schema_without_schema_path.yaml b/internal/promptdef/testdata/json_schema_without_schema_path.yaml new file mode 100644 index 0000000..6dbea5b --- /dev/null +++ b/internal/promptdef/testdata/json_schema_without_schema_path.yaml @@ -0,0 +1,9 @@ +id: json-schema-without-schema-path +version: "1.0.0" +messages: + - role: user + content: "Return JSON" +output: + format: json + validation_mode: json_schema + repair_attempts: 0 diff --git a/internal/promptdef/testdata/messages/user_prompt.tmpl b/internal/promptdef/testdata/messages/user_prompt.tmpl new file mode 100644 index 0000000..c8fc459 --- /dev/null +++ b/internal/promptdef/testdata/messages/user_prompt.tmpl @@ -0,0 +1,2 @@ +Use transcript: +{{input "transcript"}} diff --git a/internal/promptdef/testdata/missing_content_file.yaml b/internal/promptdef/testdata/missing_content_file.yaml new file mode 100644 index 0000000..42e44fa --- /dev/null +++ b/internal/promptdef/testdata/missing_content_file.yaml @@ -0,0 +1,9 @@ +id: missing-content-file +version: "1.0.0" +messages: + - role: user + content_file: ./messages/does_not_exist.tmpl +output: + format: text + validation_mode: none + repair_attempts: 0 diff --git a/internal/promptdef/testdata/missing_id.yaml b/internal/promptdef/testdata/missing_id.yaml new file mode 100644 index 0000000..6d21ccc --- /dev/null +++ b/internal/promptdef/testdata/missing_id.yaml @@ -0,0 +1,8 @@ +version: "1.0.0" +messages: + - role: user + content: "Hi" +output: + format: text + validation_mode: none + repair_attempts: 0 diff --git a/internal/promptdef/testdata/neither_content_nor_content_file.yaml b/internal/promptdef/testdata/neither_content_nor_content_file.yaml new file mode 100644 index 0000000..525419b --- /dev/null +++ b/internal/promptdef/testdata/neither_content_nor_content_file.yaml @@ -0,0 +1,8 @@ +id: neither-content-nor-content-file +version: "1.0.0" +messages: + - role: user +output: + format: text + validation_mode: none + repair_attempts: 0 diff --git a/internal/promptdef/testdata/no_messages.yaml b/internal/promptdef/testdata/no_messages.yaml new file mode 100644 index 0000000..b4b2bbf --- /dev/null +++ b/internal/promptdef/testdata/no_messages.yaml @@ -0,0 +1,6 @@ +id: no-messages +version: "1.0.0" +output: + format: text + validation_mode: none + repair_attempts: 0 diff --git a/internal/promptdef/testdata/unknown_cache_control_field.yaml b/internal/promptdef/testdata/unknown_cache_control_field.yaml new file mode 100644 index 0000000..ac8193a --- /dev/null +++ b/internal/promptdef/testdata/unknown_cache_control_field.yaml @@ -0,0 +1,12 @@ +id: unknown-cache-control-field +version: "1.0.0" +messages: + - role: system + content: "Use cached instructions." + cache_control: + type: ephemeral + unexpected: true +output: + format: markdown + validation_mode: basic + repair_attempts: 0 diff --git a/internal/promptdef/testdata/unknown_input_field.yaml b/internal/promptdef/testdata/unknown_input_field.yaml new file mode 100644 index 0000000..d8dfa35 --- /dev/null +++ b/internal/promptdef/testdata/unknown_input_field.yaml @@ -0,0 +1,13 @@ +id: unknown-input-field +version: "1.0.0" +inputs: + - name: transcript + required: true + unknown_input_setting: true +messages: + - role: user + content: "Hi" +output: + format: text + validation_mode: none + repair_attempts: 0 diff --git a/internal/promptdef/testdata/unsupported_cache_control_ttl.yaml b/internal/promptdef/testdata/unsupported_cache_control_ttl.yaml new file mode 100644 index 0000000..d6f81d4 --- /dev/null +++ b/internal/promptdef/testdata/unsupported_cache_control_ttl.yaml @@ -0,0 +1,12 @@ +id: unsupported-cache-control-ttl +version: "1.0.0" +messages: + - role: system + content: "Use cached instructions." + cache_control: + type: ephemeral + ttl: 5m +output: + format: markdown + validation_mode: basic + repair_attempts: 0 diff --git a/internal/promptdef/testdata/unsupported_cache_control_type.yaml b/internal/promptdef/testdata/unsupported_cache_control_type.yaml new file mode 100644 index 0000000..875a9b0 --- /dev/null +++ b/internal/promptdef/testdata/unsupported_cache_control_type.yaml @@ -0,0 +1,11 @@ +id: unsupported-cache-control-type +version: "1.0.0" +messages: + - role: system + content: "Use cached instructions." + cache_control: + type: persistent +output: + format: markdown + validation_mode: basic + repair_attempts: 0 diff --git a/internal/promptdef/testdata/valid_cache_control_ttl.yaml b/internal/promptdef/testdata/valid_cache_control_ttl.yaml new file mode 100644 index 0000000..4136ac4 --- /dev/null +++ b/internal/promptdef/testdata/valid_cache_control_ttl.yaml @@ -0,0 +1,14 @@ +id: valid-cache-control-ttl +version: "1.0.0" +messages: + - role: system + content: "Use cached instructions." + cache_control: + type: ephemeral + ttl: 1h + - role: user + content: "Summarize the input." +output: + format: markdown + validation_mode: basic + repair_attempts: 0 diff --git a/internal/promptdef/testdata/valid_cache_control_without_ttl.yaml b/internal/promptdef/testdata/valid_cache_control_without_ttl.yaml new file mode 100644 index 0000000..48e6c37 --- /dev/null +++ b/internal/promptdef/testdata/valid_cache_control_without_ttl.yaml @@ -0,0 +1,13 @@ +id: valid-cache-control-without-ttl +version: "1.0.0" +messages: + - role: system + content: "Use cached instructions." + cache_control: + type: ephemeral + - role: user + content: "Summarize the input." +output: + format: markdown + validation_mode: basic + repair_attempts: 0 diff --git a/internal/promptdef/testdata/valid_file_backed.yaml b/internal/promptdef/testdata/valid_file_backed.yaml new file mode 100644 index 0000000..f57f56a --- /dev/null +++ b/internal/promptdef/testdata/valid_file_backed.yaml @@ -0,0 +1,14 @@ +id: valid-file-backed +version: "1.0.0" +inputs: + - name: transcript + required: true +messages: + - role: system + content: "Return markdown." + - role: user + content_file: ./messages/user_prompt.tmpl +output: + format: markdown + validation_mode: basic + repair_attempts: 0 diff --git a/internal/promptdef/testdata/valid_inline.yaml b/internal/promptdef/testdata/valid_inline.yaml new file mode 100644 index 0000000..3f61059 --- /dev/null +++ b/internal/promptdef/testdata/valid_inline.yaml @@ -0,0 +1,18 @@ +id: valid-inline +version: "1.0.0" +inputs: + - name: transcript + required: true + content_type: text/markdown + description: Transcript content +messages: + - role: system + content: "You are concise." + - role: user + content: | + Summarize: + {{input "transcript"}} +output: + format: markdown + validation_mode: basic + repair_attempts: 0 diff --git a/internal/promptdef/testdata/valid_session_id.yaml b/internal/promptdef/testdata/valid_session_id.yaml new file mode 100644 index 0000000..b477662 --- /dev/null +++ b/internal/promptdef/testdata/valid_session_id.yaml @@ -0,0 +1,10 @@ +id: valid-session-id +version: "1.0.0" +session_id: " {{ .session_id }} " +messages: + - role: user + content: Hello. +output: + format: markdown + validation_mode: basic + repair_attempts: 0 diff --git a/internal/promptdef/testdata/with_default_profile.yaml b/internal/promptdef/testdata/with_default_profile.yaml new file mode 100644 index 0000000..1628b41 --- /dev/null +++ b/internal/promptdef/testdata/with_default_profile.yaml @@ -0,0 +1,13 @@ +id: with-default-profile +version: "1.0.0" +default_profile: local-default +inputs: + - name: transcript + required: true +messages: + - role: user + content: "Write output" +output: + format: text + validation_mode: none + repair_attempts: 0