Extract prompt sources and rendering internals
This commit is contained in:
10
README.md
10
README.md
@@ -9,10 +9,12 @@ gitea.maximumdirect.net/eric/promptkit
|
||||
|
||||
Framework extraction is in progress. The repository now contains the
|
||||
`internal/domain` model, application-neutral `internal/defaults`, and
|
||||
`internal/filecatalog` helpers that form the implementation foundation. These
|
||||
internal packages are not a consumer API, and the root package does not yet
|
||||
provide a usable public framework API, so there is no installation or usage
|
||||
example at this time.
|
||||
`internal/filecatalog` helpers that form the implementation foundation. It also
|
||||
contains internal prompt-definition and profile repositories, the embedded
|
||||
built-in profile catalog, and Go-template prompt rendering. These internal
|
||||
packages are not a consumer API, and the root package does not yet provide a
|
||||
usable public framework API, so there is no installation or usage example at
|
||||
this time.
|
||||
|
||||
Contributors should start with the [development guide](docs/development.md).
|
||||
The [architecture policy](docs/policy/architecture.md) defines the library
|
||||
|
||||
@@ -15,10 +15,14 @@ contributor workflow and validation.
|
||||
| `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, profiles, execution targets, rendering, generation, and validation. | [Domain declarations](../../internal/domain/domain.go) |
|
||||
| `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) |
|
||||
| `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) |
|
||||
| `internal/promptdef` | Loads strictly decoded, validated prompt definitions from filesystem and `fs.FS` sources, including version selection and contained file-backed message content. | [Prompt-definition repository](../../internal/promptdef/filesystem_repository.go) |
|
||||
| `internal/profile` | Loads strictly decoded, validated execution profiles from filesystem and `fs.FS` sources and composes repositories with error-preserving fallback. | [Profile repositories](../../internal/profile/filesystem_repository.go) |
|
||||
| `internal/profile/builtin` | Embeds the built-in execution profile catalog and combines it with an optional primary repository. | [Built-in profile repository](../../internal/profile/builtin/repository.go) |
|
||||
| `internal/prompt` | Renders prompt messages from Go templates with artifact, variable, session, and cache-control data. | [Go-template renderer](../../internal/prompt/go_renderer.go) |
|
||||
|
||||
These packages form the internal extraction foundation. Prompt and profile
|
||||
sources, rendering, artifact reading, validation, model clients, orchestration,
|
||||
and a usable public engine are not implemented in Promptkit yet.
|
||||
These packages provide the internal model, source, and rendering foundation.
|
||||
Artifact reading, output validation, model clients, orchestration, and a usable
|
||||
public engine are not implemented in Promptkit yet.
|
||||
|
||||
## Maintenance
|
||||
|
||||
|
||||
@@ -16,19 +16,27 @@ The module root contains package `promptkit`, which is the public facade. It
|
||||
declares the module's public package boundary but does not yet provide a usable
|
||||
exported framework API.
|
||||
|
||||
The implemented internal foundation consists of:
|
||||
The implemented internal components consist of:
|
||||
|
||||
- `internal/domain`, which owns framework data values shared by later internal
|
||||
components;
|
||||
- `internal/defaults`, which owns application-neutral framework defaults and
|
||||
constructs the default execution target; and
|
||||
- `internal/filecatalog`, which discovers YAML files and provides source-path
|
||||
helpers for filesystem and `fs.FS` consumers.
|
||||
helpers for filesystem and `fs.FS` consumers;
|
||||
- `internal/promptdef`, which loads and validates prompt definitions from
|
||||
filesystem and `fs.FS` sources;
|
||||
- `internal/profile`, which loads, validates, and overlays execution profiles
|
||||
from filesystem and `fs.FS` sources;
|
||||
- `internal/profile/builtin`, which embeds the built-in execution profile
|
||||
catalog; and
|
||||
- `internal/prompt`, which renders prompt messages from Go templates.
|
||||
|
||||
`internal/defaults` depends on `internal/domain`; the file catalog is
|
||||
independent and uses only the standard library. Prompt and profile sources,
|
||||
rendering, artifact reading, validation, model clients, orchestration, and the
|
||||
public engine have not yet been extracted.
|
||||
The defaults and renderer depend on the domain model. Prompt-definition and
|
||||
profile repositories use the domain model, file catalog, and YAML decoder. The
|
||||
built-in profile repository supplies an embedded `fs.FS` to the profile
|
||||
package. Artifact reading, output validation, model clients, orchestration, and
|
||||
the public engine have not yet been extracted.
|
||||
|
||||
Future framework extraction must follow this dependency direction:
|
||||
|
||||
|
||||
2
go.mod
2
go.mod
@@ -1,3 +1,5 @@
|
||||
module gitea.maximumdirect.net/eric/promptkit
|
||||
|
||||
go 1.25.5
|
||||
|
||||
require gopkg.in/yaml.v3 v3.0.1
|
||||
|
||||
4
go.sum
Normal file
4
go.sum
Normal file
@@ -0,0 +1,4 @@
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
9
internal/profile/builtin/assets/aion-labs/aion-2.yml
Normal file
9
internal/profile/builtin/assets/aion-labs/aion-2.yml
Normal file
@@ -0,0 +1,9 @@
|
||||
id: aion-2
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: aion-labs/aion-2.0
|
||||
temperature: 0.72
|
||||
reasoning_effort: high
|
||||
top_p: 0.95
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,7 @@
|
||||
id: claude-fable-latest
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "~anthropic/claude-fable-latest"
|
||||
reasoning_effort: high
|
||||
timeout_seconds: 600
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,7 @@
|
||||
id: claude-haiku-latest
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "~anthropic/claude-haiku-latest"
|
||||
reasoning_effort: medium
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,7 @@
|
||||
id: claude-opus-latest
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "~anthropic/claude-opus-latest"
|
||||
reasoning_effort: high
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,7 @@
|
||||
id: claude-sonnet-latest
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "~anthropic/claude-sonnet-latest"
|
||||
reasoning_effort: high
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,7 @@
|
||||
id: deepseek-3-2
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: deepseek/deepseek-v3.2
|
||||
reasoning_effort: high
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,7 @@
|
||||
id: deepseek-4-flash
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: deepseek/deepseek-v4-flash
|
||||
#reasoning_effort: medium
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,7 @@
|
||||
id: deepseek-4-pro
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: deepseek/deepseek-v4-pro
|
||||
reasoning_effort: high
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,9 @@
|
||||
id: gemini-2-flash-lite
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "google/gemini-2.5-flash-lite"
|
||||
#temperature: 0.15
|
||||
reasoning_effort: high
|
||||
#top_p: 0.98
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,9 @@
|
||||
id: gemini-2-flash
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "google/gemini-2.5-flash"
|
||||
#temperature: 0.15
|
||||
reasoning_effort: high
|
||||
#top_p: 0.98
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
9
internal/profile/builtin/assets/google/gemini-2-pro.yml
Normal file
9
internal/profile/builtin/assets/google/gemini-2-pro.yml
Normal file
@@ -0,0 +1,9 @@
|
||||
id: gemini-2-pro
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "google/gemini-2.5-pro"
|
||||
#temperature: 0.15
|
||||
reasoning_effort: high
|
||||
#top_p: 0.98
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,9 @@
|
||||
id: gemini-3-flash-lite
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "google/gemini-3.1-flash-lite"
|
||||
#temperature: 0.15
|
||||
reasoning_effort: high
|
||||
#top_p: 0.98
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,9 @@
|
||||
id: gemini-flash-latest
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "~google/gemini-flash-latest"
|
||||
#temperature: 0.15
|
||||
reasoning_effort: high
|
||||
#top_p: 0.98
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,9 @@
|
||||
id: gemini-pro-latest
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "~google/gemini-pro-latest"
|
||||
#temperature: 0.15
|
||||
reasoning_effort: high
|
||||
#top_p: 0.98
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
9
internal/profile/builtin/assets/google/gemma-4-31b.yml
Normal file
9
internal/profile/builtin/assets/google/gemma-4-31b.yml
Normal file
@@ -0,0 +1,9 @@
|
||||
id: gemma-4-31b
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: google/gemma-4-31b-it:exacto
|
||||
temperature: 0.15
|
||||
reasoning_effort: high
|
||||
top_p: 0.98
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
9
internal/profile/builtin/assets/minimax/minimax-m2.yml
Normal file
9
internal/profile/builtin/assets/minimax/minimax-m2.yml
Normal file
@@ -0,0 +1,9 @@
|
||||
id: minimax-m2
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: minimax/minimax-m2.5
|
||||
temperature: 0.5
|
||||
reasoning_effort: high
|
||||
top_p: 0.95
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
9
internal/profile/builtin/assets/minimax/minimax-m3.yml
Normal file
9
internal/profile/builtin/assets/minimax/minimax-m3.yml
Normal file
@@ -0,0 +1,9 @@
|
||||
id: minimax-m3
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: minimax/minimax-m3
|
||||
#temperature: 0.5
|
||||
reasoning_effort: high
|
||||
#top_p: 0.95
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
@@ -0,0 +1,7 @@
|
||||
id: mistral-large-2512
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: mistralai/mistral-large-2512
|
||||
temperature: 0.15
|
||||
top_p: 0.98
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
@@ -0,0 +1,8 @@
|
||||
id: mistral-medium-3-5
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: mistralai/mistral-medium-3-5
|
||||
temperature: 0.15
|
||||
reasoning_effort: high
|
||||
top_p: 0.98
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
@@ -0,0 +1,7 @@
|
||||
id: mistral-small-3
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: mistralai/mistral-small-3.2-24b-instruct
|
||||
temperature: 0.05
|
||||
top_p: 1.0
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
@@ -0,0 +1,8 @@
|
||||
id: mistral-small-4
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: mistralai/mistral-small-2603
|
||||
temperature: 0.1
|
||||
reasoning_effort: high
|
||||
top_p: 0.98
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
@@ -0,0 +1,7 @@
|
||||
id: nemotron-3-ultra
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: nvidia/nemotron-3-ultra-550b-a55b
|
||||
reasoning_effort: high
|
||||
timeout_seconds: 180
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
7
internal/profile/builtin/assets/openai/gpt-5-mini.yml
Normal file
7
internal/profile/builtin/assets/openai/gpt-5-mini.yml
Normal file
@@ -0,0 +1,7 @@
|
||||
id: gpt-5-mini
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "openai/gpt-5.4-mini"
|
||||
reasoning_effort: high
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
7
internal/profile/builtin/assets/openai/gpt-5-nano.yml
Normal file
7
internal/profile/builtin/assets/openai/gpt-5-nano.yml
Normal file
@@ -0,0 +1,7 @@
|
||||
id: gpt-5-nano
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
model: "openai/gpt-5.4-nano"
|
||||
reasoning_effort: high
|
||||
timeout_seconds: 240
|
||||
api_key_env: OPENROUTER_API_KEY
|
||||
service_tier: flex
|
||||
31
internal/profile/builtin/repository.go
Normal file
31
internal/profile/builtin/repository.go
Normal file
@@ -0,0 +1,31 @@
|
||||
package builtin
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/profile"
|
||||
)
|
||||
|
||||
const assetRoot = "assets"
|
||||
|
||||
//go:embed assets/**/*.yml
|
||||
var assets embed.FS
|
||||
|
||||
func NewRepository() profile.Repository {
|
||||
return profile.NewFSRepository(assets, assetRoot)
|
||||
}
|
||||
|
||||
func NewRepositoryWithPrimary(primary profile.Repository) profile.Repository {
|
||||
if primary == nil {
|
||||
return NewRepository()
|
||||
}
|
||||
return profile.NewOverlayRepository(primary, NewRepository())
|
||||
}
|
||||
|
||||
func NewRepositoryWithDirectory(dir string) profile.Repository {
|
||||
if strings.TrimSpace(dir) == "" {
|
||||
return NewRepository()
|
||||
}
|
||||
return NewRepositoryWithPrimary(profile.NewFilesystemRepository(dir))
|
||||
}
|
||||
127
internal/profile/builtin/repository_test.go
Normal file
127
internal/profile/builtin/repository_test.go
Normal file
@@ -0,0 +1,127 @@
|
||||
package builtin
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io/fs"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/profile"
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
func TestBuiltInProfilesValidateThroughRepository(t *testing.T) {
|
||||
repo := NewRepository()
|
||||
ids := loadBuiltInProfileIDs(t)
|
||||
if len(ids) == 0 {
|
||||
t.Fatal("expected built-in profiles")
|
||||
}
|
||||
|
||||
for id := range ids {
|
||||
t.Run(id, func(t *testing.T) {
|
||||
p, err := repo.GetProfile(context.Background(), id)
|
||||
if err != nil {
|
||||
t.Fatalf("expected built-in profile %q to load, got %v", id, err)
|
||||
}
|
||||
if p.ID != id {
|
||||
t.Fatalf("expected profile id %q, got %q", id, p.ID)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuiltInProfilesDoNotContainDuplicateIDsOrRawAPIKeys(t *testing.T) {
|
||||
loadBuiltInProfileIDs(t)
|
||||
}
|
||||
|
||||
func loadBuiltInProfileIDs(t *testing.T) map[string]string {
|
||||
t.Helper()
|
||||
|
||||
ids := map[string]string{}
|
||||
err := fs.WalkDir(assets, assetRoot, func(name string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.IsDir() || !strings.HasSuffix(name, ".yml") {
|
||||
return nil
|
||||
}
|
||||
|
||||
data, err := assets.ReadFile(name)
|
||||
if err != nil {
|
||||
t.Fatalf("failed to read built-in profile %s: %v", name, err)
|
||||
}
|
||||
|
||||
var raw map[string]any
|
||||
if err := yaml.Unmarshal(data, &raw); err != nil {
|
||||
t.Fatalf("failed to decode built-in profile %s: %v", name, err)
|
||||
}
|
||||
if _, ok := raw["api_key"]; ok {
|
||||
t.Fatalf("built-in profile %s contains raw api_key", name)
|
||||
}
|
||||
id, ok := raw["id"].(string)
|
||||
if !ok || strings.TrimSpace(id) == "" {
|
||||
t.Fatalf("built-in profile %s has missing id", name)
|
||||
}
|
||||
if previous, ok := ids[id]; ok {
|
||||
t.Fatalf("duplicate built-in profile id %q in %s and %s", id, previous, name)
|
||||
}
|
||||
ids[id] = name
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("failed to walk built-in profiles: %v", err)
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
func TestRepositoryWithPrimaryUsesPrimaryBeforeBuiltIns(t *testing.T) {
|
||||
repo := NewRepositoryWithPrimary(staticProfileRepo{
|
||||
profiles: map[string]string{"mistral-small-3": "custom-model"},
|
||||
})
|
||||
|
||||
p, err := repo.GetProfile(context.Background(), "mistral-small-3")
|
||||
if err != nil {
|
||||
t.Fatalf("expected profile to load, got %v", err)
|
||||
}
|
||||
if p.Model != "custom-model" {
|
||||
t.Fatalf("expected primary profile to override built-in, got %+v", p)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRepositoryWithPrimaryFallsBackToBuiltIns(t *testing.T) {
|
||||
repo := NewRepositoryWithPrimary(staticProfileRepo{})
|
||||
|
||||
p, err := repo.GetProfile(context.Background(), "mistral-small-3")
|
||||
if err != nil {
|
||||
t.Fatalf("expected built-in profile to load, got %v", err)
|
||||
}
|
||||
if p.ID != "mistral-small-3" {
|
||||
t.Fatalf("unexpected profile: %+v", p)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRepositoryWithPrimaryDoesNotFallBackAfterPrimaryError(t *testing.T) {
|
||||
repo := NewRepositoryWithPrimary(staticProfileRepo{err: profile.ErrInvalidProfile})
|
||||
|
||||
_, err := repo.GetProfile(context.Background(), "mistral-small-3")
|
||||
if !errors.Is(err, profile.ErrInvalidProfile) {
|
||||
t.Fatalf("expected primary error, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
type staticProfileRepo struct {
|
||||
profiles map[string]string
|
||||
err error
|
||||
}
|
||||
|
||||
func (r staticProfileRepo) GetProfile(_ context.Context, id string) (*domain.ExecutionProfile, error) {
|
||||
if r.err != nil {
|
||||
return nil, r.err
|
||||
}
|
||||
if model, ok := r.profiles[id]; ok {
|
||||
return &domain.ExecutionProfile{ID: id, Endpoint: "http://primary/v1", Model: model}, nil
|
||||
}
|
||||
return nil, profile.ErrProfileNotFound
|
||||
}
|
||||
213
internal/profile/filesystem_repository.go
Normal file
213
internal/profile/filesystem_repository.go
Normal file
@@ -0,0 +1,213 @@
|
||||
package profile
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/filecatalog"
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrProfileNotFound = errors.New("execution profile not found")
|
||||
ErrInvalidYAML = errors.New("invalid YAML format")
|
||||
ErrInvalidProfile = errors.New("invalid execution profile configuration")
|
||||
ErrRawAPIKeyNotAllowed = errors.New("raw api_key is not allowed; use api_key_env")
|
||||
)
|
||||
|
||||
type filesystemRepository struct {
|
||||
dir string
|
||||
}
|
||||
|
||||
func NewFilesystemRepository(dir string) Repository {
|
||||
return &filesystemRepository{dir: dir}
|
||||
}
|
||||
|
||||
func (r *filesystemRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
|
||||
return loadProfile(ctx, os.DirFS(r.dir), ".", id)
|
||||
}
|
||||
|
||||
type fsRepository struct {
|
||||
fsys fs.FS
|
||||
root string
|
||||
}
|
||||
|
||||
func NewFSRepository(fsys fs.FS, root string) Repository {
|
||||
return &fsRepository{fsys: fsys, root: root}
|
||||
}
|
||||
|
||||
func (r *fsRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
|
||||
return loadProfile(ctx, r.fsys, r.root, id)
|
||||
}
|
||||
|
||||
type overlayRepository struct {
|
||||
primary Repository
|
||||
fallback Repository
|
||||
}
|
||||
|
||||
func NewOverlayRepository(primary, fallback Repository) Repository {
|
||||
return &overlayRepository{primary: primary, fallback: fallback}
|
||||
}
|
||||
|
||||
func (r *overlayRepository) GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error) {
|
||||
if r.primary != nil {
|
||||
prof, err := r.primary.GetProfile(ctx, id)
|
||||
if err == nil {
|
||||
return prof, nil
|
||||
}
|
||||
if !errors.Is(err, ErrProfileNotFound) {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
if r.fallback == nil {
|
||||
return nil, ErrProfileNotFound
|
||||
}
|
||||
return r.fallback.GetProfile(ctx, id)
|
||||
}
|
||||
|
||||
func loadProfile(ctx context.Context, fsys fs.FS, root string, id string) (*domain.ExecutionProfile, error) {
|
||||
if strings.TrimSpace(id) == "" {
|
||||
return nil, fmt.Errorf("%w: profile id is required", ErrInvalidProfile)
|
||||
}
|
||||
if fsys == nil {
|
||||
return nil, fmt.Errorf("failed to read profile directory: filesystem is nil")
|
||||
}
|
||||
|
||||
files, err := filecatalog.FindFSYAMLFiles(ctx, fsys, root)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read profile directory: %w", err)
|
||||
}
|
||||
|
||||
var matches []profileMatch
|
||||
for _, fullPath := range files {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
default:
|
||||
}
|
||||
|
||||
relPath := filecatalog.DisplayPath(root, fullPath)
|
||||
fileMatch := filecatalog.Stem(path.Base(fullPath)) == id
|
||||
data, err := fs.ReadFile(fsys, fullPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read profile file %s: %w", relPath, err)
|
||||
}
|
||||
metadata := readProfileFileMetadata(data)
|
||||
idMatch := fileMatch || metadata.id == id
|
||||
if metadata.hasRawAPIKey {
|
||||
if idMatch {
|
||||
return nil, fmt.Errorf("%w: %s", ErrRawAPIKeyNotAllowed, relPath)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
var prof domain.ExecutionProfile
|
||||
decoder := yaml.NewDecoder(bytes.NewReader(data))
|
||||
decoder.KnownFields(true)
|
||||
if err := decoder.Decode(&prof); err != nil {
|
||||
if idMatch {
|
||||
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidYAML, relPath, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if prof.ID != id {
|
||||
continue
|
||||
}
|
||||
if err := validateProfile(&prof); err != nil {
|
||||
if errors.Is(err, ErrRawAPIKeyNotAllowed) {
|
||||
return nil, fmt.Errorf("%w: %s", err, relPath)
|
||||
}
|
||||
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidProfile, relPath, err)
|
||||
}
|
||||
matches = append(matches, profileMatch{
|
||||
profile: &prof,
|
||||
path: relPath,
|
||||
})
|
||||
}
|
||||
|
||||
if len(matches) > 1 {
|
||||
paths := make([]string, 0, len(matches))
|
||||
for _, match := range matches {
|
||||
paths = append(paths, match.path)
|
||||
}
|
||||
return nil, fmt.Errorf("%w: duplicate execution profile id %q found in: %s", ErrInvalidProfile, id, strings.Join(paths, ", "))
|
||||
}
|
||||
|
||||
if len(matches) == 1 {
|
||||
return matches[0].profile, nil
|
||||
}
|
||||
|
||||
return nil, ErrProfileNotFound
|
||||
}
|
||||
|
||||
type profileMatch struct {
|
||||
profile *domain.ExecutionProfile
|
||||
path string
|
||||
}
|
||||
|
||||
type profileFileMetadata struct {
|
||||
id string
|
||||
hasRawAPIKey bool
|
||||
}
|
||||
|
||||
func readProfileFileMetadata(data []byte) profileFileMetadata {
|
||||
var node yaml.Node
|
||||
if err := yaml.NewDecoder(bytes.NewReader(data)).Decode(&node); err != nil {
|
||||
return profileFileMetadata{}
|
||||
}
|
||||
if node.Kind != yaml.DocumentNode || len(node.Content) == 0 {
|
||||
return profileFileMetadata{}
|
||||
}
|
||||
mapping := node.Content[0]
|
||||
if mapping.Kind != yaml.MappingNode {
|
||||
return profileFileMetadata{}
|
||||
}
|
||||
|
||||
var metadata profileFileMetadata
|
||||
for i := 0; i+1 < len(mapping.Content); i += 2 {
|
||||
key := mapping.Content[i]
|
||||
value := mapping.Content[i+1]
|
||||
switch key.Value {
|
||||
case "id":
|
||||
metadata.id = strings.TrimSpace(value.Value)
|
||||
case "api_key":
|
||||
metadata.hasRawAPIKey = true
|
||||
}
|
||||
}
|
||||
return metadata
|
||||
}
|
||||
|
||||
func validateProfile(p *domain.ExecutionProfile) error {
|
||||
if strings.TrimSpace(p.ID) == "" {
|
||||
return errors.New("id is required")
|
||||
}
|
||||
if strings.TrimSpace(p.Endpoint) == "" {
|
||||
return errors.New("endpoint is required")
|
||||
}
|
||||
if strings.TrimSpace(p.Model) == "" {
|
||||
return errors.New("model is required")
|
||||
}
|
||||
|
||||
if p.Temperature < 0 || p.Temperature > 2 {
|
||||
return errors.New("temperature must be between 0 and 2")
|
||||
}
|
||||
if p.MaxTokens < 0 {
|
||||
return errors.New("max_tokens must be greater than or equal to 0")
|
||||
}
|
||||
if p.TopP < 0 || p.TopP > 1 {
|
||||
return errors.New("top_p must be between 0 and 1")
|
||||
}
|
||||
if p.TimeoutSeconds < 0 {
|
||||
return errors.New("timeout_seconds must be greater than or equal to 0")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
12
internal/profile/repository.go
Normal file
12
internal/profile/repository.go
Normal file
@@ -0,0 +1,12 @@
|
||||
package profile
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||
)
|
||||
|
||||
// Repository loads execution profiles.
|
||||
type Repository interface {
|
||||
GetProfile(ctx context.Context, id string) (*domain.ExecutionProfile, error)
|
||||
}
|
||||
479
internal/profile/repository_test.go
Normal file
479
internal/profile/repository_test.go
Normal file
@@ -0,0 +1,479 @@
|
||||
package profile
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||
)
|
||||
|
||||
func TestFilesystemRepository_GetProfile(t *testing.T) {
|
||||
tmpDir, err := os.MkdirTemp("", "execution_profile_test")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer os.RemoveAll(tmpDir)
|
||||
|
||||
files, err := os.ReadDir("testdata")
|
||||
if err != nil {
|
||||
t.Fatalf("failed to read testdata: %v", err)
|
||||
}
|
||||
for _, f := range files {
|
||||
src := filepath.Join("testdata", f.Name())
|
||||
dst := filepath.Join(tmpDir, f.Name())
|
||||
data, err := os.ReadFile(src)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(dst, data, 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
repo := NewFilesystemRepository(tmpDir)
|
||||
ctx := context.Background()
|
||||
|
||||
t.Run("valid local profile", func(t *testing.T) {
|
||||
p, err := repo.GetProfile(ctx, "local-default")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.ID != "local-default" {
|
||||
t.Fatalf("unexpected id: %q", p.ID)
|
||||
}
|
||||
if p.Endpoint == "" || p.Model == "" {
|
||||
t.Fatalf("expected endpoint/model to be set: %+v", p)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid profile with api_key_env", func(t *testing.T) {
|
||||
p, err := repo.GetProfile(ctx, "local-secure")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.APIKeyEnv != "SCRIPTORIUM_API_KEY" {
|
||||
t.Fatalf("unexpected api_key_env: %q", p.APIKeyEnv)
|
||||
}
|
||||
if p.ReasoningEffort != "medium" {
|
||||
t.Fatalf("unexpected reasoning_effort: %q", p.ReasoningEffort)
|
||||
}
|
||||
if p.ServiceTier != "priority" {
|
||||
t.Fatalf("unexpected service_tier: %q", p.ServiceTier)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid nested profile", func(t *testing.T) {
|
||||
nestedDir := filepath.Join(tmpDir, "local")
|
||||
if err := os.MkdirAll(nestedDir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeProfileTestFile(t, filepath.Join(nestedDir, "nested-local.yaml"), `
|
||||
id: nested-local
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: nested-model
|
||||
temperature: 0.1
|
||||
`)
|
||||
|
||||
p, err := repo.GetProfile(ctx, "nested-local")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.Model != "nested-model" {
|
||||
t.Fatalf("unexpected model: %q", p.Model)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid profile with JSON-compatible extra params", func(t *testing.T) {
|
||||
writeProfileTestFile(t, filepath.Join(tmpDir, "json-extra-params.yaml"), `
|
||||
id: json-extra-params
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: nested-model
|
||||
extra_params:
|
||||
string_value: enabled
|
||||
number_value: 42
|
||||
boolean_value: true
|
||||
object_value:
|
||||
nested: value
|
||||
count: 2
|
||||
array_value:
|
||||
- first
|
||||
- 3
|
||||
- false
|
||||
`)
|
||||
|
||||
p, err := repo.GetProfile(ctx, "json-extra-params")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
|
||||
var got map[string]any
|
||||
encoded, err := json.Marshal(p.ExtraParams)
|
||||
if err != nil {
|
||||
t.Fatalf("expected extra_params to marshal as JSON, got %v", err)
|
||||
}
|
||||
if err := json.Unmarshal(encoded, &got); err != nil {
|
||||
t.Fatalf("expected extra_params JSON to decode, got %v", err)
|
||||
}
|
||||
|
||||
if got["string_value"] != "enabled" {
|
||||
t.Fatalf("unexpected string extra param: %#v", got["string_value"])
|
||||
}
|
||||
if got["number_value"] != float64(42) {
|
||||
t.Fatalf("unexpected number extra param: %#v", got["number_value"])
|
||||
}
|
||||
if got["boolean_value"] != true {
|
||||
t.Fatalf("unexpected boolean extra param: %#v", got["boolean_value"])
|
||||
}
|
||||
objectValue, ok := got["object_value"].(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected object extra param, got %#v", got["object_value"])
|
||||
}
|
||||
if objectValue["nested"] != "value" || objectValue["count"] != float64(2) {
|
||||
t.Fatalf("unexpected object extra param: %#v", objectValue)
|
||||
}
|
||||
arrayValue, ok := got["array_value"].([]any)
|
||||
if !ok {
|
||||
t.Fatalf("expected array extra param, got %#v", got["array_value"])
|
||||
}
|
||||
if len(arrayValue) != 3 || arrayValue[0] != "first" || arrayValue[1] != float64(3) || arrayValue[2] != false {
|
||||
t.Fatalf("unexpected array extra param: %#v", arrayValue)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("duplicate profile IDs fail as ambiguous", func(t *testing.T) {
|
||||
writeProfileTestFile(t, filepath.Join(tmpDir, "duplicate-profile-a.yaml"), `
|
||||
id: duplicate-profile
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: first-model
|
||||
`)
|
||||
nestedDir := filepath.Join(tmpDir, "duplicates")
|
||||
if err := os.MkdirAll(nestedDir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeProfileTestFile(t, filepath.Join(nestedDir, "duplicate-profile-b.yaml"), `
|
||||
id: duplicate-profile
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: second-model
|
||||
`)
|
||||
|
||||
_, err := repo.GetProfile(ctx, "duplicate-profile")
|
||||
if !errors.Is(err, ErrInvalidProfile) {
|
||||
t.Fatalf("expected duplicate profile to return ErrInvalidProfile, got %v", err)
|
||||
}
|
||||
for _, want := range []string{"duplicate execution profile id", "duplicate-profile-a.yaml", filepath.Join("duplicates", "duplicate-profile-b.yaml")} {
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Fatalf("expected error to contain %q, got %v", want, err)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("nested raw api_key rejected for likely target file", func(t *testing.T) {
|
||||
nestedDir := filepath.Join(tmpDir, "secure")
|
||||
if err := os.MkdirAll(nestedDir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeProfileTestFile(t, filepath.Join(nestedDir, "not_named_like_id.yaml"), `
|
||||
id: nested_raw_api_key
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: m
|
||||
api_key: secret
|
||||
`)
|
||||
|
||||
_, err := repo.GetProfile(ctx, "nested_raw_api_key")
|
||||
if !errors.Is(err, ErrRawAPIKeyNotAllowed) {
|
||||
t.Fatalf("expected ErrRawAPIKeyNotAllowed, got %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), filepath.Join("secure", "not_named_like_id.yaml")) {
|
||||
t.Fatalf("expected nested path in error, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("raw api_key in non-target profile is ignored", func(t *testing.T) {
|
||||
writeProfileTestFile(t, filepath.Join(tmpDir, "raw-api-key-non-target.yaml"), `
|
||||
id: raw-api-key-non-target
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: m
|
||||
api_key: secret
|
||||
`)
|
||||
|
||||
_, err := repo.GetProfile(ctx, "does-not-exist-with-raw-key-nearby")
|
||||
if !errors.Is(err, ErrProfileNotFound) {
|
||||
t.Fatalf("expected ErrProfileNotFound for non-target raw api_key file, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("invalid yaml", func(t *testing.T) {
|
||||
_, err := repo.GetProfile(ctx, "invalid_yaml")
|
||||
if !errors.Is(err, ErrInvalidYAML) {
|
||||
t.Fatalf("expected ErrInvalidYAML, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("missing id", func(t *testing.T) {
|
||||
_, err := repo.GetProfile(ctx, "missing_id")
|
||||
if !errors.Is(err, ErrProfileNotFound) {
|
||||
t.Fatalf("expected ErrProfileNotFound, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("missing endpoint", func(t *testing.T) {
|
||||
_, err := repo.GetProfile(ctx, "missing-endpoint")
|
||||
if !errors.Is(err, ErrInvalidProfile) {
|
||||
t.Fatalf("expected ErrInvalidProfile, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("missing model", func(t *testing.T) {
|
||||
_, err := repo.GetProfile(ctx, "missing-model")
|
||||
if !errors.Is(err, ErrInvalidProfile) {
|
||||
t.Fatalf("expected ErrInvalidProfile, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("unknown field", func(t *testing.T) {
|
||||
_, err := repo.GetProfile(ctx, "unknown_field")
|
||||
if !errors.Is(err, ErrInvalidYAML) {
|
||||
t.Fatalf("expected ErrInvalidYAML for strict decode unknown field, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("raw api_key rejected", func(t *testing.T) {
|
||||
_, err := repo.GetProfile(ctx, "raw_api_key")
|
||||
if !errors.Is(err, ErrRawAPIKeyNotAllowed) {
|
||||
t.Fatalf("expected ErrRawAPIKeyNotAllowed, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("profile not found", func(t *testing.T) {
|
||||
_, err := repo.GetProfile(ctx, "does-not-exist")
|
||||
if !errors.Is(err, ErrProfileNotFound) {
|
||||
t.Fatalf("expected ErrProfileNotFound, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func writeProfileTestFile(t *testing.T, path string, content string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(path, []byte(strings.TrimLeft(content, "\n")), 0o644); err != nil {
|
||||
t.Fatalf("failed to write profile test file %q: %v", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSRepository(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
|
||||
t.Run("loads valid profiles from nested directories", func(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"profiles/provider/nested.yaml": profileMapFile(`
|
||||
id: nested-profile
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: nested-model
|
||||
temperature: 0.1
|
||||
`),
|
||||
}, "profiles")
|
||||
|
||||
p, err := repo.GetProfile(ctx, "nested-profile")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.ID != "nested-profile" || p.Model != "nested-model" {
|
||||
t.Fatalf("unexpected profile: %+v", p)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rejects unknown YAML fields", func(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"profiles/unknown.yaml": profileMapFile(`
|
||||
id: unknown-profile
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: model
|
||||
unknown: value
|
||||
`),
|
||||
}, "profiles")
|
||||
|
||||
_, err := repo.GetProfile(ctx, "unknown-profile")
|
||||
if !errors.Is(err, ErrInvalidYAML) {
|
||||
t.Fatalf("expected ErrInvalidYAML, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rejects raw api_key in selected profile", func(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"profiles/raw.yaml": profileMapFile(`
|
||||
id: raw-profile
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: model
|
||||
api_key: secret
|
||||
`),
|
||||
}, "profiles")
|
||||
|
||||
_, err := repo.GetProfile(ctx, "raw-profile")
|
||||
if !errors.Is(err, ErrRawAPIKeyNotAllowed) {
|
||||
t.Fatalf("expected ErrRawAPIKeyNotAllowed, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("ignores raw api_key in non-selected profiles", func(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"profiles/raw.yaml": profileMapFile(`
|
||||
id: raw-profile
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: model
|
||||
api_key: secret
|
||||
`),
|
||||
"profiles/valid.yaml": profileMapFile(`
|
||||
id: valid-profile
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: model
|
||||
`),
|
||||
}, "profiles")
|
||||
|
||||
p, err := repo.GetProfile(ctx, "valid-profile")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.ID != "valid-profile" {
|
||||
t.Fatalf("unexpected profile: %+v", p)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rejects duplicate IDs within one source", func(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"profiles/a.yaml": profileMapFile(`
|
||||
id: duplicate-profile
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: first
|
||||
`),
|
||||
"profiles/nested/b.yaml": profileMapFile(`
|
||||
id: duplicate-profile
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: second
|
||||
`),
|
||||
}, "profiles")
|
||||
|
||||
_, err := repo.GetProfile(ctx, "duplicate-profile")
|
||||
if !errors.Is(err, ErrInvalidProfile) {
|
||||
t.Fatalf("expected ErrInvalidProfile, got %v", err)
|
||||
}
|
||||
for _, want := range []string{"duplicate execution profile id", "a.yaml", "nested/b.yaml"} {
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Fatalf("expected error to contain %q, got %v", want, err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestOverlayRepository(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
primaryProfile := &domain.ExecutionProfile{ID: "shared", Endpoint: "http://primary", Model: "primary"}
|
||||
fallbackProfile := &domain.ExecutionProfile{ID: "shared", Endpoint: "http://fallback", Model: "fallback"}
|
||||
|
||||
t.Run("returns primary matches before fallback matches", func(t *testing.T) {
|
||||
repo := NewOverlayRepository(
|
||||
staticProfileRepo{profiles: map[string]*domain.ExecutionProfile{"shared": primaryProfile}},
|
||||
staticProfileRepo{profiles: map[string]*domain.ExecutionProfile{"shared": fallbackProfile}},
|
||||
)
|
||||
|
||||
p, err := repo.GetProfile(ctx, "shared")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.Model != "primary" {
|
||||
t.Fatalf("expected primary profile, got %+v", p)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("falls back on primary not found", func(t *testing.T) {
|
||||
repo := NewOverlayRepository(
|
||||
staticProfileRepo{},
|
||||
staticProfileRepo{profiles: map[string]*domain.ExecutionProfile{"shared": fallbackProfile}},
|
||||
)
|
||||
|
||||
p, err := repo.GetProfile(ctx, "shared")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.Model != "fallback" {
|
||||
t.Fatalf("expected fallback profile, got %+v", p)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("does not fall back after primary load errors", func(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
err error
|
||||
}{
|
||||
{name: "invalid yaml", err: ErrInvalidYAML},
|
||||
{name: "invalid profile", err: ErrInvalidProfile},
|
||||
{name: "raw api key", err: ErrRawAPIKeyNotAllowed},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
repo := NewOverlayRepository(
|
||||
staticProfileRepo{err: tc.err},
|
||||
staticProfileRepo{profiles: map[string]*domain.ExecutionProfile{"shared": fallbackProfile}},
|
||||
)
|
||||
|
||||
_, err := repo.GetProfile(ctx, "shared")
|
||||
if !errors.Is(err, tc.err) {
|
||||
t.Fatalf("expected %v, got %v", tc.err, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("returns not found when both sources miss", func(t *testing.T) {
|
||||
repo := NewOverlayRepository(staticProfileRepo{}, staticProfileRepo{})
|
||||
|
||||
_, err := repo.GetProfile(ctx, "missing")
|
||||
if !errors.Is(err, ErrProfileNotFound) {
|
||||
t.Fatalf("expected ErrProfileNotFound, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("nil primary uses fallback", func(t *testing.T) {
|
||||
repo := NewOverlayRepository(nil, staticProfileRepo{profiles: map[string]*domain.ExecutionProfile{"shared": fallbackProfile}})
|
||||
|
||||
p, err := repo.GetProfile(ctx, "shared")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.Model != "fallback" {
|
||||
t.Fatalf("expected fallback profile, got %+v", p)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("nil fallback returns not found after primary miss", func(t *testing.T) {
|
||||
repo := NewOverlayRepository(staticProfileRepo{}, nil)
|
||||
|
||||
_, err := repo.GetProfile(ctx, "missing")
|
||||
if !errors.Is(err, ErrProfileNotFound) {
|
||||
t.Fatalf("expected ErrProfileNotFound, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func profileMapFile(content string) *fstest.MapFile {
|
||||
return &fstest.MapFile{Data: []byte(strings.TrimLeft(content, "\n"))}
|
||||
}
|
||||
|
||||
type staticProfileRepo struct {
|
||||
profiles map[string]*domain.ExecutionProfile
|
||||
err error
|
||||
}
|
||||
|
||||
func (r staticProfileRepo) GetProfile(_ context.Context, id string) (*domain.ExecutionProfile, error) {
|
||||
if r.err != nil {
|
||||
return nil, r.err
|
||||
}
|
||||
if p, ok := r.profiles[id]; ok {
|
||||
cp := *p
|
||||
return &cp, nil
|
||||
}
|
||||
return nil, ErrProfileNotFound
|
||||
}
|
||||
3
internal/profile/testdata/invalid_yaml.yaml
vendored
Normal file
3
internal/profile/testdata/invalid_yaml.yaml
vendored
Normal file
@@ -0,0 +1,3 @@
|
||||
id: invalid_yaml
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: [broken
|
||||
2
internal/profile/testdata/missing_endpoint.yaml
vendored
Normal file
2
internal/profile/testdata/missing_endpoint.yaml
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
id: missing-endpoint
|
||||
model: gpt-4o-mini
|
||||
2
internal/profile/testdata/missing_id.yaml
vendored
Normal file
2
internal/profile/testdata/missing_id.yaml
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: gpt-4o-mini
|
||||
2
internal/profile/testdata/missing_model.yaml
vendored
Normal file
2
internal/profile/testdata/missing_model.yaml
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
id: missing-model
|
||||
endpoint: http://localhost:8000/v1
|
||||
4
internal/profile/testdata/raw_api_key.yaml
vendored
Normal file
4
internal/profile/testdata/raw_api_key.yaml
vendored
Normal file
@@ -0,0 +1,4 @@
|
||||
id: raw-api-key
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: gpt-4o-mini
|
||||
api_key: super-secret-should-not-be-here
|
||||
4
internal/profile/testdata/unknown_field.yaml
vendored
Normal file
4
internal/profile/testdata/unknown_field.yaml
vendored
Normal file
@@ -0,0 +1,4 @@
|
||||
id: unknown-field
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: gpt-4o-mini
|
||||
foo: bar
|
||||
7
internal/profile/testdata/valid_local_profile.yaml
vendored
Normal file
7
internal/profile/testdata/valid_local_profile.yaml
vendored
Normal file
@@ -0,0 +1,7 @@
|
||||
id: local-default
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: gpt-4o-mini
|
||||
temperature: 0.2
|
||||
max_tokens: 700
|
||||
top_p: 1.0
|
||||
timeout_seconds: 120
|
||||
8
internal/profile/testdata/valid_with_api_key_env.yaml
vendored
Normal file
8
internal/profile/testdata/valid_with_api_key_env.yaml
vendored
Normal file
@@ -0,0 +1,8 @@
|
||||
id: local-secure
|
||||
endpoint: http://localhost:8000/v1
|
||||
model: gpt-4o-mini
|
||||
api_key_env: SCRIPTORIUM_API_KEY
|
||||
service_tier: priority
|
||||
reasoning_effort: medium
|
||||
extra_params:
|
||||
provider: local
|
||||
125
internal/prompt/go_renderer.go
Normal file
125
internal/prompt/go_renderer.go
Normal file
@@ -0,0 +1,125 @@
|
||||
package prompt
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||
"strings"
|
||||
"text/template"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrMissingRequiredInput = errors.New("missing required input artifact")
|
||||
ErrUnknownInput = errors.New("referenced unknown input artifact")
|
||||
ErrInvalidTemplate = errors.New("invalid prompt template")
|
||||
ErrRenderFailure = errors.New("prompt render failure")
|
||||
ErrInvalidMessageRole = errors.New("invalid or empty message role")
|
||||
)
|
||||
|
||||
type goRenderer struct{}
|
||||
|
||||
func NewGoRenderer() Renderer {
|
||||
return &goRenderer{}
|
||||
}
|
||||
|
||||
func (r *goRenderer) Render(ctx context.Context, definition *domain.PromptDefinition, inputs map[string]*domain.Artifact, vars map[string]string) (*domain.RenderedPrompt, error) {
|
||||
if definition == nil {
|
||||
return nil, fmt.Errorf("%w: nil prompt definition", ErrRenderFailure)
|
||||
}
|
||||
|
||||
// 1. Verify required inputs
|
||||
for _, in := range definition.Inputs {
|
||||
if !in.Required {
|
||||
continue
|
||||
}
|
||||
art, ok := inputs[in.Name]
|
||||
if !ok || art == nil {
|
||||
return nil, fmt.Errorf("%w: %s", ErrMissingRequiredInput, in.Name)
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Setup template functions
|
||||
funcs := template.FuncMap{
|
||||
"input": func(name string) (string, error) {
|
||||
art, ok := inputs[name]
|
||||
if !ok || art == nil {
|
||||
return "", fmt.Errorf("%w: %s", ErrUnknownInput, name)
|
||||
}
|
||||
return string(art.Body), nil
|
||||
},
|
||||
}
|
||||
|
||||
sessionID, err := renderSessionID(definition.SessionID, funcs, vars)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var renderedMessages []domain.RenderedMessage
|
||||
|
||||
for i, tmplMsg := range definition.Templates {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
default:
|
||||
}
|
||||
|
||||
if tmplMsg.Role == "" {
|
||||
return nil, fmt.Errorf("%w: message %d", ErrInvalidMessageRole, i)
|
||||
}
|
||||
|
||||
// Parse and execute template
|
||||
tmpl, err := template.New(fmt.Sprintf("msg_%d", i)).Funcs(funcs).Option("missingkey=error").Parse(tmplMsg.Content)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%w: message %d: %v", ErrInvalidTemplate, i, err)
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
if err := tmpl.Execute(&buf, vars); err != nil {
|
||||
return nil, fmt.Errorf("%w: message %d: %w", ErrRenderFailure, i, err)
|
||||
}
|
||||
|
||||
renderedMessages = append(renderedMessages, domain.RenderedMessage{
|
||||
Role: tmplMsg.Role,
|
||||
Content: buf.String(),
|
||||
CacheControl: cloneCacheControl(tmplMsg.CacheControl),
|
||||
})
|
||||
}
|
||||
|
||||
return &domain.RenderedPrompt{
|
||||
SessionID: sessionID,
|
||||
Messages: renderedMessages,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func renderSessionID(raw string, funcs template.FuncMap, vars map[string]string) (string, error) {
|
||||
if strings.TrimSpace(raw) == "" {
|
||||
return "", nil
|
||||
}
|
||||
|
||||
tmpl, err := template.New("session_id").Funcs(funcs).Option("missingkey=error").Parse(raw)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("%w: session_id: %v", ErrInvalidTemplate, err)
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
if err := tmpl.Execute(&buf, vars); err != nil {
|
||||
return "", fmt.Errorf("%w: session_id: %w", ErrRenderFailure, err)
|
||||
}
|
||||
|
||||
sessionID := strings.TrimSpace(buf.String())
|
||||
if n := utf8.RuneCountInString(sessionID); n > domain.SessionIDMaxLength {
|
||||
return "", fmt.Errorf("%w: session_id length %d exceeds maximum %d", ErrRenderFailure, n, domain.SessionIDMaxLength)
|
||||
}
|
||||
return sessionID, nil
|
||||
}
|
||||
|
||||
func cloneCacheControl(in *domain.CacheControl) *domain.CacheControl {
|
||||
if in == nil {
|
||||
return nil
|
||||
}
|
||||
out := *in
|
||||
return &out
|
||||
}
|
||||
11
internal/prompt/renderer.go
Normal file
11
internal/prompt/renderer.go
Normal file
@@ -0,0 +1,11 @@
|
||||
package prompt
|
||||
|
||||
import (
|
||||
"context"
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||
)
|
||||
|
||||
// Renderer renders prompt templates using named artifacts and variables.
|
||||
type Renderer interface {
|
||||
Render(ctx context.Context, definition *domain.PromptDefinition, inputs map[string]*domain.Artifact, vars map[string]string) (*domain.RenderedPrompt, error)
|
||||
}
|
||||
345
internal/prompt/renderer_test.go
Normal file
345
internal/prompt/renderer_test.go
Normal file
@@ -0,0 +1,345 @@
|
||||
package prompt
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||
)
|
||||
|
||||
func TestGoRenderer_Render(t *testing.T) {
|
||||
renderer := NewGoRenderer()
|
||||
ctx := context.Background()
|
||||
|
||||
inputs := map[string]*domain.Artifact{
|
||||
"transcript": {Body: []byte("The quick brown fox.")},
|
||||
}
|
||||
vars := map[string]string{
|
||||
"role": "helpful assistant",
|
||||
"tone": "concise",
|
||||
}
|
||||
|
||||
t.Run("rendering inline message content", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "user", Content: "Analyze this: {{input \"transcript\"}}"},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(res.Messages) != 1 {
|
||||
t.Fatalf("expected 1 message, got %d", len(res.Messages))
|
||||
}
|
||||
if res.Messages[0].Content != "Analyze this: The quick brown fox." {
|
||||
t.Fatalf("unexpected rendered content: %q", res.Messages[0].Content)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rendering file-backed message content loaded into prompt definition", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "user", Content: "From file: {{input \"transcript\"}}", ContentFile: "/tmp/user.tmpl"},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if got := res.Messages[0].Content; got != "From file: The quick brown fox." {
|
||||
t.Fatalf("unexpected file-backed render result: %q", got)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rendering system and user messages", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "You are a {{.role}}."},
|
||||
{Role: "user", Content: "Analyze this: {{input \"transcript\"}}"},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(res.Messages) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(res.Messages))
|
||||
}
|
||||
if res.Messages[0].Role != "system" || res.Messages[1].Role != "user" {
|
||||
t.Fatalf("unexpected roles: %#v", res.Messages)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("copying cache control to rendered messages", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{
|
||||
Role: "system",
|
||||
Content: "You are concise.",
|
||||
CacheControl: &domain.CacheControl{
|
||||
Type: domain.CacheControlEphemeral,
|
||||
TTL: "1h",
|
||||
},
|
||||
},
|
||||
{Role: "user", Content: "Analyze this: {{input \"transcript\"}}"},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(res.Messages) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(res.Messages))
|
||||
}
|
||||
if res.Messages[0].CacheControl == nil {
|
||||
t.Fatal("expected rendered cache control")
|
||||
}
|
||||
if res.Messages[0].CacheControl.Type != domain.CacheControlEphemeral {
|
||||
t.Fatalf("unexpected cache control type: %q", res.Messages[0].CacheControl.Type)
|
||||
}
|
||||
if res.Messages[0].CacheControl.TTL != "1h" {
|
||||
t.Fatalf("unexpected cache control ttl: %q", res.Messages[0].CacheControl.TTL)
|
||||
}
|
||||
if res.Messages[1].CacheControl != nil {
|
||||
t.Fatalf("expected no cache control on second message, got %#v", res.Messages[1].CacheControl)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rendered cache control does not alias source template", func(t *testing.T) {
|
||||
source := &domain.CacheControl{Type: domain.CacheControlEphemeral, TTL: "1h"}
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "You are concise.", CacheControl: source},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if res.Messages[0].CacheControl == source {
|
||||
t.Fatal("expected rendered cache control to be cloned")
|
||||
}
|
||||
|
||||
res.Messages[0].CacheControl.TTL = ""
|
||||
if source.TTL != "1h" {
|
||||
t.Fatalf("source cache control was mutated, ttl=%q", source.TTL)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("accessing vars", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "Speak in a {{.tone}} tone."},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if res.Messages[0].Content != "Speak in a concise tone." {
|
||||
t.Fatalf("unexpected vars rendering: %q", res.Messages[0].Content)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rendering session id from vars", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
SessionID: " {{ .session_id }} ",
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "Speak in a {{.tone}} tone."},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, map[string]string{
|
||||
"tone": "concise",
|
||||
"session_id": "agent-session-123",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if res.SessionID != "agent-session-123" {
|
||||
t.Fatalf("unexpected session id: %q", res.SessionID)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("empty rendered session id is omitted", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
SessionID: " ",
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "Speak in a {{.tone}} tone."},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if res.SessionID != "" {
|
||||
t.Fatalf("expected empty session id, got %q", res.SessionID)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("missing session id var fails rendering", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
SessionID: "{{ .session_id }}",
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "Speak in a {{.tone}} tone."},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if !errors.Is(err, ErrRenderFailure) {
|
||||
t.Fatalf("expected ErrRenderFailure, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("too long rendered session id fails rendering", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
SessionID: "{{ .session_id }}",
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "system", Content: "Speak in a {{.tone}} tone."},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := renderer.Render(ctx, def, inputs, map[string]string{
|
||||
"tone": "concise",
|
||||
"session_id": strings.Repeat("x", domain.SessionIDMaxLength+1),
|
||||
})
|
||||
if !errors.Is(err, ErrRenderFailure) {
|
||||
t.Fatalf("expected ErrRenderFailure, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("inserting required input artifact", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "user", Content: "{{input \"transcript\"}}"},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if res.Messages[0].Content != "The quick brown fox." {
|
||||
t.Fatalf("unexpected required input rendering: %q", res.Messages[0].Content)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("optional input absent and not referenced", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{
|
||||
{Name: "transcript", Required: true},
|
||||
{Name: "glossary", Required: false},
|
||||
},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "user", Content: "Transcript: {{input \"transcript\"}}"},
|
||||
},
|
||||
}
|
||||
|
||||
res, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if len(res.Messages) != 1 {
|
||||
t.Fatalf("expected one rendered message, got %d", len(res.Messages))
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("optional input absent but referenced, expecting failure", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{
|
||||
{Name: "transcript", Required: true},
|
||||
{Name: "glossary", Required: false},
|
||||
},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "user", Content: "Glossary: {{input \"glossary\"}}"},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if !errors.Is(err, ErrRenderFailure) {
|
||||
t.Fatalf("expected ErrRenderFailure, got %v", err)
|
||||
}
|
||||
if !errors.Is(err, ErrUnknownInput) {
|
||||
t.Fatalf("expected ErrUnknownInput, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("required input missing, expecting failure", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "user", Content: "Analyze this: {{input \"transcript\"}}"},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := renderer.Render(ctx, def, map[string]*domain.Artifact{}, vars)
|
||||
if !errors.Is(err, ErrMissingRequiredInput) {
|
||||
t.Fatalf("expected ErrMissingRequiredInput, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("invalid template syntax", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "user", Content: "Hello {{.unclosed"},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if !errors.Is(err, ErrInvalidTemplate) {
|
||||
t.Fatalf("expected ErrInvalidTemplate, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("unknown input reference", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "user", Content: "Hello {{input \"ghost\"}}"},
|
||||
},
|
||||
}
|
||||
|
||||
_, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if !errors.Is(err, ErrRenderFailure) {
|
||||
t.Fatalf("expected ErrRenderFailure, got %v", err)
|
||||
}
|
||||
if !errors.Is(err, ErrUnknownInput) {
|
||||
t.Fatalf("expected ErrUnknownInput, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("empty message role", func(t *testing.T) {
|
||||
def := &domain.PromptDefinition{
|
||||
Inputs: []domain.PromptInput{{Name: "transcript", Required: true}},
|
||||
Templates: []domain.PromptMessageTemplate{
|
||||
{Role: "", Content: "Hello"},
|
||||
},
|
||||
}
|
||||
_, err := renderer.Render(ctx, def, inputs, vars)
|
||||
if !errors.Is(err, ErrInvalidMessageRole) {
|
||||
t.Fatalf("expected ErrInvalidMessageRole, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
484
internal/promptdef/filesystem_repository.go
Normal file
484
internal/promptdef/filesystem_repository.go
Normal file
@@ -0,0 +1,484 @@
|
||||
package promptdef
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/filecatalog"
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrPromptDefinitionNotFound = errors.New("prompt definition not found")
|
||||
ErrInvalidYAML = errors.New("invalid YAML format")
|
||||
ErrInvalidPromptDefinition = errors.New("invalid prompt definition configuration")
|
||||
)
|
||||
|
||||
type filesystemRepository struct {
|
||||
dir string
|
||||
}
|
||||
|
||||
type fsRepository struct {
|
||||
fsys fs.FS
|
||||
root string
|
||||
}
|
||||
|
||||
type promptDefinitionFile struct {
|
||||
ID string `yaml:"id"`
|
||||
Version string `yaml:"version"`
|
||||
DefaultProfile *string `yaml:"default_profile"`
|
||||
Description string `yaml:"description"`
|
||||
SessionID string `yaml:"session_id"`
|
||||
Inputs []promptInputFile `yaml:"inputs"`
|
||||
Messages []promptMessageFile `yaml:"messages"`
|
||||
Output promptOutputContractFile `yaml:"output"`
|
||||
}
|
||||
|
||||
type promptInputFile struct {
|
||||
Name string `yaml:"name"`
|
||||
Required bool `yaml:"required"`
|
||||
ContentType string `yaml:"content_type"`
|
||||
Description string `yaml:"description"`
|
||||
}
|
||||
|
||||
type promptMessageFile struct {
|
||||
Role string `yaml:"role"`
|
||||
Content string `yaml:"content"`
|
||||
ContentFile string `yaml:"content_file"`
|
||||
CacheControl *cacheControlFile `yaml:"cache_control"`
|
||||
}
|
||||
|
||||
type cacheControlFile struct {
|
||||
Type string `yaml:"type"`
|
||||
TTL string `yaml:"ttl"`
|
||||
}
|
||||
|
||||
type promptOutputContractFile struct {
|
||||
Format domain.OutputFormat `yaml:"format"`
|
||||
ValidationMode domain.ValidationMode `yaml:"validation_mode"`
|
||||
SchemaPath string `yaml:"schema_path"`
|
||||
RepairAttempts int `yaml:"repair_attempts"`
|
||||
}
|
||||
|
||||
func NewFilesystemRepository(dir string) Repository {
|
||||
return &filesystemRepository{dir: dir}
|
||||
}
|
||||
|
||||
func NewFSRepository(fsys fs.FS, root string) Repository {
|
||||
return &fsRepository{fsys: fsys, root: root}
|
||||
}
|
||||
|
||||
func (r *filesystemRepository) GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error) {
|
||||
if strings.TrimSpace(id) == "" {
|
||||
return nil, fmt.Errorf("%w: prompt id is required", ErrInvalidPromptDefinition)
|
||||
}
|
||||
|
||||
files, err := filecatalog.FindYAMLFiles(ctx, r.dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read prompt definition directory: %w", err)
|
||||
}
|
||||
|
||||
var matches []promptDefinitionMatch
|
||||
for _, fullPath := range files {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
default:
|
||||
}
|
||||
|
||||
relPath := filecatalog.RelativePath(r.dir, fullPath)
|
||||
fileMatch := filecatalog.Stem(filepath.Base(fullPath)) == id
|
||||
|
||||
raw, err := loadPromptDefinitionFile(fullPath)
|
||||
if err != nil {
|
||||
if fileMatch || promptDefinitionFileHasID(fullPath, id) {
|
||||
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidYAML, relPath, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
def, err := normalizePromptDefinition(raw, fullPath)
|
||||
if err != nil {
|
||||
if fileMatch || strings.TrimSpace(raw.ID) == id {
|
||||
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidPromptDefinition, relPath, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if def.ID != id {
|
||||
continue
|
||||
}
|
||||
if version != "" && def.Version != version {
|
||||
continue
|
||||
}
|
||||
matches = append(matches, promptDefinitionMatch{
|
||||
def: def,
|
||||
path: relPath,
|
||||
})
|
||||
}
|
||||
|
||||
if len(matches) > 1 {
|
||||
paths := make([]string, 0, len(matches))
|
||||
for _, match := range matches {
|
||||
paths = append(paths, match.path)
|
||||
}
|
||||
if version != "" {
|
||||
return nil, fmt.Errorf("%w: duplicate prompt definition id %q version %q found in: %s", ErrInvalidPromptDefinition, id, version, strings.Join(paths, ", "))
|
||||
}
|
||||
return nil, fmt.Errorf("%w: duplicate prompt definition id %q found in: %s", ErrInvalidPromptDefinition, id, strings.Join(paths, ", "))
|
||||
}
|
||||
|
||||
if len(matches) == 1 {
|
||||
return matches[0].def, nil
|
||||
}
|
||||
|
||||
return nil, ErrPromptDefinitionNotFound
|
||||
}
|
||||
|
||||
func (r *fsRepository) GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error) {
|
||||
return loadPromptDefinition(ctx, r.fsys, r.root, id, version)
|
||||
}
|
||||
|
||||
type promptDefinitionMatch struct {
|
||||
def *domain.PromptDefinition
|
||||
path string
|
||||
}
|
||||
|
||||
func loadPromptDefinitionFile(path string) (*promptDefinitionFile, error) {
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read prompt definition file: %w", err)
|
||||
}
|
||||
|
||||
var raw promptDefinitionFile
|
||||
decoder := yaml.NewDecoder(bytes.NewReader(data))
|
||||
decoder.KnownFields(true)
|
||||
if err := decoder.Decode(&raw); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &raw, nil
|
||||
}
|
||||
|
||||
func promptDefinitionFileHasID(path string, id string) bool {
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
var raw struct {
|
||||
ID string `yaml:"id"`
|
||||
}
|
||||
if err := yaml.NewDecoder(bytes.NewReader(data)).Decode(&raw); err != nil {
|
||||
return false
|
||||
}
|
||||
return strings.TrimSpace(raw.ID) == id
|
||||
}
|
||||
|
||||
func loadPromptDefinition(ctx context.Context, fsys fs.FS, root string, id string, version string) (*domain.PromptDefinition, error) {
|
||||
if strings.TrimSpace(id) == "" {
|
||||
return nil, fmt.Errorf("%w: prompt id is required", ErrInvalidPromptDefinition)
|
||||
}
|
||||
if fsys == nil {
|
||||
return nil, fmt.Errorf("failed to read prompt definition directory: filesystem is nil")
|
||||
}
|
||||
|
||||
files, err := filecatalog.FindFSYAMLFiles(ctx, fsys, root)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read prompt definition directory: %w", err)
|
||||
}
|
||||
cleanRoot := filecatalog.CleanFSRoot(root)
|
||||
rootInfo, err := fs.Stat(fsys, cleanRoot)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read prompt definition directory: %w", err)
|
||||
}
|
||||
|
||||
var matches []promptDefinitionMatch
|
||||
for _, fullPath := range files {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
default:
|
||||
}
|
||||
|
||||
relPath := filecatalog.DisplayPath(root, fullPath)
|
||||
fileMatch := filecatalog.Stem(path.Base(fullPath)) == id
|
||||
data, err := fs.ReadFile(fsys, fullPath)
|
||||
if err != nil {
|
||||
if fileMatch {
|
||||
return nil, fmt.Errorf("%w: %s: failed to read prompt definition file: %v", ErrInvalidYAML, relPath, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
raw, err := decodePromptDefinition(data)
|
||||
if err != nil {
|
||||
if fileMatch || promptDefinitionDataHasID(data, id) {
|
||||
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidYAML, relPath, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
def, err := normalizePromptDefinitionFromFS(raw, fsys, root, fullPath, rootInfo.IsDir())
|
||||
if err != nil {
|
||||
if fileMatch || strings.TrimSpace(raw.ID) == id {
|
||||
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidPromptDefinition, relPath, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if def.ID != id {
|
||||
continue
|
||||
}
|
||||
if version != "" && def.Version != version {
|
||||
continue
|
||||
}
|
||||
matches = append(matches, promptDefinitionMatch{
|
||||
def: def,
|
||||
path: relPath,
|
||||
})
|
||||
}
|
||||
|
||||
if len(matches) > 1 {
|
||||
paths := make([]string, 0, len(matches))
|
||||
for _, match := range matches {
|
||||
paths = append(paths, match.path)
|
||||
}
|
||||
if version != "" {
|
||||
return nil, fmt.Errorf("%w: duplicate prompt definition id %q version %q found in: %s", ErrInvalidPromptDefinition, id, version, strings.Join(paths, ", "))
|
||||
}
|
||||
return nil, fmt.Errorf("%w: duplicate prompt definition id %q found in: %s", ErrInvalidPromptDefinition, id, strings.Join(paths, ", "))
|
||||
}
|
||||
|
||||
if len(matches) == 1 {
|
||||
return matches[0].def, nil
|
||||
}
|
||||
|
||||
return nil, ErrPromptDefinitionNotFound
|
||||
}
|
||||
|
||||
func decodePromptDefinition(data []byte) (*promptDefinitionFile, error) {
|
||||
var raw promptDefinitionFile
|
||||
decoder := yaml.NewDecoder(bytes.NewReader(data))
|
||||
decoder.KnownFields(true)
|
||||
if err := decoder.Decode(&raw); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &raw, nil
|
||||
}
|
||||
|
||||
func promptDefinitionDataHasID(data []byte, id string) bool {
|
||||
var raw struct {
|
||||
ID string `yaml:"id"`
|
||||
}
|
||||
if err := yaml.NewDecoder(bytes.NewReader(data)).Decode(&raw); err != nil {
|
||||
return false
|
||||
}
|
||||
return strings.TrimSpace(raw.ID) == id
|
||||
}
|
||||
|
||||
func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*domain.PromptDefinition, error) {
|
||||
promptDir := filepath.Dir(sourcePath)
|
||||
return normalizePromptDefinitionWithContent(raw, func(contentFile string) (string, string, error) {
|
||||
resolvedPath := strings.TrimSpace(contentFile)
|
||||
if !filepath.IsAbs(resolvedPath) {
|
||||
resolvedPath = filepath.Join(promptDir, resolvedPath)
|
||||
}
|
||||
resolvedPath = filepath.Clean(resolvedPath)
|
||||
|
||||
body, err := os.ReadFile(resolvedPath)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
return string(body), resolvedPath, nil
|
||||
})
|
||||
}
|
||||
|
||||
func normalizePromptDefinitionFromFS(raw *promptDefinitionFile, fsys fs.FS, root string, sourcePath string, rootIsDir bool) (*domain.PromptDefinition, error) {
|
||||
promptDir := path.Dir(sourcePath)
|
||||
return normalizePromptDefinitionWithContent(raw, func(contentFile string) (string, string, error) {
|
||||
var resolvedPath string
|
||||
if rootIsDir {
|
||||
var err error
|
||||
resolvedPath, _, err = filecatalog.ResolveFSPath(root, promptDir, contentFile)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
} else {
|
||||
resolvedPath = strings.TrimSpace(contentFile)
|
||||
if !path.IsAbs(resolvedPath) {
|
||||
resolvedPath = path.Join(promptDir, resolvedPath)
|
||||
}
|
||||
resolvedPath = strings.TrimPrefix(path.Clean(resolvedPath), "/")
|
||||
}
|
||||
|
||||
body, err := fs.ReadFile(fsys, resolvedPath)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
return string(body), resolvedPath, nil
|
||||
})
|
||||
}
|
||||
|
||||
func normalizePromptDefinitionWithContent(raw *promptDefinitionFile, readContentFile func(string) (string, string, error)) (*domain.PromptDefinition, error) {
|
||||
if raw == nil {
|
||||
return nil, errors.New("prompt definition is nil")
|
||||
}
|
||||
|
||||
id := strings.TrimSpace(raw.ID)
|
||||
if id == "" {
|
||||
return nil, errors.New("id is required")
|
||||
}
|
||||
|
||||
version := strings.TrimSpace(raw.Version)
|
||||
if version == "" {
|
||||
return nil, errors.New("version is required")
|
||||
}
|
||||
|
||||
if len(raw.Messages) == 0 {
|
||||
return nil, errors.New("at least one message is required")
|
||||
}
|
||||
|
||||
inputs := make([]domain.PromptInput, 0, len(raw.Inputs))
|
||||
seenInputNames := make(map[string]struct{}, len(raw.Inputs))
|
||||
for i, in := range raw.Inputs {
|
||||
name := strings.TrimSpace(in.Name)
|
||||
if name == "" {
|
||||
return nil, fmt.Errorf("input %d has empty name", i)
|
||||
}
|
||||
if _, exists := seenInputNames[name]; exists {
|
||||
return nil, fmt.Errorf("duplicate input name %q", name)
|
||||
}
|
||||
seenInputNames[name] = struct{}{}
|
||||
|
||||
inputs = append(inputs, domain.PromptInput{
|
||||
Name: name,
|
||||
Required: in.Required,
|
||||
ContentType: strings.TrimSpace(in.ContentType),
|
||||
Description: strings.TrimSpace(in.Description),
|
||||
})
|
||||
}
|
||||
|
||||
templates := make([]domain.PromptMessageTemplate, 0, len(raw.Messages))
|
||||
for i, msg := range raw.Messages {
|
||||
role := strings.TrimSpace(msg.Role)
|
||||
if role == "" {
|
||||
return nil, fmt.Errorf("message %d role is required", i)
|
||||
}
|
||||
|
||||
hasContent := strings.TrimSpace(msg.Content) != ""
|
||||
hasContentFile := strings.TrimSpace(msg.ContentFile) != ""
|
||||
if hasContent == hasContentFile {
|
||||
return nil, fmt.Errorf("message %d (%s) must set exactly one of content or content_file", i, role)
|
||||
}
|
||||
|
||||
cacheControl, err := normalizeCacheControl(msg.CacheControl)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("message %d (%s) cache_control: %w", i, role, err)
|
||||
}
|
||||
|
||||
templateContent := msg.Content
|
||||
resolvedContentFile := ""
|
||||
if hasContentFile {
|
||||
body, resolvedPath, err := readContentFile(msg.ContentFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("prompt %q message %d (%s): failed to read content_file %q: %w", id, i, role, msg.ContentFile, err)
|
||||
}
|
||||
templateContent = body
|
||||
resolvedContentFile = resolvedPath
|
||||
}
|
||||
|
||||
templates = append(templates, domain.PromptMessageTemplate{
|
||||
Role: role,
|
||||
Content: templateContent,
|
||||
ContentFile: resolvedContentFile,
|
||||
CacheControl: cacheControl,
|
||||
})
|
||||
}
|
||||
|
||||
if !isValidOutputFormat(raw.Output.Format) {
|
||||
return nil, fmt.Errorf("invalid output format: %q", raw.Output.Format)
|
||||
}
|
||||
if !isValidValidationMode(raw.Output.ValidationMode) {
|
||||
return nil, fmt.Errorf("invalid validation mode: %q", raw.Output.ValidationMode)
|
||||
}
|
||||
if raw.Output.ValidationMode == domain.ValidationJSONSchema && strings.TrimSpace(raw.Output.SchemaPath) == "" {
|
||||
return nil, errors.New("output.schema_path is required when output.validation_mode is json_schema")
|
||||
}
|
||||
if raw.Output.RepairAttempts < 0 {
|
||||
return nil, errors.New("output.repair_attempts must be greater than or equal to 0")
|
||||
}
|
||||
|
||||
defaultProfile := ""
|
||||
if raw.DefaultProfile != nil {
|
||||
defaultProfile = strings.TrimSpace(*raw.DefaultProfile)
|
||||
if defaultProfile == "" {
|
||||
return nil, errors.New("default_profile must be a non-empty string when set")
|
||||
}
|
||||
}
|
||||
|
||||
return &domain.PromptDefinition{
|
||||
ID: id,
|
||||
Version: version,
|
||||
DefaultProfile: defaultProfile,
|
||||
Description: strings.TrimSpace(raw.Description),
|
||||
SessionID: strings.TrimSpace(raw.SessionID),
|
||||
Inputs: inputs,
|
||||
Templates: templates,
|
||||
OutputFormat: raw.Output.Format,
|
||||
Validation: domain.OutputContract{
|
||||
Format: raw.Output.Format,
|
||||
ValidationMode: raw.Output.ValidationMode,
|
||||
SchemaPath: strings.TrimSpace(raw.Output.SchemaPath),
|
||||
RepairAttempts: raw.Output.RepairAttempts,
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func normalizeCacheControl(raw *cacheControlFile) (*domain.CacheControl, error) {
|
||||
if raw == nil {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
cacheType := strings.TrimSpace(raw.Type)
|
||||
if cacheType == "" {
|
||||
return nil, errors.New("type is required")
|
||||
}
|
||||
if domain.CacheControlType(cacheType) != domain.CacheControlEphemeral {
|
||||
return nil, fmt.Errorf("unsupported type %q", cacheType)
|
||||
}
|
||||
|
||||
ttl := strings.TrimSpace(raw.TTL)
|
||||
if ttl != "" && ttl != "1h" {
|
||||
return nil, fmt.Errorf("unsupported ttl %q", ttl)
|
||||
}
|
||||
|
||||
return &domain.CacheControl{
|
||||
Type: domain.CacheControlType(cacheType),
|
||||
TTL: ttl,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func isValidOutputFormat(f domain.OutputFormat) bool {
|
||||
switch f {
|
||||
case domain.FormatText, domain.FormatMarkdown, domain.FormatJSON:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func isValidValidationMode(m domain.ValidationMode) bool {
|
||||
switch m {
|
||||
case domain.ValidationNone, domain.ValidationBasic, domain.ValidationJSON, domain.ValidationJSONSchema:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
12
internal/promptdef/repository.go
Normal file
12
internal/promptdef/repository.go
Normal file
@@ -0,0 +1,12 @@
|
||||
package promptdef
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||
)
|
||||
|
||||
// Repository loads prompt definitions.
|
||||
type Repository interface {
|
||||
GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error)
|
||||
}
|
||||
526
internal/promptdef/repository_test.go
Normal file
526
internal/promptdef/repository_test.go
Normal file
@@ -0,0 +1,526 @@
|
||||
package promptdef
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
"gitea.maximumdirect.net/eric/promptkit/internal/domain"
|
||||
)
|
||||
|
||||
func TestFilesystemRepository_GetPromptDefinition(t *testing.T) {
|
||||
tmpDir := t.TempDir()
|
||||
if err := copyTree("testdata", tmpDir); err != nil {
|
||||
t.Fatalf("failed to copy testdata: %v", err)
|
||||
}
|
||||
|
||||
repo := NewFilesystemRepository(tmpDir)
|
||||
ctx := context.Background()
|
||||
|
||||
t.Run("valid inline prompt", func(t *testing.T) {
|
||||
p, err := repo.GetPromptDefinition(ctx, "valid-inline", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.ID != "valid-inline" {
|
||||
t.Fatalf("unexpected id: %q", p.ID)
|
||||
}
|
||||
if p.Version != "1.0.0" {
|
||||
t.Fatalf("unexpected version: %q", p.Version)
|
||||
}
|
||||
if p.OutputFormat != domain.FormatMarkdown {
|
||||
t.Fatalf("unexpected output format: %q", p.OutputFormat)
|
||||
}
|
||||
if p.Validation.ValidationMode != domain.ValidationBasic {
|
||||
t.Fatalf("unexpected validation mode: %q", p.Validation.ValidationMode)
|
||||
}
|
||||
if len(p.Templates) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(p.Templates))
|
||||
}
|
||||
if len(p.Inputs) != 1 {
|
||||
t.Fatalf("expected 1 input, got %d", len(p.Inputs))
|
||||
}
|
||||
if p.Inputs[0].ContentType != "text/markdown" {
|
||||
t.Fatalf("expected input content_type to be preserved, got %q", p.Inputs[0].ContentType)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid file-backed prompt", func(t *testing.T) {
|
||||
p, err := repo.GetPromptDefinition(ctx, "valid-file-backed", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if len(p.Templates) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(p.Templates))
|
||||
}
|
||||
if !strings.Contains(p.Templates[1].Content, "{{input \"transcript\"}}") {
|
||||
t.Fatalf("expected content_file template body to be loaded, got %q", p.Templates[1].Content)
|
||||
}
|
||||
if p.Templates[1].ContentFile == "" {
|
||||
t.Fatal("expected ContentFile source metadata to be preserved")
|
||||
}
|
||||
if !filepath.IsAbs(p.Templates[1].ContentFile) {
|
||||
t.Fatalf("expected resolved content_file path to be absolute, got %q", p.Templates[1].ContentFile)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid cache control with ttl", func(t *testing.T) {
|
||||
p, err := repo.GetPromptDefinition(ctx, "valid-cache-control-ttl", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if len(p.Templates) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(p.Templates))
|
||||
}
|
||||
assertCacheControl(t, p.Templates[0].CacheControl, domain.CacheControlEphemeral, "1h")
|
||||
if p.Templates[1].CacheControl != nil {
|
||||
t.Fatalf("expected second message cache control to be nil, got %#v", p.Templates[1].CacheControl)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid cache control without ttl", func(t *testing.T) {
|
||||
p, err := repo.GetPromptDefinition(ctx, "valid-cache-control-without-ttl", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if len(p.Templates) != 2 {
|
||||
t.Fatalf("expected 2 messages, got %d", len(p.Templates))
|
||||
}
|
||||
assertCacheControl(t, p.Templates[0].CacheControl, domain.CacheControlEphemeral, "")
|
||||
if p.Templates[1].CacheControl != nil {
|
||||
t.Fatalf("expected second message cache control to be nil, got %#v", p.Templates[1].CacheControl)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid session id template", func(t *testing.T) {
|
||||
p, err := repo.GetPromptDefinition(ctx, "valid-session-id", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.SessionID != "{{ .session_id }}" {
|
||||
t.Fatalf("expected trimmed session_id template, got %q", p.SessionID)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("valid nested file-backed prompt resolves content file relative to nested YAML", func(t *testing.T) {
|
||||
nestedDir := filepath.Join(tmpDir, "dnd", "recap")
|
||||
if err := os.MkdirAll(nestedDir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writePromptTestFile(t, filepath.Join(nestedDir, "nested_recap.yaml"), `
|
||||
id: nested-recap
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content_file: ./nested_recap.user.tmpl
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)
|
||||
writePromptTestFile(t, filepath.Join(nestedDir, "nested_recap.user.tmpl"), `Nested recap: {{input "transcript"}}`)
|
||||
|
||||
p, err := repo.GetPromptDefinition(ctx, "nested-recap", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if len(p.Templates) != 1 {
|
||||
t.Fatalf("expected one template, got %d", len(p.Templates))
|
||||
}
|
||||
if !strings.Contains(p.Templates[0].Content, "Nested recap") {
|
||||
t.Fatalf("expected nested content file body, got %q", p.Templates[0].Content)
|
||||
}
|
||||
if !strings.Contains(p.Templates[0].ContentFile, filepath.Join("dnd", "recap", "nested_recap.user.tmpl")) {
|
||||
t.Fatalf("expected nested content file path, got %q", p.Templates[0].ContentFile)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("prompt with default_profile", func(t *testing.T) {
|
||||
p, err := repo.GetPromptDefinition(ctx, "with-default-profile", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if p.DefaultProfile != "local-default" {
|
||||
t.Fatalf("unexpected default profile: %q", p.DefaultProfile)
|
||||
}
|
||||
if len(p.Inputs) != 1 {
|
||||
t.Fatalf("expected one input, got %d", len(p.Inputs))
|
||||
}
|
||||
if p.Inputs[0].ContentType != "" {
|
||||
t.Fatalf("expected missing content_type to remain empty, got %q", p.Inputs[0].ContentType)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("duplicate prompt IDs fail as ambiguous", func(t *testing.T) {
|
||||
writePromptTestFile(t, filepath.Join(tmpDir, "duplicate_a.yaml"), `
|
||||
id: duplicate-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: First duplicate.
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)
|
||||
nestedDir := filepath.Join(tmpDir, "nested")
|
||||
if err := os.MkdirAll(nestedDir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writePromptTestFile(t, filepath.Join(nestedDir, "duplicate_b.yaml"), `
|
||||
id: duplicate-prompt
|
||||
version: "2.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: Second duplicate.
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)
|
||||
|
||||
_, err := repo.GetPromptDefinition(ctx, "duplicate-prompt", "")
|
||||
if !errors.Is(err, ErrInvalidPromptDefinition) {
|
||||
t.Fatalf("expected duplicate prompt to return ErrInvalidPromptDefinition, got %v", err)
|
||||
}
|
||||
for _, want := range []string{"duplicate prompt definition id", "duplicate_a.yaml", filepath.Join("nested", "duplicate_b.yaml")} {
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Fatalf("expected error to contain %q, got %v", want, err)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("duplicate prompt ID and requested version fails as ambiguous", func(t *testing.T) {
|
||||
writePromptTestFile(t, filepath.Join(tmpDir, "version_duplicate_a.yaml"), `
|
||||
id: duplicate-version-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: First duplicate version.
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)
|
||||
nestedDir := filepath.Join(tmpDir, "versioned")
|
||||
if err := os.MkdirAll(nestedDir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writePromptTestFile(t, filepath.Join(nestedDir, "version_duplicate_b.yaml"), `
|
||||
id: duplicate-version-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: Second duplicate version.
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)
|
||||
|
||||
_, err := repo.GetPromptDefinition(ctx, "duplicate-version-prompt", "1.0.0")
|
||||
if !errors.Is(err, ErrInvalidPromptDefinition) {
|
||||
t.Fatalf("expected duplicate prompt version to return ErrInvalidPromptDefinition, got %v", err)
|
||||
}
|
||||
for _, want := range []string{"duplicate prompt definition id", "version \"1.0.0\"", "version_duplicate_a.yaml", filepath.Join("versioned", "version_duplicate_b.yaml")} {
|
||||
if !strings.Contains(err.Error(), want) {
|
||||
t.Fatalf("expected error to contain %q, got %v", want, err)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("non-matching malformed nested prompt is ignored for not found lookup", func(t *testing.T) {
|
||||
nestedDir := filepath.Join(tmpDir, "broken")
|
||||
if err := os.MkdirAll(nestedDir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writePromptTestFile(t, filepath.Join(nestedDir, "unrelated.yaml"), "id: [")
|
||||
|
||||
_, err := repo.GetPromptDefinition(ctx, "does-not-exist-even-with-broken-nested-file", "")
|
||||
if !errors.Is(err, ErrPromptDefinitionNotFound) {
|
||||
t.Fatalf("expected ErrPromptDefinitionNotFound, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("strict decode failure in nested prompt matches by YAML ID", func(t *testing.T) {
|
||||
nestedDir := filepath.Join(tmpDir, "strict")
|
||||
if err := os.MkdirAll(nestedDir, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writePromptTestFile(t, filepath.Join(nestedDir, "not_named_like_id.yaml"), `
|
||||
id: nested-strict-error
|
||||
version: "1.0.0"
|
||||
unknown_field: true
|
||||
messages:
|
||||
- role: user
|
||||
content: Invalid because of unknown field.
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)
|
||||
|
||||
_, err := repo.GetPromptDefinition(ctx, "nested-strict-error", "")
|
||||
if !errors.Is(err, ErrInvalidYAML) {
|
||||
t.Fatalf("expected ErrInvalidYAML, got %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), filepath.Join("strict", "not_named_like_id.yaml")) {
|
||||
t.Fatalf("expected nested path in error, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("version lookup", func(t *testing.T) {
|
||||
_, err := repo.GetPromptDefinition(ctx, "valid-inline", "9.9.9")
|
||||
if !errors.Is(err, ErrPromptDefinitionNotFound) {
|
||||
t.Fatalf("expected ErrPromptDefinitionNotFound, got %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
id string
|
||||
targetErr error
|
||||
errSubstrs []string
|
||||
}{
|
||||
{name: "invalid YAML", id: "invalid_yaml", targetErr: ErrInvalidYAML},
|
||||
{name: "missing id", id: "missing_id", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"id is required"}},
|
||||
{name: "no messages", id: "no_messages", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"at least one message is required"}},
|
||||
{name: "both content and content_file", id: "both_content_and_content_file", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"exactly one"}},
|
||||
{name: "neither content nor content_file", id: "neither_content_nor_content_file", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"exactly one"}},
|
||||
{name: "missing content_file", id: "missing_content_file", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"failed to read content_file"}},
|
||||
{name: "duplicate input names", id: "duplicate_input_names", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"duplicate input name"}},
|
||||
{name: "invalid validation mode", id: "invalid_validation_mode", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"invalid validation mode"}},
|
||||
{name: "json_schema without schema_path", id: "json_schema_without_schema_path", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"schema_path"}},
|
||||
{name: "unknown input field", id: "unknown_input_field", targetErr: ErrInvalidYAML, errSubstrs: []string{"field unknown_input_setting not found"}},
|
||||
{name: "empty cache control type", id: "empty_cache_control_type", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"cache_control", "type is required"}},
|
||||
{name: "unsupported cache control type", id: "unsupported_cache_control_type", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"cache_control", "unsupported type"}},
|
||||
{name: "unsupported cache control ttl", id: "unsupported_cache_control_ttl", targetErr: ErrInvalidPromptDefinition, errSubstrs: []string{"cache_control", "unsupported ttl"}},
|
||||
{name: "unknown cache control field", id: "unknown_cache_control_field", targetErr: ErrInvalidYAML, errSubstrs: []string{"field unexpected not found"}},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
_, err := repo.GetPromptDefinition(ctx, tc.id, "")
|
||||
if !errors.Is(err, tc.targetErr) {
|
||||
t.Fatalf("expected %v, got %v", tc.targetErr, err)
|
||||
}
|
||||
for _, sub := range tc.errSubstrs {
|
||||
if !strings.Contains(err.Error(), sub) {
|
||||
t.Fatalf("expected error to contain %q, got %v", sub, err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
t.Run("prompt definition not found", func(t *testing.T) {
|
||||
_, err := repo.GetPromptDefinition(ctx, "does-not-exist", "")
|
||||
if !errors.Is(err, ErrPromptDefinitionNotFound) {
|
||||
t.Fatalf("expected ErrPromptDefinitionNotFound, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestFSRepositoryGetPromptDefinition(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"prompts/nested/prompt.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: fs-prompt
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
messages:
|
||||
- role: user
|
||||
content_file: ./messages/user.tmpl
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
"prompts/nested/messages/user.tmpl": &fstest.MapFile{Data: []byte(`Summarize {{input "transcript"}}.`)},
|
||||
}, "prompts")
|
||||
|
||||
got, err := repo.GetPromptDefinition(context.Background(), "fs-prompt", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if got.ID != "fs-prompt" {
|
||||
t.Fatalf("unexpected prompt id: %q", got.ID)
|
||||
}
|
||||
if len(got.Templates) != 1 || !strings.Contains(got.Templates[0].Content, `{{input "transcript"}}`) {
|
||||
t.Fatalf("expected content_file body to be loaded, got %+v", got.Templates)
|
||||
}
|
||||
if got.Templates[0].ContentFile != "prompts/nested/messages/user.tmpl" {
|
||||
t.Fatalf("unexpected content file path: %q", got.Templates[0].ContentFile)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSRepositoryContentFileContainment(t *testing.T) {
|
||||
t.Run("nested prompt can reference file inside root", func(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"prompts/nested/prompt.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: fs-contained-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content_file: ../shared/user.tmpl
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
"prompts/shared/user.tmpl": &fstest.MapFile{Data: []byte(`Inside root.`)},
|
||||
}, "prompts")
|
||||
|
||||
got, err := repo.GetPromptDefinition(context.Background(), "fs-contained-prompt", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if len(got.Templates) != 1 || got.Templates[0].Content != "Inside root." {
|
||||
t.Fatalf("expected contained content file, got %+v", got.Templates)
|
||||
}
|
||||
})
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
contentFile string
|
||||
wantErr string
|
||||
}{
|
||||
{name: "parent escape rejected", contentFile: "../outside.tmpl", wantErr: "escapes source root"},
|
||||
{name: "absolute path rejected", contentFile: "/outside.tmpl", wantErr: "must be relative"},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"prompts/prompt.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: fs-escaped-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content_file: ` + tc.contentFile + `
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
"outside.tmpl": &fstest.MapFile{Data: []byte(`Outside root.`)},
|
||||
}, "prompts")
|
||||
|
||||
_, err := repo.GetPromptDefinition(context.Background(), "fs-escaped-prompt", "")
|
||||
if !errors.Is(err, ErrInvalidPromptDefinition) {
|
||||
t.Fatalf("expected ErrInvalidPromptDefinition, got %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), tc.wantErr) {
|
||||
t.Fatalf("expected error to contain %q, got %v", tc.wantErr, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSRepositoryRejectsDuplicatePromptIDs(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"one.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: duplicate-fs-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: First.
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
"nested/two.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: duplicate-fs-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: Second.
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
}, ".")
|
||||
|
||||
_, err := repo.GetPromptDefinition(context.Background(), "duplicate-fs-prompt", "")
|
||||
if !errors.Is(err, ErrInvalidPromptDefinition) {
|
||||
t.Fatalf("expected ErrInvalidPromptDefinition, got %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "one.yaml") || !strings.Contains(err.Error(), "nested/two.yaml") {
|
||||
t.Fatalf("expected duplicate paths in error, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSRepositoryRejectsUnknownYAMLFields(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"not_named_like_id.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: strict-fs-prompt
|
||||
version: "1.0.0"
|
||||
unknown: true
|
||||
messages:
|
||||
- role: user
|
||||
content: Invalid.
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
}, ".")
|
||||
|
||||
_, err := repo.GetPromptDefinition(context.Background(), "strict-fs-prompt", "")
|
||||
if !errors.Is(err, ErrInvalidYAML) {
|
||||
t.Fatalf("expected ErrInvalidYAML, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func assertCacheControl(t *testing.T, got *domain.CacheControl, wantType domain.CacheControlType, wantTTL string) {
|
||||
t.Helper()
|
||||
if got == nil {
|
||||
t.Fatal("expected cache control, got nil")
|
||||
}
|
||||
if got.Type != wantType {
|
||||
t.Fatalf("unexpected cache control type: got %q want %q", got.Type, wantType)
|
||||
}
|
||||
if got.TTL != wantTTL {
|
||||
t.Fatalf("unexpected cache control ttl: got %q want %q", got.TTL, wantTTL)
|
||||
}
|
||||
}
|
||||
|
||||
func writePromptTestFile(t *testing.T, path string, content string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(path, []byte(strings.TrimLeft(content, "\n")), 0o644); err != nil {
|
||||
t.Fatalf("failed to write prompt test file %q: %v", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
func copyTree(src, dst string) error {
|
||||
return filepath.WalkDir(src, func(path string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rel, err := filepath.Rel(src, path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if rel == "." {
|
||||
return nil
|
||||
}
|
||||
|
||||
target := filepath.Join(dst, rel)
|
||||
if d.IsDir() {
|
||||
return os.MkdirAll(target, 0o755)
|
||||
}
|
||||
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return os.WriteFile(target, data, 0o644)
|
||||
})
|
||||
}
|
||||
10
internal/promptdef/testdata/both_content_and_content_file.yaml
vendored
Normal file
10
internal/promptdef/testdata/both_content_and_content_file.yaml
vendored
Normal file
@@ -0,0 +1,10 @@
|
||||
id: both-content-and-content-file
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: "Hi"
|
||||
content_file: ./messages/user_prompt.tmpl
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
14
internal/promptdef/testdata/duplicate_input_names.yaml
vendored
Normal file
14
internal/promptdef/testdata/duplicate_input_names.yaml
vendored
Normal file
@@ -0,0 +1,14 @@
|
||||
id: duplicate-input-names
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
- name: transcript
|
||||
required: false
|
||||
messages:
|
||||
- role: user
|
||||
content: "Hi"
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
10
internal/promptdef/testdata/empty_cache_control_type.yaml
vendored
Normal file
10
internal/promptdef/testdata/empty_cache_control_type.yaml
vendored
Normal file
@@ -0,0 +1,10 @@
|
||||
id: empty-cache-control-type
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control: {}
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
9
internal/promptdef/testdata/invalid_validation_mode.yaml
vendored
Normal file
9
internal/promptdef/testdata/invalid_validation_mode.yaml
vendored
Normal file
@@ -0,0 +1,9 @@
|
||||
id: invalid-validation-mode
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: "Hi"
|
||||
output:
|
||||
format: text
|
||||
validation_mode: nope
|
||||
repair_attempts: 0
|
||||
9
internal/promptdef/testdata/invalid_yaml.yaml
vendored
Normal file
9
internal/promptdef/testdata/invalid_yaml.yaml
vendored
Normal file
@@ -0,0 +1,9 @@
|
||||
id: invalid-yaml
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: [broken
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
9
internal/promptdef/testdata/json_schema_without_schema_path.yaml
vendored
Normal file
9
internal/promptdef/testdata/json_schema_without_schema_path.yaml
vendored
Normal file
@@ -0,0 +1,9 @@
|
||||
id: json-schema-without-schema-path
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: "Return JSON"
|
||||
output:
|
||||
format: json
|
||||
validation_mode: json_schema
|
||||
repair_attempts: 0
|
||||
2
internal/promptdef/testdata/messages/user_prompt.tmpl
vendored
Normal file
2
internal/promptdef/testdata/messages/user_prompt.tmpl
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
Use transcript:
|
||||
{{input "transcript"}}
|
||||
9
internal/promptdef/testdata/missing_content_file.yaml
vendored
Normal file
9
internal/promptdef/testdata/missing_content_file.yaml
vendored
Normal file
@@ -0,0 +1,9 @@
|
||||
id: missing-content-file
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content_file: ./messages/does_not_exist.tmpl
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
8
internal/promptdef/testdata/missing_id.yaml
vendored
Normal file
8
internal/promptdef/testdata/missing_id.yaml
vendored
Normal file
@@ -0,0 +1,8 @@
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: "Hi"
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
8
internal/promptdef/testdata/neither_content_nor_content_file.yaml
vendored
Normal file
8
internal/promptdef/testdata/neither_content_nor_content_file.yaml
vendored
Normal file
@@ -0,0 +1,8 @@
|
||||
id: neither-content-nor-content-file
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
6
internal/promptdef/testdata/no_messages.yaml
vendored
Normal file
6
internal/promptdef/testdata/no_messages.yaml
vendored
Normal file
@@ -0,0 +1,6 @@
|
||||
id: no-messages
|
||||
version: "1.0.0"
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
12
internal/promptdef/testdata/unknown_cache_control_field.yaml
vendored
Normal file
12
internal/promptdef/testdata/unknown_cache_control_field.yaml
vendored
Normal file
@@ -0,0 +1,12 @@
|
||||
id: unknown-cache-control-field
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
unexpected: true
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
13
internal/promptdef/testdata/unknown_input_field.yaml
vendored
Normal file
13
internal/promptdef/testdata/unknown_input_field.yaml
vendored
Normal file
@@ -0,0 +1,13 @@
|
||||
id: unknown-input-field
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
unknown_input_setting: true
|
||||
messages:
|
||||
- role: user
|
||||
content: "Hi"
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
12
internal/promptdef/testdata/unsupported_cache_control_ttl.yaml
vendored
Normal file
12
internal/promptdef/testdata/unsupported_cache_control_ttl.yaml
vendored
Normal file
@@ -0,0 +1,12 @@
|
||||
id: unsupported-cache-control-ttl
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
ttl: 5m
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
11
internal/promptdef/testdata/unsupported_cache_control_type.yaml
vendored
Normal file
11
internal/promptdef/testdata/unsupported_cache_control_type.yaml
vendored
Normal file
@@ -0,0 +1,11 @@
|
||||
id: unsupported-cache-control-type
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control:
|
||||
type: persistent
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
14
internal/promptdef/testdata/valid_cache_control_ttl.yaml
vendored
Normal file
14
internal/promptdef/testdata/valid_cache_control_ttl.yaml
vendored
Normal file
@@ -0,0 +1,14 @@
|
||||
id: valid-cache-control-ttl
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
ttl: 1h
|
||||
- role: user
|
||||
content: "Summarize the input."
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
13
internal/promptdef/testdata/valid_cache_control_without_ttl.yaml
vendored
Normal file
13
internal/promptdef/testdata/valid_cache_control_without_ttl.yaml
vendored
Normal file
@@ -0,0 +1,13 @@
|
||||
id: valid-cache-control-without-ttl
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: system
|
||||
content: "Use cached instructions."
|
||||
cache_control:
|
||||
type: ephemeral
|
||||
- role: user
|
||||
content: "Summarize the input."
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
14
internal/promptdef/testdata/valid_file_backed.yaml
vendored
Normal file
14
internal/promptdef/testdata/valid_file_backed.yaml
vendored
Normal file
@@ -0,0 +1,14 @@
|
||||
id: valid-file-backed
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
messages:
|
||||
- role: system
|
||||
content: "Return markdown."
|
||||
- role: user
|
||||
content_file: ./messages/user_prompt.tmpl
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
18
internal/promptdef/testdata/valid_inline.yaml
vendored
Normal file
18
internal/promptdef/testdata/valid_inline.yaml
vendored
Normal file
@@ -0,0 +1,18 @@
|
||||
id: valid-inline
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
content_type: text/markdown
|
||||
description: Transcript content
|
||||
messages:
|
||||
- role: system
|
||||
content: "You are concise."
|
||||
- role: user
|
||||
content: |
|
||||
Summarize:
|
||||
{{input "transcript"}}
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
10
internal/promptdef/testdata/valid_session_id.yaml
vendored
Normal file
10
internal/promptdef/testdata/valid_session_id.yaml
vendored
Normal file
@@ -0,0 +1,10 @@
|
||||
id: valid-session-id
|
||||
version: "1.0.0"
|
||||
session_id: " {{ .session_id }} "
|
||||
messages:
|
||||
- role: user
|
||||
content: Hello.
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
13
internal/promptdef/testdata/with_default_profile.yaml
vendored
Normal file
13
internal/promptdef/testdata/with_default_profile.yaml
vendored
Normal file
@@ -0,0 +1,13 @@
|
||||
id: with-default-profile
|
||||
version: "1.0.0"
|
||||
default_profile: local-default
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
messages:
|
||||
- role: user
|
||||
content: "Write output"
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
Reference in New Issue
Block a user