Add public asset source options
This commit is contained in:
@@ -21,7 +21,17 @@ if err != nil {
|
||||
}
|
||||
```
|
||||
|
||||
`PromptDir` is required. `ProfileDir` is optional; omit it to use built-in profiles only, or set it to overlay custom profiles above built-ins. `SchemaDir` defaults to the built-in schema directory. `Timeout` and `HTTPClient` configure the default OpenAI-compatible client used by `Run` when no custom LLM client is supplied.
|
||||
`PromptDir` is required unless an explicit prompt source option is supplied. `ProfileDir` is optional; omit it to use built-in profiles only, or set it to overlay custom profiles above built-ins. `SchemaDir` defaults to the built-in schema directory. `Timeout` and `HTTPClient` configure the default OpenAI-compatible client used by `Run` when no custom LLM client is supplied.
|
||||
|
||||
## Asset Sources
|
||||
|
||||
Directory fields on `Config` remain the compatibility path. Explicit source options override the matching directory field:
|
||||
|
||||
- `WithPromptFS(fsys, root)` and `WithPromptFile(path)`
|
||||
- `WithProfileFS(fsys, root)` and `WithProfileFile(path)`
|
||||
- `WithSchemaFS(fsys, root)` and `WithSchemaFile(path)`
|
||||
|
||||
Prompt and profile sources load standard Scriptorium YAML with the same strict validation as directory sources. Prompt `content_file` paths resolve relative to the prompt file in the same source. Profile options overlay custom profiles above built-ins. Schema `fs.FS` sources preserve prompt `schema_path` semantics; schema file options expose the file by its base name.
|
||||
|
||||
## Prepare A Prompt
|
||||
|
||||
|
||||
@@ -9,13 +9,13 @@ This document describes implemented adapter/repository boundaries and their curr
|
||||
- `internal/adapter/cli`: CLI command parsing, app wiring, stdout/stderr handling, exit codes.
|
||||
- `internal/adapter/http`: HTTP request/response mapping for `POST /v1/runs`.
|
||||
- root package `scriptorium`: public Go library facade for preparing and running prompt requests.
|
||||
- `internal/promptdef`: filesystem prompt-definition repository.
|
||||
- `internal/promptdef`: filesystem and `fs.FS` prompt-definition repositories.
|
||||
- `internal/profile`: filesystem, `fs.FS`, and overlay execution-profile repositories.
|
||||
- `internal/profile/builtin`: embedded built-in execution-profile repository.
|
||||
- `internal/artifact`: input artifact reader.
|
||||
- `internal/prompt`: Go-template renderer.
|
||||
- `internal/llm`: OpenAI-compatible LLM client implementation.
|
||||
- `internal/validate`: output validator.
|
||||
- `internal/validate`: filesystem and `fs.FS` output validators.
|
||||
- `internal/format`: prepared-run formatters for `render` output.
|
||||
|
||||
## Inputs And Outputs
|
||||
@@ -38,18 +38,20 @@ Public library facade:
|
||||
- Output: typed `PreparedRun` and `RunResult` values plus public sentinel errors.
|
||||
- Custom LLM behavior is injected with `WithLLMClient`; otherwise the default OpenAI-compatible client is used.
|
||||
- `RunRequest.APIKey` is a request-scoped Go value only; it is converted into internal execution state for LLM generation and stripped from public result types.
|
||||
- Prompt, profile, and schema source options can use directories, single files, or `fs.FS` roots. Explicit source options override the matching `Config` directory field.
|
||||
- Public types are facade types converted at the package boundary; internal domain types remain internal.
|
||||
|
||||
Filesystem repositories:
|
||||
Prompt/profile repositories:
|
||||
|
||||
- Input: prompt/profile YAML files under configured directories.
|
||||
- Input: prompt/profile YAML files under configured directories or `fs.FS` roots.
|
||||
- Output: normalized domain definitions/profiles or typed errors.
|
||||
- Single-file public sources are represented as `fs.FS` roots containing one YAML file; lookup still uses YAML `id` values.
|
||||
|
||||
Profile repository composition:
|
||||
|
||||
- Built-in profiles are embedded and loaded through the same profile validation rules as filesystem profiles.
|
||||
- When no custom profile directory is configured, the runner receives the built-in profile repository.
|
||||
- When a custom profile directory is configured, the runner receives an overlay repository with custom profiles as primary and built-ins as fallback.
|
||||
- When a custom profile directory/file/`fs.FS` source is configured, the runner receives an overlay repository with custom profiles as primary and built-ins as fallback.
|
||||
- Overlay lookup falls back only after custom profile-not-found errors; custom load/validation/raw-key errors are returned directly.
|
||||
|
||||
Artifact reader:
|
||||
@@ -67,6 +69,7 @@ Validator:
|
||||
|
||||
- Input: artifact body + output contract.
|
||||
- Output: validation result or runtime validation error.
|
||||
- Schema documents may be loaded from a directory, single file, or `fs.FS` root in the public package. CLI and HTTP continue to use directory-backed schema loading.
|
||||
|
||||
## Boundaries
|
||||
|
||||
@@ -103,6 +106,7 @@ Strict decoding and input checks:
|
||||
- config/prompt/profile loaders reject unknown YAML fields.
|
||||
- prompt/profile repositories scan nested subdirectories recursively.
|
||||
- prompt/profile lookup uses YAML `id` values; subdirectory paths are organizational only.
|
||||
- prompt `content_file` paths resolve relative to the prompt YAML file within the same source.
|
||||
- duplicate prompt/profile IDs are invalid and fail instead of using first-match behavior.
|
||||
- duplicate profile IDs across custom and built-in sources are allowed; the custom source overrides the built-in profile.
|
||||
- HTTP DTO decoder rejects unknown JSON fields.
|
||||
|
||||
143
engine.go
143
engine.go
@@ -4,13 +4,17 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
artifactadapter "gitea.maximumdirect.net/eric/scriptorium/internal/artifact"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/defaults"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/llm"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/profile"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/profile/builtin"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/prompt"
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/promptdef"
|
||||
@@ -51,7 +55,13 @@ type Config struct {
|
||||
type Option func(*engineOptions) error
|
||||
|
||||
type engineOptions struct {
|
||||
llmClient llm.Client
|
||||
llmClient llm.Client
|
||||
promptDefs promptdef.Repository
|
||||
profiles profile.Repository
|
||||
validator validate.Validator
|
||||
promptSource bool
|
||||
profileSource bool
|
||||
validatorSource bool
|
||||
}
|
||||
|
||||
// WithLLMClient injects a custom LLM client for execution.
|
||||
@@ -65,13 +75,87 @@ func WithLLMClient(client LLMClient) Option {
|
||||
}
|
||||
}
|
||||
|
||||
func WithPromptFS(fsys fs.FS, root string) Option {
|
||||
return func(options *engineOptions) error {
|
||||
if fsys == nil {
|
||||
return ErrInvalidConfig
|
||||
}
|
||||
if strings.TrimSpace(root) == "" {
|
||||
return ErrInvalidConfig
|
||||
}
|
||||
options.promptDefs = promptdef.NewFSRepository(fsys, root)
|
||||
options.promptSource = true
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func WithPromptFile(path string) Option {
|
||||
return func(options *engineOptions) error {
|
||||
fsys, root, err := fileSource(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
options.promptDefs = promptdef.NewFSRepository(fsys, root)
|
||||
options.promptSource = true
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func WithProfileFS(fsys fs.FS, root string) Option {
|
||||
return func(options *engineOptions) error {
|
||||
if fsys == nil {
|
||||
return ErrInvalidConfig
|
||||
}
|
||||
if strings.TrimSpace(root) == "" {
|
||||
return ErrInvalidConfig
|
||||
}
|
||||
options.profiles = profile.NewFSRepository(fsys, root)
|
||||
options.profileSource = true
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func WithProfileFile(path string) Option {
|
||||
return func(options *engineOptions) error {
|
||||
fsys, root, err := fileSource(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
options.profiles = profile.NewFSRepository(fsys, root)
|
||||
options.profileSource = true
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func WithSchemaFS(fsys fs.FS, root string) Option {
|
||||
return func(options *engineOptions) error {
|
||||
if fsys == nil {
|
||||
return ErrInvalidConfig
|
||||
}
|
||||
if strings.TrimSpace(root) == "" {
|
||||
return ErrInvalidConfig
|
||||
}
|
||||
options.validator = validate.NewFSValidator(fsys, root)
|
||||
options.validatorSource = true
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func WithSchemaFile(path string) Option {
|
||||
return func(options *engineOptions) error {
|
||||
fsys, root, err := fileSource(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
options.validator = validate.NewFSValidator(fsys, root)
|
||||
options.validatorSource = true
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// NewEngine constructs an Engine using the same default internal components as
|
||||
// the CLI and HTTP adapters.
|
||||
func NewEngine(cfg Config, opts ...Option) (*Engine, error) {
|
||||
if strings.TrimSpace(cfg.PromptDir) == "" {
|
||||
return nil, fmt.Errorf("%w: prompt directory is required", ErrInvalidConfig)
|
||||
}
|
||||
|
||||
var options engineOptions
|
||||
for _, opt := range opts {
|
||||
if opt == nil {
|
||||
@@ -82,9 +166,26 @@ func NewEngine(cfg Config, opts ...Option) (*Engine, error) {
|
||||
}
|
||||
}
|
||||
|
||||
schemaDir := cfg.SchemaDir
|
||||
if strings.TrimSpace(schemaDir) == "" {
|
||||
schemaDir = defaults.SchemaDirDefault
|
||||
promptDefs := options.promptDefs
|
||||
if !options.promptSource {
|
||||
if strings.TrimSpace(cfg.PromptDir) == "" {
|
||||
return nil, fmt.Errorf("%w: prompt directory is required", ErrInvalidConfig)
|
||||
}
|
||||
promptDefs = promptdef.NewFilesystemRepository(cfg.PromptDir)
|
||||
}
|
||||
|
||||
profiles := builtin.NewRepositoryWithDirectory(cfg.ProfileDir)
|
||||
if options.profileSource {
|
||||
profiles = builtin.NewRepositoryWithPrimary(options.profiles)
|
||||
}
|
||||
|
||||
validator := options.validator
|
||||
if !options.validatorSource {
|
||||
schemaDir := cfg.SchemaDir
|
||||
if strings.TrimSpace(schemaDir) == "" {
|
||||
schemaDir = defaults.SchemaDirDefault
|
||||
}
|
||||
validator = validate.NewStandardValidator(schemaDir)
|
||||
}
|
||||
|
||||
llmClient := options.llmClient
|
||||
@@ -101,16 +202,36 @@ func NewEngine(cfg Config, opts ...Option) (*Engine, error) {
|
||||
|
||||
return &Engine{
|
||||
runner: usecase.NewRunner(
|
||||
promptdef.NewFilesystemRepository(cfg.PromptDir),
|
||||
builtin.NewRepositoryWithDirectory(cfg.ProfileDir),
|
||||
promptDefs,
|
||||
profiles,
|
||||
artifactadapter.NewCompositeReader(),
|
||||
prompt.NewGoRenderer(),
|
||||
llmClient,
|
||||
validate.NewStandardValidator(schemaDir),
|
||||
validator,
|
||||
),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func fileSource(name string) (fs.FS, string, error) {
|
||||
cleanName := strings.TrimSpace(name)
|
||||
if cleanName == "" {
|
||||
return nil, "", ErrInvalidConfig
|
||||
}
|
||||
dir := filepath.Dir(cleanName)
|
||||
base := filepath.Base(cleanName)
|
||||
if base == "." || base == string(filepath.Separator) || strings.TrimSpace(base) == "" {
|
||||
return nil, "", ErrInvalidConfig
|
||||
}
|
||||
info, err := os.Stat(cleanName)
|
||||
if err != nil {
|
||||
return nil, "", fmt.Errorf("%w: failed to access source file %q: %v", ErrInvalidConfig, cleanName, err)
|
||||
}
|
||||
if info.IsDir() {
|
||||
return nil, "", fmt.Errorf("%w: source path %q must be a file", ErrInvalidConfig, cleanName)
|
||||
}
|
||||
return os.DirFS(dir), filepath.ToSlash(base), nil
|
||||
}
|
||||
|
||||
// Prepare resolves a prompt request without calling an LLM.
|
||||
func (e *Engine) Prepare(ctx context.Context, req RunRequest) (*PreparedRun, error) {
|
||||
if e == nil || e.runner == nil {
|
||||
|
||||
288
engine_test.go
288
engine_test.go
@@ -11,6 +11,7 @@ import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
"gitea.maximumdirect.net/eric/scriptorium"
|
||||
)
|
||||
@@ -709,6 +710,256 @@ unexpected: true
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrepareWorksWithPromptFSAndRelativeContentFile(t *testing.T) {
|
||||
promptFS := fstest.MapFS{
|
||||
"assets/prompts/fs-summary.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: fs.summary
|
||||
version: "1.0.0"
|
||||
default_profile: local-fast
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
messages:
|
||||
- role: user
|
||||
content_file: ./messages/summary.tmpl
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
"assets/prompts/messages/summary.tmpl": &fstest.MapFile{Data: []byte(`Summarize {{input "transcript"}} from prompt fs.`)},
|
||||
}
|
||||
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
PromptDir: t.TempDir(),
|
||||
ProfileDir: "./examples/profiles",
|
||||
SchemaDir: "./examples/schemas",
|
||||
}, scriptorium.WithPromptFS(promptFS, "assets/prompts"))
|
||||
if err != nil {
|
||||
t.Fatalf("expected engine construction to succeed, got %v", err)
|
||||
}
|
||||
|
||||
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
|
||||
PromptID: "fs.summary",
|
||||
Inputs: map[string]scriptorium.ArtifactRef{
|
||||
"transcript": scriptorium.Inline("Rin opens the gate."),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected prepare to succeed, got %v", err)
|
||||
}
|
||||
if len(prepared.Messages) != 1 || !strings.Contains(prepared.Messages[0].Content, "prompt fs") {
|
||||
t.Fatalf("expected content_file body from prompt fs, got %+v", prepared.Messages)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrepareWorksWithPromptFile(t *testing.T) {
|
||||
promptDir := t.TempDir()
|
||||
promptPath := filepath.Join(promptDir, "single.yaml")
|
||||
if err := os.WriteFile(promptPath, []byte(`
|
||||
id: single.file.prompt
|
||||
version: "1.0.0"
|
||||
default_profile: local-fast
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
messages:
|
||||
- role: user
|
||||
content: "Summarize {{input \"transcript\"}} from file."
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
`), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
ProfileDir: "./examples/profiles",
|
||||
SchemaDir: "./examples/schemas",
|
||||
}, scriptorium.WithPromptFile(promptPath))
|
||||
if err != nil {
|
||||
t.Fatalf("expected engine construction to succeed, got %v", err)
|
||||
}
|
||||
|
||||
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
|
||||
PromptID: "single.file.prompt",
|
||||
Inputs: map[string]scriptorium.ArtifactRef{
|
||||
"transcript": scriptorium.Inline("Rin opens the gate."),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected prepare to succeed, got %v", err)
|
||||
}
|
||||
if prepared.PromptID != "single.file.prompt" {
|
||||
t.Fatalf("unexpected prompt id: %q", prepared.PromptID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrepareWorksWithProfileFSOverBuiltIns(t *testing.T) {
|
||||
profileFS := fstest.MapFS{
|
||||
"profiles/mistral-small-3.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: mistral-small-3
|
||||
endpoint: http://profile-fs/v1
|
||||
model: profile-fs-model
|
||||
`)},
|
||||
}
|
||||
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
PromptDir: "./examples/prompts",
|
||||
SchemaDir: "./examples/schemas",
|
||||
}, scriptorium.WithProfileFS(profileFS, "profiles"))
|
||||
if err != nil {
|
||||
t.Fatalf("expected engine construction to succeed, got %v", err)
|
||||
}
|
||||
|
||||
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
|
||||
PromptID: "generic.markdown_summary",
|
||||
ProfileID: "mistral-small-3",
|
||||
Inputs: map[string]scriptorium.ArtifactRef{
|
||||
"transcript": scriptorium.Inline("Rin opens the gate."),
|
||||
"glossary": scriptorium.Inline("gate: A guarded passage."),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected prepare to succeed, got %v", err)
|
||||
}
|
||||
if prepared.EffectiveModelParams.Model != "profile-fs-model" {
|
||||
t.Fatalf("expected profile fs to override built-in, got %q", prepared.EffectiveModelParams.Model)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrepareWorksWithProfileFileOverBuiltIns(t *testing.T) {
|
||||
profileDir := t.TempDir()
|
||||
profilePath := filepath.Join(profileDir, "mistral-small-3.yaml")
|
||||
if err := os.WriteFile(profilePath, []byte(`
|
||||
id: mistral-small-3
|
||||
endpoint: http://profile-file/v1
|
||||
model: profile-file-model
|
||||
`), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
PromptDir: "./examples/prompts",
|
||||
SchemaDir: "./examples/schemas",
|
||||
}, scriptorium.WithProfileFile(profilePath))
|
||||
if err != nil {
|
||||
t.Fatalf("expected engine construction to succeed, got %v", err)
|
||||
}
|
||||
|
||||
prepared, err := engine.Prepare(context.Background(), scriptorium.RunRequest{
|
||||
PromptID: "generic.markdown_summary",
|
||||
ProfileID: "mistral-small-3",
|
||||
Inputs: map[string]scriptorium.ArtifactRef{
|
||||
"transcript": scriptorium.Inline("Rin opens the gate."),
|
||||
"glossary": scriptorium.Inline("gate: A guarded passage."),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected prepare to succeed, got %v", err)
|
||||
}
|
||||
if prepared.EffectiveModelParams.Model != "profile-file-model" {
|
||||
t.Fatalf("expected profile file to override built-in, got %q", prepared.EffectiveModelParams.Model)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunStructuredOutputWorksWithSchemaFS(t *testing.T) {
|
||||
fake := &fakeLLMClient{response: &scriptorium.GenerateResponse{Content: `{"events":[]}`}}
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
PromptDir: t.TempDir(),
|
||||
ProfileDir: "./examples/profiles",
|
||||
SchemaDir: t.TempDir(),
|
||||
},
|
||||
scriptorium.WithPromptFS(publicStructuredPromptFS("schema.fs.prompt", "events.schema.json"), "prompts"),
|
||||
scriptorium.WithSchemaFS(publicSchemaFS(), "schemas"),
|
||||
scriptorium.WithLLMClient(fake),
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("expected engine construction to succeed, got %v", err)
|
||||
}
|
||||
|
||||
result, err := engine.Run(context.Background(), scriptorium.RunRequest{
|
||||
PromptID: "schema.fs.prompt",
|
||||
Inputs: map[string]scriptorium.ArtifactRef{
|
||||
"transcript": scriptorium.Inline("Rin opens the gate."),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected run to succeed, got %v", err)
|
||||
}
|
||||
if result.Validation.Status != scriptorium.ValidationPassed || !result.Validation.IsValid {
|
||||
t.Fatalf("expected schema validation to pass, got %+v", result.Validation)
|
||||
}
|
||||
if len(fake.requests) != 1 || fake.requests[0].StructuredOutput == nil {
|
||||
t.Fatalf("expected structured output request, got %+v", fake.requests)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunStructuredOutputWorksWithSchemaFile(t *testing.T) {
|
||||
schemaDir := t.TempDir()
|
||||
schemaPath := filepath.Join(schemaDir, "events.schema.json")
|
||||
if err := os.WriteFile(schemaPath, []byte(publicSchemaJSON()), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
fake := &fakeLLMClient{response: &scriptorium.GenerateResponse{Content: `{"events":[]}`}}
|
||||
engine, err := scriptorium.NewEngine(scriptorium.Config{
|
||||
ProfileDir: "./examples/profiles",
|
||||
},
|
||||
scriptorium.WithPromptFS(publicStructuredPromptFS("schema.file.prompt", "events.schema.json"), "prompts"),
|
||||
scriptorium.WithSchemaFile(schemaPath),
|
||||
scriptorium.WithLLMClient(fake),
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("expected engine construction to succeed, got %v", err)
|
||||
}
|
||||
|
||||
result, err := engine.Run(context.Background(), scriptorium.RunRequest{
|
||||
PromptID: "schema.file.prompt",
|
||||
Inputs: map[string]scriptorium.ArtifactRef{
|
||||
"transcript": scriptorium.Inline("Rin opens the gate."),
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected run to succeed, got %v", err)
|
||||
}
|
||||
if result.Validation.Status != scriptorium.ValidationPassed || !result.Validation.IsValid {
|
||||
t.Fatalf("expected schema validation to pass, got %+v", result.Validation)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSourceOptionsRejectInvalidInputs(t *testing.T) {
|
||||
missingFile := filepath.Join(t.TempDir(), "missing.yaml")
|
||||
directoryPath := t.TempDir()
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
opt scriptorium.Option
|
||||
}{
|
||||
{name: "prompt fs nil", opt: scriptorium.WithPromptFS(nil, "prompts")},
|
||||
{name: "prompt fs empty root", opt: scriptorium.WithPromptFS(fstest.MapFS{}, "")},
|
||||
{name: "prompt file empty", opt: scriptorium.WithPromptFile("")},
|
||||
{name: "prompt file missing", opt: scriptorium.WithPromptFile(missingFile)},
|
||||
{name: "prompt file directory", opt: scriptorium.WithPromptFile(directoryPath)},
|
||||
{name: "profile fs nil", opt: scriptorium.WithProfileFS(nil, "profiles")},
|
||||
{name: "profile fs empty root", opt: scriptorium.WithProfileFS(fstest.MapFS{}, "")},
|
||||
{name: "profile file empty", opt: scriptorium.WithProfileFile("")},
|
||||
{name: "schema fs nil", opt: scriptorium.WithSchemaFS(nil, "schemas")},
|
||||
{name: "schema fs empty root", opt: scriptorium.WithSchemaFS(fstest.MapFS{}, "")},
|
||||
{name: "schema file empty", opt: scriptorium.WithSchemaFile("")},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
_, err := scriptorium.NewEngine(scriptorium.Config{PromptDir: "./examples/prompts"}, tc.opt)
|
||||
if !errors.Is(err, scriptorium.ErrInvalidConfig) {
|
||||
t.Fatalf("expected ErrInvalidConfig, got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtraParamsTypedNestedValuesAreCopiedAcrossPublicBoundary(t *testing.T) {
|
||||
fake := &fakeLLMClient{response: &scriptorium.GenerateResponse{Content: "ok"}}
|
||||
engine := newExampleEngineWithOptions(t, "./examples/schemas", scriptorium.WithLLMClient(fake))
|
||||
@@ -862,6 +1113,43 @@ api_key_env: ` + apiKeyEnv + `
|
||||
}
|
||||
}
|
||||
|
||||
func publicStructuredPromptFS(id string, schemaPath string) fstest.MapFS {
|
||||
return fstest.MapFS{
|
||||
"prompts/prompt.yaml": &fstest.MapFile{Data: []byte(`id: ` + id + `
|
||||
version: "1.0.0"
|
||||
default_profile: local-fast
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
messages:
|
||||
- role: user
|
||||
content: "Extract events from {{input \"transcript\"}}."
|
||||
output:
|
||||
format: json
|
||||
validation_mode: json_schema
|
||||
schema_path: ` + schemaPath + `
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
}
|
||||
}
|
||||
|
||||
func publicSchemaFS() fstest.MapFS {
|
||||
return fstest.MapFS{
|
||||
"schemas/events.schema.json": &fstest.MapFile{Data: []byte(publicSchemaJSON())},
|
||||
}
|
||||
}
|
||||
|
||||
func publicSchemaJSON() string {
|
||||
return `{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"type": "object",
|
||||
"required": ["events"],
|
||||
"properties": {
|
||||
"events": {"type": "array"}
|
||||
}
|
||||
}`
|
||||
}
|
||||
|
||||
type fakeLLMClient struct {
|
||||
response *scriptorium.GenerateResponse
|
||||
err error
|
||||
|
||||
@@ -5,7 +5,9 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
@@ -24,6 +26,11 @@ type filesystemRepository struct {
|
||||
dir string
|
||||
}
|
||||
|
||||
type fsRepository struct {
|
||||
fsys fs.FS
|
||||
root string
|
||||
}
|
||||
|
||||
type promptDefinitionFile struct {
|
||||
ID string `yaml:"id"`
|
||||
Version string `yaml:"version"`
|
||||
@@ -65,6 +72,10 @@ func NewFilesystemRepository(dir string) Repository {
|
||||
return &filesystemRepository{dir: dir}
|
||||
}
|
||||
|
||||
func NewFSRepository(fsys fs.FS, root string) Repository {
|
||||
return &fsRepository{fsys: fsys, root: root}
|
||||
}
|
||||
|
||||
func (r *filesystemRepository) GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error) {
|
||||
if strings.TrimSpace(id) == "" {
|
||||
return nil, fmt.Errorf("%w: prompt id is required", ErrInvalidPromptDefinition)
|
||||
@@ -132,6 +143,10 @@ func (r *filesystemRepository) GetPromptDefinition(ctx context.Context, id strin
|
||||
return nil, ErrPromptDefinitionNotFound
|
||||
}
|
||||
|
||||
func (r *fsRepository) GetPromptDefinition(ctx context.Context, id string, version string) (*domain.PromptDefinition, error) {
|
||||
return loadPromptDefinition(ctx, r.fsys, r.root, id, version)
|
||||
}
|
||||
|
||||
type promptDefinitionMatch struct {
|
||||
def *domain.PromptDefinition
|
||||
path string
|
||||
@@ -166,7 +181,187 @@ func promptDefinitionFileHasID(path string, id string) bool {
|
||||
return strings.TrimSpace(raw.ID) == id
|
||||
}
|
||||
|
||||
func loadPromptDefinition(ctx context.Context, fsys fs.FS, root string, id string, version string) (*domain.PromptDefinition, error) {
|
||||
if strings.TrimSpace(id) == "" {
|
||||
return nil, fmt.Errorf("%w: prompt id is required", ErrInvalidPromptDefinition)
|
||||
}
|
||||
if fsys == nil {
|
||||
return nil, fmt.Errorf("failed to read prompt definition directory: filesystem is nil")
|
||||
}
|
||||
|
||||
files, err := findPromptDefinitionYAMLFiles(ctx, fsys, root)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read prompt definition directory: %w", err)
|
||||
}
|
||||
|
||||
var matches []promptDefinitionMatch
|
||||
for _, fullPath := range files {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
default:
|
||||
}
|
||||
|
||||
relPath := displayPath(root, fullPath)
|
||||
fileMatch := filecatalog.Stem(path.Base(fullPath)) == id
|
||||
data, err := fs.ReadFile(fsys, fullPath)
|
||||
if err != nil {
|
||||
if fileMatch {
|
||||
return nil, fmt.Errorf("%w: %s: failed to read prompt definition file: %v", ErrInvalidYAML, relPath, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
raw, err := decodePromptDefinition(data)
|
||||
if err != nil {
|
||||
if fileMatch || promptDefinitionDataHasID(data, id) {
|
||||
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidYAML, relPath, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
def, err := normalizePromptDefinitionFromFS(raw, fsys, fullPath)
|
||||
if err != nil {
|
||||
if fileMatch || strings.TrimSpace(raw.ID) == id {
|
||||
return nil, fmt.Errorf("%w: %s: %v", ErrInvalidPromptDefinition, relPath, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if def.ID != id {
|
||||
continue
|
||||
}
|
||||
if version != "" && def.Version != version {
|
||||
continue
|
||||
}
|
||||
matches = append(matches, promptDefinitionMatch{
|
||||
def: def,
|
||||
path: relPath,
|
||||
})
|
||||
}
|
||||
|
||||
if len(matches) > 1 {
|
||||
paths := make([]string, 0, len(matches))
|
||||
for _, match := range matches {
|
||||
paths = append(paths, match.path)
|
||||
}
|
||||
if version != "" {
|
||||
return nil, fmt.Errorf("%w: duplicate prompt definition id %q version %q found in: %s", ErrInvalidPromptDefinition, id, version, strings.Join(paths, ", "))
|
||||
}
|
||||
return nil, fmt.Errorf("%w: duplicate prompt definition id %q found in: %s", ErrInvalidPromptDefinition, id, strings.Join(paths, ", "))
|
||||
}
|
||||
|
||||
if len(matches) == 1 {
|
||||
return matches[0].def, nil
|
||||
}
|
||||
|
||||
return nil, ErrPromptDefinitionNotFound
|
||||
}
|
||||
|
||||
func findPromptDefinitionYAMLFiles(ctx context.Context, fsys fs.FS, root string) ([]string, error) {
|
||||
cleanRoot := cleanFSRoot(root)
|
||||
var files []string
|
||||
err := fs.WalkDir(fsys, cleanRoot, func(name string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
default:
|
||||
}
|
||||
if d.IsDir() {
|
||||
return nil
|
||||
}
|
||||
if !isPromptDefinitionYAMLFile(d.Name()) {
|
||||
return nil
|
||||
}
|
||||
files = append(files, name)
|
||||
return nil
|
||||
})
|
||||
return files, err
|
||||
}
|
||||
|
||||
func cleanFSRoot(root string) string {
|
||||
root = strings.TrimSpace(root)
|
||||
if root == "" || root == "." {
|
||||
return "."
|
||||
}
|
||||
return path.Clean(root)
|
||||
}
|
||||
|
||||
func displayPath(root string, name string) string {
|
||||
cleanRoot := cleanFSRoot(root)
|
||||
cleanName := path.Clean(name)
|
||||
if cleanRoot == "." {
|
||||
return cleanName
|
||||
}
|
||||
prefix := strings.TrimSuffix(cleanRoot, "/") + "/"
|
||||
if strings.HasPrefix(cleanName, prefix) {
|
||||
return strings.TrimPrefix(cleanName, prefix)
|
||||
}
|
||||
return cleanName
|
||||
}
|
||||
|
||||
func isPromptDefinitionYAMLFile(name string) bool {
|
||||
return strings.HasSuffix(name, ".yaml") || strings.HasSuffix(name, ".yml")
|
||||
}
|
||||
|
||||
func decodePromptDefinition(data []byte) (*promptDefinitionFile, error) {
|
||||
var raw promptDefinitionFile
|
||||
decoder := yaml.NewDecoder(bytes.NewReader(data))
|
||||
decoder.KnownFields(true)
|
||||
if err := decoder.Decode(&raw); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &raw, nil
|
||||
}
|
||||
|
||||
func promptDefinitionDataHasID(data []byte, id string) bool {
|
||||
var raw struct {
|
||||
ID string `yaml:"id"`
|
||||
}
|
||||
if err := yaml.NewDecoder(bytes.NewReader(data)).Decode(&raw); err != nil {
|
||||
return false
|
||||
}
|
||||
return strings.TrimSpace(raw.ID) == id
|
||||
}
|
||||
|
||||
func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*domain.PromptDefinition, error) {
|
||||
promptDir := filepath.Dir(sourcePath)
|
||||
return normalizePromptDefinitionWithContent(raw, func(contentFile string) (string, string, error) {
|
||||
resolvedPath := strings.TrimSpace(contentFile)
|
||||
if !filepath.IsAbs(resolvedPath) {
|
||||
resolvedPath = filepath.Join(promptDir, resolvedPath)
|
||||
}
|
||||
resolvedPath = filepath.Clean(resolvedPath)
|
||||
|
||||
body, err := os.ReadFile(resolvedPath)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
return string(body), resolvedPath, nil
|
||||
})
|
||||
}
|
||||
|
||||
func normalizePromptDefinitionFromFS(raw *promptDefinitionFile, fsys fs.FS, sourcePath string) (*domain.PromptDefinition, error) {
|
||||
promptDir := path.Dir(sourcePath)
|
||||
return normalizePromptDefinitionWithContent(raw, func(contentFile string) (string, string, error) {
|
||||
resolvedPath := strings.TrimSpace(contentFile)
|
||||
if !path.IsAbs(resolvedPath) {
|
||||
resolvedPath = path.Join(promptDir, resolvedPath)
|
||||
}
|
||||
resolvedPath = strings.TrimPrefix(path.Clean(resolvedPath), "/")
|
||||
|
||||
body, err := fs.ReadFile(fsys, resolvedPath)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
return string(body), resolvedPath, nil
|
||||
})
|
||||
}
|
||||
|
||||
func normalizePromptDefinitionWithContent(raw *promptDefinitionFile, readContentFile func(string) (string, string, error)) (*domain.PromptDefinition, error) {
|
||||
if raw == nil {
|
||||
return nil, errors.New("prompt definition is nil")
|
||||
}
|
||||
@@ -206,7 +401,6 @@ func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*d
|
||||
}
|
||||
|
||||
templates := make([]domain.PromptMessageTemplate, 0, len(raw.Messages))
|
||||
promptDir := filepath.Dir(sourcePath)
|
||||
for i, msg := range raw.Messages {
|
||||
role := strings.TrimSpace(msg.Role)
|
||||
if role == "" {
|
||||
@@ -227,17 +421,11 @@ func normalizePromptDefinition(raw *promptDefinitionFile, sourcePath string) (*d
|
||||
templateContent := msg.Content
|
||||
resolvedContentFile := ""
|
||||
if hasContentFile {
|
||||
resolvedPath := strings.TrimSpace(msg.ContentFile)
|
||||
if !filepath.IsAbs(resolvedPath) {
|
||||
resolvedPath = filepath.Join(promptDir, resolvedPath)
|
||||
}
|
||||
resolvedPath = filepath.Clean(resolvedPath)
|
||||
|
||||
body, err := os.ReadFile(resolvedPath)
|
||||
body, resolvedPath, err := readContentFile(msg.ContentFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("prompt %q message %d (%s): failed to read content_file %q: %w", id, i, role, msg.ContentFile, err)
|
||||
}
|
||||
templateContent = string(body)
|
||||
templateContent = body
|
||||
resolvedContentFile = resolvedPath
|
||||
}
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||
)
|
||||
@@ -324,6 +325,97 @@ output:
|
||||
})
|
||||
}
|
||||
|
||||
func TestFSRepositoryGetPromptDefinition(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"prompts/nested/prompt.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: fs-prompt
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
- name: transcript
|
||||
required: true
|
||||
messages:
|
||||
- role: user
|
||||
content_file: ./messages/user.tmpl
|
||||
output:
|
||||
format: markdown
|
||||
validation_mode: basic
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
"prompts/nested/messages/user.tmpl": &fstest.MapFile{Data: []byte(`Summarize {{input "transcript"}}.`)},
|
||||
}, "prompts")
|
||||
|
||||
got, err := repo.GetPromptDefinition(context.Background(), "fs-prompt", "")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if got.ID != "fs-prompt" {
|
||||
t.Fatalf("unexpected prompt id: %q", got.ID)
|
||||
}
|
||||
if len(got.Templates) != 1 || !strings.Contains(got.Templates[0].Content, `{{input "transcript"}}`) {
|
||||
t.Fatalf("expected content_file body to be loaded, got %+v", got.Templates)
|
||||
}
|
||||
if got.Templates[0].ContentFile != "prompts/nested/messages/user.tmpl" {
|
||||
t.Fatalf("unexpected content file path: %q", got.Templates[0].ContentFile)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSRepositoryRejectsDuplicatePromptIDs(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"one.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: duplicate-fs-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: First.
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
"nested/two.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: duplicate-fs-prompt
|
||||
version: "1.0.0"
|
||||
messages:
|
||||
- role: user
|
||||
content: Second.
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
}, ".")
|
||||
|
||||
_, err := repo.GetPromptDefinition(context.Background(), "duplicate-fs-prompt", "")
|
||||
if !errors.Is(err, ErrInvalidPromptDefinition) {
|
||||
t.Fatalf("expected ErrInvalidPromptDefinition, got %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "one.yaml") || !strings.Contains(err.Error(), "nested/two.yaml") {
|
||||
t.Fatalf("expected duplicate paths in error, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSRepositoryRejectsUnknownYAMLFields(t *testing.T) {
|
||||
repo := NewFSRepository(fstest.MapFS{
|
||||
"not_named_like_id.yaml": &fstest.MapFile{Data: []byte(`
|
||||
id: strict-fs-prompt
|
||||
version: "1.0.0"
|
||||
unknown: true
|
||||
messages:
|
||||
- role: user
|
||||
content: Invalid.
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
repair_attempts: 0
|
||||
`)},
|
||||
}, ".")
|
||||
|
||||
_, err := repo.GetPromptDefinition(context.Background(), "strict-fs-prompt", "")
|
||||
if !errors.Is(err, ErrInvalidYAML) {
|
||||
t.Fatalf("expected ErrInvalidYAML, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func assertCacheControl(t *testing.T, got *domain.CacheControl, wantType domain.CacheControlType, wantTTL string) {
|
||||
t.Helper()
|
||||
if got == nil {
|
||||
|
||||
@@ -5,7 +5,9 @@ import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
@@ -18,10 +20,19 @@ type StandardValidator struct {
|
||||
schemaBaseDir string
|
||||
}
|
||||
|
||||
type FSValidator struct {
|
||||
fsys fs.FS
|
||||
root string
|
||||
}
|
||||
|
||||
func NewStandardValidator(schemaBaseDir string) Validator {
|
||||
return &StandardValidator{schemaBaseDir: schemaBaseDir}
|
||||
}
|
||||
|
||||
func NewFSValidator(fsys fs.FS, root string) Validator {
|
||||
return &FSValidator{fsys: fsys, root: root}
|
||||
}
|
||||
|
||||
func (v *StandardValidator) Validate(ctx context.Context, artifact *domain.Artifact, contract domain.OutputContract) (domain.ValidationResult, error) {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
@@ -100,6 +111,88 @@ func (v *StandardValidator) Validate(ctx context.Context, artifact *domain.Artif
|
||||
}
|
||||
}
|
||||
|
||||
func (v *FSValidator) Validate(ctx context.Context, artifact *domain.Artifact, contract domain.OutputContract) (domain.ValidationResult, error) {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return domain.ValidationResult{}, ctx.Err()
|
||||
default:
|
||||
}
|
||||
|
||||
res := domain.ValidationResult{
|
||||
Mode: contract.ValidationMode,
|
||||
SchemaPath: contract.SchemaPath,
|
||||
RepairAttempts: contract.RepairAttempts,
|
||||
}
|
||||
|
||||
if artifact == nil {
|
||||
return domain.ValidationResult{}, errors.New("artifact is required for validation")
|
||||
}
|
||||
|
||||
switch contract.ValidationMode {
|
||||
case domain.ValidationNone:
|
||||
res.Status = domain.ValidationSkipped
|
||||
res.IsValid = true
|
||||
return res, nil
|
||||
case domain.ValidationBasic:
|
||||
if strings.TrimSpace(string(artifact.Body)) == "" {
|
||||
res.Status = domain.ValidationFailed
|
||||
res.IsValid = false
|
||||
res.Errors = []string{"output is empty"}
|
||||
return res, nil
|
||||
}
|
||||
res.Status = domain.ValidationPassed
|
||||
res.IsValid = true
|
||||
return res, nil
|
||||
case domain.ValidationJSON:
|
||||
_, jsonErr := parseJSON(artifact.Body)
|
||||
if jsonErr != nil {
|
||||
res.Status = domain.ValidationFailed
|
||||
res.IsValid = false
|
||||
res.Errors = []string{fmt.Sprintf("invalid JSON: %v", jsonErr)}
|
||||
return res, nil
|
||||
}
|
||||
res.Status = domain.ValidationPassed
|
||||
res.IsValid = true
|
||||
return res, nil
|
||||
case domain.ValidationJSONSchema:
|
||||
instance, jsonErr := parseJSON(artifact.Body)
|
||||
if jsonErr != nil {
|
||||
res.Status = domain.ValidationFailed
|
||||
res.IsValid = false
|
||||
res.Errors = []string{fmt.Sprintf("invalid JSON: %v", jsonErr)}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
schemaName, schemaDoc, err := v.loadSchemaDocument(contract.SchemaPath)
|
||||
if err != nil {
|
||||
return domain.ValidationResult{}, err
|
||||
}
|
||||
|
||||
resourceURL := fsSchemaResourceURL(schemaName)
|
||||
compiler := jsonschema.NewCompiler()
|
||||
if err := compiler.AddResource(resourceURL, schemaDoc); err != nil {
|
||||
return domain.ValidationResult{}, fmt.Errorf("failed to register JSON schema %q: %w", schemaName, err)
|
||||
}
|
||||
schema, err := compiler.Compile(resourceURL)
|
||||
if err != nil {
|
||||
return domain.ValidationResult{}, fmt.Errorf("failed to compile JSON schema %q: %w", schemaName, err)
|
||||
}
|
||||
|
||||
if err := schema.Validate(instance); err != nil {
|
||||
res.Status = domain.ValidationFailed
|
||||
res.IsValid = false
|
||||
res.Errors = []string{fmt.Sprintf("json schema validation failed: %v", err)}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
res.Status = domain.ValidationPassed
|
||||
res.IsValid = true
|
||||
return res, nil
|
||||
default:
|
||||
return domain.ValidationResult{}, fmt.Errorf("unsupported validation mode: %q", contract.ValidationMode)
|
||||
}
|
||||
}
|
||||
|
||||
func parseJSON(body []byte) (any, error) {
|
||||
var v any
|
||||
if err := json.Unmarshal(body, &v); err != nil {
|
||||
@@ -132,6 +225,20 @@ func (v *StandardValidator) LoadSchemaDocument(ctx context.Context, schemaPath s
|
||||
return doc, nil
|
||||
}
|
||||
|
||||
func (v *FSValidator) LoadSchemaDocument(ctx context.Context, schemaPath string) (any, error) {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
default:
|
||||
}
|
||||
|
||||
_, doc, err := v.loadSchemaDocument(schemaPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return doc, nil
|
||||
}
|
||||
|
||||
func (v *StandardValidator) resolveSchemaPath(schemaPath string) (string, error) {
|
||||
if strings.TrimSpace(schemaPath) == "" {
|
||||
return "", errors.New("schema path is required for json_schema validation")
|
||||
@@ -149,3 +256,70 @@ func (v *StandardValidator) resolveSchemaPath(schemaPath string) (string, error)
|
||||
|
||||
return resolved, nil
|
||||
}
|
||||
|
||||
func (v *FSValidator) loadSchemaDocument(schemaPath string) (string, any, error) {
|
||||
resolved, err := v.resolveSchemaPath(schemaPath)
|
||||
if err != nil {
|
||||
return "", nil, err
|
||||
}
|
||||
|
||||
raw, err := fs.ReadFile(v.fsys, resolved)
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("failed to read schema file %q: %w", resolved, err)
|
||||
}
|
||||
|
||||
var doc any
|
||||
if err := json.Unmarshal(raw, &doc); err != nil {
|
||||
return "", nil, fmt.Errorf("failed to decode JSON schema %q: %w", resolved, err)
|
||||
}
|
||||
return resolved, doc, nil
|
||||
}
|
||||
|
||||
func (v *FSValidator) resolveSchemaPath(schemaPath string) (string, error) {
|
||||
if strings.TrimSpace(schemaPath) == "" {
|
||||
return "", errors.New("schema path is required for json_schema validation")
|
||||
}
|
||||
if v.fsys == nil {
|
||||
return "", errors.New("schema filesystem is nil")
|
||||
}
|
||||
|
||||
cleanRoot := cleanFSRoot(v.root)
|
||||
rootInfo, err := fs.Stat(v.fsys, cleanRoot)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to access schema source %q: %w", cleanRoot, err)
|
||||
}
|
||||
|
||||
cleanSchemaPath := cleanSchemaFSPath(schemaPath)
|
||||
var resolved string
|
||||
if rootInfo.IsDir() {
|
||||
resolved = path.Join(cleanRoot, cleanSchemaPath)
|
||||
} else {
|
||||
if cleanSchemaPath != path.Base(cleanRoot) {
|
||||
return "", fmt.Errorf("schema path %q does not match schema file %q", cleanSchemaPath, path.Base(cleanRoot))
|
||||
}
|
||||
resolved = cleanRoot
|
||||
}
|
||||
|
||||
if _, err := fs.Stat(v.fsys, resolved); err != nil {
|
||||
return "", fmt.Errorf("failed to access schema file %q: %w", resolved, err)
|
||||
}
|
||||
return resolved, nil
|
||||
}
|
||||
|
||||
func cleanSchemaFSPath(schemaPath string) string {
|
||||
cleaned := strings.TrimSpace(schemaPath)
|
||||
cleaned = strings.TrimPrefix(path.Clean(cleaned), "/")
|
||||
return cleaned
|
||||
}
|
||||
|
||||
func cleanFSRoot(root string) string {
|
||||
root = strings.TrimSpace(root)
|
||||
if root == "" || root == "." {
|
||||
return "."
|
||||
}
|
||||
return strings.TrimPrefix(path.Clean(root), "/")
|
||||
}
|
||||
|
||||
func fsSchemaResourceURL(schemaName string) string {
|
||||
return "scriptorium-schema:///" + strings.TrimPrefix(path.Clean(schemaName), "/")
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
"gitea.maximumdirect.net/eric/scriptorium/internal/domain"
|
||||
)
|
||||
@@ -250,3 +251,77 @@ func TestStandardValidatorLoadSchemaDocumentInvalidJSON(t *testing.T) {
|
||||
t.Fatal("expected decode error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSValidatorJSONSchemaSuccess(t *testing.T) {
|
||||
v := NewFSValidator(fstest.MapFS{
|
||||
"schemas/events.schema.json": &fstest.MapFile{Data: []byte(`{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"type": "object",
|
||||
"required": ["events"],
|
||||
"properties": {
|
||||
"events": {"type": "array"}
|
||||
}
|
||||
}`)},
|
||||
}, "schemas")
|
||||
|
||||
res, err := v.Validate(context.Background(), &domain.Artifact{Body: []byte(`{"events":[]}`)}, domain.OutputContract{
|
||||
ValidationMode: domain.ValidationJSONSchema,
|
||||
SchemaPath: "events.schema.json",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if res.Status != domain.ValidationPassed || !res.IsValid {
|
||||
t.Fatalf("expected passed/valid, got status=%q valid=%v", res.Status, res.IsValid)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSValidatorSingleSchemaFileUsesBaseName(t *testing.T) {
|
||||
v := NewFSValidator(fstest.MapFS{
|
||||
"events.schema.json": &fstest.MapFile{Data: []byte(`{
|
||||
"type": "object",
|
||||
"required": ["events"],
|
||||
"properties": {
|
||||
"events": {"type": "array"}
|
||||
}
|
||||
}`)},
|
||||
}, "events.schema.json")
|
||||
|
||||
res, err := v.Validate(context.Background(), &domain.Artifact{Body: []byte(`{"events":[]}`)}, domain.OutputContract{
|
||||
ValidationMode: domain.ValidationJSONSchema,
|
||||
SchemaPath: "events.schema.json",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
if res.Status != domain.ValidationPassed || !res.IsValid {
|
||||
t.Fatalf("expected passed/valid, got status=%q valid=%v", res.Status, res.IsValid)
|
||||
}
|
||||
|
||||
_, err = v.Validate(context.Background(), &domain.Artifact{Body: []byte(`{"events":[]}`)}, domain.OutputContract{
|
||||
ValidationMode: domain.ValidationJSONSchema,
|
||||
SchemaPath: "other.schema.json",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("expected schema path mismatch error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestFSValidatorLoadSchemaDocument(t *testing.T) {
|
||||
v := NewFSValidator(fstest.MapFS{
|
||||
"schemas/schema.json": &fstest.MapFile{Data: []byte(`{"type":"object"}`)},
|
||||
}, "schemas")
|
||||
loader, ok := v.(SchemaDocumentLoader)
|
||||
if !ok {
|
||||
t.Fatal("fs validator must implement SchemaDocumentLoader")
|
||||
}
|
||||
|
||||
doc, err := loader.LoadSchemaDocument(context.Background(), "schema.json")
|
||||
if err != nil {
|
||||
t.Fatalf("expected no error, got %v", err)
|
||||
}
|
||||
obj, ok := doc.(map[string]any)
|
||||
if !ok || obj["type"] != "object" {
|
||||
t.Fatalf("unexpected schema document: %#v", doc)
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user