Prepare v0.15.0 release documentation

README.md

@@ -37,6 +37,9 @@ boundary and constraints that framework work must preserve.
 
 ## Release Guidance
 
+The [v0.15.0 release notes](docs/releases/v0.15.0.md) cover error-only
+validation of concrete requests with `Engine.ValidateRequest`.
+
 Consumers upgrading to `v0.12.0` should read the
 [v0.12.0 release notes](docs/releases/v0.12.0.md) for the intentional ConfigKit
 configuration and public construction API migration.

docs/consumers/pkg-promptkit.md

@@ -284,6 +284,41 @@ conditional entries. Do not treat an empty partial inventory as proof that no
 input bindings are needed. See the method's [GoDoc](../../requirements.go) for
 the exact classifications, supported syntax, ownership, and error contract.
 
+## Validate A Concrete Request
+
+Use [`Engine.ValidateRequest`](../../engine.go) when the application has a
+complete `RunRequest` and needs to know whether that concrete invocation passes
+preparation:
+
+```go
+request := promptkit.RunRequest{
+	PromptID: "meeting.summary",
+	Vars:     map[string]string{"audience": "project leads"},
+	Inputs: map[string]promptkit.ArtifactRef{
+		"note": promptkit.Inline("Synthetic meeting notes"),
+	},
+}
+if err := engine.ValidateRequest(ctx, request); err != nil {
+	return err
+}
+```
+
+Unlike `InspectRequirements`, this checks supplied values, the selected template
+branches, and request bindings through the existing preparation path. It checks
+only this invocation, does not aggregate errors, and loads every supplied input
+according to ordinary preparation behavior, even if a branch does not use it.
+It makes no model call and returns no prepared snapshot, but artifact and schema
+resource I/O and its time and memory costs remain. Success does not certify
+provider acceptance or later execution success. See the method's GoDoc for its
+exact error and context behavior.
+
+Scriptorium can map the returned error to its own diagnostics and exit status.
+Its `validate` and `run` commands should construct equivalent requests for
+equivalent caller inputs so their preparation behavior agrees. Separate
+subprocess calls prepare independently; an opaque `PrepareExecution` handle
+cannot be transferred from `validate` to `run`. Use `PrepareExecution` when one
+process and engine must execute the exact prepared snapshot.
+
 ## Prepare Without Model Execution
 
 [`Engine.Prepare`](../../engine.go) resolves the selected prompt and profile,
@@ -304,8 +339,11 @@ prepared, err := engine.Prepare(ctx, promptkit.RunRequest{
 The maintained
 [offline preparation example](../../examples/go-library/prepare/main.go)
 shows a complete runnable setup with a prompt file, in-memory profile, and
-inline input. Exact request requirements and prepared-result fields belong to
+inline input. It demonstrates validation and preparation with the same request
-the [`RunRequest` and `PreparedRun` GoDoc](../../types.go).
+to make the two operations concrete; this performs preparation twice and is
+illustrative, not a requirement to validate before `Prepare` or `Run`. Exact
+request requirements and prepared-result fields belong to the
+[`RunRequest` and `PreparedRun` GoDoc](../../types.go).
 
 ## Prepare Now And Execute The Same Snapshot Later
 

docs/internal/overview.md

@@ -11,7 +11,7 @@ contributor workflow and validation.
 
 | Component | Implemented responsibility | References |
 | --- | --- | --- |
-| Root `promptkit` package | Provides the supported engine facade, ConfigKit source helpers, profile and backend mapping conveniences, injection options, public requests and results, prompt/profile/requirement inspection and identity listing, observation values, opaque prepared-execution handles, value conversion, redacted formatting, and typed capacity and generation error mapping. | [Package GoDoc](../../doc.go), [observations](../../observations.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine facade](../../engine.go), [identity listing](../../listing.go), [requirement facade](../../requirements.go) |
+| Root `promptkit` package | Provides the supported engine facade, ConfigKit source helpers, profile and backend mapping conveniences, injection options, public requests and results, prompt/profile/requirement inspection and identity listing, concrete-request validation, observation values, opaque prepared-execution handles, value conversion, redacted formatting, and typed capacity and generation error mapping. | [Package GoDoc](../../doc.go), [observations](../../observations.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine facade](../../engine.go), [identity listing](../../listing.go), [requirement facade](../../requirements.go) |
 | `examples/go-library/prepare` | Demonstrates offline preparation and explicit configuration inspection with a prompt file, programmatic profile, and inline input. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
 | `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, programmatic profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
 | `examples/go-library/observe` | Demonstrates offline built-in HTTP retry and synchronous metadata observation across structured proposal and assessment runs using embedded sources, two profiles on one finite backend, and an application decision. It is not a public library package. | [Example program](../../examples/go-library/observe/main.go) |

docs/internal/runner.md

@@ -111,6 +111,12 @@ into its private payload, while deep-copying mutable provider extras and
 creating an independent details snapshot. Claim completion or discard clears
 the execution payload without removing the details snapshot.
 
+The root facade's `Engine.ValidateRequest` uses the same complete preparation
+path as `Prepare` and discards the resulting internal prepared value without
+converting it into a public snapshot. It does not add a runner operation or
+validator component; request conversion and public error mapping remain at the
+facade boundary.
+
 Pointer-based numeric overrides preserve an explicit zero. Invalid negative or
 out-of-range values fail as invalid requests. Endpoint overrides do not change
 the selected backend identity. Non-empty extra-parameter maps replace whole

docs/releases/v0.15.0.md

@@ -0,0 +1,34 @@
+# Promptkit v0.15.0
+
+This release adds error-only validation of a concrete prompt request without
+calling a model. The minimum supported Go version remains 1.26.0.
+
+## Validate A Concrete Request
+
+`Engine.ValidateRequest` accepts an ordinary `RunRequest` and returns nil when
+it passes preparation, or the existing preparation error. Consumers can use it
+for fail-fast checks, including downstream CLI validation commands, without
+receiving a prepared snapshot.
+
+The method shares `Prepare`'s request conversion and preparation path. Unlike
+requirements inspection, it checks the actual supplied values and bindings.
+Artifact and schema resource I/O still occurs; success does not certify provider
+acceptance or later execution success. A subsequent run prepares independently.
+Validation is optional, not a required step before preparation or execution.
+
+See the [request validation guide](../consumers/pkg-promptkit.md#validate-a-concrete-request)
+and [GoDoc](../../engine.go) for usage and the exact contract. Consumers that
+need execution tied to a checked snapshot can use the existing
+[prepared execution workflow](../consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later).
+
+## Compatibility And Upgrade
+
+No application-code, configuration, or Go-toolchain migration is required from
+v0.14.0. This is an optional API addition; existing inspection, preparation, and
+execution workflows remain available.
+
+```sh
+go get gitea.maximumdirect.net/eric/promptkit@v0.15.0
+```
+
+[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.15.0/COPYRIGHT) | [LICENSE](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.15.0/LICENSE) | [ATTRIBUTION](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.15.0/docs/oss/attribution.md)

engine.go

@@ -346,6 +346,32 @@ func (e *Engine) InspectProfile(ctx context.Context, profileID string) (*Profile
 // relationship to ctx.Err is promised. Prepare returns no partial result on
 // error. It validates RunRequest.TransientRetries but makes no model attempts.
 func (e *Engine) Prepare(ctx context.Context, req RunRequest) (*PreparedRun, error) {
+	prepared, err := e.prepareRequest(ctx, req)
+	if err != nil {
+		return nil, err
+	}
+	return fromDomainPreparedRun(prepared), nil
+}
+
+// ValidateRequest reports whether req passes the same preparation path used by
+// [Engine.Prepare] at the time of this call. A nil error means this concrete
+// request passed preparation. It checks only this invocation and returns errors
+// as encountered rather than aggregating them. All supplied inputs follow the
+// same loading behavior as [Engine.Prepare], including inputs unused by the
+// selected template branch. The method does not call an LLM or retain a
+// prepared snapshot or executable handle, but preparation may still perform
+// resource I/O and incur its associated time and memory costs.
+//
+// Its errors and context handling follow [Engine.Prepare], including an error
+// matching ErrInvalidConfig when the Engine is nil. Success does not guarantee
+// provider acceptance or later execution success. Use [Engine.PrepareExecution]
+// when execution must use the snapshot produced by preparation.
+func (e *Engine) ValidateRequest(ctx context.Context, req RunRequest) error {
+	_, err := e.prepareRequest(ctx, req)
+	return err
+}
+
+func (e *Engine) prepareRequest(ctx context.Context, req RunRequest) (*domain.PreparedRun, error) {
 	ctx = observation.WithoutExecutionState(ctx)
 	if e == nil || e.internal == nil {
 		return nil, fmt.Errorf("%w: engine is nil", ErrInvalidConfig)
@@ -360,7 +386,7 @@ func (e *Engine) Prepare(ctx context.Context, req RunRequest) (*PreparedRun, err
 	if err != nil {
 		return nil, mapPublicError(err)
 	}
-	return fromDomainPreparedRun(prepared), nil
+	return prepared, nil
 }
 
 // PrepareExecution completely prepares a prompt request without calling the

examples/go-library/prepare/main.go

@@ -48,12 +48,16 @@ func main() {
 		exit(fmt.Errorf("required note input was not discovered before preparation"))
 	}
 
-	prepared, err := engine.Prepare(context.Background(), promptkit.RunRequest{
+	request := promptkit.RunRequest{
 		PromptID: "example.prepare",
 		Inputs: map[string]promptkit.ArtifactRef{
 			"note": promptkit.Inline("Ada finished the migration review."),
 		},
-	})
+	}
+	if err := engine.ValidateRequest(context.Background(), request); err != nil {
+		exit(err)
+	}
+	prepared, err := engine.Prepare(context.Background(), request)
 	if err != nil {
 		exit(err)
 	}

validate_request_contract_test.go

@@ -0,0 +1,218 @@
+package promptkit_test
+
+import (
+	"context"
+	"errors"
+	"path/filepath"
+	"sync/atomic"
+	"testing"
+	"testing/fstest"
+
+	"gitea.maximumdirect.net/eric/promptkit"
+)
+
+func TestValidateRequestMatchesPrepareContract(t *testing.T) {
+	var modelCalls atomic.Int64
+	profiles := map[string]any{
+		"offline": map[string]any{"endpoint": "http://example.test/v1", "model": "offline-model"},
+	}
+	newEngine := func(t *testing.T, definition map[string]any) *promptkit.Engine {
+		t.Helper()
+		return requirementsEngine(t, oneVersionPrompt("synthetic", "v1", definition), profiles,
+			promptkit.WithLLMClient(countingLLMClient{calls: &modelCalls}))
+	}
+
+	t.Run("successful request", func(t *testing.T) {
+		engine := newEngine(t, contractPrompt("offline", "hello", nil, "", noValidationOutput))
+		assertValidateRequestParity(t, engine, promptkit.RunRequest{PromptID: "synthetic"}, nil)
+	})
+
+	t.Run("nil engine", func(t *testing.T) {
+		var engine *promptkit.Engine
+		request := promptkit.RunRequest{PromptID: "synthetic"}
+		assertValidateRequestParity(t, engine, request, promptkit.ErrInvalidConfig)
+	})
+
+	t.Run("invalid appended message", func(t *testing.T) {
+		engine := newEngine(t, contractPrompt("offline", "hello", nil, "", noValidationOutput))
+		request := promptkit.RunRequest{
+			PromptID: "synthetic",
+			AppendedMessages: []promptkit.RenderedMessage{{
+				Role: "unsupported-role", Content: "not sent to a model",
+			}},
+		}
+		assertValidateRequestParity(t, engine, request, promptkit.ErrInvalidRequest)
+	})
+
+	t.Run("missing required input", func(t *testing.T) {
+		definition := contractPrompt("offline", `{{input "required"}}`, []any{
+			map[string]any{"name": "required", "required": true},
+		}, "", noValidationOutput)
+		engine := newEngine(t, definition)
+		assertValidateRequestParity(t, engine, promptkit.RunRequest{PromptID: "synthetic"}, promptkit.ErrPromptRender)
+	})
+
+	if got := modelCalls.Load(); got != 0 {
+		t.Fatalf("preparation operations made %d model calls, want none", got)
+	}
+}
+
+func assertValidateRequestParity(t *testing.T, engine *promptkit.Engine, request promptkit.RunRequest, want error) *promptkit.PreparedRun {
+	t.Helper()
+	prepared, prepareErr := engine.Prepare(context.Background(), request)
+	validateErr := engine.ValidateRequest(context.Background(), request)
+	if want == nil {
+		if prepareErr != nil || prepared == nil {
+			t.Fatalf("Prepare() = (%+v, %v), want a prepared result", prepared, prepareErr)
+		}
+		if validateErr != nil {
+			t.Fatalf("ValidateRequest() error = %v, want nil", validateErr)
+		}
+		return prepared
+	}
+	if prepared != nil || !errors.Is(prepareErr, want) {
+		t.Fatalf("Prepare() = (%+v, %v), want error matching %v", prepared, prepareErr, want)
+	}
+	if !errors.Is(validateErr, want) {
+		t.Fatalf("ValidateRequest() error = %v, want error matching %v", validateErr, want)
+	}
+	return nil
+}
+
+func TestValidateRequestUsesSelectedTemplateBranchesAndVariablePresence(t *testing.T) {
+	var modelCalls atomic.Int64
+	profiles := map[string]any{
+		"offline": map[string]any{"endpoint": "http://example.test/v1", "model": "offline-model"},
+	}
+	branchContent := `{{if eq .mode "detailed"}}{{input "details"}}{{end}}`
+	optionalDefinition := contractPrompt("offline", branchContent, []any{
+		map[string]any{"name": "details", "required": false},
+	}, "", noValidationOutput)
+	requiredDefinition := contractPrompt("offline", branchContent, []any{
+		map[string]any{"name": "details", "required": true},
+	}, "", noValidationOutput)
+	engine := requirementsEngine(t, map[string]any{
+		"optional": oneVersionPrompt("optional", "v1", optionalDefinition)["optional"],
+		"required": oneVersionPrompt("required", "v1", requiredDefinition)["required"],
+		"direct":   oneVersionPrompt("direct", "v1", contractPrompt("offline", `{{.audience}}`, nil, "", noValidationOutput))["direct"],
+		"index":    oneVersionPrompt("index", "v1", contractPrompt("offline", `{{index . "audience"}}`, nil, "", noValidationOutput))["index"],
+	}, profiles, promptkit.WithLLMClient(countingLLMClient{calls: &modelCalls}))
+
+	cases := []struct {
+		name    string
+		request promptkit.RunRequest
+		want    error
+	}{
+		{name: "brief branch omits optional input", request: promptkit.RunRequest{PromptID: "optional", Vars: map[string]string{"mode": "brief"}}},
+		{name: "detailed branch needs optional input", request: promptkit.RunRequest{PromptID: "optional", Vars: map[string]string{"mode": "detailed"}}, want: promptkit.ErrPromptRender},
+		{name: "detailed branch accepts inline input", request: promptkit.RunRequest{PromptID: "optional", Vars: map[string]string{"mode": "detailed"}, Inputs: map[string]promptkit.ArtifactRef{"details": promptkit.Inline("Long form details")}}},
+		{name: "declared required input is needed on brief branch", request: promptkit.RunRequest{PromptID: "required", Vars: map[string]string{"mode": "brief"}}, want: promptkit.ErrPromptRender},
+		{name: "evaluated variable needs a supplied key", request: promptkit.RunRequest{PromptID: "direct"}, want: promptkit.ErrPromptRender},
+		{name: "empty value and unrelated variable are accepted", request: promptkit.RunRequest{PromptID: "direct", Vars: map[string]string{"audience": "", "unrelated": "accepted"}}},
+		{name: "missing index lookup remains optional", request: promptkit.RunRequest{PromptID: "index"}},
+	}
+	for _, test := range cases {
+		t.Run(test.name, func(t *testing.T) {
+			assertValidateRequestParity(t, engine, test.request, test.want)
+		})
+	}
+	if got := modelCalls.Load(); got != 0 {
+		t.Fatalf("preparation operations made %d model calls, want none", got)
+	}
+}
+
+type canceledValidationArtifactReader struct{}
+
+func (canceledValidationArtifactReader) Read(context.Context, promptkit.ArtifactRef) (*promptkit.Artifact, error) {
+	return nil, context.Canceled
+}
+
+func TestValidateRequestPreservesResourceAndSelectionFailures(t *testing.T) {
+	var modelCalls atomic.Int64
+	profiles := map[string]any{
+		"offline": map[string]any{"endpoint": "http://example.test/v1", "model": "offline-model"},
+	}
+	branch := `{{if eq .mode "detailed"}}{{input "details"}}{{end}}`
+	definition := contractPrompt("offline", branch, []any{
+		map[string]any{"name": "details", "required": false},
+	}, "", noValidationOutput)
+	profilePrompt := contractPrompt("offline", "hello", nil, "", noValidationOutput)
+	schemaPrompt := contractPrompt("offline", "{}", nil, "", map[string]any{
+		"format": "json", "validation_mode": "json_schema", "schema_path": "missing-schema.json",
+	})
+	prompts := map[string]any{
+		"branch":  oneVersionPrompt("branch", "v1", definition)["branch"],
+		"profile": oneVersionPrompt("profile", "v1", profilePrompt)["profile"],
+		"schema":  oneVersionPrompt("schema", "v1", schemaPrompt)["schema"],
+	}
+	engine := requirementsEngine(t, prompts, profiles,
+		promptkit.WithSchemaFS(fstest.MapFS{}, "."),
+		promptkit.WithLLMClient(countingLLMClient{calls: &modelCalls}),
+	)
+
+	t.Run("unselected supplied input is still loaded", func(t *testing.T) {
+		missingPath := filepath.Join(t.TempDir(), "missing-input.txt")
+		request := promptkit.RunRequest{
+			PromptID: "branch",
+			Vars:     map[string]string{"mode": "brief"},
+			Inputs:   map[string]promptkit.ArtifactRef{"details": promptkit.File(missingPath)},
+		}
+		assertValidateRequestParity(t, engine, request, promptkit.ErrArtifactLoad)
+	})
+	t.Run("missing selected profile", func(t *testing.T) {
+		request := promptkit.RunRequest{PromptID: "profile", ProfileID: "not-configured"}
+		assertValidateRequestParity(t, engine, request, promptkit.ErrProfileNotFound)
+	})
+	t.Run("unavailable selected schema", func(t *testing.T) {
+		request := promptkit.RunRequest{PromptID: "schema"}
+		assertValidateRequestParity(t, engine, request, promptkit.ErrValidation)
+	})
+	if got := modelCalls.Load(); got != 0 {
+		t.Fatalf("preparation operations made %d model calls, want none", got)
+	}
+}
+
+func TestValidateRequestPreservesCanceledLoaderCategory(t *testing.T) {
+	var modelCalls atomic.Int64
+	definition := contractPrompt("offline", `{{if eq .mode "detailed"}}{{input "details"}}{{end}}`, []any{
+		map[string]any{"name": "details", "required": false},
+	}, "", noValidationOutput)
+	engine := requirementsEngine(t, oneVersionPrompt("synthetic", "v1", definition), map[string]any{
+		"offline": map[string]any{"endpoint": "http://example.test/v1", "model": "offline-model"},
+	}, promptkit.WithArtifactReader(canceledValidationArtifactReader{}),
+		promptkit.WithLLMClient(countingLLMClient{calls: &modelCalls}))
+	request := promptkit.RunRequest{
+		PromptID: "synthetic",
+		Vars:     map[string]string{"mode": "brief"},
+		Inputs:   map[string]promptkit.ArtifactRef{"details": promptkit.File("controlled-canceled-loader")},
+	}
+	assertValidateRequestParity(t, engine, request, promptkit.ErrArtifactLoad)
+	if got := modelCalls.Load(); got != 0 {
+		t.Fatalf("preparation operations made %d model calls, want none", got)
+	}
+}
+
+func TestValidateRequestForwardsProfileOverridesAndRequestSettings(t *testing.T) {
+	var modelCalls atomic.Int64
+	definition := contractPrompt("partial", "base message", nil, "", noValidationOutput)
+	engine := requirementsEngine(t, oneVersionPrompt("synthetic", "v1", definition), map[string]any{
+		"partial": map[string]any{"model": "partial-model"},
+	}, promptkit.WithLLMClient(countingLLMClient{calls: &modelCalls}))
+	request := promptkit.RunRequest{
+		PromptID:  "synthetic",
+		SessionID: "request-session",
+		Execution: &promptkit.ExecutionTargetOverride{Endpoint: "http://override.example/v1"},
+		AppendedMessages: []promptkit.RenderedMessage{{
+			Role: promptkit.RoleAssistant, Content: "caller suffix",
+		}},
+	}
+	prepared := assertValidateRequestParity(t, engine, request, nil)
+	if prepared.SelectedProfileID != "partial" || prepared.EffectiveModelParams.Model != "partial-model" ||
+		prepared.EffectiveModelParams.Endpoint != "http://override.example/v1" || prepared.SessionID != "request-session" ||
+		len(prepared.Messages) != 2 || prepared.Messages[1].Content != "caller suffix" {
+		t.Fatalf("preparation fixture did not exercise profile and request settings: %+v", prepared)
+	}
+	if got := modelCalls.Load(); got != 0 {
+		t.Fatalf("preparation operations made %d model calls, want none", got)
+	}
+}