promptkit/docs/roadmap/validate.md

Request Preflight Validation

Status: accepted; implementation pending.

Purpose

Provide an explicit success/error operation for consumers that want to check an actual prompt request without generating a model response or receiving a prepared snapshot.

The check applies to concrete variable values and input bindings for one invocation. It does not certify that an application's configured bindings cover every possible request or template branch.

Scriptorium motivates this addition: applications invoke it as a subprocess and could use a validate command to fail early when their supplied variables, inputs, or other request settings cannot pass preparation. PromptKit owns the reusable request operation; Scriptorium owns command syntax, diagnostics, process exit statuses, and any application-specific interpretation.

Current Baseline

Engine.InspectRequirements discovers recognizable input and caller-variable dependencies without loading artifacts or rendering messages. Its Complete flag describes discovery coverage, not whether supplied dependencies satisfy a request. It is useful for authoring and planning, but is not actual preflight.

Engine.Prepare already performs preflight through request conversion and the shared internal preparation pipeline. It resolves the prompt, profile, effective settings, and output contract; loads input artifacts and applicable schema resources; and renders messages and the session ID without calling an LLM. The facade then converts the internal prepared value into a caller-owned PreparedRun. Consumers can discard that result today, but the API does not directly express a success/error-only operation.

See the consumer inspection and preparation guide, existing preparation GoDoc, and internal preparation flow.

Target Public API

Add a separate method on the existing engine:

func (e *Engine) ValidateRequest(ctx context.Context, req RunRequest) error

ValidateRequest names the object being checked and distinguishes this operation from generated-output validation. validate remains the recommended downstream CLI command name; preflight describes the workflow, without implying provider or capacity readiness checks.

Use the complete existing RunRequest, including input bindings, caller variables, profile selection, execution overrides, session settings, output contract overrides, and appended messages. Apply the same request conversion and preparation rules as Prepare; validation is not a name-only check or an analysis of declared requirements.

A nil error means the supplied request passed preparation at the time of the check. A failure returns the error encountered through the existing preparation path, with the same public error categories and applicable error identities. A nil engine follows Prepare's ErrInvalidConfig convention. Context handling follows preparation rather than introducing a stronger cancellation guarantee. The result does not aggregate all missing requirements or return a prepared value, execution handle, or model output.

Keep Prepare unchanged. A separate method avoids a mode flag that changes the meaning of its returned snapshot. The intended distinction is:

• InspectRequirements: discover what a prompt may require for a supplied context, including partial discovery where appropriate.
• ValidateRequest: determine whether the actual supplied request passes preparation.
• Prepare: perform preparation and return its snapshot.
• PrepareExecution: prepare an owned snapshot that can later be executed.
• Run: prepare and execute the request.

Meaning And Limits

Validation exercises actual preparation, including loading inputs and rendering the branches selected by supplied values. It applies existing rules to missing bindings and variables rather than adding stricter required-field rules, rejecting additional caller keys, or restricting supported templates.

For example, consider a message containing {{if eq .mode "detailed"}}{{input "details"}}{{end}}, where details is not declared required. A request with mode=brief can prepare without that input; one with mode=detailed needs it. Declaring the input required makes it necessary regardless of the selected branch. Similarly, an evaluated {{.audience}} fails when the key is absent, but supplying an empty string does not itself fail preparation. Optional lookups such as {{index . "audience"}} retain their existing semantics. These examples illustrate preparation parity, not new required-field rules; see the input and template formats and requirement classifications.

No LLM request is made. This is not a promise of zero I/O: artifact and schema loaders still perform their existing resource work, including caller-supplied loader behavior. The current preparation path loads every supplied input, even if the selected template branch does not use it. Avoiding inference makes the operation useful for fail-fast workflows, but large inputs or resource processing can still cost time and memory. No fixed latency or allocation guarantee is proposed.

Success does not establish provider acceptance, context-window fit, generated output validity, or application-domain correctness. It also does not reserve execution capacity or retain a snapshot for a subsequent invocation. Later requests can differ, and referenced resources can change. Consumers that need execution tied to the checked snapshot within one process and engine should use PrepareExecution. Its opaque handle cannot connect separate subprocess invocations.

This is request-scoped validation using an already constructed engine. It does not expand configuration ownership beyond the final handoff described by ADR 0001, or introduce eager validation of all configured prompts and resources.

Downstream CLI Guidance

Scriptorium's proposed validate and existing run commands should share request construction: argument interpretation, defaults, profile selection, and overrides must describe the same request for equivalent caller inputs. This allows PromptKit's preparation parity to carry through the CLI boundary. Scriptorium owns exit statuses and diagnostic presentation.

A scriptorium validate call followed by scriptorium run performs two independent preparations; the first does not retain an executable snapshot for the second. When assessing CLI latency, measure the complete subprocess call, including process startup and engine construction, as well as preparation. Avoiding the public prepared snapshot does not eliminate the other preparation work. These are downstream integration recommendations, not Scriptorium changes included in this unit of work.

Work Needed

• Add the public error-only method through the existing facade and internal preparation path. Reuse request conversion, preparation, and error mapping rather than implementing a second validation engine or an inspection-based approximation.
• Discard the internal prepared result without converting it into a public snapshot. The existing internal prepared value may still be constructed; removing all preparation allocations is not required for this addition. Share small facade plumbing where helpful without a broad refactor.
• Preserve preparation's existing context, credential, resource, and execution separation behavior. No model generation, output-repair loop, or capacity admission is needed for the new operation.
• Add focused external-package tests for successful preparation and representative missing-variable/input, resource, selection, and invalid-request failures. Use existing fixtures where suitable to verify parity with Prepare, including a branch-sensitive example and the nil-engine boundary. Check public error identities rather than exact messages, and confirm that the injected model client is not called. Avoid duplicating the full preparation test suite.
• Document the method in GoDoc and add a short consumer example distinguishing validation from requirement discovery and prepared execution. Update the internal runner description where needed when implementation lands, and record the optional API addition in its eventual release notes.

Completion And Scope

The feature is complete when consumers can submit an ordinary RunRequest to Engine.ValidateRequest and obtain preparation success or its existing failure without a model call or public prepared snapshot, with focused tests and canonical documentation establishing that behavior.

This unit of work does not include changes to Scriptorium, CLI commands or exit codes, aggregated diagnostics, presence-only validation modes, provider probes, domain-aware validation, or new execution guarantees. It introduces no changes to prompt/profile formats or existing preparation and execution semantics.