Prepare v0.15.0 release documentation
This commit is contained in:
@@ -37,6 +37,9 @@ boundary and constraints that framework work must preserve.
|
|||||||
|
|
||||||
## Release Guidance
|
## Release Guidance
|
||||||
|
|
||||||
|
The [v0.15.0 release notes](docs/releases/v0.15.0.md) cover error-only
|
||||||
|
validation of concrete requests with `Engine.ValidateRequest`.
|
||||||
|
|
||||||
Consumers upgrading to `v0.12.0` should read the
|
Consumers upgrading to `v0.12.0` should read the
|
||||||
[v0.12.0 release notes](docs/releases/v0.12.0.md) for the intentional ConfigKit
|
[v0.12.0 release notes](docs/releases/v0.12.0.md) for the intentional ConfigKit
|
||||||
configuration and public construction API migration.
|
configuration and public construction API migration.
|
||||||
|
|||||||
34
docs/releases/v0.15.0.md
Normal file
34
docs/releases/v0.15.0.md
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
# Promptkit v0.15.0
|
||||||
|
|
||||||
|
This release adds error-only validation of a concrete prompt request without
|
||||||
|
calling a model. The minimum supported Go version remains 1.26.0.
|
||||||
|
|
||||||
|
## Validate A Concrete Request
|
||||||
|
|
||||||
|
`Engine.ValidateRequest` accepts an ordinary `RunRequest` and returns nil when
|
||||||
|
it passes preparation, or the existing preparation error. Consumers can use it
|
||||||
|
for fail-fast checks, including downstream CLI validation commands, without
|
||||||
|
receiving a prepared snapshot.
|
||||||
|
|
||||||
|
The method shares `Prepare`'s request conversion and preparation path. Unlike
|
||||||
|
requirements inspection, it checks the actual supplied values and bindings.
|
||||||
|
Artifact and schema resource I/O still occurs; success does not certify provider
|
||||||
|
acceptance or later execution success. A subsequent run prepares independently.
|
||||||
|
Validation is optional, not a required step before preparation or execution.
|
||||||
|
|
||||||
|
See the [request validation guide](../consumers/pkg-promptkit.md#validate-a-concrete-request)
|
||||||
|
and [GoDoc](../../engine.go) for usage and the exact contract. Consumers that
|
||||||
|
need execution tied to a checked snapshot can use the existing
|
||||||
|
[prepared execution workflow](../consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later).
|
||||||
|
|
||||||
|
## Compatibility And Upgrade
|
||||||
|
|
||||||
|
No application-code, configuration, or Go-toolchain migration is required from
|
||||||
|
v0.14.0. This is an optional API addition; existing inspection, preparation, and
|
||||||
|
execution workflows remain available.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go get gitea.maximumdirect.net/eric/promptkit@v0.15.0
|
||||||
|
```
|
||||||
|
|
||||||
|
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.15.0/COPYRIGHT) | [LICENSE](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.15.0/LICENSE) | [ATTRIBUTION](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.15.0/docs/oss/attribution.md)
|
||||||
@@ -1,195 +0,0 @@
|
|||||||
# Request Validation Implementation Plan
|
|
||||||
|
|
||||||
Implement [the finalized feature roadmap](validate.md) in the numbered order
|
|
||||||
below. Each stage is scoped for one Luna coding-agent prompt. The feature
|
|
||||||
roadmap owns the target behavior and downstream intent; this document owns the
|
|
||||||
implementation sequence.
|
|
||||||
|
|
||||||
## Baseline And Implementation Decisions
|
|
||||||
|
|
||||||
The baseline is commit `0c3651f`. `Engine.Prepare` in `engine.go` clears execution
|
|
||||||
observation state, checks the engine, converts `RunRequest`, calls
|
|
||||||
`internal/engine.Prepare`, maps preparation errors, and converts the returned
|
|
||||||
internal value into a caller-owned public snapshot. The internal engine already
|
|
||||||
delegates to `internal/usecase.Runner.Prepare`, which runs both preparation
|
|
||||||
phases without admission or generation.
|
|
||||||
|
|
||||||
Add exactly this public operation:
|
|
||||||
|
|
||||||
```go
|
|
||||||
func (e *Engine) ValidateRequest(ctx context.Context, req RunRequest) error
|
|
||||||
```
|
|
||||||
|
|
||||||
Extract the existing facade preparation plumbing into an unexported
|
|
||||||
`(*Engine).prepareRequest` helper returning `(*domain.PreparedRun, error)`.
|
|
||||||
`Prepare` will call that helper and retain its existing public result conversion;
|
|
||||||
`ValidateRequest` will call it and discard the internal result. This shares
|
|
||||||
request conversion, context handling, nil-engine checks, and error mapping while
|
|
||||||
avoiding public snapshot copies for validation. The private helper is an
|
|
||||||
implementation choice, not a consumer contract.
|
|
||||||
|
|
||||||
No new internal-engine or runner operation is needed. Internal prepared values,
|
|
||||||
hashing, schema preparation, and rendering may still occur; eliminating them
|
|
||||||
would change the selected preparation-parity behavior or require work outside
|
|
||||||
this small addition.
|
|
||||||
|
|
||||||
## Working Instructions
|
|
||||||
|
|
||||||
- Start with [the development guide](../development.md) and read the applicable
|
|
||||||
[agent](../policy/agents.md), [architecture](../policy/architecture.md),
|
|
||||||
[testing](../policy/testing.md), and [documentation](../policy/documentation.md)
|
|
||||||
policies. Use the existing facade GoDoc and
|
|
||||||
[runner description](../internal/runner.md#shared-preparation-pipeline) as
|
|
||||||
current-behavior references.
|
|
||||||
- Preserve `Prepare`'s behavior and error wrapping, including its existing
|
|
||||||
context classification. Do not add a general `errors.Is(err, ctx.Err())`
|
|
||||||
guarantee, new error sentinels, aggregated diagnostics, stricter binding
|
|
||||||
rules, provider checks, or capacity checks.
|
|
||||||
- Use the full `RunRequest` conversion path, not a separate subset of its fields
|
|
||||||
or a dependency inventory from `InspectRequirements`. Leave credential,
|
|
||||||
resource, template, and request-format rules unchanged.
|
|
||||||
- Keep tests offline, deterministic, and external-package at the public boundary.
|
|
||||||
Use real preparation collaborators and small injected doubles for external
|
|
||||||
effects. Test representative forwarding risks rather than reproducing the
|
|
||||||
complete preparation suite.
|
|
||||||
- Commit each completed stage after its checks and relevant diff review, with a
|
|
||||||
plain-English message. Do not push, tag, or run a publication command as part
|
|
||||||
of this plan.
|
|
||||||
|
|
||||||
## Stage 1: Share Facade Preparation And Add ValidateRequest
|
|
||||||
|
|
||||||
Scope: the small facade change, GoDoc, and basic contract tests.
|
|
||||||
|
|
||||||
1. Review `Engine.Prepare`, `toDomainRunRequest`, `mapPublicError`, and
|
|
||||||
`fromDomainPreparedRun`. Extract only the existing preparation plumbing into
|
|
||||||
the private helper described above. Keep the existing operation order and
|
|
||||||
wrapping at each error boundary; do not change error semantics as incidental
|
|
||||||
cleanup.
|
|
||||||
2. Make `Prepare` call the helper and perform the existing
|
|
||||||
`fromDomainPreparedRun` conversion on success. Its signature, snapshot
|
|
||||||
ownership, and failure behavior remain unchanged.
|
|
||||||
3. Add `ValidateRequest` in `engine.go`. Return the helper's error, discarding its
|
|
||||||
internal prepared value. Do not call the public `Prepare` method, construct a
|
|
||||||
public snapshot, or introduce an internal validation pipeline.
|
|
||||||
4. Add GoDoc explaining that nil means the concrete request passed preparation
|
|
||||||
now, no LLM call occurs, resource I/O still occurs, and no executable snapshot
|
|
||||||
is retained. State parity with preparation's errors and context handling,
|
|
||||||
including the nil-engine `ErrInvalidConfig` convention. Link conceptually to
|
|
||||||
`Prepare` and `PrepareExecution`; do not promise subsequent provider success.
|
|
||||||
5. Add `validate_request_contract_test.go` in package `promptkit_test`. Reuse
|
|
||||||
compatible helpers from `requirements_inspection_contract_test.go` and
|
|
||||||
`consumer_test_helpers_test.go` where helpful. Construct a synthetic prompt
|
|
||||||
and explicit offline profile rather than depending on catalog model choices.
|
|
||||||
Cover a successful request, a nil engine, a conversion-level invalid request
|
|
||||||
such as an invalid appended-message role, and one missing required input.
|
|
||||||
6. In those tests, invoke both `Prepare` and `ValidateRequest` for the same
|
|
||||||
test-controlled request and compare success or the relevant public sentinel
|
|
||||||
through `errors.Is`. Use one intentional failure per case rather than
|
|
||||||
expecting a particular error from several independently failing inputs.
|
|
||||||
Inject a counting client and verify neither operation generates a model call.
|
|
||||||
7. Format changed Go files and run
|
|
||||||
`go test . ./internal/usecase ./internal/prompt`, followed by
|
|
||||||
`git diff --check`. Existing `Prepare` tests must still pass.
|
|
||||||
|
|
||||||
Completion: external consumers can call the new error-only method, all facade
|
|
||||||
preparation plumbing is shared, and the public snapshot conversion remains only
|
|
||||||
on `Prepare`'s success path.
|
|
||||||
|
|
||||||
**Completed: 2026-10-12.**
|
|
||||||
|
|
||||||
## Stage 2: Verify Concrete-Request Parity And Resource Boundaries
|
|
||||||
|
|
||||||
Scope: focused tests of the finalized roadmap's clarifications.
|
|
||||||
|
|
||||||
1. Extend the root contract tests with a prompt using
|
|
||||||
`{{if eq .mode "detailed"}}{{input "details"}}{{end}}`. For an input not
|
|
||||||
declared required, verify `mode=brief` succeeds without it and
|
|
||||||
`mode=detailed` fails without it but succeeds with an inline binding. Use a
|
|
||||||
second definition with the input declared required to verify absence fails
|
|
||||||
even on the brief branch. Compare each case with `Prepare`.
|
|
||||||
2. Cover an evaluated direct variable lookup with an absent key and with an
|
|
||||||
explicitly supplied empty string. Include a missing `index` lookup to show
|
|
||||||
its existing optional semantics are retained. These assertions concern
|
|
||||||
preparation only, not nonempty model-output rules. An extra unrelated caller
|
|
||||||
variable must remain acceptable where `Prepare` accepts it.
|
|
||||||
3. Add a case with an unreadable supplied input that is not referenced on the
|
|
||||||
selected branch. A synthetic missing file under `t.TempDir()` is sufficient.
|
|
||||||
Both operations must report the applicable artifact-load error, proving
|
|
||||||
validation uses actual preparation instead of a names-only or lazy subset.
|
|
||||||
4. Cover representative selected-profile and output-schema preparation failures
|
|
||||||
with existing valid fixtures: a missing profile should retain
|
|
||||||
`ErrProfileNotFound`, and an unavailable selected schema should retain the
|
|
||||||
category already returned by `Prepare`. Avoid adding source-level or
|
|
||||||
intermediate-configuration validation.
|
|
||||||
5. Add one successful request with a profile default and a runtime override that
|
|
||||||
completes a partial profile, plus a valid direct session ID and appended
|
|
||||||
message where useful. Inspect `Prepare`'s result only as needed to establish
|
|
||||||
that the fixture exercised those settings; `ValidateRequest` must accept the
|
|
||||||
same request. Existing tests own the exhaustive override and message matrix.
|
|
||||||
6. Cover a controlled canceled-context or canceled-loader failure through the
|
|
||||||
public facade, asserting the existing preparation category for both calls.
|
|
||||||
Do not assert a stronger universal relationship to `context.Canceled` or
|
|
||||||
introduce timing-based tests. Keep the model-call count at zero across the
|
|
||||||
validation cases.
|
|
||||||
7. Run `go test -race . ./internal/usecase ./internal/prompt` and
|
|
||||||
`git diff --check`, formatting changed Go files. Review tests for redundant
|
|
||||||
preparation coverage and assertions on incidental error text or private call
|
|
||||||
order.
|
|
||||||
|
|
||||||
Completion: success/failure parity includes concrete branch choices, declared
|
|
||||||
required inputs, variable presence, all supplied artifact bindings, representative
|
|
||||||
resource/selection errors, and ordinary request forwarding. No provider or
|
|
||||||
capacity-readiness interpretation has been introduced.
|
|
||||||
|
|
||||||
**Completed: 2026-10-12.**
|
|
||||||
|
|
||||||
## Stage 3: Document Consumer Use And Verify The Integrated Feature
|
|
||||||
|
|
||||||
Scope: canonical documentation, the maintained preparation example, and final
|
|
||||||
validation.
|
|
||||||
|
|
||||||
1. Add a small `ValidateRequest` example to
|
|
||||||
`docs/consumers/pkg-promptkit.md`, next to requirements inspection and offline
|
|
||||||
preparation. Show a full `RunRequest` and nil/error handling, and link to
|
|
||||||
GoDoc for exact behavior. Distinguish concrete-request validation from
|
|
||||||
dependency discovery and from obtaining a prepared execution handle.
|
|
||||||
2. Explain that Scriptorium can map the error to its own diagnostics and exit
|
|
||||||
status, and should build equivalent requests for its `validate` and `run`
|
|
||||||
commands. Keep this as consumer guidance, not CLI code or a new PromptKit
|
|
||||||
transport contract. Explicitly note that separate subprocess calls prepare
|
|
||||||
independently and cannot transfer the opaque handle.
|
|
||||||
3. Update `examples/go-library/prepare/main.go` to demonstrate the method using
|
|
||||||
the same request value as its existing `Prepare` demonstration. Keep its
|
|
||||||
semantic output unchanged. Explain in the consumer guide that showing both
|
|
||||||
operations is illustrative, not a requirement to validate before `Prepare`
|
|
||||||
or `Run`; it performs preparation twice.
|
|
||||||
4. Update `docs/internal/runner.md` to describe the error-only facade's reuse of
|
|
||||||
the existing preparation path, and update `docs/internal/overview.md` to
|
|
||||||
include the implemented public operation. Keep business logic under the
|
|
||||||
existing internal ownership and avoid describing a new validator component.
|
|
||||||
5. Ensure GoDoc and guidance state the same boundaries: resource I/O and cost
|
|
||||||
remain, all supplied inputs follow existing loading behavior, only the
|
|
||||||
concrete invocation is checked, errors are not aggregated, and no future
|
|
||||||
provider or execution success is certified. Do not change format references
|
|
||||||
to create new requirements.
|
|
||||||
6. `v0.14.0` is already tagged at planning time; leave that historical note
|
|
||||||
unchanged. Record this optional API addition in the next release notes when
|
|
||||||
that release is prepared. Release publication and new version selection are
|
|
||||||
separate from this implementation plan.
|
|
||||||
7. Run `go test -race ./...`, `go vet ./...`,
|
|
||||||
`python3 scripts/check-examples.py`, and `git diff --check`. Check affected
|
|
||||||
documentation links and fragments. Review exported declarations to confirm
|
|
||||||
that only `Engine.ValidateRequest` was added, and review the complete diff
|
|
||||||
against [the feature roadmap](validate.md).
|
|
||||||
|
|
||||||
Completion: API, focused tests, canonical documentation, and the maintained
|
|
||||||
example are coherent; existing preparation and execution workflows remain
|
|
||||||
available; and the integrated checks pass. Report actual check results and stage
|
|
||||||
commits. The completed roadmap can be retired during later release preparation.
|
|
||||||
|
|
||||||
**Completed: 2026-10-12.**
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None. The finalized roadmap resolves the public name, request shape, result,
|
|
||||||
preparation scope, and downstream boundary required for these stages.
|
|
||||||
@@ -1,50 +0,0 @@
|
|||||||
# Request Validation Review Findings
|
|
||||||
|
|
||||||
Reviewed commits `f315620`, `f3148bf`, and `60784e7` against the finalized
|
|
||||||
[feature roadmap](validate.md) and [implementation plan](implementation.md).
|
|
||||||
|
|
||||||
## 1. Reuse The Existing Preparation-Parity Test Helper
|
|
||||||
|
|
||||||
Priority: P3 (test maintainability). Status: complete.
|
|
||||||
|
|
||||||
At reviewed commit `60784e7`, in
|
|
||||||
[validate_request_contract_test.go](../../validate_request_contract_test.go),
|
|
||||||
`TestValidateRequestMatchesPrepareContract` defines a local `testParity` closure
|
|
||||||
at lines 25–45 that duplicates the package-level
|
|
||||||
`assertValidateRequestParity` helper at lines 87–108. Both invoke `Prepare` and
|
|
||||||
`ValidateRequest` and assert the same success/result and public-error behavior;
|
|
||||||
the package-level helper additionally returns the successful prepared value.
|
|
||||||
The nil-engine subtest repeats the same failure assertions again.
|
|
||||||
|
|
||||||
Retaining separate copies makes future parity-assertion improvements require
|
|
||||||
multiple edits and increases the risk that related tests diverge unnecessarily.
|
|
||||||
This is a small cleanup, not a demonstrated functional or release-blocking bug.
|
|
||||||
|
|
||||||
### Requested Remediation
|
|
||||||
|
|
||||||
- Replace calls to the local closure with `assertValidateRequestParity` and
|
|
||||||
remove the redundant closure.
|
|
||||||
- Reuse that helper for the nil-engine subtest as well; retain its
|
|
||||||
`ErrInvalidConfig` expectation.
|
|
||||||
- Preserve all existing cases, public-error assertions, and no-model-call
|
|
||||||
checks. No production or public-contract changes are needed.
|
|
||||||
|
|
||||||
Verify with `go test -race . -run '^TestValidateRequest'` and
|
|
||||||
`git diff --check`, formatting the changed Go file.
|
|
||||||
|
|
||||||
### Completion
|
|
||||||
|
|
||||||
Removed the local closure and reused `assertValidateRequestParity` for all four
|
|
||||||
cases, including the nil-engine `ErrInvalidConfig` expectation. Existing cases,
|
|
||||||
public-error assertions, and no-model-call checks are preserved; production code
|
|
||||||
and public contracts are unchanged.
|
|
||||||
|
|
||||||
Formatted the Go file. The uncached, race-enabled `TestValidateRequest` tests
|
|
||||||
and `git diff --check` passed.
|
|
||||||
|
|
||||||
## Review Outcome
|
|
||||||
|
|
||||||
The public method and shared facade helper match the intended architecture.
|
|
||||||
No implementation gaps or unintended strengthening of contracts were found.
|
|
||||||
The uncached full race-enabled suite, vet, maintained consumer examples, and
|
|
||||||
implementation diff whitespace checks passed during review.
|
|
||||||
@@ -1,168 +0,0 @@
|
|||||||
# Request Preflight Validation
|
|
||||||
|
|
||||||
Status: accepted; implementation pending.
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Provide an explicit success/error operation for consumers that want to check an
|
|
||||||
actual prompt request without generating a model response or receiving a
|
|
||||||
prepared snapshot.
|
|
||||||
|
|
||||||
The check applies to concrete variable values and input bindings for one
|
|
||||||
invocation. It does not certify that an application's configured bindings cover
|
|
||||||
every possible request or template branch.
|
|
||||||
|
|
||||||
Scriptorium motivates this addition: applications invoke it as a subprocess and
|
|
||||||
could use a `validate` command to fail early when their supplied variables,
|
|
||||||
inputs, or other request settings cannot pass preparation. PromptKit owns the
|
|
||||||
reusable request operation; Scriptorium owns command syntax, diagnostics,
|
|
||||||
process exit statuses, and any application-specific interpretation.
|
|
||||||
|
|
||||||
## Current Baseline
|
|
||||||
|
|
||||||
`Engine.InspectRequirements` discovers recognizable input and caller-variable
|
|
||||||
dependencies without loading artifacts or rendering messages. Its `Complete`
|
|
||||||
flag describes discovery coverage, not whether supplied dependencies satisfy a
|
|
||||||
request. It is useful for authoring and planning, but is not actual preflight.
|
|
||||||
|
|
||||||
`Engine.Prepare` already performs preflight through request conversion and the
|
|
||||||
shared internal preparation pipeline. It resolves the prompt, profile, effective
|
|
||||||
settings, and output contract; loads input artifacts and applicable schema
|
|
||||||
resources; and renders messages and the session ID without calling an LLM.
|
|
||||||
The facade then converts the internal prepared value into a caller-owned
|
|
||||||
`PreparedRun`. Consumers can discard that result today, but the API does not
|
|
||||||
directly express a success/error-only operation.
|
|
||||||
|
|
||||||
See the [consumer inspection and preparation guide](../consumers/pkg-promptkit.md#inspect-requirements-before-preparation),
|
|
||||||
[existing preparation GoDoc](../../engine.go), and
|
|
||||||
[internal preparation flow](../internal/runner.md#shared-preparation-pipeline).
|
|
||||||
|
|
||||||
## Target Public API
|
|
||||||
|
|
||||||
Add a separate method on the existing engine:
|
|
||||||
|
|
||||||
```go
|
|
||||||
func (e *Engine) ValidateRequest(ctx context.Context, req RunRequest) error
|
|
||||||
```
|
|
||||||
|
|
||||||
`ValidateRequest` names the object being checked and distinguishes this operation
|
|
||||||
from generated-output validation. `validate` remains the recommended downstream
|
|
||||||
CLI command name; preflight describes the workflow, without implying provider
|
|
||||||
or capacity readiness checks.
|
|
||||||
|
|
||||||
Use the complete existing `RunRequest`, including input bindings, caller
|
|
||||||
variables, profile selection, execution overrides, session settings, output
|
|
||||||
contract overrides, and appended messages. Apply the same request conversion
|
|
||||||
and preparation rules as `Prepare`; validation is not a name-only check or an
|
|
||||||
analysis of declared requirements.
|
|
||||||
|
|
||||||
A nil error means **the supplied request passed preparation at the time of the
|
|
||||||
check**. A failure returns the error encountered through the existing
|
|
||||||
preparation path, with the same public error categories and applicable error
|
|
||||||
identities. A nil engine follows `Prepare`'s `ErrInvalidConfig` convention.
|
|
||||||
Context handling follows preparation rather than introducing a stronger
|
|
||||||
cancellation guarantee. The result does not aggregate all missing requirements
|
|
||||||
or return a prepared value, execution handle, or model output.
|
|
||||||
|
|
||||||
Keep `Prepare` unchanged. A separate method avoids a mode flag that changes the
|
|
||||||
meaning of its returned snapshot. The intended distinction is:
|
|
||||||
|
|
||||||
- `InspectRequirements`: discover what a prompt may require for a supplied
|
|
||||||
context, including partial discovery where appropriate.
|
|
||||||
- `ValidateRequest`: determine whether the actual supplied request passes
|
|
||||||
preparation.
|
|
||||||
- `Prepare`: perform preparation and return its snapshot.
|
|
||||||
- `PrepareExecution`: prepare an owned snapshot that can later be executed.
|
|
||||||
- `Run`: prepare and execute the request.
|
|
||||||
|
|
||||||
## Meaning And Limits
|
|
||||||
|
|
||||||
Validation exercises actual preparation, including loading inputs and rendering
|
|
||||||
the branches selected by supplied values. It applies existing rules to missing
|
|
||||||
bindings and variables rather than adding stricter required-field rules,
|
|
||||||
rejecting additional caller keys, or restricting supported templates.
|
|
||||||
|
|
||||||
For example, consider a message containing
|
|
||||||
`{{if eq .mode "detailed"}}{{input "details"}}{{end}}`, where `details` is not
|
|
||||||
declared required. A request with `mode=brief` can prepare without that input;
|
|
||||||
one with `mode=detailed` needs it. Declaring the input required makes it
|
|
||||||
necessary regardless of the selected branch. Similarly, an evaluated
|
|
||||||
`{{.audience}}` fails when the key is absent, but supplying an empty string does
|
|
||||||
not itself fail preparation. Optional lookups such as `{{index . "audience"}}`
|
|
||||||
retain their existing semantics. These examples illustrate preparation parity,
|
|
||||||
not new required-field rules; see the [input and template formats](../formats.md#inputs)
|
|
||||||
and [requirement classifications](../../requirements.go).
|
|
||||||
|
|
||||||
No LLM request is made. This is not a promise of zero I/O: artifact and schema
|
|
||||||
loaders still perform their existing resource work, including caller-supplied
|
|
||||||
loader behavior. The current preparation path loads every supplied input, even
|
|
||||||
if the selected template branch does not use it. Avoiding inference makes the
|
|
||||||
operation useful for fail-fast workflows, but large inputs or resource
|
|
||||||
processing can still cost time and memory. No fixed latency or allocation
|
|
||||||
guarantee is proposed.
|
|
||||||
|
|
||||||
Success does not establish provider acceptance, context-window fit, generated
|
|
||||||
output validity, or application-domain correctness. It also does not reserve
|
|
||||||
execution capacity or retain a snapshot for a subsequent invocation. Later
|
|
||||||
requests can differ, and referenced resources can change. Consumers that need
|
|
||||||
execution tied to the checked snapshot within one process and engine should use
|
|
||||||
`PrepareExecution`. Its [opaque handle](../../prepared_execution.go) cannot
|
|
||||||
connect separate subprocess invocations.
|
|
||||||
|
|
||||||
This is request-scoped validation using an already constructed engine. It does
|
|
||||||
not expand configuration ownership beyond the final handoff described by
|
|
||||||
[ADR 0001](../adr/0001-configuration-boundary.md), or introduce eager validation
|
|
||||||
of all configured prompts and resources.
|
|
||||||
|
|
||||||
## Downstream CLI Guidance
|
|
||||||
|
|
||||||
Scriptorium's proposed `validate` and existing `run` commands should share
|
|
||||||
request construction: argument interpretation, defaults, profile selection,
|
|
||||||
and overrides must describe the same request for equivalent caller inputs.
|
|
||||||
This allows PromptKit's preparation parity to carry through the CLI boundary.
|
|
||||||
Scriptorium owns exit statuses and diagnostic presentation.
|
|
||||||
|
|
||||||
A `scriptorium validate` call followed by `scriptorium run` performs two
|
|
||||||
independent preparations; the first does not retain an executable snapshot for
|
|
||||||
the second. When assessing CLI latency, measure the complete subprocess call,
|
|
||||||
including process startup and engine construction, as well as preparation.
|
|
||||||
Avoiding the public prepared snapshot does not eliminate the other preparation
|
|
||||||
work. These are downstream integration recommendations, not Scriptorium changes
|
|
||||||
included in this unit of work.
|
|
||||||
|
|
||||||
## Work Needed
|
|
||||||
|
|
||||||
- Add the public error-only method through the existing facade and internal
|
|
||||||
preparation path. Reuse request conversion, preparation, and error mapping
|
|
||||||
rather than implementing a second validation engine or an inspection-based
|
|
||||||
approximation.
|
|
||||||
- Discard the internal prepared result without converting it into a public
|
|
||||||
snapshot. The existing internal prepared value may still be constructed;
|
|
||||||
removing all preparation allocations is not required for this addition.
|
|
||||||
Share small facade plumbing where helpful without a broad refactor.
|
|
||||||
- Preserve preparation's existing context, credential, resource, and execution
|
|
||||||
separation behavior. No model generation, output-repair loop, or capacity
|
|
||||||
admission is needed for the new operation.
|
|
||||||
- Add focused external-package tests for successful preparation and
|
|
||||||
representative missing-variable/input, resource, selection, and invalid-request
|
|
||||||
failures. Use existing fixtures where suitable to verify parity with
|
|
||||||
`Prepare`, including a branch-sensitive example and the nil-engine boundary.
|
|
||||||
Check public error identities rather than exact messages, and confirm that
|
|
||||||
the injected model client is not called. Avoid duplicating the full
|
|
||||||
preparation test suite.
|
|
||||||
- Document the method in GoDoc and add a short consumer example distinguishing
|
|
||||||
validation from requirement discovery and prepared execution. Update the
|
|
||||||
internal runner description where needed when implementation lands, and
|
|
||||||
record the optional API addition in its eventual release notes.
|
|
||||||
|
|
||||||
## Completion And Scope
|
|
||||||
|
|
||||||
The feature is complete when consumers can submit an ordinary `RunRequest` to
|
|
||||||
`Engine.ValidateRequest` and obtain preparation success or its existing failure
|
|
||||||
without a model call or public prepared snapshot, with focused tests and
|
|
||||||
canonical documentation establishing that behavior.
|
|
||||||
|
|
||||||
This unit of work does not include changes to Scriptorium, CLI commands or exit
|
|
||||||
codes, aggregated diagnostics, presence-only validation modes, provider probes,
|
|
||||||
domain-aware validation, or new execution guarantees. It introduces no changes
|
|
||||||
to prompt/profile formats or existing preparation and execution semantics.
|
|
||||||
Reference in New Issue
Block a user