eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Prepare v0.15.0 release documentation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-12 20:17:52 +00:00
parent 50bc35516a
commit 35f14d58af
5 changed files with 37 additions and 413 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
README.md
View File

@@ -37,6 +37,9 @@ boundary and constraints that framework work must preserve.
## Release Guidance
The [v0.15.0 release notes](docs/releases/v0.15.0.md) cover error-only
validation of concrete requests with `Engine.ValidateRequest`.
Consumers upgrading to `v0.12.0` should read the
[v0.12.0 release notes](docs/releases/v0.12.0.md) for the intentional ConfigKit
configuration and public construction API migration.

34
docs/releases/v0.15.0.md Normal file
View File

@@ -0,0 +1,34 @@
# Promptkit v0.15.0
This release adds error-only validation of a concrete prompt request without
calling a model. The minimum supported Go version remains 1.26.0.
## Validate A Concrete Request
`Engine.ValidateRequest` accepts an ordinary `RunRequest` and returns nil when
it passes preparation, or the existing preparation error. Consumers can use it
for fail-fast checks, including downstream CLI validation commands, without
receiving a prepared snapshot.
The method shares `Prepare`'s request conversion and preparation path. Unlike
requirements inspection, it checks the actual supplied values and bindings.
Artifact and schema resource I/O still occurs; success does not certify provider
acceptance or later execution success. A subsequent run prepares independently.
Validation is optional, not a required step before preparation or execution.
See the [request validation guide](../consumers/pkg-promptkit.md#validate-a-concrete-request)
and [GoDoc](../../engine.go) for usage and the exact contract. Consumers that
need execution tied to a checked snapshot can use the existing
[prepared execution workflow](../consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later).
## Compatibility And Upgrade
No application-code, configuration, or Go-toolchain migration is required from
v0.14.0. This is an optional API addition; existing inspection, preparation, and
execution workflows remain available.
```sh
go get gitea.maximumdirect.net/eric/promptkit@v0.15.0
```
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.15.0/COPYRIGHT) | [LICENSE](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.15.0/LICENSE) | [ATTRIBUTION](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.15.0/docs/oss/attribution.md)

195
docs/roadmap/implementation.md
View File

@@ -1,195 +0,0 @@
# Request Validation Implementation Plan
Implement [the finalized feature roadmap](validate.md) in the numbered order
below. Each stage is scoped for one Luna coding-agent prompt. The feature
roadmap owns the target behavior and downstream intent; this document owns the
implementation sequence.
## Baseline And Implementation Decisions
The baseline is commit `0c3651f`. `Engine.Prepare` in `engine.go` clears execution
observation state, checks the engine, converts `RunRequest`, calls
`internal/engine.Prepare`, maps preparation errors, and converts the returned
internal value into a caller-owned public snapshot. The internal engine already
delegates to `internal/usecase.Runner.Prepare`, which runs both preparation
phases without admission or generation.
Add exactly this public operation:
```go
func (e *Engine) ValidateRequest(ctx context.Context, req RunRequest) error
```
Extract the existing facade preparation plumbing into an unexported
`(*Engine).prepareRequest` helper returning `(*domain.PreparedRun, error)`.
`Prepare` will call that helper and retain its existing public result conversion;
`ValidateRequest` will call it and discard the internal result. This shares
request conversion, context handling, nil-engine checks, and error mapping while
avoiding public snapshot copies for validation. The private helper is an
implementation choice, not a consumer contract.
No new internal-engine or runner operation is needed. Internal prepared values,
hashing, schema preparation, and rendering may still occur; eliminating them
would change the selected preparation-parity behavior or require work outside
this small addition.
## Working Instructions
- Start with [the development guide](../development.md) and read the applicable
[agent](../policy/agents.md), [architecture](../policy/architecture.md),
[testing](../policy/testing.md), and [documentation](../policy/documentation.md)
policies. Use the existing facade GoDoc and
[runner description](../internal/runner.md#shared-preparation-pipeline) as
current-behavior references.
- Preserve `Prepare`'s behavior and error wrapping, including its existing
context classification. Do not add a general `errors.Is(err, ctx.Err())`
guarantee, new error sentinels, aggregated diagnostics, stricter binding
rules, provider checks, or capacity checks.
- Use the full `RunRequest` conversion path, not a separate subset of its fields
or a dependency inventory from `InspectRequirements`. Leave credential,
resource, template, and request-format rules unchanged.
- Keep tests offline, deterministic, and external-package at the public boundary.
Use real preparation collaborators and small injected doubles for external
effects. Test representative forwarding risks rather than reproducing the
complete preparation suite.
- Commit each completed stage after its checks and relevant diff review, with a
plain-English message. Do not push, tag, or run a publication command as part
of this plan.
## Stage 1: Share Facade Preparation And Add ValidateRequest
Scope: the small facade change, GoDoc, and basic contract tests.
1. Review `Engine.Prepare`, `toDomainRunRequest`, `mapPublicError`, and
`fromDomainPreparedRun`. Extract only the existing preparation plumbing into
the private helper described above. Keep the existing operation order and
wrapping at each error boundary; do not change error semantics as incidental
cleanup.
2. Make `Prepare` call the helper and perform the existing
`fromDomainPreparedRun` conversion on success. Its signature, snapshot
ownership, and failure behavior remain unchanged.
3. Add `ValidateRequest` in `engine.go`. Return the helper's error, discarding its
internal prepared value. Do not call the public `Prepare` method, construct a
public snapshot, or introduce an internal validation pipeline.
4. Add GoDoc explaining that nil means the concrete request passed preparation
now, no LLM call occurs, resource I/O still occurs, and no executable snapshot
is retained. State parity with preparation's errors and context handling,
including the nil-engine `ErrInvalidConfig` convention. Link conceptually to
`Prepare` and `PrepareExecution`; do not promise subsequent provider success.
5. Add `validate_request_contract_test.go` in package `promptkit_test`. Reuse
compatible helpers from `requirements_inspection_contract_test.go` and
`consumer_test_helpers_test.go` where helpful. Construct a synthetic prompt
and explicit offline profile rather than depending on catalog model choices.
Cover a successful request, a nil engine, a conversion-level invalid request
such as an invalid appended-message role, and one missing required input.
6. In those tests, invoke both `Prepare` and `ValidateRequest` for the same
test-controlled request and compare success or the relevant public sentinel
through `errors.Is`. Use one intentional failure per case rather than
expecting a particular error from several independently failing inputs.
Inject a counting client and verify neither operation generates a model call.
7. Format changed Go files and run
`go test . ./internal/usecase ./internal/prompt`, followed by
`git diff --check`. Existing `Prepare` tests must still pass.
Completion: external consumers can call the new error-only method, all facade
preparation plumbing is shared, and the public snapshot conversion remains only
on `Prepare`'s success path.
**Completed: 2026-10-12.**
## Stage 2: Verify Concrete-Request Parity And Resource Boundaries
Scope: focused tests of the finalized roadmap's clarifications.
1. Extend the root contract tests with a prompt using
`{{if eq .mode "detailed"}}{{input "details"}}{{end}}`. For an input not
declared required, verify `mode=brief` succeeds without it and
`mode=detailed` fails without it but succeeds with an inline binding. Use a
second definition with the input declared required to verify absence fails
even on the brief branch. Compare each case with `Prepare`.
2. Cover an evaluated direct variable lookup with an absent key and with an
explicitly supplied empty string. Include a missing `index` lookup to show
its existing optional semantics are retained. These assertions concern
preparation only, not nonempty model-output rules. An extra unrelated caller
variable must remain acceptable where `Prepare` accepts it.
3. Add a case with an unreadable supplied input that is not referenced on the
selected branch. A synthetic missing file under `t.TempDir()` is sufficient.
Both operations must report the applicable artifact-load error, proving
validation uses actual preparation instead of a names-only or lazy subset.
4. Cover representative selected-profile and output-schema preparation failures
with existing valid fixtures: a missing profile should retain
`ErrProfileNotFound`, and an unavailable selected schema should retain the
category already returned by `Prepare`. Avoid adding source-level or
intermediate-configuration validation.
5. Add one successful request with a profile default and a runtime override that
completes a partial profile, plus a valid direct session ID and appended
message where useful. Inspect `Prepare`'s result only as needed to establish
that the fixture exercised those settings; `ValidateRequest` must accept the
same request. Existing tests own the exhaustive override and message matrix.
6. Cover a controlled canceled-context or canceled-loader failure through the
public facade, asserting the existing preparation category for both calls.
Do not assert a stronger universal relationship to `context.Canceled` or
introduce timing-based tests. Keep the model-call count at zero across the
validation cases.
7. Run `go test -race . ./internal/usecase ./internal/prompt` and
`git diff --check`, formatting changed Go files. Review tests for redundant
preparation coverage and assertions on incidental error text or private call
order.
Completion: success/failure parity includes concrete branch choices, declared
required inputs, variable presence, all supplied artifact bindings, representative
resource/selection errors, and ordinary request forwarding. No provider or
capacity-readiness interpretation has been introduced.
**Completed: 2026-10-12.**
## Stage 3: Document Consumer Use And Verify The Integrated Feature
Scope: canonical documentation, the maintained preparation example, and final
validation.
1. Add a small `ValidateRequest` example to
`docs/consumers/pkg-promptkit.md`, next to requirements inspection and offline
preparation. Show a full `RunRequest` and nil/error handling, and link to
GoDoc for exact behavior. Distinguish concrete-request validation from
dependency discovery and from obtaining a prepared execution handle.
2. Explain that Scriptorium can map the error to its own diagnostics and exit
status, and should build equivalent requests for its `validate` and `run`
commands. Keep this as consumer guidance, not CLI code or a new PromptKit
transport contract. Explicitly note that separate subprocess calls prepare
independently and cannot transfer the opaque handle.
3. Update `examples/go-library/prepare/main.go` to demonstrate the method using
the same request value as its existing `Prepare` demonstration. Keep its
semantic output unchanged. Explain in the consumer guide that showing both
operations is illustrative, not a requirement to validate before `Prepare`
or `Run`; it performs preparation twice.
4. Update `docs/internal/runner.md` to describe the error-only facade's reuse of
the existing preparation path, and update `docs/internal/overview.md` to
include the implemented public operation. Keep business logic under the
existing internal ownership and avoid describing a new validator component.
5. Ensure GoDoc and guidance state the same boundaries: resource I/O and cost
remain, all supplied inputs follow existing loading behavior, only the
concrete invocation is checked, errors are not aggregated, and no future
provider or execution success is certified. Do not change format references
to create new requirements.
6. `v0.14.0` is already tagged at planning time; leave that historical note
unchanged. Record this optional API addition in the next release notes when
that release is prepared. Release publication and new version selection are
separate from this implementation plan.
7. Run `go test -race ./...`, `go vet ./...`,
`python3 scripts/check-examples.py`, and `git diff --check`. Check affected
documentation links and fragments. Review exported declarations to confirm
that only `Engine.ValidateRequest` was added, and review the complete diff
against [the feature roadmap](validate.md).
Completion: API, focused tests, canonical documentation, and the maintained
example are coherent; existing preparation and execution workflows remain
available; and the integrated checks pass. Report actual check results and stage
commits. The completed roadmap can be retired during later release preparation.
**Completed: 2026-10-12.**
## Open Questions
None. The finalized roadmap resolves the public name, request shape, result,
preparation scope, and downstream boundary required for these stages.

50
docs/roadmap/remediation.md
View File

@@ -1,50 +0,0 @@
# Request Validation Review Findings
Reviewed commits `f315620`, `f3148bf`, and `60784e7` against the finalized
[feature roadmap](validate.md) and [implementation plan](implementation.md).
## 1. Reuse The Existing Preparation-Parity Test Helper
Priority: P3 (test maintainability). Status: complete.
At reviewed commit `60784e7`, in
[validate_request_contract_test.go](../../validate_request_contract_test.go),
`TestValidateRequestMatchesPrepareContract` defines a local `testParity` closure
at lines 25–45 that duplicates the package-level
`assertValidateRequestParity` helper at lines 87–108. Both invoke `Prepare` and
`ValidateRequest` and assert the same success/result and public-error behavior;
the package-level helper additionally returns the successful prepared value.
The nil-engine subtest repeats the same failure assertions again.
Retaining separate copies makes future parity-assertion improvements require
multiple edits and increases the risk that related tests diverge unnecessarily.
This is a small cleanup, not a demonstrated functional or release-blocking bug.
### Requested Remediation
- Replace calls to the local closure with `assertValidateRequestParity` and
remove the redundant closure.
- Reuse that helper for the nil-engine subtest as well; retain its
`ErrInvalidConfig` expectation.
- Preserve all existing cases, public-error assertions, and no-model-call
checks. No production or public-contract changes are needed.
Verify with `go test -race . -run '^TestValidateRequest'` and
`git diff --check`, formatting the changed Go file.
### Completion
Removed the local closure and reused `assertValidateRequestParity` for all four
cases, including the nil-engine `ErrInvalidConfig` expectation. Existing cases,
public-error assertions, and no-model-call checks are preserved; production code
and public contracts are unchanged.
Formatted the Go file. The uncached, race-enabled `TestValidateRequest` tests
and `git diff --check` passed.
## Review Outcome
The public method and shared facade helper match the intended architecture.
No implementation gaps or unintended strengthening of contracts were found.
The uncached full race-enabled suite, vet, maintained consumer examples, and
implementation diff whitespace checks passed during review.

168
docs/roadmap/validate.md
View File

@@ -1,168 +0,0 @@
# Request Preflight Validation
Status: accepted; implementation pending.
## Purpose
Provide an explicit success/error operation for consumers that want to check an
actual prompt request without generating a model response or receiving a
prepared snapshot.
The check applies to concrete variable values and input bindings for one
invocation. It does not certify that an application's configured bindings cover
every possible request or template branch.
Scriptorium motivates this addition: applications invoke it as a subprocess and
could use a `validate` command to fail early when their supplied variables,
inputs, or other request settings cannot pass preparation. PromptKit owns the
reusable request operation; Scriptorium owns command syntax, diagnostics,
process exit statuses, and any application-specific interpretation.
## Current Baseline
`Engine.InspectRequirements` discovers recognizable input and caller-variable
dependencies without loading artifacts or rendering messages. Its `Complete`
flag describes discovery coverage, not whether supplied dependencies satisfy a
request. It is useful for authoring and planning, but is not actual preflight.
`Engine.Prepare` already performs preflight through request conversion and the
shared internal preparation pipeline. It resolves the prompt, profile, effective
settings, and output contract; loads input artifacts and applicable schema
resources; and renders messages and the session ID without calling an LLM.
The facade then converts the internal prepared value into a caller-owned
`PreparedRun`. Consumers can discard that result today, but the API does not
directly express a success/error-only operation.
See the [consumer inspection and preparation guide](../consumers/pkg-promptkit.md#inspect-requirements-before-preparation),
[existing preparation GoDoc](../../engine.go), and
[internal preparation flow](../internal/runner.md#shared-preparation-pipeline).
## Target Public API
Add a separate method on the existing engine:
```go
func (e *Engine) ValidateRequest(ctx context.Context, req RunRequest) error
```
`ValidateRequest` names the object being checked and distinguishes this operation
from generated-output validation. `validate` remains the recommended downstream
CLI command name; preflight describes the workflow, without implying provider
or capacity readiness checks.
Use the complete existing `RunRequest`, including input bindings, caller
variables, profile selection, execution overrides, session settings, output
contract overrides, and appended messages. Apply the same request conversion
and preparation rules as `Prepare`; validation is not a name-only check or an
analysis of declared requirements.
A nil error means **the supplied request passed preparation at the time of the
check**. A failure returns the error encountered through the existing
preparation path, with the same public error categories and applicable error
identities. A nil engine follows `Prepare`'s `ErrInvalidConfig` convention.
Context handling follows preparation rather than introducing a stronger
cancellation guarantee. The result does not aggregate all missing requirements
or return a prepared value, execution handle, or model output.
Keep `Prepare` unchanged. A separate method avoids a mode flag that changes the
meaning of its returned snapshot. The intended distinction is:
- `InspectRequirements`: discover what a prompt may require for a supplied
context, including partial discovery where appropriate.
- `ValidateRequest`: determine whether the actual supplied request passes
preparation.
- `Prepare`: perform preparation and return its snapshot.
- `PrepareExecution`: prepare an owned snapshot that can later be executed.
- `Run`: prepare and execute the request.
## Meaning And Limits
Validation exercises actual preparation, including loading inputs and rendering
the branches selected by supplied values. It applies existing rules to missing
bindings and variables rather than adding stricter required-field rules,
rejecting additional caller keys, or restricting supported templates.
For example, consider a message containing
`{{if eq .mode "detailed"}}{{input "details"}}{{end}}`, where `details` is not
declared required. A request with `mode=brief` can prepare without that input;
one with `mode=detailed` needs it. Declaring the input required makes it
necessary regardless of the selected branch. Similarly, an evaluated
`{{.audience}}` fails when the key is absent, but supplying an empty string does
not itself fail preparation. Optional lookups such as `{{index . "audience"}}`
retain their existing semantics. These examples illustrate preparation parity,
not new required-field rules; see the [input and template formats](../formats.md#inputs)
and [requirement classifications](../../requirements.go).
No LLM request is made. This is not a promise of zero I/O: artifact and schema
loaders still perform their existing resource work, including caller-supplied
loader behavior. The current preparation path loads every supplied input, even
if the selected template branch does not use it. Avoiding inference makes the
operation useful for fail-fast workflows, but large inputs or resource
processing can still cost time and memory. No fixed latency or allocation
guarantee is proposed.
Success does not establish provider acceptance, context-window fit, generated
output validity, or application-domain correctness. It also does not reserve
execution capacity or retain a snapshot for a subsequent invocation. Later
requests can differ, and referenced resources can change. Consumers that need
execution tied to the checked snapshot within one process and engine should use
`PrepareExecution`. Its [opaque handle](../../prepared_execution.go) cannot
connect separate subprocess invocations.
This is request-scoped validation using an already constructed engine. It does
not expand configuration ownership beyond the final handoff described by
[ADR 0001](../adr/0001-configuration-boundary.md), or introduce eager validation
of all configured prompts and resources.
## Downstream CLI Guidance
Scriptorium's proposed `validate` and existing `run` commands should share
request construction: argument interpretation, defaults, profile selection,
and overrides must describe the same request for equivalent caller inputs.
This allows PromptKit's preparation parity to carry through the CLI boundary.
Scriptorium owns exit statuses and diagnostic presentation.
A `scriptorium validate` call followed by `scriptorium run` performs two
independent preparations; the first does not retain an executable snapshot for
the second. When assessing CLI latency, measure the complete subprocess call,
including process startup and engine construction, as well as preparation.
Avoiding the public prepared snapshot does not eliminate the other preparation
work. These are downstream integration recommendations, not Scriptorium changes
included in this unit of work.
## Work Needed
- Add the public error-only method through the existing facade and internal
preparation path. Reuse request conversion, preparation, and error mapping
rather than implementing a second validation engine or an inspection-based
approximation.
- Discard the internal prepared result without converting it into a public
snapshot. The existing internal prepared value may still be constructed;
removing all preparation allocations is not required for this addition.
Share small facade plumbing where helpful without a broad refactor.
- Preserve preparation's existing context, credential, resource, and execution
separation behavior. No model generation, output-repair loop, or capacity
admission is needed for the new operation.
- Add focused external-package tests for successful preparation and
representative missing-variable/input, resource, selection, and invalid-request
failures. Use existing fixtures where suitable to verify parity with
`Prepare`, including a branch-sensitive example and the nil-engine boundary.
Check public error identities rather than exact messages, and confirm that
the injected model client is not called. Avoid duplicating the full
preparation test suite.
- Document the method in GoDoc and add a short consumer example distinguishing
validation from requirement discovery and prepared execution. Update the
internal runner description where needed when implementation lands, and
record the optional API addition in its eventual release notes.
## Completion And Scope
The feature is complete when consumers can submit an ordinary `RunRequest` to
`Engine.ValidateRequest` and obtain preparation success or its existing failure
without a model call or public prepared snapshot, with focused tests and
canonical documentation establishing that behavior.
This unit of work does not include changes to Scriptorium, CLI commands or exit
codes, aggregated diagnostics, presence-only validation modes, provider probes,
domain-aware validation, or new execution guarantees. It introduces no changes
to prompt/profile formats or existing preparation and execution semantics.