Prepare documentation for Promptkit v0.4.0

This commit is contained in:
2026-07-30 23:48:25 +00:00
parent e40c4f182b
commit bd6cffc9d0
11 changed files with 240 additions and 2130 deletions

View File

@@ -7,10 +7,9 @@ additions to PromptKit from the perspective of the maintainers of
Weatherreporter, a downstream application planning to replace its Scriptorium
CLI integration with PromptKit.
PromptKit v0.3.0 provides the capabilities Weatherreporter needs for the
migration. None of the ideas below is a hard adoption requirement. They are
opportunities to avoid duplicate preparation, validate configuration earlier,
improve durable failure diagnostics, and make the integration more direct.
PromptKit now provides the capabilities Weatherreporter needs for the
migration. The remaining deferred ideas are optional opportunities to validate
configuration earlier and improve durable failure diagnostics.
The examples are API sketches intended to communicate the desired capability,
not prescriptive names or finalized Go contracts. The related
@@ -19,8 +18,9 @@ overlapping features from another downstream consumer's perspective.
## Priority 1: Executable Preparation Handles
**Disposition:** Accepted into the
[executable preparation handles](prepared-execution.md) feature roadmap.
**Disposition:** Implemented as [`Engine.PrepareExecution` and
`Engine.RunPrepared`](../../engine.go). See the
[consumer guidance](../consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later).
### Downstream need
@@ -35,74 +35,20 @@ needs to:
Persisting preflight before generation leaves useful evidence when a provider
call fails or the process is interrupted during generation.
### Current integration option
### Implemented behavior
With PromptKit v0.3.0, Weatherreporter can call `Engine.Prepare`, save selected
fields from the returned `PreparedRun`, and then call `Engine.Run` with the
same request. Because `Run` performs preparation internally, the work is
repeated.
Weatherreporter plans to use embedded prompt and schema files plus immutable
inline input bytes, which removes most of the consistency risk. An external
profile file or directory can still change between the two calls, and the
second preparation remains unnecessary work.
The atomic `RunDetailed` operation proposed by the
[Notarius wishlist](notarius-promptkit-wishlist.md#priority-1-atomic-execution-with-prepared-details)
would guarantee that returned preparation details describe the completed
execution. However, returning those details only after generation would not
preserve Weatherreporter's preflight-before-generation persistence boundary.
### Requested capability
Add an opt-in two-phase API that returns a prepared execution handle:
```go
prepared, err := engine.PrepareExecution(ctx, request)
if err != nil {
// Handle preparation failure.
}
details := prepared.Details()
// Persist a consumer-selected safe preparation record.
result, err := engine.RunPrepared(ctx, prepared)
```
The exact names and shapes are flexible. The important contract is that
`RunPrepared` executes the already prepared prompt and does not reload or
rerender its prompt, profile, schema, or input sources.
`Details` should return the same caller-owned public preparation information
currently represented by `PreparedRun`. The execution handle may retain opaque
engine-owned state needed to invoke the model and validate the response.
### Design considerations
- Keep `Prepare` and `Run` available for consumers that do not need a
two-phase execution boundary.
- Bind a prepared handle to the engine that constructed it.
- Define whether a handle is one-shot, reusable, or safe for concurrent use.
A one-shot contract may be the safest initial design.
- Do not give the opaque handle a stable JSON representation.
- Do not expose or serialize resolved credential values through `Details`.
- Define how a direct request API key is retained and released when an opaque
handle must carry it until execution.
- Preserve caller-owned copies for all public details.
- Make context cancellation and backend admission timing explicit.
- Document whether profile credential environment values are resolved during
preparation or execution.
- Ensure an execution error does not invalidate the public details already
returned to the consumer.
- Consider whether an atomic `RunDetailed` can share the same internal
prepared-execution implementation.
Weatherreporter can prepare one frozen execution snapshot, persist a
caller-owned and credential-redacted `Details` value, and execute that same
snapshot through `RunPrepared`. The opaque handle is engine-bound and
single-use; an unused handle can be released with `Discard`. The consumer
guide and exported GoDoc own the exact lifecycle, credential, cancellation,
and capacity contracts.
### Value to Weatherreporter
This is the highest-value upstream addition. It would preserve
Weatherreporter's durable preflight behavior, remove duplicate work, eliminate
the remaining source-consistency window, and ensure that persisted provenance
describes the actual execution.
This preserves Weatherreporter's durable preflight behavior, removes duplicate
work, eliminates the source-consistency window, and ensures that persisted
provenance describes the actual execution.
## Priority 2: Prompt-Definition Inspection
@@ -312,12 +258,13 @@ migration priority.
## Capabilities PromptKit Already Provides Well
PromptKit v0.3.0 already provides the essential Weatherreporter integration
surface:
PromptKit already provides the essential Weatherreporter integration surface:
- importable in-process engine construction;
- filesystem, `fs.FS`, and programmatic prompt, profile, and schema sources;
- offline preparation without model execution;
- prepared execution handles for a durable preflight boundary;
- exact prompt and profile inspection;
- versioned prompt selection;
- text, Markdown, JSON, and JSON Schema output contracts;
- single-pass output validation with raw output retained after completed
@@ -329,7 +276,8 @@ surface:
- injected model-client and artifact-reader interfaces;
- caller cancellation, generation timeout, and transport timeout behavior; and
- public error sentinels for configuration, prompt, profile, artifact,
validation, capacity, and generation failures.
validation, capacity, and generation failures; and
- structured backend identity for capacity rejection.
These capabilities are sufficient for Weatherreporter to adopt PromptKit
without waiting for new upstream work.
@@ -354,41 +302,30 @@ PromptKit:
## Suggested Upstream Sequence
If the PromptKit team chooses to pursue these ideas, the most useful order for
Weatherreporter would be:
For downstream adoption and any remaining upstream work, the useful order is:
1. Add executable preparation handles, ideally sharing implementation with an
atomic detailed-run API.
1. Adopt the implemented executable preparation handles.
2. Consider eager source validation after evaluating whether the two exact
inspection APIs are sufficient.
3. Add structured generation errors.
4. Use structured capacity errors and consider semantic execution-target
fingerprints as lower-priority operational improvements.
The first item removes the only material integration workaround. Prompt and
profile inspection improve fail-fast validation. The remaining items improve
ergonomics and diagnostics.
The first item removes the material integration workaround. Prompt and profile
inspection improve fail-fast validation. The remaining items are optional
ergonomic and diagnostic improvements.
## Adoption Sequencing
Weatherreporter should not wait for the complete wishlist. PromptKit v0.3.0 is
already sufficient when Weatherreporter:
Weatherreporter should not wait for the deferred wishlist items. The current
PromptKit interface is sufficient when Weatherreporter:
- embeds immutable prompt and schema assets;
- supplies immutable inline data-package bytes;
- constructs one engine per CLI invocation;
- calls `Prepare` and `Run` with the same request; and
- prepares an execution, persists selected `Details`, and calls
`RunPrepared`; and
- keeps PromptKit behind a weatherreporter-owned adapter contract.
If executable preparation handles are scheduled for a near-term PromptKit
release, Weatherreporter may defer only its final adapter implementation to
avoid implementing and then removing duplicate preparation. Prompt corpus
retrieval, application-contract design, configuration work, embedded assets,
state contracts, and offline fixtures can proceed independently.
If the feature is not scheduled, Weatherreporter can adopt v0.3.0 and keep the
duplicate `Prepare` and `Run` sequence inside its adapter. A later PromptKit
upgrade would remain localized behind that neutral boundary.
Prompt inspection, profile inspection, source validation, structured errors,
capacity details, and semantic fingerprints should not gate adoption.