Prepare documentation for Promptkit v0.4.0

This commit is contained in:
2026-07-30 23:48:25 +00:00
parent e40c4f182b
commit bd6cffc9d0
11 changed files with 240 additions and 2130 deletions

View File

@@ -6,21 +6,19 @@ This document records features and interface changes that would be useful
additions to PromptKit from the perspective of the maintainers of Notarius, a
downstream application that consumes PromptKit.
PromptKit v0.3.0 provides the capabilities Notarius currently needs. None of
the ideas below blocks current Notarius development. They are opportunities to
reduce downstream workarounds, improve integration correctness, and make
PromptKit more ergonomic for applications with configuration validation,
debugging, checkpointing, and operational-observability requirements.
PromptKit now provides the capabilities Notarius currently needs. The
remaining deferred ideas are optional opportunities to improve checkpointing
and operational observability.
The examples are API sketches intended to communicate the desired capability,
not prescriptive names or finalized Go contracts.
## Priority 1: Atomic Execution With Prepared Details
**Disposition:** Covered by the accepted
[executable preparation handles](prepared-execution.md) roadmap. The shared
two-phase capability should provide the required single-preparation
consistency; a separate `RunDetailed` method is not cataloged initially.
**Disposition:** Implemented through [`Engine.PrepareExecution` and
`Engine.RunPrepared`](../../engine.go). See the
[consumer guidance](../consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later).
A separate `RunDetailed` method is not cataloged.
### Downstream need
@@ -33,58 +31,18 @@ Notarius needs both:
Notarius uses the prepared details to construct redaction-aware debug bundles
and retain enough information to diagnose model behavior.
### Current integration
### Implemented behavior
Notarius currently calls `Engine.Prepare` and then `Engine.Run` with the same
request. Because `Run` performs preparation internally, a successful request
resolves and prepares the same work twice.
This duplicates profile resolution, input hashing, schema loading, and prompt
rendering. It also creates a theoretical consistency window in which a
filesystem-backed prompt, profile, schema, or input could change between the
explicit preparation and the preparation performed by `Run`.
### Requested capability
Add an opt-in execution method that prepares exactly once and returns both the
prepared details and completed result:
```go
type RunReport struct {
Prepared PreparedRun
Result RunResult
}
func (e *Engine) RunDetailed(
ctx context.Context,
req RunRequest,
) (*RunReport, error)
```
The exact names are flexible. The important contract is that preparation
occurs once and that the returned prepared state describes the execution that
produced the returned result.
Existing `Prepare` and `Run` behavior should remain available for consumers
that need only one side of the operation.
### Design considerations
- Keep this API additive and preserve the existing simple `Run` workflow.
- Return caller-owned copies under PromptKit's existing ownership rules.
- Define whether any prepared details are available after an operational
generation or validation error. Notarius does not require partial results
for the initial use case, but an explicit contract would be valuable.
- Preserve cancellation and backend-admission semantics.
- Do not add all prepared content directly to `RunResult`. Rendered prompt
content can be large and sensitive, and consumers should opt in to receiving
it.
Notarius can prepare one frozen execution snapshot, retain a caller-owned and
credential-redacted `Details` value for its debug bundle, and execute the same
snapshot through `RunPrepared`. The opaque handle is engine-bound and
single-use; an unused handle can be released with `Discard`. The consumer
guide and exported GoDoc own the exact lifecycle and failure contracts.
### Value to Notarius
This is the highest-value wishlist item. It would remove duplicate work from
every successful PromptKit-backed call and ensure that retained debug material
corresponds atomically to the actual execution.
This removes duplicate work from PromptKit-backed calls and ensures that
retained debug material corresponds atomically to the actual execution.
## Priority 2: Prompt-Independent Profile Inspection
@@ -275,14 +233,13 @@ application-policy decision, not an upstream PromptKit gap.
## Suggested Upstream Sequence
If the PromptKit team chooses to pursue these ideas, the most useful order for
Notarius would be:
For downstream adoption and any remaining upstream work, the useful order is:
1. Add atomic execution that returns prepared details and the completed result.
1. Adopt prepared execution for atomic details and results.
2. Add a semantic execution-target digest, preferably alongside profile
inspection.
3. Use the implemented typed capacity error where backend admission diagnostics
are needed.
The first addresses a concrete execution workaround. The second would improve
The first removes the concrete execution workaround. The second would improve
checkpoint correctness and reduce coupling. The third is operational polish.