Files
promptkit/docs/roadmap/notarius-request.md

144 lines
7.5 KiB
Markdown

# Notarius Request: Message-Content Hashing
**Status:** Requested by the Notarius consumer; proposed capability, not an
implemented API or an approved PromptKit implementation plan.
**Reviewed baseline:** PromptKit v0.23.3.
## Requested Capability
Expose a deterministic hash of a prepared request's ordered message roles and
exact text, available before model execution. `MessageContentHash` is the
suggested name. This would let consumers reuse an accepted answer to the same
message content without implementing their own message-hashing policy.
A field on `PreparedRun`, available through ordinary preparation and
`PreparedExecution.Details()`, appears sufficient. The exact public API is for
PromptKit to select; Notarius needs the value at the preparation boundary, not
only after execution or through an execution observer.
## Consumer Need
Notarius runs pipelines whose stages depend on artifacts from earlier stages.
If a later stage fails, an operator should be able to fix the problem and rerun
without paying for successful earlier model work again. Notarius is replacing
its checkpoint machinery with a small accepted-artifact cache.
The cached value is **an accepted answer to the original task**. Notarius will
prepare the initial request, obtain its message-content hash, and look for
`cache/<hash>.json`. A hit loads the accepted domain artifact directly. A miss
executes the full application operation, including domain interpretation,
validation, and any application-directed correction passes. Only the final
accepted artifact is stored, under the original request's hash.
This is intentionally a convenient reuse policy, not proof that every execution
setting, validation policy, or application implementation is identical. A change
to model settings alone should not invalidate an otherwise reusable answer.
Operators can bypass or refresh the application cache when they want fresh work.
## Hash Semantics
Hash the complete effective message sequence exposed by native preparation:
rendered definition messages, appended messages, and captured conversation
history when present. Include each message's role and exact content, preserving
message order and boundaries.
| Included | Excluded |
| --- | --- |
| Ordered message roles and exact text | Session, run, and parent-run identity; timestamps and timings |
| Message boundaries and sequence length | Provider cache-control hints and other non-content message metadata |
| Any input, instruction, or schema text actually rendered into messages | Model, backend, profile, credentials, generation settings, timeouts, budgets, and retry/repair settings |
| | Prompt ID, definition hash, input hashes/paths, and schema/output-contract metadata outside message text |
No whitespace trimming, Unicode normalization, or semantic equivalence is
requested: hash the exact prepared text. Different roles, ordering, boundaries,
or text should produce different hashes, subject to ordinary hash collision
properties. Identical role/text sequences should produce the same value across
requests, engines, and process invocations regardless of excluded metadata.
For example, changing a prompt ID while retaining identical messages should
preserve the hash. Changing settings that also changes rendered text should
change it because the text changed.
Return an opaque, filename-safe value suitable for a persisted cache key.
PromptKit should own and document the hashing semantics and compatibility
behavior. A deterministic, unambiguous encoding and a standard hash such as
SHA-256 would suffice; the precise encoding is an upstream implementation choice.
No general fingerprint configuration or selectable hashing-policy framework is
needed for this request.
## Existing APIs And The Gap
[`PreparedRun`](../../types.go) already exposes messages and
`RenderedPromptHash`. The latter includes session identity and message cache
hints, as described in the [runner documentation](../internal/runner.md).
Hashing the entire prepared value would additionally capture settings and
volatile preparation metadata. Neither supplies the requested content-only key.
Keep the existing hash's semantics intact and add the narrower capability.
Notarius could manually project and hash message fields, but that would preserve
the duplicated library mechanics this migration aims to remove.
[Native split preparation and execution](../consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later)
already provides the needed execution boundary. Preparation makes no model call
and reserves no backend capacity. On a miss, `RunPrepared` can execute the retained
snapshot. On a hit, the application can discard the handle. No new execution
protocol is requested.
## Intended Application Flow
The following is conceptual pseudocode, not an existing hash API:
```text
prepared = engine.PrepareExecution(initialRequest)
initialKey = prepared.Details().MessageContentHash // proposed addition
if cache contains a usable accepted artifact for initialKey:
discard prepared handle
return cached artifact
result = engine.RunPrepared(prepared)
artifact = interpret, validate, and correct through the Notarius operation
if the operation produces a final accepted artifact:
save artifact to cache under initialKey
return application outcome
```
Notarius retains the initial key even when acceptance requires several correction
turns. It stores the final domain artifact, not a raw intermediate response,
`RunResult`, exchange, or prepared handle. Failed, rejected, and
validation-incomplete work is not saved. A hit requires ordinary decoding and
usability checks; it does not replay model work or claim fresh domain validation.
## Ownership And Scope
PromptKit owns everything within a native request execution, including structural
response validation, transport retries, and internal repair attempts. Notarius
consumes the public response or terminal failure and then owns domain acceptance.
Those internal attempts are not inputs to the cache's eligibility policy.
This request is solely for reusable message-content hashing. It does not request
PromptKit-owned cache storage, a cache-hit execution result, cached-response
injection, validation replay, exchange restoration, or a write-through response
cache. Debug observations do not change this boundary. Notarius owns cache
lookup, final acceptance, persistence, and reuse/bypass/refresh controls.
## Suggested Verification
- Repeated preparation in separate engines produces the same hash for identical
role/text sequences despite different preparation timing or session identity.
- Changing a role, text (including whitespace), order, message count, or boundary
changes the hash; ambiguous concatenations do not collapse to the same input.
- Settings-only, prompt-ID-only, and cache-hint-only changes preserve the hash
when the effective role/text sequence is unchanged.
- Rendered inputs, appended messages, and captured history participate in the
hash when they contribute to the effective messages.
- The hash is available before model execution, and reading it does not consume
the prepared handle or alter its retained snapshot.
- Existing `RenderedPromptHash` behavior remains intact. Document the new value
in public GoDoc and add a short consumer example of lookup before execution.
Notarius will verify its own cache behavior separately: hits make no model calls,
only accepted artifacts are saved, and corrected answers remain keyed to the
initial task. The supplying PromptKit release will be selected during Notarius
roadmap incorporation; this request does not assign a release number.