Plan prepared message-content hashing
This commit is contained in:
143
docs/roadmap/notarius-request.md
Normal file
143
docs/roadmap/notarius-request.md
Normal file
@@ -0,0 +1,143 @@
|
||||
# Notarius Request: Message-Content Hashing
|
||||
|
||||
**Status:** Requested by the Notarius consumer; proposed capability, not an
|
||||
implemented API or an approved PromptKit implementation plan.
|
||||
|
||||
**Reviewed baseline:** PromptKit v0.23.3.
|
||||
|
||||
## Requested Capability
|
||||
|
||||
Expose a deterministic hash of a prepared request's ordered message roles and
|
||||
exact text, available before model execution. `MessageContentHash` is the
|
||||
suggested name. This would let consumers reuse an accepted answer to the same
|
||||
message content without implementing their own message-hashing policy.
|
||||
|
||||
A field on `PreparedRun`, available through ordinary preparation and
|
||||
`PreparedExecution.Details()`, appears sufficient. The exact public API is for
|
||||
PromptKit to select; Notarius needs the value at the preparation boundary, not
|
||||
only after execution or through an execution observer.
|
||||
|
||||
## Consumer Need
|
||||
|
||||
Notarius runs pipelines whose stages depend on artifacts from earlier stages.
|
||||
If a later stage fails, an operator should be able to fix the problem and rerun
|
||||
without paying for successful earlier model work again. Notarius is replacing
|
||||
its checkpoint machinery with a small accepted-artifact cache.
|
||||
|
||||
The cached value is **an accepted answer to the original task**. Notarius will
|
||||
prepare the initial request, obtain its message-content hash, and look for
|
||||
`cache/<hash>.json`. A hit loads the accepted domain artifact directly. A miss
|
||||
executes the full application operation, including domain interpretation,
|
||||
validation, and any application-directed correction passes. Only the final
|
||||
accepted artifact is stored, under the original request's hash.
|
||||
|
||||
This is intentionally a convenient reuse policy, not proof that every execution
|
||||
setting, validation policy, or application implementation is identical. A change
|
||||
to model settings alone should not invalidate an otherwise reusable answer.
|
||||
Operators can bypass or refresh the application cache when they want fresh work.
|
||||
|
||||
## Hash Semantics
|
||||
|
||||
Hash the complete effective message sequence exposed by native preparation:
|
||||
rendered definition messages, appended messages, and captured conversation
|
||||
history when present. Include each message's role and exact content, preserving
|
||||
message order and boundaries.
|
||||
|
||||
| Included | Excluded |
|
||||
| --- | --- |
|
||||
| Ordered message roles and exact text | Session, run, and parent-run identity; timestamps and timings |
|
||||
| Message boundaries and sequence length | Provider cache-control hints and other non-content message metadata |
|
||||
| Any input, instruction, or schema text actually rendered into messages | Model, backend, profile, credentials, generation settings, timeouts, budgets, and retry/repair settings |
|
||||
| | Prompt ID, definition hash, input hashes/paths, and schema/output-contract metadata outside message text |
|
||||
|
||||
No whitespace trimming, Unicode normalization, or semantic equivalence is
|
||||
requested: hash the exact prepared text. Different roles, ordering, boundaries,
|
||||
or text should produce different hashes, subject to ordinary hash collision
|
||||
properties. Identical role/text sequences should produce the same value across
|
||||
requests, engines, and process invocations regardless of excluded metadata.
|
||||
For example, changing a prompt ID while retaining identical messages should
|
||||
preserve the hash. Changing settings that also changes rendered text should
|
||||
change it because the text changed.
|
||||
|
||||
Return an opaque, filename-safe value suitable for a persisted cache key.
|
||||
PromptKit should own and document the hashing semantics and compatibility
|
||||
behavior. A deterministic, unambiguous encoding and a standard hash such as
|
||||
SHA-256 would suffice; the precise encoding is an upstream implementation choice.
|
||||
No general fingerprint configuration or selectable hashing-policy framework is
|
||||
needed for this request.
|
||||
|
||||
## Existing APIs And The Gap
|
||||
|
||||
[`PreparedRun`](../../types.go) already exposes messages and
|
||||
`RenderedPromptHash`. The latter includes session identity and message cache
|
||||
hints, as described in the [runner documentation](../internal/runner.md).
|
||||
Hashing the entire prepared value would additionally capture settings and
|
||||
volatile preparation metadata. Neither supplies the requested content-only key.
|
||||
|
||||
Keep the existing hash's semantics intact and add the narrower capability.
|
||||
Notarius could manually project and hash message fields, but that would preserve
|
||||
the duplicated library mechanics this migration aims to remove.
|
||||
|
||||
[Native split preparation and execution](../consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later)
|
||||
already provides the needed execution boundary. Preparation makes no model call
|
||||
and reserves no backend capacity. On a miss, `RunPrepared` can execute the retained
|
||||
snapshot. On a hit, the application can discard the handle. No new execution
|
||||
protocol is requested.
|
||||
|
||||
## Intended Application Flow
|
||||
|
||||
The following is conceptual pseudocode, not an existing hash API:
|
||||
|
||||
```text
|
||||
prepared = engine.PrepareExecution(initialRequest)
|
||||
initialKey = prepared.Details().MessageContentHash // proposed addition
|
||||
|
||||
if cache contains a usable accepted artifact for initialKey:
|
||||
discard prepared handle
|
||||
return cached artifact
|
||||
|
||||
result = engine.RunPrepared(prepared)
|
||||
artifact = interpret, validate, and correct through the Notarius operation
|
||||
if the operation produces a final accepted artifact:
|
||||
save artifact to cache under initialKey
|
||||
return application outcome
|
||||
```
|
||||
|
||||
Notarius retains the initial key even when acceptance requires several correction
|
||||
turns. It stores the final domain artifact, not a raw intermediate response,
|
||||
`RunResult`, exchange, or prepared handle. Failed, rejected, and
|
||||
validation-incomplete work is not saved. A hit requires ordinary decoding and
|
||||
usability checks; it does not replay model work or claim fresh domain validation.
|
||||
|
||||
## Ownership And Scope
|
||||
|
||||
PromptKit owns everything within a native request execution, including structural
|
||||
response validation, transport retries, and internal repair attempts. Notarius
|
||||
consumes the public response or terminal failure and then owns domain acceptance.
|
||||
Those internal attempts are not inputs to the cache's eligibility policy.
|
||||
|
||||
This request is solely for reusable message-content hashing. It does not request
|
||||
PromptKit-owned cache storage, a cache-hit execution result, cached-response
|
||||
injection, validation replay, exchange restoration, or a write-through response
|
||||
cache. Debug observations do not change this boundary. Notarius owns cache
|
||||
lookup, final acceptance, persistence, and reuse/bypass/refresh controls.
|
||||
|
||||
## Suggested Verification
|
||||
|
||||
- Repeated preparation in separate engines produces the same hash for identical
|
||||
role/text sequences despite different preparation timing or session identity.
|
||||
- Changing a role, text (including whitespace), order, message count, or boundary
|
||||
changes the hash; ambiguous concatenations do not collapse to the same input.
|
||||
- Settings-only, prompt-ID-only, and cache-hint-only changes preserve the hash
|
||||
when the effective role/text sequence is unchanged.
|
||||
- Rendered inputs, appended messages, and captured history participate in the
|
||||
hash when they contribute to the effective messages.
|
||||
- The hash is available before model execution, and reading it does not consume
|
||||
the prepared handle or alter its retained snapshot.
|
||||
- Existing `RenderedPromptHash` behavior remains intact. Document the new value
|
||||
in public GoDoc and add a short consumer example of lookup before execution.
|
||||
|
||||
Notarius will verify its own cache behavior separately: hits make no model calls,
|
||||
only accepted artifacts are saved, and corrected answers remain keyed to the
|
||||
initial task. The supplying PromptKit release will be selected during Notarius
|
||||
roadmap incorporation; this request does not assign a release number.
|
||||
Reference in New Issue
Block a user