Plan prepared message-content hashing
This commit is contained in:
220
docs/roadmap/implementation.md
Normal file
220
docs/roadmap/implementation.md
Normal file
@@ -0,0 +1,220 @@
|
||||
# Message-Content Hash Implementation Plan
|
||||
|
||||
## Objective And Authority
|
||||
|
||||
Implement the target in [message-content-hash.md](message-content-hash.md).
|
||||
That roadmap owns scope, hash semantics, and the accepted cross-release
|
||||
compatibility decision. The [Notarius request](notarius-request.md) is background;
|
||||
its application workflow does not add Promptkit requirements.
|
||||
|
||||
This is one end-to-end sprint, divided into four sequential, single-prompt
|
||||
stages sized for a Luna coding agent. Intermediate stages need not compile or
|
||||
leave a releasable repository. There are no intermediate human-review or
|
||||
external-publication gates. The final stage verifies the assembled result.
|
||||
Do not introduce temporary public APIs or compatibility shims just to make an
|
||||
intermediate stage compile.
|
||||
|
||||
Start with [development.md](../development.md) and its applicable policies:
|
||||
[agents](../policy/agents.md), [architecture](../policy/architecture.md),
|
||||
[testing](../policy/testing.md), and [documentation](../policy/documentation.md).
|
||||
Use the [release policy](../policy/release.md) when preparing release notes.
|
||||
Follow repository commit policy for completed units; commits are not approval
|
||||
gates between stages.
|
||||
|
||||
## Baseline And Fixed Design Choices
|
||||
|
||||
The reviewed baseline is Promptkit v0.23.3 plus the accepted roadmap commits.
|
||||
|
||||
- `internal/usecase/runner.go` assembles the final message sequence in
|
||||
`completePreparationWithStructuredOutput`, then builds `domain.PreparedRun`.
|
||||
`hashRenderedPrompt` includes session and cache metadata and stays unchanged.
|
||||
- `internal/usecase/prepared_execution.go` shares the prepared snapshot between
|
||||
retained details and execution payload. Public `Details` detaches through
|
||||
`fromDomainPreparedRun` in `convert.go`; `OnPrepared` uses that conversion too.
|
||||
- `PreparedRun` in `types.go` is a public data value with stable JSON field names.
|
||||
Its JSON marshaler in `json.go` uses the defined type
|
||||
`preparedRunJSONFields PreparedRun`, not a separately maintained field list.
|
||||
|
||||
Select these concrete implementation details for this sprint:
|
||||
|
||||
1. Add `MessageContentHash string` to internal and public `PreparedRun`.
|
||||
The public tag is `json:"message_content_hash"`, without `omitempty`.
|
||||
Zero/manual values retain the ordinary empty string; preparation populates
|
||||
the field. JSON decoding stores the supplied field rather than recomputing
|
||||
it or validating caller-owned message/hash consistency. Older JSON that
|
||||
omits the field decodes with the ordinary zero value.
|
||||
2. Compute the value once from the final prepared messages using the v1 scheme
|
||||
below. Carry the string through existing snapshot and conversion paths.
|
||||
Reading details does not rehash caller-modified copies or claim a handle.
|
||||
3. Keep the existing rendered hash and all its values unchanged. Do not add the
|
||||
new field to `RunResult`, `Exchange`, model requests, HTTP payloads, or
|
||||
metadata observer events merely for symmetry. Existing `OnPrepared` values
|
||||
include it because they are `PreparedRun` values.
|
||||
4. Add no hashing option, standalone public hash function, dependency, timer,
|
||||
storage feature, or application-specific interpretation. Preparation and
|
||||
execution retain their existing validation, ownership, and lifecycle rules.
|
||||
|
||||
## Stage 1 — Implement The Versioned Hash And Reference Tests
|
||||
|
||||
Work in `internal/usecase`, using a focused `message_content_hash.go` and
|
||||
`message_content_hash_test.go`. Implement the unexported helper
|
||||
`hashMessageContent(messages []domain.RenderedMessage) string`.
|
||||
|
||||
The exact v1 preimage is the concatenation of:
|
||||
|
||||
1. The literal ASCII prefix `promptkit/message-content/v1` followed by one NUL
|
||||
byte, expressed in Go as `"promptkit/message-content/v1\x00"`.
|
||||
2. The message count as an unsigned 64-bit big-endian integer.
|
||||
3. For each message in order: the role byte length as an unsigned 64-bit
|
||||
big-endian integer, the role bytes, then the content byte length in the same
|
||||
format and the content bytes. String lengths are Go `len(string)` byte
|
||||
counts, not Unicode code-point counts.
|
||||
|
||||
Hash that preimage with SHA-256 and return lowercase hexadecimal. This is an
|
||||
internal scheme specification, not a public format callers should parse.
|
||||
Include no session, cache-control, or other fields. Do not normalize strings
|
||||
or add validation to this helper; preparation already owns input validity.
|
||||
|
||||
Stream the prefix and fields into the hasher. Reuse the existing
|
||||
`writeRenderedPromptHashLength` and `writeRenderedPromptHashString` helpers
|
||||
without renaming or changing their behavior. A second linear hash traversal
|
||||
is sufficient; do not refactor the existing hash into a generic policy engine
|
||||
or build an aggregate serialized message buffer.
|
||||
|
||||
Protect the v1 scheme with these independently calculated fixed vectors:
|
||||
|
||||
| Messages, with strings in Go-style escaped notation | Expected hash |
|
||||
| --- | --- |
|
||||
| Empty slice | `4b3d27f1ccb333598f041ab4c079f285ba65b095066dda9064e23e0de57116c7` |
|
||||
| `[("user", "hello")]` | `34e867fb7cfe57c68d3a25ce6f007751230b3c41915d927492d96158c6cb0f04` |
|
||||
| `[("system", "a\x00b"), ("user", "café\n")]` | `38bcd39c878e656be34449951e24db771a5fb14d809d54b918b0c18b79c1925f` |
|
||||
|
||||
Use literal expected digests, not expectations generated by the production
|
||||
helper. The empty-vector case tests framing only; it does not authorize empty
|
||||
prompt definitions. Add a compact relational table covering role, text,
|
||||
whitespace, order, count, and boundary changes, plus cache-control invariance.
|
||||
Include a one-message/two-message ambiguous-concatenation pair. Nil and empty
|
||||
slices both encode a zero count. Do not add subprocess tests just to prove a
|
||||
deterministic pure function; fixed vectors protect cross-process stability.
|
||||
|
||||
Validation: run the new helper tests and the existing `TestHashRenderedPrompt`
|
||||
tests in `internal/usecase`. Completion means the v1 vectors and semantic
|
||||
relations pass without changing existing rendered-hash expectations.
|
||||
|
||||
## Stage 2 — Wire Prepared Values And Snapshot Lifecycle
|
||||
|
||||
Update `internal/domain/domain.go`, `types.go`, `internal/usecase/runner.go`,
|
||||
and `convert.go`:
|
||||
|
||||
- Add the internal/public field beside `RenderedPromptHash` and document its
|
||||
public semantics in GoDoc. Cover role/text inputs, excluded metadata,
|
||||
opaque filename-safe output, preparation-time computation, and the accepted
|
||||
cross-release compatibility rule. Do not promise a permanently fixed scheme.
|
||||
- Populate it in `completePreparationWithStructuredOutput` from
|
||||
`effectivePrompt.Messages`, after history, rendered messages, and appended
|
||||
messages have been assembled. Do not hash raw definitions or an intermediate
|
||||
suffix, and do not add another preparation or source-loading pass.
|
||||
- Copy it in `fromDomainPreparedRun`. Check the existing snapshot path rather
|
||||
than creating another metadata snapshot, getter computation, or locking layer.
|
||||
`Prepare`, `Details`, ordinary/prepared `OnPrepared`, and ordinary execution's
|
||||
shared preparation should all use this captured value.
|
||||
|
||||
Update `json_contract_test.go`: populate the field in
|
||||
`fullyPopulatedPreparedRun`, assert the `message_content_hash` member in its
|
||||
round-trip test, and check empty-string encoding on a zero value and zero-value
|
||||
decoding when the member is absent. The defined-type JSON projection should
|
||||
inherit the field; do not create a new serialization DTO just for this change.
|
||||
|
||||
Extend the relevant tests in `prepared_execution_contract_test.go` and
|
||||
`prepared_notification_contract_test.go`, or add a small focused external-package
|
||||
test file. Verify:
|
||||
|
||||
- Successful `Prepare` and `PrepareExecution(...).Details()` agree for identical
|
||||
effective messages and expose a nonempty value before any model call.
|
||||
- Mutating the returned details' messages and hash does not alter later details
|
||||
or execution. Reading the field does not consume a handle, and retained details
|
||||
still expose it after execution or discard under the existing lifecycle.
|
||||
- `OnPrepared` receives the same captured hash on ordinary and split execution,
|
||||
with no additional preparation/notification call introduced for hashing.
|
||||
|
||||
Reuse existing recording clients and ownership fixtures. Do not duplicate their
|
||||
complete lifecycle, concurrency, credential, or repair test inventories.
|
||||
|
||||
Validation: run the focused prepared-value, notification, and JSON contract
|
||||
tests. Completion means the value survives all supported prepared projections
|
||||
and existing handle ownership remains intact.
|
||||
|
||||
## Stage 3 — Verify Public Content Semantics And Exclusions
|
||||
|
||||
Add focused `promptkit_test` coverage, preferably in
|
||||
`message_content_hash_contract_test.go`, using real engines with small synthetic
|
||||
definitions and an injected model client only where history or execution is
|
||||
needed. Keep cases grouped by distinct risks rather than building a Cartesian
|
||||
product of every setting and API.
|
||||
|
||||
1. **Equality and exclusions:** prepare the same role/text sequence repeatedly
|
||||
and in separate engines. Vary session ID and cache hints and verify the new
|
||||
hash stays equal while the existing rendered hash changes. Independently
|
||||
vary prompt/profile/backend identifiers, effective model/settings, direct
|
||||
synthetic credentials, and output/schema metadata outside message text;
|
||||
verify unchanged prepared role/text yields the same new hash. Use a valid
|
||||
in-memory schema where necessary. Avoid sleeps merely to vary timing.
|
||||
2. **Rendered contributions:** use a template with a variable and input text.
|
||||
Changes to rendered bytes change the hash; changes to unused variables or
|
||||
artifact URI/hash metadata alone do not when rendered messages are unchanged.
|
||||
Reuse existing input helper syntax and in-memory artifact fixtures.
|
||||
3. **Composition:** exercise appended messages and a captured exchange. Compare
|
||||
against a separately prepared explicit sequence with identical roles/text
|
||||
and no history identity; hashes agree. Changing a history/appended role or
|
||||
text changes the value. This proves the complete effective sequence is hashed,
|
||||
not just the newly rendered definition. Initial history capture may execute
|
||||
a fake client; preparation itself must not execute it.
|
||||
|
||||
Assert semantic equality/difference rather than duplicating the hash algorithm
|
||||
in facade tests. Stage 1 owns exact v1 bytes; facade tests own message selection
|
||||
and preparation integration. Retain existing rendered-hash tests unchanged.
|
||||
Do not test or implement any application use of the hash.
|
||||
|
||||
Validation: run the new contract tests alongside existing appended-message and
|
||||
exchange/preparation tests. Completion means every included/excluded category
|
||||
in the feature roadmap has proportionate coverage at its owning boundary.
|
||||
|
||||
## Stage 4 — Document, Demonstrate, And Validate The End State
|
||||
|
||||
Update these existing documentation owners:
|
||||
|
||||
- `docs/consumers/pkg-promptkit.md`: explain the narrower hash versus
|
||||
`RenderedPromptHash`, and show reading `details.MessageContentHash` after
|
||||
`PrepareExecution` with ordinary error handling and deferred `Discard`.
|
||||
Keep the example neutral about application interpretation.
|
||||
- `docs/internal/runner.md`: describe the preparation point, content-only
|
||||
preimage, streaming computation, and v1 scheme stability/reference tests.
|
||||
Public GoDoc remains the exact API contract owner.
|
||||
- `examples/go-library/prepare/main.go` and its test: include the new hash in
|
||||
the existing preparation summary and assert it is nonempty. Do not hard-code
|
||||
an example digest or introduce an application workflow or another example
|
||||
project. Existing offline preparation behavior remains unchanged.
|
||||
- Prepare `docs/releases/v0.24.0.md` for this new feature and link it from the
|
||||
README. Describe the new prepared-result JSON member, no configuration
|
||||
migration, and the additive Go struct field (positional literals may require
|
||||
adjustment). Use the normal licensing footer. Do not edit published notes,
|
||||
tag, push, or deliver a release as part of this implementation sprint.
|
||||
|
||||
Use the newest installed Go toolchain following `docs/development.md`, with
|
||||
`GOTOOLCHAIN=local` and `GOWORK=off`. Keep dependency versions and the consumer
|
||||
minimum unchanged for this feature. Format changed Go files, run
|
||||
`go test -race ./...`, `go vet ./...`, and `git diff --check`. The complete Go
|
||||
suite includes maintained example tests; do not add a separate repeated example
|
||||
gate. If `/tmp` is still full, select an isolated writable `TMPDIR` and build
|
||||
cache on a filesystem with space rather than deleting unrelated files.
|
||||
|
||||
Review the final diff against the feature roadmap: both hashes have their
|
||||
distinct semantics, all prepared projections carry the new value, no new model
|
||||
request field or application behavior has appeared, and reference vectors retain
|
||||
the selected scheme. Check new local documentation links. Use focused reruns
|
||||
while fixing failures and finish with validation appropriate to the final code.
|
||||
No upstream dependency publication or consumer-project changes are needed.
|
||||
|
||||
Completion is the implemented and documented hash, successful final validation,
|
||||
and reviewed commits under repository policy. Retire the completed planning
|
||||
documents during normal release preparation, not as an extra execution gate.
|
||||
@@ -11,7 +11,7 @@ semantics. Applications own its interpretation and use. Downstream caching is
|
||||
one motivation, not a Promptkit feature or a requirement on application behavior.
|
||||
|
||||
This roadmap defines the target capability and the work needed from the current
|
||||
baseline. It is not a staged implementation plan.
|
||||
baseline. The [implementation plan](implementation.md) owns staged execution.
|
||||
|
||||
## Current Baseline
|
||||
|
||||
|
||||
143
docs/roadmap/notarius-request.md
Normal file
143
docs/roadmap/notarius-request.md
Normal file
@@ -0,0 +1,143 @@
|
||||
# Notarius Request: Message-Content Hashing
|
||||
|
||||
**Status:** Requested by the Notarius consumer; proposed capability, not an
|
||||
implemented API or an approved PromptKit implementation plan.
|
||||
|
||||
**Reviewed baseline:** PromptKit v0.23.3.
|
||||
|
||||
## Requested Capability
|
||||
|
||||
Expose a deterministic hash of a prepared request's ordered message roles and
|
||||
exact text, available before model execution. `MessageContentHash` is the
|
||||
suggested name. This would let consumers reuse an accepted answer to the same
|
||||
message content without implementing their own message-hashing policy.
|
||||
|
||||
A field on `PreparedRun`, available through ordinary preparation and
|
||||
`PreparedExecution.Details()`, appears sufficient. The exact public API is for
|
||||
PromptKit to select; Notarius needs the value at the preparation boundary, not
|
||||
only after execution or through an execution observer.
|
||||
|
||||
## Consumer Need
|
||||
|
||||
Notarius runs pipelines whose stages depend on artifacts from earlier stages.
|
||||
If a later stage fails, an operator should be able to fix the problem and rerun
|
||||
without paying for successful earlier model work again. Notarius is replacing
|
||||
its checkpoint machinery with a small accepted-artifact cache.
|
||||
|
||||
The cached value is **an accepted answer to the original task**. Notarius will
|
||||
prepare the initial request, obtain its message-content hash, and look for
|
||||
`cache/<hash>.json`. A hit loads the accepted domain artifact directly. A miss
|
||||
executes the full application operation, including domain interpretation,
|
||||
validation, and any application-directed correction passes. Only the final
|
||||
accepted artifact is stored, under the original request's hash.
|
||||
|
||||
This is intentionally a convenient reuse policy, not proof that every execution
|
||||
setting, validation policy, or application implementation is identical. A change
|
||||
to model settings alone should not invalidate an otherwise reusable answer.
|
||||
Operators can bypass or refresh the application cache when they want fresh work.
|
||||
|
||||
## Hash Semantics
|
||||
|
||||
Hash the complete effective message sequence exposed by native preparation:
|
||||
rendered definition messages, appended messages, and captured conversation
|
||||
history when present. Include each message's role and exact content, preserving
|
||||
message order and boundaries.
|
||||
|
||||
| Included | Excluded |
|
||||
| --- | --- |
|
||||
| Ordered message roles and exact text | Session, run, and parent-run identity; timestamps and timings |
|
||||
| Message boundaries and sequence length | Provider cache-control hints and other non-content message metadata |
|
||||
| Any input, instruction, or schema text actually rendered into messages | Model, backend, profile, credentials, generation settings, timeouts, budgets, and retry/repair settings |
|
||||
| | Prompt ID, definition hash, input hashes/paths, and schema/output-contract metadata outside message text |
|
||||
|
||||
No whitespace trimming, Unicode normalization, or semantic equivalence is
|
||||
requested: hash the exact prepared text. Different roles, ordering, boundaries,
|
||||
or text should produce different hashes, subject to ordinary hash collision
|
||||
properties. Identical role/text sequences should produce the same value across
|
||||
requests, engines, and process invocations regardless of excluded metadata.
|
||||
For example, changing a prompt ID while retaining identical messages should
|
||||
preserve the hash. Changing settings that also changes rendered text should
|
||||
change it because the text changed.
|
||||
|
||||
Return an opaque, filename-safe value suitable for a persisted cache key.
|
||||
PromptKit should own and document the hashing semantics and compatibility
|
||||
behavior. A deterministic, unambiguous encoding and a standard hash such as
|
||||
SHA-256 would suffice; the precise encoding is an upstream implementation choice.
|
||||
No general fingerprint configuration or selectable hashing-policy framework is
|
||||
needed for this request.
|
||||
|
||||
## Existing APIs And The Gap
|
||||
|
||||
[`PreparedRun`](../../types.go) already exposes messages and
|
||||
`RenderedPromptHash`. The latter includes session identity and message cache
|
||||
hints, as described in the [runner documentation](../internal/runner.md).
|
||||
Hashing the entire prepared value would additionally capture settings and
|
||||
volatile preparation metadata. Neither supplies the requested content-only key.
|
||||
|
||||
Keep the existing hash's semantics intact and add the narrower capability.
|
||||
Notarius could manually project and hash message fields, but that would preserve
|
||||
the duplicated library mechanics this migration aims to remove.
|
||||
|
||||
[Native split preparation and execution](../consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later)
|
||||
already provides the needed execution boundary. Preparation makes no model call
|
||||
and reserves no backend capacity. On a miss, `RunPrepared` can execute the retained
|
||||
snapshot. On a hit, the application can discard the handle. No new execution
|
||||
protocol is requested.
|
||||
|
||||
## Intended Application Flow
|
||||
|
||||
The following is conceptual pseudocode, not an existing hash API:
|
||||
|
||||
```text
|
||||
prepared = engine.PrepareExecution(initialRequest)
|
||||
initialKey = prepared.Details().MessageContentHash // proposed addition
|
||||
|
||||
if cache contains a usable accepted artifact for initialKey:
|
||||
discard prepared handle
|
||||
return cached artifact
|
||||
|
||||
result = engine.RunPrepared(prepared)
|
||||
artifact = interpret, validate, and correct through the Notarius operation
|
||||
if the operation produces a final accepted artifact:
|
||||
save artifact to cache under initialKey
|
||||
return application outcome
|
||||
```
|
||||
|
||||
Notarius retains the initial key even when acceptance requires several correction
|
||||
turns. It stores the final domain artifact, not a raw intermediate response,
|
||||
`RunResult`, exchange, or prepared handle. Failed, rejected, and
|
||||
validation-incomplete work is not saved. A hit requires ordinary decoding and
|
||||
usability checks; it does not replay model work or claim fresh domain validation.
|
||||
|
||||
## Ownership And Scope
|
||||
|
||||
PromptKit owns everything within a native request execution, including structural
|
||||
response validation, transport retries, and internal repair attempts. Notarius
|
||||
consumes the public response or terminal failure and then owns domain acceptance.
|
||||
Those internal attempts are not inputs to the cache's eligibility policy.
|
||||
|
||||
This request is solely for reusable message-content hashing. It does not request
|
||||
PromptKit-owned cache storage, a cache-hit execution result, cached-response
|
||||
injection, validation replay, exchange restoration, or a write-through response
|
||||
cache. Debug observations do not change this boundary. Notarius owns cache
|
||||
lookup, final acceptance, persistence, and reuse/bypass/refresh controls.
|
||||
|
||||
## Suggested Verification
|
||||
|
||||
- Repeated preparation in separate engines produces the same hash for identical
|
||||
role/text sequences despite different preparation timing or session identity.
|
||||
- Changing a role, text (including whitespace), order, message count, or boundary
|
||||
changes the hash; ambiguous concatenations do not collapse to the same input.
|
||||
- Settings-only, prompt-ID-only, and cache-hint-only changes preserve the hash
|
||||
when the effective role/text sequence is unchanged.
|
||||
- Rendered inputs, appended messages, and captured history participate in the
|
||||
hash when they contribute to the effective messages.
|
||||
- The hash is available before model execution, and reading it does not consume
|
||||
the prepared handle or alter its retained snapshot.
|
||||
- Existing `RenderedPromptHash` behavior remains intact. Document the new value
|
||||
in public GoDoc and add a short consumer example of lookup before execution.
|
||||
|
||||
Notarius will verify its own cache behavior separately: hits make no model calls,
|
||||
only accepted artifacts are saved, and corrected answers remain keyed to the
|
||||
initial task. The supplying PromptKit release will be selected during Notarius
|
||||
roadmap incorporation; this request does not assign a release number.
|
||||
Reference in New Issue
Block a user