13 KiB
Message-Content Hash Implementation Plan
Objective And Authority
Implement the target in message-content-hash.md. That roadmap owns scope, hash semantics, and the accepted cross-release compatibility decision. The Notarius request is background; its application workflow does not add Promptkit requirements.
This is one end-to-end sprint, divided into four sequential, single-prompt stages sized for a Luna coding agent. Intermediate stages need not compile or leave a releasable repository. There are no intermediate human-review or external-publication gates. The final stage verifies the assembled result. Do not introduce temporary public APIs or compatibility shims just to make an intermediate stage compile.
Start with development.md and its applicable policies: agents, architecture, testing, and documentation. Use the release policy when preparing release notes. Follow repository commit policy for completed units; commits are not approval gates between stages.
Baseline And Fixed Design Choices
The reviewed baseline is Promptkit v0.23.3 plus the accepted roadmap commits.
internal/usecase/runner.goassembles the final message sequence incompletePreparationWithStructuredOutput, then buildsdomain.PreparedRun.hashRenderedPromptincludes session and cache metadata and stays unchanged.internal/usecase/prepared_execution.goshares the prepared snapshot between retained details and execution payload. PublicDetailsdetaches throughfromDomainPreparedRuninconvert.go;OnPrepareduses that conversion too.PreparedRunintypes.gois a public data value with stable JSON field names. Its JSON marshaler injson.gouses the defined typepreparedRunJSONFields PreparedRun, not a separately maintained field list.
Select these concrete implementation details for this sprint:
- Add
MessageContentHash stringto internal and publicPreparedRun. The public tag isjson:"message_content_hash", withoutomitempty. Zero/manual values retain the ordinary empty string; preparation populates the field. JSON decoding stores the supplied field rather than recomputing it or validating caller-owned message/hash consistency. Older JSON that omits the field decodes with the ordinary zero value. - Compute the value once from the final prepared messages using the v1 scheme below. Carry the string through existing snapshot and conversion paths. Reading details does not rehash caller-modified copies or claim a handle.
- Keep the existing rendered hash and all its values unchanged. Do not add the
new field to
RunResult,Exchange, model requests, HTTP payloads, or metadata observer events merely for symmetry. ExistingOnPreparedvalues include it because they arePreparedRunvalues. - Add no hashing option, standalone public hash function, dependency, timer, storage feature, or application-specific interpretation. Preparation and execution retain their existing validation, ownership, and lifecycle rules.
Stage 1 — Implement The Versioned Hash And Reference Tests
Status: complete.
Work in internal/usecase, using a focused message_content_hash.go and
message_content_hash_test.go. Implement the unexported helper
hashMessageContent(messages []domain.RenderedMessage) string.
The exact v1 preimage is the concatenation of:
- The literal ASCII prefix
promptkit/message-content/v1followed by one NUL byte, expressed in Go as"promptkit/message-content/v1\x00". - The message count as an unsigned 64-bit big-endian integer.
- For each message in order: the role byte length as an unsigned 64-bit
big-endian integer, the role bytes, then the content byte length in the same
format and the content bytes. String lengths are Go
len(string)byte counts, not Unicode code-point counts.
Hash that preimage with SHA-256 and return lowercase hexadecimal. This is an internal scheme specification, not a public format callers should parse. Include no session, cache-control, or other fields. Do not normalize strings or add validation to this helper; preparation already owns input validity.
Stream the prefix and fields into the hasher. Reuse the existing
writeRenderedPromptHashLength and writeRenderedPromptHashString helpers
without renaming or changing their behavior. A second linear hash traversal
is sufficient; do not refactor the existing hash into a generic policy engine
or build an aggregate serialized message buffer.
Protect the v1 scheme with these independently calculated fixed vectors:
| Messages, with strings in Go-style escaped notation | Expected hash |
|---|---|
| Empty slice | 4b3d27f1ccb333598f041ab4c079f285ba65b095066dda9064e23e0de57116c7 |
[("user", "hello")] |
34e867fb7cfe57c68d3a25ce6f007751230b3c41915d927492d96158c6cb0f04 |
[("system", "a\x00b"), ("user", "café\n")] |
38bcd39c878e656be34449951e24db771a5fb14d809d54b918b0c18b79c1925f |
Use literal expected digests, not expectations generated by the production helper. The empty-vector case tests framing only; it does not authorize empty prompt definitions. Add a compact relational table covering role, text, whitespace, order, count, and boundary changes, plus cache-control invariance. Include a one-message/two-message ambiguous-concatenation pair. Nil and empty slices both encode a zero count. Do not add subprocess tests just to prove a deterministic pure function; fixed vectors protect cross-process stability.
Validation: run the new helper tests and the existing TestHashRenderedPrompt
tests in internal/usecase. Completion means the v1 vectors and semantic
relations pass without changing existing rendered-hash expectations.
Stage 2 — Wire Prepared Values And Snapshot Lifecycle
Status: complete.
Update internal/domain/domain.go, types.go, internal/usecase/runner.go,
and convert.go:
- Add the internal/public field beside
RenderedPromptHashand document its public semantics in GoDoc. Cover role/text inputs, excluded metadata, opaque filename-safe output, preparation-time computation, and the accepted cross-release compatibility rule. Do not promise a permanently fixed scheme. - Populate it in
completePreparationWithStructuredOutputfromeffectivePrompt.Messages, after history, rendered messages, and appended messages have been assembled. Do not hash raw definitions or an intermediate suffix, and do not add another preparation or source-loading pass. - Copy it in
fromDomainPreparedRun. Check the existing snapshot path rather than creating another metadata snapshot, getter computation, or locking layer.Prepare,Details, ordinary/preparedOnPrepared, and ordinary execution's shared preparation should all use this captured value.
Update json_contract_test.go: populate the field in
fullyPopulatedPreparedRun, assert the message_content_hash member in its
round-trip test, and check empty-string encoding on a zero value and zero-value
decoding when the member is absent. The defined-type JSON projection should
inherit the field; do not create a new serialization DTO just for this change.
Extend the relevant tests in prepared_execution_contract_test.go and
prepared_notification_contract_test.go, or add a small focused external-package
test file. Verify:
- Successful
PrepareandPrepareExecution(...).Details()agree for identical effective messages and expose a nonempty value before any model call. - Mutating the returned details' messages and hash does not alter later details or execution. Reading the field does not consume a handle, and retained details still expose it after execution or discard under the existing lifecycle.
OnPreparedreceives the same captured hash on ordinary and split execution, with no additional preparation/notification call introduced for hashing.
Reuse existing recording clients and ownership fixtures. Do not duplicate their complete lifecycle, concurrency, credential, or repair test inventories.
Validation: run the focused prepared-value, notification, and JSON contract tests. Completion means the value survives all supported prepared projections and existing handle ownership remains intact.
Stage 3 — Verify Public Content Semantics And Exclusions
Status: complete.
Add focused promptkit_test coverage, preferably in
message_content_hash_contract_test.go, using real engines with small synthetic
definitions and an injected model client only where history or execution is
needed. Keep cases grouped by distinct risks rather than building a Cartesian
product of every setting and API.
- Equality and exclusions: prepare the same role/text sequence repeatedly and in separate engines. Vary session ID and cache hints and verify the new hash stays equal while the existing rendered hash changes. Independently vary prompt/profile/backend identifiers, effective model/settings, direct synthetic credentials, and output/schema metadata outside message text; verify unchanged prepared role/text yields the same new hash. Use a valid in-memory schema where necessary. Avoid sleeps merely to vary timing.
- Rendered contributions: use a template with a variable and input text. Changes to rendered bytes change the hash; changes to unused variables or artifact URI/hash metadata alone do not when rendered messages are unchanged. Reuse existing input helper syntax and in-memory artifact fixtures.
- Composition: exercise appended messages and a captured exchange. Compare against a separately prepared explicit sequence with identical roles/text and no history identity; hashes agree. Changing a history/appended role or text changes the value. This proves the complete effective sequence is hashed, not just the newly rendered definition. Initial history capture may execute a fake client; preparation itself must not execute it.
Assert semantic equality/difference rather than duplicating the hash algorithm in facade tests. Stage 1 owns exact v1 bytes; facade tests own message selection and preparation integration. Retain existing rendered-hash tests unchanged. Do not test or implement any application use of the hash.
Validation: run the new contract tests alongside existing appended-message and exchange/preparation tests. Completion means every included/excluded category in the feature roadmap has proportionate coverage at its owning boundary.
Stage 4 — Document, Demonstrate, And Validate The End State
Status: complete.
Update these existing documentation owners:
docs/consumers/pkg-promptkit.md: explain the narrower hash versusRenderedPromptHash, and show readingdetails.MessageContentHashafterPrepareExecutionwith ordinary error handling and deferredDiscard. Keep the example neutral about application interpretation.docs/internal/runner.md: describe the preparation point, content-only preimage, streaming computation, and v1 scheme stability/reference tests. Public GoDoc remains the exact API contract owner.examples/go-library/prepare/main.goand its test: include the new hash in the existing preparation summary and assert it is nonempty. Do not hard-code an example digest or introduce an application workflow or another example project. Existing offline preparation behavior remains unchanged.- Prepare
docs/releases/v0.24.0.mdfor this new feature and link it from the README. Describe the new prepared-result JSON member, no configuration migration, and the additive Go struct field (positional literals may require adjustment). Use the normal licensing footer. Do not edit published notes, tag, push, or deliver a release as part of this implementation sprint.
Use the newest installed Go toolchain following docs/development.md, with
GOTOOLCHAIN=local and GOWORK=off. Keep dependency versions and the consumer
minimum unchanged for this feature. Format changed Go files, run
go test -race ./..., go vet ./..., and git diff --check. The complete Go
suite includes maintained example tests; do not add a separate repeated example
gate. If /tmp is still full, select an isolated writable TMPDIR and build
cache on a filesystem with space rather than deleting unrelated files.
Review the final diff against the feature roadmap: both hashes have their distinct semantics, all prepared projections carry the new value, no new model request field or application behavior has appeared, and reference vectors retain the selected scheme. Check new local documentation links. Use focused reruns while fixing failures and finish with validation appropriate to the final code. No upstream dependency publication or consumer-project changes are needed.
Completion is the implemented and documented hash, successful final validation, and reviewed commits under repository policy. Retire the completed planning documents during normal release preparation, not as an extra execution gate.