Files
promptkit/docs/roadmap/implementation.md

13 KiB

Message-Content Hash Implementation Plan

Objective And Authority

Implement the target in message-content-hash.md. That roadmap owns scope, hash semantics, and the accepted cross-release compatibility decision. The Notarius request is background; its application workflow does not add Promptkit requirements.

This is one end-to-end sprint, divided into four sequential, single-prompt stages sized for a Luna coding agent. Intermediate stages need not compile or leave a releasable repository. There are no intermediate human-review or external-publication gates. The final stage verifies the assembled result. Do not introduce temporary public APIs or compatibility shims just to make an intermediate stage compile.

Start with development.md and its applicable policies: agents, architecture, testing, and documentation. Use the release policy when preparing release notes. Follow repository commit policy for completed units; commits are not approval gates between stages.

Baseline And Fixed Design Choices

The reviewed baseline is Promptkit v0.23.3 plus the accepted roadmap commits.

  • internal/usecase/runner.go assembles the final message sequence in completePreparationWithStructuredOutput, then builds domain.PreparedRun. hashRenderedPrompt includes session and cache metadata and stays unchanged.
  • internal/usecase/prepared_execution.go shares the prepared snapshot between retained details and execution payload. Public Details detaches through fromDomainPreparedRun in convert.go; OnPrepared uses that conversion too.
  • PreparedRun in types.go is a public data value with stable JSON field names. Its JSON marshaler in json.go uses the defined type preparedRunJSONFields PreparedRun, not a separately maintained field list.

Select these concrete implementation details for this sprint:

  1. Add MessageContentHash string to internal and public PreparedRun. The public tag is json:"message_content_hash", without omitempty. Zero/manual values retain the ordinary empty string; preparation populates the field. JSON decoding stores the supplied field rather than recomputing it or validating caller-owned message/hash consistency. Older JSON that omits the field decodes with the ordinary zero value.
  2. Compute the value once from the final prepared messages using the v1 scheme below. Carry the string through existing snapshot and conversion paths. Reading details does not rehash caller-modified copies or claim a handle.
  3. Keep the existing rendered hash and all its values unchanged. Do not add the new field to RunResult, Exchange, model requests, HTTP payloads, or metadata observer events merely for symmetry. Existing OnPrepared values include it because they are PreparedRun values.
  4. Add no hashing option, standalone public hash function, dependency, timer, storage feature, or application-specific interpretation. Preparation and execution retain their existing validation, ownership, and lifecycle rules.

Stage 1 — Implement The Versioned Hash And Reference Tests

Status: complete.

Work in internal/usecase, using a focused message_content_hash.go and message_content_hash_test.go. Implement the unexported helper hashMessageContent(messages []domain.RenderedMessage) string.

The exact v1 preimage is the concatenation of:

  1. The literal ASCII prefix promptkit/message-content/v1 followed by one NUL byte, expressed in Go as "promptkit/message-content/v1\x00".
  2. The message count as an unsigned 64-bit big-endian integer.
  3. For each message in order: the role byte length as an unsigned 64-bit big-endian integer, the role bytes, then the content byte length in the same format and the content bytes. String lengths are Go len(string) byte counts, not Unicode code-point counts.

Hash that preimage with SHA-256 and return lowercase hexadecimal. This is an internal scheme specification, not a public format callers should parse. Include no session, cache-control, or other fields. Do not normalize strings or add validation to this helper; preparation already owns input validity.

Stream the prefix and fields into the hasher. Reuse the existing writeRenderedPromptHashLength and writeRenderedPromptHashString helpers without renaming or changing their behavior. A second linear hash traversal is sufficient; do not refactor the existing hash into a generic policy engine or build an aggregate serialized message buffer.

Protect the v1 scheme with these independently calculated fixed vectors:

Messages, with strings in Go-style escaped notation Expected hash
Empty slice 4b3d27f1ccb333598f041ab4c079f285ba65b095066dda9064e23e0de57116c7
[("user", "hello")] 34e867fb7cfe57c68d3a25ce6f007751230b3c41915d927492d96158c6cb0f04
[("system", "a\x00b"), ("user", "café\n")] 38bcd39c878e656be34449951e24db771a5fb14d809d54b918b0c18b79c1925f

Use literal expected digests, not expectations generated by the production helper. The empty-vector case tests framing only; it does not authorize empty prompt definitions. Add a compact relational table covering role, text, whitespace, order, count, and boundary changes, plus cache-control invariance. Include a one-message/two-message ambiguous-concatenation pair. Nil and empty slices both encode a zero count. Do not add subprocess tests just to prove a deterministic pure function; fixed vectors protect cross-process stability.

Validation: run the new helper tests and the existing TestHashRenderedPrompt tests in internal/usecase. Completion means the v1 vectors and semantic relations pass without changing existing rendered-hash expectations.

Stage 2 — Wire Prepared Values And Snapshot Lifecycle

Status: complete.

Update internal/domain/domain.go, types.go, internal/usecase/runner.go, and convert.go:

  • Add the internal/public field beside RenderedPromptHash and document its public semantics in GoDoc. Cover role/text inputs, excluded metadata, opaque filename-safe output, preparation-time computation, and the accepted cross-release compatibility rule. Do not promise a permanently fixed scheme.
  • Populate it in completePreparationWithStructuredOutput from effectivePrompt.Messages, after history, rendered messages, and appended messages have been assembled. Do not hash raw definitions or an intermediate suffix, and do not add another preparation or source-loading pass.
  • Copy it in fromDomainPreparedRun. Check the existing snapshot path rather than creating another metadata snapshot, getter computation, or locking layer. Prepare, Details, ordinary/prepared OnPrepared, and ordinary execution's shared preparation should all use this captured value.

Update json_contract_test.go: populate the field in fullyPopulatedPreparedRun, assert the message_content_hash member in its round-trip test, and check empty-string encoding on a zero value and zero-value decoding when the member is absent. The defined-type JSON projection should inherit the field; do not create a new serialization DTO just for this change.

Extend the relevant tests in prepared_execution_contract_test.go and prepared_notification_contract_test.go, or add a small focused external-package test file. Verify:

  • Successful Prepare and PrepareExecution(...).Details() agree for identical effective messages and expose a nonempty value before any model call.
  • Mutating the returned details' messages and hash does not alter later details or execution. Reading the field does not consume a handle, and retained details still expose it after execution or discard under the existing lifecycle.
  • OnPrepared receives the same captured hash on ordinary and split execution, with no additional preparation/notification call introduced for hashing.

Reuse existing recording clients and ownership fixtures. Do not duplicate their complete lifecycle, concurrency, credential, or repair test inventories.

Validation: run the focused prepared-value, notification, and JSON contract tests. Completion means the value survives all supported prepared projections and existing handle ownership remains intact.

Stage 3 — Verify Public Content Semantics And Exclusions

Status: complete.

Add focused promptkit_test coverage, preferably in message_content_hash_contract_test.go, using real engines with small synthetic definitions and an injected model client only where history or execution is needed. Keep cases grouped by distinct risks rather than building a Cartesian product of every setting and API.

  1. Equality and exclusions: prepare the same role/text sequence repeatedly and in separate engines. Vary session ID and cache hints and verify the new hash stays equal while the existing rendered hash changes. Independently vary prompt/profile/backend identifiers, effective model/settings, direct synthetic credentials, and output/schema metadata outside message text; verify unchanged prepared role/text yields the same new hash. Use a valid in-memory schema where necessary. Avoid sleeps merely to vary timing.
  2. Rendered contributions: use a template with a variable and input text. Changes to rendered bytes change the hash; changes to unused variables or artifact URI/hash metadata alone do not when rendered messages are unchanged. Reuse existing input helper syntax and in-memory artifact fixtures.
  3. Composition: exercise appended messages and a captured exchange. Compare against a separately prepared explicit sequence with identical roles/text and no history identity; hashes agree. Changing a history/appended role or text changes the value. This proves the complete effective sequence is hashed, not just the newly rendered definition. Initial history capture may execute a fake client; preparation itself must not execute it.

Assert semantic equality/difference rather than duplicating the hash algorithm in facade tests. Stage 1 owns exact v1 bytes; facade tests own message selection and preparation integration. Retain existing rendered-hash tests unchanged. Do not test or implement any application use of the hash.

Validation: run the new contract tests alongside existing appended-message and exchange/preparation tests. Completion means every included/excluded category in the feature roadmap has proportionate coverage at its owning boundary.

Stage 4 — Document, Demonstrate, And Validate The End State

Status: complete.

Update these existing documentation owners:

  • docs/consumers/pkg-promptkit.md: explain the narrower hash versus RenderedPromptHash, and show reading details.MessageContentHash after PrepareExecution with ordinary error handling and deferred Discard. Keep the example neutral about application interpretation.
  • docs/internal/runner.md: describe the preparation point, content-only preimage, streaming computation, and v1 scheme stability/reference tests. Public GoDoc remains the exact API contract owner.
  • examples/go-library/prepare/main.go and its test: include the new hash in the existing preparation summary and assert it is nonempty. Do not hard-code an example digest or introduce an application workflow or another example project. Existing offline preparation behavior remains unchanged.
  • Prepare docs/releases/v0.24.0.md for this new feature and link it from the README. Describe the new prepared-result JSON member, no configuration migration, and the additive Go struct field (positional literals may require adjustment). Use the normal licensing footer. Do not edit published notes, tag, push, or deliver a release as part of this implementation sprint.

Use the newest installed Go toolchain following docs/development.md, with GOTOOLCHAIN=local and GOWORK=off. Keep dependency versions and the consumer minimum unchanged for this feature. Format changed Go files, run go test -race ./..., go vet ./..., and git diff --check. The complete Go suite includes maintained example tests; do not add a separate repeated example gate. If /tmp is still full, select an isolated writable TMPDIR and build cache on a filesystem with space rather than deleting unrelated files.

Review the final diff against the feature roadmap: both hashes have their distinct semantics, all prepared projections carry the new value, no new model request field or application behavior has appeared, and reference vectors retain the selected scheme. Check new local documentation links. Use focused reruns while fixing failures and finish with validation appropriate to the final code. No upstream dependency publication or consumer-project changes are needed.

Completion is the implemented and documented hash, successful final validation, and reviewed commits under repository policy. Retire the completed planning documents during normal release preparation, not as an extra execution gate.