diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 876bd4b..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,228 +0,0 @@ -# Message-Content Hash Implementation Plan - -## Objective And Authority - -Implement the target in [message-content-hash.md](message-content-hash.md). -That roadmap owns scope, hash semantics, and the accepted cross-release -compatibility decision. The [Notarius request](notarius-request.md) is background; -its application workflow does not add Promptkit requirements. - -This is one end-to-end sprint, divided into four sequential, single-prompt -stages sized for a Luna coding agent. Intermediate stages need not compile or -leave a releasable repository. There are no intermediate human-review or -external-publication gates. The final stage verifies the assembled result. -Do not introduce temporary public APIs or compatibility shims just to make an -intermediate stage compile. - -Start with [development.md](../development.md) and its applicable policies: -[agents](../policy/agents.md), [architecture](../policy/architecture.md), -[testing](../policy/testing.md), and [documentation](../policy/documentation.md). -Use the [release policy](../policy/release.md) when preparing release notes. -Follow repository commit policy for completed units; commits are not approval -gates between stages. - -## Baseline And Fixed Design Choices - -The reviewed baseline is Promptkit v0.23.3 plus the accepted roadmap commits. - -- `internal/usecase/runner.go` assembles the final message sequence in - `completePreparationWithStructuredOutput`, then builds `domain.PreparedRun`. - `hashRenderedPrompt` includes session and cache metadata and stays unchanged. -- `internal/usecase/prepared_execution.go` shares the prepared snapshot between - retained details and execution payload. Public `Details` detaches through - `fromDomainPreparedRun` in `convert.go`; `OnPrepared` uses that conversion too. -- `PreparedRun` in `types.go` is a public data value with stable JSON field names. - Its JSON marshaler in `json.go` uses the defined type - `preparedRunJSONFields PreparedRun`, not a separately maintained field list. - -Select these concrete implementation details for this sprint: - -1. Add `MessageContentHash string` to internal and public `PreparedRun`. - The public tag is `json:"message_content_hash"`, without `omitempty`. - Zero/manual values retain the ordinary empty string; preparation populates - the field. JSON decoding stores the supplied field rather than recomputing - it or validating caller-owned message/hash consistency. Older JSON that - omits the field decodes with the ordinary zero value. -2. Compute the value once from the final prepared messages using the v1 scheme - below. Carry the string through existing snapshot and conversion paths. - Reading details does not rehash caller-modified copies or claim a handle. -3. Keep the existing rendered hash and all its values unchanged. Do not add the - new field to `RunResult`, `Exchange`, model requests, HTTP payloads, or - metadata observer events merely for symmetry. Existing `OnPrepared` values - include it because they are `PreparedRun` values. -4. Add no hashing option, standalone public hash function, dependency, timer, - storage feature, or application-specific interpretation. Preparation and - execution retain their existing validation, ownership, and lifecycle rules. - -## Stage 1 — Implement The Versioned Hash And Reference Tests - -Status: complete. - -Work in `internal/usecase`, using a focused `message_content_hash.go` and -`message_content_hash_test.go`. Implement the unexported helper -`hashMessageContent(messages []domain.RenderedMessage) string`. - -The exact v1 preimage is the concatenation of: - -1. The literal ASCII prefix `promptkit/message-content/v1` followed by one NUL - byte, expressed in Go as `"promptkit/message-content/v1\x00"`. -2. The message count as an unsigned 64-bit big-endian integer. -3. For each message in order: the role byte length as an unsigned 64-bit - big-endian integer, the role bytes, then the content byte length in the same - format and the content bytes. String lengths are Go `len(string)` byte - counts, not Unicode code-point counts. - -Hash that preimage with SHA-256 and return lowercase hexadecimal. This is an -internal scheme specification, not a public format callers should parse. -Include no session, cache-control, or other fields. Do not normalize strings -or add validation to this helper; preparation already owns input validity. - -Stream the prefix and fields into the hasher. Reuse the existing -`writeRenderedPromptHashLength` and `writeRenderedPromptHashString` helpers -without renaming or changing their behavior. A second linear hash traversal -is sufficient; do not refactor the existing hash into a generic policy engine -or build an aggregate serialized message buffer. - -Protect the v1 scheme with these independently calculated fixed vectors: - -| Messages, with strings in Go-style escaped notation | Expected hash | -| --- | --- | -| Empty slice | `4b3d27f1ccb333598f041ab4c079f285ba65b095066dda9064e23e0de57116c7` | -| `[("user", "hello")]` | `34e867fb7cfe57c68d3a25ce6f007751230b3c41915d927492d96158c6cb0f04` | -| `[("system", "a\x00b"), ("user", "café\n")]` | `38bcd39c878e656be34449951e24db771a5fb14d809d54b918b0c18b79c1925f` | - -Use literal expected digests, not expectations generated by the production -helper. The empty-vector case tests framing only; it does not authorize empty -prompt definitions. Add a compact relational table covering role, text, -whitespace, order, count, and boundary changes, plus cache-control invariance. -Include a one-message/two-message ambiguous-concatenation pair. Nil and empty -slices both encode a zero count. Do not add subprocess tests just to prove a -deterministic pure function; fixed vectors protect cross-process stability. - -Validation: run the new helper tests and the existing `TestHashRenderedPrompt` -tests in `internal/usecase`. Completion means the v1 vectors and semantic -relations pass without changing existing rendered-hash expectations. - -## Stage 2 — Wire Prepared Values And Snapshot Lifecycle - -Status: complete. - -Update `internal/domain/domain.go`, `types.go`, `internal/usecase/runner.go`, -and `convert.go`: - -- Add the internal/public field beside `RenderedPromptHash` and document its - public semantics in GoDoc. Cover role/text inputs, excluded metadata, - opaque filename-safe output, preparation-time computation, and the accepted - cross-release compatibility rule. Do not promise a permanently fixed scheme. -- Populate it in `completePreparationWithStructuredOutput` from - `effectivePrompt.Messages`, after history, rendered messages, and appended - messages have been assembled. Do not hash raw definitions or an intermediate - suffix, and do not add another preparation or source-loading pass. -- Copy it in `fromDomainPreparedRun`. Check the existing snapshot path rather - than creating another metadata snapshot, getter computation, or locking layer. - `Prepare`, `Details`, ordinary/prepared `OnPrepared`, and ordinary execution's - shared preparation should all use this captured value. - -Update `json_contract_test.go`: populate the field in -`fullyPopulatedPreparedRun`, assert the `message_content_hash` member in its -round-trip test, and check empty-string encoding on a zero value and zero-value -decoding when the member is absent. The defined-type JSON projection should -inherit the field; do not create a new serialization DTO just for this change. - -Extend the relevant tests in `prepared_execution_contract_test.go` and -`prepared_notification_contract_test.go`, or add a small focused external-package -test file. Verify: - -- Successful `Prepare` and `PrepareExecution(...).Details()` agree for identical - effective messages and expose a nonempty value before any model call. -- Mutating the returned details' messages and hash does not alter later details - or execution. Reading the field does not consume a handle, and retained details - still expose it after execution or discard under the existing lifecycle. -- `OnPrepared` receives the same captured hash on ordinary and split execution, - with no additional preparation/notification call introduced for hashing. - -Reuse existing recording clients and ownership fixtures. Do not duplicate their -complete lifecycle, concurrency, credential, or repair test inventories. - -Validation: run the focused prepared-value, notification, and JSON contract -tests. Completion means the value survives all supported prepared projections -and existing handle ownership remains intact. - -## Stage 3 — Verify Public Content Semantics And Exclusions - -Status: complete. - -Add focused `promptkit_test` coverage, preferably in -`message_content_hash_contract_test.go`, using real engines with small synthetic -definitions and an injected model client only where history or execution is -needed. Keep cases grouped by distinct risks rather than building a Cartesian -product of every setting and API. - -1. **Equality and exclusions:** prepare the same role/text sequence repeatedly - and in separate engines. Vary session ID and cache hints and verify the new - hash stays equal while the existing rendered hash changes. Independently - vary prompt/profile/backend identifiers, effective model/settings, direct - synthetic credentials, and output/schema metadata outside message text; - verify unchanged prepared role/text yields the same new hash. Use a valid - in-memory schema where necessary. Avoid sleeps merely to vary timing. -2. **Rendered contributions:** use a template with a variable and input text. - Changes to rendered bytes change the hash; changes to unused variables or - artifact URI/hash metadata alone do not when rendered messages are unchanged. - Reuse existing input helper syntax and in-memory artifact fixtures. -3. **Composition:** exercise appended messages and a captured exchange. Compare - against a separately prepared explicit sequence with identical roles/text - and no history identity; hashes agree. Changing a history/appended role or - text changes the value. This proves the complete effective sequence is hashed, - not just the newly rendered definition. Initial history capture may execute - a fake client; preparation itself must not execute it. - -Assert semantic equality/difference rather than duplicating the hash algorithm -in facade tests. Stage 1 owns exact v1 bytes; facade tests own message selection -and preparation integration. Retain existing rendered-hash tests unchanged. -Do not test or implement any application use of the hash. - -Validation: run the new contract tests alongside existing appended-message and -exchange/preparation tests. Completion means every included/excluded category -in the feature roadmap has proportionate coverage at its owning boundary. - -## Stage 4 — Document, Demonstrate, And Validate The End State - -Status: complete. - -Update these existing documentation owners: - -- `docs/consumers/pkg-promptkit.md`: explain the narrower hash versus - `RenderedPromptHash`, and show reading `details.MessageContentHash` after - `PrepareExecution` with ordinary error handling and deferred `Discard`. - Keep the example neutral about application interpretation. -- `docs/internal/runner.md`: describe the preparation point, content-only - preimage, streaming computation, and v1 scheme stability/reference tests. - Public GoDoc remains the exact API contract owner. -- `examples/go-library/prepare/main.go` and its test: include the new hash in - the existing preparation summary and assert it is nonempty. Do not hard-code - an example digest or introduce an application workflow or another example - project. Existing offline preparation behavior remains unchanged. -- Prepare `docs/releases/v0.24.0.md` for this new feature and link it from the - README. Describe the new prepared-result JSON member, no configuration - migration, and the additive Go struct field (positional literals may require - adjustment). Use the normal licensing footer. Do not edit published notes, - tag, push, or deliver a release as part of this implementation sprint. - -Use the newest installed Go toolchain following `docs/development.md`, with -`GOTOOLCHAIN=local` and `GOWORK=off`. Keep dependency versions and the consumer -minimum unchanged for this feature. Format changed Go files, run -`go test -race ./...`, `go vet ./...`, and `git diff --check`. The complete Go -suite includes maintained example tests; do not add a separate repeated example -gate. If `/tmp` is still full, select an isolated writable `TMPDIR` and build -cache on a filesystem with space rather than deleting unrelated files. - -Review the final diff against the feature roadmap: both hashes have their -distinct semantics, all prepared projections carry the new value, no new model -request field or application behavior has appeared, and reference vectors retain -the selected scheme. Check new local documentation links. Use focused reruns -while fixing failures and finish with validation appropriate to the final code. -No upstream dependency publication or consumer-project changes are needed. - -Completion is the implemented and documented hash, successful final validation, -and reviewed commits under repository policy. Retire the completed planning -documents during normal release preparation, not as an extra execution gate. diff --git a/docs/roadmap/message-content-hash.md b/docs/roadmap/message-content-hash.md deleted file mode 100644 index 79f59f7..0000000 --- a/docs/roadmap/message-content-hash.md +++ /dev/null @@ -1,113 +0,0 @@ -# Prepared Message-Content Hash - -## Purpose And Ownership - -Expose a deterministic hash of a prepared request's ordered message roles and -exact text. Consumers can obtain this value before model execution without -implementing their own message projection and hashing mechanics. - -Promptkit owns the hash's inputs, computation, availability, and compatibility -semantics. Applications own its interpretation and use. Downstream caching is -one motivation, not a Promptkit feature or a requirement on application behavior. - -This roadmap defines the target capability and the work needed from the current -baseline. The [implementation plan](implementation.md) owns staged execution. - -## Current Baseline - -At v0.23.3, [PreparedRun](../../types.go) exposes the effective messages and -`RenderedPromptHash`. The existing hash includes session identity and message -cache-control metadata as well as roles and text; it does not provide a -content-only equality value. Its implementation is in the -[runner](../../internal/usecase/runner.go), with a summary in the -[internal runner document](../internal/runner.md). - -`Prepare` returns public prepared details. `PrepareExecution` retains a snapshot -whose [Details method](../../prepared_execution.go) returns a caller-owned copy -before execution. These existing preparation and execution boundaries are -sufficient for this addition. - -## Target Public Surface - -Add `MessageContentHash string` to `PreparedRun`, serialized as -`message_content_hash`. Successful preparation populates it alongside the -existing hash. Both `Prepare` and `PreparedExecution.Details()` expose the same -value for equivalent effective messages under the selected hash scheme. - -Compute the value from the captured preparation snapshot, not when a caller -reads `Details`. Reading it does not consume the execution handle or alter its -retained state. Preserve existing caller-owned copying and handle lifecycle -semantics. Any other existing API that projects that same prepared snapshot, -including `OnPrepared`, carries the new field through the shared conversion. - -The addition does not require a standalone public hashing function, request -option, observer event, or new execution protocol. Keep `RenderedPromptHash` -and its existing semantics intact. - -## Hash Inputs And Meaning - -Hash the complete effective message sequence in its prepared order: captured -conversation history, rendered definition messages, then appended messages. -Include each message's prepared role and exact content, with an unambiguous -representation of message count, order, and field boundaries. - -Apply no additional whitespace trimming, Unicode normalization, or semantic -equivalence. The inputs are the already-prepared role/text values, not raw -configuration or caller messages before preparation normalizes them. - -Exclude session and run identities, timestamps, cache-control hints, model and -backend settings, profile and prompt IDs, credentials, input hashes and paths, -and output-contract or schema metadata outside the messages. Text rendered from -any of these sources participates when it actually appears in a message. - -Identical ordered role/text sequences produce identical values across requests, -engines, and processes using the same hash scheme. Changes to roles, text, -ordering, count, or boundaries change the encoded hash input; hash equality is -subject to ordinary collision properties. Changes confined to excluded metadata -do not affect the value. The hash identifies message content, not an execution -or application outcome. - -Return an opaque, filename-safe string. Use SHA-256 over a versioned, -unambiguous encoding; the precise encoding is an internal implementation choice -subject to the compatibility contract below. Stream the encoding into the -hasher in linear time in the message count and role/text bytes, without building -another complete serialized message buffer. Existing framing helpers can be -reused where suitable without changing the existing hash. - -## Hash-Value Compatibility Across Releases - -Preserve hash values for identical ordered role/text sequences across Promptkit -releases while the hash scheme is unchanged. Internal refactoring and unrelated -feature changes preserve those values. - -A deliberate change to the algorithm, encoding, or included fields is a hash -scheme change. Treat it as a compatibility change, document its effect on hash -values, and apply the normal [release policy](../policy/release.md). This does -not promise one scheme forever or make its encoding a public implementation -specification. Consumers continue to treat the string as opaque; no selectable -schemes or legacy-computation API is needed for this feature. - -## Work Needed - -- Add the value to the internal and public prepared representations and carry - it through snapshot construction, copying, and public conversion. -- Compute it in the shared preparation pipeline after the effective message - sequence is assembled. Ordinary execution, split preparation, and prepared - notification should obtain the value from that same preparation result. -- Add focused behavioral coverage for deterministic equality, role/text and - boundary sensitivity, excluded-metadata invariance, rendered inputs, - appended messages, and captured history. Verify availability before model - execution and preservation of the existing rendered hash and snapshot - ownership behavior. Protect the selected scheme's cross-release stability - with proportionate reference-vector coverage. -- Add exact semantics and compatibility language to public GoDoc when - implemented. Update the consumer guide with a short example of reading the - field after preparation, independent of any application use. Update the - internal runner description and relevant serialization tests. -- Describe the additive prepared-result field in the next feature release's - notes under the existing [release policy](../policy/release.md). No - configuration-format change is needed. - -This work does not add storage, response reuse, domain validation, application -acceptance rules, or application-specific hash consumers. Preparation retains -its existing validation and resource-loading behavior. diff --git a/docs/roadmap/notarius-request.md b/docs/roadmap/notarius-request.md deleted file mode 100644 index 0707141..0000000 --- a/docs/roadmap/notarius-request.md +++ /dev/null @@ -1,143 +0,0 @@ -# Notarius Request: Message-Content Hashing - -**Status:** Requested by the Notarius consumer; proposed capability, not an -implemented API or an approved PromptKit implementation plan. - -**Reviewed baseline:** PromptKit v0.23.3. - -## Requested Capability - -Expose a deterministic hash of a prepared request's ordered message roles and -exact text, available before model execution. `MessageContentHash` is the -suggested name. This would let consumers reuse an accepted answer to the same -message content without implementing their own message-hashing policy. - -A field on `PreparedRun`, available through ordinary preparation and -`PreparedExecution.Details()`, appears sufficient. The exact public API is for -PromptKit to select; Notarius needs the value at the preparation boundary, not -only after execution or through an execution observer. - -## Consumer Need - -Notarius runs pipelines whose stages depend on artifacts from earlier stages. -If a later stage fails, an operator should be able to fix the problem and rerun -without paying for successful earlier model work again. Notarius is replacing -its checkpoint machinery with a small accepted-artifact cache. - -The cached value is **an accepted answer to the original task**. Notarius will -prepare the initial request, obtain its message-content hash, and look for -`cache/.json`. A hit loads the accepted domain artifact directly. A miss -executes the full application operation, including domain interpretation, -validation, and any application-directed correction passes. Only the final -accepted artifact is stored, under the original request's hash. - -This is intentionally a convenient reuse policy, not proof that every execution -setting, validation policy, or application implementation is identical. A change -to model settings alone should not invalidate an otherwise reusable answer. -Operators can bypass or refresh the application cache when they want fresh work. - -## Hash Semantics - -Hash the complete effective message sequence exposed by native preparation: -rendered definition messages, appended messages, and captured conversation -history when present. Include each message's role and exact content, preserving -message order and boundaries. - -| Included | Excluded | -| --- | --- | -| Ordered message roles and exact text | Session, run, and parent-run identity; timestamps and timings | -| Message boundaries and sequence length | Provider cache-control hints and other non-content message metadata | -| Any input, instruction, or schema text actually rendered into messages | Model, backend, profile, credentials, generation settings, timeouts, budgets, and retry/repair settings | -| | Prompt ID, definition hash, input hashes/paths, and schema/output-contract metadata outside message text | - -No whitespace trimming, Unicode normalization, or semantic equivalence is -requested: hash the exact prepared text. Different roles, ordering, boundaries, -or text should produce different hashes, subject to ordinary hash collision -properties. Identical role/text sequences should produce the same value across -requests, engines, and process invocations regardless of excluded metadata. -For example, changing a prompt ID while retaining identical messages should -preserve the hash. Changing settings that also changes rendered text should -change it because the text changed. - -Return an opaque, filename-safe value suitable for a persisted cache key. -PromptKit should own and document the hashing semantics and compatibility -behavior. A deterministic, unambiguous encoding and a standard hash such as -SHA-256 would suffice; the precise encoding is an upstream implementation choice. -No general fingerprint configuration or selectable hashing-policy framework is -needed for this request. - -## Existing APIs And The Gap - -[`PreparedRun`](../../types.go) already exposes messages and -`RenderedPromptHash`. The latter includes session identity and message cache -hints, as described in the [runner documentation](../internal/runner.md). -Hashing the entire prepared value would additionally capture settings and -volatile preparation metadata. Neither supplies the requested content-only key. - -Keep the existing hash's semantics intact and add the narrower capability. -Notarius could manually project and hash message fields, but that would preserve -the duplicated library mechanics this migration aims to remove. - -[Native split preparation and execution](../consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later) -already provides the needed execution boundary. Preparation makes no model call -and reserves no backend capacity. On a miss, `RunPrepared` can execute the retained -snapshot. On a hit, the application can discard the handle. No new execution -protocol is requested. - -## Intended Application Flow - -The following is conceptual pseudocode, not an existing hash API: - -```text -prepared = engine.PrepareExecution(initialRequest) -initialKey = prepared.Details().MessageContentHash // proposed addition - -if cache contains a usable accepted artifact for initialKey: - discard prepared handle - return cached artifact - -result = engine.RunPrepared(prepared) -artifact = interpret, validate, and correct through the Notarius operation -if the operation produces a final accepted artifact: - save artifact to cache under initialKey -return application outcome -``` - -Notarius retains the initial key even when acceptance requires several correction -turns. It stores the final domain artifact, not a raw intermediate response, -`RunResult`, exchange, or prepared handle. Failed, rejected, and -validation-incomplete work is not saved. A hit requires ordinary decoding and -usability checks; it does not replay model work or claim fresh domain validation. - -## Ownership And Scope - -PromptKit owns everything within a native request execution, including structural -response validation, transport retries, and internal repair attempts. Notarius -consumes the public response or terminal failure and then owns domain acceptance. -Those internal attempts are not inputs to the cache's eligibility policy. - -This request is solely for reusable message-content hashing. It does not request -PromptKit-owned cache storage, a cache-hit execution result, cached-response -injection, validation replay, exchange restoration, or a write-through response -cache. Debug observations do not change this boundary. Notarius owns cache -lookup, final acceptance, persistence, and reuse/bypass/refresh controls. - -## Suggested Verification - -- Repeated preparation in separate engines produces the same hash for identical - role/text sequences despite different preparation timing or session identity. -- Changing a role, text (including whitespace), order, message count, or boundary - changes the hash; ambiguous concatenations do not collapse to the same input. -- Settings-only, prompt-ID-only, and cache-hint-only changes preserve the hash - when the effective role/text sequence is unchanged. -- Rendered inputs, appended messages, and captured history participate in the - hash when they contribute to the effective messages. -- The hash is available before model execution, and reading it does not consume - the prepared handle or alter its retained snapshot. -- Existing `RenderedPromptHash` behavior remains intact. Document the new value - in public GoDoc and add a short consumer example of lookup before execution. - -Notarius will verify its own cache behavior separately: hits make no model calls, -only accepted artifacts are saved, and corrected answers remain keyed to the -initial task. The supplying PromptKit release will be selected during Notarius -roadmap incorporation; this request does not assign a release number.