Plan appended request messages

2026-08-26 01:53:22 +00:00
parent 2e76003fd5
commit 8745d256bd
2 changed files with 870 additions and 0 deletions
--- a/docs/roadmap/appended-request-messages.md
+++ b/docs/roadmap/appended-request-messages.md
@@ -0,0 +1,434 @@
 # Appended Request Messages
 ## Status
 Accepted for future implementation. This roadmap records a concrete downstream
 requirement from Notarius and the Promptkit-native design selected for it. It
 describes future behavior only; the current public API and canonical
 documentation remain authoritative until the feature is implemented.
 ## Purpose
 Allow a Go consumer to execute an ordinary configured prompt with an explicit
 sequence of already-rendered chat messages appended after the
 prompt-definition messages. The capability should preserve Promptkit's normal
 preparation, inspection, hashing, validation, structured-output repair,
 capacity, cancellation, credential, error, and debug boundaries.
 The immediate consumer is Notarius, which needs to resubmit the same configured
 prompt after semantic validation rejects an LLM-produced stage candidate, then
 append:
 1. an `assistant` message containing the exact defective response; and
 2. a `user` message containing bounded application-owned feedback and a request
   for one corrected replacement response.
 Promptkit should provide the safe message-composition mechanism. Notarius
 should continue to own semantic validation, stage retry budgets, correction
 wording, terminal policy, and the decision to start another completion.
 ## Consumer Need
 Promptkit v0.8 can perform bounded structural repair inside one `Run` or
 `RunPrepared` operation. Its repairer correctly starts from the complete
 original rendered prompt, appends the latest candidate as an assistant turn,
 and adds a corrective user turn. That loop is intentionally driven by
 Promptkit's own structural validators.
 Notarius has a distinct outer workflow:
 1. Promptkit returns a structurally valid response.
 2. Notarius materializes a typed stage candidate.
 3. One or more application validators evaluate domain semantics.
 4. If the candidate is rejected and a Notarius stage attempt remains, Notarius
   makes a new Promptkit request with application-owned corrective turns
   appended after the ordinary configured prompt.
 Each `Run`, `Prepare`, or `PrepareExecution` remains an independent operation
 that resolves current sources and inputs. Promptkit does not retain or reuse a
 base prompt across calls, so the ordinary prefix is identical across attempts
 only when the selected definitions, inputs, variables, and other preparation
 inputs remain identical. A consumer that requires a pre-execution equality
 check can prepare each attempt and compare its opaque rendered-prompt hash
 before calling `RunPrepared`. A reusable prepared conversation or frozen
 cross-run prompt is outside this feature.
 Today a `RunRequest` can select and parameterize a prompt definition, but it
 cannot append downstream-owned messages. Registering a second prompt manifest
 for every correction-capable operation would duplicate prompt identity,
 profile defaults, input declarations, output contracts, and message assets.
 Calling an injected model client directly would bypass Promptkit preparation,
 capacity, structured output, repair, error adaptation, and provenance. Neither
 workaround is a suitable long-term consumer contract.
 ## Target Public API
 Add an append-only field to `RunRequest`:
 ```go
 type RunRequest struct {
 	// Existing fields omitted.
 	AppendedMessages []RenderedMessage
 }
 ```
 Retain `RenderedMessage.Role` as a string for source compatibility and expose
 the untyped string constants `RoleDeveloper`, `RoleSystem`, `RoleUser`, and
 `RoleAssistant`. Their values are `developer`, `system`, `user`, and
 `assistant`, respectively. The supported values and their semantics are part
 of the public contract.
 `AppendedMessages` contains fully rendered message values. Promptkit normalizes
 and validates their roles as described below, but does not interpret their
 content as Go templates, resolve input references inside it, or load it from
 files. Nil and empty slices are equivalent.
 Reusing `RenderedMessage` is preferred because it is already Promptkit's
 public, stable representation of a provider-bound chat message and includes
 the supported cache-control metadata. Its GoDoc should be generalized from a
 definition-supplied message to an already-rendered chat message. A new
 request-only message type would duplicate the same role, content, and cache
 control contract without adding useful type safety.
 The exact exported declarations and semantics, once implemented, belong in
 GoDoc. The name `AppendedMessages` is intentional: placement is part of the
 contract, whereas names such as `AdditionalMessages` leave ordering
 ambiguous.
 ## Composition Semantics
 For `Prepare`, `PrepareExecution`, and `Run`, Promptkit should:
 1. perform ordinary prompt selection, input loading, variable processing, and
   prompt rendering;
 2. preserve every ordinary rendered message after its role has passed the
   shared canonical normalization, including content, cache-control value,
   order, and bytes, without rerendering or rewriting content;
 3. normalize and validate request-supplied roles, then defensively copy and
   append `RunRequest.AppendedMessages` in caller-supplied order; and
 4. treat the combined sequence as the effective rendered prompt for every
   later preparation and execution boundary.
 The request-supplied messages always follow all prompt-definition messages.
 They cannot prepend, insert, replace, or delete definition messages. This
 narrow contract protects a stable prompt prefix for provider-side prompt
 caching and keeps prompt definitions as the canonical owner of ordinary
 prompt content.
 Promptkit must not synthesize separators or normalize message content. Content,
 including empty or whitespace-only content, remains byte-for-byte caller data;
 whether a provider accepts it is provider policy. The effective session ID,
 selected profile, model settings, output contract, structured-output
 constraint, and input hashes retain their ordinary resolution rules.
 `InspectPrompt` remains definition inspection and therefore does not accept or
 report request-supplied appended messages.
 ## Preparation, Hashing, And Inspection
 Appended messages must participate in the same immutable preparation boundary
 as ordinary rendered messages:
 - `PreparedRun.Messages` contains the complete combined sequence;
 - `PreparedExecution.Details().Messages` contains an independent defensive
  copy of that sequence;
 - `Run` and `RunPrepared` send exactly that sequence to the selected model
  client;
 - `RenderedPromptHash` covers the effective session ID and complete combined
  sequence, including appended-message cache-control metadata;
 - `PromptHash` continues to identify only the selected prompt definition and
  therefore does not change merely because appended messages differ; and
 - input hashes continue to describe only named artifacts.
 Two requests with identical prompt selection, inputs, variables, session, and
 appended messages should prepare equivalent message sequences and rendered
 prompt hashes. Changing any normalized appended role, content, order, or
 cache-control value should change the rendered prompt hash. The hash encoding
 must frame sessions, messages, and fields unambiguously rather than relying on
 separators that could also occur in caller-controlled content.
 Prepared execution must freeze appended messages at preparation time. Caller
 mutation after `PrepareExecution` returns must not affect its details or later
 execution, and mutation of a value returned by `Details` must not affect the
 opaque executable snapshot.
 ## Interaction With Structured-Output Repair
 The combined message sequence is the complete original prompt for one
 Promptkit operation. If Promptkit's structural validator rejects a generated
 candidate and a repair attempt is available, the existing repair workflow
 should begin from a fresh copy of that combined sequence and then append its
 own candidate and correction messages.
 For the Notarius use case, the ordering may therefore be:
 1. ordinary configured prompt messages;
 2. Notarius's previous assistant response;
 3. Notarius's semantic-correction user message;
 4. Promptkit's latest structurally defective assistant response, if any; and
 5. Promptkit's structural-repair user message, if needed.
 This is not a recursive retry policy. Promptkit still owns only its configured
 structural-repair budget within the current operation. It does not interpret
 the purpose of consumer-supplied messages, invoke application validators, or
 start a later Notarius stage attempt.
 All initial and repair generation calls must continue through the ordinary
 prepared-execution, capacity, client, cancellation, credential, structured
 output, usage, and error-adaptation paths.
 ## Message Validation And Provider Boundaries
 Promptkit should validate appended messages before publishing a prepared value
 or starting model generation. Validation should be shared by `Prepare`,
 `PrepareExecution`, and `Run` and should reject:
 - invalid UTF-8 in role or content;
 - a role that, after normalization, is not `developer`, `system`, `user`, or
  `assistant`; or
 - invalid cache-control metadata.
 Role normalization is intentionally limited to trimming surrounding whitespace
 and converting the result to lowercase. Promptkit performs no aliasing or
 semantic role translation. In particular, it must not translate `developer`
 to `system` or the reverse. `system` is generally the more portable choice
 across heterogeneous OpenAI-compatible backends; `developer` remains supported
 for providers and models that implement the newer instruction hierarchy. A
 backend or model that rejects an otherwise supported role reports that through
 the ordinary provider error path.
 The same four-role vocabulary and normalization apply to prompt-definition
 messages. This deliberately tightens the existing prompt format, which
 currently accepts any nonblank role. `tool` is not supported because
 Promptkit's text-only message value cannot express its required `tool_call_id`;
 the deprecated `function` role likewise requires a name that the value cannot
 represent. Those roles should be considered only as part of a deliberate
 future tool-call message design.
 Promptkit does not impose appended-message count, per-message size, aggregate
 size, estimated-token, or context-window limits, and it must not truncate
 caller content. Those limits are backend-, model-, and application-specific.
 Provider rejection for context length, role support, empty content, or any
 other provider rule should cross the existing generation-error boundary with
 its supported status and provider-detail information intact.
 Invalid appended messages are invalid requests and should preserve the public
 `ErrInvalidRequest` identity. Error text should identify the failing appended
 message index and violated property without echoing its role or content. An
 invalid role in a prompt definition remains a source-owned prompt-definition
 failure through its ordinary public error category.
 ## Ownership, Privacy, And Formatting
 `RunRequest.AppendedMessages` is caller-owned. Promptkit must defensively copy
 the slice and nested cache-control values before retaining them. Internal
 domain conversion, prepared-run cloning, public result conversion, and
 prepared-execution details must preserve the same non-aliasing guarantees as
 ordinary rendered messages.
 Appended content can contain model output, source excerpts, or application
 validation feedback and must be treated as sensitive prompt content:
 - `RunRequest.String` and `RunRequest.GoString` should report only an appended
  message count, not roles or content;
 - errors must not reproduce appended content;
 - no credential or secret value may be inferred, copied, or added to hashes
  beyond the existing prompt-content behavior;
 - prepared values continue to expose complete rendered messages by design and
  remain subject to the consumer's data-handling policy; and
 - Promptkit should not add logging, persistence, or a durable retry transcript
  for appended messages.
 The field does not need a stable `RunRequest` JSON contract because
 `RunRequest` has none today. The stable JSON representation of prepared values
 must continue to include the final `Messages` sequence through its existing
 contract.
 ## Application-Neutral Boundary
 This feature is a generic prompt-composition primitive. Promptkit should not
 add concepts such as semantic validator, rejection, stage, pipeline,
 correction reason code, or terminal validation policy to its public API.
 The following remain downstream responsibilities:
 - deciding whether a new request is warranted;
 - choosing and enforcing the outer retry budget;
 - retaining the exact earlier candidate;
 - constructing, bounding, and redacting application feedback;
 - determining which message roles and sequence express that workflow;
 - keeping preparation inputs stable when an identical prefix is required, or
  checking prepared hashes before execution;
 - aggregating semantic validation findings;
 - recording application provenance; and
 - deciding whether exhausted retries fail, reject, warn, or continue.
 Promptkit owns only safe composition and execution of the effective request.
 ## Documentation End State
 In the target state, the capability is described by the following canonical
 owners:
 - root GoDoc for `RunRequest`, `RenderedMessage`, `PreparedRun`, and relevant
  engine operations, including the role constants and normalization contract;
 - the [framework format reference](../formats.md) for the tightened role
  vocabulary and normalization shared by prompt-definition messages;
 - the [Go consumer guide](../consumers/pkg-promptkit.md) with one concise
  appended-message example and a warning that values are already rendered and
  potentially sensitive;
 - the [internal runner document](../internal/runner.md) for validation,
  composition, hashing, preparation, and structural-repair interaction;
 - the [internal source document](../internal/sources.md) for
  prompt-definition role normalization and source-owned failures;
 - the [internal model-client document](../internal/llm.md) only if its
  effective-message input contract needs clarification;
 - the [architecture policy](../policy/architecture.md) only if implementation
  establishes a durable request-composition invariant not already covered by
  the public-facade and application-neutral-boundary rules;
 - the
  [OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md)
  for supported provider-bound roles and provider rejection; and
 - release guidance for the new minor version, including migration guidance for
  consumers with nonstandard prompt-definition roles.
 The Go-only appended-message field does not belong in `docs/formats.md`.
 However, the prompt-definition role contract does change and must be updated in
 that canonical format reference. Profile and schema formats do not change.
 ## Verification Expectations
 Tests protect the public composition and ownership contract rather than
 private helper layout or exact diagnostic prose. Focused coverage establishes:
 - nil and empty appended slices preserve current behavior;
 - ordinary rendered messages remain an exact prefix and appended messages
  retain caller order;
 - supported roles in both sources normalize by trimming and lowercasing, while
  unsupported, `tool`, and `function` roles are rejected at the owning
  boundary;
 - message content, including empty and whitespace-only content, is preserved
  exactly and receives no Promptkit-owned count or size restriction;
 - `Prepare`, `PrepareExecution`, and `Run` agree on the combined sequence and
  rendered prompt hash;
 - `RunPrepared` uses its frozen appended-message snapshot after caller
  mutation;
 - `Details` and public conversions return independent copies;
 - changing a normalized appended role, content, order, or cache control changes
  the
  rendered prompt hash without changing `PromptHash`;
 - invalid appended roles, UTF-8, and cache controls fail as invalid requests
  before model generation without echoing sensitive content;
 - provider rejection of an otherwise structurally valid combined sequence
  retains the ordinary generation-error contract;
 - the initial model call receives the combined sequence;
 - structural repair reconstructs the complete combined sequence before adding
  Promptkit's repair turns;
 - usage, capacity, cancellation, credentials, structured output, and error
  identity retain their existing behavior; and
 - `String` and `GoString` expose only a count for appended messages.
 One external-package contract test and focused internal runner tests should
 own most of this matrix. Extend the maintained offline run example only if it
 materially improves consumer understanding; do not turn the example into a
 retry framework or duplicate the contract tests.
 ## Acceptance Criteria
 - A consumer can append already-rendered messages through `RunRequest` without
  defining a second prompt manifest or bypassing `Engine`.
 - Normalized ordinary prompt messages remain an unchanged, ordered prefix.
 - The final sequence is prepared, hashed, reported in prepared details, frozen,
  executed, and repaired consistently across all engine workflows.
 - Roles are normalized to and validated against the supported four-role
  vocabulary at both prompt-definition and appended-message boundaries.
 - Appended values are structurally validated, defensively copied, and excluded
  from ordinary request formatting and error text without Promptkit-owned
  count, size, token, or context-window limits.
 - Promptkit structural repair treats the combined sequence as its original
  prompt and retains all existing execution safeguards.
 - Promptkit introduces no application-specific retry, validation, pipeline, or
  terminal-policy concepts.
 - Existing consumers that omit appended messages and use supported roles retain
  current behavior; nonstandard prompt-definition roles are intentionally
  rejected under the tightened minor-version contract.
 - Canonical GoDoc and consumer/internal documentation describe the implemented
  contract, and the complete maintainer validation workflow passes.
 ## Non-Goals
 - A Promptkit-owned semantic validation framework.
 - A generic outer retry orchestrator.
 - Automatic interpretation or formatting of downstream validation errors.
 - Prepending, inserting, replacing, or deleting configured prompt messages.
 - Treating appended content as a template or file reference.
 - Accumulating conversation state across `Run` calls.
 - Freezing or reusing a base prompt across independent operations.
 - A durable session or retry transcript store.
 - PromptKit-owned message-count, byte-size, token, or context-window limits.
 - Tool calls, tool results, deprecated function messages, or arbitrary custom
  message roles.
 - Provider-specific branching for Notarius.
 - Changes to profile or schema file formats.
 ## Alternatives Considered
 ### Paired Initial And Correction Prompt Definitions
 A downstream application could register a second prompt definition that
 duplicates the initial manifest and adds candidate and correction inputs. This
 requires no Promptkit API change, but it duplicates prompt metadata and creates
 drift across prompt versions, profile defaults, output contracts, assets, and
 hashes. It also makes a generic request-composition need module-specific.
 ### A Purpose-Specific Repair Request
 Promptkit could expose an API accepting `PreviousOutput` and
 `ValidationErrors`, mirroring its internal structural repair request. This
 would initially fit Notarius, but it would make Promptkit interpret downstream
 policy and would conflate application semantic retries with Promptkit
 structural repair. A generic append-only message contract is smaller and more
 reusable.
 ### Direct Execution Of A Mutated Prepared Prompt
 Promptkit could expose mutable prepared messages or accept a complete caller-
 constructed prompt for execution. That would weaken definition ownership,
 prepared-execution immutability, hashing, validation, and provenance, and would
 make it easy to bypass source and preparation guarantees. The requested API
 should compose messages before immutable preparation instead.
 ### Direct Use Of The Model Client
 Notarius could invoke an injected Promptkit-compatible model client directly.
 That bypasses the engine's profile resolution, backend admission, structured
 output, repair, error mapping, and prepared-execution contracts. It is not an
 acceptable downstream integration path.
 ### A Semantic-Validation Callback
 Promptkit could accept an application callback that validates each candidate
 and drives another generation inside one `Run`. That could retain one frozen
 prompt and aggregate usage automatically, but it would make Promptkit own
 callback lifecycle, semantic diagnostics, application retry policy, and the
 boundary between typed domain values and raw model output. Keeping semantic
 validation and its outer loop downstream leaves the public API smaller and the
 library application-neutral.
 ### A Reusable Prepared Conversation
 Promptkit could expose a reusable handle whose frozen base prompt accepts new
 turns across multiple executions. That would guarantee a byte-identical prefix
 and could support longer-lived chatbot workflows, but it would require a much
 larger lifecycle, credential-retention, concurrency, source-freshness, and
 history-ownership contract. The stateless append-only request primitive meets
 the demonstrated need without precommitting to those policies.
 ### Complete Caller-Supplied Or Mutable Message Sequences
 Promptkit could let a consumer replace, insert, or delete configured messages,
 or execute an entirely caller-constructed sequence. That would weaken prompt
 definition ownership, hashing, provenance, and the prepared-execution
 boundary. Append-only composition provides the needed flexibility while
 preserving configured messages as the canonical prefix.
--- a/docs/roadmap/implementation.md
+++ b/docs/roadmap/implementation.md
@@ -0,0 +1,436 @@
 # Appended Request Messages Implementation Plan
 ## Purpose
 This document is the ordered implementation plan for the target state in
 [appended-request-messages.md](appended-request-messages.md). It is written for
 a coding agent implementing one stage per prompt, in order. The feature
 roadmap owns intent, policy, acceptance criteria, and non-goals; this document
 owns sequencing and concrete implementation decisions.
 The work adds one stateless, append-only request-composition primitive and
 tightens prompt-definition roles to the same four-role vocabulary. It does not
 add a conversation object, application validation callback, retry policy,
 message limit, context estimator, or tool-call representation.
 ## Cross-Stage Constraints
 Apply these constraints throughout every stage:
 - Preserve the architecture in
  [docs/policy/architecture.md](../policy/architecture.md): the root package is
  the public facade, `internal/domain` owns source-neutral message invariants,
  source packages own source-specific error classification, and
  `internal/usecase` owns composition and execution orchestration.
 - Keep `RenderedMessage.Role` a string. Do not introduce a new public message
  type, enum type, builder, conversation handle, callback, or alternate run
  method.
 - Support exactly `developer`, `system`, `user`, and `assistant`. Normalize a
  role only by trimming surrounding Unicode whitespace with
  `strings.TrimSpace` and lowercasing with `strings.ToLower`; do not alias one
  role to another.
 - Preserve message content exactly after confirming that request-supplied
  content is valid UTF-8. Empty and whitespace-only appended content is valid
  at the Promptkit boundary. Do not trim, template, load, truncate, estimate,
  or otherwise transform it.
 - Do not add message-count, byte-size, token, context-window, or retry limits.
  An upstream rejection of a structurally valid request must continue through
  the existing generation-error boundary.
 - Never reproduce appended role text or content in errors, formatting, or new
  logging. It is acceptable to identify an invalid message by zero-based index
  and property name.
 - Defensively copy every retained or returned message slice and every nested
  `CacheControl` pointer. Prepared execution must remain an immutable snapshot.
 - Keep `PromptHash` definition-only. Treat `RenderedPromptHash` as an opaque
  equality value covering the effective session and complete effective
  message sequence.
 - Follow [docs/policy/testing.md](../policy/testing.md): put exhaustive
  invariant matrices at their narrow owner, use external-package tests for the
  public contract, avoid repeating the same matrix at every layer, and keep
  all tests deterministic and offline.
 - Do not create a release tag, push a release, or retire either roadmap during
  implementation. Release preparation follows acceptance and review of the
  completed feature.
 ## Stage 1: Centralize Message Invariants And Tighten Prompt Roles
 ### Objective
 Establish one source-neutral owner for chat-message role and cache-control
 rules, then make prompt-definition loading and rendering publish only canonical
 supported roles. This stage intentionally does not add the public request
 field.
 ### Implementation
 1. Add an `internal/domain` message-invariant file and define the untyped
   string constants `RoleDeveloper`, `RoleSystem`, `RoleUser`, and
   `RoleAssistant` with the exact values in the feature roadmap.
 2. Add a role normalizer in `internal/domain` with the signature and behavior:
   ```go
   func NormalizeMessageRole(role string) (string, error)
   ```
   It must reject invalid UTF-8, trim surrounding whitespace, lowercase the
   result, accept exactly the four constants, and reject blank or unsupported
   values. Its errors must describe the violated role rule without embedding
   the supplied value.
 3. Move the source-neutral cache-control normalization currently owned by
   `internal/promptdef` into `internal/domain`:
   ```go
   func NormalizeCacheControl(control *CacheControl) (*CacheControl, error)
   ```
   Preserve the existing contract: nil remains nil; type and TTL are trimmed;
   type must be `ephemeral`; TTL must be empty or `1h`; the returned pointer is
   newly allocated. Reject invalid UTF-8 before normalization and do not echo
   invalid values. Keep the existing cache-control constants as the canonical
   values.
 4. Add domain-owned cloning helpers with these signatures:
   ```go
   func CloneRenderedMessages(messages []RenderedMessage) []RenderedMessage
   func ConcatRenderedMessages(prefix, suffix []RenderedMessage) []RenderedMessage
   ```
   The first returns a deep clone. The second allocates one result in prefix-
   then-suffix order and deep-copies both inputs. Both must clone nested
   cache-control pointers and preserve role, content, order, nil cache-control,
   empty content, and whitespace exactly. Do not normalize or validate inside
   a clone helper.
 5. Refactor prompt-definition normalization in
   `internal/promptdef/filesystem_repository.go` to call the domain role and
   cache-control normalizers. Convert the source-only `cacheControlFile` into a
   temporary `domain.CacheControl` before normalization; do not move YAML
   source types into the domain package. Store the normalized role. Preserve
   the existing prompt-definition error category, selected-file context, and
   message index, while ensuring diagnostics do not reproduce an invalid role
   value.
 6. Make `internal/prompt/go_renderer.go` defensively call the same role
   normalizer before publishing each rendered message. This protects the
   provider-bound invariant even for an internal repository implementation
   that did not originate in `internal/promptdef`. Retain the renderer's
   existing error identity and message index. Continue to render content and
   session templates exactly as before.
 7. Replace any literal `"assistant"` and `"user"` roles in
   `internal/usecase/repairer.go` with the domain constants. Do not otherwise
   change repair behavior in this stage.
 ### Tests
 1. Add a table at the domain owner covering:
   - all four accepted roles;
   - surrounding whitespace and mixed-case normalization;
   - blank, `tool`, `function`, and an arbitrary custom role;
   - invalid UTF-8; and
   - cache-control normalization, invalid values, and defensive-copy behavior.
 2. Add focused prompt-definition repository coverage proving that a
   mixed-case/whitespace role is published canonically and that an unsupported
   role is an invalid prompt definition with its existing public-facing error
   category. Do not repeat the entire domain role table.
 3. Add focused renderer coverage proving defensive canonicalization and
   rejection for a directly supplied internal definition. Preserve all
   existing rendering and cancellation tests.
 4. Run the tests for `internal/domain`, `internal/promptdef`,
   `internal/prompt`, and `internal/usecase`, then run `go test ./...` before
   completing the stage.
 ### Completion Criteria
 - Every definition-supplied role reaching a rendered prompt is one of the
  canonical constants.
 - Prompt-definition and renderer failures retain their owning error identities.
 - Cache-control rules and message deep-copy behavior have one source-neutral
  implementation.
 - Existing supported prompt definitions and structural repair behavior remain
  unchanged.
 ## Stage 2: Add The Public Request Contract And Boundary Validation
 ### Objective
 Expose the smallest public API for appended messages and convert it into a
 validated, caller-independent domain request before any source or model work.
 The runner may still ignore the new domain field until Stage 3.
 ### Implementation
 1. Add `AppendedMessages []RenderedMessage` to both the public and domain
   `RunRequest` values. Public GoDoc must state that values are already
   rendered, are appended after all definition messages, are not templated or
   file-resolved, preserve content exactly, and treat nil and empty slices as
   equivalent.
 2. Add a small root-package file, rather than importing `internal/domain` into
   the general public types file solely for constants, and publish untyped
   aliases to the domain constants:
   ```go
   const (
       RoleDeveloper = domain.RoleDeveloper
       RoleSystem    = domain.RoleSystem
       RoleUser      = domain.RoleUser
       RoleAssistant = domain.RoleAssistant
   )
   ```
   Give every exported declaration exact GoDoc. Do not introduce a named role
   type; assignments and comparisons with ordinary strings must remain
   frictionless.
 3. Generalize `RenderedMessage` GoDoc so it describes any prepared,
   provider-bound text chat message rather than only a definition-supplied
   message. Document the four supported roles and refer callers to the
   constants.
 4. Add one conversion helper in `convert.go` that processes appended messages
   in order. For each message it must:
   - reject invalid UTF-8 in content without reproducing it;
   - normalize and validate the role through
     `domain.NormalizeMessageRole`;
   - normalize and defensively copy cache control through
     `domain.NormalizeCacheControl`; and
   - return a newly allocated domain slice with independently allocated nested
     cache-control pointers.
   Wrap failures with the zero-based appended-message index and property name,
   but not the failing value.
 5. Call that helper from `toDomainRunRequest` and store the result on the
   domain request. Because `Prepare`, `PrepareExecution`, and `Run` already use
   this shared conversion boundary, conversion failures must emerge as the
   existing public `ErrInvalidRequest` before source resolution, preparation,
   capacity admission, or model generation.
 6. Extend `RunRequest.String` and `RunRequest.GoString` through their shared
   redacted formatter to include only `AppendedMessages:<count>`. Do not emit
   roles, content, cache-control values, or a serialized message value.
 ### Tests
 1. Add an external-package public contract test for the exact constant values
   and for ordinary string compatibility.
 2. Put the request-message validation matrix at the public conversion owner.
   Cover supported normalization, invalid UTF-8 in role and content,
   unsupported roles, invalid cache control, empty content, whitespace-only
   content, and message-indexed errors that omit the original role and content.
 3. Add a small parity test proving that `Prepare`, `PrepareExecution`, and
   `Run` all classify an invalid appended message as `ErrInvalidRequest` and do
   not invoke configured source/model fakes. Do not repeat the complete
   invalid-value matrix through all three operations.
 4. Extend the existing request-formatting test so both `String` and `GoString`
   expose the count and cannot contain distinctive role/content values.
 5. Run root-package tests and `go test ./...` before completing the stage.
 ### Completion Criteria
 - The only new public surface is one `RunRequest` field and four untyped string
  constants.
 - Every public operation rejects malformed appended messages consistently and
  before side effects.
 - Request formatting remains redacted.
 - Public-to-domain conversion owns an independent, canonical snapshot of the
  request values.
 ## Stage 3: Compose, Freeze, Hash, Execute, And Repair The Effective Prompt
 ### Objective
 Make appended messages part of the effective rendered prompt everywhere after
 ordinary rendering, including immutable preparation, client execution,
 structural repair, and opaque rendered-prompt identity.
 ### Implementation
 1. In the shared completion phase in `internal/usecase/runner.go`, append the
   already-normalized domain request messages immediately after ordinary
   rendering and application of the direct-session override. Use the domain
   concatenation helper so the resulting `RenderedPrompt.Messages` owns a deep
   copy of both the ordinary prefix and appended suffix.
 2. Use that combined rendered prompt for all subsequent work: rendered-prompt
   hashing, `PreparedRun.Messages`, `PreparedExecution` details, the initial
   generation request, `RunResult.RenderedPromptHash`, and the original-message
   input to structural repair. Do not alter `PromptHash` or input hashes.
 3. Refactor `clonePreparedRun` and any other prepared-message clone path to use
   the shared domain deep-clone helper. Keep public conversion helpers as the
   public/domain boundary, but remove duplicate internal cache-control pointer
   cloning where the new domain helper is the appropriate owner.
 4. Ensure `PreparedExecution` freezes the combined messages. The opaque
   retained value, every call to `Details`, and a later `RunPrepared` must have
   independent message slices and cache-control pointers. Caller mutation of
   the original request or of returned details must not alter execution.
 5. In the default structural repairer, start every repair request with a deep
   copy of the complete combined original sequence. Append only the latest
   nonempty candidate with `domain.RoleAssistant`, followed by the corrective
   message with `domain.RoleUser`. Preserve the existing empty-candidate rule,
   diagnostic bound, retry budget, usage accounting, capacity path,
   cancellation, target, session, credential, and structured-output behavior.
 6. Replace the delimiter-based `hashRenderedPrompt` encoding with a versioned,
   length-framed binary encoding. Use this exact field order:
   1. the fixed bytes `promptkit/rendered-prompt/v2\x00`;
   2. the effective session ID as a `uint64` big-endian byte length followed by
      its bytes;
   3. the message count as `uint64` big-endian;
   4. for each message in order, role and content as separately length-prefixed
      byte strings;
   5. one cache-control presence byte; and
   6. when present, type and TTL as separately length-prefixed byte strings.
   Hash the resulting framing with SHA-256 and retain the lowercase hexadecimal
   public representation. The terminating NUL is part of the fixed marker and
   must be written before the first length. Use a fixed eight-byte buffer with
   `binary.BigEndian.PutUint64`; do not use `binary.Write`, JSON, `fmt`,
   separators, or an input-sized staging buffer. The returned value is opaque,
   so do not preserve the old digest. This change deliberately prevents caller
   content from colliding with structural separators and streams in constant
   auxiliary space.
 7. Confirm that the built-in OpenAI-compatible client forwards the canonical
   role and exact content without provider-specific translation. No new
   provider role branching or proactive context validation is permitted.
 ### Tests
 1. At the runner owner, prove that ordinary rendered messages are an exact
   prefix, appended messages retain their order and content, and nil/empty
   suffixes retain current behavior.
 2. Prove that `Prepare`, `PrepareExecution`, and `Run` agree on the complete
   sequence and rendered-prompt hash, and that the injected model client sees
   exactly that sequence.
 3. Prove immutable ownership by mutating the original appended slice and its
   cache-control pointer after `PrepareExecution`, then mutating a value from
   `Details`; neither mutation may affect later details or `RunPrepared`.
 4. Extend hash tests to establish:
   - normalized-equivalent roles yield the same rendered hash;
   - changing role, content, order, cache-control presence/value, or session
     changes the rendered hash;
   - appended-message changes do not change `PromptHash`; and
   - the one-message sequence `{role: "user", content:
     "x\n---\nassistant\ny"}` hashes differently from the two-message sequence
     `{user, "x"}, {assistant, "y"}`, even though the legacy separator
     encoding could not distinguish them.
   Test observable relationships, not a golden SHA-256 digest.
 5. Extend repair tests to prove the exact order: ordinary prefix, consumer
   suffix, latest structurally invalid assistant candidate when nonempty, and
   Promptkit corrective user message. Also prove that a later repair begins
   again from the complete original combined sequence rather than accumulating
   prior repair turns.
 6. Add one focused OpenAI-compatible request test covering canonical role
   forwarding, including `developer`, and exact content preservation. Do not
   duplicate all HTTP client or role-validation tests.
 7. Exercise one otherwise valid combined request that the fake provider rejects
   and confirm the existing `GenerationError` status/provider-detail contract
   remains intact. Reuse the existing generation-error test scaffolding rather
   than creating a second status matrix.
 8. Run tests for `internal/domain`, `internal/usecase`, `internal/llm`, and the
   root package, then run `go test ./...` and `go test -race ./...` before
   completing the stage.
 ### Completion Criteria
 - Every prepared, inspected-prepared, executed, and repaired message sequence
  uses the same ordered combined prompt.
 - Prepared execution remains immutable across every caller-visible ownership
  boundary.
 - Rendered-prompt hashing is unambiguous, streaming, and sensitive to every
  contracted field while `PromptHash` remains definition-only.
 - Existing repair, usage, capacity, cancellation, credential, validation, and
  provider-error behavior is preserved.
 ## Stage 4: Publish Canonical Documentation And Complete Validation
 ### Objective
 Bring every canonical documentation owner into line with the implemented API,
 record the intentional prompt-format compatibility change, and run the full
 repository validation workflow.
 ### Documentation
 1. Finish the exact root-package GoDoc for `RunRequest.AppendedMessages`, the
   four role constants, `RenderedMessage`, `PreparedRun.Messages`, and the
   relevant `Engine` methods. Document normalization, ordering, ownership,
   empty-content behavior, error identity, and statelessness at the narrowest
   declaration that owns each rule.
 2. Update [docs/formats.md](../formats.md) as the canonical prompt-file
   contract:
   - enumerate the four accepted message roles;
   - state trim-and-lowercase normalization;
   - state that blank, custom, `tool`, and `function` roles are invalid; and
   - identify this as an intentional tightening from the prior nonblank-string
     rule for consumers migrating to the next minor release.
   Do not document the Go-only `AppendedMessages` field as a file-format field.
 3. Add one concise, application-neutral example to
   [docs/consumers/pkg-promptkit.md](../consumers/pkg-promptkit.md) showing a
   previous assistant response and a corrective user message passed through
   `AppendedMessages`. Explain that the messages are already rendered and may
   contain sensitive data, Promptkit remains stateless, each call re-resolves
   current sources, the consumer owns any semantic retry budget, and
   `PrepareExecution` plus opaque hash comparison is available when a
   pre-execution equality check is required.
 4. Update [docs/internal/sources.md](../internal/sources.md) with shared role
   normalization and source-owned failure behavior. Update
   [docs/internal/runner.md](../internal/runner.md) with append ordering,
   immutable preparation, the versioned framed hash, and structural-repair
   interaction. Update [docs/internal/llm.md](../internal/llm.md) only if needed
   to clarify that the client receives canonical text messages and leaves
   supported-role/context rejection to the provider.
 5. Update the
   [OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md)
   to list the four provider-bound roles, state that Promptkit sends them
   without translation, and explain that a backend/model rejection follows the
   normal provider-error path. Keep tool/function payloads outside the
   supported wire contract.
 6. Update [docs/policy/architecture.md](../policy/architecture.md) only if the
   implementation created a durable architectural invariant not already
   covered by domain ownership, the public facade, and the
   application-neutral boundary. Do not add feature-level API details to
   policy. No new internal package is expected, so do not change the internal
   package overview merely because a file was added.
 7. Do not create a release tag in this stage. Record in the final handoff that
   the next release must be a minor version and that its release note must
   summarize both the new append API and migration from nonstandard
   prompt-definition roles, linking to GoDoc and `docs/formats.md` for detail.
   Create the release document and README link only as part of the repository's
   subsequent release-preparation task.
 ### Final Verification
 1. Review new tests against [docs/policy/testing.md](../policy/testing.md).
   Consolidate repeated setup and assertions only where it clarifies behavior;
   do not replace behavioral tests with helper-implementation tests.
 2. Run the complete maintainer validation workflow from
   [docs/development.md](../development.md), including:
   - all ordinary and race-enabled Go tests;
   - `go vet` and the build;
   - both maintained examples;
   - Go and Markdown formatting/link checks;
   - diff, ignored-file, workspace, vendoring, module-replacement, and
     credential hygiene checks.
 3. Review the final diff to confirm that no provider-specific Notarius concept,
   conversation state, message limit, content logging, new credential handling,
   or unrelated refactor entered the change.
 4. Leave both roadmap documents in place for post-implementation review. Their
   retirement belongs to a later cleanup after the feature is accepted.
 ### Completion Criteria
 - Canonical GoDoc, format, consumer, internal, and integration documentation
  agree with the implemented behavior and do not duplicate ownership.
 - Migration impact for unsupported legacy prompt roles is explicit.
 - The complete maintainer workflow passes from a clean, supported Go setup.
 - The repository is ready for an implementation review followed by minor
  release preparation.
 ## Open Questions
 None. The feature roadmap and this plan resolve the public API shape, role
 vocabulary, normalization, content handling, ownership, composition order,
 hash framing, repair interaction, provider boundary, documentation ownership,
 and release compatibility policy required for implementation.