eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Plan appended request messages

Browse Source
This commit is contained in:
Eric Rakestraw
2026-08-26 01:53:22 +00:00
parent 2e76003fd5
commit 8745d256bd
2 changed files with 870 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

434
docs/roadmap/appended-request-messages.md Normal file
View File

@@ -0,0 +1,434 @@
# Appended Request Messages
## Status
Accepted for future implementation. This roadmap records a concrete downstream
requirement from Notarius and the Promptkit-native design selected for it. It
describes future behavior only; the current public API and canonical
documentation remain authoritative until the feature is implemented.
## Purpose
Allow a Go consumer to execute an ordinary configured prompt with an explicit
sequence of already-rendered chat messages appended after the
prompt-definition messages. The capability should preserve Promptkit's normal
preparation, inspection, hashing, validation, structured-output repair,
capacity, cancellation, credential, error, and debug boundaries.
The immediate consumer is Notarius, which needs to resubmit the same configured
prompt after semantic validation rejects an LLM-produced stage candidate, then
append:
1. an `assistant` message containing the exact defective response; and
2. a `user` message containing bounded application-owned feedback and a request
for one corrected replacement response.
Promptkit should provide the safe message-composition mechanism. Notarius
should continue to own semantic validation, stage retry budgets, correction
wording, terminal policy, and the decision to start another completion.
## Consumer Need
Promptkit v0.8 can perform bounded structural repair inside one `Run` or
`RunPrepared` operation. Its repairer correctly starts from the complete
original rendered prompt, appends the latest candidate as an assistant turn,
and adds a corrective user turn. That loop is intentionally driven by
Promptkit's own structural validators.
Notarius has a distinct outer workflow:
1. Promptkit returns a structurally valid response.
2. Notarius materializes a typed stage candidate.
3. One or more application validators evaluate domain semantics.
4. If the candidate is rejected and a Notarius stage attempt remains, Notarius
makes a new Promptkit request with application-owned corrective turns
appended after the ordinary configured prompt.
Each `Run`, `Prepare`, or `PrepareExecution` remains an independent operation
that resolves current sources and inputs. Promptkit does not retain or reuse a
base prompt across calls, so the ordinary prefix is identical across attempts
only when the selected definitions, inputs, variables, and other preparation
inputs remain identical. A consumer that requires a pre-execution equality
check can prepare each attempt and compare its opaque rendered-prompt hash
before calling `RunPrepared`. A reusable prepared conversation or frozen
cross-run prompt is outside this feature.
Today a `RunRequest` can select and parameterize a prompt definition, but it
cannot append downstream-owned messages. Registering a second prompt manifest
for every correction-capable operation would duplicate prompt identity,
profile defaults, input declarations, output contracts, and message assets.
Calling an injected model client directly would bypass Promptkit preparation,
capacity, structured output, repair, error adaptation, and provenance. Neither
workaround is a suitable long-term consumer contract.
## Target Public API
Add an append-only field to `RunRequest`:
```go
type RunRequest struct {
// Existing fields omitted.
AppendedMessages []RenderedMessage
}
```
Retain `RenderedMessage.Role` as a string for source compatibility and expose
the untyped string constants `RoleDeveloper`, `RoleSystem`, `RoleUser`, and
`RoleAssistant`. Their values are `developer`, `system`, `user`, and
`assistant`, respectively. The supported values and their semantics are part
of the public contract.
`AppendedMessages` contains fully rendered message values. Promptkit normalizes
and validates their roles as described below, but does not interpret their
content as Go templates, resolve input references inside it, or load it from
files. Nil and empty slices are equivalent.
Reusing `RenderedMessage` is preferred because it is already Promptkit's
public, stable representation of a provider-bound chat message and includes
the supported cache-control metadata. Its GoDoc should be generalized from a
definition-supplied message to an already-rendered chat message. A new
request-only message type would duplicate the same role, content, and cache
control contract without adding useful type safety.
The exact exported declarations and semantics, once implemented, belong in
GoDoc. The name `AppendedMessages` is intentional: placement is part of the
contract, whereas names such as `AdditionalMessages` leave ordering
ambiguous.
## Composition Semantics
For `Prepare`, `PrepareExecution`, and `Run`, Promptkit should:
1. perform ordinary prompt selection, input loading, variable processing, and
prompt rendering;
2. preserve every ordinary rendered message after its role has passed the
shared canonical normalization, including content, cache-control value,
order, and bytes, without rerendering or rewriting content;
3. normalize and validate request-supplied roles, then defensively copy and
append `RunRequest.AppendedMessages` in caller-supplied order; and
4. treat the combined sequence as the effective rendered prompt for every
later preparation and execution boundary.
The request-supplied messages always follow all prompt-definition messages.
They cannot prepend, insert, replace, or delete definition messages. This
narrow contract protects a stable prompt prefix for provider-side prompt
caching and keeps prompt definitions as the canonical owner of ordinary
prompt content.
Promptkit must not synthesize separators or normalize message content. Content,
including empty or whitespace-only content, remains byte-for-byte caller data;
whether a provider accepts it is provider policy. The effective session ID,
selected profile, model settings, output contract, structured-output
constraint, and input hashes retain their ordinary resolution rules.
`InspectPrompt` remains definition inspection and therefore does not accept or
report request-supplied appended messages.
## Preparation, Hashing, And Inspection
Appended messages must participate in the same immutable preparation boundary
as ordinary rendered messages:
- `PreparedRun.Messages` contains the complete combined sequence;
- `PreparedExecution.Details().Messages` contains an independent defensive
copy of that sequence;
- `Run` and `RunPrepared` send exactly that sequence to the selected model
client;
- `RenderedPromptHash` covers the effective session ID and complete combined
sequence, including appended-message cache-control metadata;
- `PromptHash` continues to identify only the selected prompt definition and
therefore does not change merely because appended messages differ; and
- input hashes continue to describe only named artifacts.
Two requests with identical prompt selection, inputs, variables, session, and
appended messages should prepare equivalent message sequences and rendered
prompt hashes. Changing any normalized appended role, content, order, or
cache-control value should change the rendered prompt hash. The hash encoding
must frame sessions, messages, and fields unambiguously rather than relying on
separators that could also occur in caller-controlled content.
Prepared execution must freeze appended messages at preparation time. Caller
mutation after `PrepareExecution` returns must not affect its details or later
execution, and mutation of a value returned by `Details` must not affect the
opaque executable snapshot.
## Interaction With Structured-Output Repair
The combined message sequence is the complete original prompt for one
Promptkit operation. If Promptkit's structural validator rejects a generated
candidate and a repair attempt is available, the existing repair workflow
should begin from a fresh copy of that combined sequence and then append its
own candidate and correction messages.
For the Notarius use case, the ordering may therefore be:
1. ordinary configured prompt messages;
2. Notarius's previous assistant response;
3. Notarius's semantic-correction user message;
4. Promptkit's latest structurally defective assistant response, if any; and
5. Promptkit's structural-repair user message, if needed.
This is not a recursive retry policy. Promptkit still owns only its configured
structural-repair budget within the current operation. It does not interpret
the purpose of consumer-supplied messages, invoke application validators, or
start a later Notarius stage attempt.
All initial and repair generation calls must continue through the ordinary
prepared-execution, capacity, client, cancellation, credential, structured
output, usage, and error-adaptation paths.
## Message Validation And Provider Boundaries
Promptkit should validate appended messages before publishing a prepared value
or starting model generation. Validation should be shared by `Prepare`,
`PrepareExecution`, and `Run` and should reject:
- invalid UTF-8 in role or content;
- a role that, after normalization, is not `developer`, `system`, `user`, or
`assistant`; or
- invalid cache-control metadata.
Role normalization is intentionally limited to trimming surrounding whitespace
and converting the result to lowercase. Promptkit performs no aliasing or
semantic role translation. In particular, it must not translate `developer`
to `system` or the reverse. `system` is generally the more portable choice
across heterogeneous OpenAI-compatible backends; `developer` remains supported
for providers and models that implement the newer instruction hierarchy. A
backend or model that rejects an otherwise supported role reports that through
the ordinary provider error path.
The same four-role vocabulary and normalization apply to prompt-definition
messages. This deliberately tightens the existing prompt format, which
currently accepts any nonblank role. `tool` is not supported because
Promptkit's text-only message value cannot express its required `tool_call_id`;
the deprecated `function` role likewise requires a name that the value cannot
represent. Those roles should be considered only as part of a deliberate
future tool-call message design.
Promptkit does not impose appended-message count, per-message size, aggregate
size, estimated-token, or context-window limits, and it must not truncate
caller content. Those limits are backend-, model-, and application-specific.
Provider rejection for context length, role support, empty content, or any
other provider rule should cross the existing generation-error boundary with
its supported status and provider-detail information intact.
Invalid appended messages are invalid requests and should preserve the public
`ErrInvalidRequest` identity. Error text should identify the failing appended
message index and violated property without echoing its role or content. An
invalid role in a prompt definition remains a source-owned prompt-definition
failure through its ordinary public error category.
## Ownership, Privacy, And Formatting
`RunRequest.AppendedMessages` is caller-owned. Promptkit must defensively copy
the slice and nested cache-control values before retaining them. Internal
domain conversion, prepared-run cloning, public result conversion, and
prepared-execution details must preserve the same non-aliasing guarantees as
ordinary rendered messages.
Appended content can contain model output, source excerpts, or application
validation feedback and must be treated as sensitive prompt content:
- `RunRequest.String` and `RunRequest.GoString` should report only an appended
message count, not roles or content;
- errors must not reproduce appended content;
- no credential or secret value may be inferred, copied, or added to hashes
beyond the existing prompt-content behavior;
- prepared values continue to expose complete rendered messages by design and
remain subject to the consumer's data-handling policy; and
- Promptkit should not add logging, persistence, or a durable retry transcript
for appended messages.
The field does not need a stable `RunRequest` JSON contract because
`RunRequest` has none today. The stable JSON representation of prepared values
must continue to include the final `Messages` sequence through its existing
contract.
## Application-Neutral Boundary
This feature is a generic prompt-composition primitive. Promptkit should not
add concepts such as semantic validator, rejection, stage, pipeline,
correction reason code, or terminal validation policy to its public API.
The following remain downstream responsibilities:
- deciding whether a new request is warranted;
- choosing and enforcing the outer retry budget;
- retaining the exact earlier candidate;
- constructing, bounding, and redacting application feedback;
- determining which message roles and sequence express that workflow;
- keeping preparation inputs stable when an identical prefix is required, or
checking prepared hashes before execution;
- aggregating semantic validation findings;
- recording application provenance; and
- deciding whether exhausted retries fail, reject, warn, or continue.
Promptkit owns only safe composition and execution of the effective request.
## Documentation End State
In the target state, the capability is described by the following canonical
owners:
- root GoDoc for `RunRequest`, `RenderedMessage`, `PreparedRun`, and relevant
engine operations, including the role constants and normalization contract;
- the [framework format reference](../formats.md) for the tightened role
vocabulary and normalization shared by prompt-definition messages;
- the [Go consumer guide](../consumers/pkg-promptkit.md) with one concise
appended-message example and a warning that values are already rendered and
potentially sensitive;
- the [internal runner document](../internal/runner.md) for validation,
composition, hashing, preparation, and structural-repair interaction;
- the [internal source document](../internal/sources.md) for
prompt-definition role normalization and source-owned failures;
- the [internal model-client document](../internal/llm.md) only if its
effective-message input contract needs clarification;
- the [architecture policy](../policy/architecture.md) only if implementation
establishes a durable request-composition invariant not already covered by
the public-facade and application-neutral-boundary rules;
- the
[OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md)
for supported provider-bound roles and provider rejection; and
- release guidance for the new minor version, including migration guidance for
consumers with nonstandard prompt-definition roles.
The Go-only appended-message field does not belong in `docs/formats.md`.
However, the prompt-definition role contract does change and must be updated in
that canonical format reference. Profile and schema formats do not change.
## Verification Expectations
Tests protect the public composition and ownership contract rather than
private helper layout or exact diagnostic prose. Focused coverage establishes:
- nil and empty appended slices preserve current behavior;
- ordinary rendered messages remain an exact prefix and appended messages
retain caller order;
- supported roles in both sources normalize by trimming and lowercasing, while
unsupported, `tool`, and `function` roles are rejected at the owning
boundary;
- message content, including empty and whitespace-only content, is preserved
exactly and receives no Promptkit-owned count or size restriction;
- `Prepare`, `PrepareExecution`, and `Run` agree on the combined sequence and
rendered prompt hash;
- `RunPrepared` uses its frozen appended-message snapshot after caller
mutation;
- `Details` and public conversions return independent copies;
- changing a normalized appended role, content, order, or cache control changes
the
rendered prompt hash without changing `PromptHash`;
- invalid appended roles, UTF-8, and cache controls fail as invalid requests
before model generation without echoing sensitive content;
- provider rejection of an otherwise structurally valid combined sequence
retains the ordinary generation-error contract;
- the initial model call receives the combined sequence;
- structural repair reconstructs the complete combined sequence before adding
Promptkit's repair turns;
- usage, capacity, cancellation, credentials, structured output, and error
identity retain their existing behavior; and
- `String` and `GoString` expose only a count for appended messages.
One external-package contract test and focused internal runner tests should
own most of this matrix. Extend the maintained offline run example only if it
materially improves consumer understanding; do not turn the example into a
retry framework or duplicate the contract tests.
## Acceptance Criteria
- A consumer can append already-rendered messages through `RunRequest` without
defining a second prompt manifest or bypassing `Engine`.
- Normalized ordinary prompt messages remain an unchanged, ordered prefix.
- The final sequence is prepared, hashed, reported in prepared details, frozen,
executed, and repaired consistently across all engine workflows.
- Roles are normalized to and validated against the supported four-role
vocabulary at both prompt-definition and appended-message boundaries.
- Appended values are structurally validated, defensively copied, and excluded
from ordinary request formatting and error text without Promptkit-owned
count, size, token, or context-window limits.
- Promptkit structural repair treats the combined sequence as its original
prompt and retains all existing execution safeguards.
- Promptkit introduces no application-specific retry, validation, pipeline, or
terminal-policy concepts.
- Existing consumers that omit appended messages and use supported roles retain
current behavior; nonstandard prompt-definition roles are intentionally
rejected under the tightened minor-version contract.
- Canonical GoDoc and consumer/internal documentation describe the implemented
contract, and the complete maintainer validation workflow passes.
## Non-Goals
- A Promptkit-owned semantic validation framework.
- A generic outer retry orchestrator.
- Automatic interpretation or formatting of downstream validation errors.
- Prepending, inserting, replacing, or deleting configured prompt messages.
- Treating appended content as a template or file reference.
- Accumulating conversation state across `Run` calls.
- Freezing or reusing a base prompt across independent operations.
- A durable session or retry transcript store.
- PromptKit-owned message-count, byte-size, token, or context-window limits.
- Tool calls, tool results, deprecated function messages, or arbitrary custom
message roles.
- Provider-specific branching for Notarius.
- Changes to profile or schema file formats.
## Alternatives Considered
### Paired Initial And Correction Prompt Definitions
A downstream application could register a second prompt definition that
duplicates the initial manifest and adds candidate and correction inputs. This
requires no Promptkit API change, but it duplicates prompt metadata and creates
drift across prompt versions, profile defaults, output contracts, assets, and
hashes. It also makes a generic request-composition need module-specific.
### A Purpose-Specific Repair Request
Promptkit could expose an API accepting `PreviousOutput` and
`ValidationErrors`, mirroring its internal structural repair request. This
would initially fit Notarius, but it would make Promptkit interpret downstream
policy and would conflate application semantic retries with Promptkit
structural repair. A generic append-only message contract is smaller and more
reusable.
### Direct Execution Of A Mutated Prepared Prompt
Promptkit could expose mutable prepared messages or accept a complete caller-
constructed prompt for execution. That would weaken definition ownership,
prepared-execution immutability, hashing, validation, and provenance, and would
make it easy to bypass source and preparation guarantees. The requested API
should compose messages before immutable preparation instead.
### Direct Use Of The Model Client
Notarius could invoke an injected Promptkit-compatible model client directly.
That bypasses the engine's profile resolution, backend admission, structured
output, repair, error mapping, and prepared-execution contracts. It is not an
acceptable downstream integration path.
### A Semantic-Validation Callback
Promptkit could accept an application callback that validates each candidate
and drives another generation inside one `Run`. That could retain one frozen
prompt and aggregate usage automatically, but it would make Promptkit own
callback lifecycle, semantic diagnostics, application retry policy, and the
boundary between typed domain values and raw model output. Keeping semantic
validation and its outer loop downstream leaves the public API smaller and the
library application-neutral.
### A Reusable Prepared Conversation
Promptkit could expose a reusable handle whose frozen base prompt accepts new
turns across multiple executions. That would guarantee a byte-identical prefix
and could support longer-lived chatbot workflows, but it would require a much
larger lifecycle, credential-retention, concurrency, source-freshness, and
history-ownership contract. The stateless append-only request primitive meets
the demonstrated need without precommitting to those policies.
### Complete Caller-Supplied Or Mutable Message Sequences
Promptkit could let a consumer replace, insert, or delete configured messages,
or execute an entirely caller-constructed sequence. That would weaken prompt
definition ownership, hashing, provenance, and the prepared-execution
boundary. Append-only composition provides the needed flexibility while
preserving configured messages as the canonical prefix.